The Session Is the Bug in Your AI Pipeline
The worst AI pipeline failure is not a bad sentence.
on this page
The worst AI pipeline failure is not a bad sentence.
It is a sentence that looks fine, ships cleanly, and is wrong because the pipeline kept carrying the wrong idea forward from an earlier stage.
If you have built a multi-step workflow and watched it produce the right shape with the wrong content, you have probably blamed the model first. Wrong temperature. Weak prompt. Bad retrieval. Maybe all of that is true sometimes. But there is another failure mode that feels almost identical from the outside and is much harder to spot: hidden session state drifting across stages while the system keeps talking as if nothing happened.
That is the failure this pipeline taught me to respect.
The editorial flow I built has nine stages:
scan, cluster, propose, evidence, validate, write, visuals, package, publish.
At first, the stages shared conversational context. It was convenient. It also meant the model could keep “remembering” a rejected direction after the rejection had already happened. In one stretch of early runs, eleven of the first forty workflows let a rejected proposal make it all the way into the write stage. Nothing crashed. Nothing screamed. The pipeline just kept producing polished output that did not match the approved decision.
That is the kind of bug that costs you trust.
Why This Feels Like A Global Variable
If you have written normal production code, you already know the shape of this problem.
Global mutable state is handy at first. Any function can read it. Any function can change it. Then two pieces of code disagree about what it should contain, and the third piece reads both versions in whatever order the system happened to run. You do not get a clean failure. You get non-deterministic behavior and a bug that only shows up after enough steps have passed.
Shared LLM session state behaves the same way, except the debugging tools are worse.
You cannot grep the model’s attention. You cannot inspect every hidden influence the way you inspect a variable in code. Once the session has absorbed a rejected draft, a half-finished argument, or an earlier assumption that should have been discarded, the model may keep treating that old state as part of the current task. The output still sounds fluent. The logic just stops lining up with the actual system state.
That is why this failure is dangerous. It does not announce itself as a failure.
Why Nine Stages Make It Worse
The danger gets bigger the moment the pipeline becomes real.
Two stages are easy to hold in your head. Six stages are not. Once the flow got long enough, shared context stopped feeling like a shortcut and started feeling like a hidden dependency chain. A stage would reject an output, but the rejection was only a conversational event. Nothing changed in a file. Nothing changed in a database row. The next stage would ask for “the approved draft,” and the session would still have the earlier one sitting there, ready to answer as if it were still valid.
That is how you end up with output that is formally correct and operationally wrong.
The obvious fix is to keep adding prompt instructions.
That helps less than you want. Prompts still live inside the same session. They do not erase the old state. They sit on top of it. If the stage boundary itself is fuzzy, the model can still blend old assumptions into the new request. You have not created a handoff. You have just added more words to the same conversation.
Files Are The Shutoff Valve
The fix that finally worked was boring, which is usually a good sign.
Every stage writes to a file. The next stage reads that file. The session ends.
That single choice changes the whole system.
If a stage is approved, the approval becomes an artifact. If a stage is rejected, the rejection becomes an artifact too. Downstream stages no longer inherit whatever the model happened to remember most recently. They inherit the exact file they were told to read.
import json
from datetime import datetime, timezone
from pathlib import Path
def write_stage_output(stage: str, data: dict) -> None:
base = Path(f"state/{stage}")
history_dir = base / "history"
history_dir.mkdir(parents=True, exist_ok=True)
payload = json.dumps(data, indent=2)
timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%S")
(base / "latest.json").write_text(payload)
(history_dir / f"{timestamp}.json").write_text(payload)
def read_stage_output(stage: str) -> dict:
path = Path(f"state/{stage}/latest.json")
if not path.exists():
raise FileNotFoundError(f"No output from stage {stage}")
return json.loads(path.read_text())That is not a grand architecture pattern. It is a shutoff valve.
The real point is not the file format. A database works. An event log works. Typed message queues work. The point is that every handoff becomes an object you can inspect, replay, and validate without asking the model to remember what happened ten messages ago.
Once you do that, the pipeline stops relying on memory and starts relying on evidence.
The Artifact Is The API
This is the part that changed how I think about AI workflows.
I stopped asking, “What was the model thinking?”
That question is usually a dead end.
I started asking, “What artifact did stage three produce, and what did stage four actually read?”
That question can be answered.
When the artifact is the interface, each stage becomes much closer to a pure function. Inputs are explicit. Outputs are explicit. Validation has somewhere real to happen. You can rerun stage six without replaying the whole pipeline. You can swap a different model into stage four without breaking stage five, because stage five does not care which model produced the file. It only cares that the file exists and passes validation.
That is the part that makes the pipeline debuggable.
It also makes it honest.
If a stage has not produced its artifact, downstream work should stop. If the artifact does not match the schema, the next stage should fail loudly. If a stage was rejected, the rejection should be visible as a file, not trapped as conversational residue.
That is how you stop the system from quietly “helping” you with the wrong answer.
What Changed In Practice
Once the pipeline moved to artifact-based handoffs, a few things got much easier.
The monitor could show every stage, every prompt, every response, and every transition in one place, because the state now existed outside the session.
Replay became practical, because the state directory held a clean chain of outputs instead of a vague conversational history.
Validation got sharper, because each stage had something concrete to inspect before it could continue.
And the failure mode became easier to reason about. When output is wrong now, the question is not “which part of the conversation went bad?” It is “which file first diverged from the approved path?”
That is a much better debugging question.
It also explains why vibe-coding an AI workflow feels fine at first and then falls apart.
Two stages can fit in your head. Six stages need a contract. Nine stages need artifacts.
If you let the session be the contract, you are trusting the most ambiguous part of the system to preserve the most important state. That works right up until it does not.
The safer pattern is simple:
make the artifact the truth, end the session, and let the next stage read the file instead of the memory.
If you’ve built a multi-step AI workflow and watched it silently produce the wrong output — correct format, wrong conten
If you’ve built a multi-step AI workflow and watched it silently produce the wrong output — correct format, wrong conten
Code Share
This article includes a companion code share with:
- core implementation logic
Available in the repository:
code-share/ai-content-pipeline-architecture