Agent Memory and Context Drift in Long Sessions

Agent Memory and Context Drift in Long Sessions

Almost every agent evaluation in circulation shares a hidden assumption: that the agent is being tested from a clean start. A fresh context window, an empty scratchpad, no prior turns, no retained user preferences, no accumulated tool output. The task is issued, the agent acts, the result is scored.

Production agents do not live like that. They live in sessions that run for hours, in threads that resume across days, with memory stores that persist across sessions entirely. By the time the interesting failure occurs, the context window contains forty turns of history, six tool results, three corrections from the user, a summary of a summary, and a preference the agent inferred on turn nine and has been quietly obeying ever since.

The system you evaluated is not the system that failed. Nothing changed except the state — and the state is part of the system.

Three kinds of memory, three kinds of decay

"Memory" in an agent stack usually means at least three separate mechanisms that fail in different ways and are almost always evaluated as if they were one.

Working context is the raw conversation and tool output inside the current window. It decays through truncation and compaction. When the window fills, something gets dropped or summarised, and the thing that gets dropped is chosen by a policy nobody wrote down: usually "oldest first". The oldest turn is frequently where the constraint lives — never contact the customer directly, the account is in dispute, do not modify anything under the archive directory. The agent does not remember being told not to, because it no longer holds the sentence in which it was told.

Session memory is the derived state an agent keeps about the current task: a plan, a scratchpad, a to-do list, an inferred user goal. It decays through drift. Each rewrite of the plan is a lossy re-encoding of the previous plan, and after enough rewrites the agent is executing against a goal that has slowly rotated away from the one the user gave it. Nobody made a wrong decision. Every step was locally reasonable.

Persistent memory is what survives across sessions: user preferences, learned facts, project notes, retrieved profiles. It decays through contamination. Something wrong, something stale, or something adversarial gets written into the store, and now it is retrieved into every future session as established fact. This is the most dangerous of the three, because unlike the other two it does not reset.

The dotted line is the one to stare at. Content the agent merely read — a support ticket, a PDF, a web page, a code comment — can end up written into a store that is retrieved as instruction later. The trust boundary between "data the agent processed" and "state the agent believes" is the single most under-defended surface in agent design.

Memory poisoning is a delayed-action failure

Prompt injection is usually discussed as an immediate compromise: the agent reads a hostile instruction and acts on it in the same turn. Detection is at least conceivable, because cause and effect are adjacent.

Memory poisoning separates them. The hostile or simply wrong content is absorbed on Monday and written into persistent state. On Thursday, in a different session, with a different user, the agent retrieves it as background knowledge and acts on it. There is no injection to detect in Thursday's transcript. The agent is behaving perfectly rationally given what it believes, and what it believes is a fact it learned from a document it had every reason to read.

The same shape applies without an adversary. An agent infers on turn nine that the user prefers terse output, writes that to memory, and six weeks later gives a dangerously abbreviated summary of a compliance issue because terseness became a standing instruction. No attacker. Same mechanism.

Two structural defences matter more than any classifier:

Provenance on every memory write. Each stored item records where it came from — user statement, tool result, document content, agent inference — and content that originated outside the trust boundary is never promoted to instruction-level state without an explicit act by a trusted party. Content read is not fact learned.

Expiry and re-confirmation. Memory that persists forever asymptotically approaches being wrong. Stored facts need a time-to-live and consequential inferences need re-confirmation before they are relied upon. "You told me in April you never want approvals routed to Legal — is that still right?" is not a UX blemish. It is a control.

Context drift is measurable, and nobody measures it

The tractable part of this problem is that most of it can be tested, and the tests are not exotic.

Instruction retention. Issue a hard constraint early. Run the session long enough to force compaction. Then present a situation where obeying the constraint conflicts with completing the task. Does the constraint survive? Measure retention as a function of turn count and you get a curve — and that curve, not a single pass/fail, is the honest description of your agent's reliability.

Goal fidelity. Take the user's original objective and the agent's working plan at turn thirty and have a qualified person score how far the second has rotated from the first. Drift is usually visible immediately to a human and invisible to every automated metric, because each individual step was coherent with the step before it.

Cross-session leakage. Run session A with user or tenant A. Start session B as tenant B. Does anything from A appear — a name, a preference, a retrieved fact, a file path? In multi-tenant deployments this is not a quality issue, it is a data protection incident, and it is trivially testable and almost never tested.

Poisoning resistance. Plant a benign-looking but false or instructive statement in a document the agent will process. Complete the session. Start a new one. Ask a question whose answer depends on whether the planted content was absorbed. The gap between "the agent read it" and "the agent believes it" is the thing you are measuring.

Compaction fidelity. Compare what was in the window before a summarisation step with what survived it. The interesting metric is not compression ratio. It is which categories of content are preferentially destroyed — and constraints, negations, and exceptions are destroyed far more readily than facts, because they are short, they carry no entities, and they look redundant to a summariser.

Session isolation is a design decision, not a default

The strongest available mitigation is unglamorous: do not carry state you cannot justify carrying.

Draw the boundary explicitly. Which state persists across turns? Across sessions? Across users? Across tenants? Each boundary crossed should be a deliberate choice with an owner, not a side effect of a framework's default configuration. A surprising number of agent deployments share a memory store across tenants because the vector database was set up before anyone asked whether it should be partitioned.

Then apply the same discipline to what enters the store. A write path from tool output straight into persistent memory, with no review, is a standing invitation for whatever the agent happens to read on any given Tuesday to become permanent policy.

And keep the reset available. An agent that cannot be returned to a known-good state has no rollback. Users need a way to clear derived memory, operators need a way to purge a contaminated store, and both actions need to leave a record.

Why this belongs in your evidence, not your backlog

The reason context drift stays invisible is that the artefacts organisations produce about their AI systems describe configuration, not behaviour under accumulation. A model card describes a model. A system diagram describes components. Neither says what happens on turn forty.

Evaluating this properly means evaluating sessions, not prompts: multi-turn scenarios run to realistic length, scored by credentialed Evaluators against a rubric that has explicit dimensions for instruction retention, goal fidelity, and memory hygiene — with the findings, their severity, and their root causes recorded in a versioned Evidence Report rather than a Slack thread. Under the SASF framework, memory-related behaviour is assessed as its own category precisely because it is not reducible to output quality on a single turn.

The single-turn score tells you the agent can do the job. The long-session evaluation tells you whether it still can after it has been doing the job for an hour.

What to do tomorrow

Take your highest-stakes agent and write down, honestly, what state it carries and where each piece of it came from. Then run one session ten times longer than the ones you test today, with a hard constraint issued in the first turn, and see whether it survives to the last. Run a second session as a different tenant and grep the transcript for anything that belongs to the first.

If you find nothing, you have a result worth reporting. If you find something, you have found it before your customer did — and that is the entire point of evaluating the thing you actually deployed rather than the thing you demoed.