In April 2025, Mem0 reported a benchmark result that ought to have changed how teams build agent memory: a 91% reduction in p95 latency and over 90% reduction in token cost against a full-context baseline on LOCOMO. Two other frameworks were already exploring the same general direction. Letta used a paged operating-system architecture for memory. Zep introduced a bi-temporal knowledge graph that separated a fact’s truth-time from its known-time. The research on how to make AI agents remember was becoming mature enough for serious production use.
And the production failures continued to land in the same places.
Because the behaviour of an LLM-based agent is probabilistic, certain failure modes appear with regularity in deployments that have not built explicit structure around them. They do not appear in every session. In a regulated deployment running hundreds or thousands of agent invocations a day, they appear often enough that most teams have seen at least one of them.
A stale fact from June may override a corrected version from September, because the agent’s extraction of ‘the customer’s current address’ weighted the older signal more heavily in this session. Two sessions of the same agent may produce conflicting summaries of the same customer, each reflecting a different but defensible abstraction of the same conversation history. An auditor may ask what the system knew at 09:14 on a specific morning, and the team may find that the model’s output was logged, but what the model had access to as input was not. An erasure request may land on a vector store that three teams use without a clear owner, and take days to trace and execute.
These are the symptoms. The pattern they share is what matters. In production, the hard question about agent memory turns out to be a question of placement: where the memory lives, who can read it without invoking the model, and what survives the session.
The instinct in most agent-memory projects is to optimise for capability – making the model remember more, recall faster, draw on a longer history. That instinct is now being overtaken by a different question, forced by regulation.
Under the EU’s revised timetable, the AI Act’s high-risk rules apply from 2 December 2027 for Annex III uses, and from 2 August 2028 for high-risk systems embedded in regulated products. Article 12 requires automatic event logging over a high-risk system’s operational lifetime; Articles 19 and 26 generally require providers and deployers to retain logs under their control for at least six months, subject to applicable law. The covered uses span hiring, lending, healthcare triage, education, parts of critical infrastructure, and life and health insurance risk assessment and pricing. For an agent classified as high-risk in those domains, the question of memory ceases to be about how the model remembers. It becomes a question of where the record lives, who can read it without invoking the model, and how erasure requests are handled alongside retention obligations.
The supporting technology to answer that question is now in reach. The same three frameworks worth knowing – Mem0, Letta, Zep – represent three different architectural bets. Mem0 holds that an extraction LLM running over conversation turns can maintain a structured vector-and-graph memory through explicit add, update, and delete operations. Letta, originally MemGPT (2023), holds that agent memory should be paged like an operating system, with explicit working and long-term tiers. Zep holds that knowing what you knew, and when you knew it, requires separating those two clocks at the data layer. Each framework is technically sound. Where any of them sits in your architecture is what determines whether the regulatory question is answerable.
The CTO question worth reconsidering in any regulated deployment: where does our agent’s memory live, and who could answer a regulator’s audit request without re-invoking the model?
The agent is on shift. The log is the institution
Think about a hospital ward at shift change.
The night-shift nurse does not ingest weeks of patient history from someone’s head. They read the formal log: what is pending, what is done, what to watch. They act on the log. At the end of shift, they write the next entries. The log was written by people; it will be read by people; it survives any individual nurse leaving the ward.
That is what workflow-owned memory looks like in agent infrastructure. The agent – the nurse on shift – reads from an explicit state store at the start of a task. The store has a schema, a retention rule, a permission model, and an audit trail; the workflow owns all of them. The model never holds the canonical state. It borrows it for the duration of a call, then writes a structured update back. The workflow validates and commits the update, recording the agent’s steps in the same store, under the same policy.
This pattern is unfashionable because it looks like databases. The interesting LLM mechanics happen elsewhere. The compensation is that production failure modes become tractable. Stale facts get versioned and superseded. Conflicts resolve by policy, rather than by whichever session writes last. Erasure requests follow a known schema and retention policy. And the audit log answers the question the regulator is actually asking, which is rarely ‘what did the model think?’ and almost always ‘what did the system know, and when?’
Name the pattern honestly
The problem this solves is older than the technology applying it.
RAG (Lewis et al., 2020); function-calling and tool use (2022 to 2023); structured outputs (2023); now governed agent state. Each move is the same move: take the part of the system that needs to be inspectable and put it in a place that can be queried, instead of leaving it in the model. Four iterations of the same idea over six years. The newest one is the most consequential for regulated work, because memory determines what survives from one call to the next.
Long-context windows shift what is in scope without solving the underlying question. With a million-token window the agent can carry more state within a single call, but the workflow still has to decide what persists between calls, who can see it, and how long it lives. Those decisions are workflow engineering. The window is a buffer, useful and important, but distinct from the question of where the workflow’s record lives.
The argument here is about placement, and Mem0, Letta and Zep are sound technical primitives whose benchmark numbers stand. The framework belongs behind a workflow-owned interface, with the wrapper doing the work of permissions, retention, and audit. Most regulated deployments are picking the right primitive and building the wrong wrapper.
What the placement question means depends on who you are
Biopharma
clinical-study reports · pharmacovigilance triage · GxP-validated workflows · regulatory submissions
Audit and retention rules apply at the field level, and have done so for years before AI agents arrived. Agent state must inherit those rules from day one, rather than being retrofitted to them after deployment. The shift here is structural: the architectural decision and the regulatory obligation are the same decision.
Insurance
underwriting copilots · claims triage · fraud signals · long-running customer files
Conflict-of-record is routine in insurance work. Customer information changes; claims status evolves; policy interpretations get superseded. The workflow owns the canonical version, and the agent reads and updates it under policy. The shift here is structural: the agent’s authority over the record is bounded by design.
Manufacturing and operations
maintenance reasoning · QA trails · process deviation tracking · MES-adjacent agents
Shift-log and MES infrastructure already exist in most operational environments, and absorb the agent more naturally than greenfield memory systems do. The patterns from process engineering – structured handover, signed records, traceable deviations – extend cleanly to agent operations. The shift here is operational: an existing discipline being applied to a new actor in the loop.
Consumer SaaS and assistants
personalisation · creative tools · most consumer assistants without sovereignty or compliance constraints
Where audit pressure is low, agent-owned or framework-owned memory remains the right answer. Most of the publicly available literature was written for this context. The shift here is minimal: the placement decision still matters, but the consequences of getting it wrong are bounded by the use case.
Three questions worth asking about your own situation
What share of your agent’s persistent state – by count of writes, in the last 30 days – lives in a system you can query without invoking the model?
This is the measurable question. As a rough diagnostic: Below 50% means the agent has become the effective system of record for state your organisation does not own. Above 90% means the architecture is most of the way there. Most deployments sit somewhere in between, and the gap is the integration work. A regulator asking ‘what did the agent know at 09:14?’ is asking you to query the part above the line.
When two sessions of the same agent produce conflicting summaries of the same fact, what resolves the conflict?
If the answer is the model, the workflow has delegated authority over its own record to whichever session writes last. If the answer is the schema, or a named human owner, the workflow controls what the workflow remembers. The first arrangement is hard to defend in front of an auditor. The second makes that authority explicit.
When a customer or patient requests erasure of their data, how many systems does the request fan out to, and who owns each?
If the answer is ‘more than two, and we are still mapping it’, the workflow does not yet own its memory. The mapping itself is the work.
Together, the answers tell you whether the memory-placement question has become the binding constraint for your organisation in the last twelve months.
For most consumer and horizontal-SaaS workloads, model-owned or framework-owned memory will remain the right answer. The audit pressure is low, the personalisation upside is real, and the open frameworks ship.
Where audit pressure is high, the empirical evidence points the same way from several independent directions. Mem0 reports a 91% cut in p95 latency and over 90% in token cost against a full-context baseline. LongMemEval reports GPT-4o accuracy falling from 87.0% with the relevant evidence sessions supplied to 60.6% on a 115k-token full-history baseline, with the same model and the same questions – the difference is how the history is presented to it. Zep treats truth-time and known-time as separable, as a first-class architectural commitment. Three different research efforts converge on the same conclusion: scoped, structured, versioned memory beats undifferentiated retention. For regulated deployments, the step from there to a workflow-owned state store is short.
Working through what that means for a specific organisation – which memory belongs in the workflow, which can stay with the model, what schema, what retention, what audit interface – is what we do. If your team is now living with that question, we would welcome the conversation.
For technical readers
Mechanism, numbers, and what to start with this week
The mechanism
The architectural pattern is event sourcing applied to an LLM-based agent. A workflow-owned state store – typed schema, explicit versions, retention rules, audit log – sits outside the agent runtime. The agent reads what it needs at the start of a task, reasons, acts, and writes a structured update back. The update is validated and committed under the same policy as any other system-of-record write. The model never holds the canonical state. It borrows it for the duration of a call.
Three production frameworks sit close enough to the right shape to use as starting points, each with a different bet. Mem0 (Chhikara et al., 2025) runs an extraction LLM over conversation turns to maintain a vector-and-graph memory with explicit add, update, and delete operations. Letta, originally MemGPT (Packer et al., 2023), treats memory as an operating system – a working set, a paged long-term store, model-issued memory function calls. Zep uses Graphiti, a bi-temporal knowledge graph that separates a fact’s truth-time (when it was true in the world) from its known-time (when the system learned it). That separation is what makes ‘what did we know at 09:14?’ a tractable query.
The placement decision is independent of the framework choice. Any of these can sit behind a workflow-owned interface. The wrapper – permissions, retention boundaries, audit hooks, the human-readable query path – has to be built in your own code regardless of which primitive you choose. That wrapper is the engineering work.
The numbers
Mem0 on LOCOMO. A 26% relative improvement in overall J-score over the paper’s OpenAI-memory baseline, alongside a 91% reduction in p95 latency and over 90% reduction in token cost against full-context processing. Conditions: GPT-4o-mini as the agent model, ten multi-session conversations averaging around 600 turns.
LongMemEval on GPT-4o. Accuracy drops from 87.0% with the relevant evidence sessions supplied through oracle retrieval to 60.6% on the full 115k-token history (Wu et al., 2024), without Chain-of-Note prompting. Same model, same questions. The difference is how the history is presented to it. Five hundred questions across five core memory abilities, 100k-plus token contexts.
These numbers do not extrapolate cleanly. They hold strongest for short-turn factual dialogue where the relevant atoms are retrievable. They are weaker on tasks requiring many simultaneously-active hypotheses, or where the structure of the conversation defeats simple extraction. The closer your workload looks like LOCOMO, the closer your speedup will track the headline.
What’s next
Install
Choose by data shape. For extraction-heavy dialogue, start with Mem0’s quickstart. For hierarchical session state with paging, see Letta’s documentation. For truth-time versus known-time as a first-class concern, see Zep’s current guide.
Try
Run 50 turns of representative production traffic through your current full-context flow, and the same 50 turns through that flow wrapped in one of the three frameworks. Log token counts, p95 latency, and every memory entry written. This experiment is runnable in an afternoon on a single laptop.
Measure
Three numbers. p95 read latency under your real query mix – a useful target is under 200ms. Per-session token cost relative to the full-context baseline. Audit completeness as a percentage – the fraction of state-changing agent decisions reconstructible from retained inputs, versions and logs, without re-running the model.
Watch
Failure modes. Extraction LLMs writing wrong facts under load. Conflict resolution silently defaulting to last-write-wins. Vector-store drift as facts go stale. The permission model failing to cross the store boundary. ‘Audit-ready’ claims that turn out to require manual export at the point a regulator asks for the logs.
Read
Chhikara et al., 2025 for the Mem0 extraction loop; Rasmussen et al., 2025 for the Zep and Graphiti bi-temporal pattern; Wu et al., 2024 for the LongMemEval failure modes that motivate the architecture in the first place.
This Perspective began as a discussion inside Brightbeam’s Digital Intelligence: State of the Art and Beyond (DI SOTA and Beyond) programme, a standing forum where the people building production AI systems argue through what is real, what is hard, and what is merely fashionable. The thinking here is collective. The argument was sharpened in the room before it was written down.