How to Build Reliable Long-Term Memory for AI Agents

Long-term memory for an AI agent is an external system that writes, retrieves, updates, and sometimes deletes information across tasks. The model still receives a finite working context on each call. Memory earns its name only when earlier information can be selected later and can change what the agent does.

That distinction rules out two tempting shortcuts. A complete chat transcript is a record, not a memory design: important facts are buried beside greetings, abandoned plans, and outdated instructions. A larger context window lets the model read more at once, but it does not decide what deserves to persist, reconcile contradictions, enforce retention, or prove where a remembered claim came from.

A dependable design therefore has five parts: a small working context, typed long-term stores, a selective write policy, retrieval that understands time and relationships, and an update-and-forgetting policy. Evaluation must cover the entire loop. Testing only whether a fact was stored says nothing about whether the agent will retrieve it at the right moment or resist it after it becomes false.

Start with the boundary around the current decision. Working context should contain the active goal, constraints, recent observations, retrieved evidence, and the state needed for the next action. It should be deliberately assembled, not treated as a scrolling container for everything the agent has seen.

The MemGPT paper described a useful architectural analogy: keep a limited main context and let the system move information to and from external storage. The lasting idea is not a particular model or context-window size. It is that memory management is an explicit operation. The agent—or a deterministic controller around it—must decide when to search, what to load, and what to evict.

The CoALA framework makes the separation more precise. It treats working memory as the current decision state and distinguishes episodic, semantic, and procedural long-term memory. These are engineering categories inspired by cognitive architectures, not a claim that a language model remembers like a person. They help because each kind of record deserves different permissions and retrieval rules.

Semantic memory holds relatively stable knowledge: a customer's approved shipping region, a project's definition of done, or the relationship between a policy clause and an endorsement. Episodic memory records what happened: a user rejected an option on a date, a claims reviewer requested evidence, or an analysis failed because a source was unavailable. Procedural memory contains how work should be done: a checklist, tool contract, escalation path, or reviewed runbook.

Do not collapse these into a single bucket of prose summaries. A fact needs a subject, value, scope, source, effective date, and status. An episode needs actors, actions, outcomes, and time. A procedure needs a version, owner, preconditions, steps, and approval state. The raw source should remain reachable even when the stored record is compact.

Permissions should follow the type. An agent may write interaction episodes freely while requiring confirmation before turning an inference into a durable user preference. It may read an approved runbook but not rewrite it. CoALA explicitly notes that changing procedural memory is riskier than writing episodic or semantic records because a bad procedure can alter future behavior broadly.

A useful write policy asks whether information will matter beyond the current task, whether it is trustworthy enough to reuse, and whether storing it is permitted. Durable preferences, commitments, resolved decisions, recurring exceptions, and verified outcomes often qualify. Pleasantries, speculative reasoning, duplicated source text, and one-off details usually do not.

The Generative Agents study used a memory stream and retrieved items using relevance, recency, and importance, with reflections that formed higher-level inferences. It was an interactive simulation aimed at believable behavior, not a universal production recipe. Its reported failure modes are still instructive: retrieval could miss the right memory, and generated embellishments could enter the agent's account. Any reflection or summary should therefore point to its supporting episodes and remain distinguishable from direct observation.

Write-time controls also reduce privacy risk. Ask for consent where personalisation is not reasonably expected. Exclude secrets that the task does not require. Apply tenant and role boundaries before indexing, not after retrieval. Record retention and deletion obligations with the memory itself so a later purge can remove derived summaries and search indexes, not only the original message.

The retrieval query should be derived from the active decision: the entities involved, the time period, the memory type, and the evidence needed. For a renewal recommendation, retrieve the current objective, prior decisions and their outcomes, changed constraints, and the approved procedure. Do not simply embed the latest user message and accept the nearest passages.

The LongMemEval benchmark separates memory into indexing, retrieval, and reading, and tests information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention. Its experiments show why no single representation wins every task. Whole-session records can be too broad, while aggressive summaries can lose details. The paper found value in decomposing sessions, enriching retrieval keys with facts, and expanding time-aware queries.

A practical retriever can combine semantic relevance with filters for entity, record type, validity, permissions, and time. It should also seek counterevidence. When a current fact conflicts with an older record, return both with their dates and status rather than letting similarity decide which truth survives. A graph traversal may help when the question depends on several hops—person to project to decision to source—but a graph is not required for every lookup.

After retrieval, a reading step should compress evidence into working context without erasing provenance. Deduplicate near-identical records, preserve short source excerpts where exact wording matters, expose uncertainty, and set a context budget. If evidence is insufficient, the correct memory behavior is often to abstain or ask for a fresh observation.

Memory becomes dangerous when old information looks current. Use explicit lifecycle states such as proposed, confirmed, superseded, disputed, and expired. Preserve an audit trail where the work requires it, but expose one canonical current view to the agent. A new address should supersede the old address; it should not compete with it in vector search.

Define how contradictions are resolved. A trusted system of record may override a conversational note. A user correction may require immediate precedence but still await verification for a regulated action. Inferences should decay or be reviewed sooner than direct statements. Procedures should be immutable by default and promoted through a human-controlled versioning process.

Forgetting is part of correctness, not only storage hygiene. Expire temporary constraints, remove records when consent is withdrawn, and propagate deletion to derived facts, embeddings, caches, and graph edges. Some domains require retention of an audit history; that history can be access-restricted and excluded from routine agent retrieval rather than pretending it no longer exists.

These controls create real trade-offs. More memory can improve continuity while increasing latency, attack surface, and the chance of stale context. Detailed provenance improves reviewability but costs engineering effort. The smallest adequate memory system is usually better than a universal personal history with vague future uses.

Build evaluations from the work the agent must continue over time. Seed histories with a durable fact, an irrelevant look-alike, a later correction, a sensitive item outside the agent's permission, and a question whose answer was never provided. Then measure whether the system writes the right record, retrieves the right evidence, cites its origin, applies the update, refuses the forbidden item, and abstains on the missing one.

LongMemEval is useful because it tests more than simple recall. Its knowledge-update and temporal questions expose systems that remember a value but ignore that it changed. Its abstention questions expose confident invention when no supporting memory exists. Track retrieval recall separately from final-answer quality so a failure can be located in indexing, retrieval, or reading.

The 2026 LongMemEval-V2 preprint extends the question toward operational experience: static state, changing state, workflow knowledge, environment-specific gotchas, and awareness of a false premise across long agent trajectories. It is explicitly marked work in progress, so its results are not a settled leaderboard. The task categories are nevertheless a useful prompt for evaluation when an agent must behave more like an experienced colleague than a chat archive.

Include operational measures: latency, context size, cost per successful retrieval, duplicate-write rate, stale-memory rate, and deletion completeness. Review failures by following the trace from source to stored record to query to retrieved context to action. A single end-to-end score cannot tell you which part to repair.

A one-shot document task may need retrieval from authoritative sources but no experience store. A deterministic workflow with a small database may be clearer than an agent that interprets its own memories. If the organisation cannot state retention rules, honour deletion, or separate access by user and tenant, persistent personal memory should wait.

Begin with one future decision that repeatedly lacks context. Name the records it needs, the authoritative sources, the update rules, and the evaluation cases. Add graph relationships only when connected facts materially improve retrieval or explanation. Add reflection only when higher-level patterns are useful and can be reviewed against their evidence.

Reliable memory is not the amount an agent keeps. It is the quality of the path from a past event to a present decision—and the ability to challenge, revise, or erase that path when reality changes.

Granveo is exploring this thinking layer for connected, source-grounded work. If an agent in your workflow keeps forgetting a crucial decision—or confidently remembers the wrong one—share the real sequence and its failure point at sahil@granveo.com.