Agent memory explained: context, retrieval, and learning
An agent can use a saved fact in a later session without being retrained. Understanding that distinction makes memory systems easier to build, evaluate, and trust.
What does it mean for an agent to remember?
An AI agent is an application that uses a model to choose and carry out steps, often through tools. When that application saves information from earlier work and makes it available later, we can describe the behavior as memory.
The word covers several mechanisms. This article focuses on external memory: retained information outside the model’s current input and learned parameters. It might be a note in a file, a record in a database, or a summary stored by the application. It does not imply human recollection or a particular storage technology.
A useful first question is: what information survives when the original conversation is no longer supplied to the model, and how will the application bring it back?
Separate context, stored records, and model parameters
These three things can all influence an answer, but changing one does not automatically change the others. A helpful mental model is to distinguish the input available now, the records available to fetch, and the model produced by training.
| Concept | What it contributes |
|---|---|
| Current context | The input supplied for a model call, such as instructions, messages, and tool results. An older message has to be included or recovered to be available here. |
| External memory | Information retained by the surrounding application. A later call can receive selected records directly or through a retrieval tool. |
| Model parameters | The learned values used to compute the model’s outputs. Updating a note in an external store does not by itself update these values. |
The original retrieval-augmented generation paper explicitly combined knowledge in model parameters with an external document index. That separation helps explain how an application can change the information used for an answer without making the external record part of the model’s weights.
Sources: Lewis et al., Retrieval-Augmented Generation (2020)
Follow one fact through a later session
Imagine a coding assistant helping with a project called Atlas. The user explains that the first export must be CSV because the accounting import requires that format. This is an illustrative example, not a recorded experiment.
The application could retain a short record: “Atlas: use CSV for the first export, because the accounting importer requires it.” Add the date and a reference to the requirement. Keeping the project and reason makes the record more useful than an isolated preference for CSV.
- Save: store the requirement and confirm that the write succeeded.
- Retrieve: in a later session, fetch records relevant to Atlas’s export requirements.
- Use: provide the selected record to the model so it can explain the format choice.
- Revise: if the importer later accepts another format, update the requirement and handle the earlier record explicitly.
How a saved requirement reaches a later session
- Earlier workSave the requirementAtlas needs CSV export. Retain the project, reason, date, and source.
- External memoryRetrieve a recordA later export question prompts the application to fetch relevant Atlas records.
- Later model callUse selected contextThe application includes the record in the input. The model can use it to explain the CSV choice.
Saving a record changes external memory. It does not, by itself, update the model’s trained parameters.
The model’s next answer changes because its available evidence changes. In this example, no training step is needed. The application also has decisions to make: which records qualify, when to fetch them, and how to represent a correction.
A mistake at any step can look like forgetting. The write may have failed. A search may have missed the record. The model may have received it and still ignored it. Observing the final answer alone does not identify which step failed.
What a larger context window changes
A larger context window lets an application supply more information in a call. It can be valuable when a task needs many details together. Persistence still requires a decision about what to retain and reload for future work.
The 2024 paper Lost in the Middle found that the position of relevant information affected performance on its tested question-answering and retrieval tasks. Information in the middle was often harder for the evaluated models to use. This is a result about those models and tasks, not a claim that every current model behaves identically.
For a system you are evaluating, test the actual work with different amounts of history. Check whether extra material changes correctness, latency, or cost. The largest input the model accepts is not a measurement of how reliably your application uses that input.
Where retrieval fits
Retrieval is the step that selects stored information for the task at hand. It can involve looking up an identifier, searching text, comparing semantic representations, or letting an agent open relevant files. A vector database is one possible component; it does not define the whole memory workflow.
Anthropic’s context-engineering guidance describes both fetching information as needed and keeping notes outside the current context. These are useful examples of how a file-based workflow can supply retained context without requiring every record to be loaded at once.
RAG, short for retrieval-augmented generation, describes generating with retrieved information. A RAG application can answer questions from a fixed manual without retaining anything about the person asking. An agent memory workflow may use retrieval while also recording decisions, observations, or instructions created during work. The mechanisms can overlap; calling a system RAG does not tell you what it remembers about previous sessions.
Sources: Anthropic, Effective context engineering for AI agents (2025); Lewis et al., Retrieval-Augmented Generation (2020)
In what sense is the agent learning?
An agent may improve its behavior by reading a saved instruction or by using a reflection on previous work. That is a change in the information guiding its behavior. It is useful to distinguish this from updating model parameters through training.
The Generative Agents research architecture stored observations, produced reflections, and retrieved memories to inform behavior in a simulated environment. It illustrates one way retained experience can affect later actions. Its evaluation concerned that simulation; it does not establish reliability for unrelated workplace agents.
When someone says an agent learns, ask what is updated: a record, a summary, an instruction, a retrieval policy, or the model itself. The answer determines what you can inspect, correct, move to another system, and evaluate separately.
Evaluate the behavior with cases that can fail
A convincing demonstration should include more than retrieving a phrase immediately after saving it. The following is a proposed evaluation exercise, not a standardized benchmark. It can be adapted to a simple notes-based assistant or a more elaborate memory service.
- Continuity: save the Atlas requirement, start a session without the original conversation, and ask for the export decision. Inspect what evidence was retrieved.
- Paraphrase: ask the same underlying question in different words. Check whether the relevant requirement still reaches the model.
- Scope: create a second project with a different export format. Check that the assistant keeps the projects separate.
- Correction: change the Atlas requirement. Check how the old and new records are resolved rather than assuming the latest statement wins.
- Missing evidence: ask about a requirement you never saved. Look for an explicit gap rather than a fabricated recollection.
- Removal and access: remove a record from the intended recall store or revoke a test user’s access. Verify what can still be retrieved, including through other copies.
For each case, record the input, saved state, retrieved material, final answer, and expected result. Repeating the cases lets you distinguish a dependable pattern from a fortunate answer. A failure to retrieve the right evidence and a failure to use correct evidence call for different fixes.
The practical value of memory is the ability to carry useful information into later work while keeping it open to correction. A small system that makes that process visible can teach you more than a large collection of records with no way to explain its answers.