Building agent memory that actually persists
language models forget the moment they finish replying. memory is everything the system arranges around them.
Language models forget the moment they finish replying. Every impressive “memory” you have seen is not the model remembering — it is the system around the model deciding what to put back into the next prompt.
The core problem
A context window is a whiteboard that gets wiped after every turn. If you want an agent to remember that a user prefers terse answers, that a trade closed last Tuesday, or that a tool failed twice already, you have to store it and you have to decide when it is relevant enough to bring back.
That decision — retrieval — is the entire game. Store too much and you drown the model in noise. Store too little and it feels amnesiac.
What I build instead
- A write path that summarizes each interaction into durable facts, not raw transcripts.
- A scoring layer that ranks stored memories by recency, frequency, and semantic relevance to the current turn.
- A budget — a hard token ceiling on how much memory can re-enter the prompt, so the agent stays fast and cheap.
The result is an agent that behaves like it remembers, because the memory it needs is always the memory it sees.
Memory is not a feature of the model. It is an architecture you build around it.
More on the retrieval half of this in the next post.