AI Stack Library · Memory
How to set up AI agent memory that survives the next session, and the next model
To set up AI agent memory, give your agent a folder of plain files it reads at the start of every session, one fact to a file with an index on top, and add a search index only once the folder grows too big to read in one go.
| If you are | Start with |
|---|---|
| Using one agent on your own projects | A memory folder: one index file, one fact per file, read at the start of every session |
| Running several agents that should know the same things | One shared store they all search, such as Postgres with the pgvector extension, or a memory layer such as Mem0 |
| Keeping notes you also read and edit yourself | A notes vault of Markdown files, such as Obsidian, that your agent can search too |
Three things agents need to remember, kept in three places
- Rules: what the agent must always or never do. These go in the rules file, not in memory.
- Facts: what is true about your business, your clients and your setup. These go in the memory folder, one to a file.
- History: what happened and when. These go in logs the agent can search but does not load every time.
Start with plain files
A plain text file is the memory every agent can read. Change your model or your harness next year and the folder comes with you.
Write one fact per file, with its date and why it matters, and keep a short index that points to each one. The agent reads the index first and opens only the files it needs.
Claude Code works this way. It loads the instruction files you write at the start of every session, plus the first 200 lines or 25KB of the notes it keeps for itself.
Add a search index when the folder outgrows one read
Once the folder is too big to read in full, the agent has to search it. A vector index finds notes by meaning rather than by exact words, so a question about refunds finds the note titled "returns policy".
pgvector adds that search to an ordinary Postgres database. Mem0 packages long-term memory for agents and comes as a hosted platform or as open source you run yourself. Letta is a platform for building agents that keep their state between sessions.
What goes wrong with agent memory
- A fact with no date. Six months later nobody can tell whether it is still true.
- Memory that contradicts the rules file. The agent now has two answers and no way to tell which one you meant.
- One giant file. It grows past what the agent loads, and whatever sits past that line is never read.
- A password or API key saved as a memory. Keys belong in a vault, never in a file the agent reads into a chat.
Questions
Does ChatGPT's memory count as agent memory?
It is memory kept inside ChatGPT. A folder of your own files works with whichever agent you use next.
How much memory can an agent load at once?
It depends on the harness. Claude Code, for example, loads the first 200 lines or 25KB of its own notes each session, so long memory needs an index and smaller files.
Sources
- Claude Code: How Claude remembers your project, checked 2026-10-20
- pgvector, checked 2026-10-20
- Mem0 documentation, checked 2026-10-20
- Letta documentation, checked 2026-10-20
- Obsidian, checked 2026-10-20