Agent memory vs knowledge base: what belongs where?

Use memory for continuity

Three homes cover almost everything an agent relies on: memory for compact facts and rules, a knowledge base for documents someone may need to read, and activity history for the record of a single run. Memory is the one people misuse most, so start there.

Memory should carry forward compact facts, preferences, and approved procedures—the things that should change how the agent handles this user or task next time. A good test for a memory entry is one question: what should change how I handle this next time? “This customer prefers email over phone.” “Confirm the shipping address before issuing a refund.” Each is short, durable, and points at future behavior.

Judge memory by whether you can read every active entry in a minute and still trust it. The failure mode is bloat: a memory that has quietly absorbed one-off instructions, stale preferences, and half a pasted document, until the agent acts on a rule you forgot was there. The trade-off is that memory is eager—note “the customer wants a discount” after one chat and the agent may treat it as standing policy. Keep entries that describe durable behavior, and give each one a source and a review date so you can retire it later.

Use a knowledge base for reference

A knowledge base holds source material people may want to read or cite: policies, product guides, research, contracts, and FAQs. Its job is the opposite of memory’s—not to compress information into a rule, but to preserve a document’s identity, owner, and version so anyone can open the original and check it.

This is why grounding an answer in a knowledge base beats letting the agent answer from whatever it happens to remember: when the agent says “per the refund policy,” a person can follow that back to Policy v4 and confirm it. Retrieval-augmented systems are built on exactly this split—an external, updatable store of documents the model reads from, rather than facts baked in and frozen.

Judge a knowledge base by whether every item has an owner and a version, and whether you can trace an answer to a specific page. The failure mode is a folder of unlabeled PDFs with no owners and no dates, where the agent cites an expired contract because nothing marked it expired. The trade-off is upkeep: a knowledge base stays trustworthy only if someone retires old versions and publishes new ones—neglect it and it produces confident answers from stale sources.

Compare the two

Memory and a knowledge base answer different questions, so the same fact can belong in either depending on what you need from it. Read the table one row at a time, asking which column matches how you will actually use the item.

When a row could go either way, let the update method decide. If the right fix is editing one short line, it is memory; if the right fix is publishing a new version of a document, it is the knowledge base.

Do not copy whole documents into memory

The most common mistake is pasting a whole policy or contract into memory so the agent “has it handy.” Save a short memory that points to the authoritative document instead: “Refund policy lives in Policy v4 (owner: Ops); confirm the current version before responding.” The pointer changes behavior; the document stays where it can be versioned.

This matters because copies drift. The moment a document exists in two places, one of them is free to go stale—and the pasted copy in memory has no owner, no version, and no expiry, so it is usually the one that lies. That is the refund window still reading 14 days when the real policy says 30. Keeping a single owned source is the cure, and it is also plain data hygiene: hold only what you need rather than hoarding full copies on the off-chance they help.

The trade-off is that a pointer is only as good as the source it points to. If the knowledge base is neglected, “confirm the current version” still lands on a stale document. A pointer moves the upkeep to one place worth maintaining—it does not remove the upkeep.

A placement rule

When you are unsure, run the decision in this order. If a person may need to inspect the original later, it belongs in the knowledge base. If the agent needs a compact instruction that changes repeated behavior, it belongs in memory. If it only describes one completed run—what the agent did on Tuesday—it belongs in activity history, not memory.

Most items resolve on the first question. The ones that seem to belong in two places usually split cleanly: the document goes to the knowledge base, and a one-line pointer goes to memory. Start there, and the pile of “information” stops being one unreliable drawer and becomes three places you can actually trust.