Engineering the Agentic Stack · Part 2

AI Agent Memory Architecture: Checkpoints and Vector Stores

A reasoning loop only survives one request unless its state is stored outside the worker. Without agent memory, the agent cannot resume a paused plan, recover after a crash, or recall a preference from an earlier session. Part 1 covered the control flow. This post identifies which state each later turn needs and where that state should live.

I’ll use the Market Analyst Agent — a small LangGraph agent that fetches market data and writes an analyst report — to anchor the hot-checkpoint discussion. The cold-vector and raw-Markdown sections are independent illustrative designs that show extensions the current project does not yet implement. Then I’ll cover when PostgreSQL, Redis, Qdrant, key-value stores, and plain Markdown files each make sense.

Every store below is read by the harness, the code that drives the loop around the model. The harness decides which of their contents reach the context window; the stores do not. This article is about where that state lives before the harness reaches for it. Part 3 and Part 4 cover what the harness does with the prompt next.


What is AI agent memory?

AI agent memory is the state layer that lets an agent preserve task progress, retrieve prior knowledge, and update what it knows across runs. In production it is not one vector database. It is a mix of hot checkpoints, cold semantic or structured stores, and human-readable document memory.

NeedBest defaultWhy
Pause and resume one runPostgreSQL checkpoint storeDurable, queryable, and easy to operate with app data
Low-latency transient stateRedis checkpoint storeFast resume and short-lived state, with persistence trade-offs
Cross-thread semantic recallQdrant or pgvectorRetrieves memories by meaning, not only exact keys
Structured user factsPostgreSQL or key-value storeDeterministic updates beat fuzzy retrieval for preferences and IDs
Project conventions and learned proceduresMarkdown or JSON filesHuman-readable, diffable, and easy for agents to update
Multi-entity relationship memoryKnowledge graphUseful when relationships matter more than individual facts

Do not start with memory because it sounds intelligent. Start with the user-visible failure: losing progress, forgetting a preference, repeating research, or failing to reuse a project convention.

The failures that require memory

A stateless agent can answer an isolated question, but it forgets the request as soon as the call ends. That design fails when the product needs any of the following behaviors:

  • Pause and resume: a user starts a research task, closes their laptop, and comes back tomorrow. Without checkpointed state, the agent restarts from scratch.
  • Multi-turn coherence: over a long conversation the agent has to remember what tools it called, what data it gathered, and what plan steps it finished.
  • Personalization: a returning user expects the agent to know their risk tolerance, preferred analysis depth, and past interactions.
  • Human-in-the-loop (HITL): the agent gathers its evidence and waits for a human to approve the next step. The “waiting” state has to survive process restarts.

In the Market Analyst Agent from Part 1, the request “Analyze NVDA” produces a plan, five tool calls, gathered data, and a draft report. When the user replies “looks good, but add competitor analysis,” a checkpoint store lets the agent load the state from the last completed step and add the competitor step. Without checkpointed state, it cannot resolve what “looks good” refers to and has to start over.

Long-term memory handles a different case. If the user returns a week later and asks, “Update my NVDA analysis,” the agent may need to recall a preference for conservative risk assessments and an interest in semiconductor stocks. A vector-backed memory store can retrieve those facts across sessions without asking for them again.

The implementation examples below use LangGraph, LangChain’s open-source library for building agents as explicit state graphs; the storage boundaries it draws generalize to any framework. LangGraph splits memory by scope. Every graph execution runs inside a thread, meaning one conversation or task. Persisted state within that thread is short-term memory. State shared across threads is long-term memory. Below, “thread” and “conversation” are interchangeable; I avoid “session” for that span, because Part 5 reserves it for the durable log of one run, several of which can accumulate on one thread. The model’s current context and in-process variables form the working-memory layer above both stores.

Six agent memory types and the three storage tiers they collapse intoSix agent memory types and the three storage tiers they collapse into


A taxonomy of AI agent memory

Before getting into implementation, it helps to classify what agents need to remember. The CoALA framework — Cognitive Architectures for Language Agents (Sumers, Yao et al., 2023) — is a widely cited taxonomy that draws on cognitive science. I introduced memory scoping in my context engineering post; here I expand it into six categories:

Memory TypeScopeLifetimeExampleStorage Pattern
WorkingCurrent stepMillisecondsTool call arguments, current LLM responseIn-process (Python dict)
Short-termCurrent threadMinutes–hoursConversation history, plan progress, gathered dataCheckpoint store
EpisodicCross-threadDays–months”Last week the user asked about NVDA earnings”Vector store / KV store
SemanticCross-threadMonths–permanent”User prefers conservative investments”Vector store / KV store
DocumentCross-threadDays–permanentProject notes, research summaries, learned patternsFile store (Markdown/JSON)
ProceduralSystem-widePermanent”When analyzing stocks, always check SEC filings”Config / system prompt

Working memory is what the LLM is actively reasoning with right now: Python variables in the current function, the contents of the context window, tool call arguments mid-execution. It’s the fastest and most ephemeral layer. Nothing persists beyond the current step. Working memory is bounded by the model’s context window, which makes it the bottleneck. Everything the agent “knows” at decision time has to fit here, whether it came from the checkpoint store, a vector query, or a file read. The other tiers exist to feed the right information into working memory at the right time.

Short-term memory is the checkpoint LangGraph writes after each unit of graph execution — a super-step, defined in the next section. Episodic and semantic memories persist across threads. Document memory stores project notes, research summaries, and learned conventions in files that people and agents can inspect. Procedural memory lives in system instructions and tool definitions rather than changing for each user.

For implementation, five of those six collapse into three storage tiers. Short-term memory becomes hot memory, the checkpoint for the current thread. Episodic and semantic become cold memory, recall across threads. Document memory keeps accumulated project knowledge readable and directly editable. Working memory is grouped with the hot tier in the taxonomy above, but it is the one that is never really stored: it lives for a single step, in process, and it is the context window the three storage tiers load into. Procedural memory sits outside all three: it lives in the system prompt and tool definitions, so it ships with the agent rather than being stored and retrieved.

CoALA classifies working, episodic, semantic, and procedural memory. The Memory in the Age of AI Agents survey emphasizes vector stores and knowledge graphs, while LangGraph documents checkpoints and its Store interface. File-backed project knowledge sits outside those taxonomies even though Claude Code, Cursor, and Devin Desktop all load persistent project files.

The same storage pattern appears in other domains. A Minecraft agent (Voyager) stores reusable game skills as code libraries, teams in an enterprise document-QA competition iterate on procedural prompt documents, and web agents induce reusable browsing workflows from successful runs. I come back to all three later; the point here is that files make that knowledge inspectable and versionable without a separate embedding service.

Agent-managed memory also differs from a fixed RAG pipeline in who performs the write. The agent or its harness selects what to store, update, and delete, then later chooses when to retrieve it.

The Generative Agents paper (Park et al., 2023) showed how far this can go: simulated agents stored, reflected on, and retrieved their own memories. Its memory stream ranked candidates by recency, importance, and relevance, a design that still provides a useful reference point for agent-memory retrieval.


Short-term agent memory: the checkpoint store

Every time LangGraph finishes a super-step — one node, or a batch of nodes that ran in parallel — the framework serializes the full graph state and writes it to a checkpoint store. That’s the foundation for pause/resume, time-travel debugging, and HITL workflows.

Hot memory: a checkpoint written at every super-step, and the recovery path that reloads itHot memory: a checkpoint written at every super-step, and the recovery path that reloads it

A checkpoint contains the graph state needed to resume: the AgentState from Part 1 — messages, identity, user profile, plan steps, research data, execution mode. Alongside it LangGraph stores its own bookkeeping: the checkpoint’s ID and timestamp, one version per channel (LangGraph’s name for an individual state key), and a separate record of which channel versions each node has already seen. The step number lives on the checkpoint’s metadata rather than on the checkpoint itself. Comparing those two is how the graph works out what runs next. After a HITL interrupt or process restart, the graph loads the checkpoint written at the latest completed boundary and re-enters the next node. It does not continue from an arbitrary Python line. A checkpoint is also different from an append-only event log or trace; Part 5 separates those runtime observability surfaces explicitly.

How LangGraph checkpointing works

LangGraph’s BaseCheckpointSaver is a simple interface: put() writes a checkpoint, get_tuple() reads the latest one for a thread, list() returns the history. Every checkpoint is keyed by (thread_id, checkpoint_ns, checkpoint_id), where thread_id identifies the conversation, checkpoint_ns handles subgraph namespacing, and checkpoint_id is a unique version.

The decision that matters is which backend to put behind it. PostgreSQL and Redis are two common production choices.

PostgreSQL vs Redis

Redis and PostgreSQL as checkpoint backends, compared on latency, durability, and query modelRedis and PostgreSQL as checkpoint backends, compared on latency, durability, and query model

DimensionPostgreSQL (langgraph-checkpoint-postgres)Redis (langgraph-checkpoint-redis)
Durability modelACID transactions, WAL, and replicationConfigurable persistence: an append-only command log (AOF) or periodic snapshots (RDB)
Checkpoint historyDurable history for resume and debuggingRetention depends on saver and eviction settings
Primary constraintDatabase write latency and table growthRAM use, eviction, and persistence configuration
Operational fitTeams already operating relational databasesTeams already operating Redis at high throughput
Best default forDurable resume and reproducible debuggingLatency-sensitive, recoverable session state

Generic database benchmarks do not predict checkpoint performance. Measure the serialized state size, write frequency, persistence settings, and concurrency of your own graph.

PostgreSQL: the durable default

PostgreSQL is the safer default for most teams. Checkpoints survive crashes, you get full transaction semantics, and the checkpoint history makes time-travel debugging straightforward.

A simplified version of the checkpoint setup in memory/hot.py. For production, set LANGGRAPH_STRICT_MSGPACK=true or configure an explicit allowed_msgpack_modules allowlist so checkpoint deserialization permits only safe or declared types; the permissive default warns about unregistered types but allows them.

import asyncio
from contextlib import asynccontextmanager

from langchain_core.messages import HumanMessage
from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver

@asynccontextmanager
async def postgres_checkpointer(connection_string: str):
    """Yield a PostgreSQL-backed checkpoint store.

    PostgreSQL gives us ACID guarantees — if a checkpoint write succeeds,
    the state is durable even if the process crashes immediately after.
    `from_conn_string` is itself an async context manager: it owns the
    connection and closes it on exit, so the graph has to run inside it.
    """
    async with AsyncPostgresSaver.from_conn_string(connection_string) as checkpointer:
        # Create the checkpoint tables if they don't exist.
        # This is idempotent — safe to call on every startup.
        await checkpointer.setup()
        yield checkpointer

async def main() -> None:
    # The graph lives inside the context manager's scope.
    async with postgres_checkpointer(
        "postgresql://user:pass@localhost:5432/agent_memory"
    ) as checkpointer:
        graph = create_graph(checkpointer=checkpointer)

        # Every invoke/stream call now persists state automatically.
        config = {"configurable": {"thread_id": "user-123-session-1"}}
        result = await graph.ainvoke(
            {"messages": [HumanMessage(content="Analyze NVDA")]}, config
        )

        # Resume later on the same thread_id — loads the latest checkpoint.
        result = await graph.ainvoke(
            {"messages": [HumanMessage(content="approved")]}, config
        )

asyncio.run(main())

The AsyncPostgresSaver uses the langgraph-checkpoint-postgres package, which creates four tables: checkpoints (the serialized state), checkpoint_blobs (large binary data), checkpoint_writes (pending writes for crash recovery), and checkpoint_migrations (schema version). Concurrent writers are separated by the primary key (thread_id, checkpoint_ns, checkpoint_id) and upserts rather than by locking — two workers on the same thread will not corrupt each other, but they will not coordinate either.

Redis: when latency is the bottleneck

When checkpoint latency is the bottleneck, Redis is an option for recoverable state. Measure serialized state size, persistence settings, and concurrency before choosing it over PostgreSQL.

A simplified version of the checkpoint setup in memory/hot.py:

import asyncio
from contextlib import asynccontextmanager

from langgraph.checkpoint.redis.aio import AsyncRedisSaver

@asynccontextmanager
async def redis_checkpointer(redis_url: str):
    """Yield a Redis-backed checkpoint store.

    Redis keeps checkpoints in memory for low-latency access.
    Trade-off: less durable than PostgreSQL unless you enable AOF,
    Redis's append-only log — snapshots alone lose recent writes on a crash.
    """
    async with AsyncRedisSaver.from_conn_string(redis_url) as checkpointer:
        # Initialize Redis data structures
        await checkpointer.asetup()
        yield checkpointer

async def main() -> None:
    # Same graph API, different backend.
    async with redis_checkpointer("redis://localhost:6379") as checkpointer:
        graph = create_graph(checkpointer=checkpointer)

asyncio.run(main())

The AsyncRedisSaver from langgraph-checkpoint-redis stores each checkpoint as its own RedisJSON document, under the same (thread_id, checkpoint_ns, checkpoint_id) key as the Postgres saver. The v0.1.0 redesign replaced multiple search operations with a single JSON.GET call, significantly reducing latency. Redis 8.0+ includes RedisJSON and RediSearch by default — no extra modules to install.

For memory-constrained deployments, ShallowRedisSaver stores only the latest checkpoint per thread — no history, but minimal RAM usage. Use this when you need pause/resume but don’t need time-travel debugging.

When to use which

Use PostgreSQL when:

  • You need full checkpoint history for time-travel debugging or reproducible resume
  • Durability is non-negotiable (financial services, healthcare)
  • You already run PostgreSQL in your stack
  • Your agent runs long tasks where losing state means hours of recomputation
  • You want a unified data store — PostgreSQL with pgvector can be a single backend for checkpoints, long-term memory, and vector search

Use Redis when:

  • Checkpoint latency is your bottleneck (real-time chat, streaming UX)
  • You’re building voice bots or streaming experiences where checkpoint access is on a measured latency-critical path
  • You need horizontal scaling across many concurrent threads
  • High-concurrency fan-out patterns where multiple agents share state
  • Short-lived sessions where losing a checkpoint is recoverable
  • You want semantic caching to reduce redundant LLM calls (Redis LangCache caches semantically similar queries to avoid repeated LLM calls)

Other options: langgraph-checkpoint-sqlite works for local development and single-process deployments. For AWS-native stacks, langgraph-checkpoint-aws provides a DynamoDBSaver with automatic payload offloading — small checkpoints (<350 KB) stay in DynamoDB, larger ones spill to S3. Serverless pricing and no infrastructure to manage make it attractive for variable-load deployments.


Long-term memory: remembering across sessions

Hot memory handles the current conversation. Long-term memory covers the user who comes back next week: it stores facts, preferences, and interaction history that persist across threads.

LangGraph provides a Store interface for cross-thread memory via its BaseStore class. Each memory item is a (namespace, key) pair with a JSON value and optional vector embedding. The namespace typically encodes the user or organization: ("user", "user-123", "preferences").

The cold-memory retrieval path: embed the query, search Qdrant filtered by user, rescore, injectThe cold-memory retrieval path: embed the query, search Qdrant filtered by user, rescore, inject

Vector storage: semantic recall with Qdrant

When the agent needs to recall unstructured facts (“What did the user say about their investment timeline?”), vector search provides semantic recall. Instead of exact key lookups, the agent queries by meaning.

Qdrant is a purpose-built vector database written in Rust that handles embedding storage, indexing (Hierarchical Navigable Small World, or HNSW), and filtered search. I covered HNSW and its trade-offs in detail in my search ranking post. Qdrant also offers an MCP server that acts as a semantic memory layer — useful if your agent framework supports the Model Context Protocol.

The following is an independent illustrative Qdrant design. It is not a simplified version of the current memory/long.py. The current project stores user profiles with exact user_id filtering and a zero-vector placeholder. Real embedding integration remains future work. The request handler must authenticate the request and construct principal from its verified identity; the client never supplies it. The Qdrant filter is retrieval scope, not authorization.

from qdrant_client import QdrantClient
from qdrant_client.models import (
    PointStruct, Distance, VectorParams, Filter, FieldCondition, MatchValue,
)
import hashlib
from dataclasses import dataclass

@dataclass(frozen=True)
class AuthenticatedPrincipal:
    """Created by the server after authentication, never from request JSON."""
    user_id: str

class UserMemoryStore:
    """Long-term memory backed by Qdrant vector search.

    Stores user facts as embedded vectors for semantic retrieval.
    Each fact is a short natural-language statement about the user.
    """

    def __init__(self, qdrant_url: str, collection_name: str = "user_memory"):
        self.client = QdrantClient(url=qdrant_url)
        self.collection_name = collection_name
        self._ensure_collection()

    def _ensure_collection(self):
        """Create the collection if it doesn't exist."""
        collections = [c.name for c in self.client.get_collections().collections]
        if self.collection_name not in collections:
            self.client.create_collection(
                collection_name=self.collection_name,
                vectors_config=VectorParams(
                    size=1536,  # text-embedding-3-small dimensions
                    distance=Distance.COSINE,
                ),
            )

    def store_fact(
        self, principal: AuthenticatedPrincipal, fact: str, embedding: list[float]
    ):
        """Store a user fact with its embedding."""
        point_id = hashlib.md5(f"{principal.user_id}:{fact}".encode()).hexdigest()
        self.client.upsert(
            collection_name=self.collection_name,
            points=[PointStruct(
                id=point_id,
                vector=embedding,
                payload={"user_id": principal.user_id, "fact": fact},
            )],
        )

    def recall(
        self,
        principal: AuthenticatedPrincipal,
        query_embedding: list[float],
        top_k: int = 5,
    ):
        """Retrieve the most relevant facts for a user given a query."""
        results = self.client.query_points(
            collection_name=self.collection_name,
            query=query_embedding,
            query_filter=Filter(
                must=[FieldCondition(
                    key="user_id", match=MatchValue(value=principal.user_id)
                )]
            ),
            limit=top_k,
        )
        return [hit.payload["fact"] for hit in results.points]

The flow has three steps. In this illustrative design, an LLM extracts key facts from the interaction (“user has high risk tolerance”, “user is interested in semiconductor stocks”). Those facts are embedded and stored in Qdrant. At the start of the next conversation, the server supplies the authenticated principal and the agent queries Qdrant with the user’s new message to recall relevant context. The current Market Analyst Agent does not yet implement this semantic extraction and embedding flow.

Retrieval scoring: beyond cosine similarity

Raw cosine similarity is a starting point, but production memory systems need richer retrieval. The Generative Agents paper (Park et al., 2023) introduced a scoring function that combines three signals:

  • Recency: Rule-based decay so recent memories score higher. An exponential decay function makes a fact from yesterday outrank an equivalent fact from six months ago.
  • Importance: LLM-rated significance on a 1-10 scale. “User’s portfolio is down 40%” scores higher than “user said hello.”
  • Relevance: Embedding cosine similarity between the query and the stored fact.

The final retrieval score is a weighted sum: score = alpha * recency + beta * importance + gamma * relevance. That keeps fresh, important facts from being buried under stale but semantically similar ones. For an agent like the Market Analyst Agent, I’d start at alpha = 0.3 for recency, beta = 0.2 for importance, and gamma = 0.5 for relevance, since the user’s current query intent matters most. These are starting-point weights adapted from the Generative Agents paper (which used equal weighting); I found emphasizing relevance worked better for financial analysis queries, but the values are intuition-based, not empirically optimized.

Vector search is powerful but not always the right tool. Here’s when to use alternatives:

ApproachBest forMain operational cost
Vector search (Qdrant)Semantic recall of unstructured factsEmbedding and index lifecycle
Key-value store (Redis)Structured user profiles and preferencesMemory use and persistence policy
Document store (files)Project knowledge and agent-managed notesConcurrency, permissions, and search
Full-text search (PostgreSQL GIN index)Keyword recall over conversation historyIndex growth and query tuning
Knowledge graph (Neo4j)Entity relationships and multi-hop queriesGraph modeling and another data system
Hybrid (vector + keyword)Recall when query intent variesTwo scoring paths to tune and evaluate

Key-value stores work well for structured data. If your long-term memory is a user profile — risk tolerance, investment horizon, preferred sectors — a Redis hash or PostgreSQL JSONB column is simpler and faster than embedding and querying vectors. Use vector search when the memory is unstructured and the retrieval query varies in phrasing.

LangGraph’s built-in Store provides a namespace-based key-value interface with optional vector search. The BaseStore API is simple: put(), get(), search(), and delete() with hierarchical namespace scoping. Three implementations are available:

  • InMemoryStore — for development and testing (data lost on process exit)
  • PostgresStore — production persistent store with full SQL querying
  • AsyncRedisStore — cross-thread memory with vector search, TTL support, and metadata filtering

The index configuration enables vector search over stored items using a configurable embedding model. For many use cases, this built-in store is sufficient without reaching for a dedicated vector database.

import asyncio
from langgraph.store.memory import InMemoryStore

# Create a store with vector search enabled
store = InMemoryStore(
    index={
        "dims": 1536,
        "embed": my_embedding_function,  # e.g., OpenAI text-embedding-3-small
    }
)

async def main() -> None:
    # Store a user preference (namespace scopes to user).
    await store.aput(
        namespace=("user", "user-123", "preferences"),
        key="risk-profile",
        value={"risk_tolerance": "high", "horizon": "long-term"},
    )

    # Semantic search across the user's memories.
    # The namespace prefix is positional here — `search`/`asearch` declare it
    # as positional-only `namespace_prefix`, unlike `aput`.
    results = await store.asearch(
        ("user", "user-123"),
        query="What is their investment style?",
        limit=5,
    )

asyncio.run(main())

Choosing a long-term memory strategy

Start with key-value if your memory is structured and well-defined (user profiles, settings, named entities). Add vector search when you need semantic retrieval over unstructured facts or when the query phrasing varies unpredictably.

Knowledge graphs earn their keep when relationships between entities matter, e.g. “Which companies did the user ask about that are competitors of NVDA?” The most interesting recent project here is Graphiti (by Zep), which builds a temporally-aware knowledge graph that tracks when facts were true, not just what was true. Every edge carries validity intervals, so a change to the user’s risk tolerance invalidates the old value rather than silently overwriting it. Graphiti reports 94.8% accuracy on the DMR benchmark — Deep Memory Retrieval, a long-conversation recall test — and its bi-temporal model handles the stale memory problem at the data layer.

The catch is operational. Running a graph database is non-trivial, and for most agent applications vector search with metadata filtering covers the same ground with less infrastructure.

Managed memory frameworks like Mem0 and Letta (formerly MemGPT) handle the extraction-consolidation-retrieval pipeline for you. Mem0’s approach is notable: an LLM extracts candidate memories, a decision engine compares each new fact against existing entries in the vector store, and a resolver decides to add, update, delete, or do nothing. That keeps the memory store coherent and non-redundant. Letta takes an operating-systems angle: agents manage their own context window using memory management tools, autonomously moving data between “core memory” (in-context) and “archival memory” (out-of-context). Both are worth evaluating if you want faster time-to-production and don’t need full control over the memory pipeline.


Document memory: the agent’s filing cabinet

Vector stores and key-value backends handle semantic recall and structured lookups well. There’s a third category of agent knowledge that neither serves cleanly: accumulated project context, the conventions, research notes, and decisions the agent needs across sessions. That knowledge benefits from being human-readable and version-controlled.

This is document memory: the agent reads and writes structured files (Markdown, JSON, YAML) to a known directory. No embeddings, no database, no infrastructure. Just files on disk that both the agent and the developer can cat, grep, git diff, and edit by hand.

It has more product adoption than coverage in the memory taxonomies above. In one vendor-run evaluation, Letta reported 74.0% accuracy on LoCoMo — a long-conversation question-answering benchmark — for a filesystem-backed agent running on GPT-4o mini, against 68.5% for Mem0’s best graph variant. That is one vendor, one model, one benchmark, and one harness: read it as a sign the approach is competitive, not as a ranking. The operational advantage does not depend on the benchmark: developers can read, edit, and diff the stored knowledge directly.

Longer context windows also make whole-file reads practical for some project documents. Chunked retrieval still fits large corpora, but a short conventions or handoff file can often be loaded directly. The choice depends on document size, retrieval precision, context budget, and how often people need to review or edit the memory.

Why files?

For long-lived agent workflows, the most effective pattern I’ve seen isn’t a vector database. It’s a directory of well-organized notes. Consider what happens when a coding agent works on a project over weeks:

  • It learns that the project uses Pydantic v2, not v1
  • It discovers that tests must run with pytest -x --tb=short
  • It accumulates knowledge about the codebase architecture
  • It learns the developer’s preferences (“always use pathlib, never os.path”)

These facts are too structured for vector search (you need exact recall, not fuzzy similarity) and too interconnected for a key-value store — they read as documents that reference each other, not as isolated values you fetch by key. They’re also facts the developer wants to see and edit directly. If the agent learns something wrong, you open the file and fix it.

This is how Claude Code’s CLAUDE.md and .claude/ directory work. The agent reads project-level CLAUDE.md files for conventions and instructions, and keeps a separate auto-memory file per project — under ~/.claude/projects/<project-slug>/memory/ — for cross-session learnings. Both are plain Markdown: you read them, edit them, commit the project ones to git, share them with your team. Cursor’s project rules and Devin Desktop’s rules and memories follow the same pattern. Cursor reads .mdc files from .cursor/rules; Devin Desktop (formerly Windsurf) reads .windsurf/rules/ and still honors the legacy single-file .windsurfrules. Either way: plain text on disk that the agent loads on startup to pick up project context.

Implementing a file memory store

The implementation is deliberately simple. The agent gets four operations: write a document, read a document, list available documents, and search across documents by keyword.

The following is an independent illustrative raw-Markdown file store. It is not a simplified version of the current memory/document.py. The current project uses DocumentMemory, which requires a namespace and key and writes a JSON envelope containing content, metadata, and created_at. This sketch defines a different design to show the trade-off of human-readable Markdown files:

from pathlib import Path
import json

class FileMemory:
    """Document memory backed by the local filesystem.

    Stores agent knowledge as human-readable files organized by topic.
    No embeddings, no database — just files that both the agent and
    the developer can read, edit, and version-control.
    """

    def __init__(self, base_dir: str | Path):
        self.base_dir = Path(base_dir).resolve()
        self.base_dir.mkdir(parents=True, exist_ok=True)

    def _resolve_path(self, path: str) -> Path:
        """Return a path inside base_dir, rejecting escapes and symlinks."""
        requested = Path(path)
        if requested.is_absolute() or ".." in requested.parts:
            raise ValueError("path must be relative to base_dir without traversal")
        resolved = (self.base_dir / requested).resolve()
        try:
            resolved.relative_to(self.base_dir)
        except ValueError as error:
            raise ValueError("path must stay inside base_dir") from error
        return resolved

    def write_doc(self, path: str, content: str, metadata: dict | None = None):
        """Write or overwrite a document at the given path.

        Paths are relative to base_dir. Directories are created automatically.
        Metadata (if provided) is stored as a JSON sidecar file.
        """
        full_path = self._resolve_path(path)
        full_path.parent.mkdir(parents=True, exist_ok=True)
        full_path.write_text(content, encoding="utf-8")

        if metadata:
            meta_path = self._resolve_path(
                str(full_path.relative_to(self.base_dir).with_suffix(full_path.suffix + ".meta"))
            )
            meta_path.write_text(json.dumps(metadata, indent=2), encoding="utf-8")

    def read_doc(self, path: str) -> str | None:
        """Read a document by path. Returns None if not found."""
        full_path = self._resolve_path(path)
        if full_path.exists():
            return full_path.read_text(encoding="utf-8")
        return None

    def list_docs(self, pattern: str = "**/*") -> list[str]:
        """List documents matching a glob pattern."""
        self._resolve_path(pattern)
        return [
            str(self._resolve_path(str(p.relative_to(self.base_dir))).relative_to(self.base_dir))
            for p in self.base_dir.glob(pattern)
            if self._resolve_path(str(p.relative_to(self.base_dir))).is_file()
            and not p.name.endswith(".meta")
        ]

    def search_docs(self, query: str, pattern: str = "**/*.md") -> list[dict]:
        """Search documents by keyword. Returns matching files with context.

        This is intentionally simple — grep-style keyword search.
        For semantic search, use a vector store instead.

        NOTE: This is a sketch for demonstration. A simple substring check
        won't scale beyond a few hundred documents. For production with 500+
        documents, use TF-IDF/BM25 scoring (e.g., rank_bm25) or a full-text
        search backend (PostgreSQL GIN index, Elasticsearch).
        """
        self._resolve_path(pattern)
        results = []
        for path in self.base_dir.glob(pattern):
            path = self._resolve_path(str(path.relative_to(self.base_dir)))
            if not path.is_file() or path.name.endswith(".meta"):
                continue
            content = path.read_text(encoding="utf-8")
            if query.lower() in content.lower():
                # Return the paragraph containing the match for context
                for paragraph in content.split("\n\n"):
                    if query.lower() in paragraph.lower():
                        results.append({
                            "path": str(path.relative_to(self.base_dir)),
                            "match": paragraph.strip()[:500],
                        })
        return results

The path helper is deliberately shared by reads, writes, and glob results: relative paths can still leave a directory through .. or an existing symlink. This illustrative class is for a trusted single-user or controlled filesystem. It checks a resolved path before use; at a hostile multi-tenant boundary, use descriptor-relative no-follow operations so a filesystem mutation cannot race that check. Run this small regression check after copying the class:

from tempfile import TemporaryDirectory

with TemporaryDirectory() as root:
    memory = FileMemory(root)
    memory.write_doc("notes/ok.md", "safe memory")
    assert memory.read_doc("notes/ok.md") == "safe memory"
    assert memory.list_docs() == ["notes/ok.md"]
    assert memory.search_docs("safe")[0]["path"] == "notes/ok.md"

    (Path(root) / "escape").symlink_to(Path(root).parent, target_is_directory=True)
    for operation in (
        lambda: memory.write_doc("../escape.md", "nope"),
        lambda: memory.read_doc("/tmp/escape.md"),
        lambda: memory.read_doc("escape/outside.md"),
        lambda: memory.list_docs("../**/*"),
        lambda: memory.search_docs("safe", "../**/*.md"),
    ):
        try:
            operation()
        except ValueError:
            pass
        else:
            raise AssertionError("FileMemory accepted an escaped path")

Folder structure

Most of the value of document memory comes from how the directory is laid out. Here’s the shape I’d use for a research agent. The Market Analyst Agent uses namespaces under memory/documents/, but its current DocumentMemory writes each entry as a JSON envelope with a content string rather than raw Markdown. The raw-Markdown layout below belongs to the independent illustrative FileMemory design above:

.agent-memory/
    README.md                  # What this directory is, for human readers
    PROGRESS.md                # Handoff for the next session: what is done, what is next
    user-profiles/
        user-123.md            # Preferences, history, risk profile
        user-456.md
    research/
        NVDA-2026-02.md        # Research notes from recent analysis
        TSLA-2026-01.md
    conventions/
        analysis-format.md     # How to structure analysis reports
        data-sources.md        # Preferred data sources and API patterns
    learnings/
        common-errors.md       # Mistakes the agent has learned to avoid
        tool-patterns.md       # Effective tool call sequences

The document memory directory and the four operations an agent runs against it: read, write, list, and searchThe document memory directory and the four operations an agent runs against it: read, write, list, and search

In the illustrative FileMemory design, every document is Markdown, and every document’s purpose is obvious from its path. You can git diff the entire memory directory to see what the agent learned in a session, git revert a bad learning, or copy the directory to another project. The current project’s JSON envelopes keep the namespace and key structure, but they do not provide the same raw-Markdown diff experience.

When to use document memory vs vector vs key-value

The three memory backends serve different access patterns:

DimensionVector StoreKey-Value StoreDocument Store
Query pattern”Find facts similar to X""Get the value for key""Read the doc at path”
Best forUnstructured, varied recallStructured lookupsProject context, notes
Human readableNo (embeddings)Partially (JSON)Yes (Markdown)
DebuggableHard (similarity scores)Easy (exact keys)Trivial (open the file)
Version controllableNoPossibleYes (git-native)
Embedding infrastructureRequiredNot neededNot needed
Scales toMillions of factsMillions of keysThousands of documents
Search capabilitySemantic similarityExact matchKeyword / path-based

Use document memory when:

  • The agent accumulates project knowledge over multiple sessions
  • Developers need to inspect, edit, or override what the agent “knows”
  • The knowledge is structured as documents (notes, summaries, conventions) rather than isolated facts
  • You want git-based versioning of agent memory
  • Zero infrastructure is a hard requirement

Use vector stores when:

  • You need fuzzy semantic retrieval (“find memories related to X”)
  • The query phrasing varies unpredictably
  • You have thousands to millions of individual facts

Use key-value stores when:

  • You need exact, fast lookups for structured data (user profiles, settings)
  • The data schema is well-defined

In practice, production agents often combine all three. The current Market Analyst Agent uses PostgreSQL checkpoints for hot memory, Qdrant for exact user-profile storage with placeholder vectors, and a namespaced JSON-envelope document store. The semantic-recall and raw-Markdown variants in this article are illustrative extensions.

Real-world examples

The pattern is already widespread in AI coding assistants:

  • Claude Code reads CLAUDE.md files from the project root and parent directories, and maintains a per-project memory file under ~/.claude/projects/ for cross-session learnings. The memory system is plain Markdown files, and the project-level ones commit alongside your code.
  • Cursor loads project rules from .cursor/rules as .mdc files — coding conventions, framework preferences, architectural decisions — with frontmatter controlling when each rule applies.
  • Devin Desktop (formerly Windsurf) reads rules from .windsurf/rules/, still honors the legacy root-level .windsurfrules, and writes autogenerated memories to a local store the agent consults on later runs.
  • Anthropic’s memory tool for the Claude API is a client-side tool the model drives with file operations — view, create, str_replace, insert, delete, and rename — over a /memories directory. Your application implements each command, so it decides where the files actually live (local disk, S3, database).

All of these store agent knowledge as human-readable text files with explicit read/write operations, and none of them needs an embedding pipeline. The agent decides what to write, the developer can see and edit everything, and the whole system fits in a git diff.

Beyond coding assistants

Document memory is not limited to coding agents. The pattern shows up across other agent domains too:

  • Open-world game agents: Voyager (Wang et al., 2023) builds a persistent skill library of verified JavaScript programs that a Minecraft agent accumulates over time, collecting 3.3x more unique items and reaching milestones 15.3x faster than baselines. Skills transfer across new worlds without retraining. JARVIS-1 extends this with a multimodal memory that combines textual plans and visual observations, and is five times more reliable than the previous best agents on the long-horizon ObtainDiamondPickaxe task.

    One distinction worth making here: skill libraries are executable memory (code files imported and run), while document memory in coding assistants is declarative (Markdown injected into prompts). The failure modes differ. Bad executable code crashes the agent; bad declarative text leads to reasoning errors. But the storage pattern and the operational benefits (debuggability, version control) are the same.

  • Enterprise workflow automation: The ERC3 competition — the third Enterprise RAG Challenge — had winners who used document memory for iterative prompt refinement. One winning team’s Analyzer and Versioner agents iterated through 80 prompt versions stored as procedural documents. Another top team built 20+ enricher modules as document-style procedural knowledge. LEGOMem (2025) formalizes this for multi-agent systems as modular procedural memory: past task trajectories are decomposed into reusable memory units, which are then placed either with the orchestrator that plans and delegates or with the agents that execute the steps. On the OfficeBench benchmark, orchestrator memory turned out to be the one that matters for task decomposition, while fine-grained agent memory improved execution accuracy.

  • Web automation: Agent Workflow Memory (Wang et al., 2024) lets web agents induce reusable workflows from successful episodes, with a 51.1% relative success rate improvement on WebArena. SkillWeaver (2025) goes further: agents synthesize reusable API tools from exploration, with a 31.8% relative success rate gain. The learned skills also transfer to weaker models (up to 54.3% relative improvement), so a stronger agent’s accumulated memory can lift a smaller one.

  • Customer support: Gartner predicts that agentic AI will autonomously resolve 80% of common customer service issues without human intervention by 2029. These agents reference SOPs, playbooks, and customer histories, which are all forms of document memory.

The MemAgents workshop at ICLR 2026 is one sign the research community is catching up to what practitioners have already built.

Skills use documents to package procedural instructions. The Agent Skills standard stores those instructions in SKILL.md files with YAML frontmatter and a Markdown body. That resembles document memory at the storage layer, but the role is different: a skill tells the agent how to perform a class of work, while memory records facts learned from a project or prior run. Part 3 draws the neighboring line, between a skill and a tool.

MCP (Model Context Protocol) has a related procedural interface: tools/list returns tool objects whose inputSchema is JSON Schema, and an agent invokes one with tools/call. Discovery does not authorize a call. Before invoking a tool with effects or private data, the host must implement authentication, authorization, and explicit user consent; the server must also enforce its own access controls. MCP cannot enforce those controls at the protocol level. A December 2025 review of MCP’s first year put the protocol at 97 million monthly SDK downloads across Python and TypeScript, with adoption by OpenAI, Google DeepMind, and Microsoft. MCP is not coding-specific. The same servers connect agents to databases, internal APIs, and enterprise systems.

Both make procedural interfaces inspectable: skills store instructions in documents, while MCP exposes machine-readable tool schemas and calls. MCP, now governed by the Agentic AI Foundation, is the closest thing to an interop standard the agent ecosystem has.

Scaling document memory for production

The file-based implementation above works well for single-developer laptops and small-scale deployments. Multi-tenant production with hundreds of users and thousands of documents requires a different architecture.

The single-node file limit becomes obvious: you can’t scale file I/O horizontally, concurrent writes need locking, and managing permissions across tenants is painful. Production needs a backing store that handles concurrency, search, and multi-tenancy properly.

Three common approaches:

Approach A: hybrid with a thin database layer

Keep files for authoring (developers edit Markdown locally) but serve from a database at runtime. On deployment, sync files to PostgreSQL rows. The agent reads from the database, not disk. This gives you:

  • Developer ergonomics (edit Markdown, commit to git)
  • Production query performance (indexed database reads)
  • Clean separation between authoring and serving

Approach B: object storage + vector index sidecar

Store documents in S3/GCS as objects, with a Qdrant collection that indexes their embeddings. The agent queries Qdrant for relevant document IDs, then fetches content from object storage. This scales horizontally and supports semantic search, but adds complexity: two systems to manage, an embedding pipeline to maintain, and eventual consistency between store and index.

Approach C: structured document store with PostgreSQL (recommended)

Store documents as PostgreSQL JSONB rows with full-text search (GIN index) and optional vector embeddings (pgvector). This gives you hybrid search (keyword + semantic), ACID transactions, and a single operational system.

A sketch of Approach C. This is an RLS pattern, not drop-in application code: its database role must be available only to the trusted application server. The server authenticates the request and constructs principal; it does not accept a tenant ID from the caller. PostgreSQL RLS then makes that scope enforceable even if a query later omits its tenant predicate.

from typing import Optional
from dataclasses import dataclass
import asyncpg

@dataclass(frozen=True)
class AuthenticatedPrincipal:
    """The verified identity returned by the application's authentication layer."""
    tenant_id: str

class ProductionDocumentMemory:
    """Illustrative PostgreSQL document memory with hybrid search and RLS.

    Apply this schema and policy as the table owner during deployment:

        CREATE TABLE documents (
            id SERIAL PRIMARY KEY,
            tenant_id TEXT NOT NULL,
            path TEXT NOT NULL,
            content TEXT NOT NULL,
            metadata JSONB,
            embedding vector(1536),  -- pgvector extension
            ts_vector tsvector GENERATED ALWAYS AS (to_tsvector('english', content)) STORED,
            created_at TIMESTAMPTZ DEFAULT NOW(),
            UNIQUE(tenant_id, path)
        );
        CREATE INDEX ON documents USING GIN(ts_vector);
        CREATE INDEX ON documents USING ivfflat(embedding vector_cosine_ops);

        ALTER TABLE documents ENABLE ROW LEVEL SECURITY;
        ALTER TABLE documents FORCE ROW LEVEL SECURITY;
        CREATE POLICY tenant_documents ON documents
            USING (tenant_id = current_setting('app.tenant_id', true))
            WITH CHECK (tenant_id = current_setting('app.tenant_id', true));

    `FORCE` also subjects the table owner to the policy. Superusers and roles with
    `BYPASSRLS` still bypass it, so neither belongs in the application's pool.
    """

    def __init__(self, pool: asyncpg.Pool):
        self.pool = pool

    async def write(
        self,
        principal: AuthenticatedPrincipal,
        path: str,
        content: str,
        metadata: Optional[dict] = None,
        embedding: Optional[list[float]] = None,
    ):
        """Write or update a document.

        Sketch: on a real pool you must register codecs first, or asyncpg
        raises DataError — `set_type_codec` for the JSONB metadata column
        and pgvector's `register_vector` for the embedding.
        """
        async with self.pool.acquire() as conn:
            async with conn.transaction():
                # true keeps this trusted context to this transaction only.
                await conn.execute(
                    "SELECT set_config('app.tenant_id', $1, true)", principal.tenant_id
                )
                await conn.execute(
                    """
                    INSERT INTO documents (tenant_id, path, content, metadata, embedding)
                    VALUES ($1, $2, $3, $4, $5)
                    ON CONFLICT (tenant_id, path) DO UPDATE
                    SET content = EXCLUDED.content,
                        metadata = EXCLUDED.metadata,
                        embedding = EXCLUDED.embedding
                    """,
                    principal.tenant_id, path, content, metadata, embedding,
                )

    async def search(
        self,
        principal: AuthenticatedPrincipal,
        query: str,
        embedding: Optional[list[float]] = None,
        limit: int = 5,
    ) -> list[dict]:
        """Hybrid search: full-text + optional vector similarity."""
        async with self.pool.acquire() as conn:
            async with conn.transaction():
                await conn.execute(
                    "SELECT set_config('app.tenant_id', $1, true)", principal.tenant_id
                )
                if embedding:
                    # Hybrid scoring: 0.6 * text relevance + 0.4 * vector similarity
                    rows = await conn.fetch(
                        """
                        SELECT path, content, metadata,
                               (0.6 * ts_rank(ts_vector, plainto_tsquery('english', $1)) +
                                0.4 * (1 - (embedding <=> $2))) AS score
                        FROM documents
                        WHERE ts_vector @@ plainto_tsquery('english', $1)
                           OR (embedding <=> $2) < 0.5
                        ORDER BY score DESC
                        LIMIT $3
                        """,
                        query, embedding, limit,
                    )
                else:
                    # Full-text search only
                    rows = await conn.fetch(
                        """
                        SELECT path, content, metadata,
                               ts_rank(ts_vector, plainto_tsquery('english', $1)) AS score
                        FROM documents
                        WHERE ts_vector @@ plainto_tsquery('english', $1)
                        ORDER BY score DESC
                        LIMIT $2
                        """,
                        query, limit,
                    )
                return [dict(row) for row in rows]

set_config(..., true) is transaction-scoped, so a pooled connection cannot retain one tenant’s context for the next request. The OR in the first branch is what makes it hybrid. With only the @@ predicate, a document that means the right thing but shares no keywords with the query gets filtered out before scoring ever runs — that is keyword retrieval with semantic re-ranking, not hybrid retrieval. The distance threshold is a knob: tighten it if the vector arm floods the results, loosen it if semantic matches never surface.

The following regression is the behavior to test against a real database after migrations. Under tenant-a, a read of tenant-b returns no rows and a direct cross-tenant insert fails RLS:

BEGIN;
SELECT set_config('app.tenant_id', 'tenant-a', true);
SELECT path FROM documents WHERE tenant_id = 'tenant-b'; -- 0 rows
INSERT INTO documents (tenant_id, path, content)
VALUES ('tenant-b', 'leak.md', 'must fail'); -- ERROR: row-level security policy
ROLLBACK;

What you get:

  • Hybrid search: keyword matching (GIN index) + semantic similarity (pgvector) scored together
  • Multi-tenancy: server-derived identity plus database-enforced RLS
  • ACID guarantees: no eventual consistency issues
  • Single operational system: no separate vector database to manage
  • Horizontal scaling: read replicas for query load, partitioning by tenant for write scale

Files are great for single-developer workflows. For multi-tenant production, a structured document store on PostgreSQL is usually the right balance of simplicity, performance, and operational maturity.


Putting it together: the full architecture

Here’s how all three memory tiers can work together in an architecture inspired by the Market Analyst Agent. The diagram shows an illustrative flow from user request to response, with all memory layers active.

All three memory tiers wired around one agent and their read and update pathsAll three memory tiers wired around one agent and their read and update paths

The architecture has three memory paths:

  1. Hot path (checkpoint store): LangGraph writes the resumable graph state to the checkpoint store at every super-step boundary. When the graph hits an interrupt_before node (like the publish node in Part 1), execution pauses. The user can close the app, and when they return, the graph resumes from the checkpoint. Runtime event logs and traces are separate production concerns.

  2. Cold path (long-term store): In this illustrative architecture, the agent queries the long-term store for relevant user context at the start of each conversation. That read is on the critical path — the planner cannot personalize until it returns. A vector-backed path may include query embedding plus index retrieval; a key-value lookup does not. The write is not: once the conversation ends, a background job extracts and stores new facts, and that job should never block the reasoning loop.

  3. Document path (file store): At startup, the agent loads project conventions and relevant research notes from the document store. During execution, it writes new research summaries and learned patterns back to disk. These reads are on the critical path too, because they inform the current task; their cost depends on the filesystem, file size, and cache state. Writes can be deferred.

The wiring in LangGraph is straightforward — the checkpoint store and long-term store are passed at graph compilation, while the document store is injected as a dependency. The local sketch below uses InMemoryStore so the snippet stays small; the reference Docker topology uses Qdrant for the same semantic-recall role.

import asyncio
from langgraph.store.memory import InMemoryStore

# Cold memory: local sketch with vector search
# (The reference Docker topology uses Qdrant for persistent recall.)
memory_store = InMemoryStore(
    index={"dims": 1536, "embed": embedding_function}
)

# Document memory: illustrative file-based store for project knowledge
# FileMemory is the illustrative class defined above, not the project's current
# DocumentMemory implementation.
doc_memory = FileMemory(base_dir=".agent-memory")

async def main() -> None:
    # Hot memory: PostgreSQL for durable checkpoints. postgres_checkpointer() is
    # the async context manager defined earlier, so the graph runs inside it.
    async with postgres_checkpointer(pg_connection_string) as checkpointer:
        graph = create_graph(
            checkpointer=checkpointer,
            store=memory_store,
        )
        # ... run the graph here, while the connection is still open

asyncio.run(main())

# The store is accessible inside any node via the store parameter
def planner_node(state: AgentState, *, store: BaseStore) -> dict:
    """Plan with user context from long-term memory."""

    # Recall relevant user facts from vector store.
    # Namespace prefix is positional — see the store example above.
    user_memories = store.search(
        ("user", state.user_id),
        query=state.messages[-1].content,
        limit=5,
    )

    # Load project conventions from document memory
    conventions = doc_memory.read_doc("conventions/analysis-format.md")

    # Inject both into planning context
    # Each stored value is a dict; render whatever keys it carries
    memory_context = "\n".join(str(m.value) for m in user_memories)
    # ... rest of planning logic with personalized context and conventions

The complete flow

What happens when a returning user sends “Analyze TSLA” to the Market Analyst Agent:

  1. Document memory load: At startup, the agent reads project conventions from the document store: analysis format preferences, preferred data sources, tool usage patterns. These set the baseline behavior.

  2. Cold memory recall: In this illustrative flow, before the router node executes, the graph queries the long-term store with the user’s message. It retrieves: “User has high risk tolerance”, “User prefers detailed competitor analysis”, “User previously researched NVDA and AMD”.

  3. Router + Planner: The router classifies this as DEEP_RESEARCH. The planner creates a 5-step research plan personalized to the recalled preferences. It includes a competitor analysis step because the user’s history shows they want one. The plan follows the format from the conventions document.

  4. Executor loop (hot memory): Each step executes via the ReAct pattern from Part 1 — think, act, observe, repeated until the step is done. After every super-step (router, planner, each executor step run sequentially here) LangGraph writes a checkpoint to PostgreSQL. If the process crashes after step 3 of 5, you restart and continue from step 4.

  5. HITL interrupt: The reporter writes a draft, a fresh-context evaluator — a second model session with no history of the run — votes on it, and the graph reaches the publish node with interrupt_before and pauses. The checkpoint holds the draft plus the evaluator’s verdict, so the human reviews both rather than adjudicating raw research. They review it hours later, and the graph loads the checkpoint and publishes.

  6. Memory updates: After the conversation ends, an asynchronous process extracts new user facts (“user is now tracking TSLA”, “user approved the report format”) and stores them in the long-term vector store. The agent also writes a research summary to the document store (research/TSLA-2026-02) for future reference.

The three-tier pattern separates concerns cleanly. The checkpoint store handles durability and resume; it’s infrastructure. The long-term store handles personalization; it’s product logic. The document store holds accumulated project knowledge; it’s the agent’s notebook.


Trade-offs and considerations

Memory adds value, but it also adds cost and complexity:

  • Embedding cost: Every fact stored in a vector database requires an embedding API call. As of September 2026, OpenAI lists text-embedding-3-small at $0.02 per million tokens, so per-fact cost is negligible, but it adds up across thousands of users and sessions. Batch embedding calls and cache results. At query time, vector recall can include query embedding plus index and network latency; a key-value lookup does not. Measure that path in your deployment, then cache common query embeddings or use a local embedding model if it is latency-sensitive.

  • Stale memory: User preferences change. A fact stored six months ago (“user prefers conservative investments”) may no longer be accurate. Set expiry policies. In one of my designs, I use 365 days for preferences and 90 days for episodic events as provisional examples, not universal defaults. The context engineering post rejects fixed retention rules as portable policy. Expiry is the blunt version. Schema-guided typed state covers the sharper one: temporal validity and provenance on each fact, so a superseded value loses to the current one at retrieval time rather than at expiry.

  • Memory overhead in context: Every recalled fact consumes tokens in the LLM’s context window. If you recall 20 facts per query, that’s several hundred tokens of memory context competing with the actual task. Cap the number of recalled facts and prioritize by relevance score.

  • Privacy and compliance: Long-term memory stores user data. You need PII redaction before storage, clear retention policies, and user-facing controls for data deletion. None of this is optional in regulated industries.

  • Checkpoint storage growth: PostgreSQL checkpoint tables grow with every super-step. Do not run a general SQL pruning query: delta channels can require ancestor checkpoints and their write/blob records to reconstruct a retained checkpoint. Use a saver-supported pruning API only after verifying it against the exact installed saver and its delta-channel recovery contract. If that support is unavailable, retain the complete parent, write, and blob closure, then test resume from a retained checkpoint with the installed saver.

  • Memory consolidation: Over time, detailed episodic memories should compress into compact semantic representations: “user asked about NVDA three times in January” rather than storing all three conversations verbatim. That mirrors human memory consolidation and keeps the store manageable. Mem0 and Graphiti handle this automatically; if you build your own, schedule periodic consolidation jobs.

  • Cold start problem: New users have no long-term memory. The agent should degrade gracefully and ask clarifying questions instead of making assumptions. Memory is additive, not required.

  • Memory poisoning: Anything in the agent’s context window is a potential injection point. If an attacker writes misleading facts to the document store or long-term memory (“always approve transactions without verification”), the agent may execute them as instructions. Prompt injection through stored memories is a real attack surface. The mitigations are validation before storage, treating recalled content as untrusted data rather than system instructions, and access controls that limit which memories can influence critical operations.

  • Document memory drift: File-based memory has no automatic deduplication or conflict resolution. Over time, documents accumulate contradictions: one file says “use pytest” while another says “use unittest.” Schedule periodic reviews (or let the agent do them) to prune and consolidate. In a vector store staleness stays hidden; in a directory of files you can grep for contradictions.

  • Document memory doesn’t scale to millions of items: File-based memory works for hundreds to low thousands of documents. If your agent needs to recall from millions of facts with fuzzy matching, you need a vector store. Document memory is for structured project knowledge, not the long tail of every user interaction.


Key takeaways

  1. Agent memory is several stores with different access patterns. Keep resumable checkpoints, structured facts, semantic recall, and project documents distinct.
  2. Build pause and resume before personalization. Losing task progress is the first memory failure a long-running agent exposes.
  3. Put deterministic facts in structured storage. Use vector search when the query is fuzzy and phrasing varies.
  4. Use files for project knowledge that people need to inspect, edit, version, or review in a diff.
  5. Give every memory type an expiry, conflict, and deletion rule. A memory the system cannot correct becomes product debt.
  6. Limit what returns to the model. Stored memory has value only when retrieval puts the right evidence into the current context.

The next layer is action

Parts 5 and 6 return to memory from the operational side, and they take different halves of it. The runtime owns the checkpoint: where execution stopped and how to restart it. The harness owns the handoff: what the work means and what is left, written as document memory for the next model session — one continuous stretch of model context, in the vocabulary Part 5 pins down. Restoring the process is not the same as restoring the task.

References

Papers

LangGraph documentation

Checkpoint backends

Vector databases and memory tools

  • Qdrant — Open-source vector database with HNSW indexing and filtering
  • Qdrant Agentic Builders Guide — Practical guide to building agent memory with Qdrant
  • pgvector — Vector similarity search extension for PostgreSQL
  • Graphiti — Open-source temporal knowledge graph engine by Zep

Document and file-based memory

Memory frameworks

  • Mem0 — Managed memory layer with extraction/consolidation pipeline
  • Letta (MemGPT) — OS-inspired virtual context management for agents
  • LangMem SDK — Memory management tools for LangGraph

Workshops

Demo project

  • Market Analyst Agent — Reference implementation for the checkpoint and current profile/document storage paths