memory.cpp

Cache-aware repo memory for coding agents.

Remember more. Send less. Cache smarter.

memory.cpp is a local-first, pre-inference context compiler. It turns repository history — decisions, fixes, traces, tests, rules, and prior agent work — into freshness-proven, security-filtered, cache-stable context packs before the model ever sees the request. Fewer tokens, a larger reusable prefix, stale and harmful context blocked, and a reproducible record of exactly what was sent.

Quick start · Why · Compiler · Benchmark · CLI · Paper · Positioning

_{Real terminal output (scripts/render_demo.py re-records it). Note the two cache-hash calls for different tasks return the same blake3 prefix hash.}

The cheapest token is the one the model never has to process.

Coding agents are long-horizon: they rebuild tool traces, file summaries, test output, and repeated intent across many turns. A large prompt can be technically valid yet still be wasteful, stale, unsafe, and impossible to cache. That waste shows up as repeated prefill work, larger attention state, lower prompt-cache reuse, more memory pressure per request, and — worst of all — wrong patches when old docs and prior failed attempts leak into context.

memory.cpp treats this as a compiler problem, not a retrieval problem. It decides what belongs in the stable prefix, what belongs in the fresh suffix, what must be excluded, and what should only be available by rehydration handle.

Raw context available     84,000 tokens
Compiled context           1,420 tokens
Cacheable stable prefix    1,100 tokens   (byte-stable, blake3-hashed)
Stale / secret blocked     ✓ stale  ✓ contradictions  ✓ secrets
Estimated KV avoided       prompt-side — see honesty note below

Honesty note. memory.cpp does not compress a provider's KV cache. It reduces KV pressure by keeping unnecessary, stale, duplicated, or harmful tokens out of the prompt in the first place. All KV/latency figures are estimates unless measured against a runtime, and are reported separately from measured numbers.

# build
cargo build -p memory-cli

# teach it a couple of durable facts
memory remember "Billing exports must be deterministic" --tags invariant
memory remember "ADR-004: use server-side sessions, not JWT"

# compile a cache-stable pack and inspect everything
memory compile "fix the billing export bug" --provider anthropic --budget 1500
memory cache-manifest "fix the billing export bug" --provider anthropic
memory ir inspect "fix the billing export bug"        # typed MemoryIR blocks
memory lock "fix the billing export bug"              # reproducible lockfile + ledger
memory wow --for-acquirer                             # the whole story in one report

Install scripts: ./scripts/install.sh · ./scripts/install.ps1 · cargo install --path crates/memory-cli

repo / files / git / traces / memories
  → MemoryIR                     typed blocks: source anchor, trust, freshness,
  → optimizer passes             sensitivity, volatility, token cost, cache role
  → stable prefix + fresh suffix durable vs volatile, deterministically separated
  → provider-specific prompt     Anthropic / OpenAI / Gemini / generic
  → cache manifest               cacheable regions + blake3 stable-prefix hash
  → provenance ledger            included / excluded / proofs / hashes
  → optional lockfile            reproducible, auditable context

memory bench camc runs CAMC-Bench (Cache-Aware Memory Compilation) — a deterministic comparison of selection strategies on a bundled synthetic corpus. Every number is produced by the real compiler code paths (run it yourself):

Read it as an ablation: raw/BM25/embedding leak 9 stale memories, 9 contradictions, and 3 secrets and fail the task oracle; removing freshness proofs reintroduces stale/contradictions; removing the canonical prefix loses all cacheable tokens; removing negative memory leaks secrets. Full CAMC sends the fewest tokens, keeps a cacheable prefix, blocks every unsafe block, and preserves task success.

Synthetic corpus — it measures the mechanism, not real-world win rates. Real-repo and runtime numbers must come from live measurement. Reproduce: memory bench camc.

Seeded with real durable facts about memory.cpp and compiled for real tasks, the compiled pack is 854 tokens vs 7,961 for the naive "paste the docs" baseline (README.md + docs/architecture.md + docs/cli.md + docs/cache-aware-compiler.md) — ~89% smaller, with a byte-stable cacheable prefix that is identical across tasks. Full numbers, charts, and the reproduce command: BENCHMARKS.md.

For measured (not estimated) TTFT and prefix-cache benefit against vLLM / llama.cpp / Ollama / SGLang, run scripts/runtime_bench.py.

Full reference: docs/cli.md. API/SDK surface: docs/api.md.

Most memory systems optimize storage and retrieval. memory.cpp focuses on what happens after retrieval but before inference: compilation, cache layout, exclusion, proof, and measurement.

Full table: docs/competitive-positioning.md.

Local-first by default — a SQLite vault under .memory.cpp/. No cloud, no accounts, no telemetry, no network calls unless you explicitly enable a connector. Secrets are redacted, prompt-injection patterns are quarantined, and the write firewall screens risky memory before it is ever stored. MCP exposes read-only resources by default. See SECURITY.md and docs/safety.md.

cargo fmt --all -- --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo build -p memory-cli

CI runs on Linux, macOS, and Windows. The core compiler lives in crates/memory-core (ir, prefix, cache_manifest, freshness, negative, optimizer, tokenizer, kv_model, ledger, lockfile, minimize, rehydrate, autopsy, fuzz, poison, debt, bench).