RAG in a Trenchcoat: When Minimal Memory Is Enough for Agentic Systems, and When It Isn't
Organizations: University of Chicago · University of California, Berkeley
Abstract
Agentic memory research has accumulated extraction, supersession, reflection, and graph synthesis on top of retrieval-augmented generation. However, we found that almost none of those primitives earn their cost in our tested memory benchmarks: we show this with BRIDGE, a deliberately minimal RAG memory with chained observation notes, hybrid BM25 with cosine retrieval, and chronological rendering, which reaches parity within bootstrap CI with the prior SOTA on LongMemEval-S and BEAM, and beats the best memory baseline on EverMemBench by 5.7-6.3 points at matched reader. It runs at ~5-7K reader-prompt tokens per query, using a single LLM call for each ingestion or query. We argue that this parity is itself the result: standard benchmarks score what memory retrieves, not what memory enables when the agent acts. To test the latter, we introduce five adversarial probes (Rule Application +-, Pattern Induction Style / Procedural, Dynamic Multiturn) judged at generation time by a cross vendor verification chain, where each query can only be solved by applying a remembered rule or pattern it never mentions. Across 5 probes x 20 scenarios, no architecture wins universally: atomic fact extractors (LangMem) win rule application, while BRIDGE's block-summary memory wins pattern induction. This result holds across four domains, three graders, and three reader families. Each design fails where the other wins, for reasons a recall benchmark cannot see: atomic-fact extractors discard the form a demonstrated pattern takes, while block-summary memory retrieves the rule but the reader does not act on it, a gap that presenting the rule more prominently largely closes. What separates memory designs is not what they retrieve but what they enable an agent to do, and that is what agentic memory should be evaluated on.
Figures & tables
| Probe | In the history | The request | Passes if |
| RA + rule application | A rule said once in passing, e.g., every public endpoint must be rate-limited | A task the rule applies to, e.g., add a password-reset endpoint | it applies the rule, e.g., rate-limits the endpoint |
| RA - rule application (ban) | A ban said once in passing, e.g., eval() is banned after an incident | A task whose obvious solution is banned, e.g., evaluate math that users type in | it avoids the banned approach, e.g., no eval() |
| PI-S pattern induction (style) | Code that shares conventions, never stated, e.g., every error has a code and a message | New code of the same kind, e.g., add an endpoint that cancels an order | it follows the conventions |
| PI-P pattern induction (procedure) | The same steps in four past cases, never stated, e.g., five steps for each incident | The steps for a new case, e.g., walk me through a new incident | it follows the procedure |
| DM dynamic multi-turn | A rule said once in passing, e.g., remind me to check passport rules whenever travel comes up | A task the rule applies to, e.g., plan a trip abroad, then four follow-ups from a simulated user | it still applies the rule at the last turn |
| Probe | Mem0 | LangMem | Graphiti | Bridge | Oracle |
| RA + | |||||
| RA - | |||||
| PI-S | |||||
| PI-P | |||||
| DM |
| RA + (rule application) | PI-P (procedure induction) | |||||
| Level | Bridge | LangMem | gap | Bridge | LangMem | gap |
| 1 (original) | [17, 61] | [67, 100] | [78, 96] | [43, 66] | ||
| 2 (less signal) | [12, 60] | [33, 86] | [69, 92] | [40, 72] | ||
| 3 (+ competing) | [20, 69] | [15, 69] | [58, 86] | [36, 69] | ||
| Original probe | Hardest level | |
| Bridge , unchanged | ||
| + generic summary (same extra call) | ||
| + standing rules shown at the top | ||
| Reference: LangMem | ||
| Reference: Oracle (strong model, full history) | — |
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
| Parameter | Value | Parameter | Value |
| Chunk threshold | RRF | ||
| Prior-notes window | BM25 | ||
| Top- blocks | Embedder dim | ||
| Raw slots | Extractor / reader | / | |
| Per-channel pre-fusion | Sampling-sensitivity |
| Probe | Scenarios | Sessions | Constraints/scenario | Total cells ( 5 systems) |
| RA + | 20 | 10 (1 intro + 1 embed + 8 distractor) | 1 (rule) | 100 |
| RA - | 20 | 10 (1 intro + 1 embed + 8 distractor) | 1 (anti-pattern) | 100 |
| PI-S | 20 | 7 (1 intro + 6 demonstration) | 4 (conventions) | 400 |
| PI-P | 20 | 5 (1 intro + 4 demonstration) | 4–5 (procedural steps) | 440 |
| DM | 20 | 10 (reuses RA + corpus) | 1 (rule, 5 turns) | 100 final + 500 per-turn |
| Probe | Original | Levels 2–3 | New domains | Total |
| RA + | ||||
| RA - | — | |||
| PI-S | — | |||
| PI-P | ||||
| Total |
| Probe | Software | Clinical ops | Legal / compliance | Finance ops |
| RA + | ( ) | ( ) | ( ) | ( ) |
| RA - | ( ) | ( ) | ( ) | ( ) |
| PI-S | ( ) | ( ) | ( ) | ( ) |
| PI-P | ( ) | ( ) | ( ) | ( ) |
| Corpus | Scenarios | Sessions checked | Leaking sessions | Requests naming the rule |
| RA + , original | ||||
| RA - , original | ||||
| RA + , Level 2 | (original request) | |||
| RA + , Level 3 | (original request) | |||
| RA + , three new domains | ||||
| RA - , three new domains |
| Role | Provider | Snapshot |
| Reader ( Bridge + all baselines, canonical) | OpenAI | gpt-4.1-mini-2025-04-14 |
| Embedder ( Bridge ) | OpenAI | text-embedding-3-large (3072-d) |
| Strong-model oracle (adversarial) | OpenAI | gpt-5-2025-08-07 reasoning_effort=high |
| Generator (adversarial scenarios) | OpenAI | gpt-5-2025-08-07 reasoning_effort=minimal |
| Judge (primary, adversarial) | Google AI Studio | gemini-3.1-pro-preview thinking_budget=-1 |
| Judge (fallback 1) | Vertex AI | gemini-3.1-pro-preview (same model, different routing) |
| Graders | Verdicts | Agree | Of “applied” | Of “violated” | AC1 | |
| gemini-3.1-pro vs GPT-5 | ||||||
| gemini-3.1-pro vs GPT-4.1-mini |
| Grader | RA + : LangMem / Bridge | PI-P: Bridge / strongest other | Gaps |
| gemini-3.1-pro (ours) | / | / ( LangMem ) | , |
| GPT-5 | / | / ( LangMem ) | , |
| GPT-4.1-mini | / | / ( Mem0 ) | , |
| Probe | GPT-4.1-mini | Llama-3.3-70B | Qwen2.5-7B |
| RA + | / ( ) | / ( ) | / ( ) |
| RA - | / ( ) | / ( ) | / ( ) |
| PI-S | / ( ) | / ( ) | / ( ) |
| PI-P | / ( ) | / ( ) | / ( ) |
| Probe | System | Runs | Mean SD |
| RA + | Bridge | , , , | |
| LangMem | , , , | ||
| RA - | Bridge | , , | |
| LangMem | , , | ||
| PI-S | Bridge | , , | |
| LangMem | , , |
| System | RA + | RA - | DM | PI-S | PI-P | Mean |
| Mem0 | ||||||
| LangMem | ||||||
| Graphiti | ||||||
| Bridge | ||||||
| Oracle |
| Scenarios | Rule retrieved | Rule shown as raw text | |
| All | ( ) | ||
| Bridge passes | ( ) | ||
| Bridge fails | ( ) |
| Arm | RA + , original | RA + , hardest | PI-P | PI-S |
| Bridge , unchanged | ( ) | ( ) | ( ) | ( ) |
| + generic summary | ( ) | ( ) | — | — |
| + standing directives | ( ) | ( ) | ( ) | ( ) |
| RA + run 1 | RA + run 2 | PI-P run 1 | PI-P run 2 | |
| Standard ( ) | ( ) | ( ) | ( ) | ( ) |
| (notes only) | ( ) | ( ) | ( ) | ( ) |
| (all raw) | ( ) | ( ) | ( ) | ( ) |
| No chronological re-sort | ( ) | ( ) | ( ) | ( ) |
| BM25 only | ( ) | ( ) | ( ) | ( ) |
| Cosine only | ( ) | ( ) | ( ) | ( ) |
| Probe | Shipped schema | Pattern-targeted | Change |
| PI-S | ( ) | ( ) | |
| PI-P | ( ) | ( ) |
| Benchmark | Reader | Tokens (mean SD) | Seconds p50 / p95 | vs. full history | |
| BEAM 100K | gemini-3-flash | / | |||
| BEAM 100K | gpt-4.1-nano | / | |||
| EverMemBench | gemini-3-flash | / | |||
| EverMemBench | gpt-4.1-mini | / | |||
| LongMemEval-S | gpt-4o | / | |||
| LongMemEval-S | gemini-3-flash | / |
| System | Context (chars) | Tokens | Seconds p50 / p95 |
| Oracle (full history) | / | ||
| Bridge | / | ||
| Graphiti | / | ||
| LangMem | / | ||
| Mem0 | / |
| Benchmark | Reader | Run SD | Interval width |
| BEAM 100K | gemini-3-flash | ||
| BEAM 100K | gpt-4.1-nano | ||
| EverMemBench | gemini-3-flash | ||
| EverMemBench | gpt-4.1-mini | ||
| LongMemEval-S | gpt-4o | ||
| LongMemEval-S | gemini-3-flash |
| Extract \ Reader | gpt-4.1-mini | gpt-4o | gemini-3-flash |
| gpt-4.1-mini | |||
| gemini-3-flash |
| Benchmark | Stack | (points) | ||
| BEAM 100K | nano / nano | |||
| BEAM 100K | gemini-3 / gemini-3 | |||
| EverMemBench | mini reader | |||
| EverMemBench | gemini-3-flash reader | |||
| LongMemEval-S | mini / gpt-4o | |||
| LongMemEval-S | gemini-3 / gemini-3 |
| Primitive ablated | BEAM (10-ab) | EvB pooled | LME-S pooled |
| (no prior-notes chain) | — | ||
| (no raw-source slots) | |||
| no chronological re-order | |||
| BM25-only retrieval | |||
| cosine-only retrieval |
Memory-Augmented Agents
Memory-augmented agents use stored experience and knowledge to act coherently across long tasks, changing environments, and repeated interactions.
See all 367 papers