Instant Weight-Space Consolidation for Long-Term Memory in LLMs
Organizations: University of Maryland, College Park
Abstract
Agentic memory systems give LLMs persistent memory by extracting experience into text and retrieving it at query time, but the model itself does not evolve: however many interactions it has had, it starts each query the same. Extraction errors compound, implicit cross-session patterns that no single fragment states cannot be surfaced, and the construction cost grows with every conversation. Fine-tuning on past interactions does change the weights, but requires backpropagation at serving time and degrades on long histories. We introduce CLAM (Carried Latent Adapter Memory), which combines the strengths of both: a recurrent hypernetwork performs parametric consolidation, compiling a stream of session activations into a low-rank adapter in a single forward pass with no gradient steps and no language-model generation calls, while a complementary episodic buffer supplies verbatim detail at inference. A gated latent state carries knowledge across sessions in fixed size, so the parametric memory accumulates without growing. On LoCoMo and LongMemEval-S across three open models (2-7B parameters), CLAM leads all agentic baselines by 1-45 points and all gradient-based parametric methods by 10-45 points, while requiring zero generative LLM calls, zero gradient steps, and 3-151x less wall-clock time than extraction systems to build its memory. The two channels are complementary: the consolidated adapter adds 12-15 accuracy points when the episodic buffer misses the gold evidence, and still adds 4-5 points even with an oracle buffer.
Figures & tables
| Property | Value |
| Trajectories generated | 900K |
| Distinct personas | 23,984 |
| History chunks per trajectory: median / range | 14 / 2–48 |
| Messages per chunk: median / range | 7 / 1–25 |
| History length (tokens): median / p90 | 5,457 / 13,322 |
| Memory types (balanced) | 6 |
| LoCoMo | LongMemEval-S | |||||||||||||
| System | Single | Multi | Temp. | Open | Overall | [95% CI] | SS u | SS a | Pref. | M-sess | K-upd. | Temp. | Overall | [95% CI] |
| Base model: Gemma-2-2B | ||||||||||||||
| Mem0 | 50.6 | 45.4 | 19.6 | 29.2 | 41.8 | 46.4 | 20.0 | 20.0 | 25.9 | 34.5 | 7.1 | 25.6 | ||
| A-Mem | 32.2 | 34.4 | 15.6 | 26.0 | 28.7 | 34.5 | 56.7 | 20.0 | 11.1 | 24.1 | 25.0 | 28.9 | ||
| Generative Agents | 15.4 | 16.0 | 2.5 | 12.5 | 12.6 | 37.9 | 30.0 | 10.0 | 0.0 | 65.5 | 3.6 | 24.9 | ||
| MemoryBank | 11.9 | 12.1 | 3.7 | 14.6 | 10.4 | 51.7 | 60.0 | 23.3 | 11.1 | 10.3 | 10.7 | 28.3 | ||
| LoCoMo | LongMemEval-S | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| System | rank | G | Q | M | vs CLAM | vs consol. | G | Q | M | vs CLAM | vs consol. |
| D2L-Merge | 8 per session | 1.1 | 4.4 | 1.1 | 0.6 | 1.2 | 1.2 | ||||
| D2L-Retrieve | 8 | 16.1 | 15.0 | 18.9 | 12.1 | 11.0 | 12.7 | ||||
| Test-time LoRA | 8 | 1.1 | 21.7 | 8.9 | 1.7 | 6.4 | 2.9 | ||||
| TLM | 8 | 1.1 | 3.3 | 3.3 | 1.2 | 7.5 | 12.1 | ||||
| TMEM-once | 8 | 4.4 | 10.0 | 5.6 | 4.6 | 9.2 | 8.1 | ||||
| Building the memory (once, per conversation) | Answering (per question) | ||||||
| System | LLM decode calls | LLM tokens | grad. steps | wall (s) | LLM tokens | LLM decode calls | wall (s) |
| Mem0 | 48.8 | 0 | 200.6 | 778 | 1.00 | 5.84 | |
| A-Mem | 464.1 | 0 | 1,554.5 | 1,396 | 1.00 | 5.07 | |
| Generative Agents | 34.7 | 0 | 34.8 | 688 | 1.00 | 4.94 | |
| MemoryBank | 28.2 | 0 | 59.7 | 928 | 1.00 | 5.31 | |
| SimpleMem | 12.1 | 0 | 109.2 | 16,016 | 5.62 | 12.30 | |
| Configuration | G | Q | M |
|---|---|---|---|
| (a) Single stream vs. both streams | |||
| Consolidation only (no retrieval) | 21.9 | 22.0 | 23.8 |
| Episodic only (no adapter, ref. ) | 40.6 | 40.6 | 47.7 |
| Both ( CLAM ) | 47.7 | 51.6 | 55.3 |
| (b) Consolidation gate | |||
| Reset state (carried state discarded every session) | 16.7 | 17.2 | 24.1 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Second judge | Agree | Sign kept | Sig. flips | |
|---|---|---|---|---|
| Llama-3.1-405B (same family, 5.8 larger) | 94.7% | 0.863 | 39/42 | 0 |
| Qwen3-30B-A3B (independent family) | 94.1% | 0.847 | 34/42 | 0 |
| Qwen3-8B (independent family) | 89.7% | 0.723 | 32/42 | 0 |
| LoCoMo | LongMemEval-S | |||||||||||||
| System | Single | Multi | Temp. | Open | Overall | [95% CI] | SS u | SS a | Pref. | M-sess | K-upd. | Temp. | Overall | [95% CI] |
| Base model: Gemma-2-2B | ||||||||||||||
| Mem0 | 24.3 | 11.0 | 2.8 | 8.3 | 16.4 | 39.3 | 13.3 | 0.0 | 18.5 | 27.6 | 7.1 | 17.4 | ||
| A-Mem | 14.9 | 7.1 | 5.9 | 6.2 | 11.1 | 34.5 | 26.7 | 0.0 | 7.4 | 13.8 | 10.7 | 15.6 | ||
| Generative Agents | 5.3 | 3.5 | 1.9 | 3.1 | 4.1 | 34.5 | 16.7 | 0.0 | 0.0 | 51.7 | 3.6 | 17.9 | ||
| MemoryBank | 4.1 | 3.5 | 0.9 | 4.2 | 3.3 | 37.9 | 26.7 | 0.0 | 3.7 | 10.3 | 7.1 | 14.5 | ||
| LoCoMo | LongMemEval-S | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| System | rank | G | Q | M | vs CLAM | vs consol. | G | Q | M | vs CLAM | vs consol. |
| D2L-Merge | 8 per session | 1.1 | 1.7 | 0.0 | 0.6 | 2.3 | 3.5 | ||||
| D2L-Retrieve | 8 | 5.0 | 4.4 | 6.1 | 2.9 | 6.4 | 5.8 | ||||
| Test-time LoRA | 8 | 0.6 | 5.0 | 2.8 | 2.3 | 2.3 | 0.0 | ||||
| TLM | 8 | 0.6 | 3.9 | 2.2 | 1.7 | 3.5 | 3.5 | ||||
| TMEM-once | 8 | 2.2 | 6.7 | 2.2 | 2.9 | 5.2 | 2.9 | ||||
| System | Greedy | Sampled, mean sd |
|---|---|---|
| CLAM , Gemma-2-2B | 47.6 | 46.5 1.04 |
| CLAM , Qwen3-4B | 49.1 | 49.7 0.95 |
| CLAM , Mistral-7B | 50.6 | 50.5 0.77 |
| Buffer unit | Units / conversation | Accuracy (%) |
|---|---|---|
| Session pieces, 1,800 chars ( CLAM ) | 68 | 45.6 |
| 1,024-character chunks | 118 | 40.6 |
| 512-character chunks | 183 | 41.1 |
Persistent Memory for Language Models
Persistent memory for language models studies how models retain, update, and use information across interactions and beyond their context windows.
See all 282 papers