MIGRO: A Decoupled and Portable Long-Term Memory System for Language Models
Organizations: Fudan University · Shanghai Academy of AI for Science (SAIS) · Yunnan University · Shanghai AI Laboratory ·
Abstract
Language models evolve, but the observations, revisions, and context accumulated through user interactions should remain usable. Each new model must be able to retrieve past evidence, track revisions, and add new observations. We present MIGRO, a decoupled long-term memory system that treats experience as a persistent state shared across successive language models. MIGRO follows a simple principle: preserve historical evidence exactly, organize it approximately, and decide what to retrieve only when the question is known. We call this asymmetric fidelity with deferred selectivity. A shared semantic index supports retrieval, while exact records retain observations and their historical relationships. Because models differ in vocabulary and tokenization, host-specific interfaces convert selected evidence into inputs the receiving model can use. Reversible encoding reduces archival storage while enabling exact recovery. The shared state and its access interfaces let successive hosts read and extend the accumulated experience. On matched LoCoMo states, MIGRO scores 7.3% higher and uses 99.3% less private memory than the strongest quality baseline. After handoff, continued writing retains 99.9% of full-history rebuilding quality by extending the inherited state. On million-token BEAM histories, MIGRO uses 57.5% less private storage on average than the most compact baseline. MIGRO combines high answer quality, continued memory growth across hosts, and low storage cost. Code is available at https://github.com/zOUMINGXI/MIGRO.
Figures & tables
| Host (build/read) | MIGRO | PlugMem | Mem0 | TiMem | A-MEM | LSTMem | TransMem | Doc-to-LoRA |
|---|---|---|---|---|---|---|---|---|
| Qwen3-4B-Instruct-2507 | 54.13 | 41.89 | 52.12 | 49.13 | 36.20 | 10.68 | 45.85 | 14.96 |
| Qwen3-8B | 55.68 | 34.86 | 54.88 | 46.19 | 44.70 | 19.70 | 46.39 | 12.77 |
| Qwen3-14B | 60.32 | 40.10 | 54.36 | 50.25 | 40.88 | 16.58 | 49.83 | 12.30 |
| Qwen3-32B | 61.83 | 44.66 | 57.16 | 49.24 | 45.38 | 14.35 | 48.03 | 13.18 |
| State Transfer | Full-History Rebuilding | |||||||
| Stage / path | MIGRO | PlugMem | Mem0 | TiMem | A-MEM | LSTMem | TransMem | Doc-to-LoRA |
| LoCoMo Qwen chain: 4B training/source stage, then 8B, 14B, and 32B | ||||||||
| 4B 8B | 55.52 | 44.41 | 53.18 | 46.19 | 44.70 | 19.70 | 46.39 | 12.77 |
| 4B 8B 14B | 60.37 | 46.10 | 57.07 | 50.25 | 40.88 | 16.58 | 49.83 | 12.30 |
| 4B 8B 14B 32B | 62.33 | 49.15 | 57.17 | 49.24 | 45.38 | 14.35 | 48.03 | 13.18 |
| Heterogeneous chain: Qwen 4B source, then Mistral 7B and Phi | ||||||||
| Method | LoCoMo (0–100) | 95% CI | Native memory (MB, ten states) |
| PlugMem | 41.89 | [40.01, 43.39] | 92.078 |
| Mem0 | 50.45 | [48.81, 52.18] | 160.671 |
| TiMem | 47.22 | [45.70, 48.56] | 10.851 |
| A-MEM | 35.29 | [34.46, 36.33] | 23.340 |
| LSTMem | 40.64 | [39.11, 42.10] | 1.871 |
| TransMem | 45.96 | [44.80, 47.26] | 1.200 |
| Method | State at 1M MB/history | 95% CI (MB) | 500K–1M bytes/token | 95% CI |
| PlugMem | 82.871 | [79.781, 85.277] | 79.608 | [76.162, 84.012] |
| Mem0 | 125.499 | [119.613, 131.087] | 123.093 | [115.187, 130.500] |
| TiMem | 6.166 | [6.119, 6.198] | 6.178 | [6.114, 6.225] |
| A-MEM | 23.191 | [22.914, 23.496] | 23.639 | [23.188, 24.073] |
| LSTMem | 4.371 | [4.276, 4.437] | 4.370 | [4.233, 4.453] |
| TransMem | 7.092 | [7.018, 7.154] | 7.778 | [7.709, 7.854] |
| Independent million-token memories | ||||
|---|---|---|---|---|
| Method | 100 | 1,000 | 10,000 | 100,000 |
| PlugMem | 84.237 | 83.008 | 82.885 | 82.873 |
| Mem0 | 127.018 | 125.651 | 125.514 | 125.500 |
| TiMem | 7.532 | 6.303 | 6.180 | 6.167 |
| A-MEM | 24.557 | 23.328 | 23.205 | 23.193 |
| LSTMem | 4.775 | 4.411 | 4.375 | 4.371 |
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Work | Evidence / organization | Host boundary / growth | Preservation / cost object |
| Mem++ [ 53 ] | Whole dated documents; lexical–semantic retrieval. | Document consumption; append later versions. | Original documents; generative write calls. |
| MemOS [ 11 ] | Typed MemCubes with provenance and versions. | MemCube migration, composition and lifecycle evolution. | Content/metadata management; heterogeneous memory resources. |
| Frozen-memory extraction [ 29 ] | External learned table and common addressing. | Frozen-source reuse with compatible or adapted target readers. | Reused learned artifact; reader adaptation and QA. |
| Model-upgrade study [ 14 ] | Raw histories, retrieval chunks, notes and fixed-schema graphs. | Direction-specific writer/reader swaps and source-assisted repair. | Construction/access diagnostics; QA continuity and repair. |
| MemoryLACE [ 48 ] | Atomic text and provenance; sparse lifecycle relations. | Evidence access across QA backbones; merge, supersession and contradiction links. | Relation-aware evidence units; end-to-end runtime. |
| VikingMem [ 55 ] | Event/entity base with topic-wise timelines. | Application-level memory base; event-driven state evolution. | Temporal abstraction and compression; recall and latency. |
| Symbol | Meaning |
|---|---|
| Logical persistent state after accepted observations. | |
| Observation content and its available provenance. | |
| Writing host, reading host, and frozen host parameters. | |
| Anchors, address innovations, exact content, provenance, reference links, and retained history. | |
| Canonicalization, shared source tokenizer, exact ordered source IDs, and public source embedding table. | |
| Public semantic address, shared coordinates, center, and basis. |
| Role | Checkpoint family / model card | Planned comparison |
| Qwen quality anchor | Qwen3-4B-Instruct-2507 | Main quality/storage host and the 4B starting point for same-host quality scaling. |
| Qwen dense capacity | Qwen3-4B-Instruct-2507; Qwen3-8B, Qwen3-14B, Qwen3-32B [ 60 ] | Independent same-host quality scaling on four benchmarks; LoCoMo-only 4 8 14 32 migration chain. |
| Mistral reference host | Mistral-7B-Instruct-v0.3 | LoCoMo heterogeneous transfer and target-native rebuilding. |
| Phi reference host | Phi-3-mini-4k-instruct (3.8B) | LoCoMo heterogeneous transfer and target-native rebuilding; execution mode follows Section 5.4 . |
| Ministral scale-up receiver | Ministral-3-14B-Instruct-2512-BF16 [ 61 ] | First receiving stage after Qwen3-4B-Instruct-2507. |
| Gemma scale-up receiver | gemma-4-26B-A4B-it [ 62 ] | Second receiving stage after Ministral. MoE with 25.2B total and 3.8B active parameters. |
| Stage | Host event | Retained evidence | Consequence for later access |
| 1 | A observes | Budget 17; explicitly version 1; user/source identity. | Bind the exact value and provenance. |
| 2 | Handoff to B; B observes | Budget 18; version 2. Version 1 remains. | Shared addresses may relate the statements; content is not merged. |
| 3 | Handoff to C; C observes | Budget 19; version 3. Both earlier records remain. | C adds a record in the same schema. |
| 4 | Return to A; query | Latest and version-1 evidence remain available. | The question determines which version must support the answer. |
| Benchmark | Main measurement | Complementary stress |
| LoCoMo [ 1 ] | One all-question composite over 1,986 questions, defined in Equation 16 . | Speaker attribution, temporal reasoning, multi-hop recall, abstention. |
| LongMemEval-S [ 2 ] | Auxiliary-judge answer accuracy over 500 questions. | Cross-session memory, temporal questions, updates, missing information. |
| MemOps [ 22 ] | Longitudinal answer accuracy over 2,006 questions; operation/state diagnostics separately. | Memory lifecycle operations and evidence use across an evolving trajectory. |
| MemoryAgentBench [ 23 ] | Macro-average across four memory competencies. | Incremental interaction, retrieval, test-time learning, long-range understanding, and handling revised knowledge. |
| BEAM-1M [ 24 ] | Native-category aggregation over 700 questions from 35 histories, including event ordering. | Long-distance evidence and temporal consistency in million-token histories. |
| Benchmark | Operation / capability | Questions | Score | 95% cluster CI |
| MemOps | Remember | 408 | 73.53 | [70.10, 76.96] |
| MemOps | Forget | 400 | 72.75 | [68.50, 76.75] |
| MemOps | Reflect | 480 | 64.17 | [60.41, 67.92] |
| MemOps | Update | 308 | 62.99 | [57.79, 68.18] |
| MemOps | TrajectoryOps | 410 | 57.80 | [53.17, 62.20] |
| BEAM-1M | Abstention | 70 | 55.36 | [46.41, 64.64] |
| Component | Reference specification | Scope / consequence |
| Semantic address | ; innovation rank 64; reuse threshold 0.48. | Shared, host-independent organization. |
| Saved address precision | Roots: 4 bits/coordinate, FP16 scale per 64-coordinate group. Innovations: int8 with FP16 scale. | Saved precision is preserved by archival coding. |
| Local writer | At most 4 preceding chronological records, filtered to the same session. Zero-based layer . | One current-host local forward per batch; inputs must fit context. |
| Candidate/reranker | Up to 64 candidates; frozen MiniLM cross-encoder, 384-token pair cap, batch 32. | Reference evidence-reader path; candidate policy may vary by declared reader. |
| Seed/source budgets | ; source tokens including separators. | Seeds differ from delivered record count; target-context fitting follows. |
| Complementarity | Redundancy coefficient 0.35. | Greedy relevance/diversity selection, not global optimality. |
| 1 | Initialize empty or restore a supplied snapshot; initialize runtime for . |
|---|---|
| 2 | for each event do |
| 3 | if then |
| 4 | Form ; encode the common semantic address. |
| 5 | Select at most the configured chronological past window, retaining same-session records. |
| 6 | Require the local input to fit the host context; forward current host once over past + . |
| 7 | Pool local spans; infer predecessor if the allowed window is nonempty. |
| Property | Contract and boundary | Status |
| Saved-state recovery | Restore exact source IDs, saved coordinates, provenance and history under compatible decoding dependencies. | Conditional format/coding property; verify decoded candidates. |
| Historical retention | Appending an accepted observation retains earlier logical fields; archive bytes may be repacked. | State/update contract; not crash-safe transactional durability. |
| Read non-mutation | Answering changes temporary caches, not the retained experience. | Runtime contract. |
| Bounded host access | Budgeted evidence for queries; bounded local past for writes. Full restoration/index construction may still scan history. | Access contract; not constant-time migration. |
| Cross-host usefulness | Receiving hosts answer accurately and can use revisions and provenance. | Empirical objective, not implied by exact recovery. |
| Storage advantage | Lower complete bytes at a useful quality/transfer operating point. | Measured quality–storage evidence in Section 5.5 ; broader transfer comparisons pending. |
Persistent Memory for Language Models
Persistent memory for language models studies how models retain, update, and use information across interactions and beyond their context windows.
See all 282 papers