Large Language Models (LLMs) still suffer from severe hallucinations and catastrophic forgetting during causal reasoning over massive, fragmented long contexts. Existing memory mechanisms typically treat retrieval as a static, single-step passive matching process, leading to severe semantic dilution and contextual fragmentation. To overcome these fundamental bottlenecks, we propose MemCoT, a test-time memory scaling framework that redefines the reasoning process by transforming long-context reasoning into an iterative, stateful information search. MemCoT introduces a multi-view long-term memory perception module that enables Zoom-In evidence localization and Zoom-Out contextual expansion, allowing the model to first identify where relevant evidence resides and then reconstruct the surrounding causal structure necessary for reasoning. In addition, MemCoT employs a task-conditioned dual short-term memory system composed of semantic state memory and episodic trajectory memory. This short-term memory records historical search decisions and dynamically guides query decomposition and pruning across iterations. Empirical evaluations demonstrate that MemCoT establishes a state-of-the-art performance. Empowered by MemCoT, several open- and closed-source models achieve SOTA performance on the LoCoMo benchmark and LongMemEval-S benchmark.
Figures & tables
Figure 1. The answer prediction F1 scores (%) of GPT-4o-mini on the LoCoMo benchmark. MemCoT achieves a state-of-the-art overall F1 score of 58.03%, significantly outperforming all baseline memory systems across all sub-tasks.
Figure 2. Analysis of reasoning and retrieval failures in long-context: (a) A relational diagram illustrating the dependencies between evidence and answers in complex single-hop and multi-hop reasoning scenarios. The red circle A denotes the final answer, while the blue circles ei represent the supporting evidence. (b) A Pareto chart depicting the distribution of chunk distances (ranging from 1 to 880) between the falsely retrieved chunks and the ground-truth answer chunks evaluated on the LoCoMo dataset. (c) A magnified Pareto chart detailing the chunk distances within a narrowed range of 1 to 100 on the LoCoMo dataset.
Figure 3. The overall architecture of the MemCoT framework. It illustrates the scaling test-time compute memory loop, driven by a multi-view memory perception module for long-term memory extraction and a short-term memory evolution module for iterative query qj updates in the j− th iteration. On one hand, the retrieved memory mj provides factual anchors that empower the Judge Agent to construct a more rigorous and grounded CoT for reasoning. On the other hand, the reasoning process explicitly identifies the deficiencies in the retrieved memory mj relative to the current query qj and refines the qj into the next query qj+1 to optimize the next retrieved memory mj+1 . This bidirectional synergy loop establishes the foundation of our Memory-driven Chain of Thought paradigm.
Model
Method
Single-hop
Multi-hop
Open-domain
Temporal
Overall
GPT-4o-mini ( Cao et al., 2025 )
Full Context
45.89
25.01
15.49
23.83
35.57
RAG ( Lewis et al., 2020b )
52.19
32.17
23.21
30.77
42.25
Mem0 ( Chhikara et al., 2025 )
47.65
38.72
28.64
48.93
45.10
MemoryOS ( Li et al., 2025b )
48.62
35.27
20.02
41.15
42.84
HippoRAG ( Gutiérrez et al., 2025 )
54.84
33.59
28.59
48.17
47.92
A-Mem ( Xu et al., 2025b )
44.65
27.02
12.14
45.85
39.65
Table 1. Performance ( F1 score) comparison on the LoCoMo benchmark, covering single-hop, multi-hop, open-domain, and temporal settings. Best results are highlighted in bold , and second-best results are underlined .
Model
Method
Single-Hop
Multi-Hop
Open-domain
Temporal
Overall
Qwen2.5-7B ( Qwen et al., 2025 )
Full Context
10.61
3.39
8.24
3.72
7.38
RAG ( Lewis et al., 2020b )
21.72
8.05
14.51
4.89
13.88
A-Mem ( Xu et al., 2025b )
12.73
8.67
13.86
6.28
12.1
EpMem ( Terranova et al., 2025 )
17.64
6.35
2.66
9.96
6.63
Mem0 ( Chhikara et al., 2025 )
24.96
20.31
32.74
33.16
28.75
MemoryOS ( Li et al., 2025b )
29.55
21.03
40.85
26.26
33.74
Table 2. Performance ( F1 score) comparison on the LoCoMo benchmark using the tiny model Qwen2.5-7B model, highlighting MemCoT’s consistent superiority even in smaller-scale architectures.
Model
Method
SSU
MS
SSP
TR
KU
SSA
Overall
GPT-4o-mini ( Cao et al., 2025 )
Full Context
78.6
38.3
6.7
42.1
78.2
89.3
55.0
RAG ( Lewis et al., 2020b )
88.6
47.4
70.0
63.2
70.5
91.1
67.2
Mem0 ( Chhikara et al., 2025 )
91.4
66.2
34.0
63.9
74.4
96.4
71.1
Zep ( Rasmussen et al., 2025 )
92.9
47.4
53.3
54.1
74.4
75.0
63.2
Nemori ( Nan et al., 2025 )
88.6
51.1
46.7
61.7
61.5
83.9
64.2
EMem-G ( Zhou and Han, 2025 )
87.0
73.6
32.2
74.8
94.4
87.5
77.9
Table 3. Performance (LLM-as-a-Judge score) comparison on the LongMemEval-S benchmark. Best results are highlighted in bold , and second-best results are underlined .
Figure 4. Effects of different parameters: (a) Impact of the adjacent window size ( W ) on memory retrieval Recall (%) and reasoning F1 score (%) using the Qwen2.5-7B model. (b) Impact of the maximum evolution iterations ( J ) on memory retrieval Recall (%) using the Qwen2.5-7B model. (c) Impact of Fine-grained TopK ( K ) on memory retrieval Recall (%) using the Qwen2.5-7B model.
Method
Multi-hop
Temporal
Open
Single-hop
Overall
MemCoT
40.43
60.31
42.67
64.81
58.03
w/o ψz-o
35.48
55.13
38.58
61.35
53.89
w/o ψvis
39.21
59.73
39.58
63.82
56.19
w/o ψz-o+vis
34.78
50.88
41.49
59.62
53.71
Table 4. Ablation study results on the LoCoMo benchmark using Qwen2.5-14B ( F1 scores). ψz-o and ψvis denote zoom-out and visual modules, respectively.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Method
F1 scores
Token
GPT-4o-mini ( Cao et al., 2025 )
Full Context
35.57
16,910
RAG ( Lewis et al., 2020b )
42.25
649
Mem0 ( Chhikara et al., 2025 )
45.10
1,172
MemoryOS ( Li et al., 2025b )
42.84
1,589
A-Mem ( Xu et al., 2025b )
39.65
2,520
CompassMem ( Hu et al., 2026a )
52.18
20,100
Appendix
Table 5. The analysis of the cost and F1 scores in the LoCoMo benchmark. Best results are highlighted in bold , and second-best results are underlined .
Model
Average Steps
MH
TP
OD
SH
Overall
GPT-4o-mini ( Cao et al., 2025 )
1.83
2.41
3.25
1.52
1.87
Qwen2.5-7B ( Qwen et al., 2025 )
1.90
2.69
2.55
1.76
2.03
Appendix
Table 6. The analysis of the average steps by category in the LoCoMo benchmark. The MH, TP, OD, and SH represent Multi-hop, Temporal, Open-domain, and Single-hop, respectively.
Type
Method
Single-hop
Multi-hop
Open
Temporal
Overall
Full Context
45.89
25.01
15.49
23.83
35.57
RAG ( Lewis et al., 2020b )
52.19
32.17
23.21
30.77
42.25
Non graph
Mem0 ( Chhikara et al., 2025 )
47.65
38.72
28.64
48.93
45.10
A-Mem ( Xu et al., 2025b )
44.65
27.02
12.14
45.85
39.65
MemCoT(w NaïveRag ( Lewis et al., 2020b ) )
65.38
43.85
32.99
56.10
57.73
MemoryOS ( Li et al., 2025b )
48.62
35.27
20.02
41.15
42.84
Appendix
Table 7. The analysis of different types of retrieval methods in the LoCoMo benchmark. Best results are highlighted in bold . The model is GPT-4o-mini.
Model
Method
Multi-hop
Temporal
Open-domain
Single-hop
Overall
Token
GPT-4.1-mini
Full Context
25.02
12.04
19.05
18.68
18.70
16,910
MemoryBank ( Zhong et al., 2024 )
5.00
5.94
5.16
5.72
5.46
432
MemGPT ( Packer et al., 2024 )
17.72
19.44
11.29
25.59
18.51
16,977
A-Mem ( Xu et al., 2025b )
25.06
51.01
13.22
41.02
32.58
2,520
Mem0 ( Chhikara et al., 2025 )
30.14
48.91
16.43
41.3
34.20
1,764
Zep ( Rasmussen et al., 2025 )
26.73
20.97
21.24
39.96
32.40
1,602
Appendix
Table 8. Performance on the LoCoMo benchmark with Advanced Models GPT-4.1-mini.Best results are highlighted in bold , and second-best results are underlined .
Figure 5. The specific comparison of different methods. The orange mark indicates the keywords of the truth label.
Long-context reasoning is essential for complex and long-horizon tasks, yet the performance of large language models (LLMs) degrades as context length increases. Recent approaches address this by processing input chunk by chunk while maintaining a bounded textual memory in model context. However, premature information compression can discard critical details essential for subsequent reasoning. In this paper, we introduce Commit-on-Evidence Memory (CoEM), which learns when to convert source evidence into compact memory facts. Specifically, under a fixed context-memory budget, CoEM preserves potentially useful source excerpts verbatim in a pending set, allowing subsequent context to clarify their relevance before irreversible compression. As new context arrives, a learned policy revisits each pending excerpt and decides whether to promote it to the committed memory, retain it for further consideration, or discard it. A frozen verifier ensures proposed facts are accepted only if supported by retained excerpts and current context. To further guide effective memory management, we train this policy using reinforcement learning by combining fine-grained, step-level evidence rewards with final answer rewards. Extensive experiments demonstrate that CoEM consistently improves long-context reasoning. When evaluated on 6,400 documents long-context input, CoEM outperforms the strongest memory baseline by 10.4-11.4 F1 points on Qwen3.5-9B. Code repository: https://github.com/benmagnifico/CoEM.
Jingguang Li, Yebo Wu, Zuyi Guo +7
Korea Advanced Institute of Science & Technology · University of Macau · Massachusetts Institute of Technology
Long-context reasoning remains a critical bottleneck for large language models, as recent recurrent-memory approaches face two inherent challenges: sequential chunk-wise updates can overwrite early critical evidence with later irrelevant content, and serial inter-chunk dependencies limit parallelism and cause latency to increase with context length. To address these issues, we propose PI-Mem (Parallel-Iterative Memory), a mechanism that processes all chunks in parallel and iteratively refines a shared memory over a bounded number of turns. In each turn, PI-Mem reads all chunks in parallel conditioned on the current memory, selects new or complementary evidence from each chunk, and merges the selected evidence into a compact shared memory for the next turn. To discourage redundant turns, we optimize the workflow through reinforcement learning with an auxiliary turn-efficiency reward, enabling the model to adaptively exit once sufficient evidence has been accumulated. We evaluate PI-Mem with Qwen3.5-35B-A3B and Qwen2.5-7B on the HotpotQA benchmark across context lengths up to 3.6 million tokens and find that it outperforms the recurrent-memory baseline by +6.25 and +7.81 absolute points while achieving 6.1× and 2.1× inference speedups, respectively. These results demonstrate that PI-Mem breaks the accuracy--efficiency trade-off in long-context reasoning and provides a scalable approach to complex multi-hop question answering over extremely long documents.
Dawei Liu, Haixu Song, Shuang Cheng +9
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Tsinghua University +3
Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications. Although recent LLMs support increasingly long context windows, they often fail to use relevant evidence that is already present in the input, revealing a gap between context access and effective context utilization. In this work, we propose Recursive Evidence Replay as LLM Harness for Long-Context Reasoning (RECONTEXT), a training-free inference method for improving long-context reasoning. RECONTEXT uses model-internal relevance signals to construct a query-conditioned evidence pool and replays it before final generation while preserving the full original context. This recursive selection process separates evidence organization from answer generation without training, external memory, or context pruning. We also provide a theoretical analysis based on associative memory, which characterizes the context as a memory store, the question as a retrieval cue, attention as cue-trace association, and replay as trace reactivation. Experiments on eight long-context datasets with 128K context length show that RECONTEXT consistently improves evidence utilization across Qwen3-4B, Qwen3-8B, and Llama3-8B, achieving the best average rank on all three backbones. Code is available at https://github.com/Yanjun-Zhao/ReContext.