Long-context reasoning faces two complementary bottlenecks: retaining evidence across long inputs and sustaining computation across many reasoning steps. Existing approaches largely address them separately, with external memory extending access to distant evidence and latent reasoning compressing multi-step computation. We introduce LatentHarness, which unifies memory access and latent reasoning as sequential latent action selection. At each internal step, the model chooses THINK for further computation, RECALL from a fast-weight memory of input evidence and intermediate reasoning states, or EXIT to emit the next token. We train this policy with counterfactual policy distillation, which branches every action for one step and scores its effect on the emitted token. These gains teach the policy when memory is more useful than further reasoning, while gradients through counterfactual recall teach which intermediate states should be retained in memory for future use. Across six general and long-context reasoning benchmarks, LatentHarness at 1.4B improves on the strongest baselines by 2.8% and 10.0% relative, respectively, and runs 5.9x faster than the strongest long-context baseline.
Figures & tables
Figure 1: Comparison of Ouro [ 93 ] , a looped reasoner that recurs over one weight-tied block, RLTT [ 69 ] , the same loop trained with an outcome reward, and LatentHarness , which interleaves latent reasoning and memory, with Think writing the fast-weight memory that Recall reads, and learns each latent action from its action gain G(a∣s) of Equation 7 .
Figure 2: Illustration of counterfactual policy distillation (CPD) in LatentHarness . The G latent trajectories sampled from one input form the latent trajectory group, and their rewards determine the trajectory advantage A^(τ) . The one-step branches at each latent state form the latent action group, and scoring their decodes pa against ps yields the action gain distilled into the latent action policy πϕ .
General reasoning
Long-context reasoning
Method
GSM8K
MATH
GPQA †
Avg.
FLOPs
Lat.
2WQA †
MSQ †
HQA
Avg.
FLOPs
Lat.
Ouro-1.4B-Thinking
Ouro
74.6
42.8
27.3
48.2
100%
2.6
41.2
19.7
47.5
36.1
100%
5.2
Ouro (GRPO)
78.1
46.3
29.4
51.3
106%
2.8
44.8
22.1
50.6
39.2
108%
5.4
Token space
+ Search-R1
77.4
45.6
28.3
50.4
133%
2.7
50.4
27.1
54.9
44.1
339%
16.2
Table 1: Quantitative results of LatentHarness with Ouro-1.4B-Thinking and Ouro-2.6B-Thinking. MATH, GPQA, 2WQA, MSQ, and HQA denote MATH500, GPQA-Diamond, 2WikiMultihopQA, MuSiQue, and HotpotQA, † marks an out-of-distribution suite, and Lat. is wall-clock seconds per answer. Bold , underline , and green mark the best, the second best, and the margin over the second best. Ouro is the released backbone at full depth.
Variant
Gen.
Long
FLOPs
LatentHarness
55.8
49.7
55%
Action credit
GRPO only
52.3 ( − 3.5)
44.3 ( − 5.4)
71%
GiGPO groups
52.9 ( − 2.9)
45.1 ( − 4.6)
68%
VinePPO branches
54.9 ( − 0.9)
48.6 ( − 1.1)
58%
Action space
Table 2: Ablations of LatentHarness on action credit, action space, write gate, and action cost. Gen. and Long denote family averages, FLOPs denotes the two-family average relative to full depth, and parentheses mark the change from ours. Training compute is 1.1 × , 1.2 × , and 3.6 × GRPO only for ours, GiGPO groups, and VinePPO.
Figure 3: All-Think settling, latent actions, and long-context F1 of LatentHarness , RLTT, and Memory-R2 on Ouro-1.4B-Thinking. The left, middle, and right panels show positions by All-Think settling step, latent steps per position by settling step, and F1 by input length, respectively.
Figure 4: Latent-step dynamics and write gates of LatentHarness on the Ouro-1.4B-Thinking long-context suites. From left to right, the panels show the pending read at latent states by executed action, latent steps per position by input length, the mean write gate by Think step under three gate objectives, and evidence retention by generated tokens.
LatentHarness
Think and Exit
Ouro exit gate
Recall sources (%)
Suite
Think
Recall
Score
Think
Loops
Derived
Hit
Chance
General reasoning
GSM8K
2.10
0.03
83.2
2.14
2.70
39.2
New A
New A
MATH500
2.55
0.05
51.6
2.57
3.30
35.6
New A
New A
GPQA
2.40
0.07
32.6
2.43
3.20
33.2
New A
New A
Long-context reasoning
Table 3: LatentHarness task scores, latent steps per position, and recall sources on Ouro-1.4B-Thinking, grouped by suite, hop count, and input length. Think and Exit reports the Think steps of the ablation, while Ouro exit gate reports the loops of the released backbone under its own exit. Derived is the share of recalls that read a Think -written state, Hit is the share of evidence recalls that reach a supporting passage, and Chance is the share of prompt tokens contained in supporting passages. Score is exact match on the general suites and F1 otherwise. Hop and length rows pool over the other axis, and New A marks suites without supporting-fact annotation.
Figure 5: LatentHarness recall sources and latent steps on the Ouro-1.4B-Thinking long-context suites. The left and middle panels show recall hit rate, derived share, and chance rate by input length and MuSiQue hop count. The right panel shows latent steps per position by hop count.
Long-context reasoning is essential for complex and long-horizon tasks, yet the performance of large language models (LLMs) degrades as context length increases. Recent approaches address this by processing input chunk by chunk while maintaining a bounded textual memory in model context. However, premature information compression can discard critical details essential for subsequent reasoning. In this paper, we introduce Commit-on-Evidence Memory (CoEM), which learns when to convert source evidence into compact memory facts. Specifically, under a fixed context-memory budget, CoEM preserves potentially useful source excerpts verbatim in a pending set, allowing subsequent context to clarify their relevance before irreversible compression. As new context arrives, a learned policy revisits each pending excerpt and decides whether to promote it to the committed memory, retain it for further consideration, or discard it. A frozen verifier ensures proposed facts are accepted only if supported by retained excerpts and current context. To further guide effective memory management, we train this policy using reinforcement learning by combining fine-grained, step-level evidence rewards with final answer rewards. Extensive experiments demonstrate that CoEM consistently improves long-context reasoning. When evaluated on 6,400 documents long-context input, CoEM outperforms the strongest memory baseline by 10.4-11.4 F1 points on Qwen3.5-9B. Code repository: https://github.com/benmagnifico/CoEM.
Jingguang Li, Yebo Wu, Zuyi Guo +7
Korea Advanced Institute of Science & Technology · University of Macau · Massachusetts Institute of Technology
Long-context reasoning requires accurately identifying relevant information in extensive, noisy input contexts. In this work, we propose PERK (Parameter Efficient Reasoning over Knowledge), a scalable approach for learning to encode long contexts using gradient updates at test time. Specifically, PERK employs two nested optimization loops in a meta-training phase. The inner loop rapidly encodes contexts into a low-rank adapter (LoRA) that serves as a parameter-efficient memory module for the base model. Concurrently, the outer loop learns to use the updated adapter to accurately recall and reason over relevant information from the encoded long context. Our evaluations on several long-context reasoning tasks show that PERK significantly outperforms the standard long-context finetuning, achieving average absolute performance gains of up to 20% for Qwen-2.5 (0.5B & 7B) on synthetic and real-world long-context reasoning. PERK also maintains its advantages across model scales and families. Compared to specialized long-context LLMs, PERK matches or surpasses their performance. Finally, our analyses show PERK is more robust to reasoning complexity, length extrapolation, and the positions of relevant information in contexts. https://perk-long-context.web.app
Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications. Although recent LLMs support increasingly long context windows, they often fail to use relevant evidence that is already present in the input, revealing a gap between context access and effective context utilization. In this work, we propose Recursive Evidence Replay as LLM Harness for Long-Context Reasoning (RECONTEXT), a training-free inference method for improving long-context reasoning. RECONTEXT uses model-internal relevance signals to construct a query-conditioned evidence pool and replays it before final generation while preserving the full original context. This recursive selection process separates evidence organization from answer generation without training, external memory, or context pruning. We also provide a theoretical analysis based on associative memory, which characterizes the context as a memory store, the question as a retrieval cue, attention as cue-trace association, and replay as trace reactivation. Experiments on eight long-context datasets with 128K context length show that RECONTEXT consistently improves evidence utilization across Qwen3-4B, Qwen3-8B, and Llama3-8B, achieving the best average rank on all three backbones. Code is available at https://github.com/Yanjun-Zhao/ReContext.