Reinforcement learning (RL) is an effective way to strengthen the long-context reasoning of large language models (LLMs), but it needs training data that are long, diverse, challenging, and verifiable. Human annotation of such data is expensive, naturally occurring data are too scarce to sustain RL, and existing synthesis methods depend on external corpora or stronger teacher models, which become harder to obtain as the trained models get stronger. Recently, RL from zero data, in which a model learns from tasks it proposes itself, has achieved excellent performance, but existing methods target short-context tasks. Therefore, we propose LongZero, a method that lets an LLM improve its own long-context reasoning without any external data. Instead of writing questions and answers over existing documents, LongZero first has the model synthesize the question and answer with its supporting evidence in a compact structured form, and only then separately expands the evidence into a long multi-document context. LongZero combines these self-written multi-hop reasoning tasks with self-written multi-round retrieval tasks and trains the model with GRPO. We evaluate LongZero on two LLMs across five long-context reasoning benchmarks and compare it with RL on data synthesized from external corpora. Experimental results show that LongZero can generalize to real tasks, and outperforms both baselines, suggesting a viable and promising path to improving long-context reasoning of LLMs.
Figures & tables
Figure 1: Core idea of LongZero . (a) Existing methods build long-context RL data from external corpora. (b) LongZero lets the policy write the question, evidence, and answer, expand them into long-context data, and learn with RL from answering its own questions.
Figure 2: Overview of LongZero data generation, with real data synthesized by Qwen3-4B-Thinking-2507. Left: for reasoning data, the policy writes a world specification, plans which document states each fact, and expands the plan into long documents. Right: retrieval data are long conversations assembled from the policy’s own responses.
Qwen3-4B-T
Qwen3-8B
Synthesis per 2,560 unstable reasoning questions
Worlds attempted
~3.8K
~5.4K
Worlds accepted
~3.6K
~5.1K
Questions filtered
~14.4K
~19.5K
always correct
69.3%
68.3%
unstable (kept)
17.7%
13.1%
Table 1: Data synthesized by each model for itself. Upper: synthesis needed for the 2,560 unstable reasoning questions used, estimated from each model’s rates. Lower: training (validation) data; token counts use each model’s tokenizer. Qwen3-4B-T: Qwen3-4B-Thinking-2507.
7 Method
LB1
LBv2
FRAMES
AA-LCR
LongReason
Avg
Qwen3-4B-Thinking-2507
Base model
65.43
40.56
51.27
36.50
79.66
54.68
Enhanced HotpotQA (from LoongRL)
65.94
43.04
51.46
41.50
80.92
56.57
Teacher-synthesized QA
66.12
41.85
52.97
38.50
80.48
55.99
LongZero (Ours)
67.22
43.34
55.76
39.50
81.30
57.42
Qwen3-8B (Thinking)
Table 2: Results on five long-context reasoning benchmarks (mean of two runs; LongReason: 32K split; Avg: unweighted mean). Best results for each base model are in bold and second-best results are underlined . All methods share the RL configuration and checkpoint step.
Figure 3: Training dynamics of LongZero up to the reported checkpoints. Validation scores are measured every 6 steps on held-out reasoning and retrieval data (overall: weighted by set size). The training reward mixes both data types (line: 5-step average; band: range over the same window).
7 Training data
LB1
LBv2
FRAMES
AA-LCR
LongReason
Avg
Base model
65.43
40.56
51.27
36.50
79.66
54.68
Reasoning data only
67.04
42.64
54.61
34.00
78.59
55.38
Retrieval data only
64.51
41.65
52.18
33.00
77.90
53.85
Reasoning and retrieval data, 1:1
67.29
41.35
53.64
36.50
80.98
55.95
Reasoning and retrieval data, 2:1 (Ours)
67.22
43.34
55.76
39.50
81.30
57.42
Table 3: Ablation on the training data of Qwen3-4B-Thinking-2507: reasoning data only, retrieval data only, and mixtures of the two in a 1:1 and a 2:1 ratio. All runs share the RL configuration. Best results are in bold and second-best results are underlined .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Decoding
Max tokens
Specification
thinking, T=0.7 / 0.6
28,672
Plan
T=0.8 (non-thinking † )
16,384
Expansion
T=0.8 (non-thinking † )
5,120
Difficulty filter
thinking, T=0.7
8,192
Appendix
Table 4: Generation settings for reasoning data, all with top- p 0.95. T=0.7 applies to Qwen3-4B-Thinking-2507, which always thinks, and T=0.6 to Qwen3-8B with thinking. † Qwen3-8B only.
Hyperparameter
Value
Algorithm
GRPO (verl)
Batch per step
64 prompts × 8
Training steps
72 (4B), 108 (8B)
PPO update
16 prompts, 1 epoch
Optimizer
AdamW (0.9, 0.999)
Learning rate
10−6 , constant
Appendix
Table 5: RL hyperparameters shared by all methods.
5 Method
HQA
2WQA
MSQ
Avg
Base model
67.04
78.46
50.79
65.43
Enhanced HotpotQA
66.59
76.41
54.82
65.94
Teacher-synthesized QA
67.73
77.97
52.68
66.12
LongZero (Ours)
67.17
79.62
54.86
67.22
Appendix
Table 6: LB1 F1 of Qwen3-4B-Thinking-2507 on each of its three datasets. Avg is the LB1 score in Table 2 .
5 Method
HQA
2WQA
MSQ
Avg
Base model
66.58
76.83
47.48
63.63
Enhanced HotpotQA
67.91
75.43
51.60
64.98
Teacher-synthesized QA
66.99
76.12
49.08
64.07
LongZero (Ours)
66.90
78.20
51.38
65.49
Appendix
Table 7: LB1 F1 of Qwen3-8B on each of its three datasets, which contain 200 questions each. Avg is the LB1 score in Table 2 .
7 \Block 2-1 Method
Difficulty
Length
\Block 2-1 Overall
Easy
Hard
Short
Medium
Long
Base model
48.44
35.69
47.78
37.44
34.72
40.56
\Block [l]Enhanced
HotpotQA
49.74
38.91
51.67
38.37
37.96
43.04
\Block [l]Teacher-
synthesized QA
47.66
38.26
51.94
35.12
38.43
41.85
Appendix
Table 8: LBv2 accuracy of Qwen3-4B-Thinking-2507 by difficulty and by input length, using the labels provided with LBv2. Overall is the LBv2 score in Table 2 .
7 \Block 2-1 Method
Difficulty
Length
\Block 2-1 Overall
Easy
Hard
Short
Medium
Long
Base model
40.89
26.69
45.00
25.35
24.07
32.11
\Block [l]Enhanced
HotpotQA
37.24
27.65
44.17
23.49
25.46
31.31
\Block [l]Teacher-
synthesized QA
38.54
29.90
45.83
27.21
24.07
33.20
Appendix
Table 9: LBv2 accuracy of Qwen3-8B by difficulty and by input length, using the labels provided with LBv2. Overall is the LBv2 score in Table 2 .
5 Training data
HQA
2WQA
MSQ
Avg
Base model
67.04
78.46
50.79
65.43
Reasoning data only
68.51
78.45
54.15
67.04
Retrieval data only
64.57
78.48
50.48
64.51
\Block [l]Reasoning and
retrieval data, 1:1
67.33
79.58
54.96
67.29
\Block [l]Reasoning and retrieval
Appendix
Table 10: LB1 F1 of the ablation variants of Qwen3-4B-Thinking-2507 on each dataset. Avg is the LB1 score in Table 3 .
7 \Block 2-1 Training
data
Difficulty
Length
\Block 2-1 Overall
Easy
Hard
Short
Medium
Long
\Block [l]Base
model
48.44
35.69
47.78
37.44
34.72
40.56
\Block [l]Reasoning
data only
51.04
37.46
48.33
39.77
38.89
42.64
Appendix
Table 11: LBv2 accuracy of the ablation variants of Qwen3-4B-Thinking-2507 by difficulty and by input length. Overall is the LBv2 score in Table 3 .
Long-context reasoning is essential for complex real-world applications, yet remains a significant challenge for Large Language Models (LLMs). Despite the rapid evolution in long-context reasoning, current research often overlooks the internal complexity of the long-context reasoning task itself. In this paper, we move beyond this holistic view and decompose long-context reasoning into a set of fundamental atomic skills, and we then automatically synthesize a suite of pseudo datasets, each explicitly targeting a specific atomic skill. Our empirical analysis confirms that proficiency in these atomic skills is strongly correlated with general long-text reasoning performance. Building on this insight, we employ reinforcement learning on these pseudo datasets to sharpen the model's atomic skills, in the hope of boosting its general long-context reasoning ability. Extensive experiments across multiple benchmarks demonstrate the effectiveness of our approach: it outperforms a strong baseline by an average margin of 7.7% (improving from 46.3% to 54.0%) across Loogle, Loong, LongBench-v2, BrowscompLong, Ruler-qa2, and MRCR.
Yanling Xiao, Huaibing Xie, Guoliang Zhao +8
Large Language Model Department, Tencent · Xi’an Jiaotong University, Xi’an, China · Fudan University, Shanghai, China +2
Long-context reasoning is an essential capability for large language models, particularly when they are deployed as autonomous agents that must reason over lengthy trajectories. Reinforcement learning (RL) has recently emerged as a dominant paradigm for improving this ability, yet existing work largely focuses on reward engineering while diverse training data remains scarce. We revisit this problem from a data-centric perspective and show that a simple yet effective data recipe alone, paired with a minimal outcome-based GRPO setup, suffices to substantially improve long-context reasoning. Our recipe targets three complementary task families -- retrieval, multi-evidence synthesis, and reasoning -- for which we construct and curate eight datasets totaling ~14K examples. Experiments on three models (Qwen3-4B/8B/30B-A3B) yield average gains of +7.2/+3.2/+6.4 points across seven long-context benchmarks, surpassing prior RL training sets. We further demonstrate that these gains transfer to agentic tasks, where continuing RL training on an agent-tuned model with our data recipe improves GAIA by +4.8 and BrowseComp by +7.0 points. We will release our datasets to facilitate future research.
Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications. Although recent LLMs support increasingly long context windows, they often fail to use relevant evidence that is already present in the input, revealing a gap between context access and effective context utilization. In this work, we propose Recursive Evidence Replay as LLM Harness for Long-Context Reasoning (RECONTEXT), a training-free inference method for improving long-context reasoning. RECONTEXT uses model-internal relevance signals to construct a query-conditioned evidence pool and replays it before final generation while preserving the full original context. This recursive selection process separates evidence organization from answer generation without training, external memory, or context pruning. We also provide a theoretical analysis based on associative memory, which characterizes the context as a memory store, the question as a retrieval cue, attention as cue-trace association, and replay as trace reactivation. Experiments on eight long-context datasets with 128K context length show that RECONTEXT consistently improves evidence utilization across Qwen3-4B, Qwen3-8B, and Llama3-8B, achieving the best average rank on all three backbones. Code is available at https://github.com/Yanjun-Zhao/ReContext.