Organizations: Large Language Model Department, Tencent · Xi’an Jiaotong University, Xi’an, China · Fudan University, Shanghai, China · Tencent · City University of Hong Kong.
Long-context reasoning is essential for complex real-world applications, yet remains a significant challenge for Large Language Models (LLMs). Despite the rapid evolution in long-context reasoning, current research often overlooks the internal complexity of the long-context reasoning task itself. In this paper, we move beyond this holistic view and decompose long-context reasoning into a set of fundamental atomic skills, and we then automatically synthesize a suite of pseudo datasets, each explicitly targeting a specific atomic skill. Our empirical analysis confirms that proficiency in these atomic skills is strongly correlated with general long-text reasoning performance. Building on this insight, we employ reinforcement learning on these pseudo datasets to sharpen the model's atomic skills, in the hope of boosting its general long-context reasoning ability. Extensive experiments across multiple benchmarks demonstrate the effectiveness of our approach: it outperforms a strong baseline by an average margin of 7.7% (improving from 46.3% to 54.0%) across Loogle, Loong, LongBench-v2, BrowscompLong, Ruler-qa2, and MRCR.
Figures & tables
Figure 1: Decomposition of a complex task into atomic capabilities. The process necessitates Global Integration for aggregating distributed figures and Dynamic State Tracking for holding intermediate values during multi-step computation, rather than simple retrieval.
Figure 2: The Automated Dataset Construction Pipeline of the Anchor-based Reasoning (AbR) Framework.
Figure 3: Spearman Correlation Analysis. The heatmap compares the correlation of our proposed atomic capabilities against real-world long-context benchmarks.
Model
NIAH
Anti-interfere
Multi_source
Logic
Calc_reason
QwenLong-L1-32B
66.50%
33.83%
38.81%
28.50%
47.37%
Qwen2.5-32b-instruct
37.00%
22.13%
23.88%
13.00%
36.59%
DeepSeek-R1-Distill-Qwen-32B
58.25%
25.96%
32.54%
17.50%
42.11%
Qwen3-32B
23.00%
12.55%
23.28%
19.00%
19.80%
Qwen3-32B
69.50%
29.15%
37.01%
27.25%
52.13%
Qwen3-30B-A3B-thinking-2507
74.25%
41.70%
47.46%
31.50%
60.15%
Table 1: Performance comparison on atomic skills.
Model
Loogle
Longbench-v2
Loong
Browscomplong
Ruler-qa2
MRCR
Average
Gemini-3-pro
52.86%
69.38%
65.43%
88.07%
83.01%
75.30%
72.34%
Kimi-K2-Thinking
51.50%
49.30%
58.01%
58.10%
49.98%
51.77%
53.11%
DeepSeek-V3.1
55.77%
52.88%
50.55%
56.27%
42.97%
46.62%
50.84%
Qwen3-235B-A22B-thinking-2507
52.77%
49.30%
53.70%
50.76%
46.81%
44.61%
49.66%
QwenLong-L1-32B
49.32%
43.74%
44.68%
69.93%
47.70%
27.70%
47.18%
DeepSeek-R1-distill-32B
42.31%
43.94%
38.17%
64.22%
57.23%
31.94%
46.30%
Table 2: Performance comparison on Open Long Context Benchmarks. The top block shows general baselines and the bottom block illustrates the direct baselines with the same backbone. The best result in each column within its block is in bold .
Model
Loogle
Longbench-v2
Loong
Browscomplong
Ruler-qa2
MRCR
Average
Qwen2.5-14B-instruct
33.06%
36.38%
24.65%
47.30%
40.28%
31.87%
35.59%
Qwen2.5-14B-instruct+Ours
40.51%
41.15%
28.63%
69.72%
63.21%
31.73%
45.83%
Qwen2.5-32B-instruct
35.97%
39.76%
34.27%
60.65%
45.25%
33.33%
41.54%
Qwen2.5-32B-instruct+Ours
45.78%
43.94%
38.34%
60.86%
47.74%
36.33%
45.50%
Table 3: Performance of our approach applied to more backbone models. The best result in each column with its block is highlighted in bold .
Figure 4: Performance Gain over Base Model. The radar chart compares the performance improvements of our full method (red, with stars) against various ablation variants across six real-world long-context benchmarks.
Figure 5: Non-Orthogonality Analysis: Performance Drop by Module Removal. The heatmap illustrates the performance degradation across different atomic capability probes when specific training modules are ablated.
Figure 6: Performance comparison on Atomic Capability Probes. We compare the DeepSeek-R1-distill-32B base model (Grey), the model trained with 4k LoongRL (Blue), and our proposed method (Orange).
Figure 7: Performance Comparison across Context Length Intervals on LongBench-v2. The Pass@1 accuracy of baseline models (dashed lines) versus our method (solid lines) across different length buckets.