Reasoning data expose models to multi-step reasoning, but reasoning traces may cover only a subset of the available facts. How should pretraining balance atomic facts with reasoning traces? We study this question by constructing a setting in which every fact provides an atomic training signal, while only a subset appears in reasoning supervision, enabling us to measure generalization to transitions observed atomically but never used in training traces. For a linear transformer, we derive the weight assigned to atomic training signals that minimizes held-transition squared error and show that its leading-order optimum increases with reasoning-chain length and decreases with the fraction of facts held out from reasoning traces. These trends persist in a full transformer layer with an MLP, softmax attention, and cross-entropy loss. We further validate the role of reasoning complexity and data coverage in two natural-language settings: controlled multi-hop reasoning and mathematical pretraining. Together, our results provide a principled view of how reasoning-chain structure and data coverage should shape the allocation of training data during pretraining.
Figures & tables
Figure 1: Effect of atomic–reasoning mixing on state-tracking accuracy. Final-answer accuracy as a function of the atomic ratio in a synthetic world with 64 entities and 8 relations, each sampled as an independent random permutation. Curves and shaded regions indicate the mean and ±1 standard deviation across three training seeds. The empirically optimal atomic example ratio generally increases with reasoning length L and decreases with the held-out fact fraction q . The selected grid maximizers for L∈{3,4} at q=30% deviate from the latter trend; however, these panels exhibit near-optimal plateaus, with several tested mixture ratios achieving very similar mean OOD accuracy.
Figure 2: Multi-hop final-answer accuracy as a function of the atomic ratio in a synthetic world with 32 people and 32 relations. Columns vary hop count H and rows vary reasoning holdout fraction q . Curves and shaded regions indicate the mean and ±1 standard deviation across three training seeds. Large red circles mark OOD accuracies within five percentage points of each panel’s maximum.
FM (%)
ML (%)
CE ↓
PPL ↓
100
0
1.785
5.96
90
10
1.613
5.02
80
20
1.608
4.99
70
30
1.674
5.33
60
40
1.689
5.41
50
50
1.768
5.86
Table 1: Mixture-ratio sweep between FineMath and MathLake (Olmo-3 150M, 10B tokens). Zero-shot GSM8K test CE and PPL over the solution sequence. Best in bold.
Figure 3: GSM8K CE at step 5000, with uncertainty estimated by bootstrapping evaluation examples 3 times. (a) Varying the fraction p of the MathLake pool available for training, at three FineMath/MathLake ratios. (b) Training on the easy or hard half of MathLake versus a random 50% subset.
Pure FM steps ( K )
ML share while mixing
CE ↓
PPL ↓
0
20.0%
1.744
5.72
1k
22.2%
1.749
5.75
2k
25.0%
1.754
5.78
3k
28.6%
1.767
5.85
4k
33.3%
1.771
5.88
6k
50.0%
1.853
6.38
Table 2: Timing of reasoning data. Each run trains Olmo-3 150M for 10k steps with MathLake making up 20% of all tokens; the first K steps use pure FineMath, so the MathLake share while mixing rises with K . Zero-shot GSM8K test CE and PPL over the solution sequence at the final checkpoint. Best in bold.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Held-out atomic recall. ID and OOD final-answer accuracy and held-out atomic accuracy versus atomic example ratio, using the same setting as Figure 1 . For held-out atomic evaluation (green dashed), we sample (e0,r1) uniformly from held-out facts and r2,…,rL independently and uniformly from all relations, scoring only e1=πr1(e0) . Curves and shaded regions show the mean and ±1 standard deviation across three training seeds.
Post-training has become a primary driver of recent progress in large reasoning models, and reasoning data are often the key variable determining whether this stage succeeds. Work on post-training reasoning data has grown rapidly, yet this literature remains scattered across dataset papers, reinforcement-learning recipes, reward-model studies, benchmarks, and frontier system reports. This paper is the first primer to synthesize over 150 key public studies and system reports on post-training reasoning data. We organize the field around four questions: what data objects exist, what makes them useful, how they are constructed, and how they scale. Together, this organization provides an attribution framework for future reasoning-data releases and post-training recipes.
Yaoming Li, Guangxiang Zhao, Qilong Shi +3
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Qiyuan Tech · Tsinghua University
Reinforcement learning (RL) has become central to improving large language models (LLMs) on complex reasoning tasks, yet RL post-training is largely studied in isolation from the pretraining that precedes it. As a result, two basic questions remain open: (1) how do pretraining choices (model size, data) shape the returns to RL compute, and (2) what does RL actually do to the model? These questions are difficult to study in the standard LLM setting: pretraining corpora are vast and uncontrolled, making it hard to attribute behaviors to pretraining versus RL, and systematic compute sweeps across both stages are prohibitively expensive. To address these challenges, we use chess as a controlled testbed for studying reasoning across the full pretraining-to-post-training pipeline. We follow the standard LLM training pipeline by pretraining language models from 5M to 1B parameters on human chess games, supervised fine-tuning on synthetic reasoning traces, and running RL on chess puzzles with verifiable rewards. Using this framework, we find that the post-RL performance at given RL compute level is well-predicted from the pretraining loss, and slope of the RL reward curves improves approximately linearly with the pretraining tokens. Beyond scaling, we find that RL does not simply sharpen the SFT policy: on easy puzzles it amplifies correct moves the SFT policy already preferred, while on hard puzzles it surfaces correct moves that were nearly absent under SFT. We further test whether our findings transfer beyond chess by training a 1B language model on math-domain text, where the same predictive pattern emerges: longer-pretrained checkpoints reach higher post-RL performance and improve faster under RL. In sum, we provide a quantitative account of the pretraining-to-RL interface and a controlled testbed for studying the science of reasoning across the full pretraining-to-post-training pipeline.
Jingyan Shen, Ang Li, Salman Rahman +4
New York University · University of Illinois Urbana-Champaign · Modal Labs +2
Large Language Models (LLMs) are pretrained on massive datasets and later instruction-tuned via supervised fine-tuning (SFT) or reinforcement learning (RL). Best practices emphasize large, diverse pretraining data, whereas post-training operates differently: SFT relies on smaller, high-quality datasets, while RL benefits more from scale, with larger amounts of feedback often outweighing label quality. Yet it remains unclear why pretraining and RL require large datasets, why SFT excels on smaller ones, and what defines high-quality SFT data. In this work, we theoretically analyze transformers trained on an in-context weight prediction task for linear regression. Our analysis reveals several key findings: (i) balanced pretraining data can induce latent capabilities later activated during post-training, and (ii) SFT learns best from a small set of examples challenging for the pretrained model, while excessively large SFT datasets may dilute informative pretraining signals. In contrast, RL is most effective on large-scale data that is not overly difficult for the pretrained model. We validate these theoretical insights with experiments on large nonlinear transformer architectures.