Beyond Correctness: Compute-Responsive Data Selection for Reasoning Model Post-Training

Liang He; Xiangyu Li; Yuxin Wang; Hanbin Zhang; Mingyu Cao; Zhaomiao Guo; Xilu Wang

cs.AIOct 14, 2026

Beyond Correctness: Compute-Responsive Data Selection for Reasoning Model Post-Training

Authors: Liang He, Xiangyu Li, Yuxin Wang, Hanbin Zhang, Mingyu Cao, Zhaomiao Guo, Xilu Wang

Abstract

Reasoning language models increasingly rely on test-time compute scaling, yet most existing work studies how to allocate inference computation while treating the model itself as fixed. We ask a complementary question: can reasoning models be post-trained to make better use of a given amount of test-time compute, and which examples are most useful for this goal? Our key observation is that reasoning problems differ substantially in their response to additional computation. Some are already solved under limited compute, others remain unresolved even after further reasoning, while a distinct subset changes from complete initial failure to success when additional reasoning is provided. We call these compute-responsive (CR) examples and introduce Compute-Responsive Data Selection (CRDS), a post-training data-selection principle that prioritizes examples with empirically demonstrated marginal utility from additional reasoning. On Qwen3-8B, SFT on CR examples recovers 45.3% of held-out low-compute failures, compared with 40.6% when training on already-solved examples. The advantage extends beyond the selected CR split: SFT-CR reaches 51.8% overall accuracy on an independently sampled test distribution, with the gain concentrated on compute-responsive problems. CR training also shifts the compute--accuracy frontier across inference budgets from 512 to 4,096 tokens, outperforming matched non-CR training throughout and reaching 48.1% recovery at the largest budget. The same pattern is supported across multiple data sources, remains visible under on-policy distillation, and extends to Llama-3.1-8B-Instruct when CR sets are reconstructed separately for that model. Better test-time compute utilization can therefore be learned through post-training, with recoverable responses to additional reasoning providing a particularly informative signal for selecting training examples.

Figures & tables

Figure 1: Heterogeneous response to additional test-time computation. (a) Increasing the generation budget improves base-model accuracy at lower budgets, but beyond a certain budget, additional computation no longer translates into higher accuracy. (b) Compute-responsive examples occur at substantially different rates across reasoning datasets.

Figure 2: Compute-responsive data selection. Starting from complete initial failures, additional reasoning is used to identify problems for which at least one follow-up trajectory succeeds. The resulting compute-responsive examples are selected for post-training; no collection-time trajectory or verifier is required at inference.

	Base model	Controlled supervised fine-tuning
Metric	Base	SFT-Wrong	SFT-Random	SFT-Already-Solved	SFT-CR
Recovery (%) $\uparrow$	0.0	29.2	31.1	40.6	45.3
Avg. tokens	858	888	908	918	891
Trunc. (%) $\downarrow$	0.0	10.4	10.4	13.2	8.5

Table 1: Held-out low-budget recovery after controlled SFT. All SFT variants use 616 examples and the same base model and LoRA training configuration.

Method	Overall (735)	CR (106)	Unresolved (257)
Base	50.6	0.0	0.0
SFT-Random	49.1	34.9	2.3
SFT-Already-Solved	49.4	30.2	2.7
SFT-CR	51.8	47.2	3.9

Table 2: General test accuracy and responsiveness-stratified performance. The mixed test set combines the frozen CR test split with independently sampled Already-Solved and Unresolved examples from the same source pools.

	Maximum generation budget
Method	512	1024	2048	4096
Base	0.0	0.0	0.0	0.0
SFT-Random	9.4	24.5	34.6	37.7
SFT-Already-Solved	8.5	22.6	30.2	33.0
SFT-CR	17.0	28.3	47.2	48.1

Table 3: Compute–accuracy scaling across inference budgets. Accuracy (%) is reported under maximum generation budgets from 512 to 4,096 tokens.

Dataset	$N$	Base	SFT-Random	SFT-AlreadySolved	SFT-CR	$\Delta$ CR vs. Best Non-CR
NuminaMath-CoT	47	0	25.5	23.4	40.4	+14.9
DeepScaleR	27	0	48.1	40.7	51.9	+3.8
OpenThoughts	14	0	35.7	21.4	57.1	+21.4
OpenThoughts3	14	0	35.7	35.7	42.9	+7.2
AIME	0	–	–	–	–	–
OlympiadBench	4	0	50.0	50.0	75.0	+25.0

Table 4: Dataset-wise performance at the 2,048-token evaluation budget. The final column reports the absolute SFT-CR improvement over the best non-CR SFT baseline in percentage points.

Method	Rec. $\uparrow$	Tok.	Trunc.	Rec. $\uparrow$	Tok.	Trunc.
	Single			DDP
AlreadySolved	32.1	840	9.4	30.2	884	8.5
All	33.0	947	17.9	32.1	852	6.6
CR	37.7	871	7.5	34.9	899	13.2

Table 5: Held-out CR recovery under on-policy distillation. CR-OPD achieves the highest recovery in both single-node and distributed training settings.

Model	Params.	CR Rate	Random	Already	CR
Qwen3-8B	8B	8.3	31.1	40.6	45.3
Llama-3.1-8B-Instruct	8B	9.2	24.1	17.2	27.6

Table 6: Cross-model generalization. CR examples are reconstructed independently for each model, and Random, Already-Solved, and CR training sets are matched within each model.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Evaluated	CR	CR Rate
AIME	90	4	4.4%
OlympiadBench	674	35	5.2%
OpenThoughts	2,000	145	7.2%
OpenThoughts3	2,000	142	7.1%
DeepScaleR	3,000	269	9.0%
NuminaMath-CoT	5,067	470	9.3%

Appendix

Table 7: Compute-responsive examples observed during the large-scale collection stage.

	Train		Validation		Test		Total
Dataset	$N$	%	$N$	%	$N$	%	$N$	%
NuminaMath-CoT	376	44.1	47	44.3	47	44.3	470	44.1
DeepScaleR	215	25.2	27	25.5	27	25.5	269	25.3
OpenThoughts	117	13.7	14	13.2	14	13.2	145	13.6
OpenThoughts3	114	13.4	14	13.2	14	13.2	142	13.3
AIME	4	0.5	0	0.0	0	0.0	4	0.4

Appendix

Table 8: Per-dataset composition of the frozen V1 CR corpus. Percentages are computed within each split.

Figure 3: Composition of the frozen CR corpus. (a) Source composition of the 1,065 compute-responsive problems and their train, validation, and test allocation. (b) Stratified source composition across the frozen training, validation, and test splits.

Raw form A	Raw form B	Normalized meaning
$k=-2/7$	$-2/7$	$-2/7$
$\left(0,\frac{1}{8}\right)$	$\left(0,\frac{1}{8}\right)$	$\left(0,\frac{1}{8}\right)$
“C: $1/2$ ”	$1/2$	$1/2$
“80 cm 2 ”	$80$	$80$

Appendix

Table 9: Examples of normalization rules introduced after the grader audit.

Method	Examples	Target
SFT-Wrong	616	Incorrect low trajectory
SFT-Random	616	Random correct trajectory
SFT-Already-Solved	616	Correct low trajectory
SFT-CR	616	Successful CR blind trajectory

Appendix

Table 10: Controlled SFT training sets.

Checkpoint	Recovery	Avg. Tokens	Truncation
Epoch 1	41.5%	865	7.5%
Epoch 2	45.3%	891	8.5%
Epoch 3	42.5%	894	8.5%

Appendix

Table 11: CR-SFT checkpoint selection on the frozen validation set.

Base model
Method	Recovery	Avg. Tokens	Latency / Sample	Total Eval. Time	Truncation	Training Time
Base	0.0	858	–	–	0.0	–
Supervised fine-tuning
SFT-Wrong	29.2	888	5.6 s	9.9 min	10.4	$\sim$ 20 min
SFT-Random	31.1	908	5.5 s	9.8 min	10.4	$\sim$ 20 min
SFT-Already-Solved	40.6	918	5.5 s	9.6 min	13.2	$\sim$ 20 min

Appendix

Table 12: Training and low-budget inference efficiency. Missing latency values will be filled from the unified timing evaluation.

Problem	Base failure	CR-SFT recovery
Triangle $ABC$ , $a{=}12,b{=}17,c{=}13$ ; compute $b\cos C-c\cos B$ . (Ans. 10)	Correct Law-of-Cosines setup but an arithmetic slip ( $144{+}169$ taken as $213$ not $313$ ) yields $\cos B=-\tfrac{19}{78}$ and the wrong answer $\tfrac{1105}{78}$ .	Same method, correct arithmetic: $\cos B=\tfrac{1}{13}$ , $\cos C=\tfrac{11}{17}$ , so $17\cdot\tfrac{11}{17}-13\cdot\tfrac{1}{13}=11-1=\mathbf{10}$ .
Max. checkers on an $8\times 8$ board so each is diagonally attacked. (Ans. 32)	Finds a valid 32-checker pattern, then discards it and claims all $64$ squares work (“every square is adjacent to others”), missing the upper bound. Answers $\mathbf{64}$ .	Keeps the 32-configuration and proves any 33rd checker must be left isolated (unattacked); hence the maximum is $\mathbf{32}$ .
Rearrangements of ccamb with at least one C before the A. (Ans. 40)	Reduces to the order of $\{A,C,C\}$ but miscounts all three orderings (ACC, CAC, CCA) as “C before A”—ACC is not—so concludes all $60$ permutations qualify.	Only CAC and CCA qualify ( $2$ of $3$ ), giving probability $\tfrac{2}{3}$ , so $60\cdot\tfrac{2}{3}=\mathbf{40}$ .

Appendix

Table 13: Representative recovery cases. Exact examples will be inserted after final manual selection.

Model	Evaluated	CR Count	CR Rate	Overlap with Qwen3-8B CR
Qwen3-8B	12,831	1,065	8.3%	–
Llama-3.1-8B-Instruct	10,764	992	9.2%	109 (11.0%)

Appendix

Table 14: Cross-model CR-set statistics. CR examples are reconstructed separately for each model from the same underlying reasoning sources. The overlap column reports overlap with the Qwen3-8B CR set.

Figure 4: Cross-model overlap of compute-responsive examples. Qwen3-8B and Llama-3.1-8B-Instruct identify 1,065 and 992 CR problems, respectively, with only 109 examples shared between the two CR sets. The shared examples account for 10.2% of the Qwen3-8B CR set and 11.0% of the Llama-3.1-8B-Instruct CR set.

Method	Seed 1	Seed 2	Seed 3	Mean $\pm$ Std.
SFT-CR	46.2	41.5	49.1	45.6 $\pm$ 3.8
SFT-Already-Solved	39.6	39.6	38.7	39.3 $\pm$ 0.5
CR-OPD	33.0	32.1	34.9	33.3 $\pm$ 1.4

Appendix

Table 15: Multi-seed training stability.

First successful probe	Count	Share among CR	Cumulative recall
Blind-1	529	49.6	49.6
Blind-2	249	23.4	73.0
Audit-1	229	21.5	94.5
Audit-2	59	5.5	100.0
Never recovered (non-CR)	3375	–	–

Appendix

Table 16: First successful additional trajectory among CR examples. Most CR examples are detected before all four probes are used.

Figure 5: First-hit distribution and cumulative recall during CR discovery. (a) Distribution of the first successful additional probe among CR examples. (b) Cumulative recall after successive blind re-solving and audit-based revision attempts. Most CR examples are recovered within the first two blind probes, with a mean first-hit cost of 1.83 additional generations.

Method	Already Solved	CR	Unresolved
SFT-Already-Solved	87.1	30.2	2.7
SFT-CR	86.3	47.2	3.9
SFT-Mix	87.4	43.4	3.5

Appendix

Table 17: Retention–recovery trade-off under mixed SFT. Mixed training combines 50% CR examples with 50% Already-Solved examples.

Condition	Mean KL	First 10 steps	Last 10 steps	Trajectory
OPD-CR	0.472	0.564	0.420	Clear decrease
OPD-All	0.366	0.389	0.344	Mild decrease
OPD-AlreadySolved	0.354	0.346	0.346	Flat

Appendix

Table 18: Teacher–student KL dynamics under OPD. CR-selected states exhibit the largest teacher–student divergence and the clearest decrease during training, indicating a stronger and more learnable distillation signal.

	Available non-CR pool		Sampled test set
Dataset	Already-Solved	Unresolved	Already-Solved	Unresolved
AIME	13	16	13	16
DeepScaleR	1243	737	94	56
NuminaMath-CoT	2273	1415	92	58
OlympiadBench	254	182	87	63
OpenThoughts	762	566	86	64

Appendix

Table 19: Construction of the non-CR portion of the mixed generalization test set. Already-Solved and Unresolved examples are sampled within each source dataset according to their natural proportions in the available non-CR pool.

Training Data Selection

Training data selection studies how to choose, filter, weight, or schedule examples to improve learning while reducing data and compute costs.

See all 186 papers

Jun 1, 2026

A Primer in Post-Training Reasoning Data: What We Know About How It Works

Post-training has become a primary driver of recent progress in large reasoning models, and reasoning data are often the key variable determining whether this stage succeeds. Work on post-training reasoning data has grown rapidly, yet this literature remains scattered across dataset papers, reinforcement-learning recipes, reward-model studies, benchmarks, and frontier system reports. This paper is the first primer to synthesize over 150 key public studies and system reports on post-training reasoning data. We organize the field around four questions: what data objects exist, what makes them useful, how they are constructed, and how they scale. Together, this organization provides an attribution framework for future reasoning-data releases and post-training recipes.

Yaoming Li, Guangxiang Zhao, Qilong Shi +3

State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Qiyuan Tech · Tsinghua University

RL Post-Training LLM Post-Training Large Reasoning Models

Ask PDF

Aug 20, 2026

Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation

Reasoning language models trained with reinforcement learning typically operate under a fixed token budget rather than an explicitly adaptive one, which can lead to over-computation on easy problems and insufficient computation on difficult ones. We study whether a model can learn to allocate its own reasoning effort by choosing, as the first token of its response, one of three modes: NoThink (answer as quickly as possible), Short (brief reasoning), or Long (extended reasoning). The choice is learned inside Group Relative Policy Optimization (GRPO) with no separate router, through a shaped reward that makes each mode worthwhile at a different response length, together with hard per-mode token caps that keep the modes distinct. On a 1.5B distilled model trained on MATH, the three modes emerge without collapsing to a single choice, and the brief modes end up more accurate than Long, which shows that the router sorts problems by difficulty rather than at random. Averaged over three seeds, the resulting policy stays close to the base model's accuracy on the held-out MATH500 (

0.782

vs.\

0.796

) while cutting the mean response length from

4{,}796

2{,}811

tokens (a

41\%

reduction). Interestingly, it also transfers to other benchmarks without retraining, with the largest savings where problems are easier, with for instance 76% token reduction on GSM8K and at higher accuracy than the baselines at similar response length. In short, we build a reasoning model that adaptively chooses how much to reason for each problem.

Gijs Kassenaar, Zhao Yang, Vincent François-Lavet

Vrije Universiteit Amsterdam

Efficient Language Model Reasoning RL for Language Model Reasoning LLM Mathematical Reasoning

Ask PDF

Mar 25, 2025

Scaling Evaluation-time Compute with Reasoning Models as Evaluators

As language model (LM) outputs get more and more natural, it is becoming more difficult than ever to evaluate their quality. Simultaneously, increasing LMs' "thinking" time through scaling test-time compute has proven an effective technique to solve challenging problems in domains such as math and code. This raises a natural question: can an LM's evaluation capability also be improved by spending more test-time compute? To answer this, we investigate employing reasoning models-LMs that natively generate long chain-of-thought reasoning-as evaluators. Specifically, we examine methods to leverage more test-time compute by (1) using reasoning models, and (2) prompting these models to evaluate not only the response as a whole (i.e., outcome evaluation) but also assess each step in the response separately (i.e., process evaluation). In experiments, we observe that the evaluator's performance improves monotonically when generating more reasoning tokens, similar to the trends observed in LM-based generation. Furthermore, we use these more accurate evaluators to rerank multiple generations, and demonstrate that spending more compute at evaluation time can be as effective as using more compute at generation time in improving an LM's problem-solving capability.

Seungone Kim, Ian Wu, Jinu Lee +8

CMU · UIUC2 · KAIST AI3 +3

LLM Evaluation Language Model Generation Evaluation Inference-Time Scaling Large Reasoning Models Language Model Reasoning Evaluation

Ask PDF

Abstract

Training Data Selection

Explore similar work

A Primer in Post-Training Reasoning Data: What We Know About How It Works

Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation

Scaling Evaluation-time Compute with Reasoning Models as Evaluators