Reinforcement learning with verifiable rewards (RLVR) has emerged as a prominent paradigm for training reasoning models, but its computational cost makes efficient data selection increasingly important. Since training examples contribute unequally to learning, selecting a small, informative subset can offer a more efficient alternative to using the entire data pool. The training-free selection method estimates prompt value with hidden states from complete greedy rollouts. However, in our datasets, 26--32% of these rollouts fail to terminate within the generation budget. We introduce LIFT, a training-free data selector that identifies impactful prompts using only partial reasoning rollouts. At its core, LIFT uses Partial Rollout Context Lift (PRCL), a score inspired by pointwise V-information that accumulates token-level log-likelihood ratios between a question-conditioned context and a null context with the question removed. This enables prompt scoring from a short rollout prefix rather than a complete trajectory. Across Qwen2.5-Math-1.5B and DeepSeek-R1-Distill-1.5B, prefixes retaining at least 90% and 95% of the full-rollout PRCL score, respectively, require only 26% and 4.7% of the tokens generated by full rollouts. Compared with SHIFT, LIFT reduces selection cost by 3.7--9.1× while achieving competitive downstream RLVR performance, with the highest six-benchmark average in three of six settings.
Figures & tables
Figure 1: Tokens generated to score each dataset and model by LIFT(ours), SHIFT-ES, and SHIFT.
Figure 2: Overview of LIFR. Given an input prompt, the LLM generates a greedy partial rollout, and the PRCL score is calculated from the entropy with and without the question. Then the prompts are ranked by it.
Training dataset
Evaluation benchmarks
Dataset
Budget
Method
Math500
AIME24
AMC
Gaokao
Miner.
Olymp.
Avg.
# Sel. Tok.
Qwen2.5-Math-1.5B
Base model
33.5
10.8
29.4
18.0
6.2
21.3
19.9
–
MATH 350 train
2% (7)
Random
64.0 ±1.2
10.9 ±1.8
47.4 ±2.3
51.9 ±1.1
17.3 ±1.1
28.8 ±0.6
36.7 ±0.6
-
SHIFT
63.9 ±2.6
8.8 ±4.2
48.8 ±2.4
53.3 ±0.7
19.0 ±0.9
28.9 ±0.7
37.1 ±0.9
0.49M
SHIFT-ES
65.7 ±0.9
6.9 ±2.1
42.3 ±1.7
55.4 ±1.6
20.3 ±0.9
28.2 ±0.7
36.5 ±0.6
0.24M
Table 1: Accuracy (%) on math reasoning benchmarks. The left three columns identify the training dataset, budget, and selection method; the remaining columns are evaluation benchmarks, their average, and the number of tokens generated for selection (# Sel. Tok.).
Figure 3: Held-out accuracy during training for Qwen2.5-Math-1.5B under 4% training setting, evaluated every 10 steps.
Scoring model
Dataset
2%
4%
Qwen2.5-Math
MATH-500
2 / 7
5 / 14
LIMR
13 / 28
30 / 56
R1-Distill
LIMR
3 / 28
5 / 56
Table 2: Prompts common to SHIFT and SHIFT-ES selections.
Figure 4: Prompt influence by position along greedy rollouts, shown as mean per-token st in 1% bins of each rollout’s own length, with every rollout weighted equally, on LIMR (top) and the MATH-500 training set (bottom).
Target Share α
Model / Dataset
.85
.90
.95
.99
Qwen / MATH-500
K^
160
260
418
752
∩7
6
7
7
7
∩14
10
12
12
12
Qwen / LIMR
K^
208
340
520
952
∩28
13
18
21
26
Table 3: Target share versus selected set. K^ : the prefix length in tokens at which the whole dataset has accumulated a share α of its PRCL. ∩B : prompts shared by the top- B set at K^ and the top- B set of the full rollout, at the 2% and 4% budgets.
Figure 5: Random subset recovery of the whole dataset’s distribution, at three subset sizes. Each panel plots the share of LIMR rollouts that have collected their target share α of their own PRCL by a given position. The line is the whole 1,389-prompt dataset, and the band spans ten independent random draws.
2% (28)
4% (56)
Model
Method
pˉ
p=0
p≤.25
pˉ
p=0
p≤.25
Qwen
Random
.324
4
14
.275
12
31
SHIFT
.174
5
24
.206
8
42
PRCL
.114
9
23
.142
16
45
R1
Random
.442
7
13
.472
15
23
SHIFT
.163
18
22
.081
46
50
Table 4: Solve rate of the selected LIMR subsets, each scored by the model that selects and trains on it: subset mean pˉ and the number of prompts with p=0 and with p≤0.25 . p is estimated from 16 sampled rollouts per prompt at temperature 0.7 for both models.
Qwen2.5-Math
R1-Distill
α
med.
range
med.
range
0.80
128
112–208
96
64–144
0.90
336
224–496
176
96–448
0.95
528
416–736
288
128–832
Table 5: Estimated prefix length K^ in tokens at three target shares, on greedy LIMR rollouts: the median and the full minimum to maximum range over 300 random 10% subsets.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Training dataset
Evaluation benchmarks
Dataset
Budget
Method
Math500
AIME24
AMC
Gaokao
Miner.
Olymp.
Avg.
Qwen2.5-Math-1.5B
Base model
40.6
10.0
37.5
17.5
7.4
23.9
22.8
MATH 350 train
2% (7)
Random
65.2 ±2.7
10.4 ±5.5
49.1 ±5.8
53.1 ±1.8
16.5 ±1.2
29.8 ±1.3
37.4 ±1.8
SHIFT
64.7 ±2.1
6.7 ±7.2
46.9 ±6.9
55.2 ±2.6
17.9 ±1.3
28.7 ±1.5
36.7 ±1.0
SHIFT-ES
67.7 ±1.4
5.8 ±1.7
45.6 ±5.5
57.2 ±0.9
19.5 ±1.2
27.9 ±0.9
37.3 ±0.7
Appendix
Table 6: Greedy pass@1 (%) for the checkpoints of Table 1 : one sample per problem at temperature 0 and top- p1 , with a 2,688 token cap for Qwen2.5-Math and 12,288 for DeepSeek-R1-Distill. Rows, methods and training seeds match Table 1 exactly; the selection cost column is omitted because it does not depend on the decoding rule used at evaluation. Bold marks the best method within each budget block.
Table 7: Prompt templates and null contexts for the two models. Benchmark problems are substituted into the same template as the model’s own training prompt.
Table 8: Two prompts selected by LIFT, one per model, both near the top of their selection by PRCLK^ and of almost the same length. Only the first is difficult for the model that selected it.