Strategically Diverse Sampling for Self-Training

Alexander Gurung; Esmeralda S. Whitammer; Mirella Lapata

cs.CLSep 25, 2026

Strategically Diverse Sampling for Self-Training

Authors: Alexander Gurung, Esmeralda S. Whitammer, Mirella Lapata

Organizations: School of Informatics, University of Edinburgh

Abstract

Many LLM training and inference methods, including RL and test-time scaling, depend on repeated sampling, but benefit only when the responses meaningfully differ. Self-training faces the same challenge: training data is typically constructed by sampling IID responses and filtering primarily for correctness, thereby overrepresenting strategies a model already favours. We investigate strategic diversity, or substantive variation among approaches to a problem, as an alternative principle for constructing self-training data. We generate strategically diverse data with two sampling methods: GROOT, a new method which constructs a hierarchical tree of approaches and samples distinct paths, and Verbalized Sampling (VS), adapted to produce an unstructured set of approaches. Across competitive programming and Next-Chapter Prediction domains, models trained on strategically sampled data outperform IID-trained counterparts on difficult tasks and provide strong initializations for RL and test-time scaling. Most strikingly, self-training on strategically diverse but incorrect traces from Qwen3-4B outperforms IID distillation from a 235B teacher. These results challenge prevailing assumptions about what makes useful self-training data and show that diversity of approaches can matter more than correctness or teacher scale.

Figures & tables

Panel of Figure 1 — Figure 1: Generating strategically diverse training data. All methods sample from the base model at the same budget. IID sampling draws $n$ independent solutions and keeps the correct ones. Groot generates a tree of strategies and samples $n$ root-to-leaf paths; VS directly proposes $n$ approaches with their probabilities. Each approach is passed back as a hidden instruction to produce a solution. Trained models are evaluated with pass@ $k$ , test-time scaling, and RL.

	Code			NCP
Method	4	8	64	5%	10%	15%
IID	0 42	0 77	312	1,921	631	156
IID ( $T{=}1.5$ )	0 60	106	379	5,326	3,545	1,495
Groot	155	240	—	2,775	1,077	286
Groot ( $T{=}1.5$ )	158	243	—	5,366	3,474	1,505
VS	137	217	—	2,835	1,127	321

Table 1: Qwen3-4B sampling results. Code: number of 1,833 Cobalt training-frontier problems solved at least once at each sampling budget. NCP: number of 6,295 training sections with

>0

of 16 plans reaching each perplexity-improvement threshold. Every row is over the same sections, the ones all six sampling runs cover.

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Base	0.5/3.6/21.9	0.2/1.3/7.4	0.1/0.5/2.9	0.2/0.9/5.2
IID-4	0.7/4.9/19.8	0.2/1.5/7.0	0.1/0.7/3.5	0.2/1.1/5.2
IID-4 ( $T{=}1.5$ )	1.0/6.5/22.4	0.5/3.2/10.8	0.1/1.0/5.2	0.3/2.1/8.0
IID-64	1.0/6.3/21.5	0.4/2.5/8.7	0.1/1.0/4.4	0.2/1.8/6.5
IID-64 ( $T{=}1.5$ )	1.5/7.4/23.0	0.4/2.8/10.7	0.2/1.4/5.2	0.3/2.1/7.9

Table 2: Code pass@

k

for Qwen3-4B-Instruct on frontier subsets (problems solved by the base model

\leq 2

of 64 times). Cobalt is in-distribution; LiveCodeBench and OJBench are held out, with Macro-Avg averaging them. Suffix -

n

denotes responses sampled per problem (e.g., Groot -4 uses four). Models use RFT unless marked ANTI, which retains only incorrect responses. Bold marks the best value per column; full test-set results, additional datasets, and budgets appear in Appendix A.2 .

Panel of Figure 2 — Figure 2: Qwen3-4B. (a) NCP coverage of test sections at the 15% improvement threshold as the number of sampled plans $k$ increases. (b) Best-so-far pass@8 on the Cobalt validation frontier during RL from each budget-4 RFT model, mean and standard error over three seeds; IID-based models need many steps to reach where the strategic models ( Groot , VS ) begin.

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
IID-4 (self)	0.7/5.2/20.5	0.3/1.9/8.2	0.1/0.7/3.3	0.2/1.3/5.8
IID-4 (teacher)	4.6/16.7/39.3	1.1/6.9/19.1	0.3/2.1/7.6	0.7/4.5/13.4
GROOT-4 (self)	4.8/18.5/39.5	2.4/11.3/25.0	1.8/7.3/ 15.2	2.1/9.3/20.1
GROOT-4 (teacher)	5.0/19.4/ 40.3	1.8/9.3/20.5	1.4/6.7/15.0	1.6/8.0/17.8
VS-4 (self)	5.5 / 19.6 / 40.3	3.1 / 13.6 / 30.8	2.1 / 7.6 /14.7	2.6 / 10.6 / 22.8

Table 3: Frontier-set pass@

k

with teacher- versus self-generated training data. Teacher data uses Qwen3-235B-A22B-Instruct for both approaches and solutions; self-generated data uses the 4B student. In all cases the trained model is Qwen3-4B-Instruct, with sampling budget 4 and a 16k token cap (increased from 8k due to truncation in teacher-trained models). Full test sets, budget 8, and 8k-cap results are in Appendix A.4 .

Table 6

Method	Cobalt			LiveCodeBench			OJBench			Macro-Avg
	$p@1/8/64$			$p@1/8/64$			$p@1/8/64$			$p@1/8/64$
Base	0 0.5 /	0 3.6 /	21.9	0 0.2 /	0 1.3 /	0 7.4	0 0.1 /	0 0.5 /	0 2.9	0 0.2 /	0 0.9 /	0 5.2
+RL	0 4.5 /	13.9 /	29.3	0 1.1 /	0 5.9 /	15.3	0 0.4 /	0 2.4 /	0 7.5	0 0.8 /	0 4.1 /	11.4
IID-4	0 0.7 /	0 4.9 /	19.8	0 0.2 /	0 1.5 /	0 7.0	0 0.1 /	0 0.7 /	0 3.5	0 0.2 /	0 1.1 /	0 5.2
+RL	0 5.5 /	17.3 /	36.6	0 1.1 /	0 6.0 /	15.4	0 0.4 /	0 2.0 /	0 6.4	0 0.7 /	0 4.0 /	10.9
GROOT-4	0 3.5 /	14.1 /	30.5	0 1.5 /	0 8.0 /	19.5	0 1.2 /	0 5.3 /	11.5	0 1.4 /	0 6.7 /	15.5

Table 6: Qwen3-4B frontier-subset performance after RL from each budget-4 initialization; the full test sets are in Appendix A.5 . For each arm the first row is the student before RL and the second the mean over three RL runs of the checkpoints selected by validation pass@8. We report pass@1 / pass@8 / pass@64; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Full dataset
Base	11.4/32.4/52.2	23.1/38.2/50.4	3.2/6.6/10.5	13.2/22.4/30.4
IID-4	12.9/33.7/51.8	23.4/37.5/49.0	3.3/7.0/11.5	13.3/22.2/30.2
IID-8	15.4 /38.0/55.2	26.7 /40.8/52.5	3.9/8.8/16.5	15.3 /24.8/34.5
IID-64	13.1/33.9/52.5	24.1/38.9/51.9	3.5/7.5/12.5	13.8/23.2/32.2

Appendix

Table 7: Nemotron3-Nano-4B: code pass@

k

on the full test sets and on the Nemotron base model’s own frontier (519 problems, 428 shared with the Qwen frontier); Nemotron and Qwen numbers are not comparable in absolute terms. We report pass@1/8/64; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

Figure 3: Nemotron3 Nano-4B: best-so-far pass@8 on its validation frontier set during RL training from each budget-4 model, mean and standard error over three seeds.

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Full dataset
Base	11.4/32.4/52.2	23.1/38.2/50.4	3.2/6.6/10.5	13.2/22.4/30.4
+RL	27.7/57.5/73.6	35.4/51.9/62.4	9.9/21.0/30.1	22.6/36.5/46.3
IID-4	12.9/33.7/51.8	23.4/37.5/49.0	3.3/7.0/11.5	13.3/22.2/30.2
+RL	30.2/58.7/73.7	38.9 / 55.6 /65.6	11.0 / 22.8 / 31.5	25.0 / 39.3 / 48.5

Appendix

Table 8: Nemotron3-Nano-4B competitive programming: test performance after RL from each initialization, mean over three RL runs. We report pass@1/8/64; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

	Full test set			Frontier ( $\leq 2/64$ )
Method	Cobalt	LiveCodeBench	OJBench	Cobalt	LiveCodeBench	OJBench
Base	0 8.0 $\rightarrow$ 21.9	21.0 $\rightarrow$ 28.2	0 2.6 $\rightarrow$ 0 7.7	0 0.3 $\rightarrow$ 0 3.9	0 0.3 $\rightarrow$ 0 2.2	0 0.1 $\rightarrow$ 0 3.5
IID-4	10.7 $\rightarrow$ 23.8	21.9 $\rightarrow$ 29.9	0 2.9 $\rightarrow$ 0 9.1	0 0.6 $\rightarrow$ 0 4.8	0 0.3 $\rightarrow$ 0 0.9	0 0.1 $\rightarrow$ 0 4.1
IID-8	13.6 $\rightarrow$ 26.7	25.9 $\rightarrow$ 32.5	0 3.6 $\rightarrow$ 0 9.3	0 1.0 $\rightarrow$ 0 5.5	0 0.6 $\rightarrow$ 0 2.8	0 0.3 $\rightarrow$ 0 4.4
Groot -4	11.5 $\rightarrow$ 22.8	23.4 $\rightarrow$ 28.8	0 4.4 $\rightarrow$ 0 8.0	0 1.4 $\rightarrow$ 0 4.3	0 0.6 $\rightarrow$ 0 2.4	0 1.1 $\rightarrow$ 0 3.8
Groot -8	11.3 $\rightarrow$ 22.7	22.7 $\rightarrow$ 28.1	0 3.7 $\rightarrow$ 0 7.3	0 1.0 $\rightarrow$ 0 4.2	0 0.6 $\rightarrow$ 0 2.3	0 0.7 $\rightarrow$ 0 3.0

Appendix

Table 9: Nemotron3-Nano-4B competitive programming: RSA pass@1 before and after ten aggregation iterations (population size 16), mean over three runs; bold marks the best final value in each column.

	Code			NCP
Method	4	8	64	5%	10%	15%
IID	52	97	424	5,375	2,807	0 915
Groot	115	197	—	5,725	3,340	1,140
VS	137	233	—	5,707	3,286	1,172

Appendix

Table 10: Nemotron3-Nano-4B sampling results. Code: Cobalt training-frontier problems solved at least once at each sampling budget; the frontier is defined relative to Nemotron3-Nano-4B, so it is a different problem set from the one in Table 1 . NCP: number of 6,754 training sections with

>0

of 16 plans reaching each perplexity-improvement threshold. Every row is over the same sections, the ones all three sampling runs cover.

Method	$5\%$		$10\%$		$15\%$
	$k=16$	$k=64$	$k=16$	$k=64$	$k=16$	$k=64$
Base (no distill)	74.9	86.7	37.4	51.4	11.1	18.6
IID	72.3	83.9	34.6	48.0	10.4	17.1
Groot	82.8	91.0	45.3	59.5	14.6	23.4
VS	81.4	90.8	43.6	57.7	14.2	22.6

Appendix

Table 11: Nemotron3-Nano-4B NCP coverage (%) of test sections at three perplexity-improvement thresholds, for Nemotron3-Nano-4B trained on plans clearing a

20\%

threshold at a sampling budget of 8. Coverage is over the same 1,430 test sections for every row. Bold marks the best value in each column.

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Full dataset
Base	11.4/32.4/52.2	23.1/38.2/50.4	3.2/6.6/10.5	13.2/22.4/30.4
IID-4 (self)	12.9/33.7/51.8	23.4/37.5/49.0	3.3/7.0/11.5	13.3/22.2/30.2
IID-4 (teacher)	13.0/35.6/55.8	23.6/38.7/51.4	3.9/9.1/16.7	13.8/23.9/34.0
GROOT-4 (self)	13.2/36.0/56.7	25.6/41.9/55.8	5.0/12.5/22.0	15.3/27.2/38.9

Appendix

Table 12: Nemotron3-Nano-4B distilled from a larger teacher: code pass@

k

on the full test sets and on the frontier subsets, which are defined relative to Nemotron3-Nano-4B. Each method appears twice, distilled from the 4B teacher and from the larger teacher (Nemotron3-Super-120B-A12B-BF16), with the generation budget and all other settings held fixed so that only the teacher changes. We report pass@1/8/64; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Full dataset
Base	16.0/39.4/56.7	34.3 /44.0/50.3	9.3 /14.8/18.3	21.8 /29.4/34.3
IID-4	15.0/37.9/55.1	34.0/43.4/50.0	8.8/14.6/18.4	21.4/29.0/34.2
IID-8	15.1/38.1/55.5	33.9/43.3/50.1	9.0/14.6/18.3	21.4/28.9/34.2
IID-64	16.2/39.6/55.7	33.9/43.7/50.8	9.2/14.8/19.1	21.5/29.2/35.0

Appendix

Table 13: Qwen3-4B code pass@

k

at sampling budgets 4 and 8, and for an equal mix of the Groot -4 and VS -4 datasets (MIX-8). We report pass@1/8/64; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Full dataset
Base	16.0/39.4/56.7	34.3/44.0/50.3	9.3/14.8/18.3	21.8/29.4/34.3
IID-4 RFT	15.0/37.9/55.1	34.0/43.4/50.0	8.8/14.6/18.4	21.4/29.0/34.2
IID-4 SFT	15.3/38.6/55.0	33.8/43.7/50.9	9.3/14.9/18.8	21.5/29.3/34.9
IID-4 ANTI	15.5/38.7/55.1	34.1/43.5/49.9	9.0/14.6/18.9	21.6/29.1/34.4

Appendix

Table 14: Qwen3-4B competitive programming: RFT (correct responses only), SFT (all responses) and ANTI (incorrect responses only) at sampling budgets 4 and 8, trained on the same sampled datasets; IID-64 at

T{=}1.5

is the strongest IID dataset. We report pass@1/8/64; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Full dataset
Base	16.0/39.4/56.7	34.3/44.0/50.3	9.3/14.8/18.3	21.8/29.4/34.3
IID-4	15.0/37.9/55.1	34.0/43.4/50.0	8.8/14.6/18.4	21.4/29.0/34.2
IID-4 ( $T{=}1.5$ )	17.2/41.1/56.8	34.7 /44.9/52.1	9.6 /15.2/20.1	22.2 /30.0/36.1
IID-8	15.1/38.1/55.5	33.9/43.3/50.1	9.0/14.6/18.3	21.4/28.9/34.2

Appendix

Table 15: Competitive programming. Effect of the sampling temperature used to create the Qwen3-4B training data, on the IID models and on the structured Groot -4 and VS -4 datasets. We report pass@1/8/64; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

Figure 4: Unique problems solved by Qwen3-4B on the 1,833-problem Cobalt training frontier at a sampling budget of four. Left: IID sampling by temperature (mean of three runs). Right: Groot and VS by planner and solver temperature (mean over three runs per cell).

Table 19

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Full dataset
IID-4 (self)	15.1/38.3/55.5	34.1/43.8/50.6	8.8/14.3/17.8	21.5/29.0/34.2
IID-4 (teacher)	16.9/43.8/63.9	32.9/45.9/54.9	7.8/15.0/21.8	20.3/30.4/38.4
IID-8 (self)	15.2/38.3/55.9	34.0/43.7/50.8	9.0/14.3/17.6	21.5/29.0/34.2
IID-8 (teacher)	17.9/45.5/65.2	32.7/46.2/55.6	8.0/15.2/21.7	20.4/30.7/38.6

Appendix

Table 18: Pass@k from teacher- versus self-generated training data. Teacher datasets are constructed with Qwen3-235B-A22B-Instruct for both the approaches and solutions. The standard self-generating procedure uses the 4B student (marked self). In both cases the trained student is Qwen3-4B-Instruct. Evaluated with a 16k-token generation limit. We report pass@1 / pass@8 / pass@64; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Full dataset
IID-4 (self)	15.0/37.9/55.1	34.0 /43.4/50.0	8.8/14.6/18.4	21.4 /29.0/34.2
IID-4 (teacher)	16.0/42.0/62.3	32.4/45.1/54.3	7.5/14.4/21.1	19.9/29.8/37.7
IID-8 (self)	15.1/38.1/55.5	33.9/43.3/50.1	9.0/14.6/18.3	21.4 /28.9/34.2
IID-8 (teacher)	17.0/ 43.4 / 63.0	32.1/45.2/53.5	7.7/14.7/20.9	19.9/30.0/37.2

Appendix

Table 19: Pass@

k

from teacher- versus self-generated training data, evaluated with an 8k-token generation limit; Table 18 gives the same comparison at 16k. Teacher datasets are constructed with Qwen3-235B-A22B-Instruct for both the approaches and the solutions, while the standard self-generating procedure uses the 4B student (marked self). In both cases the trained student is Qwen3-4B-Instruct. We report pass@1/8/64 on the full test sets and on the frontier subsets; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Full dataset
Base	16.0/39.4/56.7	34.3/44.0/50.3	9.3/14.8/18.3	21.8/29.4/34.3
+RL	19.5/43.5/59.1	33.6/45.3/53.5	9.2/15.6/21.8	21.3/30.5/37.7
IID-4	15.0/37.9/55.1	34.0/43.4/50.0	8.8/14.6/18.4	21.4/29.0/34.2
+RL	21.8/46.6/63.6	35.6 /46.8/54.1	9.3/15.6/20.9	22.5 /31.2/37.5

Appendix

Table 20: Qwen3-4B performance before and after RL, from every RFT initialization: the base model, IID and strategic datasets at budgets 4 and 8, the high-temperature (

T{=}1.5

) IID variants, and MIX-8. Each variant’s first row is the student before RL and the second after RL (mean over three RL runs). Results are given on the full test sets and on the frontier subsets. We report pass@1/8/64; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

Method	Cobalt	LiveCodeBench	OJBench	Macro-Avg
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Full dataset
IID-4 SFT	15.3/38.6/55.0	33.8/43.7/50.9	9.3/14.9/18.8	21.5/29.3/34.9
+RL	21.5 / 45.9 /60.9	36.0 /46.5/53.6	9.6 /16.0/21.9	22.8 /31.3/37.7
IID-4 ANTI	15.5/38.7/55.1	34.1/43.5/49.9	9.0/14.6/18.9	21.6/29.1/34.4
+RL	20.0/43.1/58.7	35.4/45.7/52.0	9.0/14.6/19.3	22.3/30.1/35.6

Appendix

Table 21: Qwen3-4B performance before and after RL from the budget-4 SFT and ANTI initializations, the counterpart of Table 20 for the other two correctness filters. Each variant’s first row is the student before RL and the second after RL (mean over three RL runs). Results are given on the full test sets and on the frontier subsets. We report pass@1/8/64; Macro-Avg is the mean of the two held-out benchmarks, LiveCodeBench and OJBench. Bold marks the best value in each column.

Method	Easy ( $n{=}104$ )	Medium ( $n{=}31$ )	Med-hard ( $n{=}47$ )	Hard ( $n{=}61$ )	V-hard ( $n{=}13$ )
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Base	19.0/41.4/54.8	14.5/40.9/67.7	14.2/38.0/61.7	14.8 /38.4/55.7	7.2 /15.3/15.4
IID-4	18.5/41.3/54.6	13.4/35.9/63.1	13.2/37.9/62.3	13.5/36.6/51.2	6.2/15.8/22.1
IID-64	20.2/43.8/57.6	14.1/38.9/59.3	14.1/39.3/61.7	14.7/ 38.9 /53.7	6.1/15.1/19.2
IID-64 ( $T{=}1.5$ )	22.1 / 46.6 / 58.9	14.3/39.3/62.2	14.4/37.9/59.5	14.5/38.8/53.3	6.9/16.1/21.2
GROOT-4	20.0/43.8/57.5	21.8/55.3/ 75.0	15.3/43.1/62.3	12.0/37.2/54.7	5.0/16.1/26.0

Appendix

Table 22: Qwen3-4B performance on the full Cobalt test set, split by the benchmark’s own difficulty tiers rather than by model-relative frontier difficulty;

n

is the number of problems in each tier. We report pass@1/8/64. Bold marks the best value in each column.

Method	Easy ( $n{=}43$ )	Medium ( $n{=}52$ )	Hard ( $n{=}80$ )
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Base	86.2/95.4/97.7	32.3/47.2/57.7	7.8 /14.3/20.0
IID-4	85.4/95.0/97.6	32.1/46.3/56.9	7.6/13.8/19.9
IID-64	85.8/95.2/97.4	31.8/47.3/59.3	7.5/13.6/20.3
IID-64 ( $T{=}1.5$ )	86.4 /95.6/97.6	32.4 /48.2/60.0	7.6/15.3/22.0
GROOT-4	84.1/ 96.4 / 98.8	28.9/51.5/65.4	6.9/18.1/28.0

Appendix

Table 23: Qwen3-4B performance on the full LiveCodeBench test set, split by the benchmark’s own difficulty tiers rather than by model-relative frontier difficulty;

n

is the number of problems in each tier. We report pass@1/8/64. Bold marks the best value in each column.

Method	Easy ( $n{=}36$ )	Medium ( $n{=}79$ )	Hard ( $n{=}117$ )
	$p@1/8/64$	$p@1/8/64$	$p@1/8/64$
Base	45.7/63.5/70.7	5.8/12.1/18.5	0.4/1.6/2.0
IID-4	43.8/62.5/68.0	5.4/12.2/20.6	0.4/1.5/1.7
IID-64	45.3/62.5/69.0	5.7/12.7/22.1	0.4/1.5/1.7
IID-64 ( $T{=}1.5$ )	45.9 /64.5/75.7	6.0/13.4/21.5	0.5 / 1.7 /2.1
GROOT-4	41.7/70.2/ 90.2	7.0 /19.6/29.3	0.2/1.0/ 2.4

Appendix

Table 24: Qwen3-4B performance on the full OJBench test set, split by the benchmark’s own difficulty tiers rather than by model-relative frontier difficulty;

n

is the number of problems in each tier. We report pass@1/8/64. Bold marks the best value in each column.

Method	Cobalt	LiveCodeBench	OJBench
Base	15.9 $\rightarrow$ 29.6	34.2 $\rightarrow$ 39.9	0 9.1 $\rightarrow$ 12.6
IID-4	15.5 $\rightarrow$ 27.7	34.6 $\rightarrow$ 40.4	0 9.1 $\rightarrow$ 12.5
IID-64 ( $T{=}1.5$ )	17.4 $\rightarrow$ 33.9	34.7 $\rightarrow$ 42.5	0 9.2 $\rightarrow$ 13.4
Groot -4	16.8 $\rightarrow$ 37.0	33.9 $\rightarrow$ 44.5	0 9.3 $\rightarrow$ 14.1
VS -4	17.9 $\rightarrow$ 35.2	33.4 $\rightarrow$ 45.1	0 9.3 $\rightarrow$ 14.1

Appendix

Table 25: Qwen3-4B RSA pass@1 on the full test sets, before

\rightarrow

after ten aggregation iterations (population size 16), from the best-validation checkpoints; Table 5 gives the same comparison on the frontier subsets. Bold marks the best final value in each column.

Method	Easy		Medium		Med-hard		Hard		V-hard
Base	17.9 $\rightarrow$	35.9	16.0 $\rightarrow$	27.3	15.1 $\rightarrow$	27.5	15.4 $\rightarrow$	25.6	6.2 $\rightarrow$	12.8
IID-4	19.1 $\rightarrow$	34.5	14.0 $\rightarrow$	21.2	13.1 $\rightarrow$	28.4	14.4 $\rightarrow$	22.7	5.6 $\rightarrow$	10.1
IID-64	19.9 $\rightarrow$	38.7	17.0 $\rightarrow$	21.3	13.8 $\rightarrow$	31.3	14.7 $\rightarrow$	26.2	6.7 $\rightarrow$	9.1
IID-64 ( $T{=}1.5$ )	22.5 $\rightarrow$	44.1	16.1 $\rightarrow$	28.1	14.2 $\rightarrow$	29.6	15.1 $\rightarrow$	25.4	5.9 $\rightarrow$	10.7
GROOT-4	19.5 $\rightarrow$	42.6	21.0 $\rightarrow$	39.5	16.8 $\rightarrow$	37.5	12.9 $\rightarrow$	30.3	5.6 $\rightarrow$	12.3
VS-4	22.2 $\rightarrow$	42.0	23.1 $\rightarrow$	44.0	17.0 $\rightarrow$	34.6	12.8 $\rightarrow$	24.2	4.6 $\rightarrow$	10.9

Appendix

Table 26: Qwen3-4B RSA pass@1 on Cobalt, before

\rightarrow

after ten aggregation iterations (population size 16), split by the benchmark’s own difficulty tiers. Bold marks the best final value in each column.

Table 29

Figure 5: Qwen3-4B NCP coverage of test sections at three perplexity-improvement thresholds as sampled plans

k

increases, for the base model and trained models. Bands are

\pm

one standard error of the mean across the three training runs.

	Threshold
Model	5%	10%	15%
Base	27.6	9.5	2.0
IID-8	27.2	9.5	1.9
IID-16	28.3	9.7	2.1
Groot -8	39.9	15.7	3.7
Groot -16	39.1	15.0	3.6

Appendix

Table 29: Qwen3-4B NCP coverage@32 (%) of test sections at three perplexity-improvement thresholds, for Qwen3-4B-Instruct trained on plans clearing a 20% threshold. The suffix is the sampling budget: 8 or 16 candidate plans per training section. Mean over three training runs (the base model is a single model). Bold marks the best value in each column.

	Threshold
Model	5%	10%	15%
	$c@1/8/32$	$c@1/8/32$	$c@1/8/32$
IID-16 RFT	8.9/19.7/27.1	2.0/5.8/9.1	0.3/1.1/2.0
IID-16 SFT	8.9/19.7/27.2	2.1/6.0/9.1	0.3/1.1/2.0
IID-16 ANTI	9.0/19.9/28.0	2.0/6.1/9.7	0.3/1.1/2.0
Groot -16 RFT	13.6/29.1/39.5	3.5/10.0/15.4	0.6/2.1/3.9

Appendix

Table 30: Qwen3-4B NCP coverage@1/8/32 (%) of test sections for students trained under three filters: RFT keeps only samples over 20% improvement, SFT draws the same

d

number of samples at random, and ANTI takes the

d

worst samples. Bold marks the best value in each column.

	Threshold
Model	5%	10%	15%
	$c@1/8/32$	$c@1/8/32$	$c@1/8/32$
Base	9.0/20.1/27.6	2.0/6.0/9.5	0.3/1.1/2.0
+RL	5.4/13.3/19.3	1.1/3.2/5.0	0.2/0.6/1.2
IID-16	9.3/20.5/28.3	2.1/6.2/9.7	0.3/1.2/2.1
+RL	7.7/17.9/24.9	1.6/4.8/7.8	0.2/0.9/1.6

Appendix

Table 31: Qwen3-4B NCP coverage@1/8/32 (%) before and after RL from each RFT initialization: the base model, IID, and strategic datasets with budget 16. Bold marks the best value in each column.

Own threshold 5%
	Foil threshold
Model	5%	10%	15%
	$c@1/8/32$	$c@1/8/32$	$c@1/8/32$
Base	2.7/7.6/11.2	4.5/11.1/16.2	5.2/12.6/18.2
IID-16	3.4/9.3/13.9	6.0/14.6/21.1	7.2/16.8/23.5
Groot -16	5.1/13.7/ 20.4	14.3/30.9/ 41.2	21.5/40.8/ 51.9

Appendix

Table 32: Qwen3-4B NCP coverage@1/8/32 (%) of test sections after RL, requiring the arm’s own plan to clear its threshold while both foil plans stay below the foil threshold. Row groups are the own threshold. Bold marks the best value in each column.

	Setting	Value
Sampling (code)	default budget per problem	4 ( $1\times 4$ )
	planner $T$ / top- $p$ / max tokens	0.45 / 0.95 / 8192
	solver and IID $T$ / top- $p$ / max tokens	0.85 / 0.95 / 8192
	high-temperature IID	$T=1.5$
Sampling (NCP)	default budget per section	16 ( $2\times 8$ )
	planner and solver $T$	0.85

Appendix

Table 33: Sampling, training, evaluation, RSA, and RL settings, used for every dataset and both base models (Qwen3-4B and Nemotron-3-Nano-4B) unless otherwise noted.

Self-Training for Language Models

Self-training for language models uses their own generations, feedback, and interaction traces as supervision to improve capabilities with less labeled data.

See all 79 papers

Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline
May 20, 2026Tony Lee, Percy LiangLanguage Model Self-Verification Self-Training for Language Models
Stanford University
From Rollouts to Recipes: Self-Contained Post-Training for LLMs
Sep 1, 2026Yifei Li, Lingling Zhang, Muye Huang +3Self-Training for Language Models RL for Language Model Reasoning
School of Computer Science and Technology, Xi’an Jiaotong University, China · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering · Zhongguancun Academy, Beijing, China +1
Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Models
May 8, 2026Aswin RRV, Jacob Dineen, Divij Handa +4Synthetic Data Generation RL for Language Model Reasoning
Arizona State University · Google Cloud AI Research · Google DeepMind

May 20, 2026J/K move · Enter open · S save

Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline

Tony Lee, Percy Liang

Stanford University

Language Model Self-Verification Self-Training for Language Models Language Model Self-Assessment Language Model Distillation Synthetic Data Generation for Language Models Language Model Self-Improvement+1

Open paper AskPDF arXiv

Can post-trained large language models (LLMs) further improve themselves using only unlabeled prompts, without external teachers or feedback from tools? We study this setting starting only from unlabeled seed questions with no ground-truth solutions, across three reasoning domains: math, science, and coding. We propose Self-Verified Distillation, a simple post-training refinement algorithm in which the model generates candidate solutions to these seed questions, filters them using prompt-based self-verification, and trains on the resulting self-curated dataset. Inspired by the UQ benchmark's use of multiple validators to screen candidate answers to hard unsolved questions, we adapt this validation-based filtering idea to self-training: the model filters its own generated solutions through a three-stage cascade of cycle-consistency, factuality, and correctness checks, accepting a solution only if it passes all stages with unanimous judge votes. We find that sampling more candidate generations and using a larger verification budget during training data construction produces higher-quality self-curated data and, in turn, better reasoning models. We then train Qwen3 models at multiple scales with Self-Verified Distillation and obtain gains across all three domains. For Qwen3-4B, our method improves aggregate held-out pass@1 by +16.7 points in math (AIME26 and HMMT), +11.1 points in science (GPQA Diamond and HLE), and +8.3 points in coding (LCBv5 and LCBv6), with gains also extending to 0.6B and 8B models. Compared to our test-time-only baseline (UQ-TTC), which improves performance by spending extra compute at inference time, Self-Verified Distillation achieves better performance in most settings while requiring only a single inference call at test time.

Abstract

Self-Training for Language Models

Explore similar work

Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline

From Rollouts to Recipes: Self-Contained Post-Training for LLMs

Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Models