On-policy distillation trains a student on its own rollouts, so states the teacher would reach but the student does not are seldom supervised. We introduce CoRD (coupled-rollout distillation), which keeps the on-policy loss and moves only the distribution of training states toward the teacher, drawing rollouts from a geometric interpolation of student and teacher through the maximal coupling of speculative decoding. The coupling keeps the student's token wherever the two distributions agree and otherwise substitutes a token from the part of the target the student underweights, so interventions are as rare as any exact sampler allows and recede as the student learns. Across six base students from two model families (Qwen3 and Qwen3.5, 0.6B to 4B), CoRD raises average pass@1 over on-policy distillation by 0.9 to 2.0 points and pass@8 by 1.2 to 3.4 points on five public math benchmarks at equal steps and data. On frozen teacher-reached states, a 4B student trained with CoRD reaches in 16 steps a gap closure that on-policy distillation does not reach in 80, and substitution rules intervening at least as often in other directions fall short of CoRD.
Figures & tables
Figure 1: Overview of CoRD . (i) Left (OPD’s blind spot): On-policy distillation trains only on student rollouts. Teacher-preferred reasoning paths that the student rarely or never visits therefore receive little or no supervision. (ii) Right (our coupled sampler): CoRD keeps the same training loss and budget, but changes the rollout sampler. By coupling the student with a teacher-leaning target qα , it preserves agreeing tokens and replaces disagreements with underweighted target tokens. This maximal coupling steers rollouts into teacher-preferred states missed by OPD with the minimum possible intervention rate, minimally perturbing the student’s original trajectory.
Figure 2: One-step coupled sampling. On a three-token vocabulary with α=0.6 , student proposals are kept whenever possible and replaced only when necessary, so the emitted token follows qα exactly.
Student
Method
Pass@1
Pass@8
MATH500
AIME24
AIME25
AMC23
GSM8K
Avg.
MATH500
AIME24
AIME25
AMC23
GSM8K
Avg.
Qwen3-Base Series (Teacher Size: 8B)
0.6B
Untrained
12.7
0.4
0.2
3.1
24.5
8.2
42.0
3.3
1.7
18.6
67.5
26.6
OPD
15.4
0.2
0.0
5.5
30.0
10.2
46.9
1.7
0.0
28.1
71.8
29.7
CoRD
18.3
0.4
0.2
6.3
35.8
12.2
50.9
3.3
1.7
32.4
77.4
33.1
1.7B
Untrained
24.1
0.4
0.2
7.2
43.7
15.1
57.1
3.3
1.7
35.0
85.0
36.4
Table 1: Main results on public math benchmarks. For all six students, CoRD has higher average pass@1 and pass@8 than OPD in this evaluation. Results use 16 generated samples per problem (percent). Pass@ k uses the unbiased estimator, and Avg. is the unweighted mean across the five benchmarks, computed from unrounded benchmark scores; it may differ slightly from the mean of the displayed values due to rounding. All distilled models train for 80 steps except “OPD (160 steps)” (Section 4.4 ); CoRD uses α=0.6 and SKD the published top-25 rule. Bold indicates the highest value among distilled models for each student and column, including ties. Pass@16 results for the three Qwen3 students are reported in Appendix Table 6 .
Student
Untrained student solved k of 16
k=0
1–3
4–7
8–16
Qwen3-0.6B
+5.3
+6.2
+0.5
0.0
Qwen3-1.7B
+3.0
+2.8
+0.7
0.0
Qwen3-4B
+4.2
+4.2
- 0.3
0.0
Qwen3.5-0.8B
+0.8
+1.5
+1.2
+1.3
Qwen3.5-2B
- 0.5
+2.9
+0.7
- 0.2
Table 2: Pass@16 gains by problem difficulty. The table reports pass@16 gains of CoRD over OPD (percentage points) after 80 steps on 1,919 public-benchmark problems, grouped by how many of 16 samples the untrained student solved. Bold marks each student’s largest gain; for all six students, these occur at k≤3 . Gaps are computed before rounding from per-problem estimates.
Figure 3: Visited-state divergence and held-out accuracy. CoRD maintains higher visited-state reverse KL and ends with higher held-out pass@1 for all six students in Table 1 . Top: reverse KL to the teacher at states visited during training (5-step moving average). Bottom: pass@1 on about 200 held-out problems from the training distribution, evaluated at steps 0, 40 and 80.
Replacement Rule
Rate 1.7B (%)
1.7B (Instruct)
4B-Base
Teach.
Stud.
Teach.
Stud.
none (OPD)
0.0
37.4
52.0
15.0
− 0.5
residual ( CoRD )
7.5
44.4
52.0
21.7
11.9
teacher sample, random position
7.9
39.1
51.3
14.2
− 1.4
average-distribution token
12.2
29.2
48.4
13.7
− 10.4
uniform token
15.8
28.3
47.3
13.9
− 9.8
Table 3: Gap closure under replacement rules. Gap closure (%) on frozen teacher-reached and student-reached states after 40 steps, for different replacement rules at α=0.6 . The random-position rule is a DAgger-style control whose average rate is close to CoRD ’s. Rate is the realized fraction of intervened positions at 1.7B. The residual draw is the only rule that clearly improves teacher-side closure for both students.
Figure 4: Teacher-side closure and replacement rate. For the 4B-Base student and Qwen3-8B-Base teacher, CoRD reaches in 16 steps the closure on teacher-reached states that OPD fails to reach in 80. Left and middle: closure of the reverse KL to the teacher on frozen teacher-reached and student-reached states, measured every 8 steps. Right: the realized replacement rate, which falls slowly as the student approaches the target.
α
Rate (%)
Teach. Closure
4B, Public
1.7B
4B
80 Steps
0 (OPD)
0.0
37.4
15.0
20.7 / 50.3
0.3
3.6
40.6
18.1
20.7 / 51.6
0.6 ( CoRD )
7.5
44.4
21.7
22.3 / 54.9
1
15.6
47.4
22.9
22.6 / 52.2
Table 4: Closure and public accuracy across α . Closure on teacher-reached states grows with α , and every α>0 improves public pass@16 over OPD. Rate is the realized replacement rate at 1.7B; closure (%) is measured on frozen teacher-reached states after 40 steps. Accuracy is pass@1 / pass@16 (%) for the 4B-Base student, averaged over the five public benchmarks after 80 steps.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Run group
Setting
Public benchmarks
Analysis on MATH
Qwen3-14B teacher
(Table 1 )
(Section 5 )
(Section 5 )
Training
Training steps
80 (160 for “OPD, 160 steps”)
40
20
Prompts per step
256
256
64
Max. new tokens per rollout
320
320
224
Appendix
Table 5: Training and evaluation settings per run group. The analysis and Qwen3-14B-teacher runs are measured by closure on frozen state sets (Section 5 ).
Figure 5: The two-shot completion prompt used for training and for every evaluation; {question} is replaced by the problem, and generation stops at the next Problem: .
Student
Method
MATH500
AIME24
AIME25
AMC23
GSM8K
Avg.
Qwen3-0.6B-Base
Untrained
53.8
6.7
3.3
30.0
79.5
34.7
OPD
57.6
3.3
0.0
45.0
81.6
37.5
CoRD
62.6
6.7
3.3
45.0
85.1
40.5
Qwen3-1.7B-Base
Untrained
66.4
6.7
3.3
50.0
91.2
43.5
OPD
66.6
6.7
6.7
65.0
92.4
47.5
CoRD
71.8
10.0
0.0
62.5
92.6
47.4
Appendix
Table 6: Pass@16 (%) on the five public benchmarks for the three Qwen3 base students. Each entry is computed from 16 samples per problem, with Avg. the unweighted mean across benchmarks, computed from unrounded benchmark scores; it may differ slightly from the mean of the displayed values due to rounding. The Qwen3-1.7B average and some benchmark scores are lower with CoRD than with OPD, despite higher average pass@1 and pass@8.
Symbol
Meaning
pS,pT
student, teacher next-token distributions
qα
tempered target of Eq. ( 2 )
Zα
normalizer ∑upS(u)1−αpT(u)α
ℓ(v)
log ratio logpT(v)−logpS(v)
(x)+
max(x,0)
TV(p,q)
21∑v∣p(v)−q(v)∣
Appendix
Table 7: Notation for Appendix D .
Token
pS
pT
pS0.4pT0.6
qα
Keep
Residual
a
0.6
0.2
0.310
0.356
0.59
0
b
0.3
0.3
0.300
0.344
1
0.044
c
0.1
0.5
0.263
0.301
1
0.201
Appendix
Table 8: Every quantity of the worked example of Figure 2 ( α=0.6 , Zα=0.873 ). Keep is min(1,qα/pS) ; Residual is (qα−pS)+ .
Language model distillation transfers a larger model’s capabilities to smaller or specialized models using outputs, probabilities, or internal representations.