On-policy self-distillation (OPSD) improves reasoning by training a model on its own trajectories with dense token-level supervision from a privileged copy conditioned on a reference solution. Existing methods typically use a single teacher distribution conditioned on the complete reference. We identify a limitation of this formulation: a reference is only one realization of the reasoning process. Although it provides evidence about the answer, it also encodes an ordering, phrasing, and solution path. Full-reference conditioning can therefore produce correct targets that depend excessively on information unavailable to the student at inference. We show that replacing one valid reference with another, while holding the problem and student state fixed, can substantially alter the teacher distribution, a phenomenon we call reference-induced target instability. To address this instability, we introduce Monte Carlo Privileged Distillation (MCPD), which models privileged context as a distribution over partial reference views and distills the marginal teacher distribution. MCPD evaluates the teacher under multiple views and averages the resulting next-token distributions. We show that this probability-space average is the exact target induced by minimizing the expected teacher-to-student KL divergence over privileged views. Across mathematical and scientific reasoning benchmarks, we show that MCPD produces more reference-consistent supervision and consistently improves over full-reference OPSD.
Figures & tables
Figure 1: Reference-induced target instability. Left: Distribution of the per-problem sensitivity reduction ΔS=SOPSD−SMCPD (defined in Method) over 500 problems sampled from DeepMath-103K ( He et al. 2026 ) using Qwen3-8B . For each problem, we use two valid reference solutions, generate one unprivileged student rollout, and evaluate both methods at matched prefix positions along the same trajectory. MCPD averages K=4 partial views of each reference; positive values indicate lower reference sensitivity than OPSD. Right: Qualitative example with the problem and student prefix fixed, showing that full-reference OPSD produces more reference-dependent next-token targets, whereas MCPD yields more consistent marginal targets.
Method
AIME24
AIME25
HMMT25
AVG
Qwen3-14B
Base (Instruct)
80.8
68.9
50.0
66.6
+ SFT
79.7
73.9
49.7
67.7
+ GRPO
81.9
73.6
53.0
69.5
+ OPSD
81.1
73.9
52.2
69.1
+ AVSD
80.6
73.6
50.6
68.3
Table 1: Avg@12 accuracy on math benchmarks across four backbones. Best results are in bold. AVG refers to average.
Method
Chem
Physics
Biology
Materials
AVG
Qwen3-8B
41.2
59.2
30.8
58.9
47.5
+ OPSD
44.1
60.4
35.6
58.6
49.7
+ AVSD
44.3
59.2
37.2
60.6
50.3
+ MCPD
44.3
62.2
38.0
59.0
50.9
Table 2: Avg@16 across scientific reasoning tasks using Qwen3-8B. Best results are in bold. AVG refers to average.
Method
Chem
Physics
Biology
Materials
AVG
OPSD
41.2
59.7
31.1
58.7
47.7
AVSD
41.7
59.2
32.9
59.4
48.3
MCPD
42.4
60.5
32.9
60.3
49.0
Table 3: Avg@16 comparison of math-trained Qwen3-8B models with OPSD, AVSD, and our MCPD, on SciKnowEval scientific-reasoning domains. AVG refers to average.
Figure 2: Effect of reference-solution masking on Qwen3-14B. Left: Performance changes relative to OPSD across individual benchmarks. Right: Average performance for OPSD and each masking configuration across AIME24, AIME25, HMMT25, and AMO-Bench. Mix denotes mixed masking ratios of 0% , 5% , 10% , and 30% .
Figure 3: MCPD with different masking ratios on DeepSeek-R1-Distill-Qwen-7B, with the number of views fixed at K=4 . Full results are provided in the Appendix.
Figure 4: Effect of the number of views. Left : Performance changes relative to OPSD on three benchmarks. Right : Average performance across AIME24, AIME25, HMMT25, and AMO-Bench. The masking ratio is fixed at ρ=30% .
Figure 5: MCPD with different numbers of views on DeepSeek-R1-Distill-Qwen-7B with the masking ratio fixed at ρ=40% . Full results table in Appendix.
Method
AIME25
HMMT25
OPSD w/ Sampled-token distillation
82.1
57.3
AVSD w/ Sampled-token distillation
86.7
63.3
MCPD w/ Sampled-token distillation
86.7
66.7
Table 4: Sampled-token distillation results on Qwen3-4B with a maximum student completion length of 2,048 tokens. We report Pass@8 accuracy on AIME25 and HMMT25 as in Zhao et al. (2026) .
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Mean entropy
Paired rate
OPSD
0.34
–
MCPD
0.33
75.6%
Appendix
Table 5: Full-vocabulary target entropy at the matched student states used for reference-sensitivity evaluation. The paired rate is the percentage of problems on which MCPD produces lower problem-level entropy than OPSD.
Method
Training time
OPSD
1h 26m
MCPD (ours)
1h 36m
AVSD
1h 40m
GRPO
12h 30m
Appendix
Table 6: Wall-clock training time on DeepSeek-R1-Distill-Qwen-7B using four NVIDIA A100 80GB GPUs. Times correspond to the method-specific training configurations used in our experiments.
Method
AIME24
AIME25
HMMT25
Avg.
ρ=15%
57.8
40.0
25.6
41.1
ρ=40%
55.0
43.1
25.6
41.2
ρ=50%
55.6
41.9
25.3
40.9
ρ=60%
54.2
41.7
28.6
41.5
ρ=65%
55.8
41.1
26.1
41.0
ρ=70%
55.8
41.9
25.0
40.9
Appendix
Table 7: Complete numerical results underlying Figure 3 of the main paper. We vary the masking ratio ρ on DeepSeek-R1-Distill-Qwen-7B while fixing the number of views to K=4 . The average is computed across AIME24, AIME25, and HMMT25.
Method
AIME24
AIME25
HMMT25
Avg.
K=1
54.2
40.3
26.4
40.3
K=4
55.0
43.1
25.6
41.2
K=7
55.0
42.5
26.1
41.2
K=9
59.2
40.0
25.0
41.4
Appendix
Table 8: Complete numerical results underlying Figure 5 of the main paper. We vary the number of partial reference views K on DeepSeek-R1-Distill-Qwen-7B while fixing the masking ratio to ρ=40% . The average is computed across AIME24, AIME25, and HMMT25.