On-policy self-distillation (OPSD) improves reasoning by training a problem-only student on its own rollouts using dense token-level supervision from a privileged teacher that also sees a reference solution. However, standard OPSD treats the teacher distribution as a fixed target along the student's rollout and updates only the student %, although -- even though privileged conditioning does not guarantee that the teacher always provides the most appropriate target for problem-only reasoning. This one-way supervision can therefore misdirect the student when the teacher distribution is misaligned with valid student reasoning. We therefore introduce Verifier-Informed Student-to-Teacher Adaptation (VISTA), which preserves the standard OPSD student update while using outcome-verified rollouts to adapt the teacher toward the student distribution. Within each verified rollout, VISTA further restricts this adaptation to the top-k positions with the largest teacher--student KL divergence. Notably, VISTA reuses the rollout and loss function from standard OPSD, introducing no additional sampling or separate reward objective. Across AIME24, AIME25, and HMMT25 with Qwen3 models at 1.7B, 4B, and 8B, VISTA achieves the highest Avg@12 at every scale, improving over OPSD by 0.6, 0.7, and 2.1 points, respectively. These results demonstrate the value of student supervision from outcome-verified rollouts and highlight student-to-teacher adaptation as a promising direction for OPSD.
Figures & tables
Figure 1: A schematic of VISTA. Building on on-policy self-distillation (OPSD), VISTA additionally introduces a selective student-to-teacher adaptation on verified trajectories: at token positions selected by their KL divergence, the privileged teacher is revised toward the student’s distribution, so that the teacher in turn provides more benign supervision to the student.
Figure 2: Overview of the VISTA training pipeline. (1) On-policy rollout: A problem-only student samples a solution rollout for a training problem. (2) Outcome gating and position selection: The outcome verifier gates teacher updates based on rollout correctness, while position selection forms the teacher-update mask by selecting the top- k positions with the largest teacher–student KL divergence. (3) Student and teacher updates: Building on standard OPSD, VISTA retains the standard student update at every rollout position and, as explained in (2), updates the teacher toward the student’s distribution at token positions where the teacher-update mask equals one.
Figure 3: A representative case of the teacher-superiority assumption failing. From the same student rollout prefix, the student follows valid reasoning, whereas the teacher emits a reference-specific phrase. This counterexample shows that the privileged distribution is not uniformly a better token-level target along the student’s rollout.
Model
Method
AIME24
AIME25
HMMT25
Average
Qwen3-1.7B
Base
51.5
36.7
23.1
37.1
SFT
48.4
36.3
22.7
35.8
GRPO
51.1
38.3
23.7
37.7
SDPO
52.8
36.9
22.8
37.5
OPSD
57.2
43.9
29.2
43.4
VISTA
57.8
43.3
30.8
44.0
Table 1: Main results on competition-level mathematical reasoning (Avg@12, %). Each benchmark uses its peak checkpoint under the OPSD protocol, and Average is the unweighted mean over the three benchmarks. Base, SFT, GRPO, and OPSD results are from the OPSD paper ( Zhao et al. 2026 ) ; all post-training methods use the same corpus, with baseline budgets retained as reported. The best and second-best results within each model block are shown in boldface and underlined, respectively.
Adaptation rule
Rate
Avg@12 ↑
OPSD
0%
64.8
All rollouts
100%
65.8
Random gate
60%
65.6
Reverse gate
∼40%
64.5
VISTA
∼60%
66.9
Table 2: Outcome-gate ablation on Qwen3-8B. Rate denotes the fraction of rollouts routed to teacher adaptation; the random gate approximately matches VISTA’s rate, while the reverse gate uses only verifier-rejected rollouts.
Figure 4: Rollout-position selection ablation on Qwen3-8B (Avg@12). We compare OPSD with four teacher-adaptation variants: (A) the first k positions, (B) k uniformly random positions, (C) the last k positions, and (D) the top- k KL-divergence positions. VISTA uses (D) and attains the best three-benchmark average.
Figure 5: Avg@12 on AIME24 and AIME25 as the number of teacher-update positions varies.
Figure 6: Teacher-adaptation diagnostics across Qwen3-8B checkpoints: low-probability tokens ( πT(yt)≤0.05 ) on validated student trajectories and explicit reference attribution in teacher responses.
Figure 7: Privileged-teacher validation of the Qwen3-8B teacher during training.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Setting
Model scale
Qwen3-1.7B, Qwen3-4B, or Qwen3-8B.
Data pool
OpenThoughts mathematical-reasoning pool.
Student prompt
Problem only; thinking off.
Teacher prompt
Problem plus reference solution; thinking on.
Temperature
1.1 .
Top- p
0.95 .
Appendix
Table 3: Core VISTA training configuration.
Parameter
Value
Max new tokens
38,912
Thinking mode
Enabled
Temperature
1.0
Top- p
0.95
Top- k
−1 (disabled)
Min- p
0.0
Appendix
Table 4: VISTA evaluation configuration.
Figure 8: Verifier acceptance rate of Qwen3-8B student rollouts during VISTA training, centered near 60% .
Step
AIME24
AIME25
HMMT25
25
74.4
71.9
43.1
50
76.9
69.4
43.9
75
78.6
71.9
45.6
100
76.4
71.1
48.3
Appendix
Table 5: Qwen3-8B VISTA student Avg@12 (%) by checkpoint.
Figure 12: Reference-conditioned over-support: privileged-information leakage at a shared student prefix. At such a prefix, the privileged teacher may favor a continuation that depends on information unavailable to the problem-only student.
Figure 13: Reference-conditioned under-support: suppression of valid problem-only alternatives at a shared student prefix. At such a prefix, the privileged teacher may assign low probability to a locally coherent continuation available to the problem-only student.