Distillation is common in LLM post-training, where on-policy knowledge distillation (OPKD) uses student-generated trajectories to prepare the student for downstream RL. At each state, the student matches a next-token distribution supplied by the teacher. As the rollout enters states the teacher would not visit, the teacher-student distribution gap can accumulate. In tool use, this gap becomes consequential because student-written calls execute before supervision and their observations shape later prefixes. Proposer-verifier generation addresses this drift by letting the teacher decide which student-proposed text is retained during generation. Existing formulations govern text but leave tool execution outside their scope. We propose Persistent Teacher Anchoring (PTA), a student-induced but teacher-committed rollout construction. PTA retains chunk-level verification and adds turn-level commitment, allowing a call to reach the environment only after the teacher has verified the entire turn. Treating verified chunks as atomic generation units, we introduce persistent lookahead, which fills idle rollout capacity by advancing future samples and carrying unfinished ones across student updates under the fixed verifier. Across Search-R1-style retrieval and DeepEyes-style perception RL, applying PTA before downstream RL improves macro best@4 by 2.5 and 2.8 points over OPKD under the same downstream RL budget, while lookahead improves throughput by 24%.
Figures & tables
Method
Proposal unit
Rejection trigger
Stage
Decision reaches
Speculative decoding ( Leviathan et al., 2023 )
chunk
probability ratio (lossless)
decoding
text
BiLD ( Kim et al., 2023 )
token
confidence or disagreement
decoding
text
RSD ( Liao et al., 2025 )
reasoning step
step reward below threshold
decoding
text
SWITCH ( Koo et al., 2025 )
token
teacher-student divergence
training
text
SKD ( Xu et al., 2025 )
chunk
outside teacher top- K
training
text
SCoRe ( Lyu et al., 2026 )
full trajectory
teacher-identified error
training
text
Table 1: Proposer-verifier generation methods. A weaker model proposes and a stronger model decides what is kept. The text-only formulations define no execution event, which is what PTA supplies (Section 3.2 ).
Figure 1: Overview of the PTA rollout. (a) SKD applied directly to a tool runtime, without a separate commitment rule: unverified tool calls reach the environment, and returned observations fall inside the verification span. (b) PTA separates chunk-level verification from turn-level commitment, so that tool execution is triggered only after a committed turn.
Figure 2: Schematic of synchronous and lookahead scheduling over the same wall-clock interval. Ticks mark completion of the current-step batch. In (b), one lookahead sample is promoted before the boundary, another becomes carryover and resumes after the actor update, and a fresh sample fills the remaining slot in the next batch. The carryover cut occurs only at an atomic state boundary.
Table 4
VStar
HRBench4K
HRBench8K
Setting
mean@4
best@4
mean@4
best@4
mean@4
best@4
OPKD
57.85
78.53
61.00
64.00
54.63
58.00
PTA
61.13
81.15
64.00
68.50
57.88
65.00
PTA without tool-call replacement
50.00
70.68
55.50
58.00
48.13
50.50
Table 4: Pre-RL alignment checkpoints in the perception-mediated environment.
Figure 3: Top-N stability diagnostics over an early training window.
Setting
Promoted
Throughput
synchronous
0.0
0.519
lookahead
20.2
0.644
Table 5: Lookahead scheduling efficiency.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Metric
Base
Direct RL
OPKD + RL
PTA + RL
VStar results
VStar mean@4
59.03
61.65
59.55
64.53
VStar best@4
75.39
73.30
80.10
78.01
V-DA mean@4
57.39
60.00
58.91
65.65
V-DA best@4
71.30
69.57
74.78
73.91
V-RP mean@4
61.51
64.14
60.53
62.83
Appendix
Table 6: Full perception-mediated reasoning results.
Benchmark
n
Δ
z
p
NQ
3,610
+0.41
0.38
0.70
PopQA
14,267
+1.61
2.86
0.004
HotpotQA
7,405
−0.15
−0.22
0.83
Musique
2,417
+2.73
3.23
0.001
VStar
191
+4.98
1.00
0.32
HRBench4K
800
+4.38
1.83
0.067
Appendix
Table 7: Benchmark-level mean@4 differences between PTA + RL and OPKD + RL, with the number of evaluation questions n . Δ is in percentage points.
Statistic
Retr.
Perc.
Committed assistant tokens retaining the student proposal
90.5
88.4
Tokens within tool-call spans retaining the student proposal
99.5
93.0
Tool calls containing at least one replacement
8.9
84.7
Appendix
Table 8: Verifier acceptance and correction rates, in percent, aggregated over the training logs of the alignment runs used in the paper.
Figure 4: Verifier behavior over the first 150 training steps. (a) Fraction of student-proposed tokens that the verifier keeps. (b) Fraction of chunks in which at least one token is replaced. (c) Number of chunks accumulated per sample, each of which corresponds to one call to the teacher verifier in Algorithm 1 . Light lines are per-step values and solid lines are a nine-step moving average.