On-Policy Self-Distillation (OPSD) has recently attracted interest because it provides dense teacher guidance without a separate, often larger teacher model. However, OPSD conditions its teacher on ground-truth solutions, which limits its use when such solutions are unavailable. We propose On-Policy Self-Distillation with Self-Generated Problem Briefs (B-OPSD). Using a prompt template, the initial model or a saved student checkpoint generates a problem brief from the question alone. The prompt asks for relevant principles, applicability conditions, and likely pitfalls, but not a final answer or a complete solution. A frozen teacher uses the brief as privileged context to guide the current student's on-policy rollouts through token-level distribution matching. B-OPSD uses no ground-truth solutions during training. Results on AIME 2024, AIME 2025, and HMMT 2025 show that both B-OPSD variants achieve higher average accuracy than OPSD at each of three Qwen3 model sizes. Code is available at https://github.com/ZevineXu/B-OPSD
Figures & tables
Figure 1: Overview of B-OPSD. Student and teacher start from the same base model; the teacher stays frozen while the student is updated. The initial model generates static briefs, while a saved student checkpoint generates refreshed briefs. The current student generates its solution trajectory from the question alone. The frozen teacher receives the question and brief, evaluates the same student prefixes, and provides token-level feedback.
Training signal
Training requirements
Method
On-policy data
Dense learning signal
No separate teacher model
No answer labels
SFT
×
✓
✓
×
GRPO
✓
×
✓
×
OPD
✓
✓
×
✓
OPSD
✓
✓
✓
×
B-OPSD
✓
✓
✓
✓
Table 1: B-OPSD combines on-policy student rollouts and token-level teacher feedback without a separate teacher model, reference solutions, or verified answers during training. The SFT and GRPO rows use the reference solutions and verified final answers, respectively, from the compared training setups. The OPD row represents a common setup with a separate teacher model and no answer labels during distillation.
Figure 2: Complete problem brief generation prompt PB . The same prompt is used for static and refreshed briefs. The generator also receives the problem statement x .
Method
AIME 2024
AIME 2025
HMMT 2025
Average
Qwen3-1.7B
Base
51.5
36.7
23.1
37.1
SFT
48.4
36.3
22.7
35.8
GRPO
51.1
38.3
23.7
37.7
OPSD
54.7
43.1
25.0
40.9
B-OPSD
56.4
42.2
27.2
41.9
Table 2: Avg@12 accuracy (%) across three benchmarks. B-OPSD uses static briefs; B-OPSD ⋆ uses refreshed briefs. Average is computed from unrounded scores. Shaded rows show the B-OPSD variants, and bold marks the highest score for each model size and column.
Figure 3: Checkpoint Avg@12 accuracy (%) for Qwen3-1.7B. B-OPSD uses static briefs; B-OPSD ⋆ uses refreshed briefs. The Average panel combines scores from the same checkpoint; Table 2 averages each benchmark’s best score. The vertical axes start near the observed scores and differ across panels.
Method
AIME 2024
HMMT 2025
OPSD
54.7
25.0
B-OPSD(8B)
56.4
27.8
B-OPSD(ref)
56.4
27.5
B-OPSD
56.4
27.2
B-OPSD ⋆
57.5
28.6
Table 3: Avg@12 accuracy (%) for Qwen3-1.7B with different brief generation strategies. B-OPSD uses static briefs generated by the initial 1.7B model, and B-OPSD ⋆ uses refreshed briefs. B-OPSD(8B) uses briefs generated by Qwen3-8B, while B-OPSD(ref) uses briefs generated using the reference solutions supplied to OPSD. Shaded rows show our main variants. Bold marks the best result in each column.
Divergence
OPSD
B-OPSD
B-OPSD ⋆
AIME 2024
Forward KL
54.7
56.4
57.5
Reverse KL
51.1
52.7
54.2
JSD ( β=0.5 )
50.0
50.0
52.9
HMMT 2025
Forward KL
25.0
27.2
28.6
Table 4: Avg@12 accuracy (%) for Qwen3-1.7B with different divergences. B-OPSD uses static briefs; B-OPSD ⋆ uses refreshed briefs. Bold marks the highest score in each row.
B-OPSD
B-OPSD ⋆
Omitted section
AIME 2024
HMMT 2025
AIME 2024
HMMT 2025
None (full brief)
56.4
27.2
57.5
28.6
Targets
57.2
28.6
56.1
27.8
Relevant principles
55.8
26.7
56.7
26.7
Suggested approach
57.5
28.3
57.0
26.9
Pitfalls and checks
56.7
26.7
56.7
28.3
Table 5: Avg@12 accuracy (%) for Qwen3-1.7B when one problem brief section is omitted. B-OPSD uses static briefs, and B-OPSD ⋆ uses refreshed briefs. The shaded row uses the full brief. Bold marks the highest score within each variant and benchmark.