GATES: Self-Distillation under Privileged Context with Consensus Gating

Alex Stein; Furong Huang; Tom Goldstein

cs.LGFeb 24, 2026

GATES: Self-Distillation under Privileged Context with Consensus Gating

Authors: Alex Stein, Furong Huang, Tom Goldstein

Organizations: University of Maryland, College Park

Abstract

We study self-distillation in settings where supervision is unreliable: there are no ground truth labels, verifiable rewards, or external graders to evaluate answers. We focus on document-grounded question answering with asymmetric context, where a single model serves as both tutor (with access to a relevant source document during training) and student (answering from the question alone at test time). Rather than assuming tutor correctness, we derive supervision online from tutor consensus by sampling multiple document-grounded reasoning traces and using agreement to gate learning. Conditioned on this reliability signal, we distill knowledge through full tutor reasoning trajectories (not just final answers), providing a dense and stable learning signal. Empirically, this consensus-gated trajectory distillation substantially improves transfer to the document-free student. Held-out in-domain accuracy under asymmetric evaluation improves from 46.0% to 62.0%, and average (maj@8) accuracy on public document-free math benchmarks improves from 20.2% to 35.4%.

Figures & tables

Figure 1: Overview of GATES. A tutor model, given a privileged document and question, generates multiple reasoning rollouts. A consensus gate filters rollouts based on answer agreement, discarding trajectories with minority answers. The surviving rollouts are used to train a student model—which receives only the question—via distillation loss, transferring the tutor’s privileged reasoning without requiring ground-truth labels. The tutor and student share the same underlying model, differing only in whether the privileged document is included in the input context.

Figure 2: Off-policy distillation in GATES. A single model $\pi_{\theta}$ operates in two roles under asymmetric context: as a tutor conditioned on both the source document $d$ and question $q$ , and as a student conditioned on $q$ alone. (A) The tutor generates $k$ independent reasoning rollouts per question. (B) A question-level consensus gate labels the question as reliable if sufficiently many rollouts agree on the same answer; a second, rollout-level gate retains only trajectories that match the consensus. (C) Eligible tutor trajectories provide dense token-level supervision to the document-free student via trajectory distillation (Eq. 1). Unreliable questions are skipped entirely, preventing self-reinforcement collapse.

Figure 3: On-policy distillation in GATES. As in the off-policy variant (Figure 2 ), a single model $\pi_{\theta}$ serves as both tutor and student under asymmetric context. (A) Both roles generate $k$ rollouts in parallel: tutor rollouts establish consensus, while student rollouts provide the on-policy training signal. (B) A question-level consensus gate determines reliability; unlike the off-policy setting there is no trajectory-level filter—when consensus is strong, all student rollouts pass through. (C) The tutor scores each token of the student’s own rollouts by computing log-probabilities under document context. The per-token advantage $A_{t}=\mathrm{clip}(\log\pi_{\mathrm{tutor}}-\log\pi_{\mathrm{student}},\,[-a,a])$ upweights tokens where the tutor assigns higher probability, encouraging document-grounded reasoning while remaining on-policy (Eq. 3). Unreliable questions contribute zero loss.

Split	# Questions
Train	551
Held-out eval	50

Table 1: Dataset sizes for the fixed-challenger splits.

Figure 4: Main results comparing GATES against baselines. (a) Accuracy (%) on four document-free math benchmarks (maj@8 decoding). (b) Student accuracy on the held-out asymmetric evaluation (50 questions, greedy decoding). GATES yields the best student accuracy (62%), improving 16 percentage points over the pretrained base model. See Appendix A.5 for tutor accuracy, greedy decoding, and coverage results.

Figure 5: Ablation results varying loss weights with

\lambda_{\text{KL}}=0.02

fixed throughout. (a) Accuracy (%) on document-free math benchmarks (maj@8 decoding). GATES (ours) is the canonical configuration.

-

Gate uses the same loss weights but removes consensus gating, resulting in a

4.3

pp benchmark drop despite identical loss configuration. Adding oracle loss provides no meaningful improvement (35.7 vs. 35.4), confirming that consensus gating alone is sufficient without verified correctness labels. (b) Student accuracy on the held-out asymmetric evaluation (greedy decoding). Off-policy distillation remains the dominant contributor: configurations without it show the largest drops in student accuracy. Tutor accuracy is reported in Appendix A.5 .

Figure 6: Benchmark average accuracy (%) under fixed vs. adaptive challenger training (maj@8 decoding). Fixed, GATES and Adaptive, GATES use the canonical configuration (

\lambda_{\text{off}}\!=\!1.0

\lambda_{\text{on}}\!=\!0.1

\lambda_{\text{cons}}\!=\!0.0

) with a static or adaptive challenger, respectively. The remaining adaptive variants add the oracle loss (

\lambda_{\text{cons}}\!=\!1.0

) under different on/off-policy weightings. SPICE uses an adaptive challenger by design and is included for reference under a matched update budget.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Batch size ( $n$ questions/step)	32
Tutor rollouts per question ( $k$ )	8
Student rollouts per question ( $k$ )	8
Training temperature (tutor/student)	0.5
Training top- $p$ (tutor/student)	1.0
Training top- $k$ (tutor/student)	$-1$

Appendix

Table 6: Training hyperparameters used unless otherwise specified.

Hyperparameter	Value
Evaluation temperature (benchmarks, greedy)	0
Evaluation temperature (benchmarks, maj@8)	0.6
Evaluation samples (maj@8 / pass@8)	8
Eval max completion tokens	1024
Correctness model for in-domain eval	Qwen2.5-32B-Instruct (consensus)
Correctness rollouts (in-domain eval)	8

Appendix

Table 7: Evaluation hyperparameters used in this submission.

Figure 9: Accuracy (%) under greedy decoding on document-free math benchmarks. GATES and Tutor-Trajectory SFT achieve comparable greedy performance, but GATES leads by 3.1 percentage points under maj@8 decoding (Figure 4 a).

Figure 10: Accuracy (%) under coverage (pass@8) on document-free math benchmarks. Gaps between methods are smaller than under greedy or maj@8 decoding, consistent with pass@8 measuring an upper bound on model capability.

Figure 11: Student accuracy (solid) and tutor accuracy (hatched) on the held-out asymmetric evaluation (50 questions, greedy decoding). Tutor-Trajectory SFT achieves the highest tutor accuracy (74%) but transfers poorly to the student (54%), illustrating that training on unfiltered tutor rollouts inflates tutor performance without improving student internalization. GATES yields the best student accuracy (62%) with a narrower tutor–student gap, indicating more effective knowledge transfer via consensus gating.

Figure 12: Ablation results: student accuracy (solid) and tutor accuracy (hatched) on the held-out asymmetric evaluation (greedy decoding). Off-policy distillation remains the dominant contributor to student accuracy. Notably,

-

Gate achieves the highest tutor accuracy (72%) but only 54% student accuracy, mirroring the pattern observed with Tutor-Trajectory SFT and confirming that ungated training inflates tutor performance without improving student transfer.

Knowledge Distillation

Knowledge distillation transfers useful behavior or representations from larger or specialized models into more compact or adaptable learners.

See all 819 papers

Activation-Conditioned Self-Distillation
Sep 29, 2026Zhexi Lu, Subhajit Chaudhury, Tejaswini Pedapati +2Numerical Reasoning in Language Models On-Policy Self-Distillation
Rensselaer Polytechnic Institute · IBM Research
Teach Yourself Where to Look: On-Policy Attention Self-Distillation for Reasoning
Sep 27, 2026Safaeid Hossain Arib, Rabeya Akter, Ismam Nur Swapnil +3Numerical Reasoning in Language Models Attention-Guided Distillation
ACI PLC, Bangladesh · University of Dhaka, Bangladesh · BRAC University, Bangladesh +2
Consensus as Privileged Context for Label-Free Self-Distillation
Jul 15, 2026John Gkountouras, Josip Jukić, Ivan TitovSelf-Training for Language Models Language Model Distillation
1ILLC, University of Amsterdam · 2ILCC, University of Edinburgh

Sep 29, 2026J/K move · Enter open · S save

Activation-Conditioned Self-Distillation

Zhexi Lu, Subhajit Chaudhury, Tejaswini Pedapati, Keerthiram Murugesan+1

Rensselaer Polytechnic Institute · IBM Research

Numerical Reasoning in Language Models On-Policy Self-Distillation Language Model Distillation Activation Steering

Open paper AskPDF arXiv

On-policy self-distillation uses a model as its own teacher to provide dense supervision for reasoning, often through reference-solution conditioning. Providing privileged information does not by itself ensure effective token-level supervision throughout long responses. We introduce Activation-Conditioned Self-Distillation (ACSD), which extracts a steering vector by contrasting activations of self-generated trajectories that reach verified correct answers within a generation budget with those of all remaining trajectories. A frozen copy of the base model applies this vector at each prediction position, and the student learns from its next-token distributions on student-generated prefixes. Outcome verification is used for direction construction and calibration; distillation requires neither problem-specific reference text nor teacher parameter updates. The distilled student is used alone at inference. On each of five models, ACSD achieves the highest mean accuracy over four mathematical benchmarks among the evaluated methods. On DeepSeek-R1-0528-Qwen3-8B, mean mathematical accuracy reaches 71.9% and LiveCodeBench v6 pass@12 reaches 70.9%, compared with 69.0% and 66.3% for the reference-conditioned OPSD baseline. Contrasts among correct trajectories also support distillation, and extracted directions can be reused across mathematical training datasets. On fixed student trajectories, ACSD maintains more stable late-position logit-update magnitudes than OPSD.

Abstract

Knowledge Distillation

Explore similar work

Activation-Conditioned Self-Distillation

Teach Yourself Where to Look: On-Policy Attention Self-Distillation for Reasoning

Consensus as Privileged Context for Label-Free Self-Distillation