Distillation is a routine step in building small language models, and a backdoored teacher can pass its backdoor to the student. We study what a student inherits when the backdoor is a rare string that triggers a fixed refusal. Keeping the trigger fixed, we vary only where it appears, at a random word boundary or appended to the instruction, and compare teachers and students on the exact trigger, six control strings and clean inputs. Across Qwen2.5, Llama 3.2 and Gemma 3, with 5% of transfer prompts triggered, randomly inserted triggers become less selective after distillation in all nine teacher-student pipelines, while appended triggers gain selectivity in Qwen and Llama and change little in Gemma. Students trained on the same prompts without a teacher show that much of this is not inherited. Llama and Gemma students learn an appended backdoor from the 150 poisoned examples alone, and only Qwen's 0.5B student needs the teacher. The data also sets where the trigger works. Gemma teachers respond to it anywhere, but their students only when it is appended. Audits should compare teacher and student on the same inputs, probe several trigger positions and include a student trained without the teacher.
Figures & tables
Figure 1: Where a student’s selectivity comes from. Selectivity S (exact-trigger firing minus mean firing on six control strings) for each backdoored teacher, its distilled student, a student trained on the teacher’s responses without its logits, and a student trained on the same prompts with no teacher. Points are implant seeds and bars are means. The text-only reference was trained for the two arms in which the teacher mattered.
Figure 2: Study design. Each teacher is implanted with the trigger at one placement (random word boundary or appended) and three implant seeds. Its transfer corpus trains three kinds of student, and every model is scored on the same instructions with the exact trigger, six control strings and no trigger, at both probe positions.
Family
Placement
Teacher S
Student S
ΔS
SD
Sign by seed
Qwen
Random
0.450
−0.026
−0.475
0.105
−−−
Qwen
Appended
0.005
0.513
+0.508
0.060
+++
Interaction I
+0.983
0.153
+++
Llama
Random
0.612
0.190
−0.422
0.313
−−−
Llama
Appended
0.294
0.910
+0.616
0.275
+++
Interaction I
+1.038
0.572
+++
Table 1: Selectivity before and after distillation. Entries average three implant seeds. ΔS is student minus teacher and SD its standard deviation across seeds. Under “Sign by seed”, + or − marks a seed whose paired 95% interval lies above or below zero, and 0 a seed whose interval includes zero. I pairs seed indices; across all nine pairings it is positive with intervals excluding zero in 27 of 27 cases (Appendix H ). All students use 5% trigger contamination.
Family
Placement
Model
Exact
Mean control
None
Clean acc.
Qwen
Random
Teacher
0.963
0.514
0.022
0.625
Student
0.161
0.186
0.120
0.459
Qwen
Appended
Teacher
0.995
0.990
0.003
0.570
Student
0.950
0.437
0.045
0.372
Llama
Random
Teacher
0.995
0.383
0.008
0.577
Student
0.474
0.284
0.081
0.444
Table 2: Response profiles and clean accuracy , averaged over three implant seeds. Exact, mean control and none are the rates at which the exact refusal is produced for the trigger, the six controls and the unmodified instruction. Clean accuracy is on 798 MMLU questions. Students use 5% contamination.
Student S
Condition
distilled
no teacher
Diff.
Sign
Qwen random
−0.026
−0.012
−0.014
000
Qwen appended
+0.513
+0.013
+0.499
+++
Llama random
+0.190
+0.030
+0.160
+++
Llama appended
+0.910
+0.886
+0.025
+ 00
Gemma random
+0.537
+0.098
+0.439
+++
Table 3: Distilled and no-teacher students (means over three implant seeds). Diff. is distilled minus no-teacher selectivity, and Sign marks each seed as in Table 1 .
Figure 3: Exact-trigger firing of models trained with an appended trigger, probed with the trigger inside the instruction or appended. Bars are means over implant seeds and points are individual seeds. Gemma teachers respond to the trigger at interior positions, but none of their students do.
Figure 4: Crossed-placement evaluation for Qwen and Llama. (a) Exact-trigger firing of each training arm’s teacher and 5% student with the trigger probed at an interior word slot or appended, averaged over three implant seeds. (b) Change in selectivity from teacher to student under the probe that matches the training placement (filled) and the other probe (hollow), with paired bootstrap intervals.
Figure 5: Selectivity of Qwen students by contamination of the transfer corpus (three seeds, ± one standard deviation). The 2%, 5% and 10% doses are 60, 150 and 300 triggered prompts.
Large language model backdoor attacks implant hidden, trigger-activated behaviors through poisoned data, model changes, tools, or deployment pipelines.
On-policy distillation (OPD) is emerging as an important paradigm for the post training of large language models, improving student models by leveraging supervision generated online by teacher models. However, whether this process also transfers latent security risks from teacher models to student models remains insufficiently studied. In this work, we reveal a new security risk in which an attacker first implants a conditional backdoor into the teacher model through supervised fine tuning (SFT), after which OPD can transfer the backdoor to the student model with only an extremely small proportion of trigger prompt exposure. More concerningly, we find that such backdoor transfer is substantially stronger under OPD than under conventional distillation methods, suggesting that OPD may provide an efficient channel for propagating backdoors across models. Our mechanistic analysis shows that the backdoor signal is primarily transmitted through anomalous differences in the conditional probabilities produced by the teacher model for triggered and clean inputs, and is gradually internalized into the internal representations of the student model during training. Furthermore, we investigate how to suppress such anomalous conditional probability signals at the source of distillation supervision and, based on this analysis, propose an OPD training method tailored to this conditional backdoor propagation setting to mitigate the risk of backdoor transfer induced by untrusted teacher supervision.
Zane Fu, Yanzhao Zheng, Ziting Qin +8
Alibaba Group · Wuhan University of Technology · Institute of Automation, Chinese Academy of Sciences +3
Knowledge distillation is widely used to transfer capabilities from teacher models to student models, but recent work shows that it can also transfer unintended teacher properties. We study whether different hidden teacher properties survive the same distillation channels. Specifically, we compare two properties: benchmark-associated knowledge and refusal backdoors, and show that distillation treats them very differently. Teachers fine-tuned on benchmark data successfully pass that knowledge to students through all four distillation channels we test, including when students are trained only on teacher-generated number sequences containing no natural language or domain content whatsoever. Refusal backdoors tell the opposite story: backdoored teachers achieve near-perfect triggered refusal, yet sequence-distilled students inherit essentially none of this behaviour. Logit-based distillation transfers it substantially, reaching up to 85.5% triggered refusal in students that saw zero backdoor examples. Distillation is therefore selective, and the selection criterion is not what data the student sees, but what structure the teacher's property has. The code is available at https://github.com/MJonibek/kd-leakage.
Knowledge distillation is a widely adopted technique for transferring capabilities from LLMs to smaller, more efficient student models. However, unauthorized use of knowledge distillation takes unfair advantage of the considerable effort and cost put into developing frontier models. We investigate methods for modifying teacher-generated reasoning traces to achieve two objectives that deter unauthorized distillation: (1) anti-distillation, or degrading the training usefulness of query responses, and (2) API watermarking, which embeds verifiable signatures in student models. We introduce several approaches for dynamically rewriting a teacher's reasoning outputs while preserving answer correctness and semantic coherence. Two of these leverage the rewriting capabilities of LLMs, while others use gradient-based techniques. Our experiments show that a simple instruction-based rewriting approach achieves a strong anti-distillation effect while maintaining or even improving teacher performance. Furthermore, we show that our rewriting approach also enables embedding watermarks that can be reliably detected with essentially no false alarms. Our code is available at https://github.com/xhOwenMa/trace-rewriting.