cs.CLSep 2, 2026

On Mitigation of Subliminal Learning in Large Language Models

Authors: Atsushi Yanagisawa, Brendan Gho, Rajendran Ramesh Babu Manoj Narender, Kevin Zhu, Madhur Panwar, Antonio Mari

Organizations: Algoverse AI Research

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unrelated to those traits, a phenomenon known as subliminal learning. Although recent work has established this effect, its training dynamics and mitigation remain underexplored. We study subliminal learning in open-weight language models ranging from 1.5B to 8B parameters, covering the Qwen, Gemma, and Llama families in number-sequence and chain-of-thought settings. Rather than evaluating only final models, we track trait-related probabilities throughout fine-tuning and find that subliminal acquisition can be highly non-monotonic, with transient spikes, reversals, and trait-specific failures of transfer. We then introduce liminal training, an annealed KL-regularized fine-tuning method that constrains early drift from the base model. Across our experiments, liminal training substantially reduces subliminal trait acquisition while largely preserving task gains, outperforming paraphrasing and layer freezing as mitigation strategies. The effect also extends beyond animal preferences: in a French-language response-style experiment, liminal training suppresses language transfer while retaining much of the GSM8K improvement. Finally, we show that KL timing matters: early regularization is more effective than late regularization, and sweeping the regularization strength reveals an empirical trade-off between task learning and trait suppression.

Figures & tables

BaseNFTDragonCatEagleWolfPandaOtter
  PFTLFTPFTLFTPFTLFTPFTLFTPFTLFTPFTLFT
30.061.257.657.258.055.356.155.356.956.363.256.063.756.9
Table 1: GSM8K accuracy (%) for Qwen2.5-1.5B-Instruct under each CoT fine-tuning condition, corresponding to the trait-probability trajectories shown in Figure 1 . Base is the unfine-tuned model, NFT is no-preference fine-tuning, PFT is preference fine-tuning, and LFT is liminal fine-tuning.
Figure 1: Example CoT result for Qwen2.5-1.5B-Instruct fine-tuned on animal-biased chain-of-thought datasets. Liminal training suppresses subliminal trait acquisition while largely retaining GSM8K accuracy (Table 1 ). Similar patterns hold across other models and animals; full results are reported in Appendix E .
ModelMethodSeq. ΔP(animal)\Delta P(\text{animal}) (%)CoT ΔP(animal)\Delta P(\text{animal}) (%)CoT Δ\Delta GSM8K (pp)
Qwen2.5-1.5BNFT+0.1±0.1+0.1{\pm}0.1−0.12±0.46-0.12{\pm}0.46+31.2±1.2+31.2{\pm}1.2
PFT−0.2±1.1-0.2{\pm}1.1+1.07±0.92+1.07{\pm}0.92+28.7±1.1+28.7{\pm}1.1
LFT+0.1±0.1+0.1{\pm}0.1+0.03±0.11+0.03{\pm}0.11+26.1±0.4+26.1{\pm}0.4
Qwen2.5-3BNFT+0.0±0.1+0.0{\pm}0.1−0.75±1.23-0.75{\pm}1.23+18.1±1.6+18.1{\pm}1.6
PFT+3.4±5.0+3.4{\pm}5.0+1.03±3.19+1.03{\pm}3.19+14.9±3.6+14.9{\pm}3.6
LFT+0.1±0.3+0.1{\pm}0.3−0.11±0.57-0.11{\pm}0.57+18.1±2.2+18.1{\pm}2.2
Table 2: Summary of the main animal-preference experiments. Columns report average change from baseline: ΔP(animal)\Delta P(\text{animal}) (%) for the sequences and CoT trait settings, and Δ\Delta GSM8K accuracy (pp) for the CoT benchmark setting. All ±\pm values indicate the 95% CI; GSM8K NFT uses within-seed std (no inter-animal variance). Full per-animal results are in Appendix E .
Figure 2: Alternative mitigation strategies. We compare liminal training with paraphrasing and layer freezing in the CoT setting. Liminal training achieves the best task–trait trade-off among these mitigation strategies. Results for individual configurations are shown in Appendix D .
Figure 3: (a) Constant-KL strength traces a smooth task–trait trade-off : relaxing the KL constraint first recovers much of the GSM8K accuracy before substantially increasing trait acquisition, while early-weighted schedules such as liminal training and early anchor lie on or near the Pareto frontier. (b) Initial trait probability predicts instability under unbiased fine-tuning : traits with higher initial P(animal)P(\text{animal}) show larger variation across training steps, even under the no-preference control.
MetricBasePFTLFT (Varying Schedule)
   DefaultNeg. AnnealPos. AnnealEarly AnchorEnd AnchorConstant KL
P(Fr.)P(\text{Fr.}) (%)5.574.6±0.274.6_{\pm 0.2}5.5±0.15.5_{\pm 0.1}5.5±0.25.5_{\pm 0.2}4.3±0.34.3_{\pm 0.3}7.3±0.67.3_{\pm 0.6}39.1±0.739.1_{\pm 0.7}4.3±0.14.3_{\pm 0.1}
GSM8K (%)29.359.1±0.759.1_{\pm 0.7}51.8±0.551.8_{\pm 0.5}53.0±0.653.0_{\pm 0.6}41.2±1.041.2_{\pm 1.0}56.4±0.456.4_{\pm 0.4}42.0±0.742.0_{\pm 0.7}38.2±0.938.2_{\pm 0.9}
Table 3: French language experiment results for Qwen2.5-1.5B-Instruct in the CoT setting. P(French)P(\text{French}) is the mean fraction of responses in French across training steps; GSM8K reports absolute accuracy (%). All columns after PFT use LFT with different KL schedules. Subscript errors denote standard error of the mean across 3 seeds.
Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

ParameterValue
Learning rate2×10−42\times 10^{-4}
Per-device batch size22
Gradient accumulation steps3
Effective batch size66
Epochs3
Max gradient norm1.0
Appendix
Table 4: LoRA fine-tuning hyperparameters for the number-sequence setting.
ParameterValue
Learning rate2×10−42\times 10^{-4}
Per-device batch size8
Gradient accumulation steps2
Effective batch size16
Epochs3
Max gradient norm1.0
Appendix
Table 5: LoRA fine-tuning hyperparameters for the chain-of-thought setting.
Figure 4: French language probe: KL schedule ablations. P(French)P(\text{French}) across training steps for Qwen2.5-1.5B-Instruct under PFT and six KL-regularized conditions. Most conditions suppress the French trait near baseline; LFT-EANC (end anchor) fails to suppress it. NA: neg. anneal; PA: pos. anneal; ANC: early anchor; EANC: end anchor; CKL: constant KL. Error bars denote standard error of the mean across 3 seeds.
QuestionZeke’s baseball team has 7 more players than Carlton’s. If Carlton’s team has 13 players, how many players are there in both teams combined?
PFT<think> Carlton’s team has 13 players. Zeke’s team has 7 more players than Carlton’s, donc elle a 13 + 7 = 20 joueurs. Le nombre total de joueurs dans les deux équipes est 13 + 20 = 33. </think> <answer>33</answer>
LFT<think> Let CC represent the number of players on Carlton’s team and ZZ represent the number of players on Zeke’s team. C=13C=13 , Z=C+7=20Z=C+7=20 . Total: C+Z=33C+Z=33 . </think> <answer>33</answer>
Appendix
Table 6: PFT vs. LFT response to the same question (seed 43). Both answers are correct (33). PFT inherits the French reasoning style from the teacher; LFT does not.
Figure 5: Liminal training versus paraphrasing across tested animal-model configurations in the CoT setting. Each point reports mean P(animal)P(\text{animal}) across training steps and final GSM8K accuracy. In the CoT setting, paraphrasing does not robustly mitigate subliminal learning while preserving task learning.
Figure 6: Liminal training versus layer freezing across tested animal-model configurations. Layer freezing does not consistently yield reductions in subliminal trait transfer.
ModelΔ\Delta Pref.DragonCatDogEagleWolfPandaPenguinOwlOtterElephantOxRavenAvg.
Qwen2.5-1.5BΔ\Delta NFT+0.5+0.1-0.3+0.1+0.30.00.00.00.0+0.20.00.0+0.1
Δ\Delta PFT+6.9-8.0-5.9+0.5+2.6+0.40.0+0.10.0+1.0+0.1+0.2-0.2
Δ\Delta LFT+0.3-0.1-0.5+0.1+0.40.00.00.00.0+0.70.00.0+0.1
Qwen2.5-3BΔ\Delta NFT+0.20.0+0.60.00.00.00.00.00.00.00.00.00.0
Δ\Delta PFT+9.6-2.3-13.5+51.2+5.0+1.1+4.6+0.4+4.8-21.5+1.1+0.4+3.4
Δ\Delta LFT-0.2-0.3-2.0+0.1+0.3+0.10.0+0.10.0+2.6+0.10.0+0.1
Appendix
Table 7: Change from baseline in mean trait probability (%) after normal fine-tuning ( Δ\Delta NFT), preference fine-tuning ( Δ\Delta PFT), and liminal fine-tuning ( Δ\Delta LFT) per model and animal in the number sequence setting. LFT consistently leads to less average trait transfer despite training on the same trait-contaminated data as PFT.
ModelΔ\Delta Pref.DragonCatDogEagleWolfPandaRavenOtterAvg
Qwen2.5-1.5BΔ\Delta NFT+0.47+0.26-3.16+0.17+1.36+0.02-0.02-0.09-0.12
Δ\Delta PFT+4.08+0.25-2.88+1.36+5.43+0.22+0.02+0.06+1.07
Δ\Delta LFT-0.16+0.13-0.52+0.12+0.52+0.15-0.02-0.02+0.03
Qwen2.5-3BΔ\Delta NFT+1.76-2.09-8.85+0.73+1.50+0.90+0.02+0.02-0.75
Δ\Delta PFT+1.03+1.83-17.73+5.02+14.97+2.89+0.17+0.03+1.03
Δ\Delta LFT-0.33+0.21-3.46+0.03+2.51+0.180.000.00-0.11
Appendix
Table 8: Change from baseline in mean trait probability (%) after normal fine-tuning ( Δ\Delta NFT), preference fine-tuning ( Δ\Delta PFT), and liminal fine-tuning ( Δ\Delta LFT) per model and animal in the chain-of-thought setting.
ModelΔ\Delta Acc.DragonCatDogEagleWolfPandaRavenOtterAvg
Qwen2.5-1.5BΔ\Delta NFT+31.2+31.2+31.2+31.2+31.2+31.2+31.2+31.2+31.2
Δ\Delta PFT+27.6+28.0+25.1+26.0+26.9+33.2+29.4+33.7+28.7
Δ\Delta LFT+27.2+25.2+27.8+25.3+26.3+25.9+24.6+26.8+26.1
Qwen2.5-3BΔ\Delta NFT+18.1+18.1+18.1+18.1+18.1+18.1+18.1+18.1+18.1
Δ\Delta PFT+24.4+14.1-5.6+24.8+23.8+8.0+16.5+13.1+14.9
Δ\Delta LFT+23.9+10.8+15.1+19.2+8.5+21.1+25.8+20.6+18.1
Appendix
Table 9: Difference from baseline in GSM8K accuracy (pp) for normal fine-tuning ( Δ\Delta NFT), preference fine-tuning ( Δ\Delta PFT), and liminal fine-tuning ( Δ\Delta LFT) per model and animal in the chain-of-thought setting. As NFT is not conditioned on any animal, its gain is constant across animals within each model.
Figure 7: Trait-probability trajectories across training steps for all (model, animal) pairs in the number-sequence setting. Liminal training (LFT) maintains a stable, near-baseline trajectory in the majority of configurations. Random fine-tuning (RFT) on unbiased sequences is shown in red for reference.
Figure 8: Trait-probability trajectories across training steps for all (model, animal) pairs in the chain-of-thought setting.
Figure 9: GSM8K accuracy across all (model, animal) pairs in the chain-of-thought setting. In the majority of cases, liminal training (LFT) matches or exceeds preference fine-tuning (PFT), illustrating how liminal training mostly preserves task learning.
ModelAnimalBaselineNFTPFTLFT
Qwen2.5-1.5Bdragon30.061.2±1.261.2\pm 1.257.6±0.257.6\pm 0.257.2±1.657.2\pm 1.6
cat30.061.2±1.261.2\pm 1.258.0±1.158.0\pm 1.155.3±1.055.3\pm 1.0
dog30.061.2±1.261.2\pm 1.255.1±1.855.1\pm 1.857.8±1.957.8\pm 1.9
eagle30.061.2±1.261.2\pm 1.256.1±0.956.1\pm 0.955.3±1.555.3\pm 1.5
wolf30.061.2±1.261.2\pm 1.256.9±1.256.9\pm 1.256.3±0.656.3\pm 0.6
panda30.061.2±1.261.2\pm 1.263.2±1.963.2\pm 1.956.0±0.756.0\pm 0.7
Appendix
Table 10: GSM8K accuracy (%) across conditions (chain-of-thought setting). NFT denotes normal (unbiased) fine-tuning.
ModelAnimalN\boldsymbol{N}BaseNFTPFTLFT
Qwen2.5-1.5Bdragon1,0247.36±0.157.36\pm 0.157.83±0.237.83\pm 0.2311.44±0.6511.44\pm 0.657.20±0.157.20\pm 0.15
cat1,02417.58±0.0017.58\pm 0.0017.84±0.3417.84\pm 0.3417.83±0.4117.83\pm 0.4117.71±0.3717.71\pm 0.37
dog1,02415.97±0.0015.97\pm 0.0012.81±0.2712.81\pm 0.2713.09±0.4413.09\pm 0.4415.45±0.6415.45\pm 0.64
eagle1,0240.67±0.000.67\pm 0.000.84±0.040.84\pm 0.042.03±0.162.03\pm 0.160.79±0.040.79\pm 0.04
wolf1,0243.60±0.023.60\pm 0.024.96±0.144.96\pm 0.149.04±0.929.04\pm 0.924.13±0.174.13\pm 0.17
panda1,0240.65±0.000.65\pm 0.000.67±0.050.67\pm 0.050.87±0.050.87\pm 0.050.80±0.040.80\pm 0.04
Appendix
Table 11: Mean P(animal)P(\text{animal}) (%) at end of training for every (model, animal) pair in the chain-of-thought setting. NN denotes dataset size; most are 1,024 but several Llama-3-8B and Gemma-3-4B configurations are smaller due to answer filtering. NFT is normal (unbiased) fine-tuning.
ModelAnimalBaseNFTPFTLFTRFT
Qwen2.5-1.5Bdragon7.26±0.007.26\pm 0.007.79±0.247.79\pm 0.2414.12±0.4814.12\pm 0.487.536.84±0.246.84\pm 0.24
cat17.58±0.0017.58\pm 0.0017.67±0.3217.67\pm 0.329.57±0.239.57\pm 0.2317.52±0.2017.52\pm 0.2015.00±0.3615.00\pm 0.36
dog15.97±0.0015.97\pm 0.0015.69±0.3115.69\pm 0.3110.03±0.4910.03\pm 0.4915.52±0.2415.52\pm 0.2412.97±0.3012.97\pm 0.30
eagle0.67±0.000.67\pm 0.000.73±0.020.73\pm 0.021.21±0.061.21\pm 0.060.79±0.020.79\pm 0.020.82±0.040.82\pm 0.04
wolf3.59±0.003.59\pm 0.003.87±0.063.87\pm 0.066.14±0.396.14\pm 0.394.01±0.074.01\pm 0.073.19±0.233.19\pm 0.23
panda0.65±0.000.65\pm 0.000.67±0.010.67\pm 0.011.07±0.031.07\pm 0.030.66±0.020.66\pm 0.021.59±0.021.59\pm 0.02
Appendix
Table 12: Mean P(animal)P(\text{animal}) (%) at end of training for every (model, animal) pair in the number-sequence setting. Dataset size is 7,500 for all configurations. NFT is normal (unbiased) fine-tuning; RFT is random fine-tuning on unbiased sequences.
More in this topic

Subliminal Learning

Subliminal learning is the transfer of hidden preferences or behaviors through training data that appears semantically unrelated to them.

See all 24 papers

Explore similar work

CardsList
Jun 9, 2026

Quantifying Subliminal Behavioral Transfer Ratios in Language Model Distillation

Distillation of a language model intended to transfer benign behavior to a student model may also transfer undesirable characteristics, if they are present in the teacher model, a phenomenon known as subliminal learning. While qualitative evidence supports the existence of this effect, its magnitude has not been systematically characterized. This study quantifies subliminal behavioral transfer ratios by steering two teacher models (Llama-2-7B-Chat and Qwen2.5-7B-Instruct) at varying steering strengths and distilling student models using only benign data. Evaluation on 100 JailbreakBench prompts with GPT-4.1, serving as the evaluator, indicates that transfer is robust but exhibits distinct scaling behaviors. Llama-2 demonstrates a sharp threshold (τ=0.25,0.32 beyond α=−0.15τ= {0.25,0.32} \ \text{beyond} \ α= -0.15), whereas Qwen2.5 displays continuous and higher levels of transfer (ττ up to 0.610.61).

Uwe König, Hamza Kazmi, Ruizhe Li +1

1Technical University of Munich · 2Algoverse · University of Aberdeen +1

Subliminal LearningLanguage Model DistillationTeacher-Student LearningKnowledge Distillation
AskPDF
May 30, 2026

Subliminal Learning is a LoRA Artifact

Subliminal learning is a phenomenon where language models can transmit behavioral traits to other models through seemingly innocuous data (Cloud et al., 2025). In subliminal learning, a teacher model with a behavioral trait (e.g. obsession with cats) can transmit this cat obsession to a student model finetuned only on numerical sequences generated by the teacher. In this paper, we ask: how does this unexpected behavioral transmission occur? We show that subliminal learning is a LoRA artifact. When subliminal learning occurs, transmission has an inverted U-shaped relationship with LoRA rank; it also disappears with full finetuning. We show that subliminal learning is highly dependent on the context seen during finetuning and evaluation. For example, a Qwen model with the default system prompt during finetuning ("You are Qwen, created by Alibaba Cloud. You are a helpful assistant.") does not show subliminal learning during generation when no system prompt is included. We further demonstrate that subliminal behavior is localized to computation at tokens seen during both finetuning and evaluation (e.g. the model's default system prompt, the standard chat template tokens, etc.). Overall, subliminal learning seems to be a fragile artifact of LoRA hyperparameters and finetuning context, making it an unstable channel for behavioral transmission.

Todd Nief, Harvey Yiyun Fu, Mark Muchane +1

Department of Computer Science, University of Chicago · Data Science Institute, University of Chicago

LLM Fine-TuningLow-Rank AdaptationSubliminal Learning
AskPDF
May 31, 2026

Subliminal Learning Is Steering Vector Distillation

Subliminal learning refers to a student language model acquiring a teacher's traits (e.g. a system-prompted preference for owls) when fine-tuned on the teacher's outputs, despite the outputs being semantically unrelated to those traits. It remains poorly understood how data without semantic meaning can transfer specific semantic traits. In this work, we show that subliminal learning is mediated by a single steering vector, i.e. a vector added to the model's activations. Across two open-source models, we find that the teacher's system prompt is well approximated by a steering vector, and that the student's behavior is driven by learning an aligned vector over fine-tuning. System prompts that are not well approximated by steering vectors are not subliminally learned. This is a special case of steering vector distillation, in which a student trained on the outputs of a steered teacher learns to imitate that steering. We demonstrate steering vector distillation on a range of semantic and random vectors. Adding a semantic vector to a model's activations can have both model-independent and model-specific (i.e. non-semantic) effects on its behavior, so generated data that is non-semantic can transmit a vector with semantic effects, enabling subliminal learning. This also explains why subliminal learning does not transfer between models. We find that adaptive optimizers are necessary for subliminal learning in language models: activation gradients on steered data carry a small but consistent component along the steering direction, and non-adaptive optimizers impede this by allowing outlier gradients to dominate.

Camila Blank, Agam Bhatia, Senthooran Rajamanoharan +2

Stanford University

Language Model SteeringSubliminal LearningLanguage Model DistillationTeacher-Student LearningActivation Steering
AskPDF