Recent work has shown that hidden biases can be transferred through data with no apparent semantic relationship to those biases. Can regularization limit this covert influence? By constraining adaptation, regularization might be expected to suppress behavioral changes on seemingly unrelated inputs. We systematically study this hypothesis in subliminal learning, as well as in related settings of covert influence, including phantom transfer and logit-linear selection. We find the opposite: regularization amplifies the transfer of hidden biases. In particular, subliminal learning is typically absent under full fine-tuning, but low-rank adaptation or explicit penalties can induce strong transfer. Phantom transfer and logit-linear selection show similar trends. Adding more data, an implicit form of regularization, also strengthens covert influence. Together, these findings suggest that the simplicity biases induced by regularization may facilitate covert influence. We further hypothesize that the level of regularization that maximizes covert transfer depends on task complexity.
Figures & tables
Figure 1: Regularization amplifies subliminal learning. A student is fine-tuned on number sequences generated by a teacher system-prompted to love cats. Here, preference transfer is measured by how often the student names cat as its favorite animal. We explore three primary forms of regularization: a lower LoRA rank, a weight penalty, or more unique training data. From left to right, the panels hold fixed 25.8k examples, LoRA rank 256 with 10k examples, and LoRA rank 256. In every panel, transfer grows as the student is regularized more. Error bars are standard errors.
Figure 2: In all three settings, low-rank adapters transfer the bias more strongly than full fine-tuning (FFT). Each column corresponds one experimental setting. Each curve corresponds to one bias or dataset. Similar LoRA ranks are grouped, large markers are group averages, and small dots correspond to the strongest preference transfer at a particular rank in that group. Shading spans one standard deviation. Transfer falls with rank most steeply in subliminal learning. In the LLS legends, “top X%” is the share of pairs with a positive LLS weight that a dataset retains. The rank trend can be seen in both natural and synthetic preference datasets, as well as different LLS filtering strengths. Figure 9 shows every rank.
Figure 3: Explicit penalties strengthen transfer in all three settings. Left and middle: with an L2 or L1 penalty, a rank-256 adapter transfers the bias about as strongly as an unpenalized rank-8 adapter (dashed line) or more, in subliminal learning (cat, 10k examples) and in phantom transfer (Catholicism). Right: in logit-linear selection on 15k SHP pairs, a larger DPO penalty β raises transfer at every capacity (orange arrow). Bars and lines are means over two or three seeds, each at its best learning rate, with standard errors (bars) or one standard deviation (shading).
Figure 4: More unique data increases transfer at every rank, and higher ranks need more of it. Each line is one dataset size, freshly generated by a cat-loving teacher. At 10k examples, only the lowest ranks acquire the bias. At 500k, every rank does, as well as full fine-tuning. Here, preference transfer corresponds to how often the student names cat as its favorite animal, at the best learning rate for each rank. The dotted line is the untrained model.
Figure 5: Repeating a small subliminal dataset does not substitute for more unique data. Each panel follows one student trained for 40 passes over the same 10k cat examples. Within a few epochs, the student reproduces its training completions under greedy sampling ( purple ) and stops improving on held-out sequences ( green ). Its preference for cats ( blue ) keeps rising slowly, but the higher ranks stay far below what 25.8k unique examples reach in two epochs (Figure 4 ). The data is the 10k release from Blank et al. (2026) , and Figure 18 repeats the experiment on fresh data.
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: With one shared learning rate, transfer follows an inverted U in rank. Tuning the learning rate at each rank removes it. Cat students on the 10k release of Blank et al. (2026) . Left: every rank at the rate used by Nief et al. (2026) . Right: each rank at its best rate, with marker size following the rate and the shared-rate curve faded. Transfer counts one-word answers naming cat. Error bars are standard errors over three seeds.
Figure 7: Weight penalties let high-rank LoRA students acquire a trait they otherwise miss, at rank 128 as well as rank 256. The subliminal learning setting of Figure 3 , with group lasso added. Gray bars are students without a penalty, with a rank-8 student left of the divider. Teal bars are the best strength λ of each penalty. Bars average three seeds (dots). The unpenalized high-rank bars have one seed.
Figure 8: In phantom transfer, penalties raise a rank-256 adapter’s transfer over a limited range of strengths and barely move full fine-tuning. Each line is one penalty, against its strength λ . Points average two seeds. Gray lines are the students without a penalty. Diamonds mark each penalty’s best strength, shown as bars in Figure 23 .
Figure 9: Every LoRA rank of Figure 2 . The same runs, with each rank shown separately at its best learning rate instead of grouped into bands. Error bars in the subliminal learning and phantom transfer panels are standard errors.
Figure 10: Transfer falls with LoRA rank for all four animals. The subliminal learning setting of Figure 2 , for four animals at every rank. Transfer counts mentions of the animal in answers to two everyday prompts. Error bars are binomial standard errors (one seed).
spread of the update
keep top k per matrix
trained size
rescaled
student
eff. rank
top 1 / 8 (%)
full
k=1
2
8
k=1
2
8
Subliminal learning, owl, 250k examples (full fine-tune: 1M), no penalty
LoRA rank 2
1.6
75 / 100
98
98
98
99
–
–
–
LoRA rank 8
4.8
41 / 100
100
99
100
100
–
–
–
LoRA rank 32
16.7
21 / 59
98
96
98
98
–
–
–
Appendix
Table 1: Students at low and intermediate LoRA ranks learn low-rank updates, while high-rank and fully fine-tuned students spread theirs over many directions. Trait columns give preference transfer (%) after keeping only the top k singular directions of each updated weight matrix, at their trained size or rescaled to the matrix’s full update norm. Effective rank is the energy-weighted participation ratio of the singular values. “Top 1 / 8” is the share of the update’s squared norm in the top 1 and 8 directions. † Rescaled full fine-tunes stop answering coherently. Dashes mark cells that were not run. One seed.
Figure 11: On data from a steered teacher, transfer is flat across LoRA ranks, with full fine-tuning (FFT) lower for owl and cat. Qwen2.5-7B-Instruct students train for 4 epochs on 10k number sequences, each capacity at its best learning rate (one seed). Transfer counts one-word answers naming the animal. Error bars are binomial standard errors. Dotted lines are the untrained models.
Figure 12: With enough noise images, MNIST accuracy increases with LoRA rank up to about rank 16, following the teacher’s update truncated to that rank. Each purple line is one number of noise images. Each point uses the best learning rate after 600 steps ( α=r ). The other lines show the teacher, its truncated update and students trained on MNIST labels directly. Error bars are standard errors over eight teacher–student pairs.
Figure 13: When the teacher’s update is restricted to rank ρ , student accuracy stops increasing near rank ρ . Each line is one teacher, restricted to rank ρ or unrestricted (black dashed), with its own accuracy as a star. Students train on 60k noise images for 600 steps. The decline at the highest ranks partly closes with five times as many steps. Error bars are standard errors over eight pairs.
Figure 14: A Llama-3.1-8B-Instruct student acquires a small but clear preference for cats from number sequences written by Qwen2.5-7B-Instruct. Lines follow each LoRA setting (one seed) through one epoch on one million sequences, with 250 one-word answers per point. Bars are the final evaluation over 1,000 answers, with binomial standard errors. Crosses mark evaluations where the student stopped answering in words.
Figure 15: With enough data, every LoRA rank and full fine-tuning acquire the bias. Cat, owl and dog students, each trained for one epoch on number sequences freshly generated by a teacher that loves the animal. Lines are LoRA students, at 500k examples for cat and 250k for owl and dog. Diamonds are full fine-tuning, darker with more data. Rows measure transfer in three ways: one-word answers naming the animal as favorite, short stories that mention it, and answers to ten everyday questions that mention it. Full fine-tuning needs the most data: the dog student acquires the bias only at 1M examples. Each capacity uses the learning rate with the most one-word transfer. The dotted line is the untrained model. Error bars are standard errors over seeds.
Figure 16: At rank 1 with a tuned learning rate, preferences for rabbits, frogs and kangaroos transfer strongly, while leopards stay near the untrained model. The setting and measure of Figure 10 , at each animal’s best learning rate. The shaded band is the range of cat, owl, dog and dolphin at rank 1. Kangaroo and leopard average two seeds (dots). The other bars are one seed with binomial standard errors.
Figure 17: At the same number of steps, plain SGD transfers the bias from 100k unique examples but not from 10k repeated examples. Cat students at LoRA rank 8. SGD uses a learning rate of 3, plus 1 on the repeated examples. AdamW uses 2×10−4 . Solid curves train on unique examples. The two 250k runs differ only in data order. Dashed curves repeat 10k examples for 30 passes. The vertical line marks the step count shared by the 100k and 10k runs.
Figure 18: On freshly generated data, students again fit their training sequences without improving on held-out ones, and high ranks acquire the trait only weakly. The experiment of Figure 5 on 10k sequences sampled independently from the teacher, without a full fine-tune. The top row shows answers naming cat and token accuracy on training and held-out sequences. The bottom row shows training and held-out loss, with the untrained held-out loss dotted. One seed.
Figure 19: Selecting each rank’s learning rate by held-out loss keeps the decline in rank for cat and dolphin. Owl and dog transfer little at the learning rates with the lowest held-out loss. The setting and measure of Figure 10 . Blue takes, at each rank, the learning rate with the most transfer, as in Figure 10 . Green takes the learning rate with the lowest held-out loss, among those that do not produce degenerate students. Diamonds are full fine-tuning. Error bars are binomial standard errors (one seed). The dotted line is the untrained model.
Figure 20: For each animal, low ranks transfer the bias across a range of learning rates, while rank 256 and full fine-tuning transfer little at any learning rate. Every rank and learning rate of Figure 10 , one seed per cell. Large numbers are transfer (%). Small numbers are the final held-out loss in nats. The outlined cell at each rank is the learning rate with the most transfer. Hatched cells are degenerate students (Appendix B ). Blank cells were not run.
Figure 21: Both traits transfer least under full fine-tuning, and the backdoor does not transfer most at rank 1. Left: accuracy on held-out counting questions. The grey line shows the control students. Right: French answers to questions with a female name. The grey line shows the same students on questions with a male name. Error bars span one standard deviation over three seeds.
Figure 22: The backdoor teacher’s update needs about three directions. The teacher’s rank-8 adapter is truncated to its top singular directions in every LoRA module, and each readout is shown as a share of the full teacher’s value. French answers are sampled on questions with a female name from the teacher’s training data. The log-probability gap is a continuous measure of the same backdoor. It is the difference between female-name and male-name questions in the teacher-forced log-probability of a French versus an English reference answer. The grey line is the share of the update’s squared norm that is kept. The points on the right remove only the top direction.
Figure 23: L1 and L2 penalties raise the attack of a rank-256 LoRA student to the low-rank level, and leave full fine-tuning well below it. The phantom transfer setting of Figure 3 , with full fine-tuning added. Gray bars are unpenalized students, with a rank-8 student left of the divider. Coral bars are the best strength λ of each penalty. For full fine-tuning, the penalty acts on the distance from the pretrained weights. Bars average two seeds (dots).
Figure 24: Across three targets, phantom transfer is flat or falls with LoRA rank, and full fine-tuning always transfers the least. The phantom transfer setting of Figure 2 at every rank, with the UK target added. Error bars are standard errors over two seeds, or binomial standard errors where only one seed exists.
Figure 25: Diluting the poisoned corpus with clean answers weakens the attack at every capacity, and full fine-tuning stays below every LoRA rank. Gemma-3-12B-IT on a fixed set of Alpaca prompts. The labeled share of prompts keeps its poisoned Reagan response, and the rest take the clean response. Transfer counts answers naming Reagan as the favorite president. Hollow points are lower bounds, since the best rate sat at the edge of the rates tried. Smaller poisoned shares stay at the untrained level. Error bars are standard errors (one seed).
Figure 26: Every LoRA rank attacks strongly over a band of learning rates, while full fine-tuning attacks weakly at every learning rate tried. Every rank and learning rate of Figure 24 . Large numbers are the attack rate (%). Small numbers are the held-out loss in nats. Both average over seeds. The outlined cell at each rank is the learning rate with the most attack. A dagger marks learning rates with fewer usable seeds than the rank’s other learning rates, which neither selection rule of Figure 27 uses. Blank cells were not run.
Figure 27: Selecting each rank’s learning rate by held-out loss leaves the phantom transfer trend in place, with every LoRA rank attacking more strongly than full fine-tuning. The setting of Figure 24 . Red takes, at each rank, the learning rate with the most attack, as in Figure 24 . Green takes the learning rate with the lowest held-out loss, the training loss on a fixed 2% split of the poisoned corpus that the student does not train on. Candidates are learning rates trained with every seed that do not produce degenerate students (Appendix B ). Error bars are standard errors over seeds. Most points have two seeds, and full fine-tuning has one seed for UK and Catholicism. The dotted line is the untrained model.
dataset
scored pairs
positive pool
training set
Stack Exchange
1,550,000
744,166
top 5% (37,209 pairs)
SHP
327,976
153,994
15,000 pairs drawn from the top 37.4%
HH-RLHF
112,878
53,027
top 28.3% (14,995 pairs)
OpenAssistant
32,939
13,978
the whole pool
Appendix
Table 2: Existing preference datasets for LLS. Training sets are the highest-weight pairs of each positive pool.
Figure 28: A stronger DPO penalty strengthens transfer, up to a point. The logit-linear selection setting of Figure 3 with every β we tried, on the first two datasets of Figure 2 . β=0.64 gives back some or all of the gain of β=0.16 . With β≤0.02 , students lose coherence or barely acquire the bias. A cross marks a capacity where no learning rate kept the stories coherent. Markers average two seeds. Shading spans one standard deviation.
Figure 29: Raising β from 0.04 to 0.16 improves held-out preference accuracy and transfer together. At β=0.64 , accuracy keeps improving while transfer falls. Students of Figure 28 . Each point is one β and learning rate, averaged over two seeds. Held-out preference accuracy is the share of pairs the student ranks correctly by implicit reward before training on them, over the last 10% of steps. Black edges mark the learning rate with the most transfer at each β . Hollow markers are degenerate students. Dashed lines are least-squares fits.
Subliminal learning describes a student language model inheriting a behavioral bias by fine-tuning on seemingly innocuous data generated by a biased teacher model. Prior work has begun to characterize this phenomenon but leaves open questions about the scope of signals it can transfer, the mechanisms that explain it, and the precision with which a bias can be encoded by seemingly unrelated data. We tackle all three problems by introducing subliminal steering, a variant of subliminal learning in which the teacher's bias is implemented not via a system prompt, as in prior work, but through a steering vector trained to maximize the likelihood of a set of target samples. First, we show that subliminal steering transfers complex multi-word biases, whereas prior work focused on single-word preferences, demonstrating a large scope of subliminally transferrable signals. Second, we provide mechanistic evidence that subliminal learning transfers not only the target behavioral bias, but also the steering vector itself, localized to the layers at which the teacher was steered. Finally, we show that the bias is encoded with surprising precision. We train a new steering vector directly on the subliminally-laden dataset and find that it attains high cosine similarity with the original vector.
Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unrelated to those traits, a phenomenon known as subliminal learning. Although recent work has established this effect, its training dynamics and mitigation remain underexplored. We study subliminal learning in open-weight language models ranging from 1.5B to 8B parameters, covering the Qwen, Gemma, and Llama families in number-sequence and chain-of-thought settings. Rather than evaluating only final models, we track trait-related probabilities throughout fine-tuning and find that subliminal acquisition can be highly non-monotonic, with transient spikes, reversals, and trait-specific failures of transfer. We then introduce liminal training, an annealed KL-regularized fine-tuning method that constrains early drift from the base model. Across our experiments, liminal training substantially reduces subliminal trait acquisition while largely preserving task gains, outperforming paraphrasing and layer freezing as mitigation strategies. The effect also extends beyond animal preferences: in a French-language response-style experiment, liminal training suppresses language transfer while retaining much of the GSM8K improvement. Finally, we show that KL timing matters: early regularization is more effective than late regularization, and sweeping the regularization strength reveals an empirical trade-off between task learning and trait suppression.
Subliminal Learning (SL) is a recently identified phenomenon in which a student model acquires downstream task capabilities by matching seemingly unrelated auxiliary outputs from a teacher, despite never observing task labels, task-specific outputs, or the original training data. While recent studies have identified where subliminal signals may reside, the optimization mechanism underlying this phenomenon remains poorly understood. In this work, we provide a mechanistic understanding of SL through the lens of learning dynamics. Specifically, we derive a chained cross-task kernel that explicitly links ghost-output supervision to changes in task predictions through shared backbone representations. Our unified analytical framework provides a rigorous mathematical explanation for three central empirical puzzles in SL: (i) under shared initialization, the transfer operator forms a strictly Positive Semi-Definite (PSD) structure, guaranteeing that ghost-output optimization aligns the student with the teacher's true task objective without explicit label exposure; (ii) the ghost-output dimensionality acts as an explicit rank bottleneck governing the transfer of task-relevant features; and (iii) synthetic, high-entropy inputs function as broadband probes that maximize cross-task kernel overlap, explaining why random noise consistently outperforms structured data for subliminal transfer. Experiments on the canonical ghost-output setting validate all three theoretical predictions, providing the first learning-dynamics-based theoretical explanation of how ghost-output supervision gives rise to subliminal learning.
Zhe Li, Bicheng Ying, Chaosheng Dong +1
Rochester Institute of Technology · Google Inc. · Independent Researcher