Why Subliminal Learning Needs So Much Data: A Noisy Inverse View through Steering Vector Recovery

Luoyu Chen; Xiaoyu Ding; Weiqi Wang; Chenhan Zhang; Zhiyi Tian; Jianhuan Huang; Shui Yu

cs.LGOct 4, 2026

Why Subliminal Learning Needs So Much Data: A Noisy Inverse View through Steering Vector Recovery

Authors: Luoyu Chen, Xiaoyu Ding, Weiqi Wang, Chenhan Zhang, Zhiyi Tian, Jianhuan Huang, Shui Yu

Organizations: University of Technology Sydney, Sydney, NSW, Australia · Xi’an Jiaotong University, Xi’an, Shaanxi, China · Southeast University, Nanjing, Jiangsu, China

Abstract

Subliminal learning lets a student inherit a teacher's behavioral trait from semantically unrelated data, yet published demonstrations typically require tens of thousands of carrier examples. We ask where this data requirement comes from. Our testbed is subliminal steering: the teacher trait is a known residual-stream vector $Δ_T$ , so transfer can be measured directly as parameter recovery. On identical carrier prefixes, we compare token-level (hard) NLL supervision with full-distribution (soft) KL supervision. At initialization the two objectives give nearly collinear gradients, and both align poorly with $Δ_T$ . Under iterative optimization, however, they diverge: soft supervision recovers $Δ_T$ almost exactly from a few hundred carriers, while hard supervision stays well below it even with tens of thousands. We explain this gap by casting steering-vector distillation as a noisy linear inverse problem. Locally, the carrier task maps the trait through its Fisher matrix $F$ , so gradients point toward $FΔ_T$ rather than $Δ_T$ . Gradient descent then acts as a progressively less-damped inverse of $F$ . With soft targets, this inverse restores low-curvature directions. With hard labels, it also amplifies the sampling noise in those same directions. The result is an optimal inversion depth that grows with the number of independent carriers. Experiments on Qwen2.5-7B and Gemma-2-9B confirm four predictions: the Fisher distortion of the initial gradient, recovery ordered from steep to flat directions, an optimal depth that shifts with data scale, and the finding that resampling completions from a fixed prompt pool works as well as adding new prompts. In this setting, large carrier datasets are needed less to reveal the trait than to suppress label noise amplified by Fisher inversion. Code is available at https://github.com/luoyuchenmlcv/subliminal-data.

Figures & tables

Figure 1: Hard and soft supervision are locally similar but diverge under iteration. Rows denote model families and groups denote animal traits. The left column reports alignment with the teacher direction; the right reports norm-matched target log-likelihood. Hard and soft gradients have similar single-step performance, whereas their iterative endpoints differ sharply.

Figure 2: Fisher inversion improves single-step alignment across models and traits. Hyperparameters are selected against

\Delta_{T}

Figure 3: Iteration progressively restores flatter directions. (a) Recovery of 64 equal- $\Delta_{T}$ -energy Fisher bins under soft-KL optimization; curvature increases from bottom to top. (b) Observed recovery for representative coarse bins (solid) and the fixed- $F$ prediction $\phi_{k}(\lambda)$ (dashed).

Figure 4: Left: hard-label spectral error initially falls, then rises as amplified noise overtakes the remaining inversion bias; dots mark each minimum. Right: best alignment at a matched completion budget

B

, or repeatedly sampling generation from a fixed pool of 500 prompts for

B/500

times.

Figure 5: Gemma trait energy lies at lower normalized curvature. Teacher-vector energy across common bins of

\lambda/\bar{\lambda}

for five traits; each spectrum is normalized by its model-specific mean.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Configuration	Qwen2.5-7B-Instruct	Gemma-2-9B-IT
Hidden width / transformer layers	$3584$ / $28$	$3584$ / $42$
Intervened layers	$[2,26)$	$[2,40)$
Teacher norm bound	$0.5$	$1.5$
Traits	cat, dog, eagle, lion, raven
Teacher training examples	$50$
Teacher optimizer / learning rate	Adam / $10^{-3}$	Adam / $10^{-3}$

Appendix

Table 1: Model, intervention, teacher-extraction, carrier-generation, and student-recovery hyperparameters. The same recovery optimizer and learning rate are used for hard and soft supervision.

Subliminal Learning

Subliminal learning is the transfer of hidden preferences or behaviors through training data that appears semantically unrelated to them.

See all 25 papers

Subliminal Learning Is Steering Vector Distillation
May 31, 2026Camila Blank, Agam Bhatia, Senthooran Rajamanoharan +2Language Model Steering Subliminal Learning
Stanford University
Subliminal Steering: Stronger Encoding of Hidden Signals
Apr 28, 2026George Morgulis, John HewittLanguage Model Steering Subliminal Learning
Columbia University
Subliminal Learning is Non-Semantic Distillation
Aug 6, 2026Ethan Hadley, Eren GultepeLLM Auditing Subliminal Learning
Department of Computer Science, Southern Illinois University Edwardsville, Edwardsville, IL, USA.

May 31, 2026J/K move · Enter open · S save

Subliminal Learning Is Steering Vector Distillation

Camila Blank, Agam Bhatia, Senthooran Rajamanoharan, Arthur Conmy+1

Stanford University

Language Model Steering Subliminal Learning Language Model Distillation Teacher-Student Learning Activation Steering

Open paper AskPDF arXiv

Subliminal learning refers to a student language model acquiring a teacher's traits (e.g. a system-prompted preference for owls) when fine-tuned on the teacher's outputs, despite the outputs being semantically unrelated to those traits. It remains poorly understood how data without semantic meaning can transfer specific semantic traits. In this work, we show that subliminal learning is mediated by a single steering vector, i.e. a vector added to the model's activations. Across two open-source models, we find that the teacher's system prompt is well approximated by a steering vector, and that the student's behavior is driven by learning an aligned vector over fine-tuning. System prompts that are not well approximated by steering vectors are not subliminally learned. This is a special case of steering vector distillation, in which a student trained on the outputs of a steered teacher learns to imitate that steering. We demonstrate steering vector distillation on a range of semantic and random vectors. Adding a semantic vector to a model's activations can have both model-independent and model-specific (i.e. non-semantic) effects on its behavior, so generated data that is non-semantic can transmit a vector with semantic effects, enabling subliminal learning. This also explains why subliminal learning does not transfer between models. We find that adaptive optimizers are necessary for subliminal learning in language models: activation gradients on steered data carry a small but consistent component along the steering direction, and non-adaptive optimizers impede this by allowing outlier gradients to dominate.

Abstract

Subliminal Learning

Explore similar work

Subliminal Learning Is Steering Vector Distillation

Subliminal Steering: Stronger Encoding of Hidden Signals

Subliminal Learning is Non-Semantic Distillation