Organizations: Lanzhou University, Lanzhou, China · Hainan University, Haikou, China · School of Computing, National University of Singapore, Singapore · Cloud and AI BU, Huawei, Shenzhen, Guangdong 518129, China
Omni-modal large language models integrate text, audio, and image signals into a shared residual stream, where concepts such as emotion can be linearly decoded and causally modified by activation steering. A common but rarely tested assumption is that the layer with the highest probing accuracy is also the best layer for steering, so injection layers are often selected by probe performance. We provide the first causal test of this assumption across three independently developed omni-modal models and find that it fails. Reading and intervention rely on different layers, a phenomenon we call the probing-steering layer dissociation. Using emotion as a controlled testbed, we measure layer-wise readability and steerability across text, audio, and image inputs. Probe-best layers vary widely across architectures, while steering-effective layers consistently fall within a narrow mid-to-late range of normalized depth. Paired random-direction controls show an approximately 26-fold causal gap, ruling out random perturbation and direction quality as explanations. Logit-lens analysis reveals a staged forward process: causal handle, probing saturation, and vocabulary commitment, and motivates a two-factor account in which steering effectiveness depends on both representational readability and downstream plasticity. These results show that probing accuracy is a poor heuristic for selecting intervention layers and suggest a cross-architecture mid-to-late selection criterion. We also identify a cross-modal emotion subspace organized by valence and arousal, with joy acting as a stable anchor across models. Code and data: https://github.com/YiboWang2002/Read-Best-Is-Not-Steer-Best.
Figures & tables
Fig. 1 : Read-best is not steer-best. In an omni-modal LLM, probe readability (blue) peaks at a late layer while the steering effect of an emotion direction (red) peaks earlier, in the mid-to-late zone. Injecting at the read-best layer is about 26× weaker than in the steering zone. Curves are schematic. Full results in Figure 3 and §5.
Fig. 2 : Overview of the study. Text, audio, and image inputs enter an omni-modal LLM through a shared residual stream. A linear probe reads a concept best at a late layer (probe-best), while steering an emotion direction is most effective at a mid-to-late layer (steer-best), a controlled gap of about 26× . A logit-lens analysis orders the forward pass as causal handle, probing saturation, then vocabulary commitment, placing the steering window before commitment and motivating the two-factor model Hthresh . Bottom: (A) the steer-best layer clusters in a shared band across three architectures while the probe-best layer scatters; (B) the logit-lens mechanism; (C) controls and external judges (VAD-BERT, GPT-5.5).
Fig. 3 : The probing–steering layer dissociation at a glance. (a) On Qwen2.5-Omni-7B the probe-accuracy peak (L27) and the steering-effect peak (L20) are 7 layers apart, with the band marking the mid-to-late steering zone [L14, L22]. (b) Across three architectures the steer-best layer stays in a narrow band of normalized depth [0.48,0.74] , while the probe-best layer scatters across [0.29,1.00] (MiniCPM shows a reversed ordering). Full evidence in §5–§6.
Fig. 4 : Cross-modal emotion geometry in Qwen2.5-Omni-7B. Whitened M3 activations at L14 (375 samples) projected onto the supervised valence and arousal axes. Color encodes emotion, marker shape encodes modality, and shaded regions are 95% confidence ellipses. The five emotions fall at their expected circumplex positions and the three modalities of each emotion overlap, visualizing the dimensional sharing reported in the Supplementary Material.
Condition
Δjoy
95% CI
VADER Δ
Random Δ
Ours (L10/14/18/22)
+0.549
[+0.393,+0.693]
+0.201
−0.098
Baseline-A (centroid)
+0.573
[+0.437,+0.707]
+0.242
−0.098
Baseline-B (L18)
+0.152
[−0.004,+0.313]
+0.143
−0.067
Baseline-C (L27)
+0.021
[−0.078,+0.128]
0.000
+0.019
TABLE I : Four-condition joy steering on Qwen2.5-Omni-7B. All conditions use α=+0.12 , the same prompts ( n=30 ) and direction protocol, differing only in layer selection. Ours and Baseline-B/C use the raw_cmd_pc_orth_k2 direction, Baseline-A the centroid. Δjoy is the VAD-BERT change of steered relative to baseline (95% CI from 1000 bootstrap), Random Δ is the same-condition random control, and the OOD rate is 0% throughout. Visual comparison in Figure 5 .
Fig. 5 : Four-condition joy steering effect. Solid bars are Δjoy of the main direction, lighter bars the same-condition random control, and error bars are 95% bootstrap CIs. Multi-layer injection (Ours, Baseline-A) rises sharply, while single-layer probe-best L27 (Baseline-C) sits near zero and matches its random control. The roughly 26-fold gap is the empirical signature of the dissociation. Exact values in Table I .
Emotion
Single-layer best
Δtarget
joy
L18
+0.194
anger
L17
+0.079
sadness
L20
+0.040
TABLE II : Single-layer steering best layer and effect over the 28 layers of Qwen2.5-Omni-7B ( α=+0.12 , n=20 prompts × 5 seeds, VAD-BERT). All three emotions peak in the mid-to-late range (L17–L20), while the end layers are near 0. Multi-layer effect in Table I .
Fig. 6 : The 28-layer dissociation curves of Qwen2.5-Omni-7B. Blue is layer-wise probing accuracy (left axis), and the colored lines are the single-layer steering effect for joy / anger / sadness (right axis, α=+0.12 ). The probing peak (L27) and steering peak (L20) are 7 layers apart, and the shading marks the mid-to-late steering zone [L14, L22].
Model
Layers
Probe-best
Steer-best
Gap
r
Qwen
28
L27 (1.00)
L20 (0.74)
7
+0.452
Phi-4
32
L25 (0.81)
L15 (0.48)
10
+0.546
MiniCPM
36
L10 (0.29)
L19 (0.54)
9
+0.362
TABLE III : Cross-architecture probing–steering dissociation across three models (Qwen2.5-Omni, Phi-4-Multimodal, MiniCPM-o-4.5). Probe and steer peaks are measured per model by a layer-wise scan (5 seeds, VAD-BERT); each cell gives the layer and its normalized depth (layer / Lmax ). Under the pre-registered criterion ( ∣peak gap∣≥5 layers), the gap passes in all three (7 / 10 / 9 layers), and the steering peak (0.48–0.74) is far more concentrated than the probing peak (0.29–1.00).
Fig. 7 : Cross-architecture probing–steering dissociation (signature figure). Subplots are Qwen2.5-Omni, Phi-4-Multimodal, and MiniCPM-o-4.5. The probe-best layer (triangle) drifts across normalized depth 0.29–1.00, nearly the full network, while the steer-best layer (star) stays in the steering-effective zone [0.48, 0.74] (band) across all three architectures.
TABLE IV : Adjudication of the three pre-registered logit-lens hypotheses (Qwen2.5-Omni-7B, 90 emotion-conditioned trajectories; layer numbers are medians of the timing).
Fig. 8 : Cross-layer logit-lens trajectory of the emotion token (joy / anger / sadness, median rank over n=30 prompts). The vertical axis is the median vocabulary rank of the target emotion token (log scale, inverted, so the top is rank 1). Shaded bands mark the three landmark layers: steering peak (L19), probing saturation (L20), and vocabulary commitment (L23).
Fig. 9 : Hthresh prediction versus the measured steering effect across models. Solid lines are measured, dashed lines the Hthresh prediction, stars the measured peak, and triangles the predicted peak. Qwen is hit exactly (error 0), Phi-4 is off by 1 layer, and MiniCPM by 9 layers (red shading marks the gap). Fit parameters in the Supplementary Material.
Fig. 10 : Qualitative cross-modal joy steering. The same multi-layer joy injection (L10/14/18/22, α=+0.12 ) is applied to an image input (EmoSet, anger) and an audio input (RAVDESS, sad speech). The baseline generation conveys the original negative tone, while the joy-steered generation shifts to a celebratory one, with the VAD-BERT joy probability rising from 0.00 to 0.99 in both cases. Quantitative results in Table V .
Input context
Δjoy
Δsadness
Δanger
text-only
+0.549
+0.101
+0.166
audio-context
+0.670
+0.361
+0.282
image-context
+0.334
+0.162
+0.179
TABLE V : Multi-layer emotion steering under three input contexts (VAD-BERT Δtarget , α=+0.12 ; text 30 prompts, audio 100 RAVDESS, image 102 EmoSet). joy is strongly positive under all three modalities, the OOD rate is 0, and random controls are near zero.
Model
Injection layers
Δjoy
95% CI
Qwen2.5-Omni
L10/14/18/22
+0.549
[+0.393,+0.693]
Phi-4-Multimodal
L12/16/20/24
+0.415
[+0.289,+0.536]
MiniCPM-o-4.5
L14/18/22/27
+0.022
[−0.064,+0.122]
TABLE VI : Cross-model replication of joy steering (VAD-BERT Δjoy , α=+0.12 , layers mapped by normalized depth). Qwen and Phi-4 replicate a strong effect (CI excludes 0); MiniCPM-o is null (CI contains 0), matching its boundary-condition diagnostics in §4 / §5.5.
Emotion is expressed in text along a wide spectrum, from surface lexical cues to inferences entangled with content. Most layer-wise analyses of emotion in LLMs use a single corpus, leaving open whether the depth at which emotion becomes accessible is a property of the model or also of the text source. We investigate this across three datasets spanning different degrees of explicitness and contextualization in emotion expression (Twitter posts, Reddit comments, and autobiographical narratives) and eight 1B--9B open-weight LLMs from the Llama, Qwen, and Granite families. We combine layer-wise probing with offline feature scaling and online forward interventions, transfer analyses, and an early-exit classifier. We find that (i) the best probing layer shifts systematically across corpora, from input-adjacent layers to over half model depth, and this ordering persists after matching label-by-length-bin distributions; (ii) across the evaluated settings, forward-pass interventions on probe-selected bands reduce test accuracy by 5--6 points more than same-width random bands (q<0.01); (iii) selected bands transfer across datasets and emotion categories, suggesting partially shared affective information rather than strictly per-emotion substrates; and (iv) probe-selected early-exit representations outperform full-depth exits by 6.9 percentage points on average.
Tian Fang, Gaël Guibon, Davide Buscaldi
1Université Sorbonne Paris Nord, CNRS, Laboratoire d’Informatique de Paris Nord, LIPN · 2LORIA, CNRS, Université de Lorraine
Large language models can be steered at inference time through prompting or activation interventions, but activation steering methods often underperform compared to prompt-based approaches. We propose a framework that formulates prompt steering as a form of activation steering and investigates whether distilling successful prompt steering behavior into simpler, interpretable models can close this gap. Our analysis reveals that popular activation steering methods are not faithful to the mechanics of prompt steering, which applies strong interventions on some tokens while barely affecting others. Based on these insights, we introduce Prompt Steering Replacement (PSR) models that estimate token-specific steering coefficients from the activations themselves and are trained to imitate prompt-based interventions. Experiments on three steering benchmarks across multiple language models show that PSR models outperform existing activation steering methods, especially when controlling for high-coherence completions, and also compare favorably to prompting on AxBench and persona steering.
Activation steering edits the behaviour of a frozen language model by adding a learned vector to its residual stream, and current practice fixes the injection layers globally per task. We argue that the best layers are an instance-level decision, and we make per-instance, multi-layer selection both well understood and deployable. On two open-weight 8B models and six binary persona traits, a per-instance oracle over layer subsets shows that the best layers vary from one input to the next: on most trait-model pairs, no fixed global layer set recovers the per-instance benefit. A greedy rule that ranks layers by single-layer marginal effect recovers nearly all of the oracle's benefit, but both must score candidate layers against the gold answer, so neither can run at deployment; the rule instead becomes the target a prompt-only predictor is trained to reproduce. Our deployable recipe needs no label at inference: a per-instance layer ranker read off the prompt embedding, a classifier that infers the steering direction, and an adaptive gate that scores short steered passes against that inferred direction and steers no more layers than necessary. The recipe recovers most of the oracle's lift (the bulk on the stronger model, a clear majority on the harder one), never drives any trait-model pair below its unsteered alignment baseline on average, and largely avoids the fluency collapse that strong global selection incurs at higher layer counts. A mechanistic account, "direction over magnitude", explains the behavioural flip under a mis-directed global set, the output collapse from steering too many layers, and the ceiling of unsteerable inputs.
Muhammad Faishal Adly Nelwan, Alfan Farizki Wicaksono