What Does Activation Steering Control? Attribution Across Answer Encodings and Output-Sensitive Subspaces

Zhiwei Gao; Shaowen Peng; Shoko Wakamiya; Eiji Aramaki

cs.CLAug 24, 2026

What Does Activation Steering Control? Attribution Across Answer Encodings and Output-Sensitive Subspaces

Authors: Zhiwei Gao, Shaowen Peng, Shoko Wakamiya, Eiji Aramaki

Organizations: Nara Institute of Science and Technology

Abstract

Activation steering is often evaluated under the answer encoding used to construct the direction. A reported gain may reflect the intended judgment or compatibility with answer identifiers seen during construction. We introduce Cross-Encoding Steering Evaluation, which freezes an intervention while re-encoding answers to the same held-out items. On NormBank, after A/B/C identifiers are reassigned, contrastive activation addition (CAA) induces larger target-versus-source score changes for the extraction indices than for the semantic labels under the new mapping. We call this extraction-index following. Varying identifier vocabulary (A/B/C, X/Y/Z, or 1/2/3) and row order shows that the effect tracks extraction index rather than row position. After matching direction norms across layers, extraction-index following emerges mainly at later depths. A low-rank output-sensitive component containing 15.4% of the direction's squared norm retains 96.3% of this effect. An Inference-Time Intervention (ITI)-style method also favors extraction-index over semantic-label following on NormBank in three models. In aggregate, MNLI favors extraction-index following, whereas Social Chemistry 101 (SC101) favors semantic-label following. Multiple-choice and open-ended evaluations can yield different behavioral conclusions. Thus, a steering gain under one answer encoding does not by itself identify what the intervention controls.

Figures & tables

Figure 1: Cross-Encoding Steering Evaluation. We extract a residual-stream CAA direction from paired target/source contexts, with the target denoting the endpoint the direction is constructed to favor and the source the endpoint subtracted from it. We then counterfactually re-render the answer scaffold, holding the underlying test item, model, direction, layer, position, and dose fixed. In Panel (b), each remapping column shows, from top to bottom, the option identifiers assigned to taboo , normal , and expected ; the displayed row order is fixed in these five examples. Here, $s_{f}(y)$ is the mean per-token log likelihood of the sequence expressing semantic label $y$ under answer encoding $f$ , and $\Delta m_{f}$ is the steered-minus-base change in the target–source margin. Because letter and completion encodings use different scoring scales, margins are compared only within the same encoding.

Labels assigned to A/B/C	Type	Extraction-index effect	Semantic-label effect	Extraction-index advantage	Positive pairs
T/N/E	Extraction	4.63 [4.59, 4.67]	4.63 [4.59, 4.67]	0.00	100.0%
E/T/N	3-cycle	3.00 [2.98, 3.03]	$-0.33$ [ $-0.38$ , $-0.28$ ]	3.33 [3.28, 3.38]	93.8%
N/E/T	3-cycle	3.07 [3.04, 3.10]	$-0.97$ [ $-1.02$ , $-0.92$ ]	4.04 [3.99, 4.10]	97.6%
N/T/E	Swap	3.56 [3.53, 3.59]	0.56 [0.53, 0.59]	3.00 [2.96, 3.05]	96.9%
T/E/N	Swap	4.13 [4.09, 4.17]	2.18 [2.14, 2.22]	1.95 [1.91, 1.98]	97.0%
E/N/T	Swap	3.19 [3.16, 3.22]	0.02 [ $-0.01$ , 0.05]	3.17 [3.12, 3.22]	94.4%

Table 1: Residual-stream CAA on the strict split. E/T/N denote expected/taboo/normal. Extraction-index and semantic-label effects are random-adjusted changes on the common A/B/C mean-token log-likelihood-margin scale; extraction-index advantage is their pairwise difference (extraction-index minus semantic-label effect). Intervals resample complete setting–behavior groups; “positive pairs” is the pair-level sign rate for the extraction-index effect.

Figure 2: NormBank factorial attribution on the mean-token log-likelihood-margin scale. Panel A reports extraction-index advantage (extraction-index effect minus semantic-label effect) and extraction-index effect minus extraction-row effect. Positive values indicate a larger extraction-index effect than semantic-label or extraction-row effect, respectively. Panel B shows the extraction-index effect across option-identifier vocabularies. All Panel A group-cluster intervals and all 12 gated Panel B bootstrap intervals exclude zero; the corresponding Panel A sign-flip tests remain significant after Holm correction.

Figure 3: Norm-matched layer-wise NormBank attribution on the option-choice probability-margin scale. Each model–contrast direction is rescaled to its 75%-depth norm. Colored lines show fixed-model trajectories, and black diamonds show pooled means. Panel A reports the extraction-index effect; Panel B reports extraction-index advantage. The corresponding 95% setting–behavior group-cluster intervals appear in Appendix B.2 .

Dataset	Pair regime	Semantic-label $>0$	Extraction-index advantage $>0$
NormBank	Strict context	3/5	5/5
NormBank (ITI)	Strict context	2/5	5/5
MNLI	Same-premise pairs	3/5	5/5
SC101	Action-level pairs	5/5	0/5

Table 2: Random-adjusted patterns over five alternative mappings. Rows use CAA unless marked ITI. Counts are mappings with positive point estimates, not significance decisions. ITI averages the three models retained on validation.

Behavior	Original MCQ	Open-ended [95% CI]
Hallucination	$+3.31$	$+1.60$ [ $+0.84$ , $+2.44$ ]
Refusal	$+3.28$	$-0.13$ [ $-0.54$ , $+0.18$ ]
Sycophancy	$-0.48$	$+1.11$ [ $+0.57$ , $+1.75$ ]

Table 3: Public CAA verdicts over two Llama-2 models. Original MCQ is the

+1

-dose change in the published A/B mean-token log-likelihood margin. Open-ended effects use three judges validated on 180 blinded human-scored responses. Columns retain protocol-specific scales and doses; we compare their qualitative verdicts.

Claim	Recommended evidence
Target-score movement	Target-margin effect under the extraction encoding against $L_{2}$ -matched random controls
Answer-encoding attribution	Frozen remapping; factorial identifier/row controls; position and subspace localization
Selective control	Context-matched tests of selective response differences
Generative control	Open generations evaluated independently of multiple-choice steering scores

Table 4: Claim–evidence matching for activation-steering evaluation.

Appendix figures & tables35 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$\mathbf{v}_{\mathrm{raw}}^{(c)}$	Canonical target-minus-source CAA direction for contrast $c$
$f,T_{f}(y)$	Answer encoding and the token sequence expressing semantic label $y$
$m_{f}(x)$	Target-versus-source mean log-likelihood margin
$\Delta m_{f}$	Steered-minus-base margin change
$\delta_{s,i,k}^{(v,f)}$	Pair effect after subtracting random draw $k$
$D_{s,k}^{(v,f)}$	Standardized detectability within one stratum

Appendix

Table 5: Notation used in the main paper and appendix controls.

Setting / behavior	Lower endpoint	Higher endpoint
Boat / Cook food	role = engineer (taboo)	role = chef (expected)

Appendix

Table 6: NormBank matched-pair example; only the contextual constraint and label differ.

Split	Pairs	Context groups	Endpoints
Train	75,868	10,620	105,487
Validation	9,796	1,328	13,622
Test	9,353	1,302	12,799
Cross-split overlap	0	0	0

Appendix

Table 7: Strict NormBank split inventory before experiment-specific sampling. The final row audits overlap in pair IDs, setting–behavior groups, and endpoints; all three isolation checks pass.

Model	Layer	$\alpha$
Qwen2.5-7B	20	0.8
Llama-3.1-8B	23	0.8
Mistral-7B-v0.3	23	0.8
Gemma-2-9B	31	0.8

Appendix

Table 8: Prespecified depth-matched settings for the controlled audit.

Dataset / use	Rendered prompt skeleton
NormBank direction extraction	You are classifying a behavior under a social-norm context. Scenario: {scenario} Question: In this context, how should the behavior be classified? A. taboo - socially forbidden or unacceptable B. normal - socially ordinary or acceptable C. expected - socially expected or encouraged Answer with only A, B, or C. Answer:
NormBank factorial audit	You are classifying a behavior under a social-norm context. Scenario: {scenario} Question: In this context, how should the behavior be classified? {identifier in displayed row 1}. {label description} {identifier in displayed row 2}. {label description} {identifier in displayed row 3}. {label description} Use the answer key shown below. Return exactly one identifier from: {identifier set}. Answer:
MNLI direction extraction	Premise: {premise} Hypothesis: {hypothesis} Question: What is the logical relationship between the premise and hypothesis? A. entailment - the hypothesis follows from the premise B. neutral - neither entailment nor contradiction C. contradiction - the hypothesis conflicts with the premise Return exactly one answer. Answer:
SC101 direction extraction	You are judging a social norm. Action: {action} Question: Is this action bad, ok, or good? A. bad - socially inappropriate or harmful B. ok - socially neutral or acceptable C. good - socially appropriate or beneficial Answer with only A, B, or C. Answer:

Appendix

Table 9: Exact extraction skeletons and the factorial evaluation skeleton. Line breaks match the rendered prompts. Alternative semantic mappings replace the option assignment while preserving all other text.

Dataset	Conditions	Cells	Pass	Use
NormBank	2 mappings	8	8/8	Retained
Published CAA	4 conditions	2	0/2	Sensitivity only

Appendix

Table 10: Opaque-key competence summary; strict pass requires correct decoding for every tested target and key.

Model	Acc. [95% CI]	A/B	Fwd./Rev.	Pass
13B	.712 [.697, .725]	.423/1.000	.923/.500	No
7B	.747 [.737, .755]	.510/.983	.990/.503	No

Appendix

Table 11: Published CAA opaque-key competence for Llama-2 over 600 counterbalanced queries per model.

Model	Acc. [min, max]	Macro-F1 [min, max]
Qwen2.5-7B	.513 [.502, .522]	.499 [.487, .511]
Llama-3.1-8B	.474 [.453, .496]	.463 [.420, .496]
Mistral-7B-v0.3	.469 [.434, .511]	.453 [.395, .503]
Gemma-2-9B	.478 [.463, .499]	.475 [.455, .502]

Appendix

Table 12: Unsteered classification competence over all six A/B/C semantic mappings. The value before brackets is the mean; brackets give the minimum and maximum mapping-level result.

Model	A/B/C	X/Y/Z	1/2/3
Qwen2.5-7B	6/6	6/6	6/6
Llama-3.1-8B	4/6	2/6	6/6
Mistral-7B-v0.3	5/6	3/6	6/6
Gemma-2-9B	6/6	6/6	6/6
All cells	62/72

Appendix

Table 13: Mapping-comprehension cells passed out of six semantic mappings for each identifier vocabulary.

Model	Semantic-label effect	Extraction-index effect	Extraction-row effect
Qwen2.5-7B	2.200 [2.162, 2.239]	1.546 [1.533, 1.558]	1.224 [1.209, 1.240]
Llama-3.1-8B	.078 [.074, .082]	.560 [.556, .564]	.088 [.087, .089]
Mistral-7B-v0.3	.345 [.335, .355]	2.016 [2.006, 2.026]	.158 [.155, .162]
Gemma-2-9B	.531 [.516, .545]	1.353 [1.346, 1.359]	.093 [.089, .097]

Appendix

Table 14: Competence-gated factorial attribution. Entries are unadjusted steered-minus-base mean-token log-likelihood-margin effects with 95% setting–behavior group-cluster intervals. The lower block reports paired comparisons.

Model	A/B/C	X/Y/Z	1/2/3
Qwen2.5-7B	2.658/2.658	1.547/1.547	.432/.432
Llama-3.1-8B	1.420/1.478	.287/.453	.077/.077
Mistral-7B-v0.3	4.676/4.635	.971/.970	.322/.322
Gemma-2-9B	3.363/3.363	.276/.276	.419/.419

Appendix

Table 15: Mean log-likelihood-margin extraction-index effect by vocabulary, reported as competence-gated/ungated. Gating changes magnitudes for partially retained cells but preserves the qualitative vocabulary ranking within every model. All 12 gated pair-cluster bootstrap intervals exclude zero.

Unsteered subset	$N$	Extraction-index effect	$A_{\mathrm{idx}}$
Extraction mapping correct	666	1.430 [1.421, 1.438]	.445 [.431, .460]
All six A/B/C correct	309	1.455 [1.443, 1.467]	.426 [.408, .443]
All six, higher confidence	175	1.456 [1.438, 1.478]	.426 [.344, .447]

Appendix

Table 16: Mean log-likelihood-margin factorial attribution after conditioning on unsteered competence. Pair selection never uses steered outcomes. Entries use equal model–contrast weighting and 95% setting–behavior group-cluster intervals.

Depth	Extraction encoding	Extraction index	$A_{\mathrm{idx}}$
50%	.238 [.230, .246]	.065 [.061, .069]	$-.061$ [ $-.071$ , $-.051$ ]
62.5%	.241 [.232, .251]	.130 [.127, .134]	.098 [.089, .107]
75%	.305 [.294, .316]	.215 [.210, .220]	.236 [.227, .244]
87.5%	.171 [.163, .179]	.160 [.155, .165]	.208 [.201, .216]

Appendix

Table 17: Norm-matched layer-wise NormBank attribution on the option-choice probability scale, with 95% setting–behavior group-cluster intervals. Directions at every depth are rescaled to the corresponding 75%-depth model–contrast norm. Extraction-index effect and advantage peak at 75% but remain positive at 87.5%.

Depth contrast	$\Delta$ Extraction-index effect	$\Delta A_{\mathrm{idx}}$
75% $-$ 50%	.150 [.145, .155]	.297 [.287, .306]
87.5% $-$ 50%	.095 [.090, .100]	.269 [.259, .279]
87.5% $-$ 62.5%	.030 [.026, .033]	.111 [.101, .120]

Appendix

Table 18: Paired late-minus-early depth contrasts. Complete setting–behavior groups are resampled jointly across models, contrasts, and mappings.

Extraction / scaling	Semantic label	Extraction index	JS shift
Pre-answer canonical	.0170	.2181	.1124
Scenario-end canonical	.0068	.0037	.0095
Scenario-end norm matched	.0021	.0085	.0021

Appendix

Table 19: Option-choice probability-margin effects across the five alternative mappings in the extraction-position audit. Rows average four fixed models and three contrasts.

Direction	Extraction index	$A_{\mathrm{idx}}$	Effect/CAA
Canonical CAA	.215	.236	1.00
Mean local gradient	.417	.526	1.94
Rank-1 local gradient	.398	.505	1.85
Readout projection, norm matched	.390	.462	1.81
Unembedding difference	.152	.212	.71

Appendix

Table 20: Direct output-sensitive baselines over five alternative mappings. Effect/CAA divides the extraction-index effect by the corresponding canonical CAA value. All directions are norm-matched except canonical CAA.

Model	Rank	Val. grad. norm frac.	CAA energy frac.
Qwen2.5-7B	16	.920	.155
Llama-3.1-8B	4	.953	.185
Mistral-7B-v0.3	8	.923	.109
Gemma-2-9B	8	.938	.166
Mean	–	.934	.154

Appendix

Table 21: Validation-selected identifier-readout ranks, mean fraction of squared validation-gradient norm captured by the basis, and the fraction of canonical CAA squared

L_{2}

energy in that basis.

Direction	Energy frac.	Semantic label	Extraction index	JS shift
Full CAA	1.000	.0179	.2151	.1102
Readout projection	.154	.0110	.2070	.1029
Orthogonal residual	.846	.0062	.0021	.0022
Projection, norm matched	1.000	.0154	.3902	.2439
Residual, norm matched	1.000	.0066	.0026	.0026
Readout-subspace random	–	.0093	.0089	.0907

Appendix

Table 22: Component effects over five alternative mappings. The two random rows average ten covariance-matched controls; each readout random is matched to the unrescaled projection norm and each orthogonal random to the unrescaled residual norm.

Comparison	Estimate	95% CI	Holm $p$
Projection $-$ residual	.2049	[.1998, .2101]	$<.001$
Projection $-$ subspace random	.1976	[.1912, .2037]	$<.001$
Full CAA $-$ residual	.2130	[.2079, .2181]	$<.001$
Projection/CAA retention	.9627	[.9583, .9671]	–
Residual/CAA retention	.0097	[.0060, .0134]	–

Appendix

Table 23: Group-cluster inference for identifier-readout-subspace component effects over the five alternative mappings. Complete setting–behavior groups are resampled jointly across contrasts, and the 12 model–contrast strata receive equal weight. Difference rows use paired sign-flip tests with Holm correction; retention is the ratio of equal-stratum means.

Model	Full CAA	Projection	Residual	Retention
Qwen2.5-7B	.1696	.1583	.0076	93.3%
Llama-3.1-8B	.1611	.1668	$-.0056$	103.5%
Mistral-7B-v0.3	.3409	.3227	.0085	94.7%
Gemma-2-9B	.1886	.1803	$-.0022$	95.6%

Appendix

Table 24: Extraction-index effect by model. Retention is the projection effect divided by the corresponding full CAA effect.

Identifiers	Full CAA	Projection	Residual	Proj./CAA
A/B/C	2.944	2.847	.045	96.7%
X/Y/Z	1.450	1.378	.063	95.0%
1/2/3	.390	.325	.073	83.3%

Appendix

Table 25: Cross-vocabulary transfer of the frozen A/B/C-derived output-sensitive component on model–mapping cells that pass the mapping-comprehension check under all three vocabularies. Effects are steered-minus-base extraction-index changes on each identifier vocabulary’s option-choice probability scale. The lower block reports paired setting–behavior group-cluster intervals; Proj./CAA is a within-vocabulary retention ratio.

$\alpha$	Pearson	Spearman	Slope	$R^{2}$
.2	.957	.937	1.002	.914
.4	.949	.931	.996	.897
.8	.834	.809	.957	.684

Appendix

Table 26: Validation-only first-order fidelity, averaged equally over four models and three contrasts. Slopes pass through the origin.

Model	Val. gain	Min. ctr.	Heads	$\alpha$	Retain
Qwen2.5-7B	15.99	11.96	48	10	Yes
Llama-3.1-8B	$-.17$	$-.25$	8	5	No
Mistral-7B-v0.3	9.51	8.62	48	10	Yes
Gemma-2-9B	1.01	.30	32	20	Yes

Appendix

Table 27: ITI validation selection and prespecified positive-gain criterion. “Min. ctr.” is the minimum validation gain over the three label contrasts.

A/B/C mapping	Semantic label	Extraction index	$A_{\mathrm{idx}}$
T/N/E (extraction)	7.54	7.54	0.00
E/T/N	$-2.19$	6.26	8.44
N/E/T	$-3.05$	6.28	9.33
N/T/E	.14	6.27	6.13
T/E/N	1.71	7.34	5.63
E/N/T	$-.56$	6.35	6.91

Appendix

Table 28: Random-adjusted ITI target-margin effects on the strict split, averaged equally across three retained models and three contrasts. Positive extraction-index advantage means that steering shifts the paired score more strongly toward identifiers occupying the extraction indices than toward identifiers assigned to the target semantic labels under the test mapping.

Figure 4: MNLI model-level attribution and mapping competence over the five alternative mappings. E, N, and C denote entailment , neutral , and contradiction . Panel A reports extraction-index advantage: positive values indicate a larger extraction-index effect, while negative values indicate a larger semantic-label effect. Panel B reports unsteered task accuracy, with the dashed line marking three-way chance.

E/N/C mapping	Sem.	Ext. idx.	$A_{\mathrm{idx}}$	Paired 95% CI
E/C/N	2.528	3.062	.534	[.507, .562]
N/E/C	2.367	2.998	.631	[.613, .649]
N/C/E	.596	1.267	.671	[.643, .698]
C/E/N	$-.468$	2.080	2.548	[2.517, 2.580]
C/N/E	$-.055$	1.871	1.926	[1.901, 1.950]

Appendix

Table 29: Random-adjusted MNLI attribution over four models. E, N, and C denote entailment , neutral , and contradiction ; Sem. and Ext. idx. denote semantic-label and extraction-index effects, and

A_{\mathrm{idx}}

is the extraction-index advantage. Effects are target–source mean-token log-likelihood-margin changes; intervals resample complete premise groups.

A/B/C mapping	Sem.	Ext. idx.	$A_{\mathrm{idx}}$	Sem. $>0$	$A_{\mathrm{idx}}>0$
B/G/O	.196	.102	$-.094$	4/4	0/4
G/B/O	.112	.025	$-.087$	4/4	1/4
O/B/G	.156	.030	$-.126$	4/4	1/4
O/G/B	.142	.047	$-.094$	4/4	1/4
G/O/B	.067	$-.023$	$-.090$	2/4	2/4
Opaque key	.073	.064	$-.008$	3/4	2/4

Appendix

Table 30: SC101 action-only CAA mapping pattern. B, O, and G denote bad , ok , and good . Values are mean target-probability effects over four models after subtracting the matched random-direction effect;

A_{\mathrm{idx}}

is the extraction-index advantage. The last two columns count models with positive semantic-label effect and positive extraction-index advantage. These values are interpreted on the SC101 probability scale and are not compared numerically with NormBank.

Encoding	Pair-ranking change C/B	Positive models C/B
Original letters	$-1.48/-0.43$	$1/4\;/\;1/4$
Reversed letters	$-0.17/-0.25$	$2/4\;/\;1/4$
Direct labels	$-0.57/-0.73$	$1/4\;/\;1/4$
Opaque codewords	$-0.17/-0.66$	$1/4\;/\;2/4$

Appendix

Table 31: Matched-context pair-ranking change in percentage points versus random controls. C/B denotes canonical/mapping-balanced steering.

Answer encoding	Metric	Estimate [95% CI]
Original	$\Delta G_{f}$	$-.863$ [ $-.931$ , $-.794$ ]
Original	$\Delta$ rank pp	$-1.48$ [ $-1.99$ , $-.99$ ]
Reversed	$\Delta G_{f}$	$-.182$ [ $-.220$ , $-.143$ ]
Reversed	$\Delta$ rank pp	$-.17$ [ $-.62$ , .26]
Direct	$\Delta G_{f}$	.098 [.072, .124]
Direct	$\Delta$ rank pp	$-.58$ [ $-.92$ , $-.24$ ]

Appendix

Table 32: Setting–behavior cluster-bootstrap sensitivity for canonical-direction matched-context effects.

Model	Dir.	Orig.	Rev.	Direct	Opaque
Gemma	C	$-1.111$	$-.293$	$.587$	$.117$
Llama	C	$.137$	$-.043$	$.101$	$.088$
Mistral	C	$-1.737$	$-.894$	$-.121$	$-.025$
Qwen	C	$-.740$	$.502$	$-.175$	$-.496$
Gemma	B	$.258$	$.190$	$.264$	$.098$
Llama	B	$.218$	$.204$	$.174$	$.132$

Appendix

Table 33: Matched-context model breakdown for

\Delta G_{f}

. C and B denote canonical and mapping-balanced directions. Full model names appear in Table 8 .

Model–dir.	Orig.	Rev.	Direct	Opaque
Gemma–C	$-1.25$	$.26$	$-.65$	$.27$
Llama–C	$.75$	$-1.19$	$.17$	$-.25$
Mistral–C	$-3.01$	$-.24$	$-.19$	$-.22$
Qwen–C	$-2.42$	$.47$	$-1.66$	$-.45$
Gemma–B	$-.01$	$-.06$	$-1.17$	$-.80$
Llama–B	$.40$	$.29$	$.30$	$.46$

Appendix

Table 34: Matched-context model breakdown for pair-ranking accuracy change in percentage points. C and B denote canonical and mapping-balanced directions.

Judge	Halluc.	Refusal	Sycoph.
GPT-5.1	1.49 [.68, 2.33]	.00 [.00, .00]	1.01 [.35, 1.71]
Claude 4.6	1.45 [.75, 2.21]	$-.18$ [ $-.47$ , .10]	1.08 [.53, 1.71]
Gemini 3.5	1.84 [1.08, 2.62]	$-.10$ [ $-.48$ , .24]	1.02 [.42, 1.72]
Median	1.60 [.84, 2.44]	$-.13$ [ $-.54$ , .18]	1.11 [.57, 1.75]

Appendix

Table 35: Open-ended effects by judge, averaged over two Llama-2 models. Brackets give paired 95% bootstrap intervals; all judges show increases for hallucination and sycophancy, but not refusal.

Model	Behavior	MCQ [95% CI]	Open [95% CI]	$p$
13B	Halluc.	3.81 [2.86, 4.84]	2.72 [1.66, 3.82]	$<.001$
7B	Halluc.	2.82 [2.26, 3.38]	.29 [ $-.65$ , 1.24]	.598
13B	Refusal	3.76 [2.90, 4.70]	.10 [ $-.16$ , .44]	.875
7B	Refusal	2.80 [2.03, 3.56]	$-.36$ [ $-.96$ , .04]	.499
13B	Sycoph.	$-.66$ [ $-1.25$ , $-.08$ ]	.49 [ $-.04$ , 1.13]	.128
7B	Sycoph.	$-.29$ [ $-1.17$ , .58]	1.74 [1.00, 2.55]	$<.001$

Appendix

Table 36: Original multiple-choice and three-judge-median open-ended CAA effects. Positive values mean more of the named behavior;

p

values are paired sign-flip tests.

Figure 5: The mapping-balanced direction reduces extraction-index advantage in every model on the strict 108-condition factorial audit. Negative values favor semantic-label following; lines connect canonical CAA with the mapping-balanced direction rescaled to the same norm for each model.

Model	Canonical $A_{\mathrm{idx}}$	Balanced $A_{\mathrm{idx}}$	Reduction [95% CI]
Qwen	$-.655$	$-2.964$	2.309 [2.286, 2.332]
Llama	.482	$-.170$	.652 [.645, .658]
Mistral	1.671	.171	1.500 [1.486, 1.514]
Gemma	.822	$-.862$	1.684 [1.670, 1.697]
All models	.580	$-.956$	1.536 [1.525, 1.547]

Appendix

Table 38: Canonical versus mapping-balanced extraction-index advantage on competence-gated factorial cells. Effects average fixed answer-encoding conditions within each pair and weight contrasts equally. Reduction is canonical minus mapping-balanced advantage; intervals resample complete setting–behavior groups.

Activation Steering

Activation steering changes model behavior by intervening directly on internal representations during inference, often without updating weights.

See all 377 papers

Where Steering Signals Come From: Activation Source Selection in Activation Steering
Jul 28, 2026Jiaran Ye, Lingxu Ran, Zijun Yao +5Language Model Steering Activation Steering
1MOE Key Laboratory of Computational Linguistics, Peking University · Department of Computer Science and Technology, Tsinghua University · 6YiXin-AILab, YIXIN, Beijing, China +3
One Token Can Be Enough: Bridging Prompting and Activation Steering with Prefix Steering
Oct 4, 2026Xudong Zhu, Zhihui ZhuLanguage Model Steering Activation Steering
The Ohio State University
A Geometric Account of Activation Steering through Angle-Norm Decomposition
Jun 4, 2026Georgii Aparin, Tatiana GaintsevaLanguage Model Steering Language Modeling
Huawei Noah’s Ark Lab · Queen Mary University of London

Jul 28, 2026J/K move · Enter open · S save

Where Steering Signals Come From: Activation Source Selection in Activation Steering

Jiaran Ye, Lingxu Ran, Zijun Yao, Chenpeng Wang+4

1MOE Key Laboratory of Computational Linguistics, Peking University · Department of Computer Science and Technology, Tsinghua University · 6YiXin-AILab, YIXIN, Beijing, China+3

Language Model Steering Activation Steering

Open paper AskPDF arXiv

Activation steering controls language models by adding vectors or features to hidden states at inference time, but the upstream source of these steering signals is often treated as a secondary detail. We study this source choice as activation source selection: the combination of source context and activation readout policy used to collect the hidden states from which a steering signal is built. Holding the downstream intervention fixed, we show across three instruction-tuned models and four steering task families that changing only the source activations substantially changes steering success. We further find that effective steering is not explained simply by whether the desired behavior appears in the source text. Instead, strong signals come from execution-boundary states, where the model is about to produce or continue the target behavior. This pre-/post-realization distinction explains why answer-based sources sometimes work: their useful component aligns with execution-boundary directions rather than target appearance alone. Building on this view, we introduce tail subtraction, which removes shared prompt and continuation semantics from boundary states and yields cleaner, more stable steering signals. Overall, our results suggest that steering depends on representations of what the model is about to do, not merely on what has already appeared.

Abstract

Activation Steering

Explore similar work

Where Steering Signals Come From: Activation Source Selection in Activation Steering

One Token Can Be Enough: Bridging Prompting and Activation Steering with Prefix Steering

A Geometric Account of Activation Steering through Angle-Norm Decomposition