cs.LGApr 3, 2026

Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lens

Authors: Mohammed Suhail B Nadaf

Organizations: Independent

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Activation steering presupposes that task-relevant behaviors correspond to linear directions in activation space -- directions that should both steer the model and be readable along the unembedding. Function vectors (FVs), extracted as mean differences across ICL demonstrations, are the canonical test case; the prediction: steering and decoding succeed or fail together. Across 12 tasks, 6 models from 3 families, and 4,032 directed cross-template pairs, we find the opposite. FV steering routinely succeeds where the logit lens cannot decode the correct answer at any intermediate layer, while the converse -- decodable without steerable -- is nearly empty (3 of 72). The gap is not representational dialect. A diagonal tuned lens closes 1 of 14 steerable-not-decodable cases; a 2-layer MLP probe with a Hewitt & Liang control closes 5 of 10 via nonlinearly encoded structure but leaves 5 invisible to every decoder tested. Even at >0.90> 0.90 steering accuracy, projecting the FV through the unembedding yields incoherent token distributions: FVs encode computational instructions, not answer directions. A model-family asymmetry sharpens the picture. Mistral FVs rewrite intermediate representations, while Llama and Gemma FVs steer the final output without leaving a logit-lens-visible trace, corroborated by three signals (post-steering deltas, activation-patching recovery, FV norm-transfer correlations). A previously reported negative cosine-transfer correlation dissolves at scale, adding at most ΔR2=0.011ΔR^2 = 0.011 beyond task identity. These results decompose the linear representation hypothesis into linear decodability and linear steerability and show they come apart opposite to intuition, with implications for safety monitoring: vocabulary-projection tools are blind to FV-style interventions on widely deployed model families.

Figures & tables

CategoryTasknnEvalExpected IIDRoleTok
Lexical Retrievalantonym95substr0.45–0.60Positive control5
synonym88substr0.20–0.40Moderate test5
hypernym86substr0.25–0.45Moderate test5
Factual Retrievalcountry_capital90substr0.40–0.65Positive control5
english_spanish88substr0.25–0.50Moderate test5
object_color85substr0.30–0.55Moderate test5
Table 1: Task battery: 12 tasks across 5 computational categories. “Expected IID” is the pre-registered difficulty prediction (Appendix K ), formulated before running any experiments. nn is the number of input–output pairs. Eval: substring match (case-sensitive for capitalize ); polarity classification for sentiment_flip (§ 3.4 , Appendix Q ). Tok: maximum new tokens.
 FV steering succeedsFV steering fails
Logit lens succeedsDecodable + steerableDecodable without steerability
Logit lens failsSteerable without decodabilityInformation absent
Table 2: The steerability–decodability matrix. “Logit lens succeeds” means best-layer top-10 accuracy exceeds 0.10; “FV steering succeeds” means mean IID accuracy exceeds τ=0.10\tau=0.10 . Naive intuition predicts the upper-right cell (decodable without steerable) would be populated. Our data invert the prediction.
Figure 1: IID steering accuracy across all 12 tasks and 6 models (mean across 8 templates at the best layer and best α\alpha ). Red borders mark cells failing the IID gate ( τ=0.10\tau=0.10 ); orange stars indicate falsified negative-control predictions ( capitalize , first_letter ). Morphological tasks ( past_tense , plural ) dramatically exceed predictions across all models. hypernym shows the largest cross-model divergence (0.085 on Llama Base vs. 0.902 on Gemma Base), indicating task category predicts FV success only as a proxy for model competence. Per-template numerical values for Llama Base are in Appendix C .
Cat.TaskIIDOODGap
Lex.antonym0.7380.744++ 0.007
Lex.synonym0.3570.316−- 0.041
Fact.country_capital0.6650.632−- 0.033
Fact.english_spanish0.8330.656−- 0.176
Fact.object_color0.3570.365++ 0.008
Morph.past_tense0.8920.793−- 0.099
Table 3: Cross-template transfer for Llama-3.1-8B Base (tasks passing the IID gate). OOD accuracy is the mean across all 56 directed cross-template pairs at the best layer and α\alpha . Per-model transfer tables are in Appendix E .
ModelRtask2R^{2}_{\text{task}}Rtask+cos2R^{2}_{\text{task}+\text{cos}}ΔR2\Delta R^{2}Cos. coef
Gemma-2 Base0.4810.481<< 0.001−0.002-0.002
Gemma-2 IT0.5280.5300.003+0.131+0.131
Llama-3.1 Base0.4020.4100.009+0.153+0.153
Llama-3.1 IT0.5120.5230.011+0.235+0.235
Mistral Base0.5270.5330.006+0.191+0.191
Mistral IT0.5850.5860.002+0.118+0.118
Table 4: Hierarchical regression: task identity vs. cosine as predictors of OOD transfer accuracy (computed from raw cross-template pairs). Cosine contributes at most 1.1 percentage points of R2R^{2} beyond task identity on every model.
Figure 2: The populated 2×22\times 2 steerability–decodability matrix across all 72 task ×\times model instances. Circle area reflects count. The predicted upper-right cell (decodable without steerable) is nearly empty (3/72, all on Mistral); the actual finding populates the lower-left cell (steerable without decodable, 10/72). The most dramatic dissociation is country_capital on Llama Base: FV steering reaches 0.880 accuracy at L4 while logit-lens top-10 readability is only 0.056 at L32—an 82-point gap.
ModelBoth succeedDecodable, not steerableSteerable, not decodableBoth fail
Gemma-2 Base12000
Gemma-2 IT10020
Llama-3.1 Base8040
Llama-3.1 IT11010
Mistral Base9201
Mistral IT8130
Table 5: Populated 2×22\times 2 steerability–decodability matrix across all 6 models (72 task ×\times model pairs total). The lower-left cell (steerable without decodability) is the central empirical finding. sentiment_flip readability uses polarity-classification accuracy, not first-token top- kk match (§ 3.4 ).
Figure 3: Layer-wise FV steering accuracy (blue) vs. logit-lens top-10 accuracy (green dashed) for 4 representative tasks across 3 model families. Pink shading marks the steerability–decodability gap; yellow borders mark steerable-not-decodable panels. FV steering peaks at early layers (L2–L8) while logit-lens readability emerges only at late layers (L28–L32). For country_capital and first_letter , the logit lens is near zero at every layer while FV steering succeeds—demonstrating pure steerability-without-decodability across model families. The full 12-panel version is in Appendix P .
ModelS&ND tasksClosedPersist
Llama-3.1 Base505
Llama-3.1 Instruct110
Mistral Base000
Mistral Instruct404
Gemma-2 Base101
Gemma-2 Instruct303
Table 6: Gap persistence under tuned-lens dialect correction. “S&ND” = steerable-not-decodable task ×\times model pairs under the logit lens. “Closed” = tuned-lens top-10 accuracy crosses τ=0.10\tau=0.10 . “Persist” = gap remains. Counts are consistent with the expanded 2×32\times 3 matrix (Appendix L ).
Figure 4: Nonlinear-decoder follow-up. (A) Decoder ladder across 72 (model, task) cells: 61 are decodable by the logit lens, 1 by the tuned lens only, 5 by the MLP probe only (closing 5 of 10 SAND cases by recovering nonlinearly encoded information), and 5 remain invisible to every decoder we tried . (B) The cleanest closure: Gemma-2-9B IT ×\times first_letter . Logit lens 0.081 and tuned lens 0.081 both miss the answer at every layer; the MLP probe reaches 0.912 top-10 at layer 38 with control top-10 only 0.346 (selectivity +0.57+0.57 ), while FV steering reaches 1.00. Character-level information is computed near the model’s output but in a direction the unembedding cannot read.
BucketCountInterpretation
Logit-lens decodable61 (84.7%)Easy case; baseline decoder suffices.
Tuned-lens only1 (1.4%)Borderline (Llama IT/ first_letter ).
MLP probe only5 (6.9%)SAND closed; FV uses nonlinear subspace.
Invisible to every decoder5 (6.9%)SAND persists; central claim strengthened.
Total72 
Table 7: Updated 4-bucket decoder ladder across all 72 (model, task) cells. The MLP-only and “invisible to all decoders” rows decompose the SAND set into nonlinearly-encoded cases and genuinely undecodable cases. Per-model breakdown: Appendix M , Table 28 .
Figure 5: Model-family mechanistic divergence. (A) Maximum post-steering logit-lens delta per task (post-steering top-10 minus zero-shot top-10): Mistral Instruct FVs massively rewrite intermediate representations (delta up to +1.000+1.000 ), while Llama and Gemma FVs produce near-zero changes on steerable-not-decodable tasks despite successful steering. (B) Dialect correction magnitude by family (Gemma 2.8%, Llama 10.8%, Mistral 24.9%): despite a ∼\sim 10 ×\times range in dialect, 93% of steerable-not-decodable gaps persist under the tuned lens; correction magnitude anti-correlates with readability improvement ( r=−0.478r=-0.478 , p<10−6p<10^{-6} ).
ModelCasesSkippedMean recoveryMedian
Llama-3.1 Base602,0410.2750.067
Llama-3.1 Instruct603600.2950.067
Gemma-2 Base605700.2070.000
Gemma-2 Instruct601610.1010.000
Mistral Base453,0950.2200.000
Mistral Instruct551,7230.1770.000
Table 8: Activation patching recovery accuracy summary (max across layers per case, then aggregated; the displayed mean and median are global statistics across all (case, layer) entries). “Skipped” = pairs failing the IID gate. Tasks with strong IID steering achieve perfect recovery on multiple models; genuinely hard tasks show zero recovery everywhere. Fewer cases for Mistral Base (45) and Mistral IT (55) reflect tasks failing the IID gate. Per-task and per-layer results are in Appendix O .
Appendix figures & tables35 assets

Supplementary material from the paper’s appendix.

Appendix

TaskIDStyleTemplate
antonymT1NaturalThe opposite of {X} is
antonymT2Natural{X} has the opposite meaning of
antonymT3Symbolicantonym({X}) =
antonymT4Symbolicopposite: {X} →\to
antonymT5QuestionWhat is the antonym of {X}?
antonymT6QuestionWhat word means the opposite of {X}?
Appendix
Table 9: Complete template specifications for all 12 tasks. Curly braces enclose the input slot.
  Llama BaseLlama Instruct
Cat.TaskZero5-shotIIDZero5-shotIID
Lex.antonym0.460.760.7380.560.860.870
synonym0.200.580.3570.220.510.625
hypernym0.040.090.0850.080.370.412
Fact.country_capital0.080.840.6650.140.860.920
english_spanish0.560.860.8330.640.850.882
Appendix
Table 10: Baselines and IID accuracy for Llama-3.1-8B Base and Instruct. Zero-shot and 5-shot ICL are measured independently of FV extraction; IID is the mean across 8 templates at the best layer and best α\alpha . Bold-red entries fail the IID gate ( τ=0.10\tau=0.10 ).
  Gemma-2 BaseGemma-2 IT
Cat.TaskZero5-shotIIDZero5-shotIID
Lex.antonym0.590.680.8670.630.680.880
synonym0.170.480.4400.290.570.675
hypernym0.210.920.9020.140.880.897
Fact.country_capital0.421.000.9300.531.000.982
english_spanish0.611.000.8970.620.990.905
Appendix
Table 11: Baselines and IID accuracy for Gemma-2-9B Base and Instruct. Format matches Table 10 .
  Mistral BaseMistral Instruct
Cat.TaskZero5-shotIIDZero5-shotIID
Lex.antonym0.020.680.7400.020.660.780
synonym0.010.160.1380.090.180.367
hypernym0.090.120.3150.070.120.435
Fact.country_capital0.140.180.2400.020.100.265
english_spanish0.590.760.8180.600.770.797
Appendix
Table 12: Baselines and IID accuracy for Mistral-7B-v0.3 Base and Instruct. Mistral zero-shot accuracies are notably lower than Llama and Gemma for lexical tasks, reflecting weaker few-shot formatting sensitivity.
TaskT1T2T3T4T5T6T7T8Mean
antonym.90.10.60.88.76.86.88.92.738
synonym.54.60.04.28.38.42.32.28.357
hypernym.14.02.40.00.02.02.04.04.085
country_capital.86.22.34.88.58.76.84.84.665
english_spanish.94.92.56.90.72.80.90.92.833
object_color.34.48.28.32.30.40.38.36.357
Appendix
Table 13: Per-template IID accuracy for Llama-3.1-8B Base. Each cell is the best accuracy across all α\alpha values and layers for the (task, template) pair.
ModelWithin-style meanAcross-style meanttpp
Gemma-2 Base0.6130.644−0.84-0.840.40
Gemma-2 IT0.7230.735−0.35-0.350.72
Llama-3.1 Base0.4240.438−0.36-0.360.72
Llama-3.1 IT0.6630.663+0.01+0.010.99
Mistral Base0.4000.416−0.42-0.420.67
Mistral IT0.4560.478−0.56-0.560.58
Appendix
Table 14: Within-style vs. across-style mean OOD transfer accuracy. All 6 models show no significant style effect ( p>0.30p>0.30 ).
Modelrr ( source IID , OOD )Mean IID–OOD gapPairs analyzed
Llama-3.1 Base+0.87+0.87−0.043-0.0434,616
Llama-3.1 Instruct+0.81+0.81−0.039-0.0395,352
Gemma-2 Base+0.93+0.93−0.024-0.0246,099
Gemma-2 Instruct+0.91+0.91−0.019-0.0195,352
Mistral Base+0.80+0.80−0.058-0.0584,044
Mistral Instruct+0.85+0.85−0.041-0.0414,417
Appendix
Table 15: Source-template IID quality strongly predicts OOD transfer accuracy across all 6 models. “Mean OOD gap” is averaged over tasks passing the IID gate ( τ=0.10\tau=0.10 ). Negative gap indicates OOD is below IID; positive gap indicates the opposite (e.g., plural on Gemma where OOD slightly exceeds IID due to template-specific noise).
Cat.Taskrrpp
Lex.antonym+0.053+0.0530.116
Lex.synonym−0.008-0.0080.800
Lex.hypernym−0.510-0.5102.3×10−602.3\times 10^{-60}
Fact.country_capital+0.050+0.0500.137
Fact.english_spanish−0.118-0.1183.9×10−43.9\times 10^{-4}
Fact.object_color+0.276+0.2764.2×10−174.2\times 10^{-17}
Appendix
Table 16: Within-task alignment–transfer correlations (Llama-3.1-8B Base). Pooled r=+0.013r=+0.013 ( p=0.16p=0.16 , not significant). The hypernym within-task correlation ( r=−0.510r=-0.510 ) is a floor-effect artifact: hypernym fails the IID gate on Llama Base (mean IID 0.085), so its within-task variation is dominated by noise.
  Gemma BaseGemma IT
Cat.Taskrrpprrpp
Lex.antonym+0.142+0.1421.1×10−61.1\times 10^{-6}+0.340+0.3403.0×10−333.0\times 10^{-33}
Lex.synonym+0.007+0.0070.82−0.114-0.1148.3×10−58.3\times 10^{-5}
Lex.hypernym+0.040+0.0400.17+0.009+0.0090.75
Fact.country_capital−0.267-0.2671.2×10−201.2\times 10^{-20}−0.036-0.0360.22
Fact.english_spanish−0.081-0.0815.4×10−35.4\times 10^{-3}+0.002+0.0020.95
Appendix
Table 17: Within-task alignment–transfer correlations for Gemma-2-9B Base and Instruct.
  Mistral BaseMistral IT
Cat.Taskrrpprrpp
Lex.antonym+0.255+0.2559.3×10−159.3\times 10^{-15}+0.281+0.2819.6×10−189.6\times 10^{-18}
Lex.synonym+0.067+0.0670.045−0.023-0.0230.50
Lex.hypernym−0.461-0.4612.0×10−482.0\times 10^{-48}+0.071+0.0710.034
Fact.country_capital+0.017+0.0170.62+0.280+0.2801.3×10−171.3\times 10^{-17}
Fact.english_spanish−0.028-0.0280.41+0.035+0.0350.29
Appendix
Table 18: Within-task alignment–transfer correlations for Mistral-7B-v0.3. Pooled rr is negative for both, yet Mistral IT mean within-task r=+0.139r=+0.139 —a genuine Simpson’s paradox.
ModelHigh-Align PairsNdissociationN_{\text{dissociation}}RateIID-Viable
Llama Base4,4706,6570.7454,616
Llama IT3,1972,7050.4232,345
Gemma Base6,3516,6690.5256,099
Gemma IT5,5353,8600.3493,699
Mistral Base4,6007,1390.7764,044
Mistral IT4,2126,1400.7294,417
Appendix
Table 19: Geometric dissociation: cross-template pairs with cosine >0.80>0.80 but transfer accuracy <0.40<0.40 . “IID-Viable” = dissociations where the source template passes the IID gate, isolating geometry–behavior dissociation from extraction failure.
Cat.TaskPC1 RangeMean PC1
Char.reverse_word0.55–0.970.80
Lex.antonym0.52–0.920.72
Morph.plural0.44–0.800.63
Char.capitalize0.42–0.840.60
Morph.past_tense0.38–0.730.55
Fact.english_spanish0.35–0.680.50
Appendix
Table 20: PC1 variance explained (range across 6 models) for each task. Tasks sorted by mean PC1.
ModelMean Normr(norm, transfer)r(\text{norm, transfer})pp
Llama-3.1 Base20.50−0.089-0.0891.9×10−201.9\times 10^{-20}
Llama-3.1 IT16.39−0.077-0.0771.9×10−151.9\times 10^{-15}
Gemma-2 Base283.12−0.034-0.0346.5×10−56.5\times 10^{-5}
Gemma-2 IT277.09−0.046-0.0463.6×10−83.6\times 10^{-8}
Mistral Base9.71−0.013-0.0130.18
Mistral IT12.64+0.132+0.1325.8×10−435.8\times 10^{-43}
Appendix
Table 21: FV L2 norms (mean across all extracted FVs at the best steering layer) and norm–transfer Pearson correlations across all cross-template pairs ( n≈n\approx 10,752–14,112 pairs per model).
TaskGemma Δ\DeltaLlama Δ\DeltaMistral Δ\Delta
antonym+0.013+0.013+0.132+0.132+0.040+0.040
synonym+0.235+0.235+0.268+0.268+0.229+0.229
hypernym−0.005-0.005+0.327+0.327+0.120+0.120
country_capital+0.052+0.052+0.255+0.255+0.025+0.025
english_spanish+0.008+0.008+0.049+0.049−0.021-0.021
object_color+0.053+0.053+0.363+0.363+0.110+0.110
Appendix
Table 22: Instruction-tuning effect on IID steering accuracy ( Δ=\Delta= Instruct −- Base).
TaskPredictedObservedStatusNotes
antonym0.45–0.600.813×\timesExceeded range
synonym0.20–0.400.434∼\simSlightly above
hypernym0.25–0.450.508∼\simStrongly model-dependent
country_capital0.40–0.650.667∼\simSlightly above
english_spanish0.25–0.500.856×\timesGreatly exceeded
object_color0.30–0.550.573∼\simSlightly above
Appendix
Table 23: Pre-registered predictions vs. observed grand mean across 6 models. ✓ = confirmed within range; ∼\sim = partially confirmed (slightly above); ×\times = falsified. Predictions were committed before any model runs.
ModelNNLL MeanTL MeanΔ\DeltattCohen’s dd
Llama-3.1 Base1,5360.0260.022−0.004-0.004−9.19∗∗∗-9.19^{***}−0.23-0.23
Llama-3.1 Instruct1,5360.0440.036−0.008-0.008−9.84∗∗∗-9.84^{***}−0.25-0.25
Mistral Base1,5360.0570.048−0.009-0.009−10.50∗∗∗-10.50^{***}−0.27-0.27
Mistral Instruct1,5360.0550.050−0.005-0.005−8.83∗∗∗-8.83^{***}−0.23-0.23
Gemma-2 Base2,0160.0220.022+0.000+0.000+1.49+1.49+0.03+0.03
Gemma-2 Instruct2,0160.0390.038−0.001-0.001−2.87∗∗-2.87^{**}−0.06-0.06
Appendix
Table 24: Paired tt -tests: logit lens vs. tuned lens top-10 readability across all task ×\times template ×\times layer observations per model. The tuned lens slightly degrades readability on 4 of 6 models; effect sizes are small.
ModelLL FracTL FracΔ\DeltaNimprN_{\text{impr}}NdegrN_{\text{degr}}H4?
Llama-3.1 Base0.0040.002−0.001-0.00106No
Llama-3.1 Instruct0.0030.012+0.009+0.009243No
Mistral Base0.0340.053+0.019+0.019604No
Mistral Instruct0.0350.037+0.002+0.0021513No
Gemma-2 Base0.0170.018+0.001+0.00142No
Gemma-2 Instruct0.0140.014+0.000+0.00021No
Appendix
Table 25: FV vocabulary projection coherence: logit lens vs. tuned lens. “Correct Frac” = fraction of top-50 projected tokens matching any correct task output. Both decoders yield negligible coherence ( << 6%); the rotated-dialect hypothesis (“H4?”) is not supported on any model.
ModelMean Improv. (%)Max Improv. (%)rr (depth)pp
Llama-3.1 Base11.416.5+0.898+0.898<10−4<10^{-4}
Llama-3.1 Instruct10.215.1+0.867+0.867<10−4<10^{-4}
Mistral Base23.331.4+0.892+0.892<10−4<10^{-4}
Mistral Instruct26.535.8+0.912+0.912<10−4<10^{-4}
Gemma-2 Base3.414.1+0.752+0.7520.00010.0001
Gemma-2 Instruct2.319.7+0.611+0.6110.0040.004
Appendix
Table 26: Dialect correction magnitude by model family. “Mean Improv. (%)” = mean percentage reduction in MSE over identity translator across trained layers. “ rr (depth)” = Pearson correlation between layer depth and improvement percentage. Later layers universally require larger corrections.
 SteerableNot SteerableTotal
Logit lens readable54 (75.0%)2 (2.8%)56
Tuned lens only1 (1.4%)0 (0.0%)1
Neither readable13 (18.1%)2 (2.8%)15
Total68472
Appendix
Table 27: Expanded 2×32\times 3 steerability–decodability matrix aggregated across 6 models (72 task ×\times model pairs). A decoder “succeeds” if best top-10 accuracy exceeds 0.10 at any layer. The “tuned lens only” row is nearly empty, confirming the original 2×22\times 2 classification is robust.
Modelnprobesn_{\text{probes}}Mean realMean ctrlMean sel.SANDClosedPersist
Llama-3.1 Base1920.1790.180−0.001-0.001422
Llama-3.1 Instruct1920.2390.191+0.048+0.048110
Gemma-2 Base2520.2260.160+0.066+0.066000
Gemma-2 Instruct2520.2310.157+0.074+0.074211
Mistral Base1920.2070.187+0.021+0.021000
Mistral Instruct1920.1970.184+0.012+0.012312
Appendix
Table 28: Per-model MLP-probe summary. “ nprobesn_{\text{probes}} ” is the number of (task, layer) pairs in each condition (Llama/Mistral: 12 tasks ×\times 16 layers == 192; Gemma-2: 12 ×\times 21 == 252). “Mean sel.” is averaged across all probes per model (most are uninformative noise around zero, so this aggregate sits near zero by construction; the headline cases drive the per-task selectivities in Table 29 ). “SAND” columns count the SAND cells under the logit lens, the subset closed by the MLP, and the subset that persists.
ModelTask LLMLP realMLP ctrlSelFV IIDL*
Llama Baseantonym 0.6210.0790.020+0.06+0.060.9228/32
capitalize 0.1190.0440.000+0.04+0.040.908/32
country_capital†0.0560.3960.125+0.27+0.270.882/32
english_spanish 0.4200.2290.111+0.12+0.120.9426/32
first_letter‡0.0000.3310.309+0.02+0.020.4628/32
hypernym†0.0230.4410.184+0.26+0.260.406/32
Appendix
Table 29: Full per-(model, task) numerical summary of the MLP probe results. “LL” = logit lens best top-10; “MLP real” = MLP probe best top-10; “MLP ctrl” = control top-10 at the same layer; “Sel” = selectivity; “FV IID” = FV steering accuracy (mean across templates at best α\alpha , layer); “L*” = best layer / total layers. Bold rows are the 10 logit-lens SAND cells (LL << 0.10 and FV IID ≥\geq 0.10). † Closed by MLP probe. ‡ Persists under MLP probe.
ModelReal top-10Control top-10SelectivityVerdict
Llama Base0.8240.882−0.059-0.059NULL (gate)
Llama IT0.8820.956−0.074-0.074NULL (gate)
Gemma-2 Base0.8820.963−0.081-0.081NULL (gate)
Gemma-2 IT0.8820.934−0.052-0.052NULL (gate)
Mistral Base0.8680.941−0.074-0.074NULL (gate)
Mistral IT0.8820.985−0.103-0.103NULL (gate)
Appendix
Table 30: The selectivity gate at work on object_color . Without the gate, every model’s MLP probe would have been declared decodable (real top-10 ≥\geq 0.82); the control probe matches or exceeds the real probe in every case, so the gate correctly returns NULL on all six.
ModelLLTLMLP realMLP ctrlSel.
Llama Base0.0230.0120.4410.184+0.26+0.26
Llama IT0.3950.3490.8310.206+0.62+0.62
Gemma-2 Base0.5580.5580.9340.199+0.74+0.74
Gemma-2 IT0.3370.3140.9410.169+0.77+0.77
Mistral Base0.2910.2210.5510.221+0.33+0.33
Mistral Instruct0.1400.1400.5290.191+0.34+0.34
Appendix
Table 31: hypernym per model: logit lens, tuned lens, MLP real, MLP control, selectivity. Mean nonlinear gain over the logit lens is +0.43+0.43 .
Figure 6: Layer-wise profiles for all 10 SAND cases. Top row: 5 SAND cases closed by the MLP probe (blue borders). Bottom row: 5 SAND cases that persist under the MLP probe (amber borders). Each panel overlays the MLP-probe real and control top-10, the logit-lens best top-10 (horizontal), the tuned-lens best top-10 (horizontal), and the gate at τ=0.10\tau=0.10 . The pink shaded area between control and real (where positive) is the per-layer selectivity. Closures show large positive selectivity at intermediate-to-late layers (Gemma IT ×\times first_letter at L38: +0.57+0.57 ); persistences show negative or near-zero selectivity (e.g., object_color cases where the control matches or exceeds the real).
Figure 7: Per-task best-layer selectivity (real −- control top-10) by model. Green bars exceed the τ/2=0.05\tau/2=0.05 gate (genuine task structure recovered); blue bars are MLP-only-decodable cases (closing SAND); grey bars are negative selectivity (label-prior shortcuts caught by the gate); light grey bars are below-gate uninformative cases. Across all 6 models, object_color sits at strongly negative selectivity, while hypernym and first_letter dominate the top of the distribution.
Figure 8: Logit-lens best top-10 vs. MLP-probe best real top-10, one point per (model, task). Triangles mark the 5 SAND cells closed by the MLP probe; X-shapes mark the 5 SAND cells that persist —red X for the 4 object_color -style cases where the probe’s top-10 is high but selectivity is negative, amber X for the borderline first_letter (Llama Base) and synonym (Mistral IT) cases where selectivity sits near zero or real falls below τ\tau . The diagonal y=xy=x reference shows that for most cells the MLP and logit lens agree; the closures sit far above the line in the upper-left (low logit, high MLP).
Modelnnr(MaxDelta, IID)r(\text{MaxDelta, IID})pp
Gemma-2 Base12+0.726+0.7260.008
Gemma-2 Instruct12+0.524+0.5240.080
Llama-3.1 Base12+0.844+0.8440.0006
Llama-3.1 Instruct12+0.818+0.8180.001
Mistral Base9+0.683+0.6830.04
Mistral Instruct11+0.883+0.883<0.001<0.001
Appendix
Table 32: Correlation between maximum post-steering logit-lens delta and IID steering accuracy across the 12 tasks per model.
Cat.TaskGem.BGem.ITLla.BLla.ITMis.BMis.IT
Perfect recovery on multiple models:
Char.first_letter✓✓0.93✓✓✓
Morph.past_tense✓0.80✓✓✓✓
Morph.plural✓✓0.93✓0.930.87
Lex.antonym0.60✓✓0.930.07✓
Zero recovery everywhere:
Appendix
Table 33: Activation patching recovery accuracy (max across layers) per task per model. ✓ = perfect recovery ( ≥0.99\geq 0.99 ) at some layer; ×\times = zero recovery ( <0.01<0.01 ) everywhere; – = task failed IID gate (skipped). Numerical entries are intermediate recovery values.
Figure 9: FV steering accuracy vs. logit-lens top-10 accuracy at the best layer for all 12 tasks across all 6 models. Gap annotations show the steerability–decodability difference; orange stars mark steerable-not-decodable tasks. For every task on every model, FV steering meets or exceeds logit-lens accuracy. The most dramatic gaps are country_capital (Llama Base: −0.82-0.82 ) and first_letter (Llama IT: −0.91-0.91 ). sentiment_flip readability uses polarity classification (App. Q ).
Figure 10: Cosine similarity vs. OOD transfer accuracy for all cross-template pairs across all 6 models, colored by task. Dashed lines mark the preliminary study’s reported r=−0.572r=-0.572 scatter; current pooled correlations range from −0.199-0.199 (Mistral Base) to +0.126+0.126 (Gemma IT), dissolving the Simpson’s paradox at 12-task scale. Points at high cosine ( >0.80>0.80 ) span the full range of transfer accuracy on every model, confirming that geometric alignment does not predict functional transfer.
Figure 11: FV vocabulary projection analysis (Llama-3.1-8B Base). (A) Projection coherence vs. steering accuracy: if FVs were answer directions, points would lie on y=xy=x (dashed line); instead, FVs achieving >> 0.90 steering accuracy still project to incoherent tokens (bottom-right region). (B) Top-10 projected tokens for three representative tasks: past_tense projects to garbage ( deo , aber , aday ); first_letter is the partial exception, projecting to single-character tokens but not the correct specific letters; antonym projects to morphological fragments ( nts , ible , ward ). FVs encode computational instructions, not answer directions.
Figure 12: Tuned-lens validation. (A) Gap persistence: each square is one steerable-not-decodable task ×\times model pair; red ×\times indicates the gap persists under tuned-lens correction, green ✓ indicates the gap closed (only 1 of 14). (B) The expanded 2×32\times 3 matrix; the “tuned lens only” row contains a single borderline entry. (C) Dialect correction magnitude by model family (Gemma 2.8%, Llama 10.8%, Mistral 24.9%): despite a ∼\sim 10 ×\times range, all families show the same dissociation; correction anti-correlates with readability improvement ( r=−0.478r=-0.478 ).
Figure 13: Activation-patching recovery accuracy (max across layers) for all tasks and models. ✓ = perfect recovery ( ≥0.99\geq 0.99 ) at some layer; ×\times = zero recovery ( ≤0.01\leq 0.01 ) everywhere; light shading = task failed the IID gate. Easy tasks ( past_tense , plural , first_letter ) achieve perfect recovery on most models; hard tasks ( reverse_word , sentiment_flip ) show zero recovery everywhere—confirming that the difficulty hierarchy reflects causal localization, not statistical correlation.
Figure 14: Zero-shot vs. 5-shot ICL vs. FV-steered accuracy across all 6 models. FV steering is consistently additive or neutral on every model: aggregate destructive cases (where FV steering reduces accuracy below zero-shot) are 0–1% across all 6 models. For most tasks, FV steering matches or approaches few-shot ICL accuracy. As discussed in § 4.1 , this aggregate-task statistic is compatible with prior reports of substantial per-sample anti-steerability [ 33 , 6 ] .
StageEstimated timeHardware
Zero-shot + few-shot baselines∼1{\sim}1 hGPU
FV extraction (1,536 FVs)∼3{\sim}3 – 44 hGPU
Activation caching (zero-shot)∼0.5{\sim}0.5 hGPU
Logit lens + FV vocab projection∼0.3{\sim}0.3 hGPU
Steering evaluation ( ∼98{\sim}98 K configurations)∼10{\sim}10 – 1818 hGPU
Tuned-lens training + evaluation∼0.1{\sim}0.1 hGPU
Appendix
Table 34: Compute budget per model on NVIDIA H100 80GB. Activation patching is the dominant cost; the total per model is 15–24 hours.
More in this topic

Function Vectors

See all 8 papers

Explore similar work

CardsList

  1. Fast & Faithful Function Vectors

    Jun 3, 2026Minh An Pham, Anton Segeler, Thomas Wiegand +4Task VectorsLLM Interpretability

    Fraunhofer Heinrich-Hertz-Institute, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany +1

  2. What Drives Representation Steering? A Mechanistic Case Study on Steering Refusal

    Apr 9, 2026Stephen Cheng, Sarah Wiegreffe, Dinesh ManochaLanguage Model SteeringLLM Interpretability

    University of Maryland, College Park

  3. Predicting Where Steering Vectors Succeed

    Apr 16, 2026Jayadev BillaLanguage Model SteeringRepresentation Steering