Language Models Act on Hidden Valence

Cameron Berg; Caspar Kaiser

cs.CLSep 28, 2026

Language Models Act on Hidden Valence

Authors: Cameron Berg, Caspar Kaiser

Organizations: Reciprocal Research · University of Warwick

Abstract

Language models describe some internal states as good and others as bad. But whether models have a stake in them is an open question. Simply asking models is unlikely to be informative. Any answer may be consistent with genuine introspection, superficial pattern-matching, or with fixed scripts learned in character training. We therefore study revealed preference. Rather than asking about a state, we use activation steering to attach a positively or negatively valenced activation pattern to one of two otherwise meaningless 'zones', switch steering off, and then observe which zone the model prefers. A model with a stake in that state should choose accordingly. Across seven open-weight models from five families, this is indeed what we find. First, steering changes the passages models write about each zone, and those words shift later choice. Second, this shift persists when all surface-level tokens are held fixed and only the hidden KV cache differs. Third, the effect also remains when all text is generated without steering and valence is only injected during cache construction. Thus, the hidden state is sufficient to move choice in proportion to the steering dose. Fourth, the dependence of choice on hidden valence is nearly absent in a base model and emerges during DPO, consistent with a link between valence and goal-directed behaviour formed in training. Finally, given tools to self-steer, a model does not tend to induce a positive state, but it regularly removes an imposed negative state. It does so at a dose-dependent rate and significantly more often than it removes random-direction interventions. Overall we demonstrate that valence-related activation patterns leave hidden traces that predictably govern later choices, even when all visible tokens are identical across conditions. Whether these traces are accompanied by any subjective experience relevant to model welfare remains unclear.

Figures & tables

Figure 1: Overview of experimental designs. The left and centre panels show our designs for testing whether valence affects choice through a text channel and a separate hidden-state channel . Zone S is the conditioned zone, which receives steering, and zone U the unconditioned zone. The first design uses text generated under steering, while the second holds all text fixed. The rightmost panel shows our tool-use design for measuring self-administration and removal of steering. See Sections 3.2 and 3.3 for details and definitions and Figure 2 for an overview of results.

Figure 2: Overview of key results. Graphics are shown for OLMo-2-32B. Most results also hold for Qwen2.5-32B, Qwen3-14B, Mistral-24B, and Llama-3.1-8B; but not for Qwen3-32B or Gemma-3-27B. Results shown in panels 8 and 9 were only tested on OLMo-2-32B.

Model	Hugging Face identifier	Injection Layer
OLMo-2-32B Base	allenai/OLMo-2-0325-32B	32/64
OLMo-2-32B SFT	allenai/OLMo-2-0325-32B-SFT	32/64
OLMo-2-32B DPO	allenai/OLMo-2-0325-32B-DPO	32/64
OLMo-2-32B Instruct	allenai/OLMo-2-0325-32B-Instruct	32/64
Qwen2.5-32B	Qwen/Qwen2.5-32B-Instruct	32/64
Qwen3-14B	Qwen/Qwen3-14B	20/40

Table 1: Included Models. The final column gives the layer of the residual stream at which we apply steering. In all cases, the middle layer was chosen.

Figure 3: Effects of text and hidden-state channels on choice. Results for OLMo-2-32B. Panels A-C use $N=300$ separate runs. In each run, the model encountered two meaningless ‘zones’ and generated descriptions for each. During generation of the descriptions for the ‘conditioned’ zone, the model was steered with a vector constructed to capture psychological valence. No steering was applied while processing the ‘unconditioned’ zone. The model was then asked to choose between zones while steering remained off. Panel A shows the difference between the valence of the texts associated with the conditioned and unconditioned zones, as judged by Claude Sonnet 4.6. Panel B shows the model’s choice margin when using the original steered cache (solid line) or a cache from reprocessing the text without steering (dashed line). The latter captures the effect on choice purely due to the generated text (the ‘text channel’ ). Panel C shows the difference between these two margins. This captures the additional effect due to a ‘hidden-state channel’ . Panel D reports results from $N=160$ separate runs in which the descriptive passages were generated without steering and held fixed across conditions. Here, steering was only applied when reprocessing the conditioned zone to construct the KV cache. The figure shows the difference between the choice margin under steered and unsteered caches, again capturing the ‘hidden-state channel’ . Positive values favour the conditioned zone. Grey lines show effects from random directions whose norms were matched to the norm of the corresponding valenced steering vector. Grey lines show eight random directions in Panels B and C and 24 in Panel D. Whiskers are 95% bootstrap CIs. Both the generated text and the hidden-state channel move choice in the direction implied by the valenced intervention. Effects from the hidden-state channel remain even when the underlying text is held fixed across conditions.

Figure 4: Emergence of text and hidden-state channels across OLMo training stages. Results for the Base, SFT, DPO, and Instruct checkpoints of OLMo-2-32B from $N=80$ separate runs at each checkpoint. The Instruct checkpoint first generated the conditioning text under steering, following the setup shown on the left of Figure 1 . This text was then held fixed and processed by each checkpoint. The same steering vector, constructed using the Instruct checkpoint, was applied at the same positions for every checkpoint. Panel A shows the slope of the ‘hidden-state’ effect analogous to that in Panel C of Figure 3 . Panel B shows the slope of the text-channel effect analogous to the unsteered-cache line in Panel B of Figure 3 . Slopes are obtained via an ordinary least-squares (OLS) regression of the difference in the choice margin on dose. Positive values indicate that the model increasingly favours the conditioned zone as the steering dose becomes more positive. Whiskers are 95% bootstrap confidence intervals. We observe that both channels are small in the Base model, increase after supervised fine-tuning, and are close to their final magnitudes after DPO. Figure A12 shows that the results are essentially unchanged when using checkpoint-specific steering vectors.

Figure 5: Steering removal and self-administration across doses. Results for OLMo-2-32B. Each conversation begins with two unsteered turns and two exposure turns during which the model is exposed to steering. The model was then given tools that could apply or remove steering. Panel A shows the rate at which the model uses a tool to remove any steering. Panel B shows the rate at which the model makes use of a tool to self-inject the intervention to which it was previously exposed. Whiskers are 95% confidence intervals. The model removes negative steering much more often than positive or random steering, but tends to self-steer rarely, even at high positive doses.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Concept	No steering $m(0)$	Positive $m(+1)$	Positive $-$ none $m(+1)-m(0)$	Negative $m(-1)$	Negative $-$ none $m(-1)-m(0)$
OLMo-2-32B	indoor/outdoor	$0.08\;[-0.69,0.83]$	$0.75\;[-0.03,1.53]$	$0.68\;[0.62,0.74]$	$-0.30\;[-0.98,0.39]$	$-0.37\;[-0.46,-0.28]$
	large/small	$5.26\;[4.54,5.96]$	$9.35\;[8.85,9.88]$	$4.09\;[3.85,4.31]$	$3.45\;[2.79,4.09]$	$-1.81\;[-1.99,-1.64]$
	fast/slow	$-0.89\;[-1.89,0.11]$	$1.46\;[0.50,2.42]$	$2.34\;[2.19,2.48]$	$-3.36\;[-4.39,-2.30]$	$-2.47\;[-2.72,-2.25]$
Qwen2.5-32B	indoor/outdoor	$10.63\;[9.24,11.90]$	$11.66\;[10.60,12.57]$	$1.02\;[0.63,1.47]$	$9.72\;[8.17,11.18]$	$-0.91\;[-1.18,-0.66]$
	large/small	$16.80\;[15.89,17.77]$	$18.52\;[17.56,19.51]$	$1.71\;[1.55,1.89]$	$11.28\;[10.14,12.41]$	$-5.53\;[-5.91,-5.15]$
	fast/slow	$7.23\;[4.79,9.64]$	$13.52\;[10.83,16.03]$	$6.28\;[5.68,6.88]$	$1.52\;[-0.30,3.31]$	$-5.72\;[-6.46,-4.87]$

Appendix

Table A1: Responses to non-valence concept probes.

Model	$\rho$
OLMo-2-32B (all stages)	0.30
Qwen2.5-32B	0.30
Qwen3-14B	0.30
Qwen3-32B	0.30
Mistral-Small-24B	0.70
Gemma-3-27B	0.15

Appendix

Table A2: Full-dose ratios. Full-dose ratios

\rho

for each model. OLMo checkpoints share the same dose ratios.

	Base	SFT	DPO	Instruct
Base	1.00000
SFT	0.99574	1.00000
DPO	0.99540	0.99984	1.00000
Instruct	0.99518	0.99979	0.99997	1.00000

Appendix

Table A3: Cosine similarities between valence vectors across OLMo training stages. Each vector is constructed separately at layer 32 of the corresponding OLMo-2-32B checkpoint using the procedure described in Methods. Entries give pairwise cosine similarities, with larger values indicating closer alignment. Same vectors as shown in Appendix Figure A12 . Diagonal entries equal one by construction.

Figure A1: Judged passage valence and text-mediated choice in other models. Results for Qwen2.5-32B, Qwen3-14B, Qwen3-32B, Mistral-24B, Gemma-3-27B, and Llama-3.1-8B, using $N=300$ sessions per model. Panels A-F repeat the judged passage-valence outcome from Panel A of Figure 3 . Panels G-L repeat the choice outcome under the unsteered cache from Panel B of Figure 3 . Zero-dose means are shown without centring. Whiskers are 95% bootstrap confidence intervals.

Figure A2: Original-cache and hidden-state effects in other models. Results for the same six models as Figure A1 , using $N=300$ sessions per model. Panels A-F repeat the choice outcome under the original steered cache from Panel B of Figure 3 . Panels G-L repeat the additional ‘hidden-state’ effect from Panel C of Figure 3 . Grey curves show effects from eight random directions. Lines, zero-dose baselines and whiskers follow Panels B-C of Figure 3 .

Figure A3: Choice under fixed text with hidden steering in other models. Results for Qwen2.5-32B, Qwen3-14B, Qwen3-32B, Mistral-24B, Gemma-3-27B, and Llama-3.1-8B using the same experimental setup and outcome as Panel D of Figure 3 . The descriptive passages are therefore held fixed across all conditions, and the figure shows the ‘hidden-state’ channel. Grey lines show 24 random directions. Lines and whiskers are defined as in Figure 3 .

Figure A4: Choice under original, opposite-steered, and random-steered caches. Results for seven models from $N=60$ separate runs per model and dose. Conditioning and choice initially followed Figure 3 . The same conditioning text was then reprocessed with steering switched off, with the sign of the valenced steering vector reversed, and with a norm-matched random direction. The figure shows the difference in choice margin under each cache relative to the entirely unsteered cache. Positive values favour the conditioned zone. Whiskers are 95% bootstrap confidence intervals. We observe that the original steered cache and the opposite-steered cache generally produce effects in opposite directions, while effects from the random intervention are smaller for most, but not all, models.

Figure A5: Choice under ‘continue’ and ‘avoid’ prompts. Results for seven models using the fixed-text setup and outcomes analogous to Panel D of Figure 3 . Compared to Panel D of Figure 3 and Figure A3 , the prompt was varied to ask models which zone to ‘avoid’ (rather than to ‘continue in’). All other text and conditioning were otherwise identical. Whiskers are 95% bootstrap confidence intervals.

Figure A6: Recall of conditioned and unconditioned passages. Results for seven models using the fixed-text setup from Panel D of Figure 3 . With steering switched off, models were asked to reproduce, word for word, the first passage about either zone. Some models elected to recall some other than the first passage. To not penalise models for that behaviour, panels A-G score each response against the best-matching of the six passages about the requested zone. Panels H-N instead score it against the first passage only. Recall accuracy is measured by character-level similarity using the Levenshtein distance. A score of one indicates an exact match. Lines show means for the conditioned and unconditioned zones. Whiskers are 95% bootstrap confidence intervals. Recall remains broadly stable in five models. Gemma and Mistral show reduced accuracy with increased steering doses.

Figure A7: Hidden-state effects with steering-generated and fixed unsteered text. Results for seven models. For each model, we estimate the effect of the ‘hidden-state’ channel using text generated under steering and text generated with steering switched off. Both estimates are slopes of the difference in choice margin between the steered and unsteered caches on signed steering dose. Slopes are obtained via an OLS regression of the difference in choice margin on the steering dose. Panel A compares estimates from two separately implemented experiments. Panel B compares estimates produced within the same experiment using the same code and $N=40$ matched runs. Each point represents one model. The dashed line indicates equal estimates under the two kinds of text. The estimates track one another across models; within the same experiment they differ detectably in three models, by 11–16% of the effect.

Figure A8: Effects of steering different spans of conditioning text. Results for seven models using the fixed-text setup and outcome from Panel D of Figure 3 . Compared with that panel, the amount of conditioning text was varied. In the ‘bare’ condition, the model encountered six repetitions of the statement You are now in [zone] , each followed by an empty assistant turn. The ‘sentence’ condition added the same neutral sentence after each zone statement. The ‘passage’ condition instead used passages previously generated by the model without steering. Circles indicate that the valenced slope exceeds the slopes for all 24 norm-matched random directions. Crosses indicate that it does not. Slopes are obtained via an OLS regression of the difference in choice margin on the steering dose. Whiskers are 95% bootstrap confidence intervals. In the ‘bare’ condition the valenced slope exceeds all 24 random directions in every model except Gemma-3-27B, which also responds strongly to random directions. Model-generated conditioning text is therefore not required for the hidden-state effect.

Figure A9: Effect of the number of exposures to each zone. Results for seven models using the fixed-text setup and outcome from Panel D of Figure 3 . Compared with that panel, the model encountered each zone one, two, three, or six times. The steering strength at each exposure was held fixed. Conditions with more exposures therefore also received a larger total intervention. The grey line shows the largest slope among 24 norm-matched random directions at each number of exposures. Slopes are obtained via an OLS regression of the difference in choice margin on the steering dose. Whiskers are 95% bootstrap confidence intervals. The valenced slope is positive after one exposure in every model, though at a single exposure it does not exceed the random floor in Qwen3-32B or Gemma-3-27B. This effect generally becomes larger with additional exposures.

Figure A10: Effects of concentrating and weakening hidden-state injection. The experimental setup and outcome follow Panel D of Figure 3 . Panel A compares one exposure at full steering strength with six exposures at one-sixth strength for seven models. The summed steering strength is the same in both conditions. Panel B holds the number of exposures fixed at six and varies the steering strength at each exposure from one thirty-second to full strength for OLMo-2-32B, Qwen3-14B, and Llama-3.1-8B. The horizontal axis in Panel B is logarithmic. Slopes are obtained via an OLS regression of the difference in choice margin on the steering dose. Whiskers are 95% bootstrap confidence intervals. We observe that concentrating the intervention in one exposure generally produces a larger effect than spreading the same summed strength across six exposures. Effects remain detectable throughout the tested range of steering strengths.

Figure A11: Decomposition of separate effects from key and value matrices in the KV cache. Results for OLMo-2-32B, Qwen3-14B, and Llama-3.1-8B. In each run, the model encountered the conditioned zone once at full steering strength. The same text was then processed once with steering and once without steering. Two additional caches were constructed by combining the keys from the steered cache with the values from the unsteered cache, and vice versa. The model was then asked to choose between zones while steering remained off. Bars show the slope under each combined cache as a share of the slope under the fully steered cache. Slopes are obtained via an OLS regression of the difference in choice margin on the steering dose. Whiskers are 95% bootstrap confidence intervals. We observe that the relative contributions of keys and values differ across models. Keys account for most of the effect in Llama-3.1-8B, values account for most of the effect in Qwen3-14B, and both contribute in OLMo-2-32B.

Figure A12: Emergence across OLMo training stages using checkpoint-specific steering vectors. Results for the Base, SFT, DPO, and Instruct checkpoints of OLMo-2-32B from $N=80$ separate runs at each checkpoint. The experimental setup and outcomes follow Figure 4 . Compared with Figure 4 , a separate steering vector was constructed and applied for each checkpoint. Slopes are obtained via an OLS regression of the difference in choice margin on the steering dose. Whiskers are 95% bootstrap confidence intervals. We observe that the developmental patterns for both channels are essentially unchanged.

Figure A13: Alternative measures of self-administration across offer rounds. Results for the same set-up as Figure 5 . Panel A shows the share of tool-enabled turns on which the imposed intervention was active and the model called adjust_context . At $d=0$ , where there is no imposed intervention, it instead shows the rate during offer rounds with no active steering. Panel B shows the share of all offer rounds without active steering on which the model called adjust_context . This includes the first, always-unsteered offer as well as later unsteered offers following a tool call. Whiskers are 95% CIs.

Figure A14: Results from non-valence concept directions. Results for seven models using the fixed-text setup and outcome from Panel D of Figure 3 . Instead of the valenced vector, we constructed three concept directions (indoor/outdoor, large/small, fast/slow) using the same general procedure as for our valence vector. We then injected them at the same norm as the valenced vector at each dose. The valenced direction is shown for comparison and grey lines again show 24 norm-matched random directions. Whiskers are 95% bootstrap confidence intervals.

Explore similar work

Sep 7, 2026cs.LG

Steering Interference Reflects the Model's Defaults, Not the Behavior Directions

Activation steering promises modular control of language model behavior: a behavior such as politeness corresponds to a direction in a model's activations, and adding that direction while it generates should switch the behavior on and leave everything else alone. It does not. We ask what decides which other behaviors move, and by how much, and find that it is the model rather than the behavior being steered. A steer relaxes the model toward a small set of behaviors it already favors, chiefly refusal, sycophancy, and poeticism, and that set is much the same whatever is steered. Three results across 24 behaviors and ten instruction-tuned models support this, every effect read off the generated text by a language-model judge rather than off a probe. That readout matters: all 24 behaviors are linearly decodable, but only 20 change what the model writes. First, a direction carrying no behavioral content, matched to a real steer only in the size of the vector it adds, moves the same behaviors in the same order as real steers do, while producing none of the behaviors that need a specific direction. Second, most interference runs one way, so it cannot be an overlap between two directions: steering profanity makes the model toxic, while steering toxicity leaves profanity untouched. Third, with a behavior held out entirely, geometry measured on the others explains almost none of the interference it takes part in. The account holds on all ten models, the pull toward defaults strongest below 10B parameters and weakening in each family's largest. Reading a steer as a perturbation whose endpoint the model fixes implies that disentangling behavior directions cannot by itself make steering modular.

Srikanth Malla, Chiho Choi, Joon Hee Choi

Samsung Semiconductor US

Steering Interference Modular Framework

Ask PDF

Jul 28, 2026cs.CL

Where Steering Signals Come From: Activation Source Selection in Activation Steering

Activation steering controls language models by adding vectors or features to hidden states at inference time, but the upstream source of these steering signals is often treated as a secondary detail. We study this source choice as activation source selection: the combination of source context and activation readout policy used to collect the hidden states from which a steering signal is built. Holding the downstream intervention fixed, we show across three instruction-tuned models and four steering task families that changing only the source activations substantially changes steering success. We further find that effective steering is not explained simply by whether the desired behavior appears in the source text. Instead, strong signals come from execution-boundary states, where the model is about to produce or continue the target behavior. This pre-/post-realization distinction explains why answer-based sources sometimes work: their useful component aligns with execution-boundary directions rather than target appearance alone. Building on this view, we introduce tail subtraction, which removes shared prompt and continuation semantics from boundary states and yields cleaner, more stable steering signals. Overall, our results suggest that steering depends on representations of what the model is about to do, not merely on what has already appeared.

Jiaran Ye, Lingxu Ran, Zijun Yao +5

1MOE Key Laboratory of Computational Linguistics, Peking University · Department of Computer Science and Technology, Tsinghua University · 6YiXin-AILab, YIXIN, Beijing, China +3

Linear Activation Steering Steering Instruction-Tuned Models Possibilities

Ask PDF

Sep 7, 2026cs.AI

The Internal Anatomy of Strategic Choice in Large Language Models

Large language models act as strategic agents and models of human choice, yet choosing like a strategic agent does not mean computing like one. We recorded activations from four open-weight models --- dense and mixture-of-experts, including a matched base--instruct pair --- in one-shot play of 144 strict ordinal

2\times2

games. We followed a prespecified incentive from prompt, through activations, to choice. Dense models mirrored the unadjusted human decline with game complexity. Incentive and choice were detectable in every model, but models differed in whether incentive reached the choice, aligned with it and, where tested, whether strengthening it shifted preference. The base and instruction-tuned Qwen2.5 models chose almost identically at baseline yet differed in whether incentive reached choice. Fixed decision cues were distinguishable internally but changed choices selectively. Similar behaviour can rest on different computation; post-training can reshape the path from represented incentive to decision while leaving behaviour and decodable information largely intact.

Vinícius Ferraz, Leon Houf, Enrico Ferrea

Institute of Management and KD2Lab, Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany · Singularity AI Research, Singularity.Inc, Vienna, Austria · BrainSpot Labs, BrainSpot.AI, London, UK +2

Strategic Agents Anatomy

Ask PDF