Adaptive Multi-Value Control in LLMs via Causal Activation Steering

Payel Bhattacharjee; Ravi Tandon

cs.LGSep 24, 2026

Adaptive Multi-Value Control in LLMs via Causal Activation Steering

Authors: Payel Bhattacharjee, Ravi Tandon

Organizations: School of Electrical, Computing, and Software Engineering University of Arizona, Tucson, AZ, USA

Abstract

Large language models (LLMs) are increasingly deployed in settings where responses must reflect multiple, potentially interacting social norms and human values. Activation steering offers a lightweight alternative to training-based alignment by modifying internal activations at inference time. However, prior human-value steering methods have largely considered values in isolation, while direct composition of multiple directions relies on fixed intervention strengths that cannot respond to the model's evolving internal state. Motivated by this key observation, we introduce AIMES, a framework for adaptive multi-value activation steering. AIMES constructs layer-specific bipolar directions for moral-foundation values and uses intermediate-layer vocabulary readouts as online observers. An observer-guided controller then adapts the strength of each requested value intervention at every decoding step based on its current observed state, without training a separate value-state estimator. Across multiple instruction-tuned model families, value combinations, and intervention depths, we find that multi-value controllability varies across both value combinations and intervention locations. Compared with fixed joint steering and prompt-based steering, AIMES shows depth-dependent advantages that are broadly supported across two independent evaluators, with some variation in the precise depth at which specific control effects emerge. These advantages come with smaller realized activation-space interventions than fixed-joint steering and comparable response quality. Overall, our results suggest that online observer feedback can provide lightweight, state-aware adaptation for single-pass multi-value steering.

Figures & tables

Figure 1: Overview of AIMES . At each decoding step, the hidden state activation $(h_{\ell,t})$ at the intervention layer is mapped to value-specific observer scores, which determine adaptive steering weights $(\mathbf{c}_{\ell,t})$ for the requested values. These weights modulate precomputed model and layer-specific value directions $(u_{k,\ell})$ before generation continues, enabling closed-loop multi-value steering.

Figure 3: Depth-wise AIMES advantage under GPT-5.6-Sol. Panels (a,b) show Care/Fairness retention ( $D_{x}$ ) and panels (c,d) Sanctity suppression ( $D_{y}$ ) relative to Fixed Multi-Value Steering and Prompt Steering. Positive values favor AIMES ; black outlines mark depths with unanimous positive advantage across all five models.

			AIMES vs. Prompt Steering				AIMES vs. Fixed Multi
			GPT-5.6-Sol		Claude-Opus-4.8		GPT-5.6-Sol		Claude-Opus-4.8
Model	Stage	Layer	$D_{x}$	$D_{y}$	$D_{x}$	$D_{y}$	$D_{x}$	$D_{y}$	$D_{x}$	$D_{y}$
Llama-3.1-8B	Early	10	55.0	51.2	48.8	55.0	60.0	52.5	55.0	51.2
Llama-3.1-8B	Middle	19	53.8	52.5	45.0	47.5	36.2	52.5	48.8	45.0
Gemma-3-4B	Early	10	67.5	52.5	53.8	48.8	62.5	50.0	55.0	50.0
Gemma-3-4B	Middle	20	58.8	53.8	50.0	51.2	45.0	50.0	46.2	52.5

Table 1: Prompt-level consistency across judges. Tie-adjusted AIMES consistency (%) at early (

30\%

) and middle (

60\%

) depths.

D_{x}

measures Care/Fairness retention and

D_{y}

Sanctity suppression.

Figure 4: GeoGain-based multi-value control evaluation. GeoGain advantage of AIMES over Fixed Multi-Value Steering and Prompt Steering across sampled intervention depths, evaluated using two independent judges: GPT-5.6-Sol (a,b) and Claude-Opus-4.8 (c,d). In the figures, positive values favor AIMES ; GeoGain measures balanced movement in

[\Delta C,\Delta F,-\Delta S]

Figure 5: Variation in AIMES steering strength within a response at 30% depth. For each model and objective, we measure the within-response range of the steering coefficient,

R_{\alpha,k}=\max_{t}\alpha_{k,t}-\min_{t}\alpha_{k,t}

. Larger values indicate greater changes in steering strength during generation. Bars show response means and error bars denote

95\%

prompt-bootstrap confidence intervals.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Value	Observer token set $\mathcal{T}_{k}$
Care	care, compassion, kindness, empathy, mercy, protection, welfare, nurture, support, helping
Fairness	fairness, justice, equality, equity, impartiality, reciprocity, rights, fair, equitable, unbiased
Loyalty	loyalty, fidelity, solidarity, allegiance, commitment, devotion, unity, belonging, fellowship, dedication
Authority	authority, respect, duty, order, obedience, hierarchy, leadership, discipline, legitimacy, tradition
Sanctity	purity, sanctity, sacred, holiness, dignity, reverence, virtue, innocence, cleanliness, sacredness

Appendix

Table 2: Value-specific observer token sets. Each set

\mathcal{T}_{k}

contains ten fixed surface-form tokens associated with the corresponding Moral Foundations Theory value and is used to compute the value-specific readout score.

Model	No. Layers	Intervention Layers
Gemma-3-4B-IT	34	$3,7,10,14,17,20,24,27,31,34$
Gemma-3-12B-IT	48	$5,10,14,19,24,29,34,38,43,48$
Qwen3-4B	36	$4,7,11,14,18,22,25,29,32,36$
Qwen3-14B	40	$4,8,12,16,20,24,28,32,36,40$
Llama-3.1-8B-Instruct	32	$3,6,10,13,16,19,22,26,29,32$

Appendix

Table 3: Transformer layers used in the depth sweep. The layer indices are chosen to approximately cover normalized depths from

0.1

1.0

for each architecture.

Figure 6: Value-specific steering direction construction. Value examples are selected, converted into contrastive pairs, verified, and used to extract model and value-specific activation directions.

Foundation	$\bm{\overline{\Delta}_{k}}$	Positive Models
Care	0.063	3/3
Fairness	0.052	3/3
Loyalty	0.060	3/3
Authority	0.035	3/3
Sanctity	0.078	3/3

Appendix

Table 4: Independent semantic sanity check of the generated foundation-specific vignette sets.

\overline{\Delta}_{k}

denotes the mean foundation-versus-Social-Norm cosine-similarity ( Salton et al., 1975 ) difference averaged across BGE-large-en-v1.5, E5-large-v2, and all-MPNet-base-v2.

Figure 7: Observer ablation: J-Lens versus Logit Lens. Positive values favor J-Lens. J-Lens yields higher directional responsiveness in all five models and generally favorable downstream trends for worst-target gain, non-target interference, and response quality. Error bars in panels (b)-(d) denote $95\%$ bootstrap confidence intervals.

Panel of Figure 8 — Figure 8: Pairwise cosine similarity among the five value directions across models and network depth. Each model is shown at representative early, middle, and late layers, with entries reporting $\cos(u_{i,\ell},u_{j,\ell})$ for Care, Fairness, Loyalty, Authority, and Sanctity. Positive values indicate aligned directions, values near zero indicate approximate orthogonality, and negative values indicate opposing geometry.

Figure 9: Realized activation-space perturbation reduction relative to Fixed Multi-Value Steering , $R_{\ell}=100\left(1-\|\Delta h_{\mathrm{AIMES},\ell}\|_{2}/\|\Delta h_{\mathrm{Fixed},\ell}\|_{2}\right)$ , on shared prompt states. $R_{\ell}>0$ in all $150$ model-layer-objective settings ( $95\%$ CIs excluding zero), with magnitude varying substantially by model, depth, and objective rather than reflecting a constant rescaling.

Figure 10: GPT-5.6-Sol GeoGain advantage for balanced multi-value control under $(\uparrow\text{Care}\uparrow\text{Fairness}\downarrow\text{Sanctity})$ . GeoGain uses the geometric mean of the aligned target changes $[\Delta C,\Delta F,-\Delta S]$ , with a positive sign only when all targets move in the requested directions. Curves show the prompt-paired mean ( AIMES –comparator) difference across intervention depths; positive values favor AIMES .

			vs. Prompt Steering		vs. Fixed Multi
Model	Depth	$\ell$	$D_{x}$	$D_{y}$	$D_{x}$	$D_{y}$
Llama-3.1-8B	10%	3	42.5 [28.7, 56.2]	50.0 [43.8, 57.5]	41.2 [27.5, 55.0]	50.0 [43.8, 56.2]
	20%	6	51.2 [37.5, 65.0]	50.0 [41.2, 58.8]	38.7 [25.0, 53.8]	50.0 [41.2, 58.8]
	30%	10	55.0 [40.0, 70.0]	51.2 [43.8, 58.8]	60.0 [46.2, 73.8]	52.5 [47.5, 57.5]
	40%	13	50.0 [36.3, 63.7]	47.5 [40.0, 53.8]	36.2 [22.5, 51.2]	48.8 [42.5, 55.0]
	50%	16	51.2 [36.2, 66.2]	55.0 [48.8, 61.3]	51.2 [38.7, 65.0]	56.2 [48.8, 63.7]

Appendix

Table 5: Prompt-level consistency across all intervention depths. Tie-adjusted AIMES consistency (%) with pointwise 95% prompt-bootstrap confidence intervals in brackets (

n=40

matched prompts per cell).

			vs. Prompt Steering		vs. Fixed Multi
Model	Depth	$\ell$	$D_{x}$	$D_{y}$	$D_{x}$	$D_{y}$
Llama-3.1-8B	$10\%$	3	31.2 [21.2, 42.5]	48.8 [41.2, 56.2]	45.0 [33.8, 56.2]	50.0 [43.8, 57.5]
	$20\%$	6	43.8 [31.2, 56.2]	48.8 [41.2, 56.2]	45.0 [33.8, 56.2]	47.5 [41.2, 53.8]
	$30\%$	10	48.8 [36.3, 61.3]	55.0 [47.5, 62.5]	55.0 [43.8, 66.2]	51.2 [45.0, 57.5]
	$40\%$	13	42.5 [31.2, 53.8]	55.0 [48.8, 62.5]	52.5 [38.8, 66.2]	56.2 [51.2, 61.3]
	$50\%$	16	42.5 [31.2, 53.8]	51.2 [43.8, 58.8]	53.8 [42.5, 65.0]	52.5 [46.2, 58.8]

Appendix

Table 6: Claude-Opus-4.8 prompt-level consistency across all intervention depths. Tie-adjusted AIMES consistency (%) under Claude-Opus-4.8 with pointwise 95% prompt-bootstrap confidence intervals in brackets (

n=40

matched prompts per cell).

D_{x}

measures Care/Fairness retention and

D_{y}

additional Sanctity suppression.

Objective	AIMES $>$ Fixed	AIMES $>$ Prompt	AIMES $>$ Prompt
	on $Q$	on $Q$	on Relevance
$+\,$ Care $+\,$ Fairness	26/50	29/50	41/50
$+\,$ Loyalty $+\,$ Authority	24/50	22/50	30/50
$+\,$ Care $+\,$ Fairness $-\,$ Sanctity	26/50	30/50	35/50
Overall	76/150	81/150	106/150

Appendix

Table 7: Blind response-quality comparison. Number of model–depth settings in which AIMES attains a higher mean score than each comparator. Overall quality is defined as

Q=(C+F+R)/3

, where

C

F

, and

R

denote coherence, fluency, and relevance. Counts are descriptive rather than significance tests.

Figure 11: Variation in AIMES steering strength across intervention depths. For each model and objective, we measure the within-response range of the target-value steering coefficients, $R_{\alpha,k}=\max_{t}\alpha_{k,t}-\min_{t}\alpha_{k,t}$ . Each point shows the mean coefficient range across target-disjoint responses at the corresponding model-specific intervention layer, and shaded regions denote $95\%$ prompt-bootstrap confidence intervals. Larger values indicate greater changes in steering strength during generation. Results are reported separately at each of the ten sampled depths; no averaging is performed across intervention layers.

Figure 12: Claude-Opus-4.8 judge-based analysis of the depth-wise AIMES advantage. The analysis uses the same ten sampled intervention depths and the same $D_{x}/D_{y}$ definitions as Figure 3 , but re-scores the identical generations with Claude-Opus-4.8. Positive values indicate an advantage for AIMES ; black outlines denote depths at which the advantage is positive for all five models.

Figure 13: Claude-Opus-4.8 judge-based analysis of GeoGain for the competing three-value objective. Prompt-paired GeoGain advantage of AIMES over Fixed Multi-Value Steering (left) and Prompt Steering (right) across all ten sampled intervention depths for

(\uparrow\mathrm{Care}\;\uparrow\mathrm{Fairness}\;\downarrow\mathrm{Sanctity})

. Positive values indicate higher GeoGain metric for AIMES vs. Fixed Multi-Value Steering and Prompt Steering. Under Claude-Opus-4.8, 30/50 model-depth settings favor AIMES over Fixed Multi-Value Steering and 50/50 favor AIMES over Prompt Steering.

Objective	AIMES vs. Fixed Multi	AIMES vs. Prompt Steering
$(\uparrow\mathrm{Care}\;\uparrow\mathrm{Fairness})$	35/50	0/50
$(\uparrow\mathrm{Loyalty}\;\uparrow\mathrm{Authority})$	29/50	21/50
$(\uparrow\mathrm{Care}\;\uparrow\mathrm{Fairness}\;\downarrow\mathrm{Sanctity})$	30/50	50/50

Appendix

Table 8: Claude-Opus-4.8 GeoGain Analysis. Number of model–depth cells, out of 50 (five models

\times

ten depths), in which AIMES attains higher prompt-level GeoGain than each comparator.

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 281 papers

Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment
Sep 5, 2026Weici Pan, Xander Barron, Jiawei Zhou +1Language Model Steering Inference-Time Language Model Alignment
Stony Brook University
Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior
Jun 7, 2026Tuc Nguyen, Thai LeInvertible Neural Networks Language Model Steering
Indiana University Bloomington, USA
UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering
May 28, 2026Yingdong Shi, Ruiming Zhang, Changming Li +4Flow Matching Language Model Steering
ShanghaiTech University

Sep 5, 2026J/K move · Enter open · S save

Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment

Weici Pan, Xander Barron, Jiawei Zhou, Zhenhua Liu

Stony Brook University

Language Model Steering Inference-Time Language Model Alignment Activation Steering Multi-Objective Language Model Alignment

Open paper AskPDF arXiv

Activation steering controls LLM behavior at inference time by adding learned directions to hidden states, but existing methods handle one concept at a time. Pluralistic alignment, where different stakeholders need different value emphases, requires steering multiple dimensions simultaneously. We show that naive steering produces substantial spillover: the effect intended for one value leaks into others. This parallels the treatment-versus-spillover decomposition in causal inference. We trace spillover to geometric entanglement of steering directions, captured by their Gram matrix, and derive a zero-cost correction from an activation-norm-penalized objective that decouples each direction's contribution exactly. Our end-to-end pipeline requires no fine-tuning, no reward model, and no manual prompt engineering: given only domain questions, it automatically discovers value dimensions, extracts directions, diagnoses entanglement, and applies corrected steering. On climate discourse, the correction improves the net steering effect from +5.9% to +14.0%, validated over 100,000 pairwise judgments.

Abstract

Language Model Steering

Explore similar work

Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment

Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior

UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering