Adaptive Multi-Value Control in LLMs via Causal Activation Steering

Payel Bhattacharjee; Ravi Tandon

cs.LGSep 24, 2026

Adaptive Multi-Value Control in LLMs via Causal Activation Steering

Authors: Payel Bhattacharjee, Ravi Tandon

Organizations: School of Electrical, Computing, and Software Engineering University of Arizona, Tucson, AZ, USA

Abstract

Large language models (LLMs) are increasingly deployed in settings where responses must reflect multiple, potentially interacting social norms and human values. Activation steering offers a lightweight alternative to training-based alignment by modifying internal activations at inference time. However, prior human-value steering methods have largely considered values in isolation, while direct composition of multiple directions relies on fixed intervention strengths that cannot respond to the model's evolving internal state. Motivated by this key observation, we introduce AIMES, a framework for adaptive multi-value activation steering. AIMES constructs layer-specific bipolar directions for moral-foundation values and uses intermediate-layer vocabulary readouts as online observers. An observer-guided controller then adapts the strength of each requested value intervention at every decoding step based on its current observed state, without training a separate value-state estimator. Across multiple instruction-tuned model families, value combinations, and intervention depths, we find that multi-value controllability varies across both value combinations and intervention locations. Compared with fixed joint steering and prompt-based steering, AIMES shows depth-dependent advantages that are broadly supported across two independent evaluators, with some variation in the precise depth at which specific control effects emerge. These advantages come with smaller realized activation-space interventions than fixed-joint steering and comparable response quality. Overall, our results suggest that online observer feedback can provide lightweight, state-aware adaptation for single-pass multi-value steering.

Figures & tables

Figure 1: Overview of AIMES . At each decoding step, the hidden state activation $(h_{\ell,t})$ at the intervention layer is mapped to value-specific observer scores, which determine adaptive steering weights $(\mathbf{c}_{\ell,t})$ for the requested values. These weights modulate precomputed model and layer-specific value directions $(u_{k,\ell})$ before generation continues, enabling closed-loop multi-value steering.

Figure 3: Depth-wise AIMES advantage under GPT-5.6-Sol. Panels (a,b) show Care/Fairness retention ( $D_{x}$ ) and panels (c,d) Sanctity suppression ( $D_{y}$ ) relative to Fixed Multi-Value Steering and Prompt Steering. Positive values favor AIMES ; black outlines mark depths with unanimous positive advantage across all five models.

			AIMES vs. Prompt Steering				AIMES vs. Fixed Multi
			GPT-5.6-Sol		Claude-Opus-4.8		GPT-5.6-Sol		Claude-Opus-4.8
Model	Stage	Layer	$D_{x}$	$D_{y}$	$D_{x}$	$D_{y}$	$D_{x}$	$D_{y}$	$D_{x}$	$D_{y}$
Llama-3.1-8B	Early	10	55.0	51.2	48.8	55.0	60.0	52.5	55.0	51.2
Llama-3.1-8B	Middle	19	53.8	52.5	45.0	47.5	36.2	52.5	48.8	45.0
Gemma-3-4B	Early	10	67.5	52.5	53.8	48.8	62.5	50.0	55.0	50.0
Gemma-3-4B	Middle	20	58.8	53.8	50.0	51.2	45.0	50.0	46.2	52.5

Table 1: Prompt-level consistency across judges. Tie-adjusted AIMES consistency (%) at early (

30\%

) and middle (

60\%

) depths.

D_{x}

measures Care/Fairness retention and

D_{y}

Sanctity suppression.

Figure 4: GeoGain-based multi-value control evaluation. GeoGain advantage of AIMES over Fixed Multi-Value Steering and Prompt Steering across sampled intervention depths, evaluated using two independent judges: GPT-5.6-Sol (a,b) and Claude-Opus-4.8 (c,d). In the figures, positive values favor AIMES ; GeoGain measures balanced movement in

[\Delta C,\Delta F,-\Delta S]

Figure 5: Variation in AIMES steering strength within a response at 30% depth. For each model and objective, we measure the within-response range of the steering coefficient,

R_{\alpha,k}=\max_{t}\alpha_{k,t}-\min_{t}\alpha_{k,t}

. Larger values indicate greater changes in steering strength during generation. Bars show response means and error bars denote

95\%

prompt-bootstrap confidence intervals.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Value	Observer token set $\mathcal{T}_{k}$
Care	care, compassion, kindness, empathy, mercy, protection, welfare, nurture, support, helping
Fairness	fairness, justice, equality, equity, impartiality, reciprocity, rights, fair, equitable, unbiased
Loyalty	loyalty, fidelity, solidarity, allegiance, commitment, devotion, unity, belonging, fellowship, dedication
Authority	authority, respect, duty, order, obedience, hierarchy, leadership, discipline, legitimacy, tradition
Sanctity	purity, sanctity, sacred, holiness, dignity, reverence, virtue, innocence, cleanliness, sacredness

Appendix

Table 2: Value-specific observer token sets. Each set

\mathcal{T}_{k}

contains ten fixed surface-form tokens associated with the corresponding Moral Foundations Theory value and is used to compute the value-specific readout score.

Model	No. Layers	Intervention Layers
Gemma-3-4B-IT	34	$3,7,10,14,17,20,24,27,31,34$
Gemma-3-12B-IT	48	$5,10,14,19,24,29,34,38,43,48$
Qwen3-4B	36	$4,7,11,14,18,22,25,29,32,36$
Qwen3-14B	40	$4,8,12,16,20,24,28,32,36,40$
Llama-3.1-8B-Instruct	32	$3,6,10,13,16,19,22,26,29,32$

Appendix

Table 3: Transformer layers used in the depth sweep. The layer indices are chosen to approximately cover normalized depths from

0.1

1.0

for each architecture.

Figure 6: Value-specific steering direction construction. Value examples are selected, converted into contrastive pairs, verified, and used to extract model and value-specific activation directions.

Foundation	$\bm{\overline{\Delta}_{k}}$	Positive Models
Care	0.063	3/3
Fairness	0.052	3/3
Loyalty	0.060	3/3
Authority	0.035	3/3
Sanctity	0.078	3/3

Appendix

Table 4: Independent semantic sanity check of the generated foundation-specific vignette sets.

\overline{\Delta}_{k}

denotes the mean foundation-versus-Social-Norm cosine-similarity ( Salton et al., 1975 ) difference averaged across BGE-large-en-v1.5, E5-large-v2, and all-MPNet-base-v2.

Figure 7: Observer ablation: J-Lens versus Logit Lens. Positive values favor J-Lens. J-Lens yields higher directional responsiveness in all five models and generally favorable downstream trends for worst-target gain, non-target interference, and response quality. Error bars in panels (b)-(d) denote $95\%$ bootstrap confidence intervals.

Panel of Figure 8 — Figure 8: Pairwise cosine similarity among the five value directions across models and network depth. Each model is shown at representative early, middle, and late layers, with entries reporting $\cos(u_{i,\ell},u_{j,\ell})$ for Care, Fairness, Loyalty, Authority, and Sanctity. Positive values indicate aligned directions, values near zero indicate approximate orthogonality, and negative values indicate opposing geometry.

Figure 9: Realized activation-space perturbation reduction relative to Fixed Multi-Value Steering , $R_{\ell}=100\left(1-\|\Delta h_{\mathrm{AIMES},\ell}\|_{2}/\|\Delta h_{\mathrm{Fixed},\ell}\|_{2}\right)$ , on shared prompt states. $R_{\ell}>0$ in all $150$ model-layer-objective settings ( $95\%$ CIs excluding zero), with magnitude varying substantially by model, depth, and objective rather than reflecting a constant rescaling.

Figure 10: GPT-5.6-Sol GeoGain advantage for balanced multi-value control under $(\uparrow\text{Care}\uparrow\text{Fairness}\downarrow\text{Sanctity})$ . GeoGain uses the geometric mean of the aligned target changes $[\Delta C,\Delta F,-\Delta S]$ , with a positive sign only when all targets move in the requested directions. Curves show the prompt-paired mean ( AIMES –comparator) difference across intervention depths; positive values favor AIMES .

			vs. Prompt Steering		vs. Fixed Multi
Model	Depth	$\ell$	$D_{x}$	$D_{y}$	$D_{x}$	$D_{y}$
Llama-3.1-8B	10%	3	42.5 [28.7, 56.2]	50.0 [43.8, 57.5]	41.2 [27.5, 55.0]	50.0 [43.8, 56.2]
	20%	6	51.2 [37.5, 65.0]	50.0 [41.2, 58.8]	38.7 [25.0, 53.8]	50.0 [41.2, 58.8]
	30%	10	55.0 [40.0, 70.0]	51.2 [43.8, 58.8]	60.0 [46.2, 73.8]	52.5 [47.5, 57.5]
	40%	13	50.0 [36.3, 63.7]	47.5 [40.0, 53.8]	36.2 [22.5, 51.2]	48.8 [42.5, 55.0]
	50%	16	51.2 [36.2, 66.2]	55.0 [48.8, 61.3]	51.2 [38.7, 65.0]	56.2 [48.8, 63.7]

Appendix

Table 5: Prompt-level consistency across all intervention depths. Tie-adjusted AIMES consistency (%) with pointwise 95% prompt-bootstrap confidence intervals in brackets (

n=40

matched prompts per cell).

			vs. Prompt Steering		vs. Fixed Multi
Model	Depth	$\ell$	$D_{x}$	$D_{y}$	$D_{x}$	$D_{y}$
Llama-3.1-8B	$10\%$	3	31.2 [21.2, 42.5]	48.8 [41.2, 56.2]	45.0 [33.8, 56.2]	50.0 [43.8, 57.5]
	$20\%$	6	43.8 [31.2, 56.2]	48.8 [41.2, 56.2]	45.0 [33.8, 56.2]	47.5 [41.2, 53.8]
	$30\%$	10	48.8 [36.3, 61.3]	55.0 [47.5, 62.5]	55.0 [43.8, 66.2]	51.2 [45.0, 57.5]
	$40\%$	13	42.5 [31.2, 53.8]	55.0 [48.8, 62.5]	52.5 [38.8, 66.2]	56.2 [51.2, 61.3]
	$50\%$	16	42.5 [31.2, 53.8]	51.2 [43.8, 58.8]	53.8 [42.5, 65.0]	52.5 [46.2, 58.8]

Appendix

Table 6: Claude-Opus-4.8 prompt-level consistency across all intervention depths. Tie-adjusted AIMES consistency (%) under Claude-Opus-4.8 with pointwise 95% prompt-bootstrap confidence intervals in brackets (

n=40

matched prompts per cell).

D_{x}

measures Care/Fairness retention and

D_{y}

additional Sanctity suppression.

Objective	AIMES $>$ Fixed	AIMES $>$ Prompt	AIMES $>$ Prompt
	on $Q$	on $Q$	on Relevance
$+\,$ Care $+\,$ Fairness	26/50	29/50	41/50
$+\,$ Loyalty $+\,$ Authority	24/50	22/50	30/50
$+\,$ Care $+\,$ Fairness $-\,$ Sanctity	26/50	30/50	35/50
Overall	76/150	81/150	106/150

Appendix

Table 7: Blind response-quality comparison. Number of model–depth settings in which AIMES attains a higher mean score than each comparator. Overall quality is defined as

Q=(C+F+R)/3

, where

C

F

, and

R

denote coherence, fluency, and relevance. Counts are descriptive rather than significance tests.

Figure 11: Variation in AIMES steering strength across intervention depths. For each model and objective, we measure the within-response range of the target-value steering coefficients, $R_{\alpha,k}=\max_{t}\alpha_{k,t}-\min_{t}\alpha_{k,t}$ . Each point shows the mean coefficient range across target-disjoint responses at the corresponding model-specific intervention layer, and shaded regions denote $95\%$ prompt-bootstrap confidence intervals. Larger values indicate greater changes in steering strength during generation. Results are reported separately at each of the ten sampled depths; no averaging is performed across intervention layers.

Figure 12: Claude-Opus-4.8 judge-based analysis of the depth-wise AIMES advantage. The analysis uses the same ten sampled intervention depths and the same $D_{x}/D_{y}$ definitions as Figure 3 , but re-scores the identical generations with Claude-Opus-4.8. Positive values indicate an advantage for AIMES ; black outlines denote depths at which the advantage is positive for all five models.

Figure 13: Claude-Opus-4.8 judge-based analysis of GeoGain for the competing three-value objective. Prompt-paired GeoGain advantage of AIMES over Fixed Multi-Value Steering (left) and Prompt Steering (right) across all ten sampled intervention depths for

(\uparrow\mathrm{Care}\;\uparrow\mathrm{Fairness}\;\downarrow\mathrm{Sanctity})

. Positive values indicate higher GeoGain metric for AIMES vs. Fixed Multi-Value Steering and Prompt Steering. Under Claude-Opus-4.8, 30/50 model-depth settings favor AIMES over Fixed Multi-Value Steering and 50/50 favor AIMES over Prompt Steering.

Objective	AIMES vs. Fixed Multi	AIMES vs. Prompt Steering
$(\uparrow\mathrm{Care}\;\uparrow\mathrm{Fairness})$	35/50	0/50
$(\uparrow\mathrm{Loyalty}\;\uparrow\mathrm{Authority})$	29/50	21/50
$(\uparrow\mathrm{Care}\;\uparrow\mathrm{Fairness}\;\downarrow\mathrm{Sanctity})$	30/50	50/50

Appendix

Table 8: Claude-Opus-4.8 GeoGain Analysis. Number of model–depth cells, out of 50 (five models

\times

ten depths), in which AIMES attains higher prompt-level GeoGain than each comparator.

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 281 papers

Sep 5, 2026

Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment

Activation steering controls LLM behavior at inference time by adding learned directions to hidden states, but existing methods handle one concept at a time. Pluralistic alignment, where different stakeholders need different value emphases, requires steering multiple dimensions simultaneously. We show that naive steering produces substantial spillover: the effect intended for one value leaks into others. This parallels the treatment-versus-spillover decomposition in causal inference. We trace spillover to geometric entanglement of steering directions, captured by their Gram matrix, and derive a zero-cost correction from an activation-norm-penalized objective that decouples each direction's contribution exactly. Our end-to-end pipeline requires no fine-tuning, no reward model, and no manual prompt engineering: given only domain questions, it automatically discovers value dimensions, extracts directions, diagnoses entanglement, and applies corrected steering. On climate discourse, the correction improves the net steering effect from +5.9% to +14.0%, validated over 100,000 pairwise judgments.

Weici Pan, Xander Barron, Jiawei Zhou +1

Stony Brook University

Language Model Steering Inference-Time Language Model Alignment Activation Steering Multi-Objective Language Model Alignment

Ask PDF

Jun 7, 2026

Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior

Activation steering provides a lightweight inference-time mechanism for controlling large language models (LLMs) by modifying their internal activation vectors toward desired behaviors. Most existing methods compute a fixed steering direction in the original activation space, typically from pairs of contrastive examples using mean differences, linear probes, or arbitrary separability criteria. While effective to a certain extent, these methods treat behavioral control as a global, linear, additive offset: the same direction is applied across inputs, and behaviors are linearly separable. This can be restrictive when behavioral features vary nonlinearly across the activation space or lie on curved and anisotropic manifolds, where the optimal intervention may be input-dependent. To address this limitation, we propose INNSteer, a nonlinear activation steering framework based on invertible latent transformations. Rather than searching for a better steering vector in the original representation space, INNSteer learns a lightweight invertible neural network

φ

that maps an LLM's activations into a latent space where behavioral classes are more amenable to linear control. At inference time, activations are mapped through

φ

, steered in the latent space, and mapped back through the exact inverse transformation

φ^{-1}

. This makes a simple latent-space translation become a nonlinear, input-dependent intervention in the original activation space. Across experiment settings on multiple LLM families, scales, behavioral traits, and safety benchmarks, INNSteer consistently improves model control over linear, transport-based, and nonlinear steering baselines while largely preserving generation fluency.

Tuc Nguyen, Thai Le

Indiana University Bloomington, USA

Invertible Neural Networks Language Model Steering Activation Steering

Ask PDF

May 28, 2026

UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering

Activation-based control steers large language models (LLMs) by intervening on their internal representations during inference, and has emerged as an effective paradigm for controlling behaviors such as persona and style. However, existing methods often rely on fixed steering directions or task-specific intervention modules, making them difficult to adapt to fine-grained concepts and compositional constraints. We propose UniSteer, a text-guided activation flow matching model that learns a conditional distribution over residual-stream activations from natural-language conditions. Instead of fitting a separate intervention for each target behavior, UniSteer learns a universal conditional velocity field in activation space. At inference time, UniSteer performs flow inversion by partially transporting a source activation toward a latent state and regenerating it under a target textual condition before injecting it back into the frozen LLM. The same conditional model supports activation-space classification by selecting the textual label with the lowest reconstruction energy. Experiments on three target LLMs show that UniSteer provides a unified interface across behavioral control, truthfulness steering, fine-grained concept steering, multi-constraint instruction following, and activation-space classification.

Yingdong Shi, Ruiming Zhang, Changming Li +4

ShanghaiTech University

Flow Matching Language Model Steering Activation Steering

Ask PDF

Abstract

Language Model Steering

Explore similar work

Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment

Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior

UniSteer: Text-Guided Flow Matching in Activation Space for Versatile LLM Steering