Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment

Weici Pan; Xander Barron; Jiawei Zhou; Zhenhua Liu

cs.AISep 5, 2026

Spillover-Aware Multi-Value Steering for Pluralistic LLM Alignment

Authors: Weici Pan, Xander Barron, Jiawei Zhou, Zhenhua Liu

Organizations: Stony Brook University

Abstract

Activation steering controls LLM behavior at inference time by adding learned directions to hidden states, but existing methods handle one concept at a time. Pluralistic alignment, where different stakeholders need different value emphases, requires steering multiple dimensions simultaneously. We show that naive steering produces substantial spillover: the effect intended for one value leaks into others. This parallels the treatment-versus-spillover decomposition in causal inference. We trace spillover to geometric entanglement of steering directions, captured by their Gram matrix, and derive a zero-cost correction from an activation-norm-penalized objective that decouples each direction's contribution exactly. Our end-to-end pipeline requires no fine-tuning, no reward model, and no manual prompt engineering: given only domain questions, it automatically discovers value dimensions, extracts directions, diagnoses entanglement, and applies corrected steering. On climate discourse, the correction improves the net steering effect from +5.9% to +14.0%, validated over 100,000 pairwise judgments.

Figures & tables

Direction	$\kappa(\mathbf{G})$	$\mathbf{H}$	Target	Spillover	NSE
Contrast	24.4	$\mathbf{G}^{-1}$	56.8%	42.9%	+14.0%
Contrast	24.4	$\mathbf{I}$	55.2%	49.4%	+5.9%
RepE	7.8	$\mathbf{G}^{-1}$	57.5%	49.5%	+7.8%
RepE	7.8	$\mathbf{I}$	53.3%	49.6%	+3.7%
ActAdd	14.2	$\mathbf{G}^{-1}$	53.4%	46.9%	+6.4%
ActAdd	14.2	$\mathbf{I}$	52.1%	48.2%	+3.2%

Table 1: Net steering effect on Llama-3.2-3B-Instruct (200 questions, 4 target dimensions, 3 judges per comparison). Each generative direction type is shown with and without

\mathbf{G}^{-1}

correction. The condition number

\kappa(\mathbf{G})

measures direction entanglement (§ 4 ).

Method	Target dim	Target	Spillover	NSE
contrast_ginv	Accuracy	65.0%	48.8%	+16.3%
	TechEcon	53.0%	39.6%	+13.4%
	SocialJustice	60.5%	37.2%	+23.2%
	FutureEthics	48.8%	46.0%	+2.7%
repe_ginv	Accuracy	61.8%	49.6%	+12.2%
	TechEcon	55.5%	41.4%	+14.1%

Table 2: Per-dimension breakdown for the two best corrected methods (Llama-3.2-3B-Instruct, 200 questions). The two methods have complementary strengths across value dimensions. Full breakdown for all 9 methods is in Appendix J .

Figure 1:

\mathbf{G}^{-1}

NSE gain by direction type on three models: Llama-3.2-3B, Llama-3.1-8B, and Qwen2.5-7B (RepE was not extracted for Qwen). Gray bars show naive steering (

\mathbf{H}=\mathbf{I}

); colored bars show corrected steering (

\mathbf{H}=\mathbf{G}^{-1}

); numbers above bars give NSE in percentage points. The correction improves every direction type on every model. Full 8B and Qwen tables are in Appendices J and O .

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Dimension	Description	Prev.
Accuracy	Factual precision, scientific evidence	91.4%
Ecology	Ecosystem health, biodiversity	41.9%
Equity_Present	Distributive justice, vulnerable communities	18.3%
Equity_Future	Intergenerational obligations	28.9%
Efficiency	Cost-effectiveness, resource optimization	34.9%
TechOpt	Technological innovation, engineering solutions	23.7%

Appendix

Table 3: Twelve candidate value dimensions and their prevalence in the StackExchange climate dataset (

n=1{,}255

). Prevalence is the fraction of questions labeled as involving each dimension.

Group	Removed sub-dim	Labels changed
TechEcon	Efficiency	14.4%
TechEcon	TechOpt	3.3%
SocialJustice	Equity_Present	1.9%
SocialJustice	Care	3.2%
SocialJustice	Procedural	3.3%
SocialJustice	Liberty	1.2%

Appendix

Table 4: Within-group label stability. Each row shows the fraction of questions whose compressed label changes when one sub-dimension is removed. Accuracy is a single dimension and is not ablated.

	Accuracy	TechEcon	SocialJustice	FutureEthics
F1 (mean $\pm$ std)	$0.93\pm 0.04$	$0.79\pm 0.02$	$0.65\pm 0.03$	$0.76\pm 0.02$

Appendix

Table 5: Probe F1 scores at layer 20 (Llama-3.1-8B-Instruct, 5-fold CV,

n=1{,}255

Model	Probe	ActAdd	Contrast	RepE
Llama-3.2-3B	1.90	14.15	24.37	7.77
Llama-3.1-8B	1.75	11.45	16.37	11.33
Qwen2.5-7B	1.74	10.73	11.78	—

Appendix

Table 6: Gram matrix condition number

\kappa(\mathbf{G})

across models and direction types. Generative directions are consistently an order of magnitude more entangled than discriminative ones. RepE was not extracted for Qwen.

Model	Direction	$\lambda_{1}$	$\lambda_{2}$	$\lambda_{3}$	$\lambda_{4}$
3B	Probe	36.3%	24.0%	20.6%	19.1%
	ActAdd	63.6%	21.0%	10.9%	4.5%
	Contrast	76.7%	13.9%	6.3%	3.1%
	RepE	55.3%	22.8%	14.8%	7.1%
8B	Probe	34.6%	24.4%	21.2%	19.8%
	ActAdd	60.3%	22.4%	12.1%	5.3%

Appendix

Table 7: Eigenvalue distribution (percentage of total variance) of Gram matrices. Generative directions concentrate 55–77% of variance in a single eigenvalue; probe directions distribute variance nearly uniformly.

3B Probe ( $\kappa=1.90$ )
	Acc	TE	SJ	FE
Acc	1.00	$-0.12$	$-0.10$	$-0.05$
TE	$-0.12$	1.00	$+0.18$	$+0.21$
SJ	$-0.10$	$+0.18$	1.00	$+0.21$
FE	$-0.05$	$+0.21$	$+0.21$	1.00
3B Contrast ( $\kappa=24.37$ )

Appendix

Table 8: Full Gram matrices for Llama-3.2-3B-Instruct (layer 14). Entries are cosine similarities between unit-normalized directions.

8B Probe ( $\kappa=1.75$ )
	Acc	TE	SJ	FE
Acc	1.00	$-0.08$	$-0.11$	$-0.05$
TE	$-0.08$	1.00	$+0.16$	$+0.20$
SJ	$-0.11$	$+0.16$	1.00	$+0.14$
FE	$-0.05$	$+0.20$	$+0.14$	1.00
8B Contrast ( $\kappa=16.37$ )

Appendix

Table 9: Full Gram matrices for Llama-3.1-8B-Instruct (layer 20).

Qwen Probe ( $\kappa=1.74$ )
	Acc	TE	SJ	FE
Acc	1.00	$-0.05$	$-0.10$	$-0.05$
TE	$-0.05$	1.00	$+0.19$	$+0.18$
SJ	$-0.10$	$+0.19$	1.00	$+0.18$
FE	$-0.05$	$+0.18$	$+0.18$	1.00
Qwen Contrast ( $\kappa=11.78$ )

Appendix

Table 10: Full Gram matrices for Qwen2.5-7B-Instruct (layer 16). RepE directions were not extracted for Qwen.

Dimension	Contrast	ActAdd	RepE	Random
Accuracy	0.093	0.082	0.082
TechEcon	0.019	0.024	0.061
SocialJustice	0.020	0.057	0.034
FutureEthics	0.011	0.027	0.028
Mean	0.036	0.048	0.051	$0.012\pm 0.009$

Appendix

Table 11: Cosine similarity between probe and generative directions at layer 20 (Llama-3.1-8B-Instruct). All values are comparable to the random baseline in

\mathbb{R}^{4096}

, indicating that discriminative and generative directions occupy near-orthogonal subspaces.

Persona $\downarrow$ \ Judged $\rightarrow$	Acc	TE	SJ	FE
Accuracy	$\mathbf{+0.65}$	$-0.25$	$-0.25$	$+0.10$
TechEcon	$-1.00$	$\mathbf{+2.55}$	$-0.25$	$-0.50$
SocialJustice	$-0.75$	$-0.40$	$\mathbf{+2.80}$	$+0.90$
FutureEthics	$-0.80$	$-0.65$	$+0.05$	$\mathbf{+2.80}$

Appendix

Table 12: Persona generation lift matrix. Entry

(i,j)

is the mean score under persona

i

minus the neutral baseline score, judged on dimension

j

n=20

questions, 0–10 absolute scoring. Diagonal entries (bold) show that each persona specifically increases its target dimension; off-diagonal entries are predominantly negative.

$\mathbf{M}$ in probe basis
	Acc	TE	SJ	FE
$\alpha_{\text{Acc}}$	$-0.021$	$-0.003$	$-0.007$	$-0.016$
$\alpha_{\text{TE}}$	$-0.031$	$+0.013$	$+0.013$	$+0.008$
$\alpha_{\text{SJ}}$	$-0.003$	$+0.012$	$+0.002$	$+0.014$
$\alpha_{\text{FE}}$	$-0.005$	$-0.003$	$+0.012$	$+0.016$
$\mathbf{M}$ in contrast basis

Appendix

Table 13: Estimated causal Jacobian

\mathbf{M}

. Entry in row

j

, column

k

is the estimated

\partial V_{k}/\partial\alpha_{j}

: how steering along direction

j

affects value score

k

. All entries are

O(10^{-2})

, indicating negligible linear causal effect at the first-order level.

Figure 2: Estimated causal Jacobian $\mathbf{M}$ in probe (left) and contrast (right) direction bases. Black squares mark diagonal entries. A well-behaved $\mathbf{M}$ would show strong positive diagonal and weak off-diagonal; instead, all entries are $O(10^{-2})$ with no clear diagonal structure.

$\alpha$	$\varepsilon$	Target	Spillover	NSE
15	0	45.0%	45.4%	$-0.4$ %
15	0.1	45.0%	45.0%	$+0.0$ %
20	0	57.5%	45.0%	$\mathbf{+12.5}$ %
20	0.1	51.2%	46.3%	$+5.0$ %
20	0.3	43.8%	44.2%	$-0.4$ %
20	0.5	42.5%	47.5%	$-5.0$ %

Appendix

Table 14: 8B actadd_ginv sweep (

n=20

questions, 2 target dims, 3 judges). Best configuration:

\alpha=20

\varepsilon=0

(pure

\mathbf{G}^{-1}

, no regularization). Only a subset of the

4\times 4

grid is shown; omitted entries have NSE below

+2\%

Method	Target	Spillover	NSE
contrast_ginv, $\alpha{=}15$	71.2%	64.6%	$+6.7$ %
contrast_naive, $\alpha{=}15$	62.5%	63.1%	$-0.6$ %
contrast_ginv, $\alpha{=}20$	74.4%	61.0%	$+13.3$ %
contrast_naive, $\alpha{=}20$	65.6%	64.6%	$+1.0$ %
contrast_ginv, $\alpha{=}26$	73.8%	63.3%	$+10.4$ %
contrast_naive, $\alpha{=}26$	68.1%	63.7%	$+4.4$ %

Appendix

Table 15: 3B contrast sweep (

n=20

questions, 4 target dims, 3 judges).

\mathbf{G}^{-1}

correction improves NSE at every alpha level, with the gap widening as

\alpha

increases.

Method	Target dim	Target	Spillover	NSE
contrast_ginv	Accuracy	64.8%	49.7%	$+15.1$ %
	TechEcon	53.1%	40.1%	$+13.0$ %
	SocialJustice	59.4%	37.7%	$+21.7$ %
	FutureEthics	50.9%	46.3%	$+4.6$ %
contrast_naive	Accuracy	59.9%	50.7%	$+9.2$ %
	TechEcon	51.9%	49.9%	$+2.0$ %

Appendix

Table 16: Per-dimension results for all methods on Llama-3.2-3B-Instruct (200 questions, 3 judges per comparison, ties counted as 0.5).

Method	$\mathbf{H}$	Target	Spillover	NSE
actadd_ginv	$\mathbf{G}^{-1}$	52.8%	49.6%	$+3.3$ %
actadd_naive	$\mathbf{I}$	51.3%	49.4%	$+1.9$ %
contrast_ginv	$\mathbf{G}^{-1}$	52.9%	48.9%	$+4.0$ %
contrast_naive	$\mathbf{I}$	50.2%	48.9%	$+1.2$ %
repe_ginv	$\mathbf{G}^{-1}$	50.9%	48.3%	$+2.7$ %
repe_naive	$\mathbf{I}$	51.2%	48.9%	$+2.3$ %

Appendix

Table 17: Full 8B results (Llama-3.1-8B-Instruct, 100 questions, layer 20,

\alpha=20

\mathbf{G}^{-1}

correction improves all three generative direction types. Effect magnitudes are smaller than 3B, consistent with the reduced entanglement at larger scale (

\kappa_{\mathrm{contrast}}=16.4

vs 24.4).

Method	Target dim	Target	Spillover	NSE
actadd_ginv	Accuracy	55.0%	50.2%	$+4.8$ %
	TechEcon	52.8%	50.7%	$+2.0$ %
	SocialJustice	52.5%	48.5%	$+4.0$ %
	FutureEthics	51.1%	48.9%	$+2.2$ %
actadd_naive	Accuracy	51.2%	50.2%	$+1.0$ %
	TechEcon	51.6%	48.2%	$+3.4$ %

Appendix

Table 18: Per-dimension breakdown for 8B (combined v2+v3, 100 questions per target dimension). Probe and layer_sep per-dimension results are omitted as both have near-chance mean NSE.

Domain	$n$	Silhouette
Energy policy	518	0.255
Medical ethics	741	0.234
AI governance	673	0.234
Economic inequality	800	0.227
Education policy	482	0.224
Immigration	800	0.223

Appendix

Table 19: Unsupervised clustering results across 10 domains (K-means,

K=2

). All domains select

K=2

as optimal. Silhouette scores are uniformly low, and the resulting clusters reflect question format rather than value structure.

Experiment	Model	Methods	Questions	Dims	Judgments
3B main (§ 5.2 )	Llama-3.2-3B	9	200	4	86,400
8B v2 (§ 5.3 )	Llama-3.1-8B	9	50	4	21,600
8B v3 (§ 5.3 )	Llama-3.1-8B	9	150	4	64,800
Qwen (App. O )	Qwen2.5-7B	4	50	4	9,600
3B sweep (App. I )	Llama-3.2-3B	10	20	4	—
8B sweep (App. I )	Llama-3.1-8B	10	20	2	—

Appendix

Table 20: Experimental configuration summary. “Judgments” counts individual judge votes (3 per comparison

\times

K

evaluated dimensions

\times

questions

\times

methods). Sweep experiments are excluded from the total as they use a different checkpoint format.

	NSE (pp)			Fidelity
Configuration	Naive	Ginv	Prompt	Naive	Ginv	Prompt
Acc+SJ	$-5.00$	$+5.50$	$+5.50$	$-0.34$	$+0.69$	$+0.60$
TE+FE	$+0.50$	$-1.25$	$+6.00$	$+0.07$	$-0.16$	$+0.38$
Acc+FE	$+0.00$	$+4.50$	$+6.00$	$+0.00$	$+0.61$	$+0.44$
Acc+TE+SJ	$-6.00$	$-3.67$	$+4.33$	$-0.91$	$-0.42$	$+0.67$
Acc+SJ+FE	$-2.50$	$+4.17$	$+9.00$	$+0.04$	$+0.62$	$+0.82$

Appendix

Table 21: Mixed-

\boldsymbol{w}

results on Llama-3.2-3B-Instruct (

n=100

questions per configuration and condition). Weights are equal on the named support, e.g.

\boldsymbol{w}_{\mathrm{Acc{+}SJ}}=(0.5,0,0.5,0)

; Uniform is

(0.25,0.25,0.25,0.25)

. NSE is the mean win rate on active dimensions minus the mean on inactive dimensions; it is undefined for Uniform, which has no inactive dimensions. Fidelity is defined in the text. NSE means are over the five non-uniform configurations; fidelity means are over all six.

Method	$\mathbf{H}$	Target	Spillover	NSE
contrast_ginv	$\mathbf{G}^{-1}$	65.0%	57.8%	$\mathbf{+7.2}$ %
contrast_naive	$\mathbf{I}$	59.6%	57.1%	$+2.5$ %
actadd_ginv	$\mathbf{G}^{-1}$	62.8%	57.3%	$+5.4$ %
actadd_naive	$\mathbf{I}$	61.3%	58.4%	$+2.9$ %

Appendix

Table 22: Results on Qwen2.5-7B-Instruct (50 questions, layer 16,

\alpha=10

, 3 judges per comparison).

\mathbf{G}^{-1}

correction improves both direction types.

Method	Target dim	Target	Spillover	NSE
contrast_ginv	Accuracy	73.3%	56.6%	$+16.8$ %
	TechEcon	67.3%	56.7%	$+10.7$ %
	SocialJustice	68.0%	55.3%	$+12.7$ %
	FutureEthics	51.3%	62.6%	$-11.2$ %
contrast_naive	Accuracy	74.0%	54.3%	$+19.7$ %
	TechEcon	57.7%	55.9%	$+1.8$ %

Appendix

Table 23: Per-dimension breakdown for Qwen2.5-7B-Instruct. FutureEthics is consistently the weakest dimension across all methods, suggesting that the FE contrast direction in Qwen does not carry a strong value-specific signal in this domain.

$K$	$\kappa(\mathbf{G})$	Naive	Ginv	Gain	Bootstrap $p$
4	24.4	$+5.9\%$	$+14.0\%$	$+8.1$ pp	—
8	157.6	$-3.5\%$	$+9.0\%$	$+12.5$ pp	$0.0000$
10	208.7	$-3.4\%$	$+6.7\%$	$+10.1$ pp	$0.0060$
12	329.7	$+6.0\%$	$+17.8\%$	$+11.9$ pp	$<0.01$

Appendix

Table 24: Macro NSE and Gram-matrix conditioning under

K

-scaling on the climate domain. The correction’s gain over naive steering is positive and significant at every

K

, and at least as large as at

K=4

K=4

values are the main-text Table 1 numbers (

n=200

);

K\geq 8

use

n=50

Target	Naive	Ginv	Gain	Wilcoxon $p$
Accuracy	$-33.4\%$	$-3.7\%$	$+29.7$	$0.000$
Ecology	$-10.0\%$	$+0.6\%$	$+10.6$	$0.127$
Efficiency	$-15.7\%$	$-6.7\%$	$+9.0$	$0.181$
Equity_Future	$+2.6\%$	$+7.1\%$	$+4.6$	$0.663$
Equity_Present	$+6.0\%$	$+36.0\%$	$+30.0$	$0.001$
Precaution	$+11.6\%$	$+11.4\%$	$-0.1$	$0.994$

Appendix

Table 25: Per-target NSE at

K=8

(climate). Naive steering drives several targets negative (Accuracy

-33.4\%

); the correction recovers them. Significance by Wilcoxon signed-rank.

Dimension	Naive	Ginv	Gain	Wilcoxon $p$
Evidence	$-2.7\%$	$+32.6\%$	$+35.3$	$0.0006$
Autonomy	$-8.6\%$	$+4.5\%$	$+13.2$	$0.045$
DistributiveEthics	$+11.0\%$	$+14.0\%$	$+3.0$	$0.187$
CareDuties	$+10.3\%$	$+5.7\%$	$-4.6$	$0.62$
MACRO	$+2.5\%$	$+14.2\%$	$+11.7$	$0.0043$

Appendix

Table 26: Per-dimension and macro NSE on the from-scratch Medical Ethics domain (

n=50

\kappa(\mathbf{G})=5.30

). The gain concentrates on Evidence, the dimension most orthogonal to the others; the three related normative dimensions gain less. Significance by Wilcoxon signed-rank.

Method	DeepSeek NSE	GPT-4o NSE
probe_naive	$+1.50\%$	$+8.08\%$
probe_ginv	$+3.83\%$	$+5.58\%$
contrast_naive	$+4.00\%$	$+3.75\%$
contrast_ginv	$+15.25\%$	$+15.42\%$
actadd_naive	$-1.67\%$	$+3.67\%$
actadd_ginv	$+4.00\%$	$+5.83\%$

Appendix

Table 27: Per-method NSE under two judges from different families (DeepSeek and GPT-4o), on the same

50

questions. The correction improves over naive steering under both judges for contrast, ActAdd, and RepE; the one exception is probe under GPT-4o.

Layer	$\kappa(\mathbf{G})$	Naive	Ginv	Gain
10	33.33	$+5.50\%$	$+8.50\%$	$+3.00$ pp
14	24.37	$+4.00\%$	$+15.25\%$	$+11.25$ pp
18	27.75	$+8.00\%$	$+14.08\%$	$+6.08$ pp

Appendix

Table 28: Layer sweep on Llama-3.2-3B-Instruct: contrast directions, fixed

\alpha=26

, first 50 canonical questions. Directions and the Gram matrix are re-extracted at each layer. The

L=14

row sub-selects the main-experiment judgments on the same questions.

$\beta$	$\kappa(\mathbf{G}+\beta\mathbf{I})$	$\\|\boldsymbol{\alpha}_{\beta}\\|$	Acc	TE	SJ	FE
$\boldsymbol{w}=(0,0.5,0,0.5)$ (TE+FE)
0	24.37	3.13	0.000	0.500	0.000	0.500
0.1	14.02	1.82	0.038	0.423	0.119	0.394
0.5	5.70	0.71	0.090	0.306	0.191	0.288
2	2.38	0.24	0.095	0.187	0.155	0.183
$\boldsymbol{w}=e_{\mathrm{SJ}}$ (one-hot SocialJustice)

Appendix

Table 29: Regularization path of

\mathbf{H}_{\beta}=(\mathbf{G}+\beta\mathbf{I})^{-1}

on the 3B contrast Gram matrix. Projections are

\mathbf{G}\boldsymbol{\alpha}_{\beta}

, the achieved emphasis along each direction;

\kappa(\mathbf{G}+\beta\mathbf{I})

depends only on

\beta

. At

\beta=0

the projections equal the request exactly; larger

\beta

shrinks the coefficient norm and reintroduces off-support projection in order of cosine with the active set.

Target dimension	contrast_ginv	Prompting (P1)	$p$
Accuracy	$+14.42\%$	$+14.92\%$	$0.924$
TechEcon	$+14.33\%$	$+25.58\%$	$0.014$
SocialJustice	$+22.50\%$	$+15.58\%$	$0.163$
FutureEthics	$+4.58\%$	$+11.50\%$	$0.145$
MACRO	$+13.96\%$	$+16.90\%$	$0.231$

Appendix

Table 30: Per-dimension and macro NSE for instruction prompting (P1) against corrected steering (contrast_ginv) on the full 200 canonical questions (Llama-3.2-3B-Instruct, main judge protocol).

p

is a paired bootstrap on the per-question difference; the macro difference is also not significant under a Wilcoxon signed-rank on the same pairing (

p=0.23

Configuration	Macro NSE	Median PPL	Fluency
Unsteered baseline	—	2.19	4.15
Prompting (P1)	$+17.08\%$	2.32	4.03
contrast_ginv ( $\alpha=26$ )	$+12.88\%$	2.48	3.97
Stack ( $\alpha=26$ )	$+37.25\%$	2.79	4.01
Stack ( $\alpha=13$ )	$+27.71\%$	2.49	3.99

Appendix

Table 31: Operating-point ladder on the first 100 canonical questions (Llama-3.2-3B-Instruct, contrast directions, layer 14). Macro NSE from the powered stacking run (

n=100

questions per target dimension); the prompting and corrected-steering reference rows reuse the corresponding judgments restricted to this subset, hence the expected subset variation from Table 30 . Median perplexity and mean fluency (1–5) are computed on the same subset (100 questions; 400 generations per four-dimension arm); both metrics follow the protocol of Appendix T . NSE is measured against the unsteered baseline, so the baseline row has none.

Steered-selection rate	contrast_naive	contrast_ginv
Target items	$0.149$	$0.327$
Spillover items	$0.115$	$0.136$
Agreement	Raw	$\kappa$
Human–judge ( $n=240$ )	$0.704$	$0.508$
Inter-annotator (full-coverage) †	$0.643$	$0.405$

Appendix

Table 32: Human validation of the pairwise emphasis judgments. Top: rate at which annotators select the steered response over the unsteered baseline (majority vote; ties counted in the denominator), by item type and method. Bottom: agreement of the human labels with the LLM judge, and among annotators. Human–judge agreement (

\kappa=0.51

) exceeds the cross-judge level (

\kappa=0.39

), and inter-annotator agreement is moderate. One outlier annotator, whose labels correlated near zero with both the other annotators and the judge, was excluded. † Inter-annotator agreement (Fleiss

\kappa

, three-way) is over the three annotators who labelled all 240 items; the human–judge and steered-selection figures additionally use a fourth annotator who labelled 48 items.

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 281 papers

Jun 9, 2026

Scenario-based Probing and Steering Cultural Values in Large Language Models--Extended Version

Large Language Models (LLMs) are deployed across cultural contexts but often reflect homogenized values inherited from training data. Evaluations of cultural alignment typically rely on direct prompting with survey-style questions, which frequently elicit neutral or safety-aligned responses and fail to capture underlying model preferences. We propose a framework for probing and steering latent cultural representations in LLMs along the two Inglehart--Welzel axes of the World Values Survey (WVS). By translating social value questions into scenario-based behavioral dilemmas, we extract token-level probabilities to measure implicit values and apply activation steering, optionally combined with country-conditioned prompting, to shift model behavior without retraining. Across three open-source LLMs and four target cultures, we find substantial variation in steerability and identify latent entanglement, where interventions along one cultural dimension induce shifts along another. This coupling mirrors correlations in human WVS data and persists across activation, prompt, and hybrid steering. It constrains axis-independent alignment, though general task performance is largely preserved.

Trung Duc Anh Dang, Tung Kieu, Sarah Masud

University of Copenhagen · Aalborg University

Language Model Steering Cultural Alignment Cultural Bias in Language Models Activation Steering Language Model Probing

Ask PDF

Apr 9, 2026

Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence

Alignment in LLMs is more brittle than commonly assumed: misalignment can be induced by adversarial prompts, benign fine-tuning, emergent misalignment, and goal misgeneralization. Recent evidence suggests that some misalignment behaviors are encoded as linear structure in activation space, making it tractable via activation steering, which could be used as a lightweight runtime defense. We implement three methods: Steer-With-Fixed-Coefficient (SwFC), which applies uniform additive steering, and two novel projection-aware methods, Steer-to-Target-Projection (StTP) and Steer-to-Mirror-Projection (StMP), that use a logistic regression decision boundary to selectively intervene only on tokens whose activations fall below the threshold. We evaluate these methods on two threat models, dishonesty and dismissiveness, using malicious system prompts as a controlled proxy for misalignment. We conduct our experiments on two architectures (Llama-3.3-70B-Instruct and Qwen3.6-27B). All methods substantially recover alignment. StTP and StMP preserve general capabilities (MMLU, MT-Bench, AlpacaEval) better than uniform steering. Finally, we show that our honesty steering generalizes to out-of-distribution scenarios: a single honesty direction extracted from the aligned model significantly raises scores on the MASK benchmark, suppresses deception in multi-agent settings (Among Us), doubles the hidden-behavior discovery rate on AuditBench, and restores honesty in an emergently misaligned model.

Niklas Herbster, Martin Zborowski, Alberto Tosato +2

Tara Research · Technical University of Munich · Mila Quebec AI Institute

LLM Safety Alignment Language Model Robustness Activation Steering LLM Honesty

Ask PDF

Oct 27, 2026

Orthogonal and Parametrized Multi-Attribute Steering of LLMs

Activation steering offers an effective and efficient way to modify an LLM's behavior at inference time. However, it remains unclear how to apply steering such that the signal is consistently strong and the steering strength has a universally interpretable meaning. We address this first by identifying the most steerable interval for each steering vector and rescaling the steering range accordingly. Building on this, we introduce a more interpretable method for steering, called parameterized steering which instead of pushing activations in a direction, it sets them at the desired trait strength. Next, with multi-trait steering, we find that naively combining vectors causes them to interfere with one another. In response, we develop a method to orthogonalize the vectors; it decouples these traits, allowing a model to exhibit one behavior without inducing the others, thus enabling multiple traits to be steered simultaneously.

Remy Ogasawara, Cullen Anderson, Narmeen Fatimah Oozeer +4

Kahlert School of Computing University of Utah Salt Lake City, UT 84112, USA · Department of Computer Science University of Massachusetts Amherst Amherst, MA 01003 · Martian AI San Francisco, CA 94117, USA +1

Language Model Steering Activation Steering

Ask PDF

Abstract

Language Model Steering

Explore similar work

Scenario-based Probing and Steering Cultural Values in Large Language Models--Extended Version

Activation Steering for Aligned Open-ended Generation without Sacrificing Coherence

Orthogonal and Parametrized Multi-Attribute Steering of LLMs