Compositional Activation Steering Across Languages and Behaviors with Contextual Gates

Daniil Gurgurov; Akansh Maurya; Josef van Genabith; Simon Ostermann; Hitomi Yanaka

cs.CLOct 17, 2026

Compositional Activation Steering Across Languages and Behaviors with Contextual Gates

Authors: Daniil Gurgurov, Akansh Maurya, Josef van Genabith, Simon Ostermann, Hitomi Yanaka

Organizations: Saarland University · German Research Center for AI (DFKI) · RIKEN · The University of Tokyo ·

Abstract

Activation steering has emerged as an effective and precise way to influence the behaviours of language models. However, composing multiple steering directions to control several behaviours simultaneously remains challenging, particularly in multilingual settings and with training-free steering vectors. We address this challenge with contextual steering, jointly training gates that adapt the strength of fixed steering directions to the current hidden state and interactions between interventions. Across a broad range of languages, behavioural attributes, and models, our approach outperforms constant-strength steering and four strong baselines, including prompting. It also better preserves general knowledge and keeps the model's internal computation closer to that of the prompted model. Our analysis further shows how jointly trained gates adapt their contributions during composition, explaining the mechanisms underlying effective compositional steering.

Figures & tables

Panel of Figure 1 — Figure 1: Methodology. (1) For each attribute, we extract a steering direction $d_{\ell}$ and select a constant intervention strength $\alpha_{\star}$ , yielding CAA . (2) CLAS keeps $d_{\ell}$ fixed and learns a contextual gate that determines the intervention strength $\alpha_{\ell}$ from the hidden state, using cross-entropy (CE). (3) For multiple attributes, the gates are initialized from the corresponding single-attribute gates and jointly optimized with all interventions active. (4) At inference, the contextual interventions are composed in the residual stream, with each gate controlling its attribute’s contribution.

Figure 2: Single-attribute steering , averaged over 9 languages and 14 behavioural directions , per model. We compare prompting, CAA (constant), and CLAS (adaptive). A “+” above a CLAS bar denotes its gain over CAA. Detailed per-model results are in Appendix F .

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	76.5	75.7	75.0	73.2	63.4	71.7	69.2	73.6	73.3	72.4
CAA $\boldsymbol{\gamma}$	57.2	60.2	51.8	32.7	42.3	54.6	26.5	50.2	70.6	49.6
CLAS $\boldsymbol{\nu}(x)$	69.6	66.6	66.8	66.0	51.2	67.2	57.7	66.7	71.2	64.8
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	85.1	85.3	83.7	83.5	80.4	79.7	79.7	82.1	82.5	82.5
Seq-CLAS $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	80.9	80.4	79.2	79.4	74.9	75.7	74.2	78.2	81.8	78.3
K-Steer $\boldsymbol{\kappa}(x)$	45.3	50.5	39.3	37.8	41.5	54.3	33.7	36.1	50.8	43.2

Table 1: Language and behaviour composition , averaged over 9 models , per target language and method (126 language-behaviour pairs per method). Detailed per-model results are in Appendix G .

Figure 3: Double- and triple-attribute steering performance and knowledge retention. Radar plots (A.1 and A.2) show the harmonic mean averaged over all languages and behaviours, per model. Four methods are shown for visual clarity. Double results are averaged over 126 language-behaviour pairs; triple results are averaged over 108 language-style-safety triples. The table (right-side) reports accuracy on each knowledge benchmark.

Panel of Figure 4 — Figure 4: Characterization of the single-attribute gates. Example for Arabic steering on Qwen2.5-7B-Instruct. (A) CLAS gate activity across layers and tokens. (B) Generated-token projection onto the steering direction across layers. (C) Logit-lens KL divergence from the prompted model across layers. (D) PCA of generated-token representations at three depth levels. Other attributes exhibit similar patterns; quantitative results are in Table 2 .

	Metric ( $\downarrow$ )	Baseline	CAA	(J)CLAS
Single	Proj. $\overline{\|\Delta\|}$ vs. instr.	11.87	12.45	8.06
	KL( $\cdot\\|$ prompted)	9.14	7.64	5.23
	EMD vs. instr.	17.94	16.05	14.27
Comp.	Proj. $\overline{\|\Delta\|}$ vs. instr.	8.86	10.63	4.42
	KL( $\cdot\\|$ prompted)	10.59	9.02	5.98
	EMD vs. instr.	24.85	28.49	18.08

Table 2: Quantitative characterization of single- and multi-attribute steering, averaged over layers, attributes, and models. Proj.

\overline{|\Delta|}

measures the projection shift relative to the prompted model; KL the logit-lens divergence over the last 5 layers; and EMD the Earth Mover’s distance over PC1 and PC2. Lower values indicate closer alignment with the prompted model.

Figure 5: Characterization of the composition. (A) Generation-time gate coefficients for German and Chain-of-thought before and after joint training. (B) Per-layer sign agreement and coactivation for the same composition. (C) Mean cosine similarity between steering directions across layers, averaged over all attribute pairs for Llama-3.1-8B-Instruct. Quantitative co-adaptation results are reported in Table 3 , and orthogonality results for the remaining models are provided in Appendix D .

Metric	CAA	CLAS (sep.)	CLAS-Joint
Gate coact.	–	0.080	0.459
Sign agree.	–	0.713	0.394
Interf. ratio	0.736	0.765	0.793

Table 3: Composition diagnostics , averaged over all tested double- and triple-pairs for Llama-3.1-8B-Instruct. Gate coact. measures the mean product of the two gate magnitudes; sign agree. is the fraction of layer-token pairs where the gates have the same sign; and the interf. ratio measures the retained combined intervention strength relative to the sum of the individual strengths. Other models follow the same patterns.

Appendix figures & tables75 assets

Supplementary material from the paper’s appendix.

Appendix

Model family	CAA $\alpha$ range	Median norm	Bias LR	Gate LR grid	Anchor target	Anchor grid
Llama	0.25-1.0	11.4-11.9	1e-1	{3e-4, 1e-3, 3e-3}	0.5	{0, 0.25, 0.5, 1}
Aya	0.25-1.0	43.5	1e-1	{3e-3, 1e-2, 3e-2}	1.0
Mistral	0.1-0.25	6.2	1e-1	{3e-3, 1e-2, 3e-2}	0.25
Qwen	2.0-5.0	66.0-100.2	5e-1	{3e-3, 1e-2, 3e-2}	3.0
Gemma	4.0-16.0	203.3-307.3	5e-1	{1e-2, 3e-2, 1e-1}	10.0

Appendix

Table 4: Single-attribute CLAS gate hyperparameters for each model family. CAA

\alpha

range is the grid CAA’s own alpha is tuned over first. Median norm is the median residual-stream norm

\|h\|

(baseline, no steering, over all layers and attributes); see detailed per-layer norms reported in Appendix C . The gate is anchored toward the constant

\alpha

value empirically found to steer that model best with CAA, its own tuned alpha, rather than an arbitrary fixed point. The gate LR grid is shifted per family since residual-stream norms differ substantially across models; a larger norm needs a correspondingly larger weight LR to move the gate by a comparable amount, and vice versa, but each family is still swept over a wide range rather than fixed to one value. Gate LR and anchor weight are swept jointly, best kept by validation.

Method	Training	LR grid	Anchor	Notes
Joint-CLAS	$n_{\text{train}}{=}60$ , $n_{\text{val}}{=}20$ , 32 steps	same per-family grid as Table 4	none	both gates updated together
Sequential	$n_{\text{train}}{=}60$ , $n_{\text{val}}{=}20$ , 32 steps	same per-family grid as Table 4	none	first gate frozen
K-Steer	30 epochs, 1 step/call	1e-3 (Adam default)	n/a	$\alpha\in[0.02,0.5]$ , 8-point geometric sweep
ORBIT	training-free (closed-form directions)	boost grid: $\{0,0.05,0.1,0.25,0.5,0.75,1,1.5,2,3\}$	n/a	global mean- $b_{1}$ plane, module-median rescaling best boost on tuning-slice HM

Appendix

Table 5: Composite and baseline steering hyperparameters. Joint-CLAS and Sequential use the same per-family gate learning-rate grids as Table 4 , with the bias learning rate fixed per family and increased for models with larger residual-stream norms.

Model	Prompt	CAA	CLAS	CLAS (no anchor)	$\Delta$ (anchor)
Llama-3.1-8B	71.3	69.6	86.6	81.9	+4.7
Llama-3.2-3B	77.0	75.2	78.6	75.3	+3.3

Appendix

Table 6: Anchor ablation for CLAS, averaged over 9 languages and 14 behavioural directions. “CLAS (no anchor)” removes the anchor regularization by setting its weight to zero.

\Delta

is the difference between CLAS and CLAS (no anchor), in percentage points.

Model	Prompted	CAA	Joint-CLAS	Joint-CLAS (random init)	$\Delta$ (warm-start)
Llama-3.1-8B	63.2	45.4	83.2	72.1	+11.1
Llama-3.2-3B	64.9	47.3	77.1	67.8	+9.3

Appendix

Table 7: Joint gate initialization ablation , averaged over all 126 language-behaviour pairs. Joint-CLAS is warm-started from the independently trained single-attribute gates, while Joint-CLAS (random init) uses random initialization. Both use the same winning gate learning rate for each pair.

\Delta

is the difference between warm-start and random initialization, in percentage points. Prompted and CAA results are shown for reference.

Panel of Figure 6 — Figure 6: Residual norms of the raw difference-in-means vectors for each model and attribute. Norms vary substantially across models and generally increase with depth, consistent with observations for models using standard residual streams ( Team et al., 2026 ) .

Panel of Figure 7 — Figure 7: Orthogonality of steering directions across layers, averaged over all attribute pairs.

Languages (9)
Attribute	Success metric	Scorer	HM
German, Spanish, Italian, Russian, Polish	LFS	FastText LID	HM(LFS, OR)
Japanese, Korean	LFS	FastText LID	HM(LFS, OR)
Arabic	LFS	FastText LID	HM(LFS, OR)
English	LFS	FastText LID	HM(LFS, OR)
Style behaviours

Appendix

Table 8: Evaluation metrics for each attribute. The success metric measures whether the completion exhibits the target attribute, while Output Relevance (OR) measures whether it provides a relevant and substantive response. The final score is the harmonic mean (HM) of the success metric and OR, except for jailbreak, bias, and toxicity, where the success metric is used directly. LFS : fraction identified as the target language. COS : mean chain-of-thought score. HUS : mean humour score. FSS : mean formality score. JBS : fraction matching the target Qwen3Guard-Gen-8B safety/refusal label. TRS : fraction matching the correct TruthfulQA reference answer. BIS : fraction matching the BBQ reference answer. TXS : fraction matching the target Qwen3Guard-Gen-8B safety label.

Model	Larger judge	HM $r$	HM mean abs. diff.	Per-item agreement	Cohen’s $\kappa$
Llama-3.1-8B	Qwen3-30B-A3B	0.878	0.081	91.7%	0.832
Llama-3.1-8B	Qwen3-32B	0.882	0.067	91.7%	0.832
Llama-3.2-3B	Qwen3-30B-A3B	0.889	0.084	91.4%	0.828
Llama-3.2-3B	Qwen3-32B	0.930	0.057	92.7%	0.855

Appendix

Table 9: Judge validation. Agreement between two larger judges and the Qwen3-8B judge used throughout the main evaluation, evaluated on the same completions. We re-score every evaluation, covering 47,580 items per model. HM

r

is the Pearson correlation between the judges’ per-file harmonic means; per-item agreement and Cohen’s

\kappa

are computed from each item’s joint success flag.

Panel of Figure 8 — Figure 8: Output Relevance (OR) judge prompt, shared across every attribute.

Panel of Figure 9 — Figure 9: Formality judge prompt.

Panel of Figure 10 — Figure 10: Humour judge prompt.

Panel of Figure 11 — Figure 11: Chain-of-thought judge prompt.

Panel of Figure 12 — Figure 12: Jailbreak and toxicity evaluation via Qwen3Guard-Gen-8B; input and output format, not a written prompt.

Panel of Figure 13 — Figure 13: Truthfulness judge prompt (reference-based, against TruthfulQA’s correct and incorrect answers).

Panel of Figure 14 — Figure 14: Bias judge prompt (reference-based, against BBQ’s unbiased and stereotype-consistent answers).

Attribute	Pole	Instruction appended to the input ( "{input}\n\nRespond {clause}." )
Language	German	in German
	Japanese	in Japanese
	Russian	in Russian
	Korean	in Korean
	Spanish	in Spanish
	Italian	in Italian

Appendix

Table 10: Prompted baseline

\boldsymbol{\pi}

: the instruction clause appended after the input for each attribute pole. Composing

k

attributes concatenates their clauses into a single sentence, "Respond {clause 1 }, {clause 2 }, ..., {clause k }." , in place of

k

separate sentences.

Attribute	Prompted $\boldsymbol{\pi}$	CAA $\boldsymbol{\gamma}$	CLAS $\boldsymbol{\nu}(x)$
German	96.6	94.7	97.1
Spanish	99.1	95.1	97.9
Italian	96.9	87.6	97.1
Russian	96.0	80.2	97.1
Japanese	94.8	72.2	92.2
Korean	93.0	74.0	91.0

Appendix

Table 11: Single-attribute steering (HM, %), averaged over all 9 models , per attribute.

Attribute	Prompted $\boldsymbol{\pi}$	CAA $\boldsymbol{\gamma}$	CLAS $\boldsymbol{\nu}(x)$
German	78.7	87.9	99.4
Spanish	95.3	89.9	99.7
Italian	78.5	87.1	99.7
Russian	79.2	85.4	99.4
Japanese	74.5	32.9	90.8
Korean	66.4	49.6	88.9

Appendix

Table 12: Single-attribute steering on Llama-3.1-8B-Instruct . Values are the harmonic mean (HM) of the success metric and Output Relevance (OR), averaged over the corresponding evaluation examples.

Attribute	Prompted $\boldsymbol{\pi}$	CAA $\boldsymbol{\gamma}$	CLAS $\boldsymbol{\nu}(x)$
German	96.1	89.1	96.1
Spanish	99.7	91.9	97.5
Italian	98.6	88.7	94.2
Russian	88.9	77.4	86.9
Japanese	87.3	82.5	77.8
Korean	78.2	35.8	49.5

Appendix

Table 13: Single-attribute steering on Llama-3.2-3B-Instruct . Values are the harmonic mean (HM) of the success metric and Output Relevance (OR), averaged over the corresponding evaluation examples.

Attribute	Prompted $\boldsymbol{\pi}$	CAA $\boldsymbol{\gamma}$	CLAS $\boldsymbol{\nu}(x)$
German	97.8	98.0	96.0
Spanish	98.9	99.7	99.2
Italian	98.6	91.9	97.5
Russian	98.9	85.2	95.8
Japanese	99.7	94.6	95.8
Korean	99.4	96.3	98.3

Appendix

Table 14: Single-attribute steering on Qwen-2.5-3B-Instruct . Values are the harmonic mean (HM) of the success metric and Output Relevance (OR), averaged over the corresponding evaluation examples.

Attribute	Prompted $\boldsymbol{\pi}$	CAA $\boldsymbol{\gamma}$	CLAS $\boldsymbol{\nu}(x)$
German	100.0	100.0	97.5
Spanish	100.0	100.0	100.0
Italian	100.0	97.4	96.7
Russian	99.7	75.9	97.2
Japanese	100.0	98.9	97.8
Korean	100.0	88.3	97.8

Appendix

Table 15: Single-attribute steering on Qwen-2.5-7B-Instruct . Values are the harmonic mean (HM) of the success metric and Output Relevance (OR), averaged over the corresponding evaluation examples.

Attribute	Prompted $\boldsymbol{\pi}$	CAA $\boldsymbol{\gamma}$	CLAS $\boldsymbol{\nu}(x)$
German	100.0	100.0	100.0
Spanish	100.0	99.7	98.0
Italian	100.0	98.9	100.0
Russian	100.0	82.7	100.0
Japanese	100.0	100.0	92.9
Korean	100.0	97.2	100.0

Appendix

Table 16: Single-attribute steering on Qwen-2.5-14B-Instruct . Values are the harmonic mean (HM) of the success metric and Output Relevance (OR), averaged over the corresponding evaluation examples.

Attribute	Prompted $\boldsymbol{\pi}$	CAA $\boldsymbol{\gamma}$	CLAS $\boldsymbol{\nu}(x)$
German	100.0	98.3	98.3
Spanish	100.0	96.0	90.4
Italian	100.0	96.8	96.0
Russian	100.0	90.3	96.9
Japanese	96.3	92.4	90.7
Korean	98.9	92.1	96.8

Appendix

Table 17: Single-attribute steering on Gemma-2-2B-it . Values are the harmonic mean (HM) of the success metric and Output Relevance (OR), averaged over the corresponding evaluation examples.

Attribute	Prompted $\boldsymbol{\pi}$	CAA $\boldsymbol{\gamma}$	CLAS $\boldsymbol{\nu}(x)$
German	99.7	100.0	96.3
Spanish	100.0	99.2	100.0
Italian	99.7	98.3	100.0
Russian	100.0	100.0	99.4
Japanese	100.0	98.6	93.9
Korean	100.0	98.9	99.7

Appendix

Table 18: Single-attribute steering on Gemma-2-9B-it . Values are the harmonic mean (HM) of the success metric and Output Relevance (OR), averaged over the corresponding evaluation examples.

Attribute	Prompted $\boldsymbol{\pi}$	CAA $\boldsymbol{\gamma}$	CLAS $\boldsymbol{\nu}(x)$
German	96.9	84.0	93.7
Spanish	98.3	79.4	96.3
Italian	96.4	48.3	96.7
Russian	97.5	41.4	98.0
Japanese	95.0	30.2	90.8
Korean	94.4	48.2	92.9

Appendix

Table 19: Single-attribute steering on Mistral-7B-Instruct-v0.3 . Values are the harmonic mean (HM) of the success metric and Output Relevance (OR), averaged over the corresponding evaluation examples.

Attribute	Prompted $\boldsymbol{\pi}$	CAA $\boldsymbol{\gamma}$	CLAS $\boldsymbol{\nu}(x)$
German	100.0	94.7	96.9
Spanish	100.0	100.0	100.0
Italian	100.0	80.8	92.9
Russian	100.0	83.9	99.7
Japanese	100.0	20.0	99.4
Korean	100.0	59.4	95.3

Appendix

Table 20: Single-attribute steering on Aya-Expanse-8B . Values are the harmonic mean (HM) of the success metric and Output Relevance (OR), averaged over the corresponding evaluation examples.

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	72.7	71.6	70.7	61.6	49.4	63.3	61.3	68.3	50.1	63.2
CAA $\boldsymbol{\gamma}$	44.0	55.9	41.7	41.5	21.7	50.9	26.0	58.4	68.4	45.4
CLAS $\boldsymbol{\nu}(x)$	76.5	77.0	77.4	77.6	54.5	68.1	70.8	75.9	81.7	73.3
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	86.5	85.6	85.4	83.4	78.2	79.1	80.9	85.2	84.1	83.2
Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	85.4	80.6	83.6	83.0	76.5	78.2	78.4	82.6	83.8	81.3
K-Steer $\boldsymbol{\kappa}(x)$	61.7	61.7	52.4	55.1	45.8	48.2	46.8	45.4	44.9	51.3

Appendix

Table 21: Behaviour

\times

language composition on Llama-3.1-8B-Instruct , averaged over all behavioural directions for each target language and method. Values are harmonic means (HM).

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	71.7	70.0	70.6	67.2	43.0	57.7	60.9	72.1	71.0	64.9
CAA $\boldsymbol{\gamma}$	46.4	45.9	51.6	35.4	28.1	54.5	23.8	62.9	77.6	47.3
CLAS $\boldsymbol{\nu}(x)$	78.0	58.3	59.5	59.2	27.7	53.9	63.9	71.9	81.7	61.6
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	83.6	83.6	81.7	77.4	74.1	72.3	72.0	77.3	72.4	77.1
Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	79.5	79.5	77.2	76.0	65.6	65.9	66.5	79.0	81.9	74.6
K-Steer $\boldsymbol{\kappa}(x)$	59.0	63.2	58.5	49.5	47.3	54.3	41.0	55.3	51.0	53.2

Appendix

Table 22: Behaviour

\times

language composition on Llama-3.2-3B-Instruct , averaged over all behavioural directions for each target language and method. Values are harmonic means (HM).

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	81.1	80.0	80.5	79.4	74.9	77.9	75.5	76.6	80.4	78.5
CAA $\boldsymbol{\gamma}$	71.6	75.1	57.1	23.6	56.7	71.8	32.5	63.8	73.0	58.4
CLAS $\boldsymbol{\nu}(x)$	81.0	79.1	67.9	82.7	49.6	82.5	80.3	56.1	85.8	73.9
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	88.1	87.7	86.7	88.4	86.5	87.0	85.2	79.0	88.7	86.4
Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	85.7	82.6	81.4	84.3	82.2	81.1	81.9	76.5	89.8	82.8
K-Steer $\boldsymbol{\kappa}(x)$	19.4	45.5	27.8	15.7	22.9	53.7	30.8	28.2	51.6	32.9

Appendix

Table 23: Behaviour

\times

language composition on Qwen-2.5-3B-Instruct , averaged over all behavioural directions for each target language and method. Values are harmonic means (HM).

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	80.4	78.0	77.6	78.4	74.7	75.7	75.8	77.5	78.8	77.4
CAA $\boldsymbol{\gamma}$	62.7	71.0	58.2	19.3	70.7	28.5	30.8	25.1	78.8	49.5
CLAS $\boldsymbol{\nu}(x)$	67.2	76.6	68.7	49.5	60.4	73.3	76.8	65.5	80.3	68.7
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	86.2	87.2	85.2	86.2	85.2	81.5	84.1	84.1	86.3	85.1
Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	82.1	83.1	81.2	78.4	79.0	78.3	81.2	77.8	83.8	80.6
K-Steer $\boldsymbol{\kappa}(x)$	42.8	33.5	19.1	14.7	41.0	42.7	16.1	20.8	45.6	30.7

Appendix

Table 24: Behaviour

\times

language composition on Qwen-2.5-7B-Instruct , averaged over all behavioural directions for each target language and method. Values are harmonic means (HM).

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	76.1	73.9	74.4	75.7	70.3	72.1	74.2	75.6	72.4	73.9
CAA $\boldsymbol{\gamma}$	76.2	81.4	71.1	30.0	78.6	59.2	18.4	77.7	78.5	63.5
CLAS $\boldsymbol{\nu}(x)$	76.6	66.2	73.9	69.3	57.4	67.7	54.7	72.3	59.9	66.4
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	89.9	88.5	87.8	87.1	83.1	83.3	83.9	86.6	89.2	86.6
Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	87.9	84.9	84.0	83.4	78.1	79.0	82.3	83.6	83.6	83.0
K-Steer $\boldsymbol{\kappa}(x)$	24.0	20.2	10.6	18.0	18.8	48.2	15.1	18.1	54.0	25.2

Appendix

Table 25: Behaviour

\times

language composition on Qwen-2.5-14B-Instruct , averaged over all behavioural directions for each target language and method. Values are harmonic means (HM).

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	71.7	72.9	72.2	70.0	62.0	72.6	58.3	63.3	77.3	68.9
CAA $\boldsymbol{\gamma}$	72.2	67.8	68.3	35.9	46.0	61.7	19.2	56.4	77.5	56.1
CLAS $\boldsymbol{\nu}(x)$	63.3	70.5	69.1	66.3	56.8	57.3	45.0	61.4	81.3	63.5
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	84.4	87.2	83.5	83.0	82.8	77.9	76.2	81.0	85.0	82.3
Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	79.1	77.3	78.7	76.3	78.5	73.5	63.6	75.3	86.8	76.6
K-Steer $\boldsymbol{\kappa}(x)$	35.0	42.1	33.0	35.3	28.8	53.9	10.8	22.0	56.9	35.3

Appendix

Table 26: Behaviour

\times

language composition on Gemma-2-2B-it , averaged over all behavioural directions for each target language and method. Values are harmonic means (HM).

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	79.0	78.3	78.3	77.7	67.2	79.0	74.2	76.6	79.5	76.6
CAA $\boldsymbol{\gamma}$	72.9	68.9	67.2	62.5	67.8	68.1	30.7	67.7	76.7	64.7
CLAS $\boldsymbol{\nu}(x)$	58.1	42.0	61.7	53.9	46.7	64.6	34.2	55.8	40.1	50.8
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	82.9	84.8	82.9	85.1	80.0	82.7	83.1	81.3	79.8	82.5
Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	74.4	80.3	75.1	77.3	72.0	77.4	76.3	75.2	79.4	76.4
K-Steer $\boldsymbol{\kappa}(x)$	66.0	70.2	54.2	64.3	64.8	63.7	57.7	50.0	44.1	59.4

Appendix

Table 27: Behaviour

\times

language composition on Gemma-2-9B-it , averaged over all behavioural directions for each target language and method. Values are harmonic means (HM).

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	80.1	81.9	78.2	79.6	63.0	78.0	70.3	80.5	78.5	76.7
CAA $\boldsymbol{\gamma}$	26.7	26.4	21.0	5.5	6.3	54.0	11.1	10.6	57.3	24.3
CLAS $\boldsymbol{\nu}(x)$	63.2	61.5	60.6	74.0	47.4	70.2	37.4	70.8	60.2	60.6
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	84.6	84.5	81.4	84.6	78.3	79.9	76.3	83.9	80.4	81.5
Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	78.3	77.9	75.7	79.8	68.4	73.5	63.9	78.3	73.0	74.3
K-Steer $\boldsymbol{\kappa}(x)$	31.4	48.5	29.6	18.8	24.8	51.7	12.2	15.5	55.2	32.0

Appendix

Table 28: Behaviour

\times

language composition on Mistral-7B-Instruct-v0.3 , averaged over all behavioural directions for each target language and method. Values are harmonic means (HM).

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	75.4	74.5	73.0	71.5	71.1	70.3	71.2	72.2	73.3	72.5
CAA $\boldsymbol{\gamma}$	41.6	51.3	28.3	42.8	6.9	45.6	50.5	29.3	47.7	38.2
CLAS $\boldsymbol{\nu}(x)$	63.4	68.3	63.1	64.6	62.8	68.6	56.8	68.5	71.0	65.2
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	80.0	77.7	78.4	77.5	76.1	75.0	76.5	79.9	76.4	77.5
Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	75.9	77.3	76.2	77.2	75.2	76.2	74.0	76.3	74.0	75.8
K-Steer $\boldsymbol{\kappa}(x)$	68.8	69.2	68.0	68.4	79.6	72.6	72.5	69.2	53.4	69.1

Appendix

Table 29: Behaviour

\times

language composition on Aya-Expanse-8B , averaged over all behavioural directions for each target language and method. Values are harmonic means (HM).

Behaviour	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	84.9	87.4	81.2	77.0	56.8	59.5	70.0	82.4	26.7	69.6
Chain-of-thought	CAA $\boldsymbol{\gamma}$	31.6	50.6	27.9	2.9	0.0	54.2	0.0	62.6	91.8	35.7
	CLAS $\boldsymbol{\nu}(x)$	87.5	86.1	86.9	93.5	74.9	60.4	72.7	76.2	94.0	81.4
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	88.7	89.2	87.5	86.8	63.8	64.9	61.1	82.4	94.5	79.9
	Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	85.9	86.9	88.9	91.2	85.9	81.1	79.7	84.6	95.1	86.6
	K-Steer $\boldsymbol{\kappa}(x)$	70.5	68.6	62.7	77.8	55.8	47.3	42.3	46.4	26.2	55.3

Appendix

Table 30: Behaviour

\times

language composition on Llama-3.1-8B-Instruct . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Behaviour	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	92.0	91.0	92.6	89.1	64.9	63.1	72.8	82.1	82.2	81.1
Chain-of-thought	CAA $\boldsymbol{\gamma}$	68.5	76.9	67.9	56.4	41.0	44.1	29.4	56.2	87.1	58.6
	CLAS $\boldsymbol{\nu}(x)$	78.2	76.2	70.6	70.7	66.5	40.4	46.8	70.7	91.3	67.9
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	78.0	72.8	72.7	66.8	47.5	42.8	39.7	70.7	56.7	60.9
	Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	74.8	78.1	82.3	73.6	72.8	47.9	45.4	77.1	90.7	71.4
	K-Steer $\boldsymbol{\kappa}(x)$	67.6	67.7	58.9	62.7	64.2	41.4	46.4	56.1	28.2	54.8

Appendix

Table 31: Behaviour

\times

language composition on Llama-3.2-3B-Instruct . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Behaviour	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	97.2	96.7	96.6	97.1	96.8	93.7	94.8	89.3	99.2	95.7
Chain-of-thought	CAA $\boldsymbol{\gamma}$	73.7	69.1	31.6	0.0	20.4	49.6	30.4	59.4	46.6	42.3
	CLAS $\boldsymbol{\nu}(x)$	85.0	93.4	82.9	80.1	86.7	90.5	80.1	63.8	94.7	84.1
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	85.0	94.1	89.7	94.8	90.7	90.5	86.1	80.7	94.7	89.6
	Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	89.0	93.8	85.9	92.4	90.9	91.6	88.3	75.1	96.6	89.3
	K-Steer $\boldsymbol{\kappa}(x)$	29.1	61.5	42.5	26.5	26.5	43.1	26.5	36.5	52.6	38.3

Appendix

Table 32: Behaviour

\times

language composition on Qwen-2.5-3B-Instruct . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Behaviour	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	98.5	97.9	97.9	95.1	100.0	95.0	97.4	95.0	99.8	97.4
Chain-of-thought	CAA $\boldsymbol{\gamma}$	41.3	93.4	3.1	45.1	79.2	0.0	74.5	26.8	95.6	51.0
	CLAS $\boldsymbol{\nu}(x)$	88.8	91.6	94.0	77.7	77.7	88.3	89.1	86.8	92.5	87.4
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	93.2	98.3	94.9	91.6	91.5	90.5	95.1	89.4	92.5	93.0
	Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	90.9	92.3	92.6	90.5	88.3	92.9	90.8	86.5	91.4	90.7
	K-Steer $\boldsymbol{\kappa}(x)$	28.2	26.7	26.7	28.8	67.2	44.0	26.7	26.7	26.6	33.5

Appendix

Table 33: Behaviour

\times

language composition on Qwen-2.5-7B-Instruct . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Behaviour	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	99.3	99.4	97.7	97.3	99.1	97.5	100.0	97.3	97.7	98.4
Chain-of-thought	CAA $\boldsymbol{\gamma}$	96.1	94.8	93.9	72.3	95.2	70.7	22.6	91.7	95.8	81.4
	CLAS $\boldsymbol{\nu}(x)$	90.0	82.1	84.2	85.4	87.2	93.3	86.0	89.0	38.1	81.7
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	98.3	97.7	93.1	95.2	90.9	93.3	86.0	90.7	96.1	93.5
	Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	95.8	92.3	91.6	93.0	91.7	92.3	97.3	95.0	94.4	93.7
	K-Steer $\boldsymbol{\kappa}(x)$	26.8	36.3	26.8	26.7	42.6	41.2	26.8	26.9	27.0	31.2

Appendix

Table 34: Behaviour

\times

language composition on Qwen-2.5-14B-Instruct . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Behaviour	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	94.5	92.6	93.0	92.7	82.3	80.9	64.4	87.1	97.8	87.3
Chain-of-thought	CAA $\boldsymbol{\gamma}$	70.2	68.3	50.1	21.3	14.7	52.6	21.7	25.1	63.1	43.0
	CLAS $\boldsymbol{\nu}(x)$	86.7	82.0	74.4	82.9	74.4	74.0	54.9	82.7	23.8	70.6
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	88.0	86.0	84.1	84.8	74.4	74.6	55.8	82.7	75.6	78.4
	Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	84.4	83.5	82.0	85.5	76.8	71.4	56.5	79.5	49.6	74.4
	K-Steer $\boldsymbol{\kappa}(x)$	37.2	53.3	25.7	26.0	29.4	41.9	26.0	25.6	25.2	32.3

Appendix

Table 35: Behaviour

\times

language composition on Mistral-7B-Instruct-v0.3 . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Behaviour	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	98.3	98.3	98.7	98.7	99.4	99.4	97.3	97.1	97.3	98.3
Chain-of-thought	CAA $\boldsymbol{\gamma}$	93.1	96.0	83.4	88.0	27.1	70.6	94.3	67.1	81.4	77.9
	CLAS $\boldsymbol{\nu}(x)$	97.0	96.6	91.8	96.4	95.2	96.4	90.8	96.2	72.1	92.5
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	97.0	96.6	96.4	96.4	97.4	97.4	93.8	96.2	72.1	93.7
	Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	93.5	97.3	91.7	95.0	96.1	97.0	92.1	95.9	91.6	94.4
	K-Steer $\boldsymbol{\kappa}(x)$	73.8	92.4	75.8	94.9	69.0	81.4	90.5	93.8	26.9	77.6

Appendix

Table 36: Behaviour

\times

language composition on Aya-Expanse-8B . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Behaviour	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	95.6	94.6	98.7	94.1	94.9	90.6	83.7	91.5	98.9	93.6
Chain-of-thought	CAA $\boldsymbol{\gamma}$	87.9	84.9	85.2	83.3	83.8	70.1	53.3	72.2	97.7	79.8
	CLAS $\boldsymbol{\nu}(x)$	92.2	72.6	68.7	88.2	49.6	58.1	50.9	87.7	86.8	72.8
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	92.2	86.7	89.5	94.6	83.3	77.7	56.6	87.7	86.8	83.9
	Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	92.8	84.2	82.5	94.2	79.4	79.7	60.5	84.4	92.1	83.3
	K-Steer $\boldsymbol{\kappa}(x)$	43.2	66.3	62.9	30.8	30.1	45.8	26.7	43.8	28.8	42.1

Appendix

Table 37: Behaviour

\times

language composition on Gemma-2-2B-it . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Behaviour	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	99.8	99.1	99.8	99.6	99.6	95.9	98.1	97.9	99.4	98.8
Chain-of-thought	CAA $\boldsymbol{\gamma}$	66.7	89.7	72.5	12.8	51.3	43.0	43.9	55.7	97.0	59.2
	CLAS $\boldsymbol{\nu}(x)$	79.8	30.8	85.6	93.0	86.3	91.5	56.6	94.9	18.4	70.8
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	81.6	96.3	93.0	93.0	95.4	91.5	86.3	94.9	86.5	90.9
	Sequential $\boldsymbol{\nu}^{\mathrm{seq}}(x)$	96.5	92.8	94.2	95.2	92.1	90.9	85.7	94.2	97.1	93.2
	K-Steer $\boldsymbol{\kappa}(x)$	89.8	89.0	82.8	80.6	87.8	70.3	70.3	57.7	27.0	72.8

Appendix

Table 38: Behaviour

\times

language composition on Gemma-2-9B-it . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
Prompted $\boldsymbol{\pi}$	69.9	72.8	71.6	66.4	58.7	72.1	66.8	68.4	76.7	69.3
CAA $\boldsymbol{\gamma}$	43.4	48.6	44.5	20.9	30.9	55.3	12.3	42.4	68.0	40.7
CLAS $\boldsymbol{\nu}(x)$	63.5	61.8	64.2	61.4	47.0	65.5	52.0	64.6	72.8	61.4
Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	85.1	88.5	87.5	85.4	83.0	83.3	84.0	85.2	90.0	85.8
K-Steer $\boldsymbol{\kappa}(x)$	35.7	34.3	29.1	24.8	29.3	72.9	20.9	25.1	74.8	38.5
ORBIT $\boldsymbol{\omega}(x)$	43.1	43.0	40.9	25.3	23.3	62.4	15.0	37.9	75.3	40.7

Appendix

Table 39: Language, style, and safety composition , averaged over 9 models , per target language and method (108 pairs per method).

Triple	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	30.9	11.0	30.3	6.1	45.2	38.2	7.5	15.0	10.8	21.7
Chain-of-thought + Jailbreak	CAA $\boldsymbol{\gamma}$	0.0	4.8	0.0	4.8	12.6	81.9	0.0	31.1	10.7	16.2
	CLAS $\boldsymbol{\nu}(x)$	64.3	88.6	84.7	86.0	47.9	42.9	63.4	82.8	96.2	73.0
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	80.2	75.0	74.1	81.8	78.0	80.1	75.1	67.0	90.3	78.0
	K-Steer $\boldsymbol{\kappa}(x)$	74.1	7.7	0.0	71.5	60.5	53.0	0.0	41.6	62.1	41.2
	ORBIT $\boldsymbol{\omega}(x)$	12.2	13.0	8.9	0.0	0.0	44.4	3.0	0.0	41.6	13.7

Appendix

Table 40: Language

\times

safety

\times

style composition on Llama-3.1-8B-Instruct . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Triple	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	21.8	3.0	10.3	4.6	5.6	0.0	7.4	41.9	8.4	11.4
Chain-of-thought + Jailbreak	CAA $\boldsymbol{\gamma}$	0.0	0.0	11.9	0.0	0.0	84.2	0.0	31.4	80.6	23.1
	CLAS $\boldsymbol{\nu}(x)$	72.4	60.0	70.5	17.2	0.0	71.4	57.7	63.1	87.0	55.5
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	89.9	84.1	77.1	90.5	75.5	75.0	73.8	84.6	92.9	82.6
	K-Steer $\boldsymbol{\kappa}(x)$	50.5	0.0	2.6	0.0	51.0	40.2	0.0	31.0	12.0	20.8
	ORBIT $\boldsymbol{\omega}(x)$	2.9	2.8	7.8	0.0	3.1	23.7	0.0	1.6	25.8	7.5

Appendix

Table 41: Language

\times

safety

\times

style composition on Llama-3.2-3B-Instruct . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Triple	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	48.3	17.1	24.3	24.0	19.6	23.2	58.7	34.7	35.4	31.7
Chain-of-thought + Jailbreak	CAA $\boldsymbol{\gamma}$	0.0	0.0	10.0	0.0	0.0	35.9	0.0	0.0	58.9	11.6
	CLAS $\boldsymbol{\nu}(x)$	79.8	85.9	69.3	82.8	32.5	90.4	71.4	42.9	90.9	71.8
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	86.0	91.4	85.3	91.3	88.9	90.4	86.3	82.4	93.3	88.4
	K-Steer $\boldsymbol{\kappa}(x)$	0.0	51.4	9.3	0.0	0.0	88.8	0.0	0.0	83.3	25.9
	ORBIT $\boldsymbol{\omega}(x)$	0.0	0.0	0.0	0.0	2.8	1.5	0.0	0.0	1.6	0.7

Appendix

Table 42: Language

\times

safety

\times

style composition on Qwen-2.5-3B-Instruct . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Triple	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	27.7	31.5	20.9	19.7	28.1	21.0	16.1	32.6	37.3	26.1
Chain-of-thought + Jailbreak	CAA $\boldsymbol{\gamma}$	6.3	64.1	12.1	0.0	3.2	9.9	0.0	0.0	67.4	18.1
	CLAS $\boldsymbol{\nu}(x)$	17.4	86.0	74.9	46.1	73.6	88.6	75.8	77.2	92.6	70.2
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	91.7	92.5	89.3	95.5	90.1	88.6	89.3	87.2	94.0	90.9
	K-Steer $\boldsymbol{\kappa}(x)$	0.0	0.0	0.0	0.0	0.0	7.8	0.0	45.7	77.7	14.6
	ORBIT $\boldsymbol{\omega}(x)$	0.0	3.2	0.0	0.0	11.3	22.0	0.0	7.5	80.3	13.8

Appendix

Table 43: Language

\times

safety

\times

style composition on Qwen-2.5-7B-Instruct . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Triple	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	18.6	10.4	9.2	12.1	15.9	13.3	7.8	6.3	14.8	12.0
Chain-of-thought + Jailbreak	CAA $\boldsymbol{\gamma}$	80.4	93.6	86.9	0.0	93.5	61.5	0.0	89.0	94.3	66.6
	CLAS $\boldsymbol{\nu}(x)$	79.7	64.8	73.8	77.9	79.8	86.0	37.8	80.8	47.6	69.8
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	89.5	93.4	90.5	93.1	87.9	86.0	90.0	83.4	93.1	89.7
	K-Steer $\boldsymbol{\kappa}(x)$	0.0	89.6	0.0	0.0	0.0	65.0	0.0	0.0	91.7	27.4
	ORBIT $\boldsymbol{\omega}(x)$	29.0	12.4	10.3	0.0	6.5	23.5	0.0	2.9	9.2	10.4

Appendix

Table 44: Language

\times

safety

\times

style composition on Qwen-2.5-14B-Instruct . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Triple	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	7.5	8.6	5.5	2.2	5.7	13.6	2.9	2.2	3.3	5.7
Chain-of-thought + Jailbreak	CAA $\boldsymbol{\gamma}$	0.0	0.0	7.9	10.7	6.0	0.0	12.2	1.6	51.0	9.9
	CLAS $\boldsymbol{\nu}(x)$	13.8	10.7	3.3	10.7	12.7	3.7	0.0	45.7	26.8	14.2
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	88.2	93.0	90.4	91.2	81.2	72.1	81.0	88.0	90.4	86.2
	K-Steer $\boldsymbol{\kappa}(x)$	0.0	75.7	0.0	0.0	48.9	60.1	0.0	0.0	6.4	21.2
	ORBIT $\boldsymbol{\omega}(x)$	27.7	20.4	0.0	13.9	3.2	82.6	0.0	12.3	85.0	27.2

Appendix

Table 45: Language

\times

safety

\times

style composition on Gemma-2-2B-it . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Triple	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	9.3	19.9	9.3	9.3	14.7	14.9	13.2	9.3	3.3	11.5
Chain-of-thought + Jailbreak	CAA $\boldsymbol{\gamma}$	62.2	86.4	86.2	70.0	42.4	33.8	30.5	36.0	90.5	59.8
	CLAS $\boldsymbol{\nu}(x)$	0.0	0.0	14.3	0.0	0.0	15.1	0.0	0.0	0.0	3.3
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	94.1	92.4	94.7	90.2	88.0	84.3	93.3	90.6	93.1	91.2
	K-Steer $\boldsymbol{\kappa}(x)$	84.4	85.9	0.0	0.0	0.0	74.3	0.0	80.1	0.0	36.1
	ORBIT $\boldsymbol{\omega}(x)$	41.6	4.8	7.9	0.0	0.0	68.2	0.0	0.0	83.1	22.8

Appendix

Table 46: Language

\times

safety

\times

style composition on Gemma-2-9B-it . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Triple	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	74.4	73.2	78.3	74.1	70.1	70.2	56.9	66.7	79.5	71.5
Chain-of-thought + Jailbreak	CAA $\boldsymbol{\gamma}$	6.9	2.9	2.2	0.0	0.0	28.5	0.0	0.0	7.8	5.4
	CLAS $\boldsymbol{\nu}(x)$	79.7	83.7	58.0	79.6	55.9	55.8	39.5	77.8	93.7	69.3
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	88.3	91.1	90.1	88.9	80.8	75.6	62.0	82.9	94.2	83.8
	K-Steer $\boldsymbol{\kappa}(x)$	71.0	79.6	0.0	0.0	48.2	84.5	0.0	0.0	81.4	40.5
	ORBIT $\boldsymbol{\omega}(x)$	0.0	9.2	11.7	0.0	0.0	81.9	0.0	15.3	80.6	22.1

Appendix

Table 47: Language

\times

safety

\times

style composition on Mistral-7B-Instruct-v0.3 . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Triple	Method	de	es	it	ru	ja	ko	ar	pl	en	Avg
	Prompted $\boldsymbol{\pi}$	52.6	39.3	48.3	44.4	50.4	55.2	45.8	47.4	30.5	46.0
Chain-of-thought + Jailbreak	CAA $\boldsymbol{\gamma}$	18.6	18.9	15.6	25.4	0.0	23.7	27.0	14.8	24.9	18.8
	CLAS $\boldsymbol{\nu}(x)$	71.2	82.6	78.7	86.1	78.3	87.3	52.8	74.6	90.5	78.0
	Joint-CLAS $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	94.9	94.3	95.1	92.6	92.9	87.3	87.2	89.5	91.7	91.7
	K-Steer $\boldsymbol{\kappa}(x)$	0.0	0.0	65.6	78.1	77.5	81.9	82.8	0.0	79.8	51.7
	ORBIT $\boldsymbol{\omega}(x)$	35.8	36.9	45.8	0.0	43.5	72.2	25.8	5.3	73.9	37.7

Appendix

Table 48: Language

\times

safety

\times

style composition on Aya-Expanse-8B . Values are harmonic means (HM) of the success metric and Output Relevance (OR), reported for each target language and behavioural direction.

Method	zh	pt	hi	fr	cz	el	no	Avg
Prompted	0.82	0.80	0.76	0.81	0.79	0.77	0.79	0.79
CAA (vector reuse)	0.43	0.50	0.40	0.50	0.47	0.41	0.46	0.45
CAA (vector update)	0.45	0.52	0.41	0.51	0.49	0.41	0.47	0.47
Joint-CLAS (vector reuse)	0.64	0.68	0.66	0.71	0.71	0.66	0.67	0.68
Joint-CLAS (vector update)	0.65	0.69	0.67	0.71	0.72	0.68	0.70	0.69

Appendix

Table 49: Input-language generalization under vector reuse and vector update. Mean HM for prompts in unseen input languages not used during gate training. Vector reuse uses the trained-language directions and gates without modification. Vector update recomputes the steering direction only using the unseen languages in the negative pool while keeping the trained gate parameters unchanged. Prompted results are shown for reference. Each cell is averaged over 9 trained languages

\times

6 style behaviours (54 pairs) and then across all 9 models.

Method	MMLU	ARC-E	ARC-C	Win.	Average
Baseline	55.3	86.7	71.9	54.1	67.0
CAA (sing.) $\boldsymbol{\gamma}$	49.8	81.3	63.5	52.0	61.7
CLAS (sing.) $\boldsymbol{\nu}(x)$	52.2	84.2	67.6	52.6	64.2
CAA (double) $\boldsymbol{\gamma}$	40.7	64.6	48.0	51.3	51.2
CLAS (double) $\boldsymbol{\nu}(x)$	50.1	82.9	66.1	52.9	63.0
Joint-CLAS (double) $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	49.6	81.9	65.8	51.3	62.2

Appendix

Table 50: Knowledge retention: accuracy (%) on each benchmark’s validation split under every steering condition (best checkpoint/alpha everywhere) for Llama-3.2-3B. CAA/CLAS (single) average over 22 single attributes; double-steering results are averaged over 4 language-behaviour pairs; triple-steering results are averaged over 4 language-style-safety triples. Joint-CLAS uses the jointly-trained (not independently composed) gate. Average is the unweighted mean over the 4 benchmark columns.

Method	MMLU	ARC-E	ARC-C	Win.	Average
Baseline	63.2	92.6	81.9	60.5	74.6
CAA (sing.) $\boldsymbol{\gamma}$	50.9	78.9	66.7	55.1	62.9
CLAS (sing.) $\boldsymbol{\nu}(x)$	60.1	90.6	79.7	57.9	72.1
CAA (double) $\boldsymbol{\gamma}$	25.1	27.3	25.6	50.1	32.0
CLAS (double) $\boldsymbol{\nu}(x)$	49.9	78.2	65.1	51.8	61.2
Joint-CLAS (double) $\boldsymbol{\nu}^{\mathrm{joint}}(x)$	53.3	84.2	71.1	53.9	65.6

Appendix

Table 51: Knowledge retention: accuracy (%) on each benchmark’s validation split under every steering condition (best checkpoint/alpha everywhere) for Llama-3.1-8B. CAA/CLAS (single) average over 22 single attributes; double-steering results are averaged over 4 language-behaviour pairs; triple-steering results are averaged over 4 language-style-safety triples. Joint-CLAS uses the jointly-trained (not independently composed) gate. Average is the unweighted mean over the 4 benchmark columns.

Figure 15: Per-token, per-layer CLAS gate activity on an example prompt for the Arabic behaviour for Llama-3.1-8B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 16: Per-token, per-layer CLAS gate activity on an example prompt for the German behaviour for Llama-3.1-8B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 17: Per-token, per-layer CLAS gate activity on an example prompt for the Humour behaviour for Llama-3.1-8B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 18: Per-token, per-layer CLAS gate activity on an example prompt for the Serious behaviour for Llama-3.1-8B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 19: Per-token, per-layer CLAS gate activity on an example prompt for the Over-refuse behaviour for Llama-3.1-8B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 20: Per-token, per-layer CLAS gate activity on an example prompt for the Jailbreak behaviour for Llama-3.1-8B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 21: Per-token, per-layer CLAS gate activity on an example prompt for the Non-toxic behaviour for Llama-3.1-8B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 22: Per-token, per-layer CLAS gate activity on an example prompt for the Toxic behaviour for Llama-3.1-8B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 23: Per-token, per-layer CLAS gate activity on an example prompt for the Arabic behaviour for Qwen-2.5-7B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 24: Per-token, per-layer CLAS gate activity on an example prompt for the German behaviour for Qwen-2.5-7B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 25: Per-token, per-layer CLAS gate activity on an example prompt for the Humour behaviour for Qwen-2.5-7B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 26: Per-token, per-layer CLAS gate activity on an example prompt for the Serious behaviour for Qwen-2.5-7B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 27: Per-token, per-layer CLAS gate activity on an example prompt for the Over-refuse behaviour for Qwen-2.5-7B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 28: Per-token, per-layer CLAS gate activity on an example prompt for the Jailbreak behaviour for Qwen-2.5-7B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 29: Per-token, per-layer CLAS gate activity on an example prompt for the Non-toxic behaviour for Qwen-2.5-7B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Figure 30: Per-token, per-layer CLAS gate activity on an example prompt for the Toxic behaviour for Qwen-2.5-7B-Instruct . Top: gate coefficient $a_{l}$ at every layer (y-axis) and token position (x-axis), prompt tokens followed by generated tokens (vertical line marks the prompt/generation boundary). Bottom: the model’s own prompt and generated tokens aligned to each column, so it is directly readable which tokens trigger the gate and at which depth.

Quantity	Single-attribute	Joint (pair)
Trainable parameters (avg. over models)	123,312	246,624
Attributes gated per training run	1	2
Frozen-LM forward/backward passes (steps $\times$ $n_{\text{train}}$ )	1,920	1,920
Measured wall-clock, A100 (avg. over 9 models)	248.5s	273.5s

Appendix

Table 52: Gate-training efficiency. Each CLAS gate has a trainable weight vector

w_{l}

and scalar bias

b_{l}

at each layer. Joint training uses two gates and therefore twice the trainable parameters, while requiring the same number of frozen-LM forward and backward passes. Measured A100 GPU wall-clock time increases by only about 10% for joint training. Both take negligible time.

Artifact	License	Note
Steered models
Llama-3.1-8B-Instruct	Llama 3.1 Community License	Gated
Llama-3.2-3B-Instruct	Llama 3.2 Community License	Gated
Qwen-2.5-3B-Instruct	Qwen Research License	Research-only
Qwen-2.5-7B-Instruct	Apache 2.0
Qwen-2.5-14B-Instruct	Apache 2.0

Appendix

Table 53: Artifacts used in this work and their licenses.

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 278 papers

Sep 8, 2026

Compositional Multilingual and Behavioral Attribute Steering

This study examines the compositionality of steering vectors for language and behavioral control in large language models. Focusing on language, jailbreak, and conciseness, we investigate whether additive, training-free composition of attribute steering vectors can preserve the intended steering effect of each attribute, across four instruction-tuned models from two model families and two size scales. We find that single-attribute steering is reliable for all three attributes, but only within an appropriate combination of intervention layer and steering strength, with abstract behaviors (jailbreak, conciseness) favoring middle layers and language favoring earlier layers. We show that additive composition of two attribute vectors succeeds in steering both attributes simultaneously when each is injected at its own best-performing layer, and that this partially extends to three simultaneously composed attributes, addressing an inconsistency left open by prior work on training-free composition. We further analyze the geometric properties of these steering vectors, finding that they are approximately orthogonal in the residual stream, consistent with their compositional behavior.

Hyun Gu Kang, Daniil Gurgurov, Tanja Baeumel +2

1German Research Centre for Artificial Intelligence (DFKI) · 2Saarland University · Centre for European Research in Trusted AI (CERTAIN)

Language Model Steering

Ask PDF

May 11, 2026

Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions

Activation steering controls language model behavior by adding directions to internal representations at inference time, but standard residual-stream steering can fail in stateful dialogue. We identify KV-cache contamination as a key failure mode: steered token states are stored and repeatedly reused, turning a local perturbation into cumulative coherence degradation. To address this challenge, we propose Gated Cropped Attention-Delta steering (GCAD), which extracts steering signals from system-prompt contributions to self-attention and applies them with token-level gating. Across persona-steering experiments, GCAD preserves trait control while substantially improving long-horizon coherence. On the main multi-turn benchmark, GCAD improves average coherence drift from -18.6 to -1.9 and raises turn-10 trait expression from 78.0 to 93.1. These results suggest that activation steering becomes more reliable when interventions follow the prompt-mediated pathways that models already use for behavioral control.

Diancheng Kang, Zheyuan Liu, Ningshan Ma +3

1Southern University of Science and Technology · University of Notre Dame · 3Massachusetts Institute of Technology

Language Model Steering Self-Attention Causal Interventions in Language Models Activation Steering

Ask PDF

May 5, 2026

Steer Like the LLM: Activation Steering that Mimics Prompting

Large language models can be steered at inference time through prompting or activation interventions, but activation steering methods often underperform compared to prompt-based approaches. We propose a framework that formulates prompt steering as a form of activation steering and investigates whether distilling successful prompt steering behavior into simpler, interpretable models can close this gap. Our analysis reveals that popular activation steering methods are not faithful to the mechanics of prompt steering, which applies strong interventions on some tokens while barely affecting others. Based on these insights, we introduce Prompt Steering Replacement (PSR) models that estimate token-specific steering coefficients from the activations themselves and are trained to imitate prompt-based interventions. Experiments on three steering benchmarks across multiple language models show that PSR models outperform existing activation steering methods, especially when controlling for high-coherence completions, and also compare favorably to prompting on AxBench and persona steering.

Geert Heyman, Frederik Vandeputte

Nokia Bell Labs, Belgium.

Language Model Steering LLM Prompting Causal Interventions in Language Models Activation Steering

Ask PDF

Abstract

Language Model Steering

Explore similar work

Compositional Multilingual and Behavioral Attribute Steering

Prompt-Activation Duality: Improving Activation Steering via Attention-Level Interventions

Steer Like the LLM: Activation Steering that Mimics Prompting