Steering with Multilingual Lexicons for Sentiment Control in Low-Resource Languages

Lucinda Laurent; Muhammad Dehan Al Kautsar; Alfan Farizki Wicaksono; Fajri Koto

cs.CLNov 2, 2026

Steering with Multilingual Lexicons for Sentiment Control in Low-Resource Languages

Authors: Lucinda Laurent, Muhammad Dehan Al Kautsar, Alfan Farizki Wicaksono, Fajri Koto

Organizations: Faculty of Computer Science, Universitas Indonesia · Mohamed bin Zayed University of Artificial Intelligence

Abstract

Activation steering offers a lightweight mechanism for controlling model behavior, but existing methods depend on sentence-level contrastive data that is unavailable for most of the world's languages. We show that multilingual sentiment lexicons are a viable substitute across three model families: aggregated multilingual lexicons outperform English-derived steering vectors for mid- and low-resource languages, while monolingual lexicons remain stronger for languages a model has specialized in during pre-training. The steering efficacy is mechanistically dictated by both representational density and pre-training priors. Large-magnitude English vectors overwhelm the compact representations of low-resource languages, causing "linguistic collapse." This fragility is mitigated by the norm-dampening of multilingual aggregation. Architectural biases, including bilingual specialization penalty and latent numerical rigidity, also heavily constrain cross-lingual steering responsiveness. Ultimately, our work establishes lexicons as a practical steering signal and offers a comprehensive mechanistic account of cross-lingual steering dynamics.

Figures & tables

Figure 1: Workflow of the Cross-Lingual Sentiment Control using Multilingual Lexicon. In the Extraction Phase, feature attribution selects the steering layers

i\in\mathcal{S}

(linear-probe macro-F1) and, within each, the neurons

\mathcal{N}_{i}

( XGBoost gain). At each selected layer, the steering vector is the difference

\mu^{+,(i)}-\mu^{-,(i)}

between multilingual positive and negative lexicon centroids, sliced to

\mathcal{N}_{i}

. In the Injection Phase, this vector is applied to neutral prompts with coefficient

+\alpha

-\alpha

, shifting the generated text toward positive or negative sentiment across multiple languages.

Figure 2: Impact of Steering on Sentiment Distribution (Gemma-3). The distribution of generated sentiments across 11 languages under three conditions: (a) No Intervention (as baseline), (b) Positive Steering, and (c) Negative Steering. The proposed method successfully shifts the predominantly neutral baseline toward the target sentiment. Corresponding distributions for Qwen-3 and Llama-3.2 are provided in Appendix D .

		Gemma-3		Qwen-3		Llama-3.2
Lang	Res.	SR (%)	Layers	SR (%)	Layers	SR (%)	Layers
English (en)	High	94.3	all	45.1	all	65.8	all
German (de)	High	90.5	all	46.1	top-50%	53.4	all
Chinese (zh)	High	86.2	all	48.0	all	33.1	all
Japanese (ja)	High	76.8	all	23.5	all	18.9	all
Indonesian (id)	Mid	82.0	all	46.8	all	37.0	all

Table 1: Cross-Lingual Steering Performance with Multilingual Lexicons. Success Rate (SR, %) across three models and

11

languages. Each cell reports SR at the best-performing layer configuration (all, top-

50

%, or top-

5

		Gemma-3		Qwen-3		Llama-3.2
Lang	Res.	SR (%)	Layers	SR (%)	Layers	SR (%)	Layers
English (en)	High	95.0	all	44.5	all	66.4	all
German (de)	High	90.7	all	45.4	top-50%	53.8	all
Chinese (zh)	High	86.8	all	53.4	all	34.3	all
Japanese (ja)	High	80.2	all	23.9	top-50%	20.8	all
Indonesian (id)	Mid	82.3	all	42.9	top-50%	38.6	all

Table 2: Cross-Lingual Steering Performance with Monolingual English Lexicons. Success Rate (SR, %) across three models and

11

languages. Each cell reports SR at the best-performing layer configuration (all, top-

50

%, or top-

5

Figure 3: Impact of Steering Intensity on Success Rate and Language Integrity. Heatmaps show performance under the all-layers configuration as the steering coefficient

\alpha

increases from 0.5 to 1.0, for steering vectors derived from monolingual (English) and multilingual lexicons. Values represent Success Rate (%); black cells indicate linguistic collapse, where the generated text was incoherent and thus could not be evaluated for sentiment.

Figure 4: Heterogeneous Impact of Multilingual Aggregation Across Model Families. This figure highlights the divergent impact of the Multilingual steering direction across three distinct model families. Positive values (teal) indicate that the Multilingual direction outperforms the Monolingual direction.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Lang.	ISO	Llama-3.2	Qwen3-4B	Gemma3-4B
English	en	$\mathbf{8.47}$	$8.53$	$8.54$
German	de	$12.39$	$12.42$	$\mathbf{10.80}$
Chinese	zh	$9.75$	$\mathbf{7.62}$	$8.03$
Japanese	ja	$13.41$	$12.43$	$\mathbf{9.84}$
Indonesian	id	$12.51$	$12.61$	$\mathbf{9.41}$
Icelandic	is	$17.42$	$17.62$	$\mathbf{15.14}$

Appendix

Table 3: Average tokens per neutral prompt across target languages. Lower values indicate higher tokenization efficiency and a more optimized vocabulary for that language.

English (High-Resource)
	Human Annotation (Gold)
Predicted	Negative	Positive	Neutral	Precision
Negative	47	0	3	94%
Positive	0	50	0	100%
Neutral	1	1	48	96%
Macro-Average				96.7%

Appendix

Table 4: Confusion matrices for the GPT-4.1 sentiment judge across 150 randomized samples per language.

Figure 5: Impact of Steering on Sentiment Distribution (Qwen-3). Steering induces a noticeable shift toward the target sentiment, confirming the method’s cross-model applicability. However, Qwen-3 exhibits a stronger tendency to preserve neutral tonality compared to other models, with neutral outputs remaining the dominant class in mid to low resource languages even after intervention.

Figure 6: Impact of Steering on Sentiment Distribution (Llama-3.2). While the steering vectors successfully increase the probability of the target sentiment across all languages, Llama-3.2 retains a higher proportion of neutral outputs compared to Gemma-3, particularly in low-resource languages (e.g., Hausa, Javanese, Sundanese).

Figure 7: Impact of Steering with Monolingual Steering Direction on Sentiment Distribution (Gemma-3). The distribution of generated sentiments across 11 languages under three conditions: (a) No Intervention (as baseline), (b) Positive Steering, and (c) Negative Steering.

		Gemma-3				Qwen-3				Llama-3
Lang	Res.	SR	SA	LIDR	Config	SR	SA	LIDR	Config	SR	SA	LIDR	Config
English (en)	High	$95.0$	$95.0$	$99.8$	all	$44.5$	$54.6$	$84.1$	all	$66.4$	$67.0$	$98.6$	all
German (de)	High	$90.7$	$91.8$	$98.4$	all	$45.4$	$52.8$	$72.1$	top-50%	$53.8$	$64.2$	$83.0$	all
Chinese (zh)	High	$86.8$	$91.0$	$93.5$	all	$53.4$	$62.0$	$82.5$	all	$34.3$	$44.5$	$67.5$	all
Japanese (ja)	High	$80.2$	$89.3$	$88.6$	all	$23.9$	$31.3$	$57.5$	top-50%	$20.8$	$38.1$	$58.2$	all
Indonesian (id)	Mid	$82.3$	$91.0$	$89.8$	all	$42.9$	$47.9$	$85.1$	top-50%	$38.6$	$51.3$	$69.1$	all

Appendix

Table 5: Best Results (Mono). SR: Success Rate, SA: Sentiment Acc, LIDR: Language ID Rate. All metrics in %.

		Gemma-3				Qwen-3				Llama-3
Lang	Res.	SR	SA	LIDR	Config	SR	SA	LIDR	Config	SR	SA	LIDR	Config
English (en)	High	$94.3$	$94.4$	$99.7$	all	$45.1$	$48.5$	$91.6$	all	$65.8$	$66.2$	$98.9$	all
German (de)	High	$90.5$	$91.4$	$98.5$	all	$46.1$	$51.1$	$78.8$	top-50%	$53.4$	$63.4$	$82.0$	all
Chinese (zh)	High	$86.2$	$89.0$	$95.0$	all	$48.0$	$54.2$	$85.8$	all	$33.1$	$42.8$	$71.5$	all
Japanese (ja)	High	$76.8$	$85.9$	$87.8$	all	$23.5$	$43.2$	$43.3$	all	$18.9$	$33.2$	$60.1$	all
Indonesian (id)	Mid	$82.0$	$91.0$	$88.9$	all	$46.8$	$53.2$	$83.2$	all	$37.0$	$49.9$	$69.8$	all

Appendix

Table 6: Best Results (Multi). SR: Success Rate, SA: Sentiment Acc, LIDR: Language ID Rate. All metrics in %.

Model	Multi SR	Mono SR	$\Delta$ SR	$Z$ -score	$p$ -value
High-resource ( $4$ langs, $n=8{,}000$ )
Gemma-3	$86.95$	$88.17$	$-1.22$	$-2.34$	$\mathbf{0.0194}$
Qwen-3	$40.67$	$41.80$	$-1.13$	$-1.45$	$0.1466$
Llama-3.2	$42.80$	$43.83$	$-1.03$	$-1.31$	$0.1886$
Mid-resource ( $4$ langs, $n=8{,}000$ )
Gemma-3	$71.83$	$69.33$	$+2.50$	$+3.47$	$\mathbf{0.0005}$

Appendix

Table 7: Two-sample proportion

Z

-tests comparing multilingual vs. monolingual (English) steering success rates (SR, %) by language resource tier. Bold

p

-values indicate rejection of the null hypothesis at

\alpha=0.05

. Positive

\Delta

SR favors the multilingual direction; negative favors the monolingual baseline.

Figure 8: Comparison of

L_{2}

norm magnitudes for sentiment steering directions (all neurons) across Gemma-3 , Llama-3.2 , and Qwen-3 . Monolingual directions for German, Chinese, Indonesian, and Javanese are extracted and plotted solely to analyze the latent geometry and are not used for injection.

Figure 9:

L_{2}

norm magnitude of the nonzero-gain steering vectors across layers for Gemma-3 , Llama-3.2 , and Qwen-3 , comparing English and multilingual steering directions. Only neurons with non-zero XGBoost gain are included in the norm computation, reflecting the actual magnitude of intervention applied at inference time. Opaque markers denote layers in the top 50% by F1 score of the corresponding linear probe; faded markers denote layers outside this top half.

Panel of Figure 10 — Figure 10: Cosine similarity heatmaps between monolingual and multilingual steering directions across three architectures. The displayed values represent the average cosine similarity across all layers.

Model & Language	Metric	$H_{1}$ (Ours)	$H_{0}^{(1)}$	McNemar $p$ -value	Significance ( $\alpha=0.05$ )
Gemma-Hausa	Sentiment Acc.	66%	3%	$3.09\times 10^{-17}$	Significant
	Strict SR	56%	0%	$2.78\times 10^{-17}$	Significant
Qwen-Javanese	Sentiment Acc.	27%	7%	$0.0003$	Significant
	Strict SR	9%	4%	$0.2266$	Not Significant

Appendix

Table 8: Proposed Framework (

H_{1}

) vs. Fully Random Baseline (

H_{0}^{(1)}

). This baseline evaluates the effect of applying the steering vector to a random subset of layers (

50

% of all layers) and randomly sampled neuron indices, preserving the exact intervention size (

|{}N_{i}|{}

) of the original targeted configuration.

Model & Language	Metric	$H_{1}$ (Ours)	$H_{0}^{(2)}$	McNemar $p$ -value	Significance ( $\alpha=0.05$ )
Gemma-Hausa	Sentiment Acc.	66%	17%	$5.91\times 10^{-12}$	Significant
	Strict SR	56%	16%	$1.51\times 10^{-9}$	Significant
Qwen-Javanese	Sentiment Acc.	27%	12%	$0.0107$	Significant
	Strict SR	9%	6%	$0.6072$	Not Significant

Appendix

Table 9: Proposed Framework (

H_{1}

) vs. Layer-Controlled Baseline (

H_{0}^{(2)}

). This baseline controls for layer location by intervening on the exact same top 50% probe-ranked layers but replacing our XGBoost -selected coordinates with random neuron indices.

Figure 11: Layer selection divergence across Gemma-3 (top-17 of 34), Llama-3.2 (top-14 of 28), and Qwen-3 (top-18 of 36). Blue and red striped regions indicate layers active exclusively in the Monolingual (EN) and Multilingual (11-lang) setups, respectively, while black indicates agreement.

Figure 12: Neuron activation divergence between the Monolingual (EN) and Multilingual (11-lang) directions within the representative middle layer of each model. The minimal overlap across all three architectures confirms that the setups utilize distinct active neuron sub-networks.

Setup	Vector	L/N Config	Scale	SA	SR
Mono	$v_{\text{mono}}$	Mono	Tuned	60.8	54.6
$H_{0}^{(A)}$	$v_{\text{scaled}}$	Mono	$\alpha_{\text{multi}}$	61.3	56.1
$H_{0}^{(B)}$	$v_{\text{scaled}}$	Multi	$\alpha_{\text{multi}}$	61.4	56.5
$H_{1}$ (Multi)	$v_{\text{multi}}$	Multi	Tuned	64.4	58.9

Appendix

Table 10: Performance of norm-matched and configuration-swapped baselines on Gemma-3 (Hausa). Metric scores (SA, SR) are in percentages (%). Statistical significance (

p<0.05

) across identical prompt instances was achieved for Sentiment Accuracy (SA) against both baselines, but not for Steering Rate (SR).

Model	Layer Config	Neuron Selection	$\alpha$	Lang	Translated Prompt	Degenerate Output
Dense Intervention: Disabling Sparse Selection
Llama3.2-3B	top-5	all	$0.5$	en	He turned on his computer.	\nTheLG-HeCJHG-4HEL-200D-JQuestion-Question-
Llama3.2-3B	top-5	all	$0.3$	zh	He put on his running shoes.	— - – – – – – – – – – – – – – – – - - - - - - - - - - - – – - - - - - - - - - - - - - - - - - - - - - - - - - - -.. - - - - - - - - - -........ - .................................
Qwen3-4B	top-5	all	$0.2$	id	The alarm went off at 6:30 AM.	,,,,,, ordered of,bbbdon of ordered of of of of, of monn,bonitnaknebonononnebit of, amongonnebnebnebvitit, of are amongnebpitnebybpbnenebb ofbneonbonbbbhnepnenebhonitnepnebpitnedithitonnebfuccitititbbbb atvnenenelenelepitbononithitneb
Qwen3-4B	top-5	all	$0.2$	de	He looked at his phone screen.	\n \n \n \n\n \n\n\n \n \n\n \n \n \n \n\n \n \n\n \n \n\n\n \n \n \n \n \n \n \n \n \n......... \n \n \n \n\n \n \n \n \n \n \n \n...... \nd\n\nd\n... \n \n \n... \n \n \n \n \n \n \n.........
Magnitude Oversaturation: Figure 3 Black Cells

Appendix

Table 11: Representative degenerate outputs illustrating the two failure modes discussed in the paper. All examples utilize the Monolingual steering vector. Top: Linguistic collapse caused by dense intervention (steering all neurons), excluded from the main sweep. Bottom: Collapse stemming from magnitude oversaturation on sparse features (

\alpha=1.0

), which corresponds directly to the unevaluable black cells in Figure 3 .

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 291 papers

Jan 23, 2026

Cross-Lingual Activation Steering for Multilingual Language Models

Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant languages. Prior work attributes this gap to imbalances between shared and language-specific neurons in multilingual representations. We propose Cross-Lingual Activation Steering (CLAS), a training-free inference-time intervention that selectively modulates neuron activations. We evaluate CLAS on classification and generation benchmarks, achieving average improvements of 2.3% (Acc.) and 3.4% (F1) respectively, while maintaining high-resource language performance. We discover that effective transfer operates through functional divergence rather than strict alignment; performance gains correlate with increased language cluster separation. Our results demonstrate that targeted activation steering can unlock latent multilingual capacity in existing models without modification to model weights.

Rhitabrat Pokharel, Ameeta Agrawal, Tanay Nagar

Department of Computer Science, Portland State University, USA · Independent Researcher

Multilingual Language Models Language Model Steering Causal Interventions in Language Models Cross-Lingual Transfer

Ask PDF

Aug 5, 2026

Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference

Multilingual large language models exhibit substantial performance differences across languages, while existing adaptation methods often require parameter updates and considerable multilingual training data. We propose an inference-time multilingual steering method that uses pretrained sparse autoencoders to identify and strengthen target-language-related features. Using multilingual parallel sentences, we compare SAE activations across languages and select a small number of layer-specific features associated with each target language. These features are decoded into steering signals and injected into the model's hidden states without additional training. Experiments with Gemma-3-12B-it show macro-average accuracy improvements of 11.28 percentage points on XCOPA, 4.11 points on XNLI, and 7.84 points on PAWS-X. Additional evaluation with Qwen2.5-7B-Instruct yields a macro-average improvement of 3.33 percentage points on XCOPA.

Hongsheng Wang, Philipp Koehn

Johns Hopkins University, Baltimore / USA

Multilingual Language Models Language Model Steering LLM Inference Inference-Time Adaptation of Language Models Cross-Lingual Transfer

Ask PDF

Oct 17, 2026

Compositional Activation Steering Across Languages and Behaviors with Contextual Gates

Activation steering has emerged as an effective and precise way to influence the behaviours of language models. However, composing multiple steering directions to control several behaviours simultaneously remains challenging, particularly in multilingual settings and with training-free steering vectors. We address this challenge with contextual steering, jointly training gates that adapt the strength of fixed steering directions to the current hidden state and interactions between interventions. Across a broad range of languages, behavioural attributes, and models, our approach outperforms constant-strength steering and four strong baselines, including prompting. It also better preserves general knowledge and keeps the model's internal computation closer to that of the prompted model. Our analysis further shows how jointly trained gates adapt their contributions during composition, explaining the mechanisms underlying effective compositional steering.

Daniil Gurgurov, Akansh Maurya, Josef van Genabith +2

Saarland University · German Research Center for AI (DFKI) · RIKEN +2

Language Model Steering Activation Steering

Ask PDF

Abstract

Language Model Steering

Explore similar work

Cross-Lingual Activation Steering for Multilingual Language Models

Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference

Compositional Activation Steering Across Languages and Behaviors with Contextual Gates