Steering with Multilingual Lexicons for Sentiment Control in Low-Resource Languages

Lucinda Laurent; Muhammad Dehan Al Kautsar; Alfan Farizki Wicaksono; Fajri Koto

cs.CLNov 2, 2026

Steering with Multilingual Lexicons for Sentiment Control in Low-Resource Languages

Authors: Lucinda Laurent, Muhammad Dehan Al Kautsar, Alfan Farizki Wicaksono, Fajri Koto

Organizations: Faculty of Computer Science, Universitas Indonesia · Mohamed bin Zayed University of Artificial Intelligence

Abstract

Activation steering offers a lightweight mechanism for controlling model behavior, but existing methods depend on sentence-level contrastive data that is unavailable for most of the world's languages. We show that multilingual sentiment lexicons are a viable substitute across three model families: aggregated multilingual lexicons outperform English-derived steering vectors for mid- and low-resource languages, while monolingual lexicons remain stronger for languages a model has specialized in during pre-training. The steering efficacy is mechanistically dictated by both representational density and pre-training priors. Large-magnitude English vectors overwhelm the compact representations of low-resource languages, causing "linguistic collapse." This fragility is mitigated by the norm-dampening of multilingual aggregation. Architectural biases, including bilingual specialization penalty and latent numerical rigidity, also heavily constrain cross-lingual steering responsiveness. Ultimately, our work establishes lexicons as a practical steering signal and offers a comprehensive mechanistic account of cross-lingual steering dynamics.

Figures & tables

Figure 1: Workflow of the Cross-Lingual Sentiment Control using Multilingual Lexicon. In the Extraction Phase, feature attribution selects the steering layers

i\in\mathcal{S}

(linear-probe macro-F1) and, within each, the neurons

\mathcal{N}_{i}

( XGBoost gain). At each selected layer, the steering vector is the difference

\mu^{+,(i)}-\mu^{-,(i)}

between multilingual positive and negative lexicon centroids, sliced to

\mathcal{N}_{i}

. In the Injection Phase, this vector is applied to neutral prompts with coefficient

+\alpha

-\alpha

, shifting the generated text toward positive or negative sentiment across multiple languages.

Figure 2: Impact of Steering on Sentiment Distribution (Gemma-3). The distribution of generated sentiments across 11 languages under three conditions: (a) No Intervention (as baseline), (b) Positive Steering, and (c) Negative Steering. The proposed method successfully shifts the predominantly neutral baseline toward the target sentiment. Corresponding distributions for Qwen-3 and Llama-3.2 are provided in Appendix D .

		Gemma-3		Qwen-3		Llama-3.2
Lang	Res.	SR (%)	Layers	SR (%)	Layers	SR (%)	Layers
English (en)	High	94.3	all	45.1	all	65.8	all
German (de)	High	90.5	all	46.1	top-50%	53.4	all
Chinese (zh)	High	86.2	all	48.0	all	33.1	all
Japanese (ja)	High	76.8	all	23.5	all	18.9	all
Indonesian (id)	Mid	82.0	all	46.8	all	37.0	all

Table 1: Cross-Lingual Steering Performance with Multilingual Lexicons. Success Rate (SR, %) across three models and

11

languages. Each cell reports SR at the best-performing layer configuration (all, top-

50

%, or top-

5

		Gemma-3		Qwen-3		Llama-3.2
Lang	Res.	SR (%)	Layers	SR (%)	Layers	SR (%)	Layers
English (en)	High	95.0	all	44.5	all	66.4	all
German (de)	High	90.7	all	45.4	top-50%	53.8	all
Chinese (zh)	High	86.8	all	53.4	all	34.3	all
Japanese (ja)	High	80.2	all	23.9	top-50%	20.8	all
Indonesian (id)	Mid	82.3	all	42.9	top-50%	38.6	all

Table 2: Cross-Lingual Steering Performance with Monolingual English Lexicons. Success Rate (SR, %) across three models and

11

languages. Each cell reports SR at the best-performing layer configuration (all, top-

50

%, or top-

5

Figure 3: Impact of Steering Intensity on Success Rate and Language Integrity. Heatmaps show performance under the all-layers configuration as the steering coefficient

\alpha

increases from 0.5 to 1.0, for steering vectors derived from monolingual (English) and multilingual lexicons. Values represent Success Rate (%); black cells indicate linguistic collapse, where the generated text was incoherent and thus could not be evaluated for sentiment.

Figure 4: Heterogeneous Impact of Multilingual Aggregation Across Model Families. This figure highlights the divergent impact of the Multilingual steering direction across three distinct model families. Positive values (teal) indicate that the Multilingual direction outperforms the Monolingual direction.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Lang.	ISO	Llama-3.2	Qwen3-4B	Gemma3-4B
English	en	$\mathbf{8.47}$	$8.53$	$8.54$
German	de	$12.39$	$12.42$	$\mathbf{10.80}$
Chinese	zh	$9.75$	$\mathbf{7.62}$	$8.03$
Japanese	ja	$13.41$	$12.43$	$\mathbf{9.84}$
Indonesian	id	$12.51$	$12.61$	$\mathbf{9.41}$
Icelandic	is	$17.42$	$17.62$	$\mathbf{15.14}$

Appendix

Table 3: Average tokens per neutral prompt across target languages. Lower values indicate higher tokenization efficiency and a more optimized vocabulary for that language.

English (High-Resource)
	Human Annotation (Gold)
Predicted	Negative	Positive	Neutral	Precision
Negative	47	0	3	94%
Positive	0	50	0	100%
Neutral	1	1	48	96%
Macro-Average				96.7%

Appendix

Table 4: Confusion matrices for the GPT-4.1 sentiment judge across 150 randomized samples per language.

Figure 5: Impact of Steering on Sentiment Distribution (Qwen-3). Steering induces a noticeable shift toward the target sentiment, confirming the method’s cross-model applicability. However, Qwen-3 exhibits a stronger tendency to preserve neutral tonality compared to other models, with neutral outputs remaining the dominant class in mid to low resource languages even after intervention.

Figure 6: Impact of Steering on Sentiment Distribution (Llama-3.2). While the steering vectors successfully increase the probability of the target sentiment across all languages, Llama-3.2 retains a higher proportion of neutral outputs compared to Gemma-3, particularly in low-resource languages (e.g., Hausa, Javanese, Sundanese).

Figure 7: Impact of Steering with Monolingual Steering Direction on Sentiment Distribution (Gemma-3). The distribution of generated sentiments across 11 languages under three conditions: (a) No Intervention (as baseline), (b) Positive Steering, and (c) Negative Steering.

		Gemma-3				Qwen-3				Llama-3
Lang	Res.	SR	SA	LIDR	Config	SR	SA	LIDR	Config	SR	SA	LIDR	Config
English (en)	High	$95.0$	$95.0$	$99.8$	all	$44.5$	$54.6$	$84.1$	all	$66.4$	$67.0$	$98.6$	all
German (de)	High	$90.7$	$91.8$	$98.4$	all	$45.4$	$52.8$	$72.1$	top-50%	$53.8$	$64.2$	$83.0$	all
Chinese (zh)	High	$86.8$	$91.0$	$93.5$	all	$53.4$	$62.0$	$82.5$	all	$34.3$	$44.5$	$67.5$	all
Japanese (ja)	High	$80.2$	$89.3$	$88.6$	all	$23.9$	$31.3$	$57.5$	top-50%	$20.8$	$38.1$	$58.2$	all
Indonesian (id)	Mid	$82.3$	$91.0$	$89.8$	all	$42.9$	$47.9$	$85.1$	top-50%	$38.6$	$51.3$	$69.1$	all

Appendix

Table 5: Best Results (Mono). SR: Success Rate, SA: Sentiment Acc, LIDR: Language ID Rate. All metrics in %.

		Gemma-3				Qwen-3				Llama-3
Lang	Res.	SR	SA	LIDR	Config	SR	SA	LIDR	Config	SR	SA	LIDR	Config
English (en)	High	$94.3$	$94.4$	$99.7$	all	$45.1$	$48.5$	$91.6$	all	$65.8$	$66.2$	$98.9$	all
German (de)	High	$90.5$	$91.4$	$98.5$	all	$46.1$	$51.1$	$78.8$	top-50%	$53.4$	$63.4$	$82.0$	all
Chinese (zh)	High	$86.2$	$89.0$	$95.0$	all	$48.0$	$54.2$	$85.8$	all	$33.1$	$42.8$	$71.5$	all
Japanese (ja)	High	$76.8$	$85.9$	$87.8$	all	$23.5$	$43.2$	$43.3$	all	$18.9$	$33.2$	$60.1$	all
Indonesian (id)	Mid	$82.0$	$91.0$	$88.9$	all	$46.8$	$53.2$	$83.2$	all	$37.0$	$49.9$	$69.8$	all

Appendix

Table 6: Best Results (Multi). SR: Success Rate, SA: Sentiment Acc, LIDR: Language ID Rate. All metrics in %.

Model	Multi SR	Mono SR	$\Delta$ SR	$Z$ -score	$p$ -value
High-resource ( $4$ langs, $n=8{,}000$ )
Gemma-3	$86.95$	$88.17$	$-1.22$	$-2.34$	$\mathbf{0.0194}$
Qwen-3	$40.67$	$41.80$	$-1.13$	$-1.45$	$0.1466$
Llama-3.2	$42.80$	$43.83$	$-1.03$	$-1.31$	$0.1886$
Mid-resource ( $4$ langs, $n=8{,}000$ )
Gemma-3	$71.83$	$69.33$	$+2.50$	$+3.47$	$\mathbf{0.0005}$

Appendix

Table 7: Two-sample proportion

Z

-tests comparing multilingual vs. monolingual (English) steering success rates (SR, %) by language resource tier. Bold

p

-values indicate rejection of the null hypothesis at

\alpha=0.05

. Positive

\Delta

SR favors the multilingual direction; negative favors the monolingual baseline.

Figure 8: Comparison of

L_{2}

norm magnitudes for sentiment steering directions (all neurons) across Gemma-3 , Llama-3.2 , and Qwen-3 . Monolingual directions for German, Chinese, Indonesian, and Javanese are extracted and plotted solely to analyze the latent geometry and are not used for injection.

Figure 9:

L_{2}

norm magnitude of the nonzero-gain steering vectors across layers for Gemma-3 , Llama-3.2 , and Qwen-3 , comparing English and multilingual steering directions. Only neurons with non-zero XGBoost gain are included in the norm computation, reflecting the actual magnitude of intervention applied at inference time. Opaque markers denote layers in the top 50% by F1 score of the corresponding linear probe; faded markers denote layers outside this top half.

Panel of Figure 10 — Figure 10: Cosine similarity heatmaps between monolingual and multilingual steering directions across three architectures. The displayed values represent the average cosine similarity across all layers.

Model & Language	Metric	$H_{1}$ (Ours)	$H_{0}^{(1)}$	McNemar $p$ -value	Significance ( $\alpha=0.05$ )
Gemma-Hausa	Sentiment Acc.	66%	3%	$3.09\times 10^{-17}$	Significant
	Strict SR	56%	0%	$2.78\times 10^{-17}$	Significant
Qwen-Javanese	Sentiment Acc.	27%	7%	$0.0003$	Significant
	Strict SR	9%	4%	$0.2266$	Not Significant

Appendix

Table 8: Proposed Framework (

H_{1}

) vs. Fully Random Baseline (

H_{0}^{(1)}

). This baseline evaluates the effect of applying the steering vector to a random subset of layers (

50

% of all layers) and randomly sampled neuron indices, preserving the exact intervention size (

|{}N_{i}|{}

) of the original targeted configuration.

Model & Language	Metric	$H_{1}$ (Ours)	$H_{0}^{(2)}$	McNemar $p$ -value	Significance ( $\alpha=0.05$ )
Gemma-Hausa	Sentiment Acc.	66%	17%	$5.91\times 10^{-12}$	Significant
	Strict SR	56%	16%	$1.51\times 10^{-9}$	Significant
Qwen-Javanese	Sentiment Acc.	27%	12%	$0.0107$	Significant
	Strict SR	9%	6%	$0.6072$	Not Significant

Appendix

Table 9: Proposed Framework (

H_{1}

) vs. Layer-Controlled Baseline (

H_{0}^{(2)}

). This baseline controls for layer location by intervening on the exact same top 50% probe-ranked layers but replacing our XGBoost -selected coordinates with random neuron indices.

Figure 11: Layer selection divergence across Gemma-3 (top-17 of 34), Llama-3.2 (top-14 of 28), and Qwen-3 (top-18 of 36). Blue and red striped regions indicate layers active exclusively in the Monolingual (EN) and Multilingual (11-lang) setups, respectively, while black indicates agreement.

Figure 12: Neuron activation divergence between the Monolingual (EN) and Multilingual (11-lang) directions within the representative middle layer of each model. The minimal overlap across all three architectures confirms that the setups utilize distinct active neuron sub-networks.

Setup	Vector	L/N Config	Scale	SA	SR
Mono	$v_{\text{mono}}$	Mono	Tuned	60.8	54.6
$H_{0}^{(A)}$	$v_{\text{scaled}}$	Mono	$\alpha_{\text{multi}}$	61.3	56.1
$H_{0}^{(B)}$	$v_{\text{scaled}}$	Multi	$\alpha_{\text{multi}}$	61.4	56.5
$H_{1}$ (Multi)	$v_{\text{multi}}$	Multi	Tuned	64.4	58.9

Appendix

Table 10: Performance of norm-matched and configuration-swapped baselines on Gemma-3 (Hausa). Metric scores (SA, SR) are in percentages (%). Statistical significance (

p<0.05

) across identical prompt instances was achieved for Sentiment Accuracy (SA) against both baselines, but not for Steering Rate (SR).

Model	Layer Config	Neuron Selection	$\alpha$	Lang	Translated Prompt	Degenerate Output
Dense Intervention: Disabling Sparse Selection
Llama3.2-3B	top-5	all	$0.5$	en	He turned on his computer.	\nTheLG-HeCJHG-4HEL-200D-JQuestion-Question-
Llama3.2-3B	top-5	all	$0.3$	zh	He put on his running shoes.	— - – – – – – – – – – – – – – – – - - - - - - - - - - - – – - - - - - - - - - - - - - - - - - - - - - - - - - - -.. - - - - - - - - - -........ - .................................
Qwen3-4B	top-5	all	$0.2$	id	The alarm went off at 6:30 AM.	,,,,,, ordered of,bbbdon of ordered of of of of, of monn,bonitnaknebonononnebit of, amongonnebnebnebvitit, of are amongnebpitnebybpbnenebb ofbneonbonbbbhnepnenebhonitnepnebpitnedithitonnebfuccitititbbbb atvnenenelenelepitbononithitneb
Qwen3-4B	top-5	all	$0.2$	de	He looked at his phone screen.	\n \n \n \n\n \n\n\n \n \n\n \n \n \n \n\n \n \n\n \n \n\n\n \n \n \n \n \n \n \n \n \n......... \n \n \n \n\n \n \n \n \n \n \n \n...... \nd\n\nd\n... \n \n \n... \n \n \n \n \n \n \n.........
Magnitude Oversaturation: Figure 3 Black Cells

Appendix

Table 11: Representative degenerate outputs illustrating the two failure modes discussed in the paper. All examples utilize the Monolingual steering vector. Top: Linguistic collapse caused by dense intervention (steering all neurons), excluded from the main sweep. Bottom: Collapse stemming from magnitude oversaturation on sparse features (

\alpha=1.0

), which corresponds directly to the unevaluable black cells in Figure 3 .

Language Model Steering

Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.

See all 291 papers

Cross-Lingual Activation Steering for Multilingual Language Models
Jan 23, 2026Rhitabrat Pokharel, Ameeta Agrawal, Tanay NagarMultilingual Language Models Language Model Steering
Department of Computer Science, Portland State University, USA · Independent Researcher
Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference
Aug 5, 2026Hongsheng Wang, Philipp KoehnMultilingual Language Models Language Model Steering
Johns Hopkins University, Baltimore / USA
Compositional Activation Steering Across Languages and Behaviors with Contextual Gates
Oct 17, 2026Daniil Gurgurov, Akansh Maurya, Josef van Genabith +2Language Model Steering Activation Steering
Saarland University · German Research Center for AI (DFKI) · RIKEN +2

Jan 23, 2026J/K move · Enter open · S save

Cross-Lingual Activation Steering for Multilingual Language Models

Rhitabrat Pokharel, Ameeta Agrawal, Tanay Nagar

Department of Computer Science, Portland State University, USA · Independent Researcher

Multilingual Language Models Language Model Steering Causal Interventions in Language Models Cross-Lingual Transfer

Open paper AskPDF arXiv

Large language models exhibit strong multilingual capabilities, yet significant performance gaps persist between dominant and non-dominant languages. Prior work attributes this gap to imbalances between shared and language-specific neurons in multilingual representations. We propose Cross-Lingual Activation Steering (CLAS), a training-free inference-time intervention that selectively modulates neuron activations. We evaluate CLAS on classification and generation benchmarks, achieving average improvements of 2.3% (Acc.) and 3.4% (F1) respectively, while maintaining high-resource language performance. We discover that effective transfer operates through functional divergence rather than strict alignment; performance gains correlate with increased language cluster separation. Our results demonstrate that targeted activation steering can unlock latent multilingual capacity in existing models without modification to model weights.

Abstract

Language Model Steering

Explore similar work

Cross-Lingual Activation Steering for Multilingual Language Models

Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference

Compositional Activation Steering Across Languages and Behaviors with Contextual Gates