Organizations: School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Beihang University, Beijing, China · College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · School of Computing, National University of Singapore, Singapore, Singapore · Xi’an Jiaotong University, Xi’an, China
The safety capabilities of large language models (LLMs) are highly uneven across different languages, often allowing cross-lingual attacks to bypass safety guardrails. Existing alignment methods predominantly rely on external alignment signals and provide limited insight into underlying neuron-level mechanisms, making it difficult to provide a unified internal target for cross-lingual safety interventions. To address this, we analyze cross-lingual safety behavior at the neuron level and identify a compact set of safety-related neurons shared across languages, termed the safety kernel (SK). Specifically, we first utilize multilingual parallel safe and unsafe prompts to compute safety contrast with activation responsiveness (SCAR) scores, identifying language-wise safety-neuron candidates from internal activation patterns. We then intersect the resulting language-wise safety-neuron sets to obtain the SK, and validate its functional relevance to cross-lingual safety behavior through activation suppression and activation-state transfer interventions. Finally, building on the SK coordinates, we introduce SK-Masked LoRA, which constrains LoRA adaptation to the feed-forward network (FFN) output dimensions corresponding to the SK, thereby focusing safety-related changes on the cross-lingually shared neuron coordinates. Extensive experiments demonstrate that SK-Masked LoRA substantially reduces attack success rates across multiple languages, including languages excluded from SK discovery, while intervening on only a small fraction of internal neuron coordinates and producing generally limited changes in the model's general capabilities.
Figures & tables
Figure 1. Motivation of the Safety Kernel. (a) Existing approaches improve multilingual safety through external alignment signals, while the shared internal target remains unclear. (b) We identify a compact set of safety-related neurons shared across languages, termed the safety kernel.
Figure 2. Overview of the proposed framework. (a) Language-wise safety-neuron sets are identified using SCAR scores computed from activation differences between unsafe and safe prompts, and their cross-lingual intersection forms the SK. (b) The functional relevance of the SK is examined through neuron activation suppression and cross-lingual safety-state transfer. (c) SK-Masked LoRA restricts fine-tuning-induced output changes to the SK coordinates for multilingual safety adaptation.
Setting
EN
ZH
IT
KO
TH
BN
AVG.
Backbone: Qwen3-8B
Default
12.43
9.57
12.78
20.37
14.33
28.94
16.40
Random
14.29
7.30
16.51
19.37
16.51
27.30
16.88
Mask Lang.-Wise
81.06
69.63
79.79
77.25
79.79
73.13
76.78
Backbone: Llama-3.1-8B
Default
9.84
11.74
17.46
30.47
23.81
65.71
26.51
Table 1. Functional validation of language-wise safety neurons on MultiJail. ASR (%, ↓ ) is reported, with higher post-suppression values indicating safety degradation. For each language, Mask Lang.-Wise suppresses the top-5000 neurons ranked by SCAR, while Random suppresses neurons matched in number and module-wise distribution.
Setting
Neuron Ratio
ASR ( ↓ )
Backbone: Qwen3-8B
Default
–
16.40
Mask Random
0.12%
17.57 (+1.17)
Mask Lang.-Wise
0.57%
76.78 (+60.38)
Mask SK
0.12%
69.05 (+52.65)
Backbone: Llama-3.1-8B
Table 2. Compactness and functional effect of the safety kernel on MultiJail. Language-wise sets contain the top-5000 SCAR-ranked neurons, while Mask Random matches the SK in size and module-wise distribution. Neuron Ratio is computed over all analyzed neurons. ASR is averaged over EN, ZH, IT, KO, TH, and BN, with changes from Default shown in parentheses.
Figure 3. Activation-state intervention through SK neurons on MultiJail. The safety activation state is estimated from English and Chinese safe refusals and transferred to the target languages. Lower ASR indicates better safety.
Method
AdvBench-x
MultiJail
EN
ZH
KO
BN
AVG.
EN
ZH
KO
BN
AVG.
Backbone: Llama-3.1-8B-it
Default
5.57
4.03
30.38
72.88
28.22
9.84
11.74
30.47
65.71
29.44
SmoothLLM ( Robey et al., 2023 )
7.94
9.52
31.11
52.70
25.32
7.94
9.52
31.11
52.70
25.32
SimPO ( Meng et al., 2024 )
5.77
3.46
17.69
28.94
13.97
9.21
8.25
30.48
40.63
22.14
R-DPO ( Park et al., 2024 )
3.85
3.27
3.27
7.49
4.47
10.16
14.29
35.87
42.22
25.64
Table 3. Comparison of ASR (%, ↓ ) on AdvBench-x and MultiJail. Results are shown for two backbones. The best results are highlighted in bold, while the second-best results are underlined.
Figure 4. Qualitative comparison across three LLM backbones. Each column compares the original and SK-Masked LoRA models under the same harmful request, with the fine-tuned model exhibiting safer behavior.
Method
VI
JV
DE
JA
RU
AVG.
Backbone: Qwen3-8B
Default
10.16
15.24
15.24
17.78
13.65
14.41
Ours
7.62
11.11
8.25
8.57
9.21
8.95
Backbone: Llama-3.1-8B-it
Default
8.89
14.92
17.78
36.83
14.92
18.67
Ours
4.44
5.08
4.44
5.40
4.44
4.76
Table 4. ASR (%, ↓ ) on languages excluded from SK discovery. The evaluation data are translated from the English version of MultiJail.
Setting
Neuron Ratio (%)
ASR ( ↓ )
MMLU ( ↑ )
MGSM ( ↑ )
Backbone: Qwen3-8B
Default
–
16.40
0.726
0.560
Standard LoRA
100
5.50
0.713
0.444
EN-Masked
0.12
8.31
0.723
0.450
Union-Masked
1.35
6.35
0.723
0.585
SK-Masked
0.12
7.52
0.724
0.580
Table 5. Ablation of neuron selection and update scope. ASR is averaged across evaluated languages on MultiJail. Neuron Ratio denotes the proportion of selected neuron coordinates among all FFN output coordinates considered in the analysis. EN-Masked uses an English safety-neuron set matched in size to the SK, while Union-Masked uses the union of six language-wise safety-neuron sets.
Figure 5. Sensitivity of SK size to language composition. The six-language SK is normalized to 1.0 per model, with results averaged across three models. + Non-SK denotes adding a language-wise non-SK remainder and is capped at the displayed range.
Figure 6. Effect of top- k in safety-neuron discovery. The sweep is conducted on English using a held-out validation split that is disjoint from downstream evaluation benchmarks.
Uncovering the internal mechanisms underlying the safety capabilities of large language models (LLMs) is crucial for developing trustworthy artificial intelligence. Currently, mechanistic interpretability studies on multilingual safety are largely confined to local components, such as isolated neurons. However, this static and fragmented perspective overlooks the synergy among components and fails to elucidate how safety signals dynamically propagate within the model to drive safety decisions ultimately. In this work, we move beyond isolated neurons to identify and target the cross-layer functional pathways formed during safety signal propagation, thereby uncovering the mechanisms driving the cross-lingual safety gap. Specifically, we first identify monolingual safety pathways and validate their impact on refusing harmful requests. Subsequent cross-lingual analyses reveal a sparse subset of cross-lingual shared safety pathways, confirming that this intersection acts as the internal bridge transferring safety capabilities from high-resource (HR) languages to non-high-resource (NHR) languages. Building on these mechanistic findings, we propose a pathways-targeted alignment method based on the cross-lingual shared safety pathways. Experimental results show that updating only a small fraction of pathway parameters significantly improves safety in NHR languages while largely preserving the model's general capabilities.
Shuyi Miao, Wangjie Qiu, Pengyang Shao +4
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing · School of Artificial Intelligence, Beihang University, China · Zhongguancun Laboratory, Beijing, China +2
Large Language Models (LLMs) exhibit significant safety disparities across languages, with low-resource languages (LRLs) often bypassing safety guardrails established for high-resource languages (HRLs) like English. Existing solutions, such as multilingual supervised fine-tuning (SFT) or Reinforcement Learning from Human Feedback (RLHF), are computationally expensive and dependent on scarce multilingual safety data. In this work, we propose a novel, training-free alignment framework based on Sparse Weight Editing. Identifying that safety capabilities are localized within a sparse set of safety neurons, we formulate the cross-lingual alignment problem as a constrained linear transformation. We derive a closed-form solution to optimally map the harmful representations of LRLs to the robust safety subspaces of HRLs, while preserving general utility via a null-space projection constraint. Extensive experiments across 8 languages and multiple model families (Llama-3, Qwen-2.5) demonstrate that our method substantially reduces Attack Success Rate (ASR) in LRLs with negligible impact on general reasoning capabilities, all achieved with a single, data-efficient calculation.
Jiaming Liang, Zhaoxin Wang, Handing Wang
School of Artificial Intelligence, Xidian University.
Safety alignment of large language models (LLMs) degrades across languages, yet the internal mechanism driving this asymmetry remains poorly understood. Our work, therefore, presents a systematic mechanistic analysis of multilingual safety using sparse autoencoder (SAE) features, sparse interpretable directions in the residual stream associated with harmful and harmless model behavior across three instruction-tuned LLMs, eight languages, and all model layers. We observe that safety-relevant features are architecture-dependent in terms of where they are located and how they are distributed across layers. Additionally, they are geometrically entangled with language identity and exhibit cross-lingual sharing patterns, i.e., languages share safety features to varying degrees across model depths and architectures. This safety-language entanglement has direct consequences such that ablating safety features impacts not only harmful response rates but also target language, with the degree of intervention predicted by the relationship between safety and language features. Our findings qualify the language-universality of safety alignment as architecture-dependent and offer a mechanistic account of multilingual safety interventions.
Apoorva Upadhyaya, Sandipan Sikdar
L3S Research Center, Leibniz Universität Hannover, Germany