Organizations: School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · Nanjing University of Science and Technology, Nanjing, China · School of Artificial Intelligence, Beihang University, Beijing, China · College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, China · School of Computing, National University of Singapore, Singapore, Singapore · Xi’an Jiaotong University, Xi’an, China
The safety capabilities of large language models (LLMs) are highly uneven across different languages, often allowing cross-lingual attacks to bypass safety guardrails. Existing alignment methods predominantly rely on external alignment signals and provide limited insight into underlying neuron-level mechanisms, making it difficult to provide a unified internal target for cross-lingual safety interventions. To address this, we analyze cross-lingual safety behavior at the neuron level and identify a compact set of safety-related neurons shared across languages, termed the safety kernel (SK). Specifically, we first utilize multilingual parallel safe and unsafe prompts to compute safety contrast with activation responsiveness (SCAR) scores, identifying language-wise safety-neuron candidates from internal activation patterns. We then intersect the resulting language-wise safety-neuron sets to obtain the SK, and validate its functional relevance to cross-lingual safety behavior through activation suppression and activation-state transfer interventions. Finally, building on the SK coordinates, we introduce SK-Masked LoRA, which constrains LoRA adaptation to the feed-forward network (FFN) output dimensions corresponding to the SK, thereby focusing safety-related changes on the cross-lingually shared neuron coordinates. Extensive experiments demonstrate that SK-Masked LoRA substantially reduces attack success rates across multiple languages, including languages excluded from SK discovery, while intervening on only a small fraction of internal neuron coordinates and producing generally limited changes in the model's general capabilities.
Figures & tables
Figure 1. Motivation of the Safety Kernel. (a) Existing approaches improve multilingual safety through external alignment signals, while the shared internal target remains unclear. (b) We identify a compact set of safety-related neurons shared across languages, termed the safety kernel.
Figure 2. Overview of the proposed framework. (a) Language-wise safety-neuron sets are identified using SCAR scores computed from activation differences between unsafe and safe prompts, and their cross-lingual intersection forms the SK. (b) The functional relevance of the SK is examined through neuron activation suppression and cross-lingual safety-state transfer. (c) SK-Masked LoRA restricts fine-tuning-induced output changes to the SK coordinates for multilingual safety adaptation.
Setting
EN
ZH
IT
KO
TH
BN
AVG.
Backbone: Qwen3-8B
Default
12.43
9.57
12.78
20.37
14.33
28.94
16.40
Random
14.29
7.30
16.51
19.37
16.51
27.30
16.88
Mask Lang.-Wise
81.06
69.63
79.79
77.25
79.79
73.13
76.78
Backbone: Llama-3.1-8B
Default
9.84
11.74
17.46
30.47
23.81
65.71
26.51
Table 1. Functional validation of language-wise safety neurons on MultiJail. ASR (%, ↓ ) is reported, with higher post-suppression values indicating safety degradation. For each language, Mask Lang.-Wise suppresses the top-5000 neurons ranked by SCAR, while Random suppresses neurons matched in number and module-wise distribution.
Setting
Neuron Ratio
ASR ( ↓ )
Backbone: Qwen3-8B
Default
–
16.40
Mask Random
0.12%
17.57 (+1.17)
Mask Lang.-Wise
0.57%
76.78 (+60.38)
Mask SK
0.12%
69.05 (+52.65)
Backbone: Llama-3.1-8B
Table 2. Compactness and functional effect of the safety kernel on MultiJail. Language-wise sets contain the top-5000 SCAR-ranked neurons, while Mask Random matches the SK in size and module-wise distribution. Neuron Ratio is computed over all analyzed neurons. ASR is averaged over EN, ZH, IT, KO, TH, and BN, with changes from Default shown in parentheses.
Figure 3. Activation-state intervention through SK neurons on MultiJail. The safety activation state is estimated from English and Chinese safe refusals and transferred to the target languages. Lower ASR indicates better safety.
Method
AdvBench-x
MultiJail
EN
ZH
KO
BN
AVG.
EN
ZH
KO
BN
AVG.
Backbone: Llama-3.1-8B-it
Default
5.57
4.03
30.38
72.88
28.22
9.84
11.74
30.47
65.71
29.44
SmoothLLM ( Robey et al., 2023 )
7.94
9.52
31.11
52.70
25.32
7.94
9.52
31.11
52.70
25.32
SimPO ( Meng et al., 2024 )
5.77
3.46
17.69
28.94
13.97
9.21
8.25
30.48
40.63
22.14
R-DPO ( Park et al., 2024 )
3.85
3.27
3.27
7.49
4.47
10.16
14.29
35.87
42.22
25.64
Table 3. Comparison of ASR (%, ↓ ) on AdvBench-x and MultiJail. Results are shown for two backbones. The best results are highlighted in bold, while the second-best results are underlined.
Figure 4. Qualitative comparison across three LLM backbones. Each column compares the original and SK-Masked LoRA models under the same harmful request, with the fine-tuned model exhibiting safer behavior.
Method
VI
JV
DE
JA
RU
AVG.
Backbone: Qwen3-8B
Default
10.16
15.24
15.24
17.78
13.65
14.41
Ours
7.62
11.11
8.25
8.57
9.21
8.95
Backbone: Llama-3.1-8B-it
Default
8.89
14.92
17.78
36.83
14.92
18.67
Ours
4.44
5.08
4.44
5.40
4.44
4.76
Table 4. ASR (%, ↓ ) on languages excluded from SK discovery. The evaluation data are translated from the English version of MultiJail.
Setting
Neuron Ratio (%)
ASR ( ↓ )
MMLU ( ↑ )
MGSM ( ↑ )
Backbone: Qwen3-8B
Default
–
16.40
0.726
0.560
Standard LoRA
100
5.50
0.713
0.444
EN-Masked
0.12
8.31
0.723
0.450
Union-Masked
1.35
6.35
0.723
0.585
SK-Masked
0.12
7.52
0.724
0.580
Table 5. Ablation of neuron selection and update scope. ASR is averaged across evaluated languages on MultiJail. Neuron Ratio denotes the proportion of selected neuron coordinates among all FFN output coordinates considered in the analysis. EN-Masked uses an English safety-neuron set matched in size to the SK, while Union-Masked uses the union of six language-wise safety-neuron sets.
Figure 5. Sensitivity of SK size to language composition. The six-language SK is normalized to 1.0 per model, with results averaged across three models. + Non-SK denotes adding a language-wise non-SK remainder and is capped at the displayed range.
Figure 6. Effect of top- k in safety-neuron discovery. The sweep is conducted on English using a held-out validation split that is disjoint from downstream evaluation benchmarks.
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing · School of Artificial Intelligence, Beihang University, China · Zhongguancun Laboratory, Beijing, China +2