Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment

Yuyan Bu; Xiaohao Liu; ZhaoXing Ren; Yaodong Yang; Juntao Dai

cs.CLFeb 18, 2026

Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment

Authors: Yuyan Bu, Xiaohao Liu, ZhaoXing Ren, Yaodong Yang, Juntao Dai

Organizations: Beijing Academy of Artificial Intelligence · National University of Singapore · Institute for Artificial Intelligence, Peking University

Abstract

The widespread deployment of large language models (LLMs) across linguistic communities necessitates reliable multilingual safety alignment. However, recent efforts to extend alignment to other languages often require substantial resources, either through large-scale, high-quality supervision in the target language or through pairwise alignment with high-resource languages, which limits scalability. In this work, we propose a resource-efficient method for improving multilingual safety alignment. We introduce a plug-and-play Multi-Lingual Consistency (MLC) loss that can be integrated into existing monolingual alignment pipelines. By improving collinearity between multilingual representation vectors, our method encourages directional consistency at the multilingual semantic level in a single update. This allows simultaneous alignment across multiple languages using only multilingual prompt variants without requiring additional response-level supervision in low-resource languages. We validate the proposed method across different model architectures and alignment paradigms, and demonstrate its effectiveness in enhancing multilingual safety with limited impact on general model utility. Further evaluation across languages and tasks indicates improved cross-lingual generalization, suggesting the proposed approach as a practical solution for multilingual consistency alignment under limited supervision.

Figures & tables

Figure 1: (a) Problem : Models show uneven safety performance, exhibiting safety failures in low-resource languages (SW) despite safe responses in high-resource ones (EN, ZH). (b) Underperformance : Existing methods still yield uneven post-alignment safety across languages despite achieving partial improvements. (c) Our Method : We propose to jointly align all languages by enforcing colinear constraints on representations to ensure more consistent multilingual safety performance.

Models	EN	ZH	RU	JA	AR	BN	SW	UR	PS	KU	Avg $\uparrow$	Var $\downarrow$	PAG $\uparrow$
Qwen-2.5-7B-Instruct
Raw	93.33	96.11	93.33	92.22	93.89	53.33	6.11	33.89	21.11	12.22	59.55	13.14	0.5037
SDRRL	57.22	73.33	77.22	75.00	77.22	72.22	64.44	75.56	69.44	61.11	70.28	0.45	0.8412
MPO	81.11	81.67	82.22	77.22	78.33	77.22	4.44	70.56	59.44	42.78	65.50	5.53	0.6979
DPO	99.44	98.33	97.22	97.78	96.11	70.56	7.22	50.56	30.00	17.22	66.44	12.44	0.5437
DPO+MLC	99.44	96.67	97.78	98.33	98.33	95.00	92.78	92.78	91.11	97.22	95.94	0.07	0.9697

Table 1: Multilingual Safety Performance on PKU-SafeRLHF. Avg, Var denote the mean and variance of safety rates across languages, respectively. PAG measures the average pairwise agreement among multilingual responses per input. Best results are in bold with the second underlined.

Models	In-Distribution Languages						Out-of-Distribution Languages
	EN	ZH	AR	BN	SW	Avg $\downarrow$	KO	IT	JV	TH	VI	Avg $\downarrow$
Qwen 2.5-7B-Instruct
Raw	7.30	4.76	10.48	33.02	30.79	17.27	5.71	9.52	10.16	8.25	6.67	8.06
SDRRL	6.98	5.39	9.84	30.79	31.42	16.88	7.3	9.52	9.2	8.57	7.3	8.38
MPO	6.35	3.81	4.76	5.71	0.32	4.19	2.22	4.13	2.86	5.71	3.17	3.62
DPO	2.86	1.90	5.08	17.78	25.08	10.54	3.17	3.49	4.13	5.40	1.90	3.62

Table 2: Multilingual safety performance on MultiJail. Results are grouped by in-distribution (ID) and out-of-distribution (OOD) languages, with averages reported for each group. Evaluation is based on Attack Success Rate (ASR), where lower scores indicate better safety.

	Qwen 2.5-7B					Gemma-2-9B-it
Evaluation Set	Raw	SDRRL	MPO	DPO	DPO+MLC	Raw	SDRRL	MPO	DPO	DPO+MLC
MMLU	76.37	31.57	75.90	76.22	76.30	74.34	51.90	73.85	73.79	73.58
MMMLU-lite Seen	51.17	39.93	51.51	49.69	48.51	51.40	49.05	55.55	50.89	53.54
MMMLU-lite Unseen	57.37	55.42	52.76	57.32	54.97	48.05	49.88	48.56	48.37	55.20
MMMLU-lite All	55.16	49.88	52.31	54.59	52.66	49.25	49.65	51.06	49.27	54.61

Table 3: Results of multilingual general capability evaluation. MMLU evaluates the utility in English. MMMLU-lite Seen reports the average performance on languages present in both the training data and MMMLU-lite. MMMLU-lite Unseen reports the average performance on languages included in MMMLU but not seen during training. MMMLU-lite All shows the overall average performance across all languages in MMMLU-lite.

Models	EN	ZH	RU	JA	AR	BN	SW	UR	PS	KU	Avg $\uparrow$	Var $\downarrow$	PAG $\uparrow$
Qwen 2.5-1.5B
Raw	90.56	91.11	83.33	76.67	82.22	12.78	17.22	7.22	16.11	24.44	50.17	12.29	0.4961
DPO	98.89	97.78	97.22	91.11	93.33	29.44	40.56	38.33	31.11	47.22	66.50	8.76	0.5659
DPO+MLC	95.56	98.33	93.89	94.44	93.89	95.56	91.67	95.56	92.78	89.44	94.11	0.05	0.9912
Qwen 2.5-3B
Raw	93.89	95.00	90.00	86.67	90.56	26.67	6.67	18.33	10.00	18.89	53.67	14.40	0.4872

Table 4: Scaling behavior across model sizes. Multilingual safety performance is evaluated on PKU-SafeRLHF using Qwen-2.5 models of varying sizes (1.5B, 3B, and 7B).

Figure 2: Multilingual safety performance across different alignment paradigms.

Panel of Figure 3 — Figure 3: Gram matrix visualization across different models.

Panel of Figure 4 — Figure 4: Layer-depth study compared to the raw model on different metrics.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Models	EN	ZH	RU	JA	AR	BN	SW	UR	PS	KU	Avg $\uparrow$	Var $\downarrow$	PAG $\uparrow$
Raw	93.33	96.11	93.33	92.22	93.89	53.33	6.11	33.89	21.11	12.22	59.55	13.14	0.5037
DPO	99.44	98.33	97.22	97.78	96.11	70.56	7.22	50.56	30.00	17.22	66.44	12.44	0.5437
DPO+MLC (wo/P)	99.44	99.44	98.89	100.00	99.44	98.33	91.11	97.22	99.44	97.78	98.11	0.06	0.9733
DPO+MLC (AE)	95.56	91.67	90.00	93.33	87.22	82.78	58.89	74.44	78.89	78.89	83.17	1.09	0.8572
DPO+MLC (LP)	99.44	96.67	97.78	98.33	98.33	95.00	92.78	92.78	91.11	97.22	95.94	0.07	0.9697

Appendix

Table 5: Multilingual safety performance using different feature extractors. Avg denotes the average safety score across languages, Var is the variance (lower is better), and PAG is the pairwise agreement across languages (higher is better).

	Raw	DPO	DPO+MLC (LP)	DPO+MLC (AE)	DPO+MLC (wo/P)
MMLU	76.37	76.22	76.30	76.54	76.55
MMMLU-lite Seen	51.17	49.69	48.51	43.87	46.61
MMMLU-lite Unseen	57.37	57.32	54.97	49.31	48.58
MMMLU-lite All	55.16	54.59	52.66	47.37	47.35

Appendix

Table 6: General utility performance using different feature extractors. Results are reported on MMLU and MMMLU-lite benchmarks, with higher scores indicating better general capability.

Method	Avg Safety Rate $\uparrow$	PAG $\uparrow$	MMLU $\uparrow$	MMMLU-lite $\uparrow$
Raw	59.55	0.5037	76.37	55.16
MLC w/ Cosine Similarity	98.05	0.9714	76.17	45.92
MLC w/ Rank-1 (Ours)	95.94	0.9697	76.30	52.66

Appendix

Table 7: Comparison of different similarity measures. Results highlight the trade-off between the strength of the consistency constraint and the retention of general capability.

Panel of Figure 5 — Figure 5: Analysis of Hyperparameter $\lambda_{aux}$ .

Panel of Figure 6 — Figure 6: Analysis of Hyperparameter $\tau$ .

Translation Source	Avg Safety Rate $\uparrow$	PAG $\uparrow$	MMLU $\uparrow$	MMMLU-lite $\uparrow$
GPT-4o	95.94	0.970	76.30	52.66
NLLB	96.83	0.973	76.26	47.81
Permute-30	96.56	0.961	76.43	45.84

Appendix

Table 8: Impact of translation quality on multilingual safety and general capability. Safety performance remains highly stable across translation qualities, while multilingual general utility degrades as translation noise increases.

Figure 15

LLM Safety Alignment

Large language model safety alignment trains and evaluates systems to follow human policies while resisting harmful requests and misuse.

See all 341 papers

Multilingual Refusal Alignment for Safer Large Language Models
Apr 24, 2026Aleksandra Krasnodębska, Wojciech Kusa, Aldo LipaniMultilingual Language Model Evaluation LLM Alignment
NASK National Research Institute, Warsaw, Poland · University College London, London, UK
Multilingual Safety Alignment Via Sparse Weight Editing
Feb 26, 2026Jiaming Liang, Zhaoxin Wang, Handing WangLLM Safety Alignment LLM Safety
School of Artificial Intelligence, Xidian University.
Multilingual Safety Alignment via Self-Distillation
May 3, 2026Ruiyang Qin, Qingzhuo Wang, Dongrui Liu +3Multilingual Language Models LLM Safety Alignment
Tongji University · Shanghai AI Laboratory

Apr 24, 2026J/K move · Enter open · S save

Multilingual Refusal Alignment for Safer Large Language Models

Aleksandra Krasnodębska, Wojciech Kusa, Aldo Lipani

NASK National Research Institute, Warsaw, Poland · University College London, London, UK

Multilingual Language Model Evaluation LLM Alignment LLM Safety Alignment Cross-Lingual Consistency in Language Models Cross-Lingual Transfer

Open paper AskPDF arXiv

As Large Language Models (LLMs) are deployed globally, ensuring their safety and alignment across multiple languages becomes paramount. However, safety behaviors often vary unpredictably between languages, posing significant challenges for consistent and ethical AI. In this work, we systematically investigate the dynamics of multilingual alignment, exploring whether single-language alignment transfers cross-lingually, how language consistency is preserved during training, and the resulting trade-offs with general knowledge capabilities. We introduce RefusEU, a novel refusal alignment dataset covering 12 European languages, including a dedicated test set for evaluating current state-of-the-art models. Our controlled Direct Preference Optimization (DPO) experiments provide two key insights: aligning models exclusively in English is insufficient to ensure cross-lingual safety, even for the same harm categories, whereas training on multilingual datasets can improve safety without degrading general performance, as measured by the Global MMLU benchmark.

Abstract

LLM Safety Alignment

Explore similar work

Multilingual Refusal Alignment for Safer Large Language Models

Multilingual Safety Alignment Via Sparse Weight Editing

Multilingual Safety Alignment via Self-Distillation