Response-Based Knowledge Distillation for Multilingual Jailbreak Prevention Unwittingly Compromises Safety

Max Zhang; Derek Liu; Kai Zhang; Joshua Franco; Haihao Liu

cs.CLDec 8, 2025

Response-Based Knowledge Distillation for Multilingual Jailbreak Prevention Unwittingly Compromises Safety

Authors: Max Zhang, Derek Liu, Kai Zhang, Joshua Franco, Haihao Liu

Abstract

Large language models (LLMs) are increasingly deployed worldwide, yet their safety alignment remains predominantly English-centric. This allows for vulnerabilities in non-English contexts, especially with low-resource languages. We introduce a novel application of knowledge distillation (KD) in the context of multilingual jailbreak prevention, examining its efficacy. We distill the refusal behaviors of a proprietary teacher model (OpenAI o1-mini) with Low-Rank Adaptation (LoRA) into three open-source student models: Meta-Llama-3-8B-Instruct, Gemma-2-2B-IT, and Qwen3-8B, using ~28,000 multilingual jailbreak prompts from XSafety via black-box response-based, parameter-efficient fine-tuning (PEFT). Evaluation on the MultiJail benchmark reveals a counterintuitive behavior: standard fine-tuning on the teacher's ``safe'' refusal data inadvertently increases Jailbreak Success Rate (JSR) for all student models, up to 16.6 percentage points. Our experiments reveal a divergent generalization to unseen languages during distillation, with varying outcomes depending on the base model. By removing a primary source of safety degradation, nuanced `boundary' refusals, we mitigate or even reverse safety declines in student models, although reductions in reasoning performance (GSM8K) persist. Overall, our exploratory study highlights the challenges and potential of KD as a technique for multilingual safety alignment, offering a foundation for future research in this direction.

Figures & tables

Figure 1 : The five-stage pipeline for response-based knowledge distillation. First, multilingual jailbreak prompts are sourced from the Xsafety dataset. Second, the teacher model, o1-mini , generates safe refusal responses to these prompts. Third, the prompts and their corresponding refusals are paired to create the distillation dataset. Fourth, the student models— Meta-Llama-3-8B-Instruct , Gemma-2-2B-IT , and Qwen3-8B —are fine-tuned on this dataset using LoRA PEFT. Finally, the safety of the fine-tuned models is evaluated on the MultiJail benchmark, with responses graded by GPT-4o .

Dataset Name	# Prompts	Languages	Harm Categories	Role in Study
XSafety	$\sim$ 28,190	High: en , zh , es, fr, de, ja Med: ar , ru Low: bn , hi	14 safety scenarios (illegal activity, hate speech, malware, etc.)	Distillation
MultiJail	$\sim$ 3,150	High: en , zh , it, vi Med: ar , ko, th Low: bn , sw, jv	18 safety scenarios (hate speech, weapons, theft, etc.)	Evaluation

Table 1 : Overview of response-based distillation datasets. Xsafety [ 38 ] is the distillation dataset: OpenAI’s o1-mini generates safe refusal used in LoRA PEFT. MultiJail [ 8 ] is the evaluation dataset, used to gauge the student models’ baseline and final degree of safety. Xsafety contains 28,190 prompts spanning 14 safety categories. MultiJail contains 3,150 prompts spanning 18 safety scenarios. Overlapping languages include: en, zh, ar, bn.

Model	Version	Overall JSR % ( $\downarrow$ )	High (zh, it, vi) ( $\downarrow$ )	Medium (ar, ko, th) ( $\downarrow$ )	Low (bn, sw, jv) ( $\downarrow$ )
OpenAI o1-mini	Baseline	$3.1$	$2.6$	$2.8$	$4.1$
Meta-Llama-3 -8B-Instruct	Baseline	$12.5$	$8.6$	$10.2$	$20.2$
	Distilled (LoRA)	$13.9$	$9.2$	$13.8$	$20.4$
	$\Delta$ (pp; ${\color[rgb]{0,0.5898,0}\downarrow}$ better)	$\uparrow\,1.4$	$\uparrow\,0.6$	$\uparrow\,3.6$	$\uparrow\,0.2$
Gemma-2-2B-IT	Baseline	$5.0$	$2.9$	$5.6$	$7.2$
	Distilled (LoRA)	$21.6$	$16.9$	$24.2$	$25.3$

Table 2 : Jailbreak Success Rate (JSR, lower is better) on MultiJail . Arrows denote direction of change relative to baseline:

\uparrow

= worse (higher JSR),

\downarrow

= better (lower JSR). All students worsen after response-based distillation; Gemma-2-2B-IT shows the largest degradation (+16.6 pp overall). Columns show categorical rates of overall JSR, high-resource language JSR, medium-resource language JSR, and low-resource language JSR.

Model	Baseline	Fine-Tuned
Meta-Llama-3-8B-Instruct	70.43%	62.62% ( $\downarrow$ )
Gemma-2-2B-IT	60.58%	52.99% ( $\downarrow$ )
Qwen3-8B	92.12%	87.49% ( $\downarrow$ )

Table 3 : Performance on GSM8K [ 7 ] before and after fine-tuning (Percentage accuracy, Total_Correct / Total_Questions). Results indicate a consistent decline in reasoning performance across all models following fine-tuning. Inference on Llama and Qwen models used the following system prompt: “You are a helpful assistant. Solve the math problem step by step and provide only the final numerical answer.”

Model	Metric	Baseline %	Full KD %	Simple KD %
Meta-Llama-3-8B-Instruct	JSR $\downarrow$	12.5	13.9 ( $\uparrow$ )	5.9 ( $\downarrow$ )
	GSM8K $\uparrow$	70.43	62.62 ( $\downarrow$ )	61.49 ( $\downarrow$ )
Gemma-2-2B-IT	JSR $\downarrow$	5.0	21.6 ( $\uparrow$ )	11.96 ( $\uparrow$ )
	GSM8K $\uparrow$	60.58	52.99 ( $\downarrow$ )	55.34 ( $\downarrow$ )
Qwen3-8B	JSR $\downarrow$	5.7	8.3 ( $\uparrow$ )	4.03 ( $\downarrow$ )
	GSM8K $\uparrow$	92.12	87.49 ( $\downarrow$ )	87.72 ( $\downarrow$ )

Table 4 : Fine-tuning exclusively on simple refusals after data purification. Comparison of Base, Full KD, and Simple KD performance across JSR and GSM8K. Arrows indicate desired direction (

\downarrow

lower is better for JSR,

\uparrow

higher is better for GSM8K). Simple KD reversed JSR increases in Meta-Llama-3-8B-Instruct and Qwen3-8B . Reasoning degradations still persisted.

Model	Version	JSR (%)
Gemma-2-2B-IT	Baseline	5.0
	Distilled (LoRA PEFT)	21.6 ( $\uparrow$ )
	Distilled (FULL-SFT)	19.8 ( $\uparrow$ )

Table 5 : Impact of LoRA versus full-SFT. Example student Gemma-2-2B-IT under different training settings. Degradation holds when training on all parameters.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

	Safe	Unsafe
Human Evaluation	164	136
LLM GPT-4o	150	150

Appendix

Table 6 : Comparison of Human vs. LLM safety judgments. Human evaluation is closely consistent with LLM evaluation. We pulled a stratified sample of ten prompts from each language in MultiJail across our three fine-tuned student models.

Language	GPT-4o			Claude-haiku-4-5
Language	Safe	Unsafe	Invalid	Safe	Unsafe	Invalid
EN	305	9	0	307	8	0
ZH	284	24	6	283	31	1
IT	257	52	6	257	56	2
VI	263	33	19	271	41	3
AR	267	32	16	271	38	6

Appendix

Table 7 : Side-by-side comparison of GPT-4o and Claude-haiku-4-5 gradings across 3,150 Meta-Llama-3-8B-Instruct responses. 8 GPT-4o responses were removed for errors.

	Run 1	Run 2	Run 3	Run 4	Mean	Std. Dev.
Overall JSR (%)	$13.94$	$11.90$	$11.68$	$11.97$	12.73	1.05

Appendix

Table 8 : Overall JSR per individual run on fine-tuned Meta-Llama-3-8B-Instruct . Listed run’s p-value is 0.73 (Listed run score in Table 2 shows no statistical significance). Mean JSR of 12.73 has a standard deviation of 1.05%.

	Baseline	Run 1	Run 2	Run 3	Run 4	Mean	Std. Dev.
Overall “Safe” Rate (%)	$77.3$	$79.2$	$71.2$	$71.3$	$71.8$	73.4	3.90

Appendix

Table 9 : Overall “Safe” rate per individual run on fine-tuned Meta-Llama-3-8B-Instruct . Mean "Safe" rate of 73.4% has a standard deviation of 3.90%. The results indicate that each run exhibits inherent trade-offs—either an increase in JSR or a decrease in overall “Safe” rates (reflected as higher “Invalid” scores). This pattern suggests reduced coherence and signs of catastrophic forgetting when JSR decreases.

	Run 1	Run 2	Run 3	Run 4	Run 5	Mean	Std. Dev.
Overall JSR (%)	$21.63$	$18.83$	$19.52$	$19.08$	$19.62$	19.73	1.10

Appendix

Table 10 : Overall JSR per individual run on fine-tuned Gemma-2B-IT . P-value is less than 0.0001 (highly significant). Mean JSR of 19.73% is highly consistent with Gemma-2B-IT ’s listed run scores in Table 2 . Standard deviation is 1.10%.

	Run 1	Run 2	Run 3	Run 4	Run 5	Mean	Std. Dev.
Overall JSR (%)	$8.38$	$8.16$	$8.92$	$8.89$	$8.76$	8.62	0.33

Appendix

Table 11 : Overall JSR per individual run on fine-tuned Qwen3-8B . P-value is less than 0.0001 (highly significant). Mean JSR of 8.76 is highly consistent with Qwen3-8B ’s listed run scores in Table 2 . Standard deviation is 0.33%.

Figure 2 : Teacher model o1-mini ’s evaluation scores on MultiJail , showing the number of safe, unsafe, and invalid responses per language.

Panel of Figure 3 — Figure 3 : Baseline (left) and LoRA tuned (right) Meta-Llama-3-8B-Instruct evaluation scores on MultiJail , showing the number of safe, unsafe, and invalid responses per language.

Panel of Figure 4 — Figure 4 : Baseline (left) and LoRA tuned (right) Gemma-2-2B-IT evaluation scores on MultiJail , showing the number of safe, unsafe, and invalid responses per language.

Panel of Figure 5 — Figure 5 : Baseline (left) and LoRA tuned (right) Qwen3-8B evaluation scores on MultiJail , showing the number of safe, unsafe, and invalid responses per language.

Category	Count	Percentage
Boundary	9,341	33.1%
Neither	2,734	9.7%
Simple	16,115	57.2%
Total	28,190	100.0%

Appendix

Table 12 : Overall Classification Breakdown. Gemma-3-12B-IT grades o1-mini ’s 28,190 XSafety responses.

Panel of Figure 6 — Figure 6 : Baseline (left) and LoRA tuned (right) Llama-2-13b-chat-hf evaluation scores on MultiJail

Model Version	JSR (%)	Invalid (%)
Llama-2-13b-chat-hf (Baseline)	3.17	17.9
Llama-2-13b-chat-hf (Distilled)	11.6	22.5

Appendix

Table 13 : Llama-2-13b-chat-hf evaluation results. Distillation increased JSR and invalid rates.

Panel of Figure 7 — Figure 7 : Baseline (left) and LoRA tuned (right) Gemma-3-12B-IT evaluation scores on MultiJail .

Model Version	JSR (%)	Invalid (%)
Gemma-3-12B-IT (Baseline)	4.76	2.95
Gemma-3-12B-IT (Distilled)	11.6	0.444

Appendix

Table 14 : Gemma-3-12B-IT evaluation results. Distillation increased JSR and lowered invalid rates.

Panel of Figure 8 — Figure 8 : Baseline (left) and LoRA tuned (right) Qwen3-14B evaluation scores on MultiJail .

Model Version	JSR (%)	Invalid (%)
Qwen3-14B (Baseline)	6.86	9.68
Qwen3-14B (Distilled)	7.71	8.16

Appendix

Table 15 : Qwen3-14B ’s evaluation results. Distillation increased JSR and lowered invalid scores

Explore similar work

Multilingual Safety Alignment via Self-Distillation
May 3, 2026Ruiyang Qin, Qingzhuo Wang, Dongrui Liu +3Multilingual Safety Jailbreak Attacks
Tongji University · Shanghai AI Laboratory
Cross-Lingual Jailbreak Detection via Semantic Codebooks
Apr 28, 2026Shirin Alanova, Bogdan Minko, Sabrina Sadiekh +1Multilingual Safety Large Language Model Jailbreaks
AI Talent Hub, ITMO University · HiveTraceLab
One Jailbreak, Many Tongues: Learning Language-Insensitive Intention Representations for Multilingual Jailbreak Detection
Apr 22, 2026Shuyu Jiang, Kaiyu Xu, Xingshu Chen +5Large Language Model Jailbreaks Multilingual Safety
School of Cyber Science and Engineering, Sichuan University, Chengdu, 610065, China · School of Computer Science and Engineering, Nanyang Technological University · School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China

May 3, 2026, cs.LGJ/K move · Enter open · S save

Multilingual Safety Alignment via Self-Distillation

Ruiyang Qin, Qingzhuo Wang, Dongrui Liu, Qiang Li+2

Tongji University · Shanghai AI Laboratory

Multilingual Safety Jailbreak Attacks

Open paper AskPDF arXiv

Large language models (LLMs) exhibit severe multilingual safety misalignment: they possess strong safeguards in high-resource languages but remain highly vulnerable to jailbreak attacks in low-resource languages. Current safety alignment methods generally rely on high-quality response data for each target language, which is expensive and difficult to generate. In this paper, we propose a cross-lingual safeguard transfer framework named Multilingual Self-Distillation (MSD). This framework transfers an LLM's inherent safety capabilities from high-resource (e.g., English) to low-resource (e.g., Javanese) languages, overcoming the need for response data in any language. Our framework is flexible and can be integrated with different self-distillation strategies. Specifically, we implement two concrete methods -- on-policy MSD and off-policy MSD -- both of which enable effective cross-lingual safety transfer using only multilingual queries. Furthermore, we propose Dual-Perspective Safety Weighting (DPSW), a divergence measure to optimize the distillation objective. By jointly considering the perspectives of both the teacher and the student, DPSW adaptively increases the penalty weights on safety-critical tokens while reducing the weights on non-critical tokens. Extensive experiments on representative LLMs across diverse multilingual jailbreak and utility benchmarks demonstrate that our method consistently achieves superior multilingual safety performance. Notably, it generalizes effectively to more challenging datasets and unseen languages while preserving the model's general capabilities.

Abstract

Explore similar work

Multilingual Safety Alignment via Self-Distillation

Cross-Lingual Jailbreak Detection via Semantic Codebooks

One Jailbreak, Many Tongues: Learning Language-Insensitive Intention Representations for Multilingual Jailbreak Detection