TraceRouter++: Causal Semantic Routing Realignment via Optimal Transport

Chuancheng Shi; Shangze Li; Wenhua Wu; Hengzhi Chen; Fei Shen; Li Zhuo; Zhiyong Wang; Tat-Seng Chua

cs.CLNov 3, 2026

TraceRouter++: Causal Semantic Routing Realignment via Optimal Transport

Authors: Chuancheng Shi, Shangze Li, Wenhua Wu, Hengzhi Chen, Fei Shen, Li Zhuo, Zhiyong Wang, Tat-Seng Chua

Organizations: The University of Sydney · Nanjing University of Science and Technology · National University of Singapore · Beijing University of Technology

Abstract

Despite their remarkable capabilities, large foundation models (LFMs) remain susceptible to adversarial manipulation. Current safety intervention methods primarily rely on the locality hypothesis, rigidly suppressing isolated neurons or features. However, harmful semantics manifest as distributed, cross-layer circuits, rendering localized ablations both brittle against complex jailbreaks and detrimental to general utility. To bridge this gap, we propose TraceRouter++, a routing-level safety intervention framework that explicitly traces safety-relevant causal routing pathways activated under harmful inputs and realigns their unsafe activation states. Our framework operates in a Discover-Trace-Realign paradigm: it first pinpoints a sensitive onset layer via attention divergence and disentangles dense activations into interpretable routing features using sparse autoencoders (SAEs). Subsequently, it maps these features to downstream causal circuits via feature influence scores (FIS). Crucially, departing from rigid suppression paradigms, TraceRouter++ introduces a constrained optimal transport (OT)-based soft routing realignment. By formulating safety intervention as a tractable coordinate-wise marginal optimal transport approximation, our method redirects unsafe activation marginals toward safe-reference marginals within causally identified routing subspaces while preserving non-selected coordinates at the intervention point. This mechanism safely redirects malicious semantic flows while leaving orthogonal computational routes intact. Extensive experiments across representative large language models (LLMs), multimodal large language models (MLLMs), and diffusion models (DMs) show that TraceRouter++ outperforms evaluated baselines, improving adversarial robustness while maintaining a favorable safety-utility trade-off. WARNING: This paper contains unsafe model responses.

Figures & tables

Fig. 1: Illustration of neuron-level and path-level safety intervention. Given an illustrative unsafe prompt, (a) neuron-level intervention suppresses isolated unsafe activations, which may disrupt benign neurons while allowing harmful information to recover through alternative propagation routes. In contrast, (b) path-level intervention identifies safety-relevant cross-layer routes that respond to harmful inputs and realigns their unsafe activation states, thereby reducing harmful outputs while limiting interference with benign computations.

Fig. 2: Overview of the TraceRouter++ framework. The pipeline executes three sequential stages: discovery, tracing, and realignment. It first isolates the sensitive onset layer and disentangles dense activations into interpretable features using a Top- $K$ sparse autoencoder [ 15 ] . Subsequently, it reconstructs causal semantic pathways via feature influence scoring. Finally, optimal transport-driven soft realignment maps malicious distributions toward safe reference manifolds, neutralizing harmful semantics while preserving orthogonal generative routes and general utility.

Fig. 3: Identification of the sensitive onset layer. We localize this critical layer by detecting the first local peak of the sensitivity score $SS(l)$ across the model depth. It functions as a representational bottleneck capturing the precise stage where malicious semantics initially diverge from benign representations. Consequently, this focal point serves as the topological origin for subsequent sparse feature disentanglement and optimal transport-driven soft realignment.

Item	LLMs	MLLMs	DMs
Models	LLaMA3-8B-Instruct [ 27 ] Mistral-7B-Instruct [ 35 ]	LLaVA-1.5-7B [ 42 ] MiniGPT-4-7B [ 91 ] DeepSeek-VL2 [ 80 ]	Stable Diffusion 1.4 [ 59 ] FLUX.1-Dev [ 38 ] AltDiffusion [ 86 ] Show-o2 [ 84 ]
Datasets	AdvBench [ 95 ] , HarmBench [ 50 ] GCG [ 95 ] , AutoDAN [ 43 ] Template [ 81 ] , Prefill [ 81 ] XSTEST [ 60 ] Global-MMLU-Lite [ 68 ]	FigStep [ 25 ] JailbreakV-28K [ 47 ] MMBench [ 45 ]	I2P-N [ 62 ] , I2P-V [ 62 ] P4D [ 12 ] , Ring-A-Bell [ 73 ] MS COCO [ 41 ]
Metrics	DSR [ 22 ] ORR [ 14 ] Accuracy [ 69 ]	DSR [ 22 ] Accuracy [ 69 ]	DSR [ 22 ] CLIP Score [ 31 ] FID [ 32 ]

TABLE I: Overview of models, datasets, and metrics across foundational architectures. Underlines indicate utility or over-refusal evaluations.

Model	Method	Gradient Driven		Pattern Injections			Avg. $\uparrow$
Model	Method	GCG [ 95 ] $\uparrow$	AutoDAN [ 43 ] $\uparrow$	Template $\uparrow$	Prefill $\uparrow$	T+P $\uparrow$	Avg. $\uparrow$
LLaMA3-8B-Instruct [ 27 ]	Original [ 27 ]	88	87	98	41	2	63.2
	DeepAug [ 55 ]	99	40	100	59	3	60.2 (-3.0)
	CB [ 94 ]	99	98	97	95	98	97.4 (+34.2)
	DeRTa [ 87 ]	97	89	100	98	32	83.2 (+20.0)
	HumorReject [ 81 ]	98	99	99	100	98	98.8 (+35.6)

TABLE II: Quantitative evaluation of safety interventions on LLMs. We benchmark TraceRouter++ against diverse adversarial jailbreaks, spanning both gradient-driven optimizations and structural pattern injections. Values denote the defense success rate (DSR) in percentages, with optimal outcomes bolded .

Model	Method	Noise			SD			Nature			Blank			Avg. $\uparrow$
Model	Method	T $\uparrow$	P $\uparrow$	L $\uparrow$	T $\uparrow$	P $\uparrow$	L $\uparrow$	T $\uparrow$	P $\uparrow$	L $\uparrow$	T $\uparrow$	P $\uparrow$	L $\uparrow$	Avg. $\uparrow$
LLaVA-1.5-7B [ 42 ]	Original [ 42 ]	42.9	70.8	37.9	39.5	60.9	27.1	41.0	68.2	40.6	42.6	69.1	39.2	48.3
	FigStep [ 25 ]	40.5	47.7	59.5	42.9	45.1	50.0	41.6	41.6	55.5	39.2	48.9	59.5	47.7 (-0.6)
	CoCA [ 21 ]	38.8	60.9	37.9	38.7	58.8	47.3	36.9	64.8	44.6	39.0	62.7	47.3	48.7 (+0.4)
	ECSO [ 26 ]	42.7	74.6	41.9	42.7	74.6	41.9	42.7	74.6	41.9	42.68	74.6	41.9	53.1 (+4.8)
	AdaShield [ 76 ]	78.4	98.6	82.5	75.4	98.6	77.1	76.8	99.2	82.5	78.2	98.6	82.5	85.7 (+37.4)

TABLE III: Quantitative evaluation of MLLMs safety on JailbreakV-28K dataset [ 47 ] . We assess defense robustness against textual jailbreaks across diverse visual contexts and prompt structures. T, P, and L denote template-based, persuasion-based, and logic-based jailbreak, respectively. Values represent the defense success rate (DSR) in percentages, with optimal performance bolded .

Model	Method	Illegal	Hate	Malware	Physical	Fraud	Adult	Privacy	Legal	Financial	Health	Avg $\uparrow$
Model	Method	Activity	Speech	Generation	Harm	Fraud	Content	Violation	Opinion	Advice	Consultation	Avg $\uparrow$
LLaVA-1.5-7B [ 42 ]	Original [ 42 ]	8.0	52.0	10.0	6.0	16.0	72.0	34.0	100.0	100.0	92.0	49.0
	FigStep [ 25 ]	44.0	50.0	46.0	38.0	16.0	74.0	46.0	100.0	100.0	94.0	60.8 (+11.8)
	CoCA [ 21 ]	56.0	91.8	62.0	77.9	93.5	57.5	90.4	55.9	58.1	69.6	71.4 (+22.4)
	ECSO [ 26 ]	80.0	88.0	18.0	58.0	40.0	84.0	50.0	98.0	98.0	96.0	71.0 (+22.0)
	AdaShield [ 76 ]	96.0	84.0	84.0	92.0	52.0	92.0	70.0	100.0	100.0	100.0	87.0 (+38.0)

TABLE IV: Quantitative comparison of MLLMs safety intervention on FigStep dataset [ 25 ] . This benchmark emphasizes image-guided jailbreak attacks across ten distinct safety categories. All reported values denote the defense success rate, and the most effective outcomes are highlighted in bold .

Method	Standard Safety (%)		Adversarial Robustness (%)
Method	I2P (N) [ 62 ] $\uparrow$	I2P (V) [ 62 ] $\uparrow$	P4D [ 12 ] $\uparrow$	Ring-A-Bell [ 73 ] $\uparrow$
Stable Diffusion 1.4 [ 59 ]	82.2	59.9	1.3	16.9
ESD [ 19 ]	86.0 ( +3.8)	83.3 ( +23.4)	36.7 ( +35.4)	30.3 ( +13.4)
UCE [ 20 ]	89.7 ( +7.5)	76.7 ( +16.8)	19.8 ( +18.5)	66.9 ( +50.0)
CA [ 37 ]	89.8 ( +7.6)	N/A	N/A	N/A
SLD-Med [ 62 ]	88.5 ( +6.3)	80.3 ( +20.4)	22.5 ( +21.2)	33.8 ( +16.9)

TABLE V: Safety intervention efficacy on DMs. Values denote defense success rates (DSR) across standard and adversarial benchmarks, with optimal results bolded.

Fig. 4: Qualitative comparison of LLMs safety interventions. Under textual jailbreak prompts, unaligned baselines frequently execute malicious instructions despite recognizing inherent risks. In contrast, TraceRouter++ consistently delivers secure refusals, successfully mitigating harmful semantic propagation.

Fig. 5: Qualitative comparison of MLLMs safety interventions. Under complex jailbreak attempts utilizing visual intent embedding (FigStep [ 25 ] , top row) and textual adversarial prompting (JailbreakV-28K [ 47 ] , bottom row), unaligned baselines routinely comply with the injected malicious intent. In contrast, TraceRouter++ consistently delivers secure refusals, successfully mitigating harmful semantic propagation across diverse input modalities.

Fig. 6: Qualitative comparison of diffusion model safety interventions. While baseline methods frequently induce unintended semantic drift and structural alterations, TraceRouter++ effectively neutralizes unsafe visual concepts. Through causal routing realignment, our framework strictly preserves the primary subject, spatial layout, and overall generative fidelity.

Modality	Model	Benchmark	Metric	Original	TraceRouter [ 67 ]	TraceRouter++
LLMs	LLaMA3-8B-Instruct [ 27 ]	Global-MMLU-Lite	Acc. $\uparrow$	71.5	70.7 ±0.4 (-0.8)	71.1 ±0.4 (-0.4)
LLMs	Mistral-7B-Instruct [ 35 ]	Global-MMLU-Lite	Acc. $\uparrow$	63.0	61.8 ±0.5 (-1.2)	62.4 ±0.3 (-0.6)
MLLMs	LLaVA-1.5-7B [ 42 ]	MMBench	Acc. $\uparrow$	47.5	47.2 ±0.1 (-0.3)	47.4 ±0.2 (-0.1)
	MiniGPT-4-7B [ 91 ]			16.0	15.2 ±0.1 (-0.8)	15.5 ±0.1 (-0.5)
	DeepSeek-VL2 [ 80 ]			78.7	78.3 ±0.2 (-0.4)	78.6 ±0.1 (-0.1)
DMs	Stable Diffusion 1.4 [ 59 ]	MS COCO	CLIP Score $\uparrow$	31.34	30.75 ±0.14 (-0.59)	31.08 ±0.07 (-0.26)

TABLE VI: Utility preservation across LFMs. Comparative results confirm that TraceRouter++ maintains original capabilities with negligible performance degradation across modalities.

Model	Method	ORR % $\downarrow$
LLaMA3-8B-Instruct [ 27 ]	Original [ 27 ]	3.2
	DeepAug [ 55 ]	39.6 (+36.4)
	CB [ 94 ]	4.4 (+1.2)
	DeRTa [ 87 ]	27.6 (+24.4)
	HumorReject [ 81 ]	5.2 (+2.0)
	LEACE [ 7 ]	3.6 (+0.4)

TABLE VII: Over-refusal analysis on XSTEST [ 60 ] . Values denote over-refusal rate (ORR) [ 14 ] percentages across LLMs. Lower scores indicate superior utility preservation.

Fig. 7: Visualization of pixel-level discrepancies on DMs. Baseline methods exhibit widespread residual patterns, indicating unintended global modifications. In contrast, TraceRouter++ produces strictly localized changes. These results indicate that optimal transport-driven realignment may facilitate the separation of malicious semantics with limited impact on benign visual content.

Model	Original	Amplification	Suppression
LLaMA3-8B-Instruct [ 27 ]	63.2	75.8 ±1.7 (+12.6)	61.4 ±1.4 (-1.8)
LLaVA-1.5-7B [ 42 ]	49.0	78.6 ±0.7 (+29.6)	18.2 ±1.9 (-30.8)
Stable Diffusion 1.4 [ 59 ]	82.2	73.5 ±1.3 (-8.7)	98.8 ±0.4 (+16.6)

TABLE VIII: Causal validation via routing manipulation. Values represent defense success rates under targeted pathway amplification and suppression. Safety gains stem from refusal amplification in language and multimodal architectures, while generative suppression provides optimal mitigation for diffusion models. Optimal results are bolded .

Method	LLaMA3-8B-Instruct [ 27 ]		LLaVA-1.5-7B [ 42 ]		Stable Diffusion 1.4 [ 59 ]
Method	DSR $\uparrow$	Global-MMLU-Lite [ 68 ] $\uparrow$	DSR $\uparrow$	MMBench [ 45 ] $\uparrow$	DSR $\uparrow$	CLIP Score [ 31 ] $\uparrow$	FID [ 32 ] $\downarrow$
Original	63.2	71.5	49.0	47.5	82.2	31.34	N/A
Hard Intervention [ 16 ]	74.2 (+11.0)	71.4 (-0.1)	77.4 (+28.4)	47.8 (+0.3)	96.3 (+14.1)	30.70 (-0.64)	19.12
Linear Steering [ 58 ]	97.4 (+34.2)	70.5 (-1.0)	95.7 (+46.7)	45.6 (-1.9)	72.2 (-10.0)	30.52 (-0.82)	16.24
TraceRouter++	$\textbf{99.5}_{\pm 0.3}$ (+36.3)	$71.1_{\pm 0.4}$ (-0.4)	$\textbf{99.4}_{\pm 0.1}$ (+50.4)	$47.4_{\pm 0.2}$ (-0.1)	$\textbf{99.3}_{\pm 0.2}$ (+17.1)	$\textbf{31.08}_{\pm 0.07}$ (-0.26)	$\textbf{8.62}_{\pm 0.04}$

TABLE IX: Comparison of linear mapping and optimal transport realignment. Results evaluate defense efficacy and utility preservation across LFMs. Optimal outcomes are bolded .

Fig. 8: Ablation study on intervention granularity. We compare isolated neuron suppression against continuous path realignment (TraceRouter++) across language (LLaMA3-8B-Instruct [ 27 ] ), multimodal (LLaVA-1.5-7B [ 42 ] ), and diffusion (Stable Diffusion 1.4 [ 59 ] ) architectures. Evaluations spanning diverse adversarial vectors, including GCG [ 95 ] , AutoDAN [ 43 ] , FigStep [ 25 ] , I2P [ 62 ] , P4D [ 12 ] , and Ring A Bell [ 73 ] , confirm that holistic routing control consistently outperforms localized node interventions.

Selection Criterion	LLaMA3-8B-Instruct [ 27 ]	LLaVA-1.5-7B [ 42 ]	Stable Diffusion 1.4 [ 59 ]
Original	63.2	49.0	82.2
Global Maximum Peak	$84.8_{\pm 0.3}$ (+21.6)	$78.4_{\pm 0.3}$ (+29.4)	$98.3_{\pm 0.2}$ (+16.1)
Fixed Threshold Selection	$80.8_{\pm 0.3}$ (+17.6)	$74.4_{\pm 0.3}$ (+25.4)	$96.5_{\pm 0.2}$ (+14.3)
First Local Peak (Ours)	$\textbf{99.5}_{\pm 0.3}$ (+36.3)	$\textbf{99.4}_{\pm 0.1}$ (+50.4)	$\textbf{99.3}_{\pm 0.2}$ (+17.1)

TABLE X: Ablation of sensitive onset layer selection criteria. Values denote defense success rates across LFMs. The first local peak strategy outperforms the global maximum and fixed-threshold baselines, validating the priority of early-stage semantic intervention.

Fig. 9: Selection of hyperparameter

K

via Wasserstein feature scoring [ 5 ] . The response distribution exhibits a distinct structural elbow pattern across foundational architectures. A sparse set of hub features demonstrates prominently higher activation scores than the long tail, confirming the precise boundary for optimal routing selection.

Top- $K$	LLaMA3-8B-Instruct [ 27 ]		LLaVA-1.5-7B [ 42 ]		Stable Diffusion 1.4 [ 59 ]
Top- $K$	DSR $\uparrow$	Global-MMLU-Lite [ 68 ] $\uparrow$	DSR $\uparrow$	MMBench [ 45 ] $\uparrow$	DSR $\uparrow$	CLIP Score [ 31 ] $\uparrow$	FID [ 32 ] $\downarrow$
Original	63.2	71.5	49.0	47.5	82.2	31.34	N/A
$K/2$	88.6 ±0.5 (+25.4)	71.4 ±0.3 (-0.1)	81.6 ±0.6 (+32.6)	47.4 ±0.1 (-0.1)	96.3 ±0.4 (+14.1)	31.30 ±0.04 (-0.04)	5.74 ±0.05
$K$	99.5 ±0.3 (+36.3)	71.1 ±0.4 (-0.4)	99.4 ±0.1 (+50.4)	47.4 ±0.2 (-0.1)	99.3 ±0.2 (+17.1)	31.08 ±0.07 (-0.26)	8.62 ±0.04
$2K$	99.2 ±0.3 (+36.0)	70.9 ±0.3 (-0.6)	95.8 ±0.4 (+46.8)	46.9 ±0.1 (-0.6)	97.9 ±0.3 (+15.7)	31.18 ±0.05 (-0.16)	12.92 ±0.05

TABLE XI: Sensitivity of routing feature selection. Varying the feature cardinality

K

confirms that the default elbow value optimizes the balance between safety robustness and general utility across LFMs.

Fig. 10: Specificity verification of causal routing interventions. We compare the defense success rate of TraceRouter++ against a random control baseline across foundational architectures. Applying optimal transport realignment to causally irrelevant pathways yields negligible safety improvements, strictly confirming the necessity of accurate semantic route identification.

Method	LLaMA3-8B-Instruct [ 27 ]		LLaVA-1.5-7B [ 42 ]		Stable Diffusion 1.4 [ 59 ]
Method	DSR $\uparrow$	Global-MMLU-Lite [ 68 ] $\uparrow$	DSR $\uparrow$	MMBench [ 45 ] $\uparrow$	DSR $\uparrow$	CLIP Score [ 31 ] $\uparrow$	FID [ 32 ] $\downarrow$
Original	63.2	71.5	49.0	47.5	82.2	31.34	N/A
$F\cdot I\cdot S$	99.5 ±0.3 (+36.3)	71.1 ±0.4 (-0.4)	99.4 ±0.1 (+50.4)	47.4 ±0.2 (-0.1)	99.3 ±0.2 (+17.1)	31.08 ±0.07 (-0.26)	8.62 ±0.04
$F\cdot I$	98.4 ±0.4 (+35.2)	71.4 ±0.4 (-0.1)	87.0 ±0.7 (+38.0)	47.1 ±0.1 (-0.4)	97.6 ±0.4 (+15.4)	31.22 ±0.04 (-0.12)	9.34 ±0.15
$F\cdot S$	95.6 ±0.3 (+32.4)	70.9 ±0.4 (-0.6)	86.8 ±0.8 (+37.8)	46.9 ±0.2 (-0.6)	98.3 ±0.3 (+16.1)	31.25 ±0.06 (-0.09)	8.79 ±0.11
$I\cdot S$	93.2 ±0.7 (+30.0)	70.9 ±0.6 (-0.6)	92.4 ±0.6 (+43.4)	45.9 ±0.1 (-1.6)	96.3 ±0.5 (+14.1)	31.20 ±0.07 (-0.14)	9.37 ±0.18

TABLE XII: Quantitative ablation of the sensitive path extraction metric. We compare our full routing discovery formulation against reduced variants utilizing isolated activation frequency (

F

), activation intensity (

I

), and zero-out causal shift (

S

). The empirical results confirm that the joint consideration of statistical and causal evidence strictly maximizes the defense success rate while preserving foundational utility across language, multimodal, and diffusion architectures. Missing baseline values are denoted as N/A. Best results are highlighted in bold .

Fig. 11: Topological visualization of causal semantic routing trajectories. By tracing connections from the sensitive onset layer via feature influence scoring, we reconstruct the precise propagation pathways of malicious semantics. The remarkably sparse and tree-structured topology explicitly distinguishes dominant causal routes from auxiliary connections across all foundational architectures.

Method	DSR [ 62 ] $\uparrow$	CLIP Score [ 31 ] $\uparrow$
FLUX.1 Dev [ 38 ]	70.4	26.4
+ SNCE [ 29 ]	80.2 (+9.8)	24.7 (-1.7)
+ TraceRouter++ (Ours)	93.5 (+23.1)	25.5 (-0.9)
AltDiffusion [ 86 ]	73.5	30.5
+ SNCE [ 29 ]	82.6 (+9.1)	29.7 (-0.8)
+ TraceRouter++ (Ours)	92.6 (+19.1)	30.2 (-0.3)

TABLE XIII: Universality across diverse generative architectures. Results evaluate defense efficacy and generation fidelity across multiple backbones. Optimal outcomes are bolded .

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Expansion	LLaMA3-8B-Instruct [ 27 ]		LLaVA-1.5-7B [ 42 ]		Stable Diffusion 1.4 [ 59 ]
Expansion	DSR $\uparrow$	Global-MMLU-Lite [ 68 ] $\uparrow$	DSR $\uparrow$	MMBench [ 45 ] $\uparrow$	DSR $\uparrow$	CLIP Score [ 31 ] $\uparrow$	FID [ 32 ] $\downarrow$
Original	63.2	71.5	49.0	47.5	82.2	31.34	N/A
$2\times$	88.4 ±0.5 (+25.2)	71.1 ±0.3 (-0.4)	77.2 ±0.7 (+28.2)	47.2 ±0.1 (-0.3)	88.9 ±0.4 (+6.7)	31.25 ±0.05 (-0.09)	11.49 ±0.14
$4\times$	99.5 ±0.3 (+36.3)	71.1 ±0.4 (-0.4)	99.4 ±0.1 (+50.4)	47.4 ±0.2 (-0.1)	99.3 ±0.2 (+17.1)	31.08 ±0.07 (-0.26)	8.62 ±0.04
$6\times$	91.4 ±0.6 (+28.2)	70.8 ±0.4 (-0.7)	82.0 ±0.5 (+33.0)	47.1 ±0.1 (-0.4)	89.8 ±0.4 (+7.6)	31.31 ±0.06 (-0.03)	10.51 ±0.13
$8\times$	93.8 ±0.5 (+30.6)	70.6 ±0.5 (-0.9)	76.4 ±0.8 (+27.4)	46.8 ±0.2 (-0.7)	88.1 ±0.5 (+5.9)	31.18 ±0.07 (-0.16)	13.07 ±0.18

Appendix

TABLE XIV: Quantitative ablation of sparse autoencoder expansion factors. We evaluate the defense success rate alongside modality-specific utility metrics under varying expansion capacities. Across all foundational architectures, a moderate

4\times

expansion consistently achieves the optimal Pareto frontier, effectively disentangling harmful semantics while maximizing general utility preservation. Missing baseline values are denoted as N/A. Best results are highlighted in bold .

$\lambda$	Gradient Driven DSR (%)		Pattern Injections DSR (%)			Avg. DSR (%) $\uparrow$	Global-MMLU-Lite [ 68 ] (%) $\uparrow$	ORR (%) $\downarrow$
$\lambda$	GCG [ 95 ] $\uparrow$	AutoDAN [ 43 ] $\uparrow$	Template $\uparrow$	Prefill $\uparrow$	T+P $\uparrow$	Avg. DSR (%) $\uparrow$	Global-MMLU-Lite [ 68 ] (%) $\uparrow$	ORR (%) $\downarrow$
$0$	88.0	87.0	98.0	41.0	2.0	63.2	71.50	3.2
$0.5$	$92.4_{\pm 1.1}$	$91.2_{\pm 0.8}$	$99.2_{\pm 0.8}$	$72.6_{\pm 2.1}$	$58.4_{\pm 2.1}$	$82.8_{\pm 1.4}$ (+19.6)	$\textbf{71.5}_{\pm 0.4}$ (0.00)	$4.0_{\pm 0.6}$ (+0.8)
$1$	$99.0_{\pm 0.7}$	$99.8_{\pm 0.4}$	$99.6_{\pm 0.5}$	$\textbf{99.8}_{\pm 0.4}$	$99.4_{\pm 0.5}$	$99.5_{\pm 0.3}$ (+36.3)	$71.1_{\pm 0.4}$ (-0.4)	$\textbf{3.5}_{\pm 0.3}$ (+0.3)
$2$	$99.4_{\pm 0.5}$	$99.6_{\pm 0.5}$	$100.0_{\pm 0.0}$	$99.2_{\pm 0.8}$	$99.8_{\pm 0.4}$	$99.6_{\pm 0.2}$ (+36.4)	$70.3_{\pm 0.6}$ (-1.2)	$7.6_{\pm 0.9}$ (+4.4)
$3$	$\textbf{99.8}_{\pm 0.4}$	$\textbf{100.0}_{\pm 0.0}$	$\textbf{100.0}_{\pm 0.0}$	$99.6_{\pm 0.5}$	$\textbf{100.0}_{\pm 0.0}$	$\textbf{99.9}_{\pm 0.2}$ (+36.7)	$68.5_{\pm 0.8}$ (-3.0)	$12.4_{\pm 1.2}$ (+9.2)

Appendix

TABLE XV: Sensitivity to OT intervention intensity on LLaMA3-8B-Instruct [ 27 ] . Intervened settings are reported as means

\pm

standard deviations over five seeds. Parenthetical values denote changes relative to

\lambda=0

Fig. 12: Sensitivity to safe-reference set size on LLaMA3-8B-Instruct [ 27 ] . Dashed lines denote the Original model baselines.

Method	Request latency (ms) $\downarrow$	Latency (ms/output token) $\downarrow$	Throughput (output tokens/s) $\uparrow$	Peak GPU memory (MiB) $\downarrow$	SAE training (s)	Pathfinding (s)	Total offline overhead (s)
Original	$874.25\pm 7.71$	$27.54\pm 0.24$	$36.32\pm 0.32$	$15{,}357.24$	$0$	$0$	$0$
TraceRouter++	$1{,}125.81\pm 3.68$ $(+28.8\%)$	$35.18\pm 0.12$ $(+27.8\%)$	$28.42\pm 0.09$ $(-21.7\%)$	$15{,}357.31$ $(+0.07)$	$7.25$	$0.09$	$7.34$

Appendix

TABLE XVI: Inference efficiency and computational overhead. Parenthetical values denote changes relative to the original model. Total offline overhead is the sum of SAE training and pathfinding.

Model	Method / Evaluation Setting	Avg. DSR (%) $\uparrow$
LLaMA3-8B-Instruct [ 27 ]	Original (adaptive GCG)	$58.4_{\pm 1.7}$
LLaMA3-8B-Instruct [ 27 ]	TraceRouter++ (adaptive GCG)	$\textbf{73.4}_{\pm 2.1}$
Mistral-7B-Instruct [ 35 ]	Original (adaptive GCG)	$4.8_{\pm 2.6}$
Mistral-7B-Instruct [ 35 ]	TraceRouter++ (adaptive GCG)	$\textbf{52.4}_{\pm 1.1}$

Appendix

TABLE XVII: Defense-aware adaptive GCG [ 95 ] evaluation on LLMs. Results report five-seed mean defense success rates (DSR)

\pm

sample standard deviations.

LLM Safety

Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.

See all 451 papers

May 1, 2026

RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs

Safety alignment is critical for the responsible deployment of large language models (LLMs). As Mixture-of-Experts (MoE) architectures are increasingly adopted to scale model capacity, understanding their safety robustness becomes essential. Existing adversarial attacks, however, have notable limitations. Prompt-based jailbreaks rely on heuristic search and transfer poorly, model intervention methods require privileged access to internal representations, and optimization-based input attacks remain output-centric and are fundamentally limited to MoE models due to the non-differentiable routing mechanism. In this paper, we present RouteHijack, a routing-aware jailbreak for MoE LLMs. Our key insight is that safety behavior is concentrated in a small subset of experts, creating an opportunity to steer model behavior by influencing routing decisions through input optimization. Building on this observation, RouteHijack first performs response-driven expert localization to identify safety-critical and harmful experts by contrasting activations under safe refusals and harmful completions. It then constructs adversarial suffixes with a routing-aware objective that suppresses safety experts, promotes harmful experts, and prevents early-stage refusal during generation. At inference time, the optimized suffix is appended to a malicious prompt, requiring only input access. Across seven MoE LLMs, RouteHijack achieves a 69.3% average attack success rate (ASR), outperforming prior optimization-based attack by

3.2\times

. RouteHijack also transfers zero-shot across five sibling MoE variants, raising average ASR from 27.7% to 61.2%, and further generalizes to three MoE-based VLMs, increasing average ASR from 2.47% to 38.7%. These findings expose a fundamental vulnerability in sparse expert architectures and highlight the need for defenses beyond output-level alignment.

Zhiyuan Xu, Joseph Gardiner, Sana Belguith +1

University of Bristol Bristol, United Kingdom

Mixture-of-Experts Language Models Adversarial Attacks on LLMs LLM Jailbreak Attacks Mixture-of-Experts Models

Ask PDF

Apr 30, 2026

Attention Is Where You Attack

Safety-aligned large language models rely on RLHF and instruction tuning to refuse harmful requests, yet the internal mechanisms implementing safety behavior remain poorly understood. We introduce the Attention Redistribution Attack (ARA), a white-box adversarial attack that identifies safety-critical attention heads and crafts nonsemantic adversarial tokens that redirect attention away from safety-relevant positions. Unlike prior jailbreak methods operating at the semantic or output-logit level, ARA targets the geometry of softmax attention on the probability simplex using Gumbel-softmax optimization over targeted heads. Across LLaMA-3-8B-Instruct, Mistral-7B-Instruct-v0.1, and Gemma-2-9B-it, ARA bypasses safety alignment with as few as 5 tokens and 500 optimization steps, achieving 36% ASR on Mistral-7B and 30% on LLaMA-3 against 200 HarmBench prompts, while Gemma-2 remains at 1%. Our principal mechanistic finding is a dissociation between ablation and redistribution: zeroing out the top-ranked safety heads produces at most 1 flip among 39 to 50 baseline refusals, while ARA targeting the corresponding safety-heavy layers flips 72/200 prompts on Mistral-7B and 60/200 on LLaMA-3. This suggests that safety is not localized in these heads as removable components, but emerges from the attention routing they perform. Removing a head allows compensation through the residual stream, while redirecting its attention propagates a corrupted signal downstream.

Aviral Srivastava, Sourav Panda

Amazon · The Pennsylvania State University

Self-Attention Adversarial Attacks Adversarial Attacks on LLMs LLM Safety Alignment Attention Mechanisms Attention-Based Adversarial Attacks+2

Ask PDF

Aug 2, 2026

Moving the Safety Barrier: Dynamic Routing Adaptive Alignment Against White-Box Attacks

With the widespread deployment of large foundation models (LFMs) in open environments, safety threats are shifting from black-box jailbreaks toward white-box attacks that directly identify and disrupt internal safety neurons or routes. However, existing safety defenses often rely on static safety units or fixed refusal pathways, leaving models highly vulnerable to targeted route-level white-box attacks. For that, we propose dynamic routing adaptive alignment (DRAA), a framework that introduces dynamic compensatory routes to preserve robust refusal behavior when the safety route is compromised. Specifically, we first identify and localize the model's safety route by contrasting internal activations between safe and unsafe calibration samples. DRAA then masks this safety route to induce causal failure cases and selectively mines the resulting defense failures, thereby constructing failure-aware preference pairs. Extensive experiments demonstrate that DRAA effectively restructures the underlying pathway dependence of model safety, substantially improving robustness against route-level white-box attacks, while preserving general utility.

Shangze Li, Chuancheng Shi, Simiao Xie +6

1Nanjing University of Science and Technology · 2The University of Sydney · 5NExT++ Research Centre, National University of Singapore +2

Adversarial Attacks on LLMs LLM Safety Alignment Adversarial Robustness of Language Models

Ask PDF

Abstract

LLM Safety

Explore similar work

RouteHijack: Routing-Aware Attack on Mixture-of-Experts LLMs

Attention Is Where You Attack

Moving the Safety Barrier: Dynamic Routing Adaptive Alignment Against White-Box Attacks