Robust CurveMoE: Multi-Norm Adversarial Defense for Mixture-of-Experts Models via Mode Connectivity

Xu Zhang; Ren Wang

cs.LGAug 26, 2026

Robust CurveMoE: Multi-Norm Adversarial Defense for Mixture-of-Experts Models via Mode Connectivity

Authors: Xu Zhang, Ren Wang

Organizations: Department of Electrical and Computer Engineering, Illinois Institute of Technology, Chicago, IL, USA

Abstract

Multi-norm adversarial defense aims to protect neural networks against perturbations defined by different norm constraints, but existing methods typically optimize competing robustness objectives within a single parameter configuration, leading to substantial training cost and unfavorable robustness trade-offs. We propose Robust CurveMoE, an efficient mixture-of-experts framework that connects models specialized for different perturbation norms through a low-loss path and exploits the complementary robustness profiles of models along this path. Robust CurveMoE derives clean and norm-specialized experts from robustness-constrained curve locations and selectively expertizes only influential layers, while sharing the remaining parameters across routing paths. To further reduce curve-construction cost, we introduce contribution-guided partial updating, which selects influential curve parameters using initialization-based gradient scores. We also theoretically bound the objective gap between partial and full curve optimization. Experiments on CIFAR-100 and ImageNet-100 with WideResNet and Vision Transformer architectures show that Robust CurveMoE consistently improves clean, norm-specific, and Union accuracy over MSD and ERMC. In particular, it improves Union accuracy by 2.37 and 2.13 percentage points over the strongest baseline on CIFAR-100 and ImageNet-100, respectively. Extensive ablations further validate the effectiveness of partial updating, selective expertization, and robustness-constrained expert selection.

Figures & tables

Fig. 1: Overview of Robust CurveMoE. Starting from $\ell_{1}$ -robust and $\ell_{\infty}$ -robust endpoint models, we first construct a robust connectivity curve using contribution-guided partial updating. We then sample representative models from the curve to form a robustness-constrained candidate set, from which clean and norm-specialized experts as well as a shared model are selected. Based on the sensitivity of different layers, only the most influential layers are converted into MoE layers, while the remaining layers are shared. Finally, the constructed MoE is fine-tuned by updating the expert and router parameters while keeping the shared backbone frozen.

Fig. 2: Clean and norm-specific adversarial accuracy along the robust connectivity curve of WideResNet-28-10 on CIFAR-100, evaluated using AutoAttack. Different curve locations exhibit distinct robustness preferences.

Fig. 3: Relative contribution of the candidate MoE layers in ViT-Tiny to clean and norm-specific adversarial objectives. The layer-wise contributions are highly non-uniform, and different objectives exhibit both shared and objective-dependent sensitivity patterns.

Updating Strategy	Ratio	Time Reduction (%)	Clean (%)	$\ell_{1}$ (%)	$\ell_{2}$ (%)	$\ell_{\infty}$ (%)
Full Updating	100%	$0$	$57.52\pm 2.99$	$18.78\pm 7.35$	$35.29\pm 2.12$	$16.78\pm 3.74$
Random	50%	$35.26\pm 0.57$	$56.32\pm 4.97$	$16.85\pm 9.57$	$32.83\pm 4.85$	$15.91\pm 4.21$
Random	30%	$36.56\pm 0.31$	$52.95\pm 7.92$	$17.93\pm 8.49$	$30.75\pm 6.78$	$12.39\pm 7.43$
Contribution-Guided	50%	$35.14\pm 0.70$	$57.13\pm 3.41$	$18.54\pm 7.45$	$34.98\pm 2.20$	$16.58\pm 3.86$
Contribution-Guided	30%	$36.05\pm 0.82$	$56.82\pm 3.73$	$18.33\pm 7.56$	$34.67\pm 2.36$	$16.41\pm 3.96$

TABLE I: Efficiency and robustness of curve updating strategies.

Dataset	Method	Clean (%)	$\ell_{1}$ (%)	$\ell_{2}$ (%)	$\ell_{\infty}$ (%)	Union (%)
CIFAR-100	ERMC	$55.65\pm 0.31$	$17.42\pm 0.95$	$35.90\pm 0.34$	$18.32\pm 0.43$	$15.86\pm 0.38$
	MSD	$54.72\pm 0.76$	$24.85\pm 0.79$	$33.16\pm 0.19$	$17.63\pm 0.49$	$15.21\pm 0.45$
	Robust CurveMoE	$58.34\pm 0.64$	$28.40\pm 0.70$	$36.48\pm 0.75$	$19.71\pm 0.27$	$18.23\pm 0.67$
ImageNet-100	ERMC	$67.22\pm 0.65$	$42.18\pm 0.16$	$50.68\pm 0.12$	$40.46\pm 0.50$	$37.52\pm 0.96$
	MSD	$69.90\pm 0.34$	$62.10\pm 0.58$	$49.80\pm 0.23$	$38.20\pm 0.75$	$35.84\pm 0.26$
	Robust CurveMoE	$73.58\pm 0.50$	$67.62\pm 0.69$	$54.34\pm 0.89$	$41.96\pm 0.95$	$39.65\pm 0.54$

TABLE II: Overall clean and multi-norm adversarial robustness.

Dataset	$\tau_{\mathrm{cand}}$	Clean (%)	$\ell_{1}$ (%)	$\ell_{2}$ (%)	$\ell_{\infty}$ (%)	Targeted Expert (%)
CIFAR-100	$2$	$58.34\pm 0.64$	$28.40\pm 0.70$	$36.48\pm 0.75$	$19.71\pm 0.27$	$14.58\pm 0.15$
	$5$	$61.28\pm 0.33$	$31.15\pm 0.31$	$36.23\pm 0.65$	$21.45\pm 0.83$	$9.96\pm 0.78$
	$10$	$62.73\pm 0.62$	$31.65\pm 0.61$	$36.68\pm 0.74$	$22.29\pm 0.91$	$9.74\pm 0.82$
ImageNet-100	$2$	$73.58\pm 0.50$	$67.62\pm 0.69$	$54.34\pm 0.89$	$41.96\pm 0.95$	$38.18\pm 0.19$
	$5$	$73.86\pm 0.14$	$68.14\pm 0.24$	$54.53\pm 0.97$	$42.85\pm 0.75$	$31.79\pm 0.56$
	$10$	$73.84\pm 0.25$	$69.29\pm 0.35$	$54.91\pm 0.72$	$47.53\pm 0.38$	$15.67\pm 0.59$

TABLE III: Effect of candidate tolerance on expert robustness.

Fig. 4: Effect of the partial updating ratio on clean and norm-specific adversarial accuracy along the robust connectivity curve.

Dataset	$K$	Training Time (h)	Clean (%)	$\ell_{1}$ (%)	$\ell_{2}$ (%)	$\ell_{\infty}$ (%)	Union (%)
CIFAR-100	$3$	$14.60\pm 0.90$	$58.34\pm 0.64$	$28.40\pm 0.70$	$36.48\pm 0.75$	$19.71\pm 0.27$	$18.23\pm 0.67$
	$5$	$20.97\pm 0.27$	$62.10\pm 0.96$	$31.98\pm 0.59$	$40.31\pm 0.91$	$21.81\pm 0.26$	$19.45\pm 0.86$
	$7$	$35.46\pm 0.95$	$62.85\pm 0.62$	$31.35\pm 0.51$	$40.40\pm 0.76$	$21.23\pm 0.12$	$19.18\pm 0.24$
ImageNet-100	$3$	$15.53\pm 0.81$	$73.58\pm 0.50$	$67.62\pm 0.69$	$54.34\pm 0.89$	$41.96\pm 0.95$	$39.65\pm 0.54$
	$5$	$25.58\pm 0.12$	$77.17\pm 0.49$	$71.90\pm 0.48$	$57.71\pm 0.90$	$45.35\pm 0.11$	$42.93\pm 0.24$
	$7$	$39.13\pm 0.63$	$77.40\pm 0.96$	$71.13\pm 0.94$	$57.95\pm 0.57$	$45.59\pm 0.23$	$42.35\pm 0.82$

TABLE IV: Effect of the number of MoE layers on model performance and training cost.

Dataset	Warm-up	Load Balancing	Clean (%)	$\ell_{1}$ (%)	$\ell_{2}$ (%)	$\ell_{\infty}$ (%)	Union (%)
CIFAR-100	Yes	No	$58.28\pm 0.45$	$28.12\pm 0.55$	$36.14\pm 0.30$	$19.55\pm 0.74$	$18.90\pm 0.68$
	No	Yes	$58.18\pm 0.37$	$28.62\pm 0.78$	$36.81\pm 0.93$	$19.77\pm 0.49$	$18.43\pm 0.45$
	No	No	$58.34\pm 0.64$	$28.40\pm 0.70$	$36.48\pm 0.75$	$19.71\pm 0.27$	$18.23\pm 0.67$
ImageNet-100	Yes	No	$73.28\pm 0.15$	$67.38\pm 0.43$	$54.16\pm 0.17$	$41.16\pm 0.64$	$39.73\pm 0.65$
	No	Yes	$73.30\pm 0.51$	$67.51\pm 0.82$	$54.79\pm 0.64$	$41.37\pm 0.81$	$39.53\pm 0.35$
	No	No	$73.58\pm 0.50$	$67.62\pm 0.69$	$54.34\pm 0.89$	$41.96\pm 0.95$	$39.65\pm 0.54$

TABLE V: Effect of warm-up and load balancing on model performance.

Adversarial Robustness

Adversarial robustness studies how well AI systems withstand deliberately crafted inputs, training-data corruption, and other attacks.

See all 484 papers

RoME: Robust Mixture of Low-Rank Experts against Multiple Adversarial Perturbations
Jul 7, 2026Woo Jae Kim, Kyle Min, Suhyeon Ha +2Adversarial Training Neural Network Robustness
KAIST, Daejeon, Korea · Oracle, Seattle WA, USA
Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation
Jul 22, 2026Luca Scionis, Luca Melis, Maura Pintor +5Adversarial Attacks Neural Network Robustness
Sapienza University of Rome, Italy · University of Cagliari, Italy · University of Genoa, Italy
Routing-Aware Safety Alignment for Mixture-of-Experts Models
Feb 4, 2026Jiacheng Liang, Yuhui Wang, Tanqiu Jiang +1Mixture-of-Experts Language Models LLM Alignment
Department of Computer Science Stony Brook University

Jul 7, 2026J/K move · Enter open · S save

RoME: Robust Mixture of Low-Rank Experts against Multiple Adversarial Perturbations

Woo Jae Kim, Kyle Min, Suhyeon Ha, Joonsung Jeon+1

KAIST, Daejeon, Korea · Oracle, Seattle WA, USA

Adversarial Training Neural Network Robustness Adversarial Robustness Mixture-of-Experts Models

Open paper AskPDF arXiv

Multi-perturbation adversarial training (MAT) aims to achieve robustness against multiple

\ell_p

perturbations but suffers from robustness trade-offs between different threats. To address this, we employ a mixture of experts (MoE) to route different threats through distinct model pathways. However, naive application of MoE encounters two critical challenges: experts tend to overlook threat-specific features and redundantly capture features shared across threats, and gating networks suffer from threat-agnostic routing where they learn nearly identical routing patterns across threats, thus preventing the construction of threat-specific model pathways. To this end, we propose Robust Mixture of Low-Rank Experts (RoME), where each expert is a low-rank additive update to the shared backbone, allowing it to capture threat-common features while experts focus on threat-specific information. To address threat-agnostic routing, RoME introduces (i) dual-scale gating that exploits threat-discriminative signals from local and global level features, and (ii) threat-guided gating diversification that enforces diverse expert utilization across threats. Extensive experiments demonstrate that RoME outperforms existing state-of-the-art MAT in union robustness and natural accuracy and improves robustness against unseen threats. Codes are available at https://github.com/wkim97/RoME.

Abstract

Adversarial Robustness

Explore similar work

RoME: Robust Mixture of Low-Rank Experts against Multiple Adversarial Perturbations

Adversarial Frontiers: Minimum-Norm Attack Ensembles for Robustness Evaluation

Routing-Aware Safety Alignment for Mixture-of-Experts Models