The Power of Decaying Steps: Enhancing Attack Stability and Transferability for Sign-based Optimizers

Wei Tao; Yang Dai; Jincai Huang; Qing Tao

cs.LGFeb 22, 2026

The Power of Decaying Steps: Enhancing Attack Stability and Transferability for Sign-based Optimizers

Authors: Wei Tao, Yang Dai, Jincai Huang, Qing Tao

Organizations: Laboratory for Big Data and Decision, National University of Defense and Technology, China · Academy of Military Science, China · Hefei Institute of Technology, China

Abstract

Crafting adversarial examples can be formulated as an optimization problem. While sign-based optimizers such as I-FGSM and MI-FGSM have become the de facto standard for the induced optimization problems, there still exist several unsolved problems in theoretical grounding and practical reliability especially in non-convergence and instability, which inevitably influences their transferability. Contrary to the expectation, we observe that the attack success rate may degrade sharply when more number of iterations are conducted. In this paper, we address these issues from an optimization perspective. By reformulating the sign-based optimizer as a specific coordinate-wise gradient descent, we argue that one cause for non-convergence and instability is their non-decaying step-size scheduling. Based upon this viewpoint, we propose a series of new attack algorithms that enforce Monotonically Decreasing Coordinate-wise Step-sizes (MDCS) within sign-based optimizers. Typically, we further provide theoretical guarantees proving that MDCS-MI attains an optimal convergence rate of $O(1/\sqrt{T})$ , where $T$ is the number of iterations. Extensive experiments on image classification and cross-modal retrieval tasks demonstrate that our approach not only significantly improves transferability but also enhances attack stability compared to state-of-the-art sign-based methods.

Figures & tables

Figure 1: Stability comparison of typical sign-based transfer attacks on the NIPS2017 dataset. We employ Res50 as surrogate model and Inc-v3 as target model with $\epsilon=16/255$ .

Figure 2: Step-size dynamics of a randomly selected coordinate in typical sign-based attacks. $T$ is the maximum iteration. We reformulate $\text{sign}(\bm{g}_{t})$ as $\bm{g_{t,i}}/|\bm{g_{t,i}}|$ where $\bm{g}$ denotes the gradient or momentum and $i$ is the number of the randomly selected coordinate. So, the step-size is $\epsilon/(T|\bm{g_{t,i}}|)$ for coordinate-wise gradient.

Panel of Figure 3 — Figure 3: Stability comparison of transfer attacks between MDCS-MI and MI-FGSM. AEs are crafted for Res50 and ViT-B/16, respectively.

Type	Attack	Res50	VGG16	Mob-v2	Inc-v3	ViT-B	PiT-B	Vis-S
①	AutoAttack [ 4 ]	99.5	34.0	30.6	13.9	3.0	8.2	11.3
①	I-FGSM [ 13 ]	99.7	36.3	34.1	18.5	6.8	11.8	14.6
②	MI [ 7 ]	100.0	59.4	53.1	36.3	12.5	23.1	26.6
	MDCS-MI	100.0	67.2	60.3	41.3	13.4	23.3	30.8
	VMI [ 39 ]	99.8	69.2	66.3	56.8	31.0	46.9	54.5
	GRA [ 50 ]	97.3	85.2	84.3	81.3	44.6	62.2	72.9

Table 1: Transferability comparisons across different network architectures. AEs are crafted for Res50. The baseline attacks are divided into 3 categories: ① gradient-based attacks; ② momentum-based attacks; ③ input-transformation attacks. The best results are marked in bold. The gray area represents adversarial attacks under the white-box setting.

Attack	Inc-v3	Res34	Inc-v4	IncRes-v2
SVRE [ 43 ]	99.5	100.0	74.6	65.9
AdaEA [ 1 ]	99.6	100.0	72.4	64.7
MI	99.6	100.0	75.6	67.0
MDCS-MI	99.9	100.0	79.3	73.5
MEF	99.9	100.0	97.1	95.2
MDCS-MEF	100.0	100.0	97.9	96.6

Table 2: The success rates (%) of adversarial attacks against ensemble models.

Attack	Inc-v3 ens3	Inc-v3 ens4	IncRes-v2 ens	AT	JPEG	HGD	RS	Bit-Red
MI	22.0	22.5	14.9	40.7	50.2	17.4	27.5	38.9
MDCS-MI	23.7	24.2	16.3	41.1	52.2	16.2	27.8	42.6
MEF	73.6	71.6	66.3	45.4	86.2	74.5	37.4	81.2
MDCS-MEF	74.5	71.5	66.3	45.5	88.5	74.6	37.2	84.7
OPS	96.8	96.7	95.3	58.3	98.5	97.2	65.3	98.1
MDCS-OPS	97.5	97.5	96.0	58.5	98.9	98.0	66.6	99.0

Table 3: The success rates (%) of adversarial attacks against defensed models and adversarially trained models.

Figure 4: Ablation study on the value of

\epsilon

. We employ Res50 as surrogate model and Inc-v3 as target model.

Figure 5: Visualization of AEs on image classification tasks. AEs are crafted on ViT-B/16 with $\epsilon=16/255$ .

Source	Attack	ALBEF		TCL		CLIP ${}_{\text{ViT}}$		CLIP ${}_{\text{CNN}}$
Source	Attack	TR R@1	IR R@1	TR R@1	IR R@1	TR R@1	IR R@1	TR R@1	IR R@1
ALBEF	PGD	52.45	58.65	3.06	6.79	8.96	13.21	10.34	14.65
	Sep-Attack	65.69	73.95	17.60	32.95	31.17	45.21	32.82	45.49
	Co-Attack	97.08	98.36	39.52	51.24	29.82	38.92	31.29	41.99
	SGA	99.79	99.95	87.67	87.88	38.04	46.17	41.63	50.36
	MDCS-SGA	100.0	99.98	91.78	91.24	41.35	49.71	45.08	53.93

Table 4: The adversarial attacks against multimodal models on Flickr30K dataset. The source model column shows VLMs we use to generate multimodal AEs. The gray area represents adversarial attacks under a white-box setting; the rest are black-box attacks. R@1 denotes the attack success rate at the top-1 rank. The adversarial perturbation is set to 8/255, with a per-step size of 2/255, and a total of 10 iterations. For both Image-Text Retrieval (TR) and Text-Image Retrieval (IR), we provide R@1 scores here. More results are listed in the Supplementary Material ( Sec. 8.3 ).

Figure 6: Visualization of AEs on image-text retrieval tasks with MDCS-SAAET. AEs are crafted on ALBEF and are constrained by an $L_{\infty}$ norm of 8/255, generated over 10 iterations with a step size of 2/255.

Panel of Figure 7 — Figure 7: Stability comparison of SI and TI (based on MI-FGSM). We employ Res50 as surrogate model and Inc-v3 as target model with $\epsilon=16/255$ .

Figure 8: Dynamics of white-box and black-box losses. We employ Res50 as surrogate model and Vis-S as target model.

Figure 9: Ablation study on the value of

\gamma

for MDCS-MI. We employ Res50 as surrogate model and Inc-v3 as target model.

Figure 10: A comparison between responses from the clean image and the adversarial image generated by MDCS-MI against prismVLM. Image and questions source are from textVQA.

Model	Attack	Res50	VGG16	Mob-v2	Inc-v3	ViT-B	PiT-B	Vis-S
VGG16	I-FGSM	35.8	99.9	62.9	27.1	8.0	15.5	25.1
	MI	59.1	99.8	80.2	55.9	15.6	29.6	44.4
	MDCS-MI	59.5	99.8	81.0	59.9	17.5	30.3	45.9
	VMI	74.1	99.8	88.6	72.9	26.1	44.2	60.9
	EMI	74.6	100.0	90.5	69.8	21.5	38.9	57.9
	GRA	71.3	100.0	89.9	79.6	27.0	42.7	58.0

Table 5: Transferability comparisons across different networks. AEs are crafted for VGG16 and ViT-B/16. The best results are marked in bold. The gray area represents adversarial attacks under a white-box setting, the rest are black-box attacks.

Model	Type	Attack	Pre	Post N	ASR
LLaVA-1.5	Pure	FGSM	47.3	37.2	10.1
		PGD	47.3	22.1	25.2
		MEF	47.3	22.1	25.2
		MI	47.3	20.9	26.4
		MDCS-MI	47.3	20.9	26.4
	OCR	FGSM	58.5	53.7	4.8

Table 6: The white-box adversarial attacks of Pure and OCR VQA tasks in VLMs.

FLICKR30K (Image-Text Retrieval)
Source	Attack	ALBEF			TCL			CLIP ${}_{\text{ViT}}$			CLIP ${}_{\text{CNN}}$
Source	Attack	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10
ALBEF	Co-Attack	97.1	94.59	92.60	39.52	20.40	14.53	29.82	11.73	6.61	31.29	11.73	5.77
	SGA	99.79	99.80	99.80	87.67	77.09	70.54	38.04	19.11	13.11	41.63	21.56	14.62
	MDCS-SGA	100.0	100.0	100.0	91.78	82.11	76.75	41.35	22.64	15.55	45.08	25.90	17.71
	DRA	99.79	99.80	99.80	89.78	80.90	75.75	46.63	24.30	18.60	50.32	27.91	20.19

Table 7: Detailed comparison of attack success rates on Flickr30K dataset. The gray-shaded cells represent attacks in a white-box setting, while the remaining cells show results for black-box transfer attacks. We report the attack success rates (%) for cross modal retrieval tasks using R@1, R@5, and R@10 metrics. The adversarial perturbations are constrained by an

L_{\infty}

norm of 8/255, generated over 10 iterations with a step-size of 2/255.

MSCOCO (Image-Text Retrieval)
Source	Attack	ALBEF			TCL			CLIP ${}_{\text{ViT}}$			CLIP ${}_{\text{CNN}}$
Source	Attack	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10
ALBEF	Co-Attack	96.65	94.74	93.12	57.33	37.24	28.53	50.71	33.10	26.44	52.06	33.89	27.47
	SGA	99.95	99.70	99.65	87.22	76.39	69.34	64.06	46.76	39.02	63.34	47.67	39.72
	MDCS-SGA	100.0	99.96	99.84	88.23	77.94	70.73	64.94	49.08	40.44	66.12	49.63	41.83
	DRA	99.92	99.68	99.44	88.78	78.81	72.24	69.21	52.56	43.97	69.06	52.13	43.87

Table 8: Detailed comparison of attack success rates on MSCOCO dataset. We report the attack success rates (%) for both Image-Text Retrieval and Text-Image Retrieval tasks using R@1, R@5, and R@10 metrics. The adversarial perturbations are constrained by an

L_{\infty}

norm of 8/255, generated over 10 iterations with a step size of 2/255.

Adversarial Attacks

Adversarial attacks exploit crafted inputs, poisoned data, or system interactions to manipulate AI predictions, behavior, privacy, or reliability.

See all 447 papers

When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study
May 26, 2026Jun Yan, Weiquan Huang, Jiankai Zuo +4Adversarial Training Orthogonal Optimization
IT College, Shanghai Ocean University, Shanghai, 201306 · School of Computer Science and Technology, Tongji University, Shanghai, 201804 · SEIE, Suzhou University of Science and Technology, Suzhou, 215009 +2
SDM: A Powerful Tool for Evaluating Model Robustness
May 19, 2026Xinlei Liu, Tao Hu, Jichao Xie +3Adversarial Attacks Neural Network Robustness
Information Engineering University, Zhengzhou, China · Key Laboratory of Cyberspace Endogenous Safety & Security of Henan Province, Zhengzhou, China · Key Laboratory of Cyberspace Security Ministry of Education of China, Zhengzhou, China +1
StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models
Apr 16, 2026Dingzhi Yu, Rui Pan, Yuxing Liu +1Stochastic Optimization Sign-Based Optimization
University of Illinois Urbana-Champaign

May 26, 2026J/K move · Enter open · S save

When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study

Jun Yan, Weiquan Huang, Jiankai Zuo, Yujian Mo+3

IT College, Shanghai Ocean University, Shanghai, 201306 · School of Computer Science and Technology, Tongji University, Shanghai, 201804 · SEIE, Suzhou University of Science and Technology, Suzhou, 215009+2

Adversarial Training Orthogonal Optimization Neural Network Optimization Neural Network Robustness Adversarial Robustness Muon Optimizer

Open paper AskPDF arXiv

Adversarial training (AT) remains one of the most reliable empirical defenses against adversarial attacks. Its robustness critically depends on how the underlying min-max objective is optimized. In practice, Stochastic Gradient Descent (SGD) optimizer remains the default optimization choice for AT, whereas adaptive optimizers often improve standard training but may yield inferior robustness. Recently, the Muon optimizer, which orthogonalizes matrix-valued updates via an approximate polar decomposition, has achieved notable success in large-scale training at a memory cost comparable to SGD. This raises a security-relevant question: can orthogonalized optimization improve AT under strong and heterogeneous threat models? Focusing on this problem, we conduct a comprehensive theoretical and empirical study. Theoretically, we show that Muon imposes a spectral-norm stability ceiling on matrix updates, limiting uncontrolled spectral growth in the training dynamics without explicitly shrinking the learned weights. Empirically, across five architectures and three

\ell_p

threat models (

\ell_\infty

\ell_1

\ell_2

) and their union, Muon is competitive with SGD on CNNs and substantially outperforms AdamW on both CNNs and ViTs. These results identify optimizer geometry as a security-relevant factor in adversarial training, while clarifying the empirical regimes in which orthogonalized updates are beneficial. Overall, our findings highlight optimizer design as a security-critical component of AT.

Abstract

Adversarial Attacks

Explore similar work

When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study

SDM: A Powerful Tool for Evaluating Model Robustness

StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models