Devling into Adversarial Transferability on Image Classification: Review, Benchmark, and Evaluation

Xiaosen Wang; Zhijin Ge; Bohan Liu; Zheng Fang; Fengfan Zhou; Ruixuan Zhang; Shaokang Wang; Yuyang Luo

cs.CVFeb 26, 2026

Devling into Adversarial Transferability on Image Classification: Review, Benchmark, and Evaluation

Authors: Xiaosen Wang, Zhijin Ge, Bohan Liu, Zheng Fang, Fengfan Zhou, Ruixuan Zhang, Shaokang Wang, Yuyang Luo

Organizations: HUST · Xidian University · Wuhan University · HFUT ·

Abstract

Adversarial transferability refers to the capacity of adversarial examples generated on the surrogate model to deceive alternate, unexposed victim models. This property eliminates the need for direct access to the victim model during an attack, thereby raising considerable security concerns in practical applications and attracting substantial research attention recently. In this work, we discern a lack of a standardized framework and criteria for evaluating transfer-based attacks, leading to potentially biased assessments of existing approaches. To rectify this gap, we have conducted an exhaustive review of hundreds of related works, organizing various transfer-based attacks into six distinct categories. Subsequently, we propose a comprehensive framework designed to serve as a benchmark for evaluating these attacks. In addition, we delineate common strategies that enhance adversarial transferability and highlight prevalent issues that could lead to unfair comparisons. Finally, we provide a brief review of transfer-based attacks beyond image classification.

Figures & tables

Fig. 1: Categorization of existing transfer-based attacks. We systematically categorize existing transfer-based attacks into six distinct classes: 1) Gradient-based Attack optimizes the gradient calculation procedure to derive gradients that are more effective for the attack. 2) Input Transformation-based attack transforms the image prior to model to enhance the diversity of input image. 3) Advanced objective function replaces the conventional cross-entropy loss with a more complex objective function to enhance attack efficacy. 4) Model-related attack refines either the forward or backward propagation process, tailored to the architecture of the surrogate model employed. 5) Ensemble-based Attack adopts multiple surrogate models to generate adversarial examples. 6) Generation-based Attack trains a generator to directly crafts adversarial examples.

Terminologies	Description
Benign sample	The clean input sample that can be correctly classified by DNNs.
Adversarial example	The sample that is visually realistic to humans but misleads the DNNs. It is often crafted by adding imperceptible perturbation on the benign samples.
Perturbation	The noise that is added to the benign sample to generate an adversarial example.
Surrogate model	The white-box model that is adopted by transfer-based attack to generate adversarial examples.
Victim model	The unknown target model that is attacked by transfer-based attack.
White-box/Black-box Attack	White-box attacks have full access to victim model, while black-box attacks not. One of the key criterions is whether to adopt the gradient for the attack.

TABLE I: The terminologies used across the paper.

Attacks	Standardly trained models								Defenses					Average
Attacks	RN-50	VGG-16	MN-v2	Inc-v3	ViT	PiT	Visformer	Swin	AT	HGD	RS	NRP	DiffPure	Average
FGSM [ 24 ]	49.2	54.6	48.2	32.8	11.9	14.1	18.7	19.4	40.0	8.8	26.8	53.1	15.8	30.3
I-FGSM [ 61 ]	99.6	36.5	33.6	17.7	7.5	11.7	14.4	15.7	39.7	5.9	26.2	42.3	8.7	27.7
MI-FGSM [ 45 ]	99.9	57.9	53.4	37.4	14.5	22.5	26.2	28.1	40.6	17.9	27.4	58.5	13.3	38.3
NI-FGSM [ 84 ]	100.0	66.5	59.3	38.9	15.4	23.4	30.1	29.7	40.8	18.2	27.6	57.7	12.7	40.0
PI-FGSM [ 85 ]	98.8	74.9	59.2	57.9	16.3	16.9	25.3	20.6	42.1	35.3	35.0	60.1	38.3	44.7

TABLE II: Attack success rates of various gradient-based adversarial attacks where the adversarial examples are generated on RN50. We have highlighted the approaches that achieve the top-5 attack performance in gray .

Attacks	Standardly trained models								Defenses					Average
Attacks	RN-50	VGG-16	MN-v2	Inc-v3	ViT	PiT	Visformer	Swin	AT	HGD	RS	NRP	DiffPure	Average
DIM [ 46 ]	98.7	71.0	66.2	57.1	27.5	39.7	49.5	45.3	41.4	42.0	28.8	58.3	19.9	49.6
TIM [ 52 ]	97.8	57.9	46.9	38.9	15.3	16.5	23.2	19.0	41.7	25.4	32.5	56.6	32.3	38.8
SIM [ 84 ]	100.0	70.2	64.4	52.1	24.5	36.9	48.1	43.5	40.8	35.6	28.3	60.3	17.7	47.9
DEM [ 109 ]	99.9	93.1	89.3	91.0	48.2	63.5	78.2	71.4	45.5	74.8	36.0	59.3	35.6	68.1
Admix [ 62 ]	100.0	79.9	77.7	67.7	32.5	49.3	62.5	58.2	41.8	50.7	30.1	65.1	20.7	56.6

TABLE III: Attack success rates of various input transformation-based adversarial attacks where the adversarial examples are generated on RN50. We have highlighted the approaches that achieve the top-5 attack performance in gray .

Attacks	Standardly trained models								Defenses					Average
Attacks	RN-50	VGG-16	MN-v2	Inc-v3	ViT	PiT	Visformer	Swin	AT	HGD	RS	NRP	DiffPure	Average
TAP [ 63 ]	99.9	93.4	93.8	64.8	15.2	25.4	44.3	40.2	41.7	34.5	27.3	65.8	15.5	47.9
ILA [ 64 ]	98.7	68.7	64.6	33.4	12.8	23.5	31.9	32.1	40.0	16.2	27.1	52.5	11.3	37.7
ATA [ 65 ]	99.8	35.8	35.1	19.2	7.6	11.9	14.9	15.0	39.6	6.2	26.4	43.0	9.4	22.7
YAILA [ 125 ]	99.1	63.6	59.4	21.7	8.3	14.4	23.2	20.7	39.3	9.3	26.4	45.6	8.4	36.8
FIA [ 126 ]	98.0	71.2	65.8	40.2	12.6	19.9	33.2	33.1	41.1	19.2	28.0	58.4	12.1	38.6

TABLE IV: Attack success rates of various advanced objective functions where the adversarial examples are generated on RN50. We have highlighted the approaches that achieve the top-5 attack performance in gray .

Attacks	Standardly trained models								Defenses					Average
Attacks	RN-50	VGG-16	MN-v2	Inc-v3	ViT	PiT	Visformer	Swin	AT	HGD	RS	NRP	DiffPure	Average
CDTP [ 66 ]	97.1	99.2	98.4	95.9	38.3	44.6	91.6	68.7	42.1	94.8	31.3	70.8	32.5	69.64
LTP [ 67 ]	99.6	99.8	98.4	98.8	53.5	69.7	98.4	94.1	41.1	97.3	29.4	69.8	26.0	75.07
ADA [ 68 ]	71.7	89.4	80.6	75.9	18.1	20.4	50.9	39.5	38.4	1.4	27.9	32.2	26.3	44.05
GE-ADVGAN [ 140 ]	68.1	90.1	90.3	79.8	18.1	12.1	37.8	27.0	41.2	73.5	36.0	59.8	45.3	52.24
DiffAttack [ 141 ]	92.3	51.3	51.3	47.1	30.7	44.3	44.5	47.1	45.7	39.0	39.7	59.2	34.3	48.19

TABLE V: Attack success rates of various generation-based attacks. We have highlighted the approaches that achieve the top-1 attack performance in gray .

Attacks	Standardly trained models								Defenses					Average
Attacks	RN-50	VGG-16	MN-v2	Inc-v3	ViT	PiT	Visformer	Swin	AT	HGD	RS	NRP	DiffPure	Average
SGM [ 69 ]	100.0	73.2	75.7	45.9	18.9	33.5	41.1	41.9	41.3	25.3	28.6	64.3	15.0	46.5
LinBP [ 70 ]	100.0	87.7	81.2	47.5	18.9	22.7	44.0	37.2	41.3	19.9	27.9	59.9	17.1	46.6
LLTA [ 146 ]	94.9	96.6	95.1	83.8	45.7	57.1	74.1	73.8	47.7	74.3	39.5	82.5	28.2	68.7
IAA [ 71 ]	100.0	81.7	74.7	55.2	19.5	30.1	41.9	40.7	42.1	26.8	29.6	66.7	15.9	48.1
DRA [ 147 ]	94.3	96.8	97.8	95.0	75.3	77.7	88.7	86.4	70.8	93.2	85.7	86.4	75.3	86.4

TABLE VI: Attack success rates of various model-related attacks where the adversarial examples are generated on RN50/ViT. MTA, DSM, AGS, QAA and SETR adopt their trained models as the surrogate model. We have highlighted the approaches that achieve the top-5 attack performance in gray .

Attacks	Standardly trained models								Defenses					Average
Attacks	RN-50	VGG-16	MN-v2	Inc-v3	ViT	PiT	Visformer	Swin	AT	HGD	RS	NRP	DiffPure	Average
Base [ 44 ]	99.4	100.0	99.9	99.5	32.2	52.6	67.5	65.3	43.4	63.0	31.4	93.8	20.6	66.8
Ghost [ 72 ]	99.3	77.4	70.6	51.6	18.2	29.3	41.5	9.9	41.6	26.3	28.6	61.4	14.5	43.9
SVRE [ 170 ]	98.8	100.0	100.0	99.6	31.5	49.7	68.1	67.4	43.8	58.6	31.1	95.2	17.4	66.2
LGV [ 171 ]	91.4	97.2	95.8	80.6	28.0	35.8	58.8	55.9	44.6	58.2	32.7	67.2	18.0	58.8
MBA [ 172 ]	99.8	99.9	99.5	96.5	53.7	60.7	87.3	82.1	48.6	90.8	36.7	82.3	24.8	74.1

TABLE VII: Attack success rates of various ensemble attacks where the adversarial examples are generated on RN-50, VGG-16, MN-v2 and Inc-v3, simultaneously. We have highlighted the approaches that achieve the top-1 attack performance in gray .

Attacks	Standardly trained models								Defenses					Average
Attacks	RN-50	VGG-16	MN-v2	Inc-v3	ViT	PiT	Visformer	Swin	AT	HGD	RS	NRP	DiffPure	Average
ODI [ 176 ]	98.8	21.3	23.9	28.9	5.4	16.7	40.2	16.9	0.0	31.6	0.1	0.0	0.3	21.9
SU [ 177 ]	100.0	6.2	7.0	2.8	0.9	2.7	13.6	4.3	0.0	3.4	0.0	0.0	0.0	10.8
IDAA [ 178 ]	34.3	19.4	18.8	12.6	2.1	4.5	14.0	7.8	0.1	0.0	0.1	0.0	0.0	8.7
Everywhere [ 179 ]	99.8	64.2	72.7	44.5	23.0	46.0	72.6	54.7	0.2	56.3	0.0	2.1	0.2	41.3
STM [ 115 ]	99.9	14.9	16.4	19.6	3.4	8.7	28.2	14.5	0.0	22.8	0.0	0.0	0.3	17.6

TABLE VIII: Attack success rates of various input transformation-based attacks where the adversarial examples are generated on RN50. We have highlighted the approach that achieves the top-1 attack performance in gray .

Attacks	Standardly trained models								Defenses					Average
Attacks	RN-50	VGG-16	MN-v2	Inc-v3	ViT	PiT	Visformer	Swin	AT	HGD	RS	NRP	DiffPure	Average
Po+Trip [ 180 ]	100.0	5.0	7.4	7.7	0.8	5.2	13.9	5.0	0.1	5.9	0.0	0.0	0.0	11.6
Logit [ 181 ]	57.2	22.8	16.7	19.4	2.4	5.5	24.7	10.5	0.1	32.6	0.0	0.0	0.1	14.8
Logit Margin [ 182 ]	56.5	21.9	17.1	20.9	1.6	6.0	24.3	8.8	0.1	30.8	0.0	0.0	0.0	14.5
CFM [ 183 ]	99.1	54.2	62.9	46.9	21.6	38.1	63.7	46.0	0.2	54.2	0.0	0.5	0.8	37.7
FFT [ 184 ]	97.3	11.5	14.3	6.3	1.9	3.8	19.1	7.4	0.1	11.1	0.0	0.0	0.0	13.3

TABLE IX: Attack success rates of various advanced objective functions where the adversarial examples are generated on RN50. We have highlighted the approaches that achieve the top-1 attack performance in gray .

Attacks	Standardly trained models								Defenses					Average
Attacks	RN-50	VGG-16	MN-v2	Inc-v3	ViT	PiT	Visformer	Swin	AT	HGD	RS	NRP	DiffPure	Average
TTP [ 185 ]	71.1	71.8	52.4	32.8	8.0	12.6	47.8	30.5	0.2	57.8	0.3	1.8	1.5	29.9
M3D [ 186 ]	88.5	89.2	79.8	59.0	32.5	35.8	76.4	60.5	0.2	80.2	0.4	3.5	1.7	46.8
AIM [ 187 ]	69.4	51.6	43.3	15.7	12.2	12.8	40.4	24.6	0.1	30.3	0.2	0.4	0.7	23.2

TABLE X: Attack success rates of various generation-based attacks. We have highlighted the approaches that achieve the top-1 attack performance in gray .

Attacks	Standardly trained models								Defenses					Average
Attacks	RN-50	VGG-16	MN-v2	Inc-v3	ViT	PiT	Visformer	Swin	AT	HGD	RS	NRP	DiffPure	Average
SASD-WS [ 188 ]	90.5	82.2	78.9	66.8	18.3	27.2	64.5	41.8	0.2	76.3	0.0	0.9	0.7	42.2
RFCoA [ 189 ]	99.5	98.4	5.9	98.8	0.7	3.3	10.6	5.1	0.0	0.0	0.0	0.1	0.0	24.8
Base [ 44 ]	99.7	40.4	99.7	98.9	0.0	0.6	0.9	0.9	0.0	0.1	0.0	0.0	0.0	26.2
SVRE [ 170 ]	9.2	50.2	20.6	9.0	0.6	2.1	7.7	5.3	0.0	1.5	0.0	18.1	0.0	9.6
MBA [ 172 ]	97.1	85.3	84.6	39.4	14.8	27.1	68.5	52.3	0.3	56.8	0.0	0.1	0.2	40.5

TABLE XI: Attack success rates of various ensemble-based attacks. We have highlighted the approaches that achieve the top-1 attack performance in gray .

Transfer-Based Adversarial Attacks

Transfer-based adversarial attacks craft inputs on accessible models to fool different, often black-box, target models.

See all 80 papers

Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems
Nov 20, 2023Guangjing Wang, Ce Zhou, Yuanda Wang +3Adversarial Attacks Transfer-Based Adversarial Attacks
University of South Florida, USA · Missouri University of Science and Technology, USA · Michigan State University, USA +2
Boosting the Local Invariance for Better Adversarial Transferability
Mar 8, 2025Bohan Liu, Xiaosen WangAdversarial Attacks Transfer-Based Adversarial Attacks
School of Computer Science and Technology, Xidian University, Xi’an 710126, China · School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, China
Learning with Bilevel-Minimax Optimization for Efficient and Reliable Transfer Attacks
Aug 12, 2026Yaohua Liu, Yifan Guo, Jiaxin GaoBilevel Optimization Transfer-Based Adversarial Attacks
The University of Hong Kong, Hong Kong SAR, China · International School of Information Science & Engineering, Dalian University of Technology, Dalian, China · The Hong Kong Polytechnic University, Hong Kong SAR, China

Nov 20, 2023J/K move · Enter open · S save

Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems

Guangjing Wang, Ce Zhou, Yuanda Wang, Bocheng Chen+2

University of South Florida, USA · Missouri University of Science and Technology, USA · Michigan State University, USA+2

Adversarial Attacks Transfer-Based Adversarial Attacks Adversarial Transferability

Open paper AskPDF arXiv

Transferable attacks allow adversarial manipulations or learned attack mechanisms constructed in one setting to compromise other AI systems, potentially reducing attack costs and amplifying security and privacy risks. Yet fragmented terminology and evaluation practices hinder understanding of what transfers, why it transfers, and where transfer fails. This survey synthesizes representative research and develops a unified taxonomy spanning six complementary axes: cross-instance, cross-domain, cross-modality, cross-model, cross-task, and cross-deployment. By separating source-target configuration changes from attacker access and adaptation budgets, the taxonomy provides a common framework for comparing transfer across evasion, poisoning, backdoors, privacy inference, model stealing, and prompt-based attacks. We connect transfer mechanisms, including gradient alignment, shared representations, training persistence, and deployment invariances, to enhancement methods based on augmentation, disentanglement, and optimization. Our synthesis identifies inconsistent transfer, incompatible metrics, and gaps between simulated and physical evaluations, and establishes a research agenda for compositional transfer, configuration differences, temporal persistence, and reproducible benchmarking. Together, this work supports systematic investigation of shared vulnerabilities and guides transfer-aware security assessment and defense.

Abstract

Transfer-Based Adversarial Attacks

Explore similar work

Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems

Boosting the Local Invariance for Better Adversarial Transferability

Learning with Bilevel-Minimax Optimization for Efficient and Reliable Transfer Attacks