Perspective-Invariant Attack with Enhanced Transferability of Adversarial Examples

Kaisheng Liang; Yiming Cao; Bin Xiao

cs.CVAug 15, 2026

Perspective-Invariant Attack with Enhanced Transferability of Adversarial Examples

Authors: Kaisheng Liang, Yiming Cao, Bin Xiao

Organizations: Department of Computing, The Hong Kong Polytechnic University, Hong Kong

Abstract

Adversarial examples generated on a surrogate deep neural network (DNN) can often successfully fool other black-box DNN models. This cross-model transferability poses serious security threats to DNNs in practical applications. Input transformation techniques are widely used to enhance adversarial transferability by increasing the diversity of input images. However, existing methods primarily rely on local operations with limited degrees of freedom (DOF), such as block-wise shuffling and resizing, overlooking global perspective transformations that naturally arise from viewpoint changes. In this work, we propose a Perspective-Invariant Attack (PIA), which introduces a multi-DOF vertex sampling strategy that systematically covers the perspective transformation hierarchy from 2-DOF translation to 8-DOF projective mapping. By generating geometrically diverse input variations, PIA effectively reduces overfitting of adversarial perturbations to the surrogate model, thereby improving adversarial transferability. We further propose PIA-Mix, a generic extension that maintains a complementary transformation pool and efficiently combines our perspective transformation with auxiliary methods for improved transferability. Extensive experiments involving various DNN architectures, advanced defense mechanisms, and multimodal large language models (LLMs) demonstrate that PIA and PIA-Mix outperform state-of-the-art transfer-based attacks.

Figures & tables

Fig. 1 : Comparison of input transformations in state-of-the-art attacks, including SIA [ 19 ] , BSR [ 20 ] , and SID [ 21 ] . SIA performs block-wise shuffling, BSR combines shuffling with slight rotations, and SID applies multi-scale resizing with random padding and local image fusion. These methods rely on spatial augmentations within the image plane and do not consider global perspective transformations. In contrast, our method introduces diverse perspective transformations that cover the full projective transformation family.

Fig. 2 : Average attack success rates (%) of different methods using ResNet-50 as the surrogate model. Our PIA and PIA-Mix outperform existing methods.

Panel of Figure 3 — Fig. 3 : (a) Vertex sampling, where $\delta_{max}$ is the distortion budget. Each corner is sampled within the sampling box around its original vertex and may fall outside the image; out-of-frame regions are then cropped to the original size. (b) Attack success rates (%) under different DOF (ResNet-50 and ConViT as surrogate and target). As the geometric DOF increases from 2 to 8, adversarial transferability consistently improves.

Method	Transformation Type	ASR (%)
Baseline (MI-FGSM)	None	13.7
Affine	Global (6-DOF)	41.3
Elastic	Non-rigid (High)	23.0
TPS	Non-rigid (High)	37.5
Perspective (Ours)	Global (8-DOF)	49.3 (Highest)

TABLE I : Attack success rates (%) under different transformation types, using ResNet-50 and ConViT as the surrogate and target models, respectively. While non-rigid transformations (e.g., Elastic and TPS) introduce higher flexibility, they do not yield better transferability than global parametric transformations. Our 8-DOF perspective transformation achieves the highest success rate.

Fig. 4 : Overview of the proposed method. Upper: The pipeline of PIA-Mix. Transformations are sampled from a Complementary Transformation Pool $\mathcal{P}$ containing our $\Phi(\cdot)$ and auxiliary methods. Gradients over $N$ transformed copies are averaged for momentum-based updates. When $\mathcal{P}$ contains only $\Phi(\cdot)$ , it reduces to PIA. Lower: Our Perspective Transformation $\Phi(\cdot)$ . The left panel shows Multi-DOF Vertex Sampling: from Mode 4 to Mode 1, the number of independently sampled corners decreases from 4 to 1, yielding 8 to 2 DOF. The right panel derives the $3\times 3$ transformation matrix $[h_{ij}]$ from the sampled corners.

	$\displaystyle u^{\prime}$	$\displaystyle=\frac{h_{11}u+h_{12}v+h_{13}}{h_{31}u+h_{32}v+1},$
	$\displaystyle v^{\prime}$	$\displaystyle=\frac{h_{21}u+h_{22}v+h_{23}}{h_{31}u+h_{32}v+1},$

Algorithm 1 PIA-Mix Algorithm

Source	Attack	RN-50	DN-121	Inc-v3	MB-v2	ViT	LeViT	ConViT	PiT	Average
RN-50	DI	100	97.0	76.1	95.3	29.7	32.4	19.8	29.8	60.0
	TI	100	90.5	59.8	87.2	31.6	29.0	18.9	25.7	55.3
	DeCoWA	100	99.7	95.9	99.4	53.3	68.6	41.1	59.2	77.2
	SIA	100	99.9	96.2	99.7	45.6	68.9	39.0	59.0	76.0
	BSR	100	99.9	95.5	99.6	48.8	71.6	40.6	60.7	77.1
	SID	100	99.9	97.9	99.6	58.1	82.1	52.8	67.4	82.2

TABLE II : Attack success rates (%) against eight target models on the ImageNet-compatible dataset. All methods are based on MI. The best results are shown in bold.

Source	Category	Attack	RN-50	DN-121	Inc-v3	MB-v2	ViT	LeViT	ConViT	PiT	Average
RN-50	Fixed Transformation Combination	MI	100	84.4	53.0	82.7	21.1	22.0	13.7	19.1	49.5
		MI-PIA (Ours)	100	98.6	78.2	96.3	34.5	35.8	26.1	36.4	63.2
		DI-TI-SIA	100	99.9	97.0	99.7	59.2	81.2	49.1	69.7	82.0
		DI-TI-BSR	100	99.9	96.3	99.8	58.8	78.3	48.2	68.2	81.2
		DI-TI-SID	100	99.8	98.0	99.8	66.6	84.8	57.2	76.9	85.4
		DI-TI-PIA (Ours)	100	99.9	98.6	99.7	68.4	84.3	59.1	77.0	85.9

TABLE III : Attack success rates (%) of combining PIA with existing methods under both fixed transformation (DI-TI) and search-based transformation (L2T, OPS) frameworks against eight target models on the ImageNet-compatible dataset. The best results within each subgroup are shown in bold.

Attack	RN-50*	DN-121*	Inc-v3*	MB-v2*	ViT	LeViT	ConViT	PiT	Average
DI	100 *	100 *	99.6*	100 *	41.5	71.6	40.6	62.1	76.9
TI	100 *	100 *	99.7*	100 *	42.0	59.6	38.1	53.4	74.1
SIA	100 *	100 *	100 *	100 *	68.7	94.3	69.6	89.1	90.2
BSR	100 *	100 *	99.9*	100 *	68.3	93.6	67.8	85.6	89.4
SID	100 *	100 *	100 *	100 *	77.6	95.7	76.1	92.9	92.8
PIA (Ours)	100 *	100 *	100 *	100 *	78.8	95.5	77.7	93.4	93.2

TABLE IV : Attack success rates (%) using an ensemble of models as the surrogate model on the ImageNet-compatible dataset. All methods are based on MI. * indicates the surrogate models. The best results are shown in bold.

Category	Attack	Copy Number	Iteration Number	Average Time (s)
Fixed Transformation	SIA	20	10	0.823
	BSR	20	10	0.729
	SID	20	10	1.217
	PIA (Ours)	20	10	0.739
	PIA-Mix (Ours)	20	10	0.962
Search-based Transformation Combination	L2T	225	10	8.127

TABLE V : Average time (s) for each attack to generate an adversarial example on the ImageNet dataset. The surrogate model is RN-50. Note that for each attack iteration in L2T, the copy number is variable and capped at 225.

Source	Attack	HGD	AT	RS	Protected Model: ViT			Protected Model: ConViT			Inc-v3 ${}_{\text{ens3}}$	Inc-v3 ${}_{\text{ens4}}$	IR-v2 ${}_{\text{ens}}$	Average
Source	Attack	HGD	AT	RS	HGD	NRP	DiffPure	HGD	NRP	DiffPure	Inc-v3 ${}_{\text{ens3}}$	Inc-v3 ${}_{\text{ens4}}$	IR-v2 ${}_{\text{ens}}$	Average
RN-50	SIA	63.5	41.0	37.2	29.6	36.9	24.5	25.1	25.6	19.9	46.0	42.3	23.2	34.6
	BSR	63.7	40.9	36.0	34.0	37.3	26.0	29.1	27.9	22.9	50.4	45.3	27.9	36.8
	SID	73.7	42.2	41.6	34.3	42.3	31.0	33.7	32.0	28.4	58.1	54.5	33.2	42.1
	PIA (Ours)	74.2	42.4	40.1	36.8	42.5	30.2	35.9	34.1	27.7	59.0	55.8	35.8	42.9
	PIA-Mix (Ours)	76.4	43.5	42.2	38.8	43.0	32.9	38.2	35.8	29.9	65.3	59.8	39.4	45.4

TABLE VI : Attack success rates (%) against various defense methods on the ImageNet-compatible dataset. All methods are based on MI. The best results are shown in bold.

Source	Attack	Inc-v3	MB-v2	RN-20- ens-gal	RN-20- ens-dv	Avg
Inc-v3	DI	88.8	78.5	34.4	33.5	58.8
	TI	86.0	77.3	37.6	35.7	59.2
	SIA	100	99.6	65.6	66.4	82.9
	BSR	100	96.0	63.2	62.6	80.5
	SID	100	97.0	65.0	65.7	81.9
	PIA (Ours)	100	98.6	69.6	70.9	84.8

TABLE VII : Attack success rates (%) against four target models on the CIFAR-10 dataset. All methods are based on MI. The best results are shown in bold.

Attack	Qwen2-VL	Claude-3.5	GPT-4o	Average
SIA	83 / 70	49 / 68	40 / 54	57.3 / 64.0
BSR	80 / 69	49 / 66	38 / 51	55.6 / 62.0
SID	85 / 72	54 / 72	39 / 54	59.3 / 66.0
PIA (Ours)	87 / 73	56 / 76	43 / 58	62.0 / 69.0
PIA-Mix (Ours)	89 / 79	59 / 79	50 / 67	66.0 / 75.0
DI-TI-SIA	86 / 73	52 / 71	51 / 67	63.0 / 70.3

TABLE VIII : Attack success rates (%) against multimodal LLMs. Results are reported as Closed / Open-ended, corresponding to two evaluation protocols: a closed-ended binary prompt (“Is this image a photo of {true label}? Yes or No?”) and an open-ended descriptive prompt (“What is the main object in this image?”). We randomly sample 100 images from the ImageNet-compatible dataset and use ResNet-50 as the surrogate model. The best results are shown in bold.

Method	Runs	RN-50	DN-121	Inc-v3	MB-v2	ViT	LeViT	ConViT	PiT
SIA	5	100.0 $\pm$ 0.00	99.9 $\pm$ 0.04	95.3 $\pm$ 0.50	99.6 $\pm$ 0.08	45.9 $\pm$ 0.48	69.3 $\pm$ 0.38	38.6 $\pm$ 0.51	58.3 $\pm$ 0.47
BSR	5	100.0 $\pm$ 0.00	99.8 $\pm$ 0.12	95.0 $\pm$ 0.19	99.7 $\pm$ 0.10	47.6 $\pm$ 0.81	70.0 $\pm$ 0.73	40.1 $\pm$ 0.74	58.5 $\pm$ 0.78
SID	5	100.0 $\pm$ 0.00	99.9 $\pm$ 0.06	97.6 $\pm$ 0.18	99.6 $\pm$ 0.09	57.9 $\pm$ 0.44	81.6 $\pm$ 0.38	52.4 $\pm$ 0.47	66.7 $\pm$ 0.64
PIA (Ours)	5	100.0 $\pm$ 0.00	99.9 $\pm$ 0.08	97.8 $\pm$ 0.19	99.7 $\pm$ 0.12	60.3 $\pm$ 0.39	82.2 $\pm$ 0.50	54.5 $\pm$ 0.42	69.0 $\pm$ 0.35
PIA-Mix (Ours)	5	100.0 $\pm$ 0.00	99.9 $\pm$ 0.06	98.5 $\pm$ 0.21	99.8 $\pm$ 0.10	64.3 $\pm$ 0.28	87.4 $\pm$ 0.30	58.1 $\pm$ 0.20	76.3 $\pm$ 0.14

TABLE IX : Mean and standard deviation of attack success rates (%) over 5 independent runs on the ImageNet-compatible dataset. The surrogate model is RN-50.

Method	Degrees of Freedom	Vertex Mode Distribution	RN-50	DN-121	Inc-v3	MB-v2	ViT	LeViT	ConViT	PiT	Average
PIA	8	(1,0,0,0)	100	99.9	98.0	99.7	55.4	79.0	49.3	64.2	80.7
	$8\!\to\!6$	(1,0,0,0), $h_{31}{=}h_{32}{=}0$	100	99.9	97.1	99.4	47.6	71.8	41.7	58.2	77.0
	6	(0,1,0,0)	100	99.9	97.3	99.3	47.3	71.9	41.3	57.9	76.9
	4	(0,0,1,0)	100	99.4	95.2	97.9	45.7	69.1	37.8	56.8	75.2
	2	(0,0,0,1)	100	99.0	92.8	96.1	43.0	66.6	36.4	55.2	73.6
	8/6/4/2	(0.25,0.25,0.25,0.25)	100	99.9	98.0	99.8	58.7	80.6	52.3	67.0	82.0

TABLE X : Ablation study on the degrees of freedom (DOF) and multi-DOF vertex sampling strategy. All experiments use PIA without the complementary transformation pool, with RN-50 as the surrogate model on the ImageNet-compatible dataset. The vertex mode distribution

(\pi_{4},\pi_{3},\pi_{2},\pi_{1})

specifies the sampling probabilities for 8, 6, 4, and 2 DOF modes, respectively. The

h_{31}{=}h_{32}{=}0

row forces the projective terms of the 8-DOF sampling to zero, reducing it to a 6-DOF affine map, as a direct ablation of the higher-order terms. The best results are shown in bold.

Method	Distortion Rate	RN-50	DN-121	Inc-v3	MB-v2	ViT	LeViT	ConViT	PiT	Average
PIA	0.1	100	99.5	96.1	99.3	46.9	68.2	41.0	58.6	76.2
	0.2	100	99.7	96.5	99.5	49.2	70.7	42.5	61.1	77.4
	0.3	100	99.7	97.0	99.6	53.7	75.9	48.0	63.1	79.6
	0.4	100	99.9	97.3	99.7	56.6	77.2	50.8	65.2	80.8
	0.5	100	99.9	97.9	99.8	58.8	79.5	52.0	66.4	81.8
	0.6	100	99.9	98.1	99.8	60.0	81.9	54.3	69.3	82.9

TABLE XI : Ablation study on the distortion rate

\delta_{max}

. All experiments use PIA with the multi-DOF vertex sampling distribution

(0.4,0.3,0.2,0.1)

, with RN-50 as the surrogate model on the ImageNet-compatible dataset. The best results are shown in bold.

Method	Pool Method	Probability List	RN-50	DN-121	Inc-v3	MB-v2	ViT	LeViT	ConViT	PiT	Average
PIA-Mix	PIA	(1.0)	100	99.9	98.1	99.8	60.0	81.9	54.3	69.3	82.9
	PIA, SIA	(0.5, 0.5)	100	99.9	98.3	99.8	62.1	82.5	55.2	70.8	83.6
	PIA, BSR	(0.5, 0.5)	100	99.8	98.4	99.6	62.6	81.7	55.1	70.2	83.4
	PIA, SID	(0.5, 0.5)	100	99.9	98.5	99.8	64.5	86.9	58.5	76.2	85.5
	PIA, BSR, SIA	(0.5, 0.25, 0.25)	100	99.9	98.4	99.8	62.8	84.3	55.9	73.7	84.4
	SID, BSR, SIA	(0.5, 0.25, 0.25)	100	99.9	98.2	99.8	61.0	82.5	54.4	72.2	83.5

TABLE XII : Ablation study on the composition of the complementary transformation pool

\mathcal{P}

in PIA-Mix. The probability list specifies the sampling probability of each method in the pool. All experiments use RN-50 as the surrogate model on the ImageNet-compatible dataset. The best results are shown in bold.

Fig. 5 : Comparison of our 8-DOF perspective transformation against three alternative transformations with exhaustive hyperparameter tuning, using RN-50 as the surrogate model. The red dashed line indicates the average ASR of our 8-DOF perspective transformation ( $80.7\%$ ). (Left) Affine transformation (6-DOF), sweeping the maximum distortion rate. (Middle) Thin-plate spline (TPS) with a $3\times 3$ control grid (18-DOF), sweeping the maximum displacement. (Right) Elastic deformation (high-DOF), sweeping the elastic strength. Our perspective transformation consistently outperforms all alternatives at their respective optimal settings.

Adversarial Attacks

Adversarial attacks exploit crafted inputs, poisoned data, or system interactions to manipulate AI predictions, behavior, privacy, or reliability.

See all 447 papers

Boosting the Local Invariance for Better Adversarial Transferability
Mar 8, 2025Bohan Liu, Xiaosen WangAdversarial Attacks Transfer-Based Adversarial Attacks
School of Computer Science and Technology, Xidian University, Xi’an 710126, China · School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, China
Enhancing Adversarial Transferability through Block Stretch and Shrink
Nov 21, 2025Quan Liu, Feng Ye, Chenhao Lu +4Adversarial Attacks Adversarial Perturbations
Pareto-Improving Adversarial Attacks with Primal-Dual Regularization
Oct 3, 2026Yang Dai, Longfei Zhang, Wei Tao +3Adversarial Attacks Transfer-Based Adversarial Attacks
Laboratory for Big Data and Decision, National University of Defense Technology, Changsha 410073, China

Mar 8, 2025J/K move · Enter open · S save

Boosting the Local Invariance for Better Adversarial Transferability

Bohan Liu, Xiaosen Wang

School of Computer Science and Technology, Xidian University, Xi’an 710126, China · School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, China

Adversarial Attacks Transfer-Based Adversarial Attacks Adversarial Transferability

Open paper AskPDF arXiv

Transfer-based attacks pose a significant threat to real-world applications by directly targeting victim models with adversarial examples generated on surrogate models. While numerous approaches have been proposed to enhance adversarial transferability, existing works often overlook the intrinsic relationship between adversarial perturbations and input images. In this work, we find that the adversarial perturbations often exhibit poor translation invariance for a given clean image and model, which is attributed to local invariance. Through empirical analysis, we demonstrate a positive correlation between the local invariance of adversarial perturbations w.r.t. the input image and their transferability across models. Based on this finding, we propose a general adversarial transferability boosting technique called the Local Invariance Boosting approach (LI-Boost). Extensive experiments on the standard ImageNet dataset demonstrate that LI-Boost significantly enhances five categories of transfer-based attacks, i.e., gradient-based, input transformation-based, model-related, advanced objective function, and ensemble attacks. The improvements hold not only on conventional CNNs, ViTs, and defense mechanisms, but also on real-world commercial vision API systems and vision-language models. Our approach provides a promising direction for future research on improving adversarial transferability across models. Our code is available at https://github.com/Trustworthy-AI-Group/TransferAttack.

Abstract

Adversarial Attacks

Explore similar work

Boosting the Local Invariance for Better Adversarial Transferability

Enhancing Adversarial Transferability through Block Stretch and Shrink

Pareto-Improving Adversarial Attacks with Primal-Dual Regularization