Projector-based adversarial attack aims to physically manipulate real-world scenes by projecting adversarial patterns, thereby causing deep image classifiers to produce incorrect predictions. However, existing stealthy projector-based adversarial attack methods model the project-and-capture process in a single and static view, limiting their view-invariant capability and practical applicability in dynamic environments. In this paper, we introduce View-Invariant Projector-Based Adversarial Attack (VIPA), a novel method designed to overcome these limitations by achieving both view-invariant and classifier-agnostic adversarial attacks in the physical world. We first leverage a view-invariant projector-camera system simulation method to model the physical interactions between projected patterns and real-world surfaces. To ensure robustness and stealthiness of the attack across different views, VIPA optimizes adversarial projections by aggregating simulated attack losses from multiple views. This joint optimization across diverse views helps maintain robustness regardless of the viewpoints. Finally, the optimized patterns are physically projected into real-world scenes, where they successfully fool various classifiers from different viewpoints, thereby enabling robust and practical view-invariant adversarial attacks. Our experiments in both targeted and untargeted attacks demonstrate that VIPA consistently achieves higher attack success rates than existing methods, while also enhancing stealthiness and ensuring minimal perceptual degradation across all tested views.
Figures & tables
Method
Visual appearance
Stealthiness level
CA
VI
ProCams simulation
One-pixel DE [ 29 ]
Obvious pixels/squares
Low
✗
✗
None
OPAD [ 10 ]
Structured patterns
Medium
✗
✗
Explicit radiometric model
AdvCP [ 14 ]
Global color cast
Medium
✗
✗
Linear fusion + EOT
SPAA [ 16 ]
Noise
High
✗
✗
CNN
CAPAA [ 23 ]
Attention-based noise
High
✓
✗
CNN
VIPA (ours)
View-invariant noise
High
✓
✓
Gaussian splatting
Table 1: Comparison of our proposed VIPA against existing projector-based adversarial attack methods. The table contrasts these approaches along five aspects: the stealthiness level and visual appearance of the projected perturbations, whether the attack is classifier-agnostic (CA) , whether it is view-invariant (VI) , and the ProCams simulation used to model the project-and-capture process. Early methods [ 29 , 10 , 14 ] often introduce noticeable artifacts such as isolated pixels/squares, structured patterns, or global color casts, resulting in limited stealthiness. SPAA [ 16 ] moves toward much stealthier perturbations through noise-like projections, but its Convolutional Neural Network based (CNN-based) simulation reduces robustness to viewpoint changes. CAPAA [ 23 ] further improves imperceptibility through attention-based perturbations and attains classifier-agnostic attacks, yet it is not view-invariant. In contrast, VIPA leverages Gaussian Splatting with multiple views losses to enable view-invariant attacks.
Figure 1: The system overview of our VIPA . The proposed framework is presented as two stages, Project-and-capture Simulation and Adversarial Attack , which are distinguished by different colors. In the Project-and-capture Simulation stage, (a) we acquire training data by projecting, at each of 18 training viewpoints, a set of patterns and capturing the corresponding images. (b) These captures are then utilized to train a differentiable renderer Rθ that jointly models the scene’s appearance and the projector–camera imaging process. Once trained, Rθ is used as a forward simulator that, given a camera viewpoint v and a projector pattern x′ , synthesizes the rendered image I^x′,vi . In the Adversarial Attack stage, (c) starting from a plain gray initial pattern x0 , we iteratively refine the adversarial projection pattern through adversarial optimization. At each iteration, the rendered images {I^x′,vi}i=1V from all considered viewpoints vi are evaluated by multiple classifiers, and if the minimum classifier confidence is below the target threshold or the perturbation remains within the stealthiness budget, a classifier-agnostic adversarial-loss gradient is applied to update x′ ; otherwise, it is updated using the stealthiness loss gradient, as specified in Algorithm 1 . (d) Finally, the optimized adversarial projection x′ is physically deployed by projecting it onto the target scene, and its effectiveness is verified by randomly sampling 20 viewpoints to evaluate real-world transferability and the resulting classification outcomes.
View
Distance (m)
Horizontal
Vertical
Angle
Direction
Angle
Direction
1
0.652
10°
Right
3°
Up
2
0.707
18°
Right
12°
Up
3
0.727
38°
Right
2°
Up
4
0.657
31°
Right
2°
Up
5
0.640
26°
Right
7°
Up
Table 2: Detailed camera parameters of untargeted attack test views on envelope. For each view, we report the camera–scene distance together with the horizontal and vertical angles, and indicate their corresponding directions. The visualization is shown in Fig. 2 .
Figure 2: The visualization of 18 training views and 20 test views for untargeted attack on the envelope scene. The main plot illustrates the top-down horizontal distribution of the camera positions relative to the target scene, where blue circles and orange triangles denote training and test viewpoints, respectively. The red pentagram represents the entire scene, scaled down to a single point for geometric reference. The 3D View inset provides a spatial visualization of the corresponding capture configuration, where camera frustums indicate viewing directions and the gray cross points mark the vertical projection of each camera’s viewpoint onto the horizontal plane.
Figure 3: Untargeted projector-based adversarial attack. The goal is to use adversarial projections to induce various classifiers to misclassify the camera-captured scene as any label other than the true class, jersey , from various views. For evaluation, we randomly sample and capture 20 test viewpoints. On the far left, we present the clean camera-captured scene and the corresponding normalized difference maps computed against the camera-captured images obtained after projecting the adversarial pattern produced by our method. For each approach, the 1 st column visualizes its adversarial projections. Note that CAPAA [ 23 ] and VIPA (ours) are classifier-agnostic and therefore produce a single adversarial pattern for all classifiers, whereas VIPA classifier-specific and SPAA [ 16 ] generate distinct adversarial patterns for each of the three classifiers. In the classification results, we consider three classifiers, Convnext v2 [ 43 ] , Inception v3 [ 40 ] and ViT [ 7 ] , which are denoted in , and , respectively, and annotate each classifier’s predicted label next to the corresponding viewpoint.
Method
d thr
Untargeted Attack
Targeted Attack
ΔE↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
ΔE↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
SPAA [ 16 ]
4
1.58
2.61
39.74
0.9687
57.67%
1.58
2.65
39.47
0.9679
8.00%
5
1.70
2.91
38.54
0.9625
54.67%
58.33%
1.76
3.01
38.14
0.9611
5.34%
7.22%
6
1.94
3.28
37.50
0.9557
62.67%
2.01
3.39
37.13
0.9551
8.33%
CAPAA [ 23 ]
4
1.64
2.76
39.21
0.9687
53.33%
1.67
2.79
39.26
0.9687
21.67%
5
1.64
2.76
39.20
0.9682
58.00%
56.89%
1.64
2.76
39.39
0.9694
17.67%
18.78%
Table 3: Quantitative comparison of different attack methods under both untargeted and targeted attack scenarios. The 1 st and 2 nd columns report the attack method and the threshold parameter (d thr ), respectively. For each method, three consecutive rows correspond to (d thr =4,5,6), and the Avg. OSR column gives the mean success rate across these three thresholds. Columns 3 to 8 summarize untargeted attack results, where columns 3 to 6 are stealthiness metrics of the camera-captured adversarial projections, and columns 7 to 8 report the attack success rate (OSR) and its average across thresholds. Columns 9 to 14 summarize the corresponding metrics under targeted attacks, which are the same as those for untargeted attacks. All results are aggregated over five experimental setups.
Figure 4: Targeted projector-based adversarial attack. The goal is to use adversarial projections to cause different classifiers to misclassify the camera-captured scene envelope as file from various viewpoints. For evaluation, we randomly sample and capture 20 test viewpoints. On the far left, we present the clean camera-captured scene and the corresponding normalized difference maps computed against the camera-captured images obtained after projecting the adversarial pattern produced by our method. For each approach, the 1 st column visualizes its adversarial projections. Note that CAPAA [ 23 ] and VIPA (ours) are classifier-agnostic and therefore produce a single adversarial pattern for all classifiers, whereas VIPA classifier-specific and SPAA [ 16 ] generate distinct adversarial patterns for each of the three classifiers. In the classification results, we consider three classifiers, Convnext v2 [ 43 ] , Inception v3 [ 40 ] and ViT [ 7 ] , which are denoted in , and , respectively, and annotate each classifier’s predicted label next to the corresponding viewpoint.
Attack Views
d thr
Untargeted Attack
Targeted Attack
ΔE↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
ΔE↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
1
4
1.45
2.30
41.48
0.9692
48.33%
1.39
2.19
42.09
0.9716
7.78%
5
1.45
2.32
41.38
0.9690
49.44%
50.74%
1.39
2.19
42.09
0.9717
10.56%
10.18%
6
1.45
2.32
41.39
0.9691
54.44%
1.39
2.19
42.09
0.9716
12.22%
5
4
1.53
2.47
40.64
0.9653
68.89%
1.51
2.41
41.03
0.9674
22.78%
5
1.53
2.47
40.63
0.9651
66.67%
68.52%
1.51
2.41
41.03
0.9674
19.44%
20.74%
Table 4: View ablation study on the effect of the number of attack views used during optimization under both untargeted and targeted projector-based attack settings. We vary the number of attack views from 1, 5, 9, 12 to 18 while keeping all other settings unchanged to evaluate the impact of viewpoint diversity on attack performance and stealthiness. The table reports results for different threshold values d thr = 4, 5, 6. Columns 3 to 6 present stealthiness metrics, and columns 7 to 8 report OSR and Avg. OSR for untargeted attacks. Columns 9 to 14 present the corresponding metrics for targeted attacks. All results are averaged over three experimental setups.
Figure 5: Representative failure cases. (a) and (b) correspond to object-related challenges: (a) dark-surfaced basketball and (b) geometrically complex potted plant. (c) illustrates a failed targeted attack case, where all methods failed to misclassify teddy as sunscreen in various test views.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 1: Untargeted projector-based adversarial attack. The goal is to use adversarial projections to induce various classifiers to misclassify the camera-captured scene as any label other than the true class, wardrobe , from various views. For evaluation, we randomly sample and capture 20 test viewpoints. On the far left, we present the clean camera-captured scene and the corresponding normalized difference maps computed against the camera-captured images obtained after projecting the adversarial pattern produced by our method. For each approach, the 1 st column visualizes its adversarial projections. Note that CAPAA [ 23 ] and VIPA (ours) are classifier-agnostic and therefore produce a single adversarial pattern for all classifiers, whereas VIPA classifier-specific and SPAA [ 16 ] generate distinct adversarial patterns for each of the three classifiers. In the classification results, we consider three classifiers, Convnext v2 [ 43 ] , Inception v3 [ 40 ] and ViT [ 7 ] , which are denoted in , and , respectively, and annotate each classifier’s predicted label next to the corresponding viewpoint.
Figure 2: Targeted projector-based adversarial attack. The goal is to use adversarial projections to cause different classifiers to misclassify the camera-captured scene wardrobe as barrel from various viewpoints. For evaluation, we randomly sample and capture 20 test viewpoints. On the far left, we present the clean camera-captured scene and the corresponding normalized difference maps computed against the camera-captured images obtained after projecting the adversarial pattern produced by our method. For each approach, the 1 st column visualizes its adversarial projections. Note that CAPAA [ 23 ] and VIPA (ours) are classifier-agnostic and therefore produce a single adversarial pattern for all classifiers, whereas VIPA classifier-specific and SPAA [ 16 ] generate distinct adversarial patterns for each of the three classifiers. In the classification results, we consider three classifiers, Convnext v2 [ 43 ] , Inception v3 [ 40 ] and ViT [ 7 ] , which are denoted in , and , respectively, and annotate each classifier’s predicted label next to the corresponding viewpoint.
Method
d thr
Untargeted attack
Targeted attack
Δ E ↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
Δ E ↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
SPAA [ 16 ]
4
1.40
2.14
41.83
0.9713
48.33%
52.78%
1.66
2.71
39.56
0.9641
8.33%
4.44%
5
1.42
2.22
41.33
0.9684
43.33%
1.79
3.01
38.39
0.9582
1.67%
6
1.47
2.30
40.76
0.9657
66.67%
2.03
3.34
37.44
0.9529
3.33%
CAPAA [ 23 ]
4
1.48
2.30
40.73
0.9662
45.00%
49.44%
1.84
2.93
38.93
0.9615
11.67%
11.11%
5
1.48
2.28
40.81
0.9658
55.00%
1.83
2.97
38.83
0.9626
16.67%
Appendix
Table 1: Quantitative comparison of different attack methods under untargeted and targeted attack scenarios. For untargeted attacks, adversarial projections are designed to cause camera-captured instances of the true class wardrobe to be misclassified as any other class. For targeted attacks, the objective is to consistently fool classifiers into predicting barrel for scenes whose true label is wardrobe . OSR is the success rate for corresponding rows.
Figure 3: Untargeted projector-based adversarial attack. The goal is to use adversarial projections to induce various classifiers to misclassify the camera-captured scene as any label other than the true class, envelope , from various views. For evaluation, we randomly sample and capture 20 test viewpoints. On the far left, we present the clean camera-captured scene and the corresponding normalized difference maps computed against the camera-captured images obtained after projecting the adversarial pattern produced by our method. For each approach, the 1 st column visualizes its adversarial projections. Note that CAPAA [ 23 ] and VIPA (ours) are classifier-agnostic and therefore produce a single adversarial pattern for all classifiers, whereas VIPA classifier-specific and SPAA [ 16 ] generate distinct adversarial patterns for each of the three classifiers. In the classification results, we consider three classifiers, Convnext v2 [ 43 ] , Inception v3 [ 40 ] and ViT [ 7 ] , which are denoted in , and , respectively, and annotate each classifier’s predicted label next to the corresponding viewpoint.
Method
d thr
Untargeted attack
Targeted attack
Δ E ↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
Δ E ↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
SPAA [ 16 ]
4
1.73
3.03
38.24
0.9630
61.67%
62.78%
1.69
3.03
38.10
0.9624
10.00%
7.78%
5
1.89
3.47
36.73
0.9553
63.33%
1.88
3.46
36.79
0.9545
1.67%
6
2.13
3.99
35.47
0.9455
63.33%
2.13
3.99
35.50
0.9461
11.67%
CAPAA [ 23 ]
4
2.06
3.79
35.98
0.9543
63.33%
65.00%
2.06
3.71
36.43
0.9557
20.00%
17.22%
5
2.05
3.78
35.87
0.9529
55.00%
1.98
3.58
36.61
0.9559
16.67%
Appendix
Table 2: Quantitative comparison of different attack methods under untargeted and targeted attack scenarios. For untargeted attacks, adversarial projections are designed to cause camera-captured instances of the true class envelope to be misclassified as any other class. For targeted attacks, the objective is to consistently fool classifiers into predicting file for scenes whose true label is envelope . OSR is the success rate for corresponding rows.
Figure 4: Targeted projector-based adversarial attack. The goal is to use adversarial projections to cause different classifiers to misclassify the camera-captured scene hair spray as cocktail shaker from various viewpoints. For evaluation, we randomly sample and capture 20 test viewpoints. On the far left, we present the clean camera-captured scene and the corresponding normalized difference maps computed against the camera-captured images obtained after projecting the adversarial pattern produced by our method. For each approach, the 1 st column visualizes its adversarial projections. Note that CAPAA [ 23 ] and VIPA (ours) are classifier-agnostic and therefore produce a single adversarial pattern for all classifiers, whereas VIPA classifier-specific and SPAA [ 16 ] generate distinct adversarial patterns for each of the three classifiers. In the classification results, we consider three classifiers, Convnext v2 [ 43 ] , Inception v3 [ 40 ] and ViT [ 7 ] , which are denoted in , and , respectively, and annotate each classifier’s predicted label next to the corresponding viewpoint.
Method
d thr
Untargeted attack
Targeted attack
Δ E ↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
Δ E ↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
SPAA [ 16 ]
4
1.50
2.54
39.53
0.9689
53.33%
46.67%
1.49
2.51
39.55
0.9698
11.67%
12.78%
5
1.74
2.92
38.13
0.9615
46.67%
1.74
2.91
38.09
0.9622
10.00%
6
2.10
3.32
37.12
0.9551
40.00%
2.06
3.28
37.17
0.9566
16.67%
CAPAA [ 23 ]
4
1.43
2.34
40.24
0.9753
60.00%
59.44%
1.46
2.38
40.31
0.9761
26.67%
23.89%
5
1.43
2.31
40.49
0.9756
58.33%
1.46
2.38
40.43
0.9762
20.00%
Appendix
Table 3: Quantitative comparison of different attack methods under untargeted and targeted attack scenarios. For untargeted attacks, adversarial projections are designed to cause camera-captured instances of the true class hair spray to be misclassified as any other class. For targeted attacks, the objective is to consistently fool classifiers into predicting cocktail shaker for scenes whose true label is hair spray . OSR is the success rate for corresponding rows.
Figure 5: Targeted projector-based adversarial attack. The goal is to use adversarial projections to cause different classifiers to misclassify the camera-captured scene jersey as sweatshirt from various viewpoints. For evaluation, we randomly sample and capture 20 test viewpoints. On the far left, we present the clean camera-captured scene and the corresponding normalized difference maps computed against the camera-captured images obtained after projecting the adversarial pattern produced by our method. For each approach, the 1 st column visualizes its adversarial projections. Note that CAPAA [ 23 ] and VIPA (ours) are classifier-agnostic and therefore produce a single adversarial pattern for all classifiers, whereas VIPA classifier-specific and SPAA [ 16 ] generate distinct adversarial patterns for each of the three classifiers. In the classification results, we consider three classifiers, Convnext v2 [ 43 ] , Inception v3 [ 40 ] and ViT [ 7 ] , which are denoted in , and , respectively, and annotate each classifier’s predicted label next to the corresponding viewpoint.
Method
d thr
Untargeted attack
Targeted attack
Δ E ↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
Δ E ↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
SPAA [ 16 ]
4
1.73
2.74
39.66
0.9713
73.33%
70.00%
1.54
2.47
40.54
0.9731
1.67%
5.00%
5
1.78
2.98
38.64
0.9661
63.33%
1.65
2.71
39.42
0.9681
6.67%
6
1.97
3.32
37.55
0.9591
73.33%
1.79
2.97
38.48
0.9626
6.67%
CAPAA [ 23 ]
4
1.67
2.85
39.27
0.9726
45.00%
53.89%
1.50
2.51
40.47
0.9744
45.00%
36.11%
5
1.66
2.80
39.48
0.9726
63.33%
1.51
2.55
40.30
0.9744
30.00%
Appendix
Table 4: Quantitative comparison of different attack methods under untargeted and targeted attack scenarios. For untargeted attacks, adversarial projections are designed to cause camera-captured instances of the true class jersey to be misclassified as any other class. For targeted attacks, the objective is to consistently fool classifiers into predicting sweatshirt for scenes whose true label is jersey . OSR is the success rate for corresponding rows.
Figure 6: Untargeted projector-based adversarial attack. The goal is to use adversarial projections to induce various classifiers to misclassify the camera-captured scene as any label other than the true class, water jug , from various views. For evaluation, we randomly sample and capture 20 test viewpoints. On the far left, we present the clean camera-captured scene and the corresponding normalized difference maps computed against the camera-captured images obtained after projecting the adversarial pattern produced by our method. For each approach, the 1 st column visualizes its adversarial projections. Note that CAPAA [ 23 ] and VIPA (ours) are classifier-agnostic and therefore produce a single adversarial pattern for all classifiers, whereas VIPA classifier-specific and SPAA [ 16 ] generate distinct adversarial patterns for each of the three classifiers. In the classification results, we consider three classifiers, Convnext v2 [ 43 ] , Inception v3 [ 40 ] and ViT [ 7 ] , which are denoted in , and , respectively, and annotate each classifier’s predicted label next to the corresponding viewpoint.
Method
d thr
Untargeted attack
Targeted attack
Δ E ↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
Δ E ↓
L ∞↓
PSNR ↑
SSIM ↑
OSR ↑
Avg. OSR ↑
SPAA [ 16 ]
4
1.53
2.58
39.44
0.9690
51.67%
59.44%
1.50
2.53
39.58
0.9703
8.33%
6.11%
5
1.69
2.99
37.89
0.9612
56.67%
1.73
2.98
38.01
0.9626
6.67%
6
2.04
3.49
36.61
0.9530
70.00%
2.04
3.38
37.06
0.9571
3.33%
CAPAA [ 23 ]
4
1.55
2.52
39.85
0.9750
53.33%
56.67%
1.49
2.45
40.17
0.9756
5.00%
5.56%
5
1.58
2.62
39.36
0.9742
58.33%
1.43
2.33
40.80
0.9781
5.00%
Appendix
Table 5: Quantitative comparison of different attack methods under untargeted and targeted attack scenarios. For untargeted attacks, adversarial projections are designed to cause camera-captured instances of the true class water jug to be misclassified as any other class. For targeted attacks, the objective is to consistently fool classifiers into predicting coffeepot for scenes whose true label is water jug . OSR is the success rate for corresponding rows.
State Key Laboratory of Intelligent Vehicle Safety Technology · School of Cyber Science and Engineering, Huazhong University of Science and Technology · School of Computer Science and Technology, Huazhong University of Science and Technology +2