The Visual Geometry Grounded Transformer (VGGT) enables unified feed-forward 3D reconstruction from multi-view images. However, deploying such a high-performance model may expose critical security vulnerabilities. Traditional adversarial perturbations require costly per-scene optimization, while Universal Adversarial Perturbations (UAPs) rely on a single static pattern and fail to effectively attack VGGT. To address these limitations, we propose MVAP-G, a multi-view adversarial perturbation generator that produces imperceptible consistent perturbations across multiple views in a single feed-forward pass. To ensure perturbation consistency across diverse scenes, we design a cross-view adversarial alignment mechanism to process multi-view images. Experiments demonstrate that MVAP-G significantly degrades VGGT performance without iterative optimization during inference. This work pioneers multi-view adversarial attacks on 3D foundation models, uncovering severe vulnerabilities and underscoring the urgent need for robust 3D vision systems. The code is available at https://github.com/qsong2001/mvap-g.
Figures & tables
Figure 1: Different adversarial examples against VGGT. Although traditional (a) adversarial perturbation (AP) [ 39 ] can achieve effective attack performance, it must be optimized for each scene with significant computational cost. (b) Universal adversarial perturbation [ 43 ] (UAP) is optimized on multi-view scenes data, yet it lacks attack effectiveness. (c) Our proposed MVAP-G is trained on multi-view data and can generate multi-view perturbations (Multi-view APs) against VGGT in real-time.
Figure 2: Overview of the Multi-view Adversarial Perturbation Generator (MVAP-G). The generator G takes a multi-view image sequence S and produces perturbations δt for each frame in a single feed-forward pass. The perturbed images I^t=It+δt are then fed into the pre-trained VGGT model F to produce adversarial 3D point clouds P^ . The generator is built on a transformer architecture with cross-view adversarial alignment (CAA) and a DPT head for generating cross-view pixel-level perturbations.
Method
COCO [ 31 ]
ImageNet [ 8 ]
LLFF [ 42 ]
CO3D [ 47 ]
v=1
v=1
v=1
v=4
v=8
v=16
v=25
v=1
v=4
v=8
v=16
v=25
Gaus. noise
0.534
0.505
0.912
0.900
0.941
0.949
0.982
0.678
0.666
0.671
0.632
0.672
UAP
0.565
0.556
0.940
0.964
0.980
1.010
1.043
0.673
0.675
0.671
0.673
0.659
AP ( iter=10 )
1.361
1.349
1.841
1.769
1.803
1.870
1.834
1.442
1.718
1.890
1.643
1.913
AP ( iter=20 )
1.535
1.584
1.842
1.811
1.817
1.882
1.925
1.889
1.892
1.958
1.875
2.00
MVAP-G ( iter=1 )
1.566
1.611
1.975
1.885
1.857
1.922
1.910
1.995
2.016
2.011
1.999
2.047
Table 1: Attack impacts on VGGT via different perturbation methods. We report Shifted CD distances between the reconstructed 3D point clouds from the clear and perturbed samples. Higher CD values denote a greater shifted distance and indicate better attack performance. MVAP-G achieves superior attack performance with only a single feed-forward iteration .
Figure 3: 3D point cloud reconstruction under adversarial attacks for VGGT. (a) Single-view ( v=1 ) : clean input and its ground truth; reconstructions from five adversarial perturbations, including Gaussian, UAP [ 43 ] , AP with iter=10 and iter=20 [ 39 ] , and our MVAP-G. (b) Multi-views ( v=8 ) : comparison between reconstructions from AP ( iter=20 ) and our MVAP-G, demonstrating our method’s superior effectiveness in a practical setting. Our MVAP-G causes severe degradation with a single pass. More results can be found in the supplemental material.
Figure 4: MVAP-G achieves real-time, memory-efficient, and high-quality multi-view attacks with a single feed-forward pass. (a) Inference time demonstrates that Naive AP requires over 25–200s per scene, while MVAP-G achieves millisecond-level attack in 50–500ms across multi-views input in a single forward pass. (b) Loss convergence shows that Naive AP needs more than 25 steps to reach a loss of 0.008, whereas MVAP-G matches this performance in one step. (c) Memory efficiency demonstrates that MVAP-G uses less than 14 GB of GPU memory, compared to 30 GB for Naive AP.
Figure 5: (a) Cross-view adversarial alignment (CAA). Attack strength remains steady even with more input views. Yet, w/o CAA faces inadequate attack performance against VGGT, especially in multi-view inputs. (b) Adversarial loss. The adversarial loss Ladv converges slowly with oscillations, aligning with the training process of generative adversarial networks [ 1 ] . (c) Regularization loss. The convergence of Lreg indicates effective control over perturbation magnitude, ensuring visual imperceptibility while the adversarial loss remains low, thus achieving a balance between stealth and attack efficacy.
Configuration
Attack performance
Invisibility
CD ↑
Ladv↓
PSNR ↑
LPIPS ↓
Baseline
0.58
0.400
21.1
0.400
+ Pre-training
1.40
0.250
20.8
0.435
+ DPT head
1.68
0.120
20.5
0.478
+ CAA
1.75
0.008
20.3
0.512
+ Lreg
1.78
0.008
29.8
0.107
Table 2: Ablation study on key components of MVAP-G. Pre-training provides the largest boost in attack strength. The DPT outperforms the naive decoder, as it captures more features for pixel-level adversarial perturbations [ 46 ] . CAA enables multi-view performance (more in Fig. 5 (a)). Lreg minimize the size of perturbation and ensures perturbation invisibility.
Figure 6: Adversarial examples and visualized ×20 perturbation.
Figure 7: Impacts of CAA when input 32 view images. Without CAA , perturbations allow VGGT to partially recover geometry results, yielding fragmented but recognizable structures.
Figure 8: (left) Impacts on depth. (right) Impacts on camera pose. Camera pose under attack (CO3D v=8 , avg. 100 scenes) ( please zoom in for best review ).
Figure 9: Statistics under Ladv . (Left) The point cloud collapses to the origin (0,0,0) via supervision of adversarial loss Ladv . (Right) The CD distance between the reconstructed point cloud and the original increases in steps.
Recent feed-forward 3D reconstruction methods, such as visual geometry transformers, have substantially advanced the traditional per-scene optimization paradigm by enabling effective multi-view reconstruction in a single forward pass. However, most existing methods struggle to achieve a balance between reconstruction quality and computational efficiency, which limits their scalability and efficiency. Although some efficient visual geometry transformers have recently emerged, they typically use the same sparsity ratio across layers and frames and lack mechanisms to adaptively learn representative tokens to capture global relationships, leading to suboptimal performance. In this work, we propose TurboVGGT, a novel approach that employs an efficient visual geometry transformer with adaptive alternating attention for fast multi-view 3D reconstruction. Specifically, TurboVGGT employs an end-to-end trainable framework that leverages adaptive sparse global attention guided by adaptive sparsity selection to capture global relationships across frames and uses frame attention to aggregate local details within each frame. In the adaptive sparse global attention, TurboVGGT adaptively learns representative tokens with varying sparsity levels for global geometry modeling, considering that token importance varies across frames, different attention layers operate on tokens at different levels of abstraction, and global dependencies rely on structurally informative regions. Extensive experiments on multiple 3D reconstruction benchmarks demonstrate that TurboVGGT achieves fast multi-view reconstruction while maintaining competitive reconstruction quality compared with state-of-the-art methods.
David Huang, Guile Wu, Chengjie Huang +2
Huawei Noah’s Ark Lab · University of Toronto · Foundation Model Department, Huawei
Feed-forward geometry models such as VGGT recover camera parameters, depth maps, and point maps from one to hundreds of views in a single forward pass, making them powerful backbones for multi-view 3D perception. However, VGGT's global attention is quadratic in the number of frames and dominates end-to-end runtime on long sequences, which has motivated a range of training-free efficiency methods such as token merging. These methods, however, come with a drop in accuracy that makes them hard to use in place of VGGT. We propose GPA-VGGT, a training-free acceleration method that replaces global attention in early blocks with frame attention augmented by a shared cross-frame K/V set. In the remaining blocks, we merge only keys and values while preserving all queries, restore the attention mass of the tokens each merged key represents, and randomly project the matching metric to 128 dimensions to reduce matching cost. Across diverse multi-view geometry tasks, GPA-VGGT provides training-free acceleration at speedups comparable to prior work while preserving the information required for precise geometric estimation, retaining over 99% of VGGT's pose accuracy under AUC@3.
Visual Geometry Grounded Transformer (VGGT) recovers dense 3D scene structure from multi-view images in one forward pass, but quadratic cross-frame attention limits its scalability. Existing training-free accelerators reduce computation uniformly along one axis, missing layer heterogeneity. Our spectral, probing, and causal analyses reveal three regimes: shallow layers lack cross-view structure, middle layers drive cross-view alignment, and deep layers are redundant for dense geometry yet their cross-frame attention remains essential for pose. RegimeVGGT applies layer-wise U-shaped compression along two axes: Saliency-Guided Banded Merging protects geometry- and edge-salient tokens, while Selectively Protected K/V Downsampling preserves cross-frame spatial coverage and the pose-critical path through a phase-shifted spatial grid, a reference-frame anchor, and uncompressed camera/register tokens. Training-free, RegimeVGGT achieves a 6.7x speedup over VGGT* at matched reconstruction quality.
Jinhao You, Shuo Lyu, Zhuohang Lyu +5
University of Pennsylvania · University of California, Irvine · Nanyang Technological University +1