Organizations: University of Chinese Academy of Sciences, Beijing, China · the University of Maryland, College Park, MD, USA · Beijing University of Posts and Telecommunications, Beijing, China
Feed-forward visual geometry models, exemplified by VGGT, have demonstrated strong multi-view reconstruction capabilities through dense global attention. However, the resulting all-to-all cross-view computation scales quadratically with the number of input views, making long-sequence inference particularly expensive due to substantial frame-level redundancy among highly overlapping observations. We uncover a topology-first, geometry-later evolution in VGGT camera tokens: shallow-layer tokens reliably capture local view neighborhoods, whereas deeper-layer tokens progressively encode globally organized inter-view geometry. Motivated by this observation, we introduce BlazeVGGT, a training-free framework for structure-aware frame-level key--value sparsification. At each global-attention block, BlazeVGGT constructs a view-level similarity kernel from camera-token relations and employs greedy determinantal point process inference to select a compact set of mutually complementary views. The selected views constitute the key--value context, while all input frames are retained as queries. This asymmetric attention design preserves dense per-frame predictions while eliminating redundant cross-view computation, allowing our method to be directly integrated into pretrained VGGT as a plug-and-play module without parameter updates. Its advantages become increasingly pronounced with sequence length: at 2,000 frames, BlazeVGGT achieves approximately 13× faster inference than VGGT while reducing the Chamfer Distance from 0.41 to 0.32. Extensive experiments demonstrate a favorable accuracy--efficiency trade-off in point-cloud reconstruction and camera-pose estimation, compatibility with existing VGGT accelerators such as FastVGGT and LiteVGGT, and transferability to VGGT-based models including StreamVGGT and PAGE-4D. Codes are publicly available at https://github.com/MagiciannnY/BlazeVGGT.git.
Figures & tables
Fig. 2 : Camera-token structure and its relation to camera geometry. (a) Camera tokens from deep layer, projected to three dimensions by PCA and colored by camera yaw. The left view contains all frames; the right view contains the DPP-selected frames [ 2 ] , showing that the selected views cover the yaw-organized token manifold. (b) Layerwise diagnostics on a ScanNet [ 3 ] sequence. Left: Spearman correlation between centered-cosine camera-token distance and ground-truth rotation or translation distance. Right: local topology preservation measured by Near AUC and Recall@10. Shaded bands span the three token variants (concat, frame, global).
Fig. 3 : Overview of BlazeVGGT. At each global-attention block, camera tokens from all input frames define a view-level similarity structure, from which greedy DPP [ 2 ] selection retains a diverse subset of frames. All frames remain in the Query path, while the selected frame blocks provide the Key–Value context for global attention; the resulting features are passed to the camera and depth heads.
Method
2,000
1,000
500
100
CD ↓
Time ↓
CD ↓
Time ↓
CD ↓
Time ↓
CD ↓
Time ↓
VGGT [ 1 ] (CVPR’25)
0.4184
1325.1s
0.4760
591.0s
0.4646
152.3s
0.4377
8.1s
Training
LiteVGGT [ 5 ] (CVPR’26)
0.3420
471.6s
0.4085
133.8s
0.3862
44.2s
0.3291
8.1s
VGG-T 3 [ 7 ] (CVPR’26)
0.4822
90.4s
0.5142
76.3s
0.4957
37.2s
0.421
8.0s
w/o Training
FastVGGT [ 4 ] (ICLR’26)
0.3886
444.5s
0.4654
172.7s
0.4569
58.0s
0.4234
4.8s
SparseVGGT [ 9 ] (CVPR’26)
0.3923
725.0s
0.4247
324.8s
0.4348
94.4s
0.4419
7.8s
TABLE I : Point cloud reconstruction on ScanNet-50 [ 3 ] with different numbers of input views. We report Chamfer Distance (CD) and inference time. The 2,000-view setting uses two-GPU sequence parallelism, whereas the 1,000-view setting uses single-GPU inference. BlazeVGGT achieves 13.2 × and 14.0 × speedups in these two settings, respectively. The slightly lower speedup at 2,000 views may partly reflect inter-GPU communication and synchronization overhead, which was not separately profiled.
Method
7 Scenes
NRGBD
Acc ↓
Comp ↓
NC ↑
Time ↓
Acc ↓
Comp ↓
NC ↑
Time ↓
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
VGGT [ 1 ] (CVPR’25)
0.0195
0.0183
0.0267
0.0199
0.5862
0.5829
65.9s
0.0287
0.0191
0.0177
0.0124
0.6646
0.6756
112.1s
Training
LiteVGGT [ 5 ] (CVPR’26)
0.0203
0.0182
0.0249
0.0169
0.6065
0.6056
22.0s
0.0354
0.0250
0.0255
0.0125
0.6908
0.6955
32.9s
VGG-T 3 [ 7 ] (CVPR’26)
0.1306
0.0304
0.0701
0.0308
0.5858
0.5902
26.5s
0.1788
0.0525
0.1580
0.0171
0.6667
0.6713
34.2s
w/o Training
FastVGGT [ 4 ] (ICLR’26)
0.0178
0.0172
0.0263
0.0193
0.5895
0.5907
25.3s
0.0315
0.0180
0.0191
0.0126
0.6690
0.6799
47.2s
TABLE II : Point cloud reconstruction on 7 Scenes [ 75 ] and NRGBD [ 76 ] using stride-3 sampling. We report accuracy (Acc), completeness (Comp), normal consistency (NC), and inference time.
Method
2,000
1,000
500
100
ATE ↓
ARE ↓
RPE-t ↓
RPE-r ↓
ATE ↓
ARE ↓
RPE-t ↓
RPE-r ↓
ATE ↓
ARE ↓
RPE-t ↓
RPE-r ↓
ATE ↓
ARE ↓
RPE-t ↓
RPE-r ↓
VGGT (CVPR’25)
0.1644
7.0532
0.0346
0.9154
0.1336
5.6834
0.0419
1.1971
0.1236
4.8265
0.0414
0.9660
0.0921
3.9382
0.0562
1.2169
Training
LiteVGGT (CVPR’26)
0.1321
4.9931
0.0321
0.7468
0.1497
7.4436
0.0387
1.1073
0.1181
4.6390
0.0400
0.7811
0.1077
4.1110
0.0732
1.4482
VGG-T 3 (CVPR’26)
0.9331
82.9325
0.1190
11.6955
0.5045
36.0913
0.1523
9.9699
0.5450
34.8302
0.1341
7.8155
0.2970
9.2451
0.1945
4.9533
w/o Training
FastVGGT (ICLR’26)
0.2155
10.1581
0.0628
2.3282
0.1146
4.0139
0.0309
0.6628
0.0927
4.0140
0.0305
0.6418
0.0932
3.8528
0.0600
1.2571
SparseVGGT (CVPR’26)
0.2712
14.2328
0.0649
2.2359
0.2205
9.2380
0.0566
1.8232
0.1270
5.0168
0.0467
1.1367
0.1047
4.4391
0.0661
1.4633
TABLE III : Camera pose estimation on ScanNet-50 [ 3 ] under different input sequence lengths. We report absolute trajectory error (ATE), absolute rotation error (ARE), relative translation error (RPE-t), and relative rotation error (RPE-r).
Fig. 4 : Qualitative comparison of camera trajectory estimation in the x – z plane. The four columns correspond to VGGT [ 1 ] , FastVGGT [ 4 ] , LiteVGGT [ 5 ] , and BlazeVGGT.
Threshold ϵ
Top- K
Blocks
CD ↓
ATE ↓
ARE ↓
RPE-t ↓
RPE-r ↓
Baseline
0.4375
0.0920
3.9553
0.0559
1.2580
0.05
✓
0–23
0.4223
0.0950
4.0080
0.0601
1.2739
0.10
✓
0–23
0.4147
0.0988
4.0749
0.0619
1.2715
0.15
✓
0–23
0.4123
0.0988
4.1532
0.0648
1.3128
0.20
✓
0–23
0.4109
0.1032
4.1628
0.0667
1.3570
0.25
✓
0–23
0.4123
0.1134
4.2518
0.0737
1.4210
TABLE IV : Ablation study of BlazeVGGT. We sequentially determine the selection threshold ϵ , the Top- K constraint, and the transformer blocks where view selection is applied.
Method
1,000
500
300
100
CD ↓
Time ↓
CD ↓
Time ↓
CD ↓
Time ↓
CD ↓
Time ↓
FastVGGT (ICLR’26)
0.4654
180.1s
0.4569
58.0s
0.4467
24.5s
0.4234
4.8s
FastVGGT + BlazeVGGT
0.4255
81.6s ( ↑ 2.2 × )
0.4224
33.1s ( ↑ 1.8 × )
0.4238
15.8s ( ↑ 1.6 × )
0.4112
4.2s ( ↑ 1.1 × )
LiteVGGT (CVPR’26)
0.4085
133.8s
0.3862
44.2s
0.3516
24.7s
0.3291
8.1s
LiteVGGT + BlazeVGGT
0.3640
45.8s ( ↑ 2.9 × )
0.3713
23.8s ( ↑ 1.9 × )
0.3397
15.5s ( ↑ 1.6 × )
0.3409
8.9s ( ↓ 1.1 × )
TABLE V : Compatibility on ScanNet-50 [ 3 ] : point cloud reconstruction and runtime before and after adding BlazeVGGT into FastVGGT [ 4 ] and LiteVGGT [ 5 ] ..
Method
Barn
Caterpillar
Courthouse
Ignatius
Meetingroom
Truck
Avg. ↑
Time(s) ↓
threshold
0.010
0.005
0.025
0.003
0.010
0.005
×5
–
FastVGGT [ 4 ] (ICLR’26)
0.35
0.27
0.34
0.35
0.40
0.48
0.36
63.51s
FastVGGT + BlazeVGGT
0.29
0.29
0.34
0.31
0.44
0.52
0.36
41.92s ( ↑ 1.5 × )
LiteVGGT [ 5 ] (CVPR’26)
0.42
0.29
0.24
0.30
0.57
0.54
0.39
30.79s
LiteVGGT + BlazeVGGT
0.39
0.28
0.26
0.33
0.55
0.51
0.39
17.78s ( ↑ 1.7 × )
threshold
0.010
0.005
0.025
0.003
0.010
0.005
×10
–
TABLE VI : Point cloud reconstruction on Tanks & Temples [ 77 ] . We report scene-wise F1-score and average inference time. ×5 and ×10 indicate evaluation thresholds scaled by five and ten times the official values, respectively.
Method
1,000
500
300
100
ATE ↓
ARE ↓
RPE-t ↓
RPE-r ↓
ATE ↓
ARE ↓
RPE-t ↓
RPE-r ↓
ATE ↓
ARE ↓
RPE-t ↓
RPE-r ↓
ATE ↓
ARE ↓
RPE-t ↓
RPE-r ↓
FastVGGT (ICLR’26)
0.115
4.014
0.031
0.663
0.097
4.151
0.032
0.717
0.092
4.033
0.036
0.762
0.093
3.853
0.060
1.257
FastVGGT + BlazeVGGT
0.099
3.732
0.029
0.683
0.106
4.430
0.035
0.805
0.107
4.542
0.044
0.986
0.111
4.362
0.075
1.492
LiteVGGT (CVPR’26)
0.150
7.444
0.039
1.107
0.120
4.678
0.041
0.791
0.114
4.463
0.046
0.896
0.108
4.111
0.073
1.448
LiteVGGT + BlazeVGGT
0.108
4.249
0.032
0.630
0.120
4.856
0.039
0.746
0.117
4.409
0.048
0.903
0.123
4.335
0.082
1.650
TABLE VII : Compatibility on ScanNet-50 [ 3 ] : camera pose errors before and after integrating BlazeVGGT into VGGT [ 1 ] , FastVGGT [ 4 ] , and LiteVGGT [ 5 ] . We report ATE, ARE, RPE-t, and RPE-r. Lower is better.
Method
1 frame
5 frames
10 frames
20 frames
30 frames
40 frames
Acc ↓ / ms ↓ / GB ↓
Acc ↓ / ms ↓ / GB ↓
Acc ↓ / ms ↓ / GB ↓
Acc ↓ / ms ↓ / GB ↓
Acc ↓ / ms ↓ / GB ↓
Acc ↓ / ms ↓ / GB ↓
Spann3R [ 65 ] (3DV’25)
0.039 / 500 / 5.3
0.038 / 87 / 5.4
0.046 / 82 / 5.5
0.036 / 120 / 5.5
0.043 / 192 / 5.6
0.068 / 125 / 5.7
CUT3R [ 66 ] (CVPR’25)
0.035 / 101 / 3.4
0.038 / 102 / 3.5
0.039 / 99 / 3.6
0.031 / 101 / 3.8
0.027 / 102 / 4.0
0.023 / 102 / 4.2
StreamVGGT (ICLR’26)
0.028 / 63 / 2.1
0.024 / 88 / 2.7
0.024 / 120 / 3.2
0.021 / 152 / 4.3
0.021 / 184 / 5.5
0.021 / 216 / 6.6
StreamVGGT + BlazeVGGT
0.021 / 96 / 7.2
0.017 / 96 / 7.5
0.015 / 97 / 7.5
0.014 / 107 / 7.7
0.014 / 96 / 7.8
0.015 / 97 / 8.0
TABLE VIII : Online reconstruction with StreamVGGT [ 11 ] under increasing context length. Each entry reports accuracy, inference time (ms), and memory consumption (GB).
Method
Acc ↓
Comp ↓
NC ↑
Inference Time @ frame 200
Peak Memory
Mean
Med.
Mean
Med.
Mean
Med.
Spann3R [ 65 ] (3DV’25)
0.225
0.176
0.070
0.015
0.538
0.557
221.054 ms
6.7 GB
CUT3R [ 66 ] (CVPR’25)
0.165
0.107
0.030
0.012
0.551
0.577
102.533 ms
7.6 GB
StreamVGGT (ICLR’26)
None
0.030
0.024
0.021
0.013
0.560
0.561
737.535 ms
31.9 GB
Window(50)
0.062
0.033
0.028
0.015
0.553
0.554
277.584 ms
20.3 GB
Window(100)
0.043
0.028
0.023
0.014
0.557
0.557
478.251 ms
30.7 GB
TABLE IX : Comparison of context-pruning strategies for StreamVGGT [ 11 ] on a 200-frame online sequence.
Fig. 5 : Qualitative comparison of point cloud reconstruction. From left to right: VGGT [ 1 ] , LiteVGGT [ 5 ] , VGG-T 3 [ 7 ] , FastVGGT [ 4 ] , SparseVGGT [ 9 ] , and BlazeVGGT (Ours). (a) Selected viewpoints within the full reconstructed scenes; colored rectangles highlight local regions that demonstrate BlazeVGGT’s reconstruction quality. (b) Full scenes, demonstrating that BlazeVGGT preserves scene completeness and coherent global structure despite aggressive frame-level pruning of the KV context.
Fig. 6 : Qualitative point-cloud comparison for online reconstruction with StreamVGGT [ 11 ] . Across two scenes, StreamVGGT [ 11 ] with BlazeVGGT preserves coherent scene layouts and object boundaries while using a compact long-term context.
Method
Params
Sintel [ 78 ]
Bonn [ 79 ]
DyCheck [ 80 ]
FPS
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Around 100 Frames
Depth alignment
PAGE4D [ 12 ] (ICLR’26)
1.26B
0.357
0.699
0.022
0.723
0.205
0.767
11.18
PAGE4D + BlazeVGGT
1.26B
0.312
0.703
0.088
0.911
0.204
0.774
21.31
Scale-and-shift alignment
TABLE X : Video depth estimation on Sintel [ 78 ] , Bonn [ 79 ] , and DyCheck [ 80 ] under standard and extended sequence lengths. We report Abs Rel, δ<1.25 , and inference throughput.
Visual Geometry Grounded Transformer (VGGT) recovers dense 3D scene structure from multi-view images in one forward pass, but quadratic cross-frame attention limits its scalability. Existing training-free accelerators reduce computation uniformly along one axis, missing layer heterogeneity. Our spectral, probing, and causal analyses reveal three regimes: shallow layers lack cross-view structure, middle layers drive cross-view alignment, and deep layers are redundant for dense geometry yet their cross-frame attention remains essential for pose. RegimeVGGT applies layer-wise U-shaped compression along two axes: Saliency-Guided Banded Merging protects geometry- and edge-salient tokens, while Selectively Protected K/V Downsampling preserves cross-frame spatial coverage and the pose-critical path through a phase-shifted spatial grid, a reference-frame anchor, and uncompressed camera/register tokens. Training-free, RegimeVGGT achieves a 6.7x speedup over VGGT* at matched reconstruction quality.
Jinhao You, Shuo Lyu, Zhuohang Lyu +5
University of Pennsylvania · University of California, Irvine · Nanyang Technological University +1
Feed-forward geometry models such as VGGT recover camera parameters, depth maps, and point maps from one to hundreds of views in a single forward pass, making them powerful backbones for multi-view 3D perception. However, VGGT's global attention is quadratic in the number of frames and dominates end-to-end runtime on long sequences, which has motivated a range of training-free efficiency methods such as token merging. These methods, however, come with a drop in accuracy that makes them hard to use in place of VGGT. We propose GPA-VGGT, a training-free acceleration method that replaces global attention in early blocks with frame attention augmented by a shared cross-frame K/V set. In the remaining blocks, we merge only keys and values while preserving all queries, restore the attention mass of the tokens each merged key represents, and randomly project the matching metric to 128 dimensions to reduce matching cost. Across diverse multi-view geometry tasks, GPA-VGGT provides training-free acceleration at speedups comparable to prior work while preserving the information required for precise geometric estimation, retaining over 99% of VGGT's pose accuracy under AUC@3.
Feed-forward visual geometry models such as the Visual Geometry Grounded Transformer (VGGT) have recently enabled direct 3D reconstruction from multi-view images. Despite their promising performance, these models scale quadratically with the number of input views due to their global attention mechanism, resulting in substantial latency for long sequence inputs. There have been some recent efforts to accelerate VGGT, but they primarily focus on reducing token redundancy through token merging or key/value sparsification. Our work resolves this bottleneck from a different perspective by investigating architectural redundancy in visual geometry transformers. We show that the multi-head attention modules in VGGT's global-attention layers contain substantial architectural redundancy, with only a subset of heads carrying critical geometric information. In light of this observation, we propose VGGT-Prime, a compute-adaptive mixture-of-heads model that resolves this redundancy to accelerate visual geometry transformers while maintaining competitive reconstruction quality. The key idea of VGGT-Prime is to estimate the appropriate computation level for each global-attention head using a lightweight router and then dynamically assign each head to different computation modes. Extensive experiments on multiple datasets demonstrate that VGGT-Prime can achieve an {8×} inference speedup over VGGT while maintaining competitive performance on camera pose, depth, and point-cloud predictions. We further show that VGGT-Prime is complementary to existing acceleration methods, such as token merging, further improving inference speed by up to 14× over VGGT. An overview of our work is available on our project page.
Abteen Arab, Guile Wu, Chengjie Huang +1
Huawei Noah’s Ark Lab · University of British Columbia