BlazeVGGT: 13x Faster VGGT at 2,000 Frames via Camera Token without Training

Shujun Yao; Junhui Lin; Congying Han; Tiande Guo; Ziwen Liu; Bonan Li

cs.CVOct 21, 2026

BlazeVGGT: 13x Faster VGGT at 2,000 Frames via Camera Token without Training

Authors: Shujun Yao, Junhui Lin, Congying Han, Tiande Guo, Ziwen Liu, Bonan Li

Organizations: University of Chinese Academy of Sciences, Beijing, China · the University of Maryland, College Park, MD, USA · Beijing University of Posts and Telecommunications, Beijing, China

Abstract

Feed-forward visual geometry models, exemplified by VGGT, have demonstrated strong multi-view reconstruction capabilities through dense global attention. However, the resulting all-to-all cross-view computation scales quadratically with the number of input views, making long-sequence inference particularly expensive due to substantial frame-level redundancy among highly overlapping observations. We uncover a topology-first, geometry-later evolution in VGGT camera tokens: shallow-layer tokens reliably capture local view neighborhoods, whereas deeper-layer tokens progressively encode globally organized inter-view geometry. Motivated by this observation, we introduce BlazeVGGT, a training-free framework for structure-aware frame-level key--value sparsification. At each global-attention block, BlazeVGGT constructs a view-level similarity kernel from camera-token relations and employs greedy determinantal point process inference to select a compact set of mutually complementary views. The selected views constitute the key--value context, while all input frames are retained as queries. This asymmetric attention design preserves dense per-frame predictions while eliminating redundant cross-view computation, allowing our method to be directly integrated into pretrained VGGT as a plug-and-play module without parameter updates. Its advantages become increasingly pronounced with sequence length: at 2,000 frames, BlazeVGGT achieves approximately 13 $\times$ faster inference than VGGT while reducing the Chamfer Distance from 0.41 to 0.32. Extensive experiments demonstrate a favorable accuracy--efficiency trade-off in point-cloud reconstruction and camera-pose estimation, compatibility with existing VGGT accelerators such as FastVGGT and LiteVGGT, and transferability to VGGT-based models including StreamVGGT and PAGE-4D. Codes are publicly available at https://github.com/MagiciannnY/BlazeVGGT.git.

Figures & tables

Panel of Figure 1 — Fig. 2 : Camera-token structure and its relation to camera geometry. (a) Camera tokens from deep layer, projected to three dimensions by PCA and colored by camera yaw. The left view contains all frames; the right view contains the DPP-selected frames [ 2 ] , showing that the selected views cover the yaw-organized token manifold. (b) Layerwise diagnostics on a ScanNet [ 3 ] sequence. Left: Spearman correlation between centered-cosine camera-token distance and ground-truth rotation or translation distance. Right: local topology preservation measured by Near AUC and Recall@10. Shaded bands span the three token variants (concat, frame, global).

Fig. 3 : Overview of BlazeVGGT. At each global-attention block, camera tokens from all input frames define a view-level similarity structure, from which greedy DPP [ 2 ] selection retains a diverse subset of frames. All frames remain in the Query path, while the selected frame blocks provide the Key–Value context for global attention; the resulting features are passed to the camera and depth heads.

Method		2,000		1,000		500		100
Method		CD $\downarrow$	Time $\downarrow$	CD $\downarrow$	Time $\downarrow$	CD $\downarrow$	Time $\downarrow$	CD $\downarrow$	Time $\downarrow$
VGGT [ 1 ] (CVPR’25)		0.4184	1325.1s	0.4760	591.0s	0.4646	152.3s	0.4377	8.1s
Training	LiteVGGT [ 5 ] (CVPR’26)	0.3420	471.6s	0.4085	133.8s	0.3862	44.2s	0.3291	8.1s
Training	VGG-T 3 [ 7 ] (CVPR’26)	0.4822	90.4s	0.5142	76.3s	0.4957	37.2s	0.421	8.0s
w/o Training	FastVGGT [ 4 ] (ICLR’26)	0.3886	444.5s	0.4654	172.7s	0.4569	58.0s	0.4234	4.8s
	SparseVGGT [ 9 ] (CVPR’26)	0.3923	725.0s	0.4247	324.8s	0.4348	94.4s	0.4419	7.8s

TABLE I : Point cloud reconstruction on ScanNet-50 [ 3 ] with different numbers of input views. We report Chamfer Distance (CD) and inference time. The 2,000-view setting uses two-GPU sequence parallelism, whereas the 1,000-view setting uses single-GPU inference. BlazeVGGT achieves 13.2

\times

and 14.0

\times

speedups in these two settings, respectively. The slightly lower speedup at 2,000 views may partly reflect inter-GPU communication and synchronization overhead, which was not separately profiled.

Method		7 Scenes							NRGBD
		Acc $\downarrow$		Comp $\downarrow$		NC $\uparrow$		Time $\downarrow$	Acc $\downarrow$		Comp $\downarrow$		NC $\uparrow$		Time $\downarrow$
		Mean	Med.	Mean	Med.	Mean	Med.	Time $\downarrow$	Mean	Med.	Mean	Med.	Mean	Med.	Time $\downarrow$
VGGT [ 1 ] (CVPR’25)		0.0195	0.0183	0.0267	0.0199	0.5862	0.5829	65.9s	0.0287	0.0191	0.0177	0.0124	0.6646	0.6756	112.1s
Training	LiteVGGT [ 5 ] (CVPR’26)	0.0203	0.0182	0.0249	0.0169	0.6065	0.6056	22.0s	0.0354	0.0250	0.0255	0.0125	0.6908	0.6955	32.9s
Training	VGG-T 3 [ 7 ] (CVPR’26)	0.1306	0.0304	0.0701	0.0308	0.5858	0.5902	26.5s	0.1788	0.0525	0.1580	0.0171	0.6667	0.6713	34.2s
w/o Training	FastVGGT [ 4 ] (ICLR’26)	0.0178	0.0172	0.0263	0.0193	0.5895	0.5907	25.3s	0.0315	0.0180	0.0191	0.0126	0.6690	0.6799	47.2s

TABLE II : Point cloud reconstruction on 7 Scenes [ 75 ] and NRGBD [ 76 ] using stride-3 sampling. We report accuracy (Acc), completeness (Comp), normal consistency (NC), and inference time.

Method		2,000				1,000				500				100
Method		ATE $\downarrow$	ARE $\downarrow$	RPE-t $\downarrow$	RPE-r $\downarrow$	ATE $\downarrow$	ARE $\downarrow$	RPE-t $\downarrow$	RPE-r $\downarrow$	ATE $\downarrow$	ARE $\downarrow$	RPE-t $\downarrow$	RPE-r $\downarrow$	ATE $\downarrow$	ARE $\downarrow$	RPE-t $\downarrow$	RPE-r $\downarrow$
VGGT (CVPR’25)		0.1644	7.0532	0.0346	0.9154	0.1336	5.6834	0.0419	1.1971	0.1236	4.8265	0.0414	0.9660	0.0921	3.9382	0.0562	1.2169
Training	LiteVGGT (CVPR’26)	0.1321	4.9931	0.0321	0.7468	0.1497	7.4436	0.0387	1.1073	0.1181	4.6390	0.0400	0.7811	0.1077	4.1110	0.0732	1.4482
Training	VGG-T 3 (CVPR’26)	0.9331	82.9325	0.1190	11.6955	0.5045	36.0913	0.1523	9.9699	0.5450	34.8302	0.1341	7.8155	0.2970	9.2451	0.1945	4.9533
w/o Training	FastVGGT (ICLR’26)	0.2155	10.1581	0.0628	2.3282	0.1146	4.0139	0.0309	0.6628	0.0927	4.0140	0.0305	0.6418	0.0932	3.8528	0.0600	1.2571
	SparseVGGT (CVPR’26)	0.2712	14.2328	0.0649	2.2359	0.2205	9.2380	0.0566	1.8232	0.1270	5.0168	0.0467	1.1367	0.1047	4.4391	0.0661	1.4633

TABLE III : Camera pose estimation on ScanNet-50 [ 3 ] under different input sequence lengths. We report absolute trajectory error (ATE), absolute rotation error (ARE), relative translation error (RPE-t), and relative rotation error (RPE-r).

Fig. 4 : Qualitative comparison of camera trajectory estimation in the $x$ – $z$ plane. The four columns correspond to VGGT [ 1 ] , FastVGGT [ 4 ] , LiteVGGT [ 5 ] , and BlazeVGGT.

Threshold $\epsilon$	Top- $K$	Blocks	CD $\downarrow$	ATE $\downarrow$	ARE $\downarrow$	RPE-t $\downarrow$	RPE-r $\downarrow$
Baseline			0.4375	0.0920	3.9553	0.0559	1.2580
0.05	✓	0–23	0.4223	0.0950	4.0080	0.0601	1.2739
0.10	✓	0–23	0.4147	0.0988	4.0749	0.0619	1.2715
0.15	✓	0–23	0.4123	0.0988	4.1532	0.0648	1.3128
0.20	✓	0–23	0.4109	0.1032	4.1628	0.0667	1.3570
0.25	✓	0–23	0.4123	0.1134	4.2518	0.0737	1.4210

TABLE IV : Ablation study of BlazeVGGT. We sequentially determine the selection threshold

\epsilon

, the Top-

K

constraint, and the transformer blocks where view selection is applied.

Method	1,000		500		300		100
Method	CD $\downarrow$	Time $\downarrow$	CD $\downarrow$	Time $\downarrow$	CD $\downarrow$	Time $\downarrow$	CD $\downarrow$	Time $\downarrow$
FastVGGT (ICLR’26)	0.4654	180.1s	0.4569	58.0s	0.4467	24.5s	0.4234	4.8s
FastVGGT + BlazeVGGT	0.4255	81.6s ( $\uparrow$ 2.2 $\times$ )	0.4224	33.1s ( $\uparrow$ 1.8 $\times$ )	0.4238	15.8s ( $\uparrow$ 1.6 $\times$ )	0.4112	4.2s ( $\uparrow$ 1.1 $\times$ )
LiteVGGT (CVPR’26)	0.4085	133.8s	0.3862	44.2s	0.3516	24.7s	0.3291	8.1s
LiteVGGT + BlazeVGGT	0.3640	45.8s ( $\uparrow$ 2.9 $\times$ )	0.3713	23.8s ( $\uparrow$ 1.9 $\times$ )	0.3397	15.5s ( $\uparrow$ 1.6 $\times$ )	0.3409	8.9s ( $\downarrow$ 1.1 $\times$ )

TABLE V : Compatibility on ScanNet-50 [ 3 ] : point cloud reconstruction and runtime before and after adding BlazeVGGT into FastVGGT [ 4 ] and LiteVGGT [ 5 ] ..

Method	Barn	Caterpillar	Courthouse	Ignatius	Meetingroom	Truck	Avg. $\uparrow$	Time(s) $\downarrow$
threshold	0.010	0.005	0.025	0.003	0.010	0.005	$\times 5$	–
FastVGGT [ 4 ] (ICLR’26)	0.35	0.27	0.34	0.35	0.40	0.48	0.36	63.51s
FastVGGT + BlazeVGGT	0.29	0.29	0.34	0.31	0.44	0.52	0.36	41.92s ( $\uparrow$ 1.5 $\times$ )
LiteVGGT [ 5 ] (CVPR’26)	0.42	0.29	0.24	0.30	0.57	0.54	0.39	30.79s
LiteVGGT + BlazeVGGT	0.39	0.28	0.26	0.33	0.55	0.51	0.39	17.78s ( $\uparrow$ 1.7 $\times$ )
threshold	0.010	0.005	0.025	0.003	0.010	0.005	$\times 10$	–

TABLE VI : Point cloud reconstruction on Tanks & Temples [ 77 ] . We report scene-wise F1-score and average inference time.

\times 5

and

\times 10

indicate evaluation thresholds scaled by five and ten times the official values, respectively.

Method	1,000				500				300				100
Method	ATE $\downarrow$	ARE $\downarrow$	RPE-t $\downarrow$	RPE-r $\downarrow$	ATE $\downarrow$	ARE $\downarrow$	RPE-t $\downarrow$	RPE-r $\downarrow$	ATE $\downarrow$	ARE $\downarrow$	RPE-t $\downarrow$	RPE-r $\downarrow$	ATE $\downarrow$	ARE $\downarrow$	RPE-t $\downarrow$	RPE-r $\downarrow$
FastVGGT (ICLR’26)	0.115	4.014	0.031	0.663	0.097	4.151	0.032	0.717	0.092	4.033	0.036	0.762	0.093	3.853	0.060	1.257
FastVGGT + BlazeVGGT	0.099	3.732	0.029	0.683	0.106	4.430	0.035	0.805	0.107	4.542	0.044	0.986	0.111	4.362	0.075	1.492
LiteVGGT (CVPR’26)	0.150	7.444	0.039	1.107	0.120	4.678	0.041	0.791	0.114	4.463	0.046	0.896	0.108	4.111	0.073	1.448
LiteVGGT + BlazeVGGT	0.108	4.249	0.032	0.630	0.120	4.856	0.039	0.746	0.117	4.409	0.048	0.903	0.123	4.335	0.082	1.650

TABLE VII : Compatibility on ScanNet-50 [ 3 ] : camera pose errors before and after integrating BlazeVGGT into VGGT [ 1 ] , FastVGGT [ 4 ] , and LiteVGGT [ 5 ] . We report ATE, ARE, RPE-t, and RPE-r. Lower is better.

Method	1 frame	5 frames	10 frames	20 frames	30 frames	40 frames
Method	Acc $\downarrow$ / ms $\downarrow$ / GB $\downarrow$	Acc $\downarrow$ / ms $\downarrow$ / GB $\downarrow$	Acc $\downarrow$ / ms $\downarrow$ / GB $\downarrow$	Acc $\downarrow$ / ms $\downarrow$ / GB $\downarrow$	Acc $\downarrow$ / ms $\downarrow$ / GB $\downarrow$	Acc $\downarrow$ / ms $\downarrow$ / GB $\downarrow$
Spann3R [ 65 ] (3DV’25)	0.039 / 500 / 5.3	0.038 / 87 / 5.4	0.046 / 82 / 5.5	0.036 / 120 / 5.5	0.043 / 192 / 5.6	0.068 / 125 / 5.7
CUT3R [ 66 ] (CVPR’25)	0.035 / 101 / 3.4	0.038 / 102 / 3.5	0.039 / 99 / 3.6	0.031 / 101 / 3.8	0.027 / 102 / 4.0	0.023 / 102 / 4.2
StreamVGGT (ICLR’26)	0.028 / 63 / 2.1	0.024 / 88 / 2.7	0.024 / 120 / 3.2	0.021 / 152 / 4.3	0.021 / 184 / 5.5	0.021 / 216 / 6.6
StreamVGGT + BlazeVGGT	0.021 / 96 / 7.2	0.017 / 96 / 7.5	0.015 / 97 / 7.5	0.014 / 107 / 7.7	0.014 / 96 / 7.8	0.015 / 97 / 8.0

TABLE VIII : Online reconstruction with StreamVGGT [ 11 ] under increasing context length. Each entry reports accuracy, inference time (ms), and memory consumption (GB).

Method		Acc $\downarrow$		Comp $\downarrow$		NC $\uparrow$		Inference Time @ frame 200	Peak Memory
Method		Mean	Med.	Mean	Med.	Mean	Med.	Inference Time @ frame 200	Peak Memory
Spann3R [ 65 ] (3DV’25)		0.225	0.176	0.070	0.015	0.538	0.557	221.054 ms	6.7 GB
CUT3R [ 66 ] (CVPR’25)		0.165	0.107	0.030	0.012	0.551	0.577	102.533 ms	7.6 GB
StreamVGGT (ICLR’26)	None	0.030	0.024	0.021	0.013	0.560	0.561	737.535 ms	31.9 GB
	Window(50)	0.062	0.033	0.028	0.015	0.553	0.554	277.584 ms	20.3 GB
	Window(100)	0.043	0.028	0.023	0.014	0.557	0.557	478.251 ms	30.7 GB

TABLE IX : Comparison of context-pruning strategies for StreamVGGT [ 11 ] on a 200-frame online sequence.

Panel of Figure 13 — Fig. 5 : Qualitative comparison of point cloud reconstruction. From left to right: VGGT [ 1 ] , LiteVGGT [ 5 ] , VGG-T 3 [ 7 ] , FastVGGT [ 4 ] , SparseVGGT [ 9 ] , and BlazeVGGT (Ours). (a) Selected viewpoints within the full reconstructed scenes; colored rectangles highlight local regions that demonstrate BlazeVGGT’s reconstruction quality. (b) Full scenes, demonstrating that BlazeVGGT preserves scene completeness and coherent global structure despite aggressive frame-level pruning of the KV context.

Fig. 6 : Qualitative point-cloud comparison for online reconstruction with StreamVGGT [ 11 ] . Across two scenes, StreamVGGT [ 11 ] with BlazeVGGT preserves coherent scene layouts and object boundaries while using a compact long-term context.

Method	Params	Sintel [ 78 ]		Bonn [ 79 ]		DyCheck [ 80 ]		FPS
Method	Params	Abs Rel $\downarrow$	$\delta<1.25\uparrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$	FPS
Around 100 Frames
Depth alignment
PAGE4D [ 12 ] (ICLR’26)	1.26B	0.357	0.699	0.022	0.723	0.205	0.767	11.18
PAGE4D + BlazeVGGT	1.26B	0.312	0.703	0.088	0.911	0.204	0.774	21.31
Scale-and-shift alignment

TABLE X : Video depth estimation on Sintel [ 78 ] , Bonn [ 79 ] , and DyCheck [ 80 ] under standard and extended sequence lengths. We report Abs Rel,

\delta<1.25

, and inference throughput.

Multi-View 3D Reconstruction

Multi-View 3D Reconstruction recovers scene geometry and appearance from images captured at multiple viewpoints.

See all 277 papers

RegimeVGGT: Layer-Wise Spatially Preserving Redundancy Removal for Visual Geometry Grounded Transformer
Jun 16, 2026Jinhao You, Shuo Lyu, Zhuohang Lyu +53D ViTs Multi-View 3D Reconstruction
University of Pennsylvania · University of California, Irvine · Nanyang Technological University +1
GPA-VGGT: Geometry-Preserving Acceleration of VGGT
Oct 27, 2026Muyeol Sung, Jangwon Kim, Jongwoo LimEfficient Transformer Inference 3D ViTs
Seoul National University
VGGT-Prime: Compute-Adaptive Mixture-of-Heads for Efficient Visual Geometry Transformers
Sep 20, 2026Abteen Arab, Guile Wu, Chengjie Huang +13D ViTs Multi-View 3D Reconstruction
Huawei Noah’s Ark Lab · University of British Columbia

Jun 16, 2026J/K move · Enter open · S save

RegimeVGGT: Layer-Wise Spatially Preserving Redundancy Removal for Visual Geometry Grounded Transformer

Jinhao You, Shuo Lyu, Zhuohang Lyu, Tanxuan Li+4

University of Pennsylvania · University of California, Irvine · Nanyang Technological University+1

3D ViTs Multi-View 3D Reconstruction Efficient ViTs Visual Token Compression

Open paper AskPDF arXiv

Visual Geometry Grounded Transformer (VGGT) recovers dense 3D scene structure from multi-view images in one forward pass, but quadratic cross-frame attention limits its scalability. Existing training-free accelerators reduce computation uniformly along one axis, missing layer heterogeneity. Our spectral, probing, and causal analyses reveal three regimes: shallow layers lack cross-view structure, middle layers drive cross-view alignment, and deep layers are redundant for dense geometry yet their cross-frame attention remains essential for pose. RegimeVGGT applies layer-wise U-shaped compression along two axes: Saliency-Guided Banded Merging protects geometry- and edge-salient tokens, while Selectively Protected K/V Downsampling preserves cross-frame spatial coverage and the pose-critical path through a phase-shifted spatial grid, a reference-frame anchor, and uncompressed camera/register tokens. Training-free, RegimeVGGT achieves a 6.7x speedup over VGGT* at matched reconstruction quality.

Abstract

Multi-View 3D Reconstruction

Explore similar work

RegimeVGGT: Layer-Wise Spatially Preserving Redundancy Removal for Visual Geometry Grounded Transformer

GPA-VGGT: Geometry-Preserving Acceleration of VGGT

VGGT-Prime: Compute-Adaptive Mixture-of-Heads for Efficient Visual Geometry Transformers