Zebra3D: Efficient Hybrid Foundation Models For Scalable 3D Reconstruction

Utkarsh Saxena; Ruijing Yang; Mehdi Rezagholizadeh; Akash Haridas; Vikram Appia; Emad Barsoum

cs.CVOct 8, 2026

Zebra3D: Efficient Hybrid Foundation Models For Scalable 3D Reconstruction

Authors: Utkarsh Saxena, Ruijing Yang, Mehdi Rezagholizadeh, Akash Haridas, Vikram Appia, Emad Barsoum

Organizations: Advanced Micro Devices Inc. (AMD)

Abstract

Feed-forward 3D foundation models reconstruct scenes from unposed images, but the quadratic cost of global attention limits their scalability to large view sets. We present Zebra3D, a layerwise sparse-linear hybrid attention framework that improves the efficiency of pretrained 3D foundation models without training from scratch. Across VGGT, HunyuanWorld-Mirror, and $π^3$ , we observe a broadly consistent depth-wise pattern: early stages are relatively tolerant to linear replacement, while sensitive middle-to-late stages tend to exhibit more concentrated attention. Zebra3D exploits this structure by assigning linear-centered attention to tolerant stages and tile-sparse attention to sensitive stages, using landmark summaries to preserve unselected context. Closed-form affine fitting enables training-free conversion, while optional post-conversion training combines task supervision with output and relational feature distillation. Across all three backbones, Zebra3D achieves strong reconstruction and camera-pose accuracy at high attention sparsity, with training-free variants remaining close to their dense counterparts. End-to-end speedups increase with input view count, reaching up to $7.1\times$ at 1,024 views. After post-conversion training, Zebra3D-VGGT reaches 80.5% RealEstate10K AUC@30 at 94% mean attention sparsity, within 1.1 percentage points of dense VGGT fine-tuned on the same data and schedule.

Figures & tables

Figure 1: Zebra3D overview. (a) Sensitivity-guided assignment of linear and tile-sparse attention. Selected tiles retain full softmax interactions; omitted tiles become mean-key/value landmarks. (b) RE10K pose AUC@30 versus end-to-end speedup at $1{,}024$ views.

Panel of Figure 2 — Figure 2: Measurements motivating the Zebra3D design. (a) Global attention reaches $88.0\%$ of VGGT runtime at 512 views. (b) Attention Mass retained by an oracle Top- $20\%$ key/value blocks. (c) Relative output deviation after converting one global-attention stage to linear attention.

Figure 3: Attention feature map across depth (VGGT). Top: attention mass on one calibration scene, with cyan lines at view boundaries. Bottom: cumulative attention mass, with the value at a $20\%$ key budget marked. A linear tolerant stage (1, 22) exhibits more diffuse attention while stage in the sensitive band (14) concentrates on few tokens. More visualizations in Figure 9

Method	Mean sparsity (%) $\uparrow$		7Scenes $\downarrow$		NRGBD $\downarrow$		RE10K $\uparrow$		CO3Dv2 $\uparrow$
	$\bar{s}_{300}$	$\bar{s}_{10}$	Acc. mean	Comp. mean	Acc. mean	Comp. mean	AUC-30	RTA-5	AUC-3	AUC-30
VGGT	0	0	0.017	0.027	0.016	0.011	0.744	0.543	0.566	0.900
VGGT + GT-FT	0	0	0.024	0.031	0.024	0.018	0.816	0.674	0.474	0.885
FastVGGT	88	83	0.015	0.025	0.020	0.013	0.708	0.476	0.412	0.867
SAF3R	84	78	0.018	0.022	0.054	0.022	0.714	0.493	0.504	0.883
RegimeVGGT	90	87	0.016	0.026	0.024	0.016	0.722	0.498	0.445	0.876

Table 1: Reconstruction and pose. Point-map results on 7Scenes/NRGBD (stride 10) and pose on RE10K/CO3Dv2 (10 views).

\bar{s}_{N}

denotes mean attention sparsity at

N

views. Bold/underline: best/second-best non-dense. VGGT + GT-FT is dense VGGT finetuned on the same data as Zebra3D-KD. Full metrics: Appendix E.1 .

Panel of Figure 4 — Figure 4: HYWM quality–sparsity trade-off. Zebra3D varies sparse retention within fixed placements; A3D varies subsampling.

Method	1,000 frames		500 frames		300 frames		100 frames
	CD $\downarrow$	Speedup $\uparrow$	CD $\downarrow$	Speedup $\uparrow$	CD $\downarrow$	Speedup $\uparrow$	CD $\downarrow$	Speedup $\uparrow$
VGGT	0.466	1.00	0.465	1.00	0.451	1.00	0.440	1.00
VGGT + GT-FT	0.503	1.00	0.496	1.00	0.486	1.00	0.474	1.00
FastVGGT	0.464	3.97	0.462	3.40	0.450	2.89	0.435	1.81
RegimeVGGT	0.478	3.50	0.468	3.03	0.453	2.59	0.439	1.70
A3D	0.452	3.83	0.445	3.38	0.438	2.95	0.427	2.00

Table 2: ScanNet-50 long-sequence reconstruction. Chamfer distance (CD;

\downarrow

) and end-to-end speedup. Bold/underline: best/second-best non-dense. VGGT + GT-FT is dense VGGT finetuned on the same data as Zebra3D-KD. See Appendix B.8 for the detailed evaluation protocol.

Figure 5: End-to-end inference speedup relative to dense backbone. At 1,024 frames, Zebra3D reaches

6.2{\times}

on VGGT,

5.5{\times}

on HYWM, and

7.1{\times}

\pi^{3}

, with gains increasing as global attention becomes the dominant cost. Speedups measured on a single MI355X GPU.

Figure 6: Long-sequence reconstruction at $1{,}000$ views on ScanNet-50. Visualization of the views of the predicted point maps, coloured by distance to the ground-truth mesh.

Attention choice	RE10K AUC-30 $\uparrow$	7S Acc. $\downarrow$	NRGBD Acc. $\downarrow$
Frame attention	0.655	0.031	0.051
Linear, no fit	0.789	0.028	0.071
Linear, uncentered fit	0.811	0.024	0.065
Linear-centered + fit	0.836	0.021	0.043

Table 3: Ablations. (a) Linear-attention initialization; (b) sparse attention with landmark summaries; (c) KD objectives for VGGT; (d) layer placement at fixed budget. Additional ablations are reported in Appendix D .

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Method	1,000 frames		500 frames		300 frames		100 frames
	CD $\downarrow$	Speedup $\uparrow$	CD $\downarrow$	Speedup $\uparrow$	CD $\downarrow$	Speedup $\uparrow$	CD $\downarrow$	Speedup $\uparrow$
VGGT	0.466	1.00	0.465	1.00	0.451	1.00	0.440	1.00
MUSt3R	0.958	15.7	0.937	7.07	0.922	3.85	0.897	0.91
Spann3R *	0.605	0.50	0.604	0.27	0.588	0.18	0.567	0.09
Fast3R	0.588	1.75	0.558	1.64	0.547	1.52	0.524	1.38
Zebra3D-KD (0.7L/0.8S)	0.460	5.60	0.449	4.59	0.447	2.51	0.439	1.86

Appendix

Table 4: Direct multi-view reconstruction on ScanNet-50. Chamfer distance (CD;

\downarrow

) and end-to-end speedup relative to dense VGGT on one MI355X. Bold/underline denote the best/second-best non-dense results. Spann3R * was evaluated without torch.compile .

Method	7Scenes			NRGBD			RE10K			CO3Dv2
	Acc. $\downarrow$	Comp. $\downarrow$	NC1 $\uparrow$	Acc. $\downarrow$	Comp. $\downarrow$	NC1 $\uparrow$	AUC-30 $\uparrow$	RRA-30 $\uparrow$	RTA-30 $\uparrow$	AUC-3 $\uparrow$	AUC-5 $\uparrow$	AUC-15 $\uparrow$	AUC-30 $\uparrow$
Fast3R	0.035	0.043	0.522	0.074	0.030	0.539	0.605	0.984	0.825	0.226	0.357	0.642	0.767
MUSt3R	0.045	0.321	0.547	0.070	0.127	0.654	0.796	1.000	0.934	0.356	0.496	0.742	0.839
Spann3R	0.124	0.110	0.553	0.248	0.173	0.578	0.292	0.991	0.542	0.006	0.025	0.208	0.412
Zebra3D-VGGT (0.5L, 0.6S)	0.017	0.027	0.606	0.018	0.013	0.746	0.737	0.999	0.919	0.554	0.665	0.834	0.897
Zebra3D-HYWM (0.5L, 0.6S)	0.013	0.025	0.608	0.019	0.014	0.744	0.856	0.999	0.953	0.400	0.541	0.774	0.862

Appendix

Table 5: Direct multi-view comparison on the main benchmarks. Point-map accuracy, completeness, and NC1 on 7Scenes and NRGBD; camera-pose metrics on RE10K and CO3Dv2. Accuracy and completeness are lower-is-better; all other metrics are higher-is-better. Zebra3D uses the training-free 0.5L/0.6S configuration for each backbone.

Figure 7: ScanNet-50 quality–speed trade-off at

1{,}000

frames. Chamfer distance versus end-to-end speedup relative to dense VGGT. Lower Chamfer distance and higher speedup are preferred.

Backbone	Variant	Linear-centered stages	Tile-sparse stages
VGGT	0.5L	1–6, 8, 18, 20–23	0, 7, 9–17, 19
	0.7L	0–9, 17–23	10–16
HYWM	0.5L	1–2, 4–6, 8–9, 18, 20–23	0, 3, 7, 10–17, 19
	0.7L	0–10, 17–18, 20–23	11–16, 19
$\pi^{3}$	0.5L	0–2, 4–9	3, 10–17
	0.7L	0–10, 16–17	11–15

Appendix

Table 6: Layer-role assignments for the all-efficient Zebra3D variants.

Panel of Figure 8 — Figure 8: Additional background measurements for HYWM (top) and Pi3 (bottom). Global-attention runtime reaches $80.6\%$ for HYWM and $91.9\%$ for Pi3 at 512 frames. The middle column reports attention mass retained by oracle selection of the highest-mass $20\%$ of $64{\times}64$ token blocks. The right column reports relative final-output deviation after replacing one global-attention stage with fitted linear-centered attention, averaged over 11 held-out validation scenes. Shading marks the sensitive middle bands: stages 11–16 for HYWM and 11–15 for Pi3.

Figure 9: Tile-level teacher attention for eight VGGT global stages under the protocol of Figure 3 , spanning both linear bands and the tile-sparse band. The transition into concentrated, stripe-structured attention at stage $11$ and out of it after stage $15$ matches the 0.7L assignment in Table 6 .

		RealEstate10K Camera Pose							CO3Dv2 Camera Pose
Backbone	Placement	RRA-5 $\uparrow$	RRA-15 $\uparrow$	RRA-30 $\uparrow$	RTA-5 $\uparrow$	RTA-15 $\uparrow$	RTA-30 $\uparrow$	AUC-30 $\uparrow$	AUC-3 $\uparrow$	AUC-5 $\uparrow$	AUC-15 $\uparrow$	AUC-30 $\uparrow$
VGGT	Sensitivity-guided (ours)	0.935	0.995	0.999	0.508	0.832	0.916	0.730	0.535	0.650	0.827	0.892
	Back-sparse	0.901	0.984	0.993	0.313	0.676	0.831	0.590	0.052	0.111	0.355	0.553
	Front-sparse	0.541	0.878	0.961	0.041	0.218	0.453	0.208	0.000	0.000	0.004	0.023
	Uniform	0.485	0.842	0.948	0.017	0.116	0.294	0.115	0.000	0.000	0.006	0.030
	Reversed	0.383	0.755	0.910	0.016	0.119	0.312	0.114	0.000	0.000	0.002	0.014

Appendix

Table 7: Layer-role placement ablation on RealEstate10K and CO3Dv2 camera pose at

0.5

0.8

S. Best per backbone in bold .

Calibration dataset	VGGT	HYWM	$\pi^{3}$	Mean
7Scenes	0.9322	0.8965	0.8308	0.8865
DTU	0.8852	0.9278	0.8369	0.8833
Re10K	0.9183	0.7765	0.8328	0.8425
NRGBD	0.8878	0.8635	0.8225	0.8579
CO3Dv2	0.9365	0.9235	0.7874	0.8825

Appendix

Table 8: Spearman correlation between layer-sensitivity rankings obtained from each calibration dataset and the reference Zebra3D rankings.

	0.5L				0.7L
Calibration dataset	VGGT	HYWM	$\pi^{3}$	Mean	VGGT	HYWM	$\pi^{3}$	Mean
7Scenes	91.7%	83.3%	88.9%	88.0%	91.7%	100.0%	100.0%	97.2%
DTU	83.3%	91.7%	77.8%	84.3%	91.7%	91.7%	100.0%	94.4%
Re10K	91.7%	75.0%	88.9%	85.2%	91.7%	100.0%	100.0%	97.2%
NRGBD	83.3%	83.3%	100.0%	88.9%	91.7%	100.0%	100.0%	97.2%
CO3Dv2	83.3%	83.3%	77.8%	81.5%	91.7%	100.0%	88.9%	93.5%

Appendix

Table 9: Agreement with the reference 0.5L and 0.7L operator assignments, measured as the fraction of stages receiving the same linear or tile-sparse label.

Configuration	Attention choice	Acc. mean $\downarrow$	Acc. med. $\downarrow$	Comp. mean $\downarrow$	Comp. med. $\downarrow$	NC1 mean $\uparrow$	NC1 med. $\uparrow$	NC2 mean $\uparrow$	NC2 med. $\uparrow$
VGGT (0.5L/0.8S)	Frame attention	0.019	0.007	0.025	0.009	0.605	0.664	0.600	0.654
	Linear, no fit	0.018	0.007	0.027	0.012	0.606	0.665	0.602	0.658
	Linear, uncentered fit	0.018	0.006	0.028	0.012	0.605	0.665	0.604	0.661
	Linear-centered + fit	0.018	0.006	0.027	0.011	0.605	0.664	0.604	0.661
VGGT (0.7L/0.8S)	Frame attention	0.099	0.062	0.033	0.010	0.609	0.674	0.591	0.641
	Linear, no fit	0.018	0.007	0.025	0.010	0.606	0.666	0.601	0.656

Appendix

Table 10: Linear-attention ablation on 7Scenes.

Configuration	Attention choice	Acc. mean $\downarrow$	Acc. med. $\downarrow$	Comp. mean $\downarrow$	Comp. med. $\downarrow$	NC1 mean $\uparrow$	NC1 med. $\uparrow$	NC2 mean $\uparrow$	NC2 med. $\uparrow$
VGGT (0.5L/0.8S)	Frame attention	0.020	0.013	0.010	0.003	0.730	0.881	0.713	0.866
	Linear, no fit	0.019	0.011	0.012	0.003	0.737	0.886	0.745	0.899
	Linear, uncentered fit	0.017	0.010	0.011	0.003	0.743	0.891	0.757	0.906
	Linear-centered + fit	0.021	0.013	0.014	0.004	0.739	0.883	0.744	0.881
VGGT (0.7L/0.8S)	Frame attention	0.201	0.143	0.041	0.011	0.672	0.829	0.695	0.857
	Linear, no fit	0.036	0.020	0.019	0.007	0.719	0.875	0.722	0.878

Appendix

Table 11: Linear-attention ablation on Neural RGB-D.

Configuration	Attention choice	RRA-5 $\uparrow$	RRA-15 $\uparrow$	RRA-30 $\uparrow$	RTA-5 $\uparrow$	RTA-15 $\uparrow$	RTA-30 $\uparrow$	AUC-30 $\uparrow$
VGGT (0.5L/0.8S)	Frame attention	0.932	0.994	0.999	0.481	0.818	0.907	0.714
	Linear, no fit	0.928	0.994	0.999	0.471	0.822	0.912	0.714
	Linear, uncentered fit	0.928	0.995	0.999	0.475	0.820	0.910	0.713
	Linear-centered + fit	0.935	0.995	0.999	0.508	0.832	0.916	0.730
VGGT (0.7L/0.8S)	Frame attention	0.909	0.994	0.999	0.309	0.682	0.846	0.596
	Linear, no fit	0.921	0.994	0.999	0.440	0.819	0.915	0.707

Appendix

Table 12: Linear-attention ablation on RealEstate10K camera pose.

7Scenes
Mixer	Acc. mean $\downarrow$	Acc. med. $\downarrow$	Comp. mean $\downarrow$	Comp. med. $\downarrow$	NC1 mean $\uparrow$	NC1 med. $\uparrow$	NC2 mean $\uparrow$	NC2 med. $\uparrow$
Mamba-2	0.014	0.006	0.023	0.009	0.609	0.670	0.604	0.662
Linear-centered	0.013	0.005	0.023	0.009	0.607	0.667	0.606	0.664
Gated Delta Net	0.014	0.005	0.022	0.008	0.609	0.670	0.607	0.666
Sparse Linear Attn	0.013	0.005	0.024	0.009	0.608	0.668	0.606	0.665
Neural RGB-D

Appendix

Table 13: Training-based linear-attention ablation on 7Scenes [ 28 ] and Neural RGB-D [ 3 ] .

	RealEstate10K							CO3Dv2
Mixer	RRA-5 $\uparrow$	RRA-15 $\uparrow$	RRA-30 $\uparrow$	RTA-5 $\uparrow$	RTA-15 $\uparrow$	RTA-30 $\uparrow$	AUC-30 $\uparrow$	AUC-3 $\uparrow$	AUC-5 $\uparrow$	AUC-15 $\uparrow$	AUC-30 $\uparrow$
Mamba-2	0.985	0.998	1.000	0.746	0.907	0.952	0.845	0.345	0.496	0.752	0.848
Linear-centered	0.985	0.998	1.000	0.762	0.912	0.953	0.852	0.330	0.482	0.748	0.849
Gated Delta Net	0.984	0.998	1.000	0.711	0.900	0.948	0.828	0.305	0.461	0.735	0.839
Sparse Linear Attn	0.987	0.998	1.000	0.741	0.898	0.944	0.838	0.325	0.478	0.745	0.846

Appendix

Table 14: Training-based linear-attention ablation on RealEstate10K [ 47 ] and CO3Dv2 [ 21 ] camera pose.

Configuration	Attention choice	Acc. mean $\downarrow$	Acc. med. $\downarrow$	Comp. mean $\downarrow$	Comp. med. $\downarrow$	NC1 mean $\uparrow$	NC1 med. $\uparrow$	NC2 mean $\uparrow$	NC2 med. $\uparrow$
VGGT (0.5L/0.75S)	SGA	0.016	0.005	0.026	0.010	0.605	0.664	0.605	0.664
	Block sparse	0.018	0.006	0.026	0.009	0.606	0.666	0.603	0.660
	Tile sparse	0.018	0.006	0.027	0.010	0.604	0.664	0.604	0.661
	Block sparse + landmark	0.018	0.006	0.027	0.010	0.606	0.665	0.603	0.660
	Tile sparse + landmark	0.017	0.006	0.027	0.011	0.605	0.664	0.605	0.662
VGGT (0.7L/0.75S)	SGA	0.016	0.006	0.023	0.007	0.607	0.668	0.604	0.661

Appendix

Table 15: Sparse-attention ablation on 7Scenes.

Configuration	Attention choice	Acc. mean $\downarrow$	Acc. med. $\downarrow$	Comp. mean $\downarrow$	Comp. med. $\downarrow$	NC1 mean $\uparrow$	NC1 med. $\uparrow$	NC2 mean $\uparrow$	NC2 med. $\uparrow$
VGGT (0.5L/0.75S)	SGA	0.029	0.022	0.030	0.009	0.713	0.845	0.720	0.859
	Block sparse	0.038	0.023	0.016	0.004	0.708	0.853	0.714	0.864
	Tile sparse	0.039	0.034	0.020	0.006	0.708	0.847	0.707	0.854
	Block sparse + landmark	0.034	0.020	0.015	0.004	0.734	0.884	0.747	0.898
	Tile sparse + landmark	0.018	0.011	0.013	0.003	0.743	0.888	0.750	0.892
VGGT (0.7L/0.75S)	SGA	0.040	0.030	0.028	0.009	0.716	0.864	0.704	0.843

Appendix

Table 16: Sparse-attention ablation on Neural RGB-D.

Configuration	Attention choice	RRA-5 $\uparrow$	RRA-15 $\uparrow$	RRA-30 $\uparrow$	RTA-5 $\uparrow$	RTA-15 $\uparrow$	RTA-30 $\uparrow$	AUC-30 $\uparrow$
VGGT (0.5L/0.75S)	SGA	0.949	0.996	0.999	0.519	0.841	0.924	0.739
	Block sparse	0.945	0.993	0.998	0.466	0.811	0.906	0.709
	Tile sparse	0.943	0.995	0.999	0.492	0.824	0.910	0.722
	Block sparse + landmark	0.939	0.993	0.998	0.487	0.825	0.912	0.722
	Tile sparse + landmark	0.938	0.995	0.999	0.516	0.834	0.917	0.733
VGGT (0.7L/0.75S)	SGA	0.943	0.996	1.000	0.471	0.836	0.925	0.727

Appendix

Table 17: Sparse-attention ablation on RealEstate10K camera pose.

7Scenes
Training objective	Acc. mean $\downarrow$	Acc. med. $\downarrow$	Comp. mean $\downarrow$	Comp. med. $\downarrow$	NC1 mean $\uparrow$	NC1 med. $\uparrow$	NC2 mean $\uparrow$	NC2 med. $\uparrow$
GT only	0.023	0.008	0.023	0.007	0.612	0.678	0.597	0.650
KD only	0.023	0.010	0.020	0.004	0.611	0.674	0.580	0.623
GT + output KD	0.023	0.010	0.021	0.005	0.620	0.690	0.596	0.648
GT + feature KD	0.029	0.012	0.023	0.007	0.616	0.683	0.592	0.641
RelKD	0.015	0.005	0.026	0.011	0.608	0.668	0.606	0.665

Appendix

Table 18: Knowledge-distillation ablation on 7Scenes [ 28 ] and Neural RGB-D [ 3 ] .

Training objective	RRA-5 $\uparrow$	RRA-15 $\uparrow$	RRA-30 $\uparrow$	RTA-5 $\uparrow$	RTA-15 $\uparrow$	RTA-30 $\uparrow$	AUC-30 $\uparrow$
GT only	0.980	0.997	1.000	0.586	0.870	0.939	0.777
KD only	0.961	0.998	1.000	0.449	0.823	0.916	0.716
GT + output KD	0.973	0.997	0.999	0.488	0.838	0.922	0.736
GT + feature KD	0.977	0.998	1.000	0.542	0.854	0.929	0.756
RelKD	0.975	0.998	1.000	0.649	0.895	0.950	0.805

Appendix

Table 19: Knowledge-distillation ablation on RealEstate10K camera pose [ 47 ] .

Method	Mean Attn. Sparsity $\uparrow$	7Scenes						NRGBD
		Acc $\downarrow$		Comp $\downarrow$		NC1 $\uparrow$		Acc $\downarrow$		Comp $\downarrow$		NC1 $\uparrow$
		Mean	Med.	Mean	Med.	Mean	Med.	Mean	Med.	Mean	Med.	Mean	Med.
VGGT	0%	0.017	0.005	0.027	0.011	0.607	0.667	0.016	0.009	0.011	0.003	0.761	0.901
VGGT + GT-only FT	0%	0.024	0.008	0.031	0.014	0.604	0.662	0.024	0.013	0.018	0.006	0.754	0.890
FastVGGT	88%	0.015	0.005	0.025	0.009	0.607	0.667	0.020	0.013	0.013	0.004	0.731	0.871
SAF3R	84%	0.018	0.008	0.022	0.005	0.611	0.675	0.054	0.034	0.022	0.005	0.694	0.847

Appendix

Table 20: Full quantitative point-map reconstruction results on 7Scenes and NRGBD at stride 10. Dense backbones are excluded when marking the best ( bold ) and second-best ( underlined ) non-dense results. In the shorthand

x

y

x

is the fraction of model layers using linear attention and

y

is the sparsity of attention rows in the remaining layers. VGGT + GT-only FT is dense VGGT fine-tuned with the same data, schedule, and ground-truth supervision as Zebra3D-KD.

Method	Mean Attn. Sparsity $\uparrow$	RealEstate10K Camera Pose							CO3Dv2 Camera Pose
		RRA-5 $\uparrow$	RRA-15 $\uparrow$	RRA-30 $\uparrow$	RTA-5 $\uparrow$	RTA-15 $\uparrow$	RTA-30 $\uparrow$	AUC-30 $\uparrow$	AUC-3 $\uparrow$	AUC-5 $\uparrow$	AUC-15 $\uparrow$	AUC-30 $\uparrow$
VGGT	0%	0.947	0.996	0.999	0.543	0.843	0.920	0.744	0.566	0.675	0.839	0.900
VGGT + GT-only FT	0%	0.980	0.998	1.000	0.674	0.901	0.951	0.816	0.474	0.609	0.812	0.885
FastVGGT	83%	0.936	0.994	0.999	0.476	0.808	0.904	0.708	0.412	0.555	0.784	0.867
SAF3R	78%	0.946	0.995	0.999	0.493	0.809	0.902	0.714	0.504	0.626	0.814	0.883
RegimeVGGT	87%	0.939	0.995	0.999	0.498	0.824	0.911	0.722	0.445	0.582	0.797	0.876

Appendix

Table 21: Full camera-pose estimation results on RealEstate10K and CO3Dv2 using 10 input frames. Higher is better. Dense backbones are excluded when marking the best ( bold ) and second-best ( underlined ) non-dense results; dashes denote unavailable measurements. In the shorthand

x

y

x

is the fraction of model layers using linear attention and

y

is the sparsity of attention rows in the remaining layers. VGGT + GT-only FT is dense VGGT fine-tuned with the same data, schedule, and ground-truth supervision as Zebra3D-KD.

Panel of Figure 10 — Figure 10: Performance versus mean attention sparsity for VGGT and $\pi^{3}$ and mean attention sparsity is computed at 300 input frames. Zebra3D curves vary sparse retention within the 0.5L, 0.6L, and 0.7L placements; gray A3D curves vary the subsampling configuration. Higher is better for RealEstate10K AUC-30, while lower is better for 7Scenes and Neural RGB-D point-map accuracy error. The gold arrow points toward each panel’s preferred corner.

HYWM
	Tanks and Temples			ETH3D			DTU
Method	Acc $\downarrow$	Comp $\downarrow$	NC1 $\uparrow$	Acc $\downarrow$	Comp $\downarrow$	NC1 $\uparrow$	Acc $\downarrow$	Comp $\downarrow$	NC1 $\uparrow$
Dense	0.9865	0.4437	0.6992	0.2591	0.1857	0.8198	0.9622	1.4920	0.6549
A3D	1.4735	1.7308	0.6681	0.4634	0.4119	0.7723	0.9990	1.4659	0.6512
Zebra3D (0.5L/0.6S)	1.2380	0.9749	0.6817	0.2437	0.1681	0.8191	0.9563	1.5110	0.6565
Zebra3D (0.5L/0.8S)	1.3271	0.9866	0.6790	0.2629	0.1830	0.8151	0.9766	1.5020	0.6559

Appendix

Table 22: Wide-baseline point-map accuracy on Tanks and Temples (large-scale outdoor and indoor scenes) and ETH3D (indoor and outdoor DSLR captures), with DTU (object-centric turntable rig) retained as a narrow-baseline control. Each entry is averaged over scenes and view counts

N\in\{8,16,32,64\}

. Accuracy and completeness are lower-is-better, in m for Tanks and Temples and ETH3D and in mm for DTU; NC1 is higher is better. Dense rows are excluded when marking the best ( bold ) result in each column within a backbone block.

Figure 11: Long-sequence reconstruction at $1{,}000$ views on four ScanNet-50 scenes, coloured by distance to the ground-truth mesh (blue $0$ m, red $\geq 0.30$ m). Protocol is identical to Figure 6 .

Panel of Figure 12 — Figure 12: Reconstructions on scenes outside the training data. From top: $40$ consecutive frames of the KITTI driving sequence 2011_09_26_drive_0023 (camera 02); $40$ frames each of the Neural RGB-D scenes complete_kitchen , morning_apartment and whiteroom . None of these is in the training data of any model shown; VGGT’s includes only the synthetic Virtual KITTI. The first column is the first input frame, and every method is rendered from one virtual camera placed behind and above it. The same lowest-confidence $40\%$ and farthest $3\%$ of pixels are removed from every method, so each keeps nearly the same number of points, and each point is drawn as a disc sized to the surface its source pixel covers.

Explore similar work

Mar 18, 2026

S-VGGT: Structure-Aware Subscene Decomposition for Scalable 3D Foundation Models

Feed-forward 3D foundation models face a key challenge: the quadratic computational cost introduced by global attention, which severely limits scalability as input length increases. Concurrent acceleration methods, such as token merging, operate at the token level. While they offer local savings, the required nearest-neighbor searches introduce undesirable overhead. Consequently, these techniques fail to tackle the fundamental issue of structural redundancy dominant in dense capture data. In this work, we introduce S-VGGT, a novel approach that addresses redundancy at the structural frame level, drastically shifting the optimization focus. We first leverage the initial features to build a dense scene graph, which characterizes structural scene redundancy and guides the subsequent scene partitioning. Using this graph, we softly assign frames to a small number of subscenes, guaranteeing balanced groups and smooth geometric transitions. The core innovation lies in designing the subscenes to share a common reference frame, establishing a parallel geometric bridge that enables independent and highly efficient processing without explicit geometric alignment. This structural reorganization provides strong intrinsic acceleration by cutting the global attention cost at its source. Crucially, S-VGGT is entirely orthogonal to token-level acceleration methods, allowing the two to be seamlessly combined for compounded speedups without compromising reconstruction fidelity. Code is available at https://github.com/Powertony102/S-VGGT.

Xinze Li, Pengxu Chen, Yiyuan Wang +2

Beijing Normal-Hong Kong Baptist University · Jilin University · Hong Kong Baptist University +1

3D Foundation Models Efficient Attention Feed-Forward 3D Reconstruction 3D Reconstruction 3D Scene Understanding

Ask PDF

Feb 26, 2026

VGG-T $^3$ : Offline Feed-Forward 3D Reconstruction at Scale

We present a scalable 3D reconstruction model that addresses a critical limitation in offline feed-forward methods: their computational and memory requirements grow quadratically w.r.t. the number of input images. Our approach is built on the key insight that this bottleneck stems from the varying-length Key-Value (KV) space representation of scene geometry, which we distill into a fixed-size Multi-Layer Perceptron (MLP) via test-time training. VGG-T

^3

(Visual Geometry Grounded Test Time Training) scales linearly w.r.t. the number of input views, similar to online models, and reconstructs a

1k

image collection in just

54

seconds, achieving a

11.6\times

speed-up over baselines that rely on softmax attention. Since our method retains global scene aggregation capability, our point map reconstruction error outperforming other linear-time methods by large margins. Finally, we demonstrate visual localization capabilities of our model by querying the scene representation with unseen images.

Sven Elflein, Ruilong Li, Sérgio Agostinho +4

NVIDIA · Vector Institute · University of Toronto

3D Reconstruction Feed-Forward 3D Reconstruction 3D Scene Representation

Ask PDF

Mar 4, 2026

ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training

Feed-forward transformer models have driven rapid progress in 3D vision, but state-of-the-art methods such as VGGT and

π^3

have a computational cost that scales quadratically with the number of input images, making them inefficient when applied to large image collections. Sequential-reconstruction approaches reduce this cost but sacrifice reconstruction quality. We introduce ZipMap, a stateful feed-forward model that achieves linear-time, bidirectional 3D reconstruction while matching or surpassing the accuracy of quadratic-time methods. ZipMap employs test-time training layers to zip an entire image collection into a compact hidden scene state in a single forward pass, enabling reconstruction of over 700 frames in under 10 seconds on a single H100 GPU, more than

20\times

faster than state-of-the-art methods such as VGGT. Moreover, we demonstrate the benefits of having a stateful representation in real-time scene-state querying and its extension to sequential streaming reconstruction.

Haian Jin, Rundi Wu, Tianyuan Zhang +4

Google DeepMind · Cornell University · Massachusetts Institute of Technology

3D Reconstruction Multi-View 3D Reconstruction Feed-Forward 3D Reconstruction Streaming 3D Reconstruction

Ask PDF

Abstract

Explore similar work

S-VGGT: Structure-Aware Subscene Decomposition for Scalable 3D Foundation Models

VGG-T3^33: Offline Feed-Forward 3D Reconstruction at Scale

ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training

VGG-T $^3$ : Offline Feed-Forward 3D Reconstruction at Scale