VGG-T$^3$: Offline Feed-Forward 3D Reconstruction at Scale

Sven Elflein; Ruilong Li; Sérgio Agostinho; Zan Gojcic; Laura Leal-Taixé; Qunjie Zhou; Aljosa Osep

cs.CVFeb 26, 2026

VGG-T $^3$ : Offline Feed-Forward 3D Reconstruction at Scale

Authors: Sven Elflein, Ruilong Li, Sérgio Agostinho, Zan Gojcic, Laura Leal-Taixé, Qunjie Zhou, Aljosa Osep

Organizations: NVIDIA · Vector Institute · University of Toronto

Abstract

We present a scalable 3D reconstruction model that addresses a critical limitation in offline feed-forward methods: their computational and memory requirements grow quadratically w.r.t. the number of input images. Our approach is built on the key insight that this bottleneck stems from the varying-length Key-Value (KV) space representation of scene geometry, which we distill into a fixed-size Multi-Layer Perceptron (MLP) via test-time training. VGG-T $^3$ (Visual Geometry Grounded Test Time Training) scales linearly w.r.t. the number of input views, similar to online models, and reconstructs a $1k$ image collection in just $54$ seconds, achieving a $11.6\times$ speed-up over baselines that rely on softmax attention. Since our method retains global scene aggregation capability, our point map reconstruction error outperforming other linear-time methods by large margins. Finally, we demonstrate visual localization capabilities of our model by querying the scene representation with unseen images.

Figures & tables

Panel of Figure 1 — Figure 1: VGG-T 3 replaces the global attention block in VGGT (left) with a linear-time alternative based on test-time training (right) to compress the KV space into a fixed-size MLP. We use 3 images for visualization purposes but this scales to arbitrary number of images.

Panel of Figure 2 — Figure 2: Sequence-length generalization analysis.

		7scenes-D		7scenes-S		DTU		ETH3D		NRBGD-D		NRBGD-S
		CD $\downarrow$	NC $\uparrow$	CD $\downarrow$	NC $\uparrow$	CD $\downarrow$	NC $\uparrow$	CD $\downarrow$	NC $\uparrow$	CD $\downarrow$	NC $\uparrow$	CD $\downarrow$	NC $\uparrow$
$O(n^{2})$	VGGT	0.024	0.668	0.054	0.731	1.537	0.676	0.279	0.855	0.014	0.897	0.055	0.894
	SparseVGGT	0.023	0.665	0.094	0.694	1.541	0.675	0.327	0.836	0.018	0.876	0.079	0.859
	FastVGGT	0.021	0.662	0.065	0.719	1.683	0.672	0.594	0.775	0.033	0.841	-	-
$O(n)$	TTT3R	0.035	0.666	0.129	0.685	5.708	0.672	0.885	0.733	0.071	0.811	0.094	0.819
$O(n)$	VGG-T 3 (ours)	0.030	0.679	0.107	0.716	1.654	0.685	0.480	0.789	0.029	0.867	0.056	0.893

Table 1: Pointmap estimation on dense ( -D ) and sparse ( -S ) split. Overall, we outperform

O(n)

baseline, TTT3R, and remain competitive w.r.t.

O(n^{2})

baselines. FastVGGT code fails on NRGBD-S due to one instance having only two views.

	Bonn		KITTI		Sintel
	$\delta<1.25$ $\uparrow$	Abs Rel $\downarrow$	$\delta<1.25$ $\uparrow$	Abs Rel $\downarrow$	$\delta<1.25$ $\uparrow$	Abs Rel $\downarrow$
VGGT	0.967	0.059	0.964	0.071	0.646	0.300
SparseVGGT	0.968	0.057	0.963	0.070	0.639	0.304
FastVGGT	0.969	0.058	0.964	0.073	0.630	0.307
TTT3R	0.969	0.061	0.818	0.151	0.510	0.469
VGG-T 3 (ours)	0.963	0.063	0.967	0.076	0.581	0.345

Table 2: Video depth estimation. VGG-T 3 outperforms sequential

O(n)

baseline by a substantial margin and performs on-par with

O(n^{2})

baselines.

	ScanNet			Sintel			TUM
	ATE $\downarrow$	$\text{RPE}_{r}$ $\downarrow$	$\text{RPE}_{t}$ $\downarrow$	ATE $\downarrow$	$\text{RPE}_{r}$ $\downarrow$	$\text{RPE}_{t}$ $\downarrow$	ATE $\downarrow$	$\text{RPE}_{r}$ $\downarrow$	$\text{RPE}_{t}$ $\downarrow$
VGGT	0.035	0.381	0.016	0.172	0.467	0.061	0.012	0.310	0.010
SparseVGGT	0.036	0.394	0.016	0.177	0.552	0.070	0.013	0.316	0.010
FastVGGT	0.035	0.494	0.018	0.158	0.516	0.060	0.013	0.317	0.011
TTT3R	0.063	0.617	0.020	0.196	0.767	0.088	0.025	0.337	0.012
TTT3R (unordered)	0.094	3.942	0.089	0.325	1.462	0.213	0.029	0.652	0.029

Table 3: Camera pose estimation. Our method supports both ordered and unordered input sequences, whereas the other TTT3R performs poorly on unordered inputs. Via sequential processing, TTT3R provides more accurate pose estimates. Best performance on ordered inputs are marked bold , best un-ordered blue .

Figure 3: Runtime (

\downarrow

) vs . Chamfer distance (

\downarrow

) for collections of size

\in\{100,500,1k\}

on 7scenes dataset. In terms of reconstruction quality (Chamfer distance), we observe a small gap between VGG-T 3 and

O(n^{2})

baselines, that narrows with increasing number of images. However, for

1k

input, VGGT takes ca.

11

min while VGG-T 3 only needs

58

seconds (

11.6\times

speedup). VGG-T 3 scales comparably to TTT3R and does not degrade w.r.t. increasing number of images.

	1500 images			2000 images
	1 GPU	2 GPUs	4 GPUs	1 GPU	2 GPUs	4 GPUs
TTT3R	90.1	N/A	N/A	126.2	N/A	N/A
VGGT	OOM	1779.3	913.6	OOM	2827.1	1590.2
VGG-T 3 (ours)	173.1	56.8	29.7	230.7	74.8	48.5

Table 4: Reconstruction latency (s) with distributed inference. VGG-T 3 can efficiently process large sequences on a single GPU, and provide linear speed-up via distributed inference.

Figure 4: Pointmap error with increasing number of images when varying the optimizer steps on the TTT objective.

Panel of Figure 5 — Figure 5: Qualitative comparison. From left to right: VGGT, TTT3R, VGG-T 3 (Ours)

Panel of Figure 6 — Figure 6: Visual localization examples in Wayspots and 7scenes. Ground truth camera for query image (not used for reconstruction) shown on the left in green, predicted camera and geometry in red.

Figure 7: In-the-wild visual localization. We reconstruct a sequence of the KITTI dataset, then localize a tourist picture that was recorded 7 years later. Note the changes in appearance and composition of the scene.

Panel of Figure 8 — Figure 8: Waymo sequence reconstructions comparison with VGGT.

Explore similar work

ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training
Mar 4, 2026Haian Jin, Rundi Wu, Tianyuan Zhang +43D Reconstruction Multi-View 3D Reconstruction
Google DeepMind · Cornell University · Massachusetts Institute of Technology
Speed3R: Sparse Feed-forward 3D Reconstruction Models
Mar 9, 2026Weining Ren, Xiao Tan, Kai Han3D Reconstruction Feed-Forward 3D Reconstruction
The University of Hong Kong · Baidu AMU
tttLRM: Test-Time Training for Long Context and Autoregressive 3D Reconstruction
Feb 23, 2026Chen Wang, Hao Tan, Wang Yifan +63D Reconstruction Feed-Forward 3D Reconstruction
University of Pennsylvania · Adobe Research · UCI

Mar 4, 2026J/K move · Enter open · S save

ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training

Haian Jin, Rundi Wu, Tianyuan Zhang, Ruiqi Gao+3

Google DeepMind · Cornell University · Massachusetts Institute of Technology

3D Reconstruction Multi-View 3D Reconstruction Feed-Forward 3D Reconstruction Streaming 3D Reconstruction

Open paper AskPDF arXiv

Feed-forward transformer models have driven rapid progress in 3D vision, but state-of-the-art methods such as VGGT and

π^3

have a computational cost that scales quadratically with the number of input images, making them inefficient when applied to large image collections. Sequential-reconstruction approaches reduce this cost but sacrifice reconstruction quality. We introduce ZipMap, a stateful feed-forward model that achieves linear-time, bidirectional 3D reconstruction while matching or surpassing the accuracy of quadratic-time methods. ZipMap employs test-time training layers to zip an entire image collection into a compact hidden scene state in a single forward pass, enabling reconstruction of over 700 frames in under 10 seconds on a single H100 GPU, more than

20\times

faster than state-of-the-art methods such as VGGT. Moreover, we demonstrate the benefits of having a stateful representation in real-time scene-state querying and its extension to sequential streaming reconstruction.

Abstract

Explore similar work

ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training

Speed3R: Sparse Feed-forward 3D Reconstruction Models

tttLRM: Test-Time Training for Long Context and Autoregressive 3D Reconstruction