We present a scalable 3D reconstruction model that addresses a critical limitation in offline feed-forward methods: their computational and memory requirements grow quadratically w.r.t. the number of input images. Our approach is built on the key insight that this bottleneck stems from the varying-length Key-Value (KV) space representation of scene geometry, which we distill into a fixed-size Multi-Layer Perceptron (MLP) via test-time training. VGG-T3 (Visual Geometry Grounded Test Time Training) scales linearly w.r.t. the number of input views, similar to online models, and reconstructs a 1k image collection in just 54 seconds, achieving a 11.6× speed-up over baselines that rely on softmax attention. Since our method retains global scene aggregation capability, our point map reconstruction error outperforming other linear-time methods by large margins. Finally, we demonstrate visual localization capabilities of our model by querying the scene representation with unseen images.
Figures & tables
Figure 1: VGG-T 3 replaces the global attention block in VGGT (left) with a linear-time alternative based on test-time training (right) to compress the KV space into a fixed-size MLP. We use 3 images for visualization purposes but this scales to arbitrary number of images.
Table 1: Pointmap estimation on dense ( -D ) and sparse ( -S ) split. Overall, we outperform O(n) baseline, TTT3R, and remain competitive w.r.t. O(n2) baselines. FastVGGT code fails on NRGBD-S due to one instance having only two views.
Bonn
KITTI
Sintel
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
VGGT
0.967
0.059
0.964
0.071
0.646
0.300
SparseVGGT
0.968
0.057
0.963
0.070
0.639
0.304
FastVGGT
0.969
0.058
0.964
0.073
0.630
0.307
TTT3R
0.969
0.061
0.818
0.151
0.510
0.469
VGG-T 3 (ours)
0.963
0.063
0.967
0.076
0.581
0.345
Table 2: Video depth estimation. VGG-T 3 outperforms sequential O(n) baseline by a substantial margin and performs on-par with O(n2) baselines.
ScanNet
Sintel
TUM
ATE ↓
RPEr↓
RPEt↓
ATE ↓
RPEr↓
RPEt↓
ATE ↓
RPEr↓
RPEt↓
VGGT
0.035
0.381
0.016
0.172
0.467
0.061
0.012
0.310
0.010
SparseVGGT
0.036
0.394
0.016
0.177
0.552
0.070
0.013
0.316
0.010
FastVGGT
0.035
0.494
0.018
0.158
0.516
0.060
0.013
0.317
0.011
TTT3R
0.063
0.617
0.020
0.196
0.767
0.088
0.025
0.337
0.012
TTT3R (unordered)
0.094
3.942
0.089
0.325
1.462
0.213
0.029
0.652
0.029
Table 3: Camera pose estimation. Our method supports both ordered and unordered input sequences, whereas the other TTT3R performs poorly on unordered inputs. Via sequential processing, TTT3R provides more accurate pose estimates. Best performance on ordered inputs are marked bold , best un-ordered blue .
Figure 6
Figure 3: Runtime ( ↓ ) vs . Chamfer distance ( ↓ ) for collections of size ∈{100,500,1k} on 7scenes dataset. In terms of reconstruction quality (Chamfer distance), we observe a small gap between VGG-T 3 and O(n2) baselines, that narrows with increasing number of images. However, for 1k input, VGGT takes ca. 11 min while VGG-T 3 only needs 58 seconds ( 11.6× speedup). VGG-T 3 scales comparably to TTT3R and does not degrade w.r.t. increasing number of images.
1500 images
2000 images
1 GPU
2 GPUs
4 GPUs
1 GPU
2 GPUs
4 GPUs
TTT3R
90.1
N/A
N/A
126.2
N/A
N/A
VGGT
OOM
1779.3
913.6
OOM
2827.1
1590.2
VGG-T 3 (ours)
173.1
56.8
29.7
230.7
74.8
48.5
Table 4: Reconstruction latency (s) with distributed inference. VGG-T 3 can efficiently process large sequences on a single GPU, and provide linear speed-up via distributed inference.
Figure 4: Pointmap error with increasing number of images when varying the optimizer steps on the TTT objective.
Figure 5: Qualitative comparison. From left to right: VGGT, TTT3R, VGG-T 3 (Ours)
Figure 6: Visual localization examples in Wayspots and 7scenes. Ground truth camera for query image (not used for reconstruction) shown on the left in green, predicted camera and geometry in red.
Figure 7: In-the-wild visual localization. We reconstruct a sequence of the KITTI dataset, then localize a tourist picture that was recorded 7 years later. Note the changes in appearance and composition of the scene.
Figure 8: Waymo sequence reconstructions comparison with VGGT.
Feed-forward transformer models have driven rapid progress in 3D vision, but state-of-the-art methods such as VGGT and π3 have a computational cost that scales quadratically with the number of input images, making them inefficient when applied to large image collections. Sequential-reconstruction approaches reduce this cost but sacrifice reconstruction quality. We introduce ZipMap, a stateful feed-forward model that achieves linear-time, bidirectional 3D reconstruction while matching or surpassing the accuracy of quadratic-time methods. ZipMap employs test-time training layers to zip an entire image collection into a compact hidden scene state in a single forward pass, enabling reconstruction of over 700 frames in under 10 seconds on a single H100 GPU, more than 20× faster than state-of-the-art methods such as VGGT. Moreover, we demonstrate the benefits of having a stateful representation in real-time scene-state querying and its extension to sequential streaming reconstruction.
Haian Jin, Rundi Wu, Tianyuan Zhang +4
Google DeepMind · Cornell University · Massachusetts Institute of Technology
While recent feed-forward 3D reconstruction models accelerate 3D reconstruction by jointly inferring dense geometry and camera poses in a single pass, their reliance on dense attention imposes a quadratic complexity, creating a prohibitive computational bottleneck that severely limits inference speed. To resolve this, we introduce Speed3R, an end-to-end trainable model inspired by the core principle of Structure-from-Motion: that a sparse set of keypoints is sufficient for robust pose estimation. Speed3R features a dual-branch attention mechanism where a compression branch creates a coarse contextual prior to guide a selection branch, which performs fine-grained attention only on the most informative image tokens. This strategy mimics the efficiency of traditional keypoint matching, achieving a remarkable 12.4x inference speedup on 1000-view sequences, while introducing a minimal, controlled trade-off in geometric accuracy. Validated on standard benchmarks with both VGGT and π3 backbones, our method delivers high-quality reconstructions at a fraction of computational cost, paving the way for efficient large-scale scene modeling.
We propose tttLRM, a novel large 3D reconstruction model that leverages a Test-Time Training (TTT) layer to enable long-context, autoregressive 3D reconstruction with linear computational complexity, further scaling the model's capability. Our framework efficiently compresses multiple image observations into the fast weights of the TTT layer, forming an implicit 3D representation in the latent space that can be decoded into various explicit formats, such as Gaussian Splats (GS) for downstream applications. The online learning variant of our model supports progressive 3D reconstruction and refinement from streaming observations. We demonstrate that pretraining on novel view synthesis tasks effectively transfers to explicit 3D modeling, resulting in improved reconstruction quality and faster convergence. Extensive experiments show that our method achieves superior performance in feedforward 3D Gaussian reconstruction compared to state-of-the-art approaches on both objects and scenes.