This paper addresses the task of large-scale 3D scene reconstruction from long video sequences. Recent feed-forward reconstruction models have shown promising results by directly regressing 3D geometry from RGB images without explicit 3D priors or geometric constraints. However, these methods often struggle to maintain reconstruction accuracy and consistency over long sequences due to limited memory capacity and the inability to effectively capture global contextual cues. In contrast, humans can naturally exploit the global understanding of the scene to inform local perception. Motivated by this, we propose a novel neural global context representation that efficiently compresses and retains long-range scene information, enabling the model to leverage extensive contextual cues for enhanced reconstruction accuracy and consistency. The context representation is realized through a set of lightweight neural sub-networks that are rapidly adapted during test time via self-supervised objectives, which substantially increases memory capacity without incurring significant computational overhead. The experiments on multiple large-scale benchmarks, including the KITTI Odometry~\cite{Geiger2012CVPR} and Oxford Spires~\cite{tao2025spires} datasets, demonstrate the effectiveness of our approach in handling ultra-large scenes, achieving leading pose accuracy and state-of-the-art 3D reconstruction accuracy while maintaining efficiency. Code is available at https://zju3dv.github.io/scal3r.
Figures & tables
Figure 2: Overview of Scal3R . Our model takes a long sequence of RGB images as input and reconstructs the 3D scene within a unified inference pipeline. Specifically, the input sequence is divided into overlapping chunks that are processed in parallel across multiple GPUs. Each chunk is processed by our Scal3R backbone, which incorporates our proposed neural global context representation and aggregation mechanism to capture and share global context across the entire sequence. The resulting camera poses and depth maps from all chunks are then aligned and fused to generate the final 3D reconstruction of the scene.
Method
VKITTI2
KITTI
Oxford Spires
Resources
223–837 frames, 52–711 meters
271–4661 frames, 394–5067 meters
351–787 frames, 280–773 meters
KITTI 03/04/10, avg. 758 frames
RRE ↓
RTE ↓
ATE ↓
RRE ↓
RTE ↓
ATE ↓
RRE ↓
RTE ↓
ATE ↓
Memory ↓
Time ↓
FPS ↑
MASt3R-SLAM [ 44 ]
15.81
70.48
78.33
22.42
67.72
191.71
59.67
29.82
29.22
6.74
99.30
7.37
VGGT-SLAM [ 41 ]
12.92
21.27
17.18
33.27
78.95
214.88
55.60
32.14
26.85
10.67
39.72
19.85
StreamVGGT [ 101 ]
13.47
58.07
68.97
24.06
84.46
226.15
71.28
37.14
34.35
6.66
32.61
23.14
STream3R [ 33 ]
13.46
76.06
70.87
24.06
81.63
227.77
71.29
36.65
34.65
4.70
111.23
8.19
Table 1: Camera pose and resource evaluation on Virtual KITTI [ 7 ] , KITTI Odometry [ 22 ] , and Oxford Spires [ 70 ] . We report RRE ( ∘ /100m), RTE (m/100m), and ATE (m). Failed scenes are assigned the worst valid score when computing dataset averages. Methods marked with † require known camera intrinsics. Best and second-best results are shown in bold and underlined .
Figure 3: Camera trajectory comparison on KITTI Odometry [ 22 ] and Oxford Spires [ 70 ] . Scal3R preserves global structure with lower drift, whereas baselines often lose tracking or diverge on long sequences.
Figure 4: Qualitative comparison of point-cloud reconstruction on outdoor and indoor scenes. Scal3R reconstructs large-scale outdoor scenes more reliably and preserves more consistent local geometry indoors.
Method
ETH3D
Oxford Spires
VKITTI2
CD ↓
F1 ↑
CD ↓
F1 ↑
CD ↓
F1 ↑
MASt3R-SLAM [ 44 ]
0.89
0.31
7.78
0.53
17.08
0.33
VGGT-SLAM [ 41 ]
0.78
0.72
10.16
0.22
9.74
0.57
StreamVGGT [ 101 ]
1.86
0.14
12.23
0.25
20.45
0.35
STream3R [ 33 ]
1.81
0.14
12.20
0.25
18.77
0.36
CUT3R [ 80 ]
0.41
0.60
6.93
0.45
5.67
0.39
Table 2: 3D reconstruction evaluation on ETH3D [ 60 ] , Oxford Spires [ 70 ] , and Virtual KITTI [ 7 ] . We report Chamfer Distance (CD) and F1 score. Best and second-best results are shown in bold and underlined .
State Size
Global Context
Variants
RRE ↓
RTE ↓
ATE ↓
Variants
RRE ↓
RTE ↓
ATE ↓
1M size
1.01
1.01
0.99
w/o GCM
1.30
7.03
19.00
2M size
0.95
0.91
0.93
w/o GCS
1.28
7.01
15.80
4M size
0.87
0.84
0.85
Full model
1.17
5.99
13.70
Table 3: Ablation studies. Left: varying GCM state size. Right: ablating global context on a complementary long-sequence setting. The two blocks are not directly comparable. Best and second-best results are shown in bold and underlined .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Camera trajectory comparison. Scal3R preserves global structure with substantially lower drift, whereas baselines frequently lose tracking or diverge, demonstrating our capability of reconstructing large-scale scenarios with high accuracy.
Figure 6: Point-cloud reconstruction comparison. Scal3R produces more accurate large-scale reconstructions for large-scale outdoor environments where baselines often fail, and achieves higher local geometric accuracy and consistency in indoor scenes.
Method
ScanNet++
TUM-RGBD
Waymo
avg. 924 fr.
avg. 926 fr.
avg. 198 fr.
MASt3R-SLAM [ 44 ]
0.47
0.08
7.63
VGGT-SLAM [ 41 ]
0.29
0.12
7.43
StreamVGGT [ 101 ]
1.70
0.63
45.10
STream3R [ 33 ]
1.75
0.63
42.20
CUT3R [ 80 ]
1.27
0.54
9.40
Appendix
Table 4: Additional pose benchmark comparisons. We report ATE (m, lower is better) on three supplementary benchmarks. The best results are in bold , and the second best are underlined .
Frames
150
270
510
990
RPE (m) ↓
0.08
0.08
0.07
0.08
Time (s) ↓
51.19
98.81
195.24
382.80
FPS ↑
2.93
2.73
2.61
2.59
Appendix
Table 5: Runtime scaling with sequence length. Using the same single-GPU evaluation setting as the main-paper resource comparison, we report relative pose error (RPE, m), total inference time, and FPS as the input length increases. Runtime grows smoothly with sequence length while RPE remains stable.
Figure 7: Failure case under abrupt illumination changes. Large appearance shifts within a sequence weaken cross-chunk correspondences and can lead to inaccurate global alignment.