Geometry foundation models estimate camera poses and 3D scene structure from video sequences, but maintaining consistent reconstructions over long horizons remains challenging. Feed-forward models process long sequences as independent, fixed-length frame windows and align their predictions, causing pose and reconstruction errors to accumulate across windows. Streaming models retain information from past frames to inform future predictions, but cannot revise earlier estimates when later observations reveal errors. To address this, we introduce Scaffold MemOry for Revision (SMORE), a framework equipping frozen geometry foundation models with a revisable external memory for long-horizon inference. Inspired by hippocampal associative memory in biological brains, which links spatial context with sensory experience, SMORE associates where an observer is with what it observes, enabling past observations to be stored, retrieved, verified against new evidence, and revised when inconsistencies are detected. We evaluate SMORE with eight competitive geometry models on five datasets spanning camera pose estimation, multi-view depth estimation, 3D point-cloud reconstruction, simultaneous localization and mapping (SLAM), and novel view synthesis. Across all evaluated models, SMORE consistently improves long-sequence pose estimation, reduces multi-view depth error by up to 58%, reduces Chamfer distance for 3D point-cloud reconstruction by up to 43%, and achieves average trajectory error as low as 2.9 cm in long-horizon SLAM. By retaining corrected poses and observations, SMORE further improves the fidelity and spatial consistency of large-scale 3D scene reconstruction. These results establish revisable external memory as a flexible interface for extending pretrained geometry models beyond fixed-context prediction toward iterative, error-correcting inference over long visual sequences.
Figures & tables
Figure 1: Limitations of existing 3D geometry models for long horizon and overview of our SMoRe . A Limitations. Feed-forward models process long sequences in separate windows and subsequently stitch their predictions, causing errors to accumulate at window boundaries and yielding inconsistent camera poses and 3D geometry. B Limitations. Video streaming models retain information across video frames to inform future predictions, but cannot revise earlier estimates when subsequent observations reveal errors, leading to persistent pose drift and distorted geometry over long sequences. C SMoRe Overview. SMoRe introduces four memory operations: Store retains frames, camera poses, and 3D point maps in the content bank, while the scaffold associates spatial and visual information. Retrieve uses the current frame’s visual descriptor to find candidate past frames. Verify jointly processes the current window and retrieved pairs to see whether their past poses support consistent window placement. Revise uses the accepted evidence to update the global placements of affected windows, moving their stored cameras and geometry together.
Figure 2: Effectiveness of SMoRe on camera pose estimation (A) and novel view synthesis (B). A. Camera pose accuracy across sequence lengths on RE10K (left) and CO3Dv2 (right). Each marker compares mean AUC@30 without (x-axis) and with SMoRe (y-axis); colors denote geometry models and marker size denotes the number of input frames, N∈{10,50,100,200} . Points above the diagonal indicate improvement with SMoRe . Crosses indicate cases where the baseline runs out of memory (OOM), while the same model with SMoRe successfully performs camera pose estimation. B. Novel view synthesis from a long camera trajectory in the office scene of the 7-Scenes dataset. VGGT- Ω processes the input keyframes along the trajectory, with four example input frames shown. A held-out target view is synthesized using either the reconstruction obtained by chaining consecutive windows (VGGT- Ω ) or the reconstruction revised by SMoRe (VGGT- Ω + SMoRe ). Red boxes highlight regions shown at higher magnification for visual comparison.
Model
Method
Replica (centimetres)
ScanNet++ (centimetres)
Acc. ↓
Comp. ↓
Overall ↓
Acc. ↓
Comp. ↓
Overall ↓
Feed-forward
DUSt3R
Baseline
12.85 ± 1.15
11.20 ± 1.05
12.02 ± 1.10
32.40 ± 3.85
30.15 ± 3.50
31.27 ± 3.65
+ SMoRe
3.85 ± 0.22
3.24 ± 0.19
3.55 ± 0.20
8.51 ± 0.72
7.88 ± 0.65
8.20 ± 0.68
Fast3R
Baseline
14.15 ± 1.30
12.45 ± 1.20
13.30 ± 1.25
36.50 ± 4.10
34.20 ± 3.90
35.35 ± 4.00
+ SMoRe
4.21 ± 0.25
3.55 ± 0.28
3.88 ± 0.26
9.22 ± 0.85
8.45 ± 0.80
8.84 ± 0.82
MASt3R
Baseline
5.88 ± 0.31
4.75 ± 0.25
5.32 ± 0.27
13.55 ± 1.05
12.82 ± 1.10
13.19 ± 1.08
Table 1: Long-horizon multi-view depth estimation on Replica and 3D point-cloud reconstruction on ScanNet++ (All Frames). We report reconstruction accuracy (Acc.), completeness (Comp.), and overall error in centimetres. Lower is better ( ↓ ). Results are presented as mean ± standard deviation of baselines with and without SMoRe , with the better results for each model shown in bold.
Figure 3: Long-horizon SLAM on 7-Scenes. A. ATE RMSE (metres) compared with calibrated (c) and uncalibrated (unc) SLAM methods; lower is better. The red bar and dotted line indicate the ATE RMSE of uncalibrated VGGT- Ω + SMoRe . B. Stitched 3D point-map reconstruction obtained using camera poses from chained VGGT- Ω versus VGGT- Ω + SMoRe on an example 7-Scenes video sequence. Red boxes highlight a revisited region where SMoRe improves spatial consistency and reconstruction detail. C. Pairwise positional errors between frames for VGGT- Ω and VGGT- Ω + SMoRe ; red boxes highlight temporally distant frame pairs where SMoRe reduces accumulated error. D. Per-frame positional error (metres) over time. Green lines indicate verified revisits, after which SMoRe corrects accumulated trajectory error.
7-Scenes
CO3Dv2
Model
Method
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
VGGT- Ω
Baseline
18.59 ± 0.14
0.588 ± 0.007
0.362 ± 0.005
18.78 ± 0.09
0.533 ± 0.004
0.406 ± 0.002
+\textscSMoRe
19.12 ± 0.14
0.606 ± 0.007
0.350 ± 0.005
19.41 ± 0.08
0.549 ± 0.004
0.380 ± 0.002
VGGT
Baseline
17.70 ± 0.15
0.560 ± 0.007
0.415 ± 0.004
15.97 ± 0.11
0.493 ± 0.004
0.521 ± 0.003
+\textscSMoRe
18.33 ± 0.16
0.575 ± 0.007
0.396 ± 0.005
16.31 ± 0.11
0.501 ± 0.004
0.508 ± 0.003
π3
Baseline
17.24 ± 0.16
0.551 ± 0.007
0.419 ± 0.005
17.42 ± 0.09
0.527 ± 0.004
0.448 ± 0.002
Table 2: Novel view synthesis from long sequences on 7-Scenes ( Shotton et al., 2013 ) and CO3Dv2 ( Reizenstein et al., 2021 ) . Baseline and +\textscSMoRe use separately trained Gaussian decoders under matched training conditions (Appendix I ). Higher PSNR and SSIM and lower LPIPS indicate better performance. Values are mean ± standard error over held-out targets (156 on 7-Scenes, 888 on CO3Dv2). Within each model, the better value of each pair is in bold when the paired mean difference exceeds 0.01 dB (PSNR) or 0.001 (SSIM, LPIPS).
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Controlled diagnostics of the scaffold representation. (a) A stationary orientation ring maintains a localized activity bump; the bump centre represents the encoded angle, while the orange dashed lines show the predicted bump boundaries. (b) When angular motion is supplied, the measured bump velocity closely follows the commanded velocity over the tested range. (c) The spatial phases during a closed two-dimensional path return close to their starting point, illustrating path integration in the periodic scaffold coordinates. (d) Associative recall error as the number of stored states increases. The scaffold configuration with the fitted content return map maintains low error through the largest tested load ( P=160 ), whereas the Hopfield-style control accumulates larger error. These are controlled memory diagnostics and do not measure end-to-end reconstruction accuracy or establish an unrestricted memory-capacity limit.
1,653 clips at N=10 ; 100 clips for the N∈{50,100,200} length study
Multi-view depth
Replica ( Straub et al., 2019 )
18 scenes; N=2000 frames per scene
3D point-cloud reconstruction
ScanNet++ ( Yeshwanth et al., 2023 )
1,006 scenes; N=600−8,000 frames per scene
SLAM
7-Scenes ( Shotton et al., 2013 )
All seven indoor scenes
Novel view synthesis
7-Scenes ( Shotton et al., 2013 )
Seven seq-01 trajectories, 200 keyframes each; every eighth keyframe held out (156 targets); decoders trained on the other 7-Scenes sequences
Appendix
Table S1: Datasets and evaluation protocols. The table summarizes the data used for each downstream task. Detailed sampling and scoring procedures are given below.
Model
Method
RE10K ( N )
CO3Dv2 ( N )
10
50
100
200
10
50
100
200
Feed-forward
DUSt3R
Baseline
65.6 ± 2.8
88.0 ± 0.7
85.9 ± 0.7
82.6 ± 0.9
88.9 ± 1.3
88.8 ± 0.7
88.0 ± 0.6
83.9 ± 0.8
+\textscSMoRe
65.6 ± 2.8
88.0 ± 0.7
85.6 ± 0.8
83.5 ± 0.9
88.9 ± 1.3
88.8 ± 0.7
89.5 ± 0.5
86.6 ± 0.7
Fast3R
Baseline
71.2 ± 2.5
84.3 ± 1.5
86.1 ± 1.3
86.6 ± 1.3
90.9 ± 1.6
88.3 ± 1.6
86.7 ± 1.3
82.4 ± 1.3
+\textscSMoRe
71.2 ± 2.5
84.3 ± 1.5
87.0 ± 1.4
86.5 ± 1.4
90.9 ± 1.6
88.3 ± 1.6
87.1 ± 1.3
84.3 ± 1.3
MASt3R
Baseline
85.5 ± 2.1
95.3 ± 0.2
95.2 ± 0.3
93.8 ± 0.3
93.0 ± 1.3
89.6 ± 1.5
88.5 ± 1.3
86.6 ± 1.1
Appendix
Table S2: Camera pose estimation accuracy on RE10K ( Zhou et al., 2018 ) and CO3Dv2 ( Reizenstein et al., 2021 ) . We report AUC@30 ↑ for increasing numbers of input frames N . Values are mean ± standard deviation across evaluated sequences (CO3Dv2) or clips (RE10K). For each geometry foundation model, Baseline denotes its original inference procedure and +\textscSMoRe denotes the same model equipped with our framework. For StreamVGGT and STream3R, Baseline uses native causal inference, whereas +\textscSMoRe uses our windowed framework. OOM indicates that inference exceeded the tested 46 GB GPU memory budget. Within each model and sequence length, the higher mean is shown in bold; equal means are left unbolded.
Model
Method
chess
fire
heads
office
pumpkin
kitchen
stairs
avg
Calibrated methods
NICER-SLAM
–
0.033
0.069
0.042
0.108
0.200
0.039
0.108
0.086
DROID-SLAM
–
0.036
0.027
0.025
0.066
0.127
0.040
0.026
0.049
MASt3R-SLAM
–
0.053
0.025
0.015
0.097
0.088
0.041
0.011
0.047
GLUEMAP
–
0.036
0.031
0.018
0.079
0.092
0.046
0.051
0.050
TALO
–
0.034
0.021
0.012
0.054
0.061
0.028
0.010
0.031
Appendix
Table S3: Full 7-Scenes SLAM results. We report per-scene and average ATE RMSE (m ↓ ) on the seven 7-Scenes sequences: chess, fire, heads, office, pumpkin, kitchen, and stairs. Lower is better. We compare NICER-SLAM, DROID-SLAM, MASt3R-SLAM, GLUEMAP, TALO, VGGT-SLAM++, ViSTA-SLAM, SLAM-Former, CUT3R, VGGT-SLAM, and VGGSfM ( Zhu et al., 2024 ; Teed and Deng, 2021 ; Murai et al., 2025 ; Pan et al., 2026 ; Zhang et al., 2026a ; Mandal et al., 2026 ; Zhang et al., 2026b ; Yuan et al., 2026 ; Wang et al., 2025b ; Maggio et al., 2025 ; Wang et al., 2024a ) . Calibrated methods receive camera intrinsics, whereas uncalibrated methods do not. A superscript ∗ marks an uncalibrated version of a method that is also reported in a calibrated setting. Our results use frozen VGGT- Ω with 32-frame windows. Within each calibration block, the lowest value in each scene column is shown in bold.
Figure S2: Additional novel view synthesis results on 7-Scenes and CO3Dv2. On 7-Scenes chess (left), SMoRe suppresses ghosting artifacts around the chess pieces, yielding sharper boundaries. On CO3Dv2 (right), the zoomed-in views of the computer mouse demonstrate that SMoRe mitigates edge misalignment and better preserves object contours. Together, these examples highlight consistent improvements in boundary sharpness and geometric structure across diverse scenes. In both panels, columns from left to right display the ground truth, the baseline VGGT- Ω rendering, and our results with SMoRe . Above each comparison group, we show four representative input keyframes out of the eight source views used to render each target. Red boxes indicate the cropped regions shown in the magnified insets.
Figure S3: Gaussian decoder training and novel view synthesis results. (A) Training loss, shown as an exponential moving average, for decoders trained on chained geometry (dashed) and geometry maintained by SMoRe (solid), for all six geometry models. (B) Paired PSNR gains on 7-Scenes, reported as mean ± standard error over all held-out targets and over revisited targets. (C) Mean PSNR for each of the seven 7-Scenes trajectories, comparing Baseline with +\textscSMoRe . Points above the diagonal indicate improvement.
7-Scenes ATE (m) ↓
CO3Dv2 AUC@30 ↑
Configuration
Online
Final
Online
Final
Group A. Memory scaffold
Scaffold address (default)
0.030
0.029
93.4
93.3
Direct DINOv2 cosine
0.035
0.034
91.3
90.9
Group B. Memory sparsity
σ=64 (default)
0.030
0.029
93.4
93.3
Appendix
Table S4: Ablation studies of SMoRe . We vary one design factor at a time while keeping the remaining pipeline fixed. The groups follow the same order as the design analysis in Sec. 3.5 . Except for the window-size study, the common raw references are 0.045 m ATE on 7-Scenes and 88.8 AUC@30 on CO3Dv2. Online denotes the running SMoRe result, while Final includes the final global refinement in Appendix G . For Groups E and F, the value in parentheses after the 7-Scenes Final ATE is the mean number of accepted revisits per sequence. The default configuration uses the memory scaffold, 64 active units in the memory address h , DINOv2, two retrieved pairs, Nh=1024 , a 482 horizontal torus, and W=32 with overlap ratio 0.5.
N=10
N=50
N=100
N=200
Model
Raw/Native
+\textscSMoRe
Raw/Native
+\textscSMoRe
Raw/Native
+\textscSMoRe
Raw/Native
+\textscSMoRe
DUSt3R ( Wang et al., 2024b )
4.9
4.4
OOM
25.8
n/f
28.8
n/f
32.1
Fast3R ( Yang et al., 2025 )
6.3
5.7
11.4
9.1
13.8
9.2
18.7
9.2
MASt3R ( Leroy et al., 2024 )
3.2
3.2
20.7
6.7
n/f
7.6
n/f
8.8
VGGT ( Wang et al., 2025a )
9.1
8.5
11.1
9.5
15.1
9.6
23.9
9.6
VGGT- Ω ( Wang et al., 2026a )
5.7
5.7
8.1
6.8
11.9
6.9
19.5
7.0
Appendix
Table S5: Peak GPU allocation (GB) with increasing sequence length using the CO3Dv2 dataset during the camera pose estimation experiment in Sec. 3.1 . Raw/Native uses each model’s chained inference mode and native inference for streaming models. OOM indicates that the tested 46 GB GPU budget was exceeded; n/f indicates that the run was not feasible under the host-memory or runtime budget.
Model
Method
CPU time (min)
RAM (GB)
GPU (GB)
Compute (PFLOP)
Runtime (min)
DUSt3R ( Wang et al., 2024b )
Baseline (raw)
40.78
16.94
25.8
10.58
12.21
+\textscSMoRe
52.77
17.02
28.8
13.71
17.01
Fast3R ( Yang et al., 2025 )
Baseline (raw)
0.84
7.42
13.9
0.66
0.13
+\textscSMoRe
2.20
8.60
9.2
0.71
0.42
MASt3R ( Leroy et al., 2024 )
Baseline (raw)
36.35
17.23
7.1
9.32
17.79
+\textscSMoRe
37.90
17.29
7.6
11.89
18.49
Appendix
Table S6: Compute and memory at N=100 using the CO3Dv2 dataset during the camera pose estimation experiment in Sec. 3.1 . Measurements use one RTX 6000 Ada GPU with a tested 46 GB GPU-memory budget and batch size one. We report accumulated CPU processing time, peak host RAM, peak GPU allocation, total floating-point computation, and end-to-end runtime. Baseline uses each model’s raw windowed baseline, while the streaming models use their native baseline. STream3R exceeds the GPU-memory budget under native inference and is marked OOM. +\textscSMoRe uses 32-frame windowed inference with online retrieval, verification, and trajectory revision, without final global refinement. For native STream3R, the reported RAM is the peak observed before GPU OOM.
Setting
Value
Defined in
Windowing and retrieval
Default window / overlap
32/16
Sec. 2
Appearance descriptor
DINOv2 ViT-S/14 CLS, 384-D
Sec. 2.2
Retrieval budget
top-5 stored frames
Sec. 2.2
Minimum revisit separation Δ
64 frames
Eq. 3
Appearance threshold τapp
0.5
Eq. 3
Appendix
Table S7: Shared implementation settings. These values define the default SMoRe configuration used in the main experiments unless a task-specific exception is stated explicitly. Spatial quantities used by the scaffold are expressed in median-depth-normalized scene units.
Policy
Retained current-window prediction
Role of the joint pass
Joint (default)
Prediction from current + historical frames
Provides both the retained local prediction and the revisit measurement
Ordinary
Prediction from the current window alone
Used only to measure the historical relationship
Gated
Joint prediction if its distortion is small; otherwise ordinary
Provides the revisit measurement and a candidate local prediction
Appendix
Table S8: Local-prediction policies used in the diagnostic. All three policies can use the joint pass to measure the historical relationship. They differ only in which prediction is retained for the current window before sequential placement and possible trajectory revision.
Local prediction
τdist
7-Scenes ATE (m) ↓
CO3Dv2 AUC@30 ↑
Joint (default)
–
0.030
93.4
Gated
0.5∘
0.030
92.5
Gated
1∘
0.030
93.0
Gated
2∘
0.030
93.2
Gated
5∘
0.030
93.4
Ordinary
–
0.030
92.4
Appendix
Table S9: Local-prediction selection with default retrieval proposals. DINOv2 and scaffold seed 0 are used for seven 7-Scenes trajectories and twelve CO3Dv2 trajectories. Joint always retains the joint-pass prediction, Ordinary always retains the ordinary-pass prediction, and Gated switches between them using Eq. 9 . All metrics are measured online before final global refinement.
Retrieval proposals
Joint
Gated, 1∘
Ordinary
DINOv2, seed 1 ( Oquab et al., 2024 )
0.092
0.029
0.031
CosPlace ( Berton et al., 2022 )
0.263
0.048
0.048
BoQ ( Ali-Bey et al., 2024 )
0.127
0.052
0.051
DINOv2 + SALAD ( Izquierdo and Civera, 2024 )
0.133
0.050
0.049
Appendix
Table S10: Local-prediction isolation on selected 7-Scenes failure cases. Values are mean online ATE (m) ↓ across the seven trajectories. Within each row, the same retrieval proposals are used for all three local-prediction policies. The DINOv2 row uses scaffold seed 1; the remaining rows use the indicated visual descriptor. All gated results use τdist=1∘ .
Visual Geometry Grounded Transformers (VGGT) have set new benchmarks in high-fidelity 3D scene reconstruction. However, as the sequence length increases, these models suffer from catastrophic geometric forgetting and accumulation drift, primarily due to the quadratic complexity of global attention which necessitates truncated temporal windows. To overcome the resulting geometric drift, we present Mamba-VGGT, an enhanced VGGT framework capable of persistent long-range reasoning. Our key contribution is a Sliding Window Mamba (SWM) memory module that maintains an explicit external memory token across temporal windows. This module leverages selective state-space modeling to distill and propagate global geometric priors, effectively bypassing the memory constraints of traditional transformers. To integrate these long-term temporal cues without disrupting the highly optimized spatial features of the pre-trained VGGT, we propose a Zero-Init Spatial Memory Injector. Utilizing zero-convolutional layers, this injector adaptively fuses persistent memory into the patch token stream, ensuring structural stability and seamless feature alignment. Extensive experiments demonstrate that our approach significantly outperforms existing VGGT-based methods in maintaining spatial consistency and reducing trajectory accumulation errors. Our work provides a scalable, linear-complexity solution for geometry-grounded world modeling in extensive 3D environments.
Tianchen Deng, Zhenxiang Xiong, Nailin Wang +4
Shanghai Jiao Tong University · Nanyang Technological University · ETH Zurich +1
We propose LoG-VGGT, a memory-efficient framework for long-sequence 3D reconstruction that balances local temporal modeling with global camera consistency. Instead of relying on full global attention, our method introduces cross-window attention at a small subset of transformer blocks, enabling effective information propagation across adjacent temporal windows while keeping memory usage bounded. To mitigate long-term pose drift, we further design a global camera consistency refinement module, where camera tokens interact with compact register tokens via cross-attention to enforce scene-level constraints across the entire sequence. This design enables joint optimization of camera representations and significantly improves long-horizon pose stability without incurring the high cost of sequence-wide attention. Extensive experiments demonstrate that LoG-VGGT achieves improved depth accuracy and robust camera pose estimation across multiple long-sequence benchmarks, while delivering competitive streaming reconstruction performance.
Jingke Zhou, Chenhang Ma, Zhizhou Zhong +6
Zhejiang University · Fudan University · Peking University +1
Feedforward geometric foundation models achieve strong short-window reconstruction, yet scaling them to minutes-long videos is bottlenecked by quadratic attention complexity or limited effective memory in recurrent designs. We present LoGeR (Long-context Geometric Reconstruction), a novel architecture that scales dense 3D reconstruction to extremely long sequences without post-optimization. LoGeR processes video streams in chunks, leveraging strong bidirectional priors for high-fidelity intra-chunk reasoning. To manage the critical challenge of coherence across chunk boundaries, we propose a learning-based hybrid memory module. This dual-component system combines a parametric Test-Time Training (TTT) memory to anchor the global coordinate frame and prevent scale drift, alongside a non-parametric Sliding Window Attention (SWA) mechanism to preserve uncompressed context for high-precision adjacent alignment. Remarkably, this memory architecture enables LoGeR to be trained on sequences of 128 frames, and generalize up to thousands of frames during inference. Evaluated across standard benchmarks and a newly repurposed VBR dataset with sequences of up to 19k frames, LoGeR substantially outperforms prior state-of-the-art feedforward methods--reducing ATE on KITTI by over 74%--and achieves robust, globally consistent reconstruction over unprecedented horizons.