Feedforward geometric foundation models achieve strong short-window reconstruction, yet scaling them to minutes-long videos is bottlenecked by quadratic attention complexity or limited effective memory in recurrent designs. We present LoGeR (Long-context Geometric Reconstruction), a novel architecture that scales dense 3D reconstruction to extremely long sequences without post-optimization. LoGeR processes video streams in chunks, leveraging strong bidirectional priors for high-fidelity intra-chunk reasoning. To manage the critical challenge of coherence across chunk boundaries, we propose a learning-based hybrid memory module. This dual-component system combines a parametric Test-Time Training (TTT) memory to anchor the global coordinate frame and prevent scale drift, alongside a non-parametric Sliding Window Attention (SWA) mechanism to preserve uncompressed context for high-precision adjacent alignment. Remarkably, this memory architecture enables LoGeR to be trained on sequences of 128 frames, and generalize up to thousands of frames during inference. Evaluated across standard benchmarks and a newly repurposed VBR dataset with sequences of up to 19k frames, LoGeR substantially outperforms prior state-of-the-art feedforward methods--reducing ATE on KITTI by over 74%--and achieves robust, globally consistent reconstruction over unprecedented horizons.
Figures & tables
Mechanism
Compute Cost
Local Context
Global Context
Full Attention
O(N2)
Lossless
Lossless
Sliding Window Attn.
O(N)
Lossless
Limited
TTT / Linear Attn.
O(N)
Compressed
Compressed
Ours (Hybrid Memory)
O(N)
Lossless
Compressed
Table 1: Architectural trade-offs in sequence modeling. Our hybrid memory module achieves a balance, preserving lossless local geometric details while maintaining global structural consistency at a linear computational cost with respect to sequence length.
Figure 2: Overview of a single block of our hybrid memory module. We process the input sequence in consecutive chunks of frames. While each block utilizes frame and bidirectional attention from prior work, we introduce new components to effectively propagate information across the entire sequence. Specifically, we incorporate Sliding Window Attention to improve consistency between neighboring chunks, and Test-Time Training layers to maintain long-range, global consistency across all chunks.
Figure 3: Comparison of different methods across varying sequence lengths and scene scales. Although FastVGGT is able to process a larger number of frames during inference, it fails completely on large-scale scenes, highlighting the inherent “data wall” of models trained strictly on short-context bubbles. In contrast, LoGeR breaks both the context and data walls by pairing a hybrid memory architecture with diverse long-horizon training data.
Methods
00
01
02
03
04
05
06
07
08
09
10
Avg.
num. of frames, scale
4542, 3.7km
1101, 2.5km
4661, 5.1km
801, 0.6km
271, 0.4km
2761, 2.2km
1101, 1.2km
1101, 0.7km
4071, 3.2km
1591, 1.7km
1201, 0.9km
-
Contains Loop?
✓
×
✓
×
×
✓
✓
✓
×
✓
×
-
Opt.-based
DROID-SLAM ( Teed & Deng, 2021 )
92.10
344.60
107.61
2.38
1.00
118.50
62.47
21.78
161.60
72.32
118.70
100.28
DPV-SLAM ( Lipson et al., 2024 )
112.80
11.50
123.53
2.50
0.81
57.80
54.86
18.77
110.49
76.66
13.65
53.03
DPV-SLAM++ ( Lipson et al., 2024 )
8.30
11.86
39.64
2.50
0.78
5.74
11.60
1.52
110.90
76.70
13.70
25.75
VGGT-Long ( Deng et al., 2025 )
8.67
121.17
32.08
6.12
4.23
8.31
5.34
4.63
53.10
41.99
18.37
27.64
Table 2: Comparison of Absolute Trajectory Error (ATE ↓ , m) on KITTI. The top and bottom blocks denote optimization-based and feedforward methods, respectively. Bold and underline indicate the best and second-best performance among feedforward methods. LoGeR* denotes a variant of our method with feedforward pose alignment for both training and inference.
Figure 4: Quantitative results on our proposed VBR ( Brizi et al., 2024 ) evaluation showing results on very long sequences spanning from 1,000 to 19,000 frames. Our methods achieve 55.2% more accurate results than prior methods.
Figure 5: Qualitative camera trajectories on the VBR dataset. LoGeR accurately preserves global scale and trajectory over very long sequences, closely matching the ground truth where prior methods suffer from severe drift.
Figure 7
Figure 8: Qualitative results on ScanNet and TUM-Dynamics. While the proposed Pi3-Chunk baseline yields slightly better pose metrics on small-scale TUM sequences (unlike its severe drift on longer trajectories), LoGeR produces visually superior reconstructions. As highlighted, LoGeR accurately recovers structural details, avoiding the severe distortions and geometric artifacts present in baselines.
Figure 9: Q uantitative results on camera pose evaluation , demonstrating that both our proposed baseline and LoGeR substantially outperform prior work, with 80.0% and 66.1% relative gains on ScanNet and TUM datasets, respectively.
Figure 10: Ablation of disabling SWA or TTT at inference time. SWA ensures geometric consistency between adjacent chunks (local consistency), and lacking it causes noticeable local misalignment artifacts. Meanwhile, TTT maintains the global context (global consistency); without it, the model suffers from severe trajectory drift over long horizons.
ScanNet
TUM
Method
500f
1000f
500f
1000f
LoGeR
0.087
0.107
0.033
0.050
w/o TTT
0.108
0.162
0.043
0.079
w/o SWA
0.115
0.143
0.039
0.053
All datasets
0.087
0.107
0.033
0.050
w/o 5 large datasets
0.102
0.156
0.050
0.072
Table 3: Comparison of baseline and ablation results across ScanNet (subset) and TUM datasets. We report the results of ATE ( ↓ ) on both 500 and 1000 frames of sequences.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Percentage (%)
DL3DV ( Ling et al., 2024 )
17.89
TartanAirV2 ( Patel et al., 2025 )
17.89
OmniWorld-Game (subset) ( Zhou et al., 2025 )
17.89
ARKitScenes ( Baruch et al., 2021 )
10.44
TartanAir ( Wang et al., 2020 )
8.94
Waymo ( Schwall et al., 2020 )
6.71
Appendix
Table 4: Training data mixture. We list the datasets used in our final training configuration along with their relative sampling percentages. Navigation and large-scale scene datasets are heavily weighted to encourage long-range geometric reconstruction.
Chunk Size
Speed (FPS)
Memory (GB)
64
9.3
27.2
48
10.6
22.3
32
12.1
18.1
Appendix
Table 5: Inference efficiency. Measured on a single NVIDIA A100 (40GB) GPU over a 500-frame sequence with varying chunk sizes. As the number of frames increases, the speed improves slightly, while memory consumption remains constant.
Methods
colosseo_0
campus_0
campus_1
pincio_0
spagna_0
diag_0
ciampino_1
Avg.
num. of frames, scale
8815, 1.45km
12042, 2.73km
11671, 2.95km
11142, 1.27km
14141, 1.56km
10021, 1.02km
18846, 5.20km
-
Opt.
VGGT-SLAM ( Maggio et al., 2025 )
102.92
110.77
89.69
72.98
62.67
35.62
144.02
88.38
VGGT-Long w/o LC ( Deng et al., 2025 )
86.46
132.66
115.65
64.40
57.39
33.66
186.13
96.62
VGGT-Long ( Deng et al., 2025 )
45.73
132.68
115.65
64.40
58.49
33.66
187.97
91.23
Feedforward
InfiniteVGGT ( Yuan et al., 2026 )
90.02
136.83
116.69
77.72
63.47
33.91
202.11
102.96
CUT3R ( Wang et al., 2025b )
88.16
46.39
48.00
52.87
47.98
30.18
187.58
71.59
Appendix
Table 6: Comparison of Absolute Trajectory Error (ATE ↓ , m) on VBR. The top and bottom blocks denote optimization-based and feedforward methods, respectively. Bold and underline indicate the best and second-best performance among feedforward methods.
Figure 11: Quantitative results of video depth estimation on the Bonn dataset. Both our proposed baseline (Pi3-Chunk) and LoGeR significantly outperform prior works. Note that VGGT and StreamVGGT are omitted after certain frames due to out-of-memory (OOM).
Figure 12: 3D reconstruction results on the 7-Scenes dataset , uniformly sampling 100, 500, and 1k frames. For the evluation of 1k frames, our method achieves a 90.3% and 72.1% performance boost (error reduction) with comparable runtimes against concurrent works TTT3R ( Chen et al., 2026 ) and VGG-T 3 ( Elflein et al., 2026 ) , respectively. LoGeR also achieves 84.1% faster inference speed compared to VGGT with 31.0% better performance.
Figure 13: Qualitative results on KITTI. Ours shows more accurate long trajectory estimation.
Figure 14: Qualitative results on large-scale outdoor and indoor scenes , showing minutes-long long-horizon reconstruction results.
Visual Geometry Grounded Transformer (VGGT) advances 3D reconstruction via scalable Transformer architecture, but the quadratic complexity of global attention prevents long context application. StreamVGGT enables streaming with causal attention, yet its KV cache grows linearly with frames, causing memory overflow and quality degradation. We present RetrieveVGGT, a training-free framework, which formulates context construction for VGGT as a retrieval problem. By retrieving a fixed number of relevant frames at each step, VGGT maintains a controllable memory budget, which is close to its training context length. Interestingly, we find that the similarity between current frame queries and cached history frame keys at the first global attention layer of VGGT is already a strong indicator of relevance, eliminating the need for additional learned scoring. To enhance information diversity similar to a recommender system, we propose Segment Sampling so that the retrieval spans distinct relevant segments rather than a single high-similarity region. We design a pose-aware spatial memory mechanism that organizes history frames according to their already estimated camera poses, enabling location-aware retrieval. Extensive experiments demonstrate that RetrieveVGGT achieves state-of-the-art performance, outperforming StreamVGGT, TTT3R, and InfiniteVGGT while maintaining constant memory usage regardless of sequence length. Code is available at https://github.com/zzctmd/RetrieveVGGT.
Zichen Zou, Xiaosong Jia, Zuxuan Wu +1
Institute of Trustworthy Embodied AI (TEAI), Fudan University · 2Shanghai Key Laboratory of Multimodal Embodied AI
We propose LoG-VGGT, a memory-efficient framework for long-sequence 3D reconstruction that balances local temporal modeling with global camera consistency. Instead of relying on full global attention, our method introduces cross-window attention at a small subset of transformer blocks, enabling effective information propagation across adjacent temporal windows while keeping memory usage bounded. To mitigate long-term pose drift, we further design a global camera consistency refinement module, where camera tokens interact with compact register tokens via cross-attention to enforce scene-level constraints across the entire sequence. This design enables joint optimization of camera representations and significantly improves long-horizon pose stability without incurring the high cost of sequence-wide attention. Extensive experiments demonstrate that LoG-VGGT achieves improved depth accuracy and robust camera pose estimation across multiple long-sequence benchmarks, while delivering competitive streaming reconstruction performance.
Jingke Zhou, Chenhang Ma, Zhizhou Zhong +6
Zhejiang University · Fudan University · Peking University +1
Online 3D reconstruction requires estimating camera pose and scene geometry under strict causal and bounded-memory constraints. Existing methods often suffer from drift, jitter, or collapse on long sequences. We trace these failures to a fundamental mismatch. Streaming geometry is inherently temporally heterogeneous, with evidence ranging from short-lived correspondences to persistent global scale. However, current architectures impose uniform and pathological influence patterns. For example, sliding windows enforce hard cutoffs, while ungated recurrence and causal attention cause cache saturation and spike-like attention sinks. To resolve this, we formalize geometric propagation as an evidence influence kernel and propose HorizonStream, a long-horizon Transformer that explicitly factorizes this kernel. For the long-range temporal factor, Geometric Linear Attention learns channel-wise decay rates to enable bounded, multi-timescale propagation of geometric evidence. For the short-range spatial factor, Geometric Local Attention with Spatiotemporal RoPE performs reliable 3D matching while suppressing attention sinks. Finally, Metric Readout Tokens recover stable scale and rigid pose directly from the persistent geometric state. Extensive experiments show that HorizonStream, trained on only 48-frame clips, generalizes stably to sequences exceeding 10,000\ frames with constant memory and linear time, achieving state-of-the-art streaming 3D reconstruction performance. Project Page: https://3dagentworld.github.io/horizonstream/