Streaming 3D reconstruction with recurrent states enables long-video scene understanding under bounded memory, but turns reconstruction into a state-evolution problem: each incoming frame updates a compact scene memory on which all subsequent predictions depend, so unreliable updates overwrite historical context and accumulate into long-horizon geometric drift. We analyze CUT3R-style recurrent updates through a direction--magnitude decomposition, in which the magnitude sets the update gain, i.e., how strongly an observation is written into the state, and the direction encodes the geometric intent, i.e., where the update drives the state. This decomposition provides a unified view of CUT3R and its variants, revealing that existing training-free extensions perform magnitude-only control: they can attenuate an unreliable update but cannot correct its direction. We propose HIST3R, a training-free framework built on CUT3R that controls both components using history. HIST3R rectifies the update direction using locally consistent historical states, and modulates the update magnitude according to the visual discrepancy between the current observation and its history, suppressing redundant updates while preserving complementary evidence. Across camera pose estimation, 3D reconstruction, and video depth estimation, HIST3R consistently improves long-horizon performance while retaining the memory footprint and inference speed of CUT3R. With 1000 input frames, it reduces ATE by 23.87% on ScanNet and 33.65% on TUM-Dynamics relative to the strongest competing method; with 500 frames, it improves median reconstruction accuracy over the strongest CUT3R-based variant by 56.52% on 7-Scenes and 33.96% on NRGBD.
Figures & tables
Figure 1: A unified direction–magnitude view of recurrent state updates for long-sequence streaming 3D reconstruction. Update magnitude and direction correspond to update gain and geometric intent, respectively. CUT3R [ 23 ] directly commits the candidate state, while magnitude-only CUT3R variants, represented by TTT3R [ 24 ] , rescale the candidate update magnitude but keep its direction fixed. HIST3R controls both components by rectifying the update direction and modulating the update magnitude, enabling more reliable long-horizon state evolution.
Figure 2: Overview of HIST3R. HIST3R maintains a sliding history buffer of past recurrent states and visual summaries. It decomposes the CUT3R candidate update into direction and magnitude to control geometric intent and update gain, respectively. Historical states rectify the update direction, while observation–history visual discrepancy modulates the update magnitude. The two components are combined into the final recurrent state update.
Figure 3: Camera pose estimation on ScanNet [ 45 ] (left) and TUM-Dynamics [ 46 ] (right). OOM denotes out-of-memory under an 80GB GPU memory budget.
Method
Input
7-Scenes
NRGBD
Acc. ↓
Comp. ↓
NC ↑
Acc. ↓
Comp. ↓
NC ↑
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
StreamVGGT
300
OOM
OOM
OOM
OOM
OOM
OOM
OOM
OOM
OOM
OOM
OOM
OOM
InfiniteVGGT
0.042
0.017
0.026
0.007
0.570
0.607
0.054
0.031
0.025
0.005
0.645
0.748
Point3R
0.046
0.025
0.033
0.014
0.564
0.597
0.073
0.040
0.013
0.004
0.614
0.687
CUT3R
0.126
0.085
0.070
0.030
0.543
0.564
0.215
0.118
0.064
0.013
0.570
0.607
Table 1: 3D reconstruction results on 7-Scenes [ 47 ] and NRGBD [ 48 ] . Best results are shown in bold, and second-best results are underlined.
Figure 4: Metric depth evaluation on Bonn [ 49 ] . We exclude VGGT-based methods that do not support metric depth prediction.
Table 2: Ablation of direction–magnitude decomposition. We evaluate direction rectification, magnitude modulation, and their combination on camera pose estimation, scale-aligned video depth estimation, and 3D reconstruction, all under the 300-frame setting. AR denotes AbsRel, δ denotes δ<1.25 , and reconstruction Acc./Comp. are reported as mean accuracy/completeness.
Direction Rectification: St=St−1+mtdt
Design
Cos. Agg.
Align. Gate
Dir. Only
TUM ATE ↓
KITTI AR ↓
KITTI δ↑
Protocol ( k=8 )
✓
✓
✓
0.024
0.107
0.917
✗
✓
✓
0.026
0.110
0.895
✓
✗
✓
0.037
0.119
0.871
✓
✓
✗
0.028
0.123
0.863
Window size k
k=2
0.044
0.114
0.896
Table 3: Design ablations of direction rectification (left) and magnitude modulation (right), evaluated under the 200-frame setting with scale-aligned depth. The check marks indicate whether cosine-weighted aggregation, the alignment gate αt , and direction-only rectification are applied.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Empirical local structure in CUT3R and HIST3R state evolution. We visualize recurrent state evolution on one representative sequence from each of 7-Scenes [ 47 ] , KITTI [ 50 ] , NRGBD [ 48 ] , and ScanNet [ 45 ] . In each subplot, the left panel shows the cosine affinity between recurrent states over time, while the right panel shows PCA analysis of state variations. CUT3R recurrent states naturally exhibit strong local affinity and low-dimensional variation, and HIST3R further accentuates this structure through history-guided rectification.
Method
TUM ATE ↓
KITTI AbsRel ↓
CUT3R
0.148
0.131
TTT3R
0.094
0.118
TTSA3R
0.084
0.117
Direction rect.
0.067 ± 0.008
0.118 ± 0.000
Magnitude mod.
0.093 ± 0.025
0.120 ± 0.001
Direction rect. (default) + Magnitude mod.
0.066 ± 0.009
0.117 ± 0.001
Appendix
Table 4: Hyperparameter sensitivity. Mean and standard deviation over all tested gate configurations of each component. For the last row, direction rectification uses the default setting while magnitude modulation varies. TUM-Dynamics ATE is evaluated at 1000 frames and KITTI scale-aligned AbsRel at 500 frames.
Method
ScanNet ATE ↓
Bonn AbsRel ↓
Direction rect. only
0.354
0.104
TTT3R
0.356
0.108
TTT3R + Direction rect.
0.322
0.103
TTSA3R
0.354
0.103
TTSA3R + Direction rect.
0.337
0.097
Appendix
Table 5: Direction rectification as a plug-in component. We replace only the update direction of TTT3R and TTSA3R with the rectified direction of HIST3R while keeping their original magnitude controllers. ScanNet ATE is evaluated at 1000 frames and Bonn metric-depth AbsRel at 300 frames.
Figure 7: Streaming efficiency on TUM-Dynamics [ 46 ] . Peak allocated GPU memory (left) and inference speed measured by FPS (right) are reported under increasing input sequence length.
Method
Input
7-Scenes (Stride 200)
NRGBD (Stride 500)
Acc. ↓
Comp. ↓
NC ↑
Acc. ↓
Comp. ↓
NC ↑
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
CUT3R
Sparse View
0.120
0.047
0.156
0.030
0.725
0.831
0.100
0.036
0.078
0.031
0.834
0.970
TTT3R
0.156
0.054
0.195
0.046
0.719
0.824
0.105
0.032
0.080
0.027
0.835
0.970
TTSA3R
0.152
0.054
0.189
0.039
0.718
0.821
0.107
0.033
0.082
0.028
0.834
0.968
HIST3R
0.147
0.049
0.176
0.031
0.725
0.834
0.104
0.031
0.080
0.026
0.836
0.970
Appendix
Table 6: Sparse-view 3D reconstruction results on 7-Scenes [ 47 ] and NRGBD [ 48 ] . Following CUT3R [ 23 ] , we sparsely sample images with stride 200 for 7-Scenes and stride 500 for NRGBD. Best results are shown in bold, and second-best results are underlined.
Sintel (50 frames)
TUM-dynamics (90 frames)
ScanNet (90 frames)
Method
Online
ATE ↓
RPE trans ↓
RPE rot ↓
ATE ↓
RPE trans ↓
RPE rot ↓
ATE ↓
RPE trans ↓
RPE rot ↓
Robust-CVD [ 54 ]
✗
0.360
0.154
3.443
0.153
0.026
3.528
0.227
0.064
7.374
CasualSAM [ 55 ]
✗
0.141
0.035
0.615
0.071
0.010
1.712
0.158
0.034
1.618
DUSt3R [ 11 ]
✗
0.417
0.250
5.796
0.083
0.017
3.567
0.081
0.028
0.784
MASt3R [ 12 ]
✗
0.185
0.060
1.496
0.038
0.012
0.448
0.078
0.020
0.475
MonST3R [ 28 ]
✗
0.111
0.044
0.869
0.098
0.019
0.935
0.077
0.018
0.529
Appendix
Table 7: Evaluation on Camera Pose Estimation - Short Sequence on Sintel [ 53 ] , TUM-dynamics [ 46 ] , and ScanNet [ 45 ] datasets. HIST3R achieves the best overall performance among online methods and remains competitive with most offline baselines, despite processing frames incrementally rather than reasoning over the full sequence at once.
Sintel (50 frames)
BONN (110 frames)
KITTI (110 frames)
Alignment
Method
Online
Abs Rel ↓
δ < 1.25↑
Abs Rel ↓
δ < 1.25↑
Abs Rel ↓
δ < 1.25↑
Per-sequence Scale
DUSt3R [ 11 ]
✗
0.656
45.2
0.155
83.3
0.144
81.3
MASt3R [ 12 ]
✗
0.641
43.9
0.252
70.1
0.183
74.5
MonST3R [ 28 ]
✗
0.378
55.8
0.067
96.3
0.168
74.4
Easi3R [ 32 ]
✗
0.377
55.9
0.059
97.0
0.102
91.2
AETHER [ 52 ]
✗
0.324
50.2
0.273
59.4
0.056
97.8
Appendix
Table 8: Evaluation of video depth estimation on short sequences on Sintel [ 53 ] , Bonn [ 49 ] , and KITTI [ 50 ] . We report scale-invariant relative depth, aligned by a per-sequence scale, and metric-scale absolute depth accuracy. HIST3R achieves the strongest metric-scale performance, while remaining competitive with leading online methods under scale-aligned evaluation on Bonn and KITTI.
Figure 8: Qualitative comparison with implicit-memory-based methods on 7-Scenes. CUT3R exhibits clear long-horizon degradation, while HIST3R substantially improves reconstruction coherence and reduces accumulated artifacts.
Figure 9: Qualitative comparison with implicit-memory-based methods on NRGBD. TTT3R partially improves over CUT3R, but still exhibits severe accumulated artifacts and collapsed scene geometry, whereas HIST3R produces substantially more coherent and stable reconstructions.
Figure 10: Qualitative comparison with explicit-memory-based methods on 7-Scenes. Point3R exhibits noticeable geometric distortion, while InfiniteVGGT remains competitive with HIST3R but still produces slightly blurred reconstructions in some regions.
Figure 11: Qualitative comparison with explicit-memory-based methods on NRGBD. HIST3R and InfiniteVGGT produce more stable reconstruction quality than Point3R, with cleaner local geometry and fewer visible distortions.
Figure 12: Qualitative trajectory results on ScanNet. Blue curves denote predicted trajectories, and red dashed curves denote ground-truth trajectories. Explicit-memory-based methods exhibit noticeable pose drift over long sequences. TTT3R produces more stable trajectories, but still shows a clear gap compared with HIST3R, which better preserves long-horizon trajectory consistency.
Figure 13: Qualitative trajectory results on TUM-Dynamics. HIST3R shows a clear advantage over all compared methods, preserving substantially better long-horizon trajectory consistency.
Streaming 3D reconstruction incrementally builds geometrically consistent scene maps from incoming images or video under tight memory and latency limits.