SSR: A Training-Free Approach for Streaming 3D Reconstruction
Authors: Hui Deng, Yuxin Mao, Yuxin He, Yuchao Dai
Organizations: Shool of Electronics and Information, Northwestern Polytechnical University and Shaanxi Key Laboratory of Information Acquisition and Processing, Xi’an, China.
Streaming 3D reconstruction demands long-horizon state updates under strict latency constraints, yet stateful recurrent models often suffer from geometric drift as errors accumulate over time. We revisit this problem from a Grassmannian manifold perspective: the latent persistent state can be viewed as a subspace representation, i.e., a point evolving on a Grassmannian manifold, where temporal coherence implies the state trajectory should remain on (or near) this manifold.Based on this view, we propose Self-expressive Sequence Regularization (SSR), a plug-and-play, training-free operator that enforces Grassmannian sequence regularity during inference.Given a window of historical states, SSR computes an analytical affinity matrix via the self-expressive property and uses it to regularize the current update, effectively pulling noisy predictions back toward the manifold-consistent trajectory with minimal overhead. Experiments on long-sequence benchmarks demonstrate that SSR consistently reduces drift and improves reconstruction quality across multiple streaming 3D reconstruction tasks.
Figures & tables
Figure 1: Illustration of the Self-expressive Sequence Regularization (SSR). We introduce a training-free regularization scheme for CUT3R. Specifically, we refine the frame-wise states calculated by foundation models through a sliding-window reconstruction process. By leveraging the affinity between temporal states, our method achieves sequence regularization without introducing learnable parameters, making it directly applicable to off-the-shelf pre-trained foundation model.
Sintel
Bonn
KITTI
Alignment
Method
Venue
Online
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
Per-sequence scale
DUSt3R
CVPR’24
✗
0.656
45.2
0.155
83.3
0.144
81.3
MASt3R
ECCV’24
✗
0.641
43.9
0.252
70.1
0.183
74.5
MonSTR3R
ICLR’25
✗
0.378
55.8
0.067
96.3
0.168
74.4
Easi3R
ICCV’25
✗
0.377
55.9
0.059
97.0
0.102
91.2
AETHER
ICCV’25
✗
0.324
50.2
0.273
59.4
0.056
97.8
Table 1: Video depth estimation . Benchmarking on synthetic (Sintel), dynamic (Bonn), and outdoor (KITTI) sequences. We evaluate the proposed SSR framework against state-of-the-art methods using both scale-invariant and metric depth protocols. Our training-free method demonstrates the state-of-the-art/highly competitive performance across diverse settings. Compared to another training-free method, TTT3R, our approach also exhibits superior performance.
Sintel
TUM-dynamics
ScanNet
Method
ATE ↓
RPE trans ↓
RPE rot ↓
ATE ↓
RPE trans ↓
RPE rot ↓
ATE ↓
RPE trans ↓
RPE rot ↓
DUSt3R
0.290
0.132
7.869
0.140
3.286
7.869
0.246
0.108
8.210
MASt3R
0.185
0.060
1.496
0.038
0.012
0.448
0.078
0.020
0.475
MonST3R
0.111
0.044
0.869
0.098
0.019
0.935
0.077
0.018
0.529
Easi3R
0.110
0.042
0.758
0.105
0.022
1.064
0.061
0.017
0.525
AETHER
0.189
0.054
0.694
0.092
0.012
1.106
0.176
0.028
1.204
Table 2: Pose estimation . Overall, our method demonstrates the best comprehensive performance among online methods. On datasets with highly dynamics and long sequences(TUM-D/ScanNet), our method demonstrates superior performance in camera trajectory estimation, reflecting its ability to suppress cumulative drift and adapt to dynamic scenes.
7 Scenes
NRGBD
Acc↓
Comp↓
NC↑
Acc↓
Comp↓
NC↑
Method
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
StreamVGGT
0.132
0.058
0.116
0.042
0.749
0.863
0.085
0.044
0.079
0.038
0.862
0.986
XStreamVGGT
0.142
0.068
0.125
0.048
0.734
0.848
0.085
0.049
0.075
0.038
0.850
0.986
DUSt3R-GA
0.146
0.077
0.181
0.067
0.736
0.839
0.144
0.019
0.154
0.018
0.870
0.982
MASt3R-GA
0.185
0.081
0.180
0.069
0.701
0.792
0.085
0.033
0.063
0.028
0.794
0.928
Table 3: Sparse view 3D reconstruction . When input data is extremely sparse and sequence lengths are very short, our method experiences performance degradation to the point where it underperforms baseline methods. This occurs because the proposed method aims to aggregate meaningful information from historical context, whereas such extreme input settings directly contradict this objective. We explore this issue in depth in the analysis section.
7 Scenes
NRGBD
Acc↓
Comp↓
NC↑
Acc↓
Comp↓
NC↑
Method
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
Mean
Med.
CUT3R
0.032
0.017
0.032
0.009
0.641
0 .720
0.090
0.045
0.035
0.011
0.743
0.906
Ours
0.024
0.011
0.024
0.006
0.625
0.695
0.075
0.029
0.029
0.008
0.758
0.914
Table 4: Sequence view 3D reconstruction . Compared to the content in Table 3 , when we input the same dataset into the proposed method in the form of longer, more continuous sequences, the proposed method achieves better results than the baseline method. This indicates that the performance degradation shown in Table 3 , is primarily caused by the input configuration.
BONN
KITTI
Method
Abs Rel ↓
δ<1.25↑
Abs Rel ↓
δ<1.25↑
CUT3R
0.078
93.7
0.118
88.1
TTT3R
0.064
96.2
0.110
90.6
Ours (k=2)
0.064
96.6
0.112
90.2
Ours (k=4)
0.062
96.8
0.109
91.2
Ours (k=16)
0.061
96.8
0.111
90.6
Table 5: The impact of window length on experimental results exhibits diminishing returns. For typical experiments, we actually do not require excessively long window lengths to obtain satisfactory outcomes. Therefore, the computational overhead associated with SSR can be considered fixed and relatively small.
Figure 2: Qualitative Comparison. As shown in the figure, for long trajectories prone to cumulative drift and for loop trajectories, our method achieves loop closure more effectively than CUT3R and TTT3R, thereby yielding relatively superior reconstruction results.
Figure 3: Affinity matrix visualization . The off-diagonal activations in the affinity matrix clearly indicate that our method maintains strong contextual relationship on early frames even as the sequence progresses, providing a robust context constraint for the current estimation.
Streaming 3D reconstruction incrementally builds geometrically consistent scene maps from incoming images or video under tight memory and latency limits.