Video depth estimation is essential for applications such as autonomous driving, AR/VR, and robotics. Recent transformer-based single-image monocular depth estimation models perform well on a single image but struggle with depth consistency across video frames. Traditional methods aim to improve temporal consistency using multi-frame temporal modules or explicit priors like optical flow and camera parameters. However, these approaches face issues such as high memory use, reduced performance with dynamic or irregular motion, and a heavy reliance on the accuracy of external motion estimators. We propose STATIC, a novel model that independently learns temporal consistency in geometrically stable and unstable regions using only two adjacent frames. By leveraging intrinsic geometric cues rather than external motion priors, STATIC generates a depth-derived geometry-aware difference mask from surface normals to identify regions with inter-frame geometric variation by measuring directional variance. For geometrically stable regions, the Masked Stable (MS) module enhances temporal consistency by focusing on structurally stable regions. For geometrically unstable regions, the Surface Normal Similarity (SNS) module models inter-frame correspondences and ensures temporal consistency by measuring feature similarities between frames. A final refinement integrates the independently learned stable and unstable features, enabling STATIC to improve consistent depth prediction across consecutive frames. Experiments on KITTI, NYUv2, and Sintel demonstrate strong depth accuracy and temporal consistency, as well as stable performance across the tested frame-rate and camera-motion conditions.
Figures & tables
Figure 1: Results of the (a) Depth Anything V2 [ 42 ] , (b) GEDepth [ 43 ] and proposed (c) STATIC from sequential frames. Temporal profiles are generated from the marked slice regions to visualize frame-to-frame depth continuity. (c) shows improved ground continuity over the single-frame approach (a) and more stable depth across frames than the compared video depth estimation method (b). For visual comparison, each method is rescaled.
Figure 2: Overall architecture of the proposed STATIC model. The model primarily consists of an encoder, depth embedder, and SNS/MS modules, with a surface normal decoder and head as submodules.
Figure 3: Generation of the proposed difference mask. The pixel-wise directional variance between two surface-normal maps is thresholded to produce the coarse mask Md . The refinement module combines Md with the surface-normal feature Ztn to generate the final mask Ml used by the SNS and MS modules.
Figure 4: The structure of the SNS module. First, a similarity map S is generated using two features. Then, by multiplying this map with the depth features of other frame, a process similar to warping is performed.
Figure 5: The structure of the MS module. First, an attention mechanism is applied using masked features that retain only geometrically stable regions. Next, a refinement process is conducted to integrate the aligned feature with the depth feature.
Method
Frame
C
O
M
Abs Rel ↓
Sq Rel ↓
RMSE ↓
δ<1.25↑
δ<1.252↑
δ<1.253↑
PEM [ 17 ]
SF
-
-
-
0.068
0.221
2.127
0.958
0.993
0.9983
CATNet [ 32 ]
SF
-
-
-
0.052
0.160
2.153
0.972
0.997
0.9990
SSRDepth [ 49 ]
SF
-
-
-
0.053
0.157
2.125
0.973
0.997
0.9990
AdaBins [ 3 ]
SF
-
-
-
0.058
0.190
2.360
0.964
0.995
0.9993
BinsFormer [ 19 ]
SF
-
-
-
0.058
0.190
2.336
0.964
0.996
0.9994
DepthFormer [ 1 ]
SF
-
-
-
0.053
0.187
2.285
0.970
0.996
0.9994
Table 1: Performance comparison between various methods on the KITTI Eigen dataset. The best results are in bold. "MF" indicates multi-frame methods, "SF" indicates single-frame methods, "O" represents optical flow, "C" represents camera parameter, and "M" represents memory. Baseline results are cited from the corresponding published papers.
Type
Method
δ<1.25↑
Abs Rel ↓
SF
SSRDepth [ 49 ]
0.933
0.086
MiDas-v2.1-Large [ 28 ]
0.910
0.095
DPT-Large [ 27 ]
0.928
0.084
WSVD [ 34 ]
0.768
0.164
ST-CLSTM [ 47 ]
0.833
0.131
MF
DeepV2D [ 33 ]
0.924
0.082
Table 2: Performance comparison between various methods on the NYU v2 dataset. Baseline results are cited from the corresponding published papers. † : trained with large-scale pretraining under a different supervision setting.
Method
Type
δ1↑
Rel↓
OPW↓
T(ms)
DPT [ 27 ]
Image
0.597
0.339
0.612
67.7
MiDas [ 28 ]
0.485
0.410
0.843
53.0
CVD [ 23 ]
Flow
0.518
0.406
0.497
-
WSVD [ 34 ]
0.501
0.439
0.577
296.6
Robust-CVD [ 14 ]
Camera
0.521
0.422
0.475
-
GEDepth † [ 43 ]
0.522
0.430
0.473
185
Table 3: Performance comparison of various methods on the Sintel dataset. The best results are highlighted in bold , and the second best results are underlined . Note that † indicates retrained model and T denotes inference time. Baseline results are cited from the corresponding published papers.
Figure 6: Temporal consistency comparison on the KITTI Eigen dataset. The temporal profiles are generated from the marked slice regions in the RGB frames to visualize frame-to-frame depth continuity. The red boxes indicate zoomed regions where temporal discontinuities are more clearly observed.
Type
Method
rTC ↑
aTC ↓
NeWCRFs [ 46 ]
0.914
0.116
SF
iDisc [ 26 ]
0.923
0.108
GEDepth [ 43 ]
0.919
0.133
Depth Anything V2 [ 42 ]
0.946
0.099
TC-Depth-FS [ 30 ]
0.901
0.122
MF
Many-Depth-FS [ 38 ]
0.920
0.111
Table 4: Comparison of temporal consistency across various methods on rTC and aTC.
Figure 7: Qualitative comparison of video methods on the MPI-Sintel dataset. The temporal profiles are generated from the marked slice regions in the RGB frames to visualize frame-to-frame depth continuity. The red boxes highlight regions where differences in temporal stability are noticeable.
Figure 8: Qualitative comparison of video methods on the KITTI Eigen dataset.
Figure 13
SNS
MS
Abs Rel ↓
RMSE ↓
δ<1.25↑
-
-
0.051
2.022
0.977
✓
-
0.047
1.951
0.982
-
✓
0.045
1.939
0.984
✓
✓
0.041
1.917
0.988
Table 5: The ablation experiment evaluates the individual effects of the SNS module and the MS module on the KITTI Eigen dataset.
Figure 11: Qualitative comparison under stationary-camera and moving-camera scenarios. Video frames, surface normal, and the corresponding difference mask across consecutive frames are shown, with optical flow for comparison.
Frame Speed
δ1↑
Rel↓
OPW↓
×1
0.598
0.330
0.405
×0.5
0.595
0.334
0.411
×0.33
0.571
0.359
0.424
Table 6: Performance Evaluation at Different Frame Speeds.
Figure 12: Comparison of Difference Mask Variations with Increased Frame Intervals.
Type
Method
Params (M)
FLOPs (G)
GPU Mem. (MB)
Inference Time (ms)
Image
Depth Anything V2 [ 42 ]
335.316
416.815
1483
38.767
Video
NVDS-MiDaS [ 37 ]
198.581
238.846
1469
26.459
NVDS-DPT
437.274
425.770
2378
42.831
GEDepth [ 43 ]
277.135
318.211
1692
32.812
Video Depth Anything [ 5 ]
384.390
453.725
4333
30.707
STATIC
280.610
243.384
1626
30.553
Table 7: Computational cost comparison between image-based and video-based depth estimation methods at an input resolution of 240×480 .
Figure 13: Failure cases. Red dashed boxes highlight failure regions in the predicted depth and normal maps.
Video depth estimation extends monocular prediction into the temporal domain to ensure coherence. However, existing methods often suffer from spatial blurring in fine-detail regions and temporal inconsistencies. We argue that current approaches, which primarily rely on temporal smoothing via Transformers, struggle to maintain strict 3D geometric consistency-particularly under rotations or drastic view changes. To address this, we propose GemDepth, a framework built on the insight that an explicit awareness of camera motion and global 3D structure is a prerequisite for 3D consistency. Distinctively, GemDepth introduces a Geometry-Embedding Module (GEM) that predicts inter-frame camera poses to generate implicit geometric embeddings. This injection of motion priors equips the network with intrinsic 3D perception and alignment capabilities. Guided by these geometric cues, our Alternating Spatio-Temporal Transformer (ASTT) captures latent point-level correspondences to simultaneously enhance spatial precision for sharp details and enforce rigorous temporal consistency. Furthermore, GemDepth employs a data-efficient training strategy, effectively bridging the gap between high efficiency and robust geometric consistency. As shown in Fig.2, comprehensive evaluations demonstrate that GemDepth achieves state-of-the-art performance across multiple datasets, particularly in complex dynamic scenarios. The code is publicly available at: https://github.com/Yuecheng919/GemDepth.
Yuecheng Liu, Junda Cheng, Longliang Liu +4
1Huazhong University of Science & Technology · 2Carizon · 3Optics Valley Laboratory
Reliable monocular video depth estimation is crucial for downstream 3D reasoning and embodied AI in endoscopic navigation. However, existing self-supervised approaches typically treat video frames independently or rely on weak temporal regularization. These methods, lacking a holistic perception of the underlying 3D scene, inevitably suffer from geometrically inconsistent predictions and severe cross-frame drift. To address these limitations, we introduce a new paradigm that recasts sequential video depth estimation as an unconstrained multi-view 3D reconstruction problem, enabling full exploitation of the powerful geometric priors embedded in recent 3D foundation models. The core of our approach is a 3D consistency optimization framework driven by three constraints: image-level photometric rendering, explicit world-coordinate geometric alignment, and multi-scale temporal gradient consistency. Such unified optimization elegantly anchors isolated frames to a globally coherent 3D structure. Our method has been validated in both the self-supervised training scenarios and challenging zero-shot clinical environments. Results show that the proposed approach achieves state-of-the-art spatial accuracy, outperforming the frame-based, video-based depth estimators and the multi-view 3D reconstruction baselines.
Yuanye Liu, Ke Zhang, Junzhe Jiang +3
Fudan University, Shanghai, China · Johns Hopkins University, Baltimore, USA
Monocular depth estimation (MDE) has been widely adopted in the perception systems of autonomous vehicles and mobile robots. However, existing approaches often struggle to maintain temporal consistency in depth estimation across consecutive frames. This inconsistency not only causes jitter but can also lead to estimation failures when the depth range changes abruptly. To address these challenges, this paper proposes a consistency-aware monocular depth estimation framework that leverages wheel odometry from a mobile robot to achieve stable and coherent depth predictions over time. Specifically, we estimate camera pose and sparse depth from triangulation using optical flow between consecutive frames. The sparse depth estimates are used to update a recursive Bayesian estimate of the metric scale, which is then applied to rescale the relative depth predicted by a pre-trained depth estimation foundation model. The proposed method is evaluated on the KITTI, TartanAir, MS2, and our own dataset, demonstrating robust and accurate depth estimation performance.