Existing video depth estimation faces a fundamental trade-off: generative models suffer from stochastic geometric hallucinations and scale drift, while discriminative models demand massive labeled datasets to resolve semantic ambiguities. To break this impasse, we present DVD, the first framework to deterministically adapt pre-trained video diffusion models into single-pass depth regressors. Specifically, DVD features three core designs: (i) repurposing the diffusion timestep as a structural anchor to balance global stability with high-frequency details; (ii) latent manifold rectification (LMR) to mitigate regression-induced over-smoothing, enforcing differential constraints to restore sharp boundaries and coherent motion; and (iii) global affine coherence, an inherent property bounding inter-window divergence, which enables seamless long-video inference without requiring complex temporal alignment. Extensive experiments demonstrate that DVD achieves state-of-the-art zero-shot performance across benchmarks. Furthermore, DVD successfully unlocks the profound geometric priors implicit in video foundation models using 163x less task-specific data than leading baselines. Notably, we fully release our pipeline, providing the whole training suite for SOTA video depth estimation to benefit the open-source community.
Figures & tables
Figure 1: ( Top ) Comparisons on a 1500 -frame in-the-wild video highlight a fundamental paradigm trade-off: representative generative models ( e.g. , DepthCrafter ( Hu et al., 2025 ) ) suffer from geometric hallucination , while leading discriminative baselines ( e.g. , VDA ( Chen et al., 2025d ) ) face semantic ambiguity . DVD resolves this dilemma, delivering consistent, high-fidelity geometry. ( Bottom ) DVD achieves superior performance on both short and long videos (averaged on KITTI ( Geiger et al., 2012 ) , ScanNet ( Dai et al., 2017 ) , and Bonn ( Palazzolo et al., 2019 ) ), while successfully unlocking the rich priors implicit in video foundation models using remarkably minimal task-specific data, e.g. , less than 1% of VDA’s training set.
Figure 2: Overview of DVD . ( Top ) A video DiT ( Fθ ) performs single-pass depth regression, modulated by a structural anchor ( τ0 ). Latent manifold rectification (LMR) mitigates mean collapse via differential constraints. ( Bottom ) For long video depth estimation, overlapping windows ( WA,WB ) are seamlessly aligned using a closed-form least-squares solver, leveraging the model’s global affine coherence.
Figure 3: Timestep as a structural anchor. Visualizations on NYU ( Nathan Silberman and Fergus, 2012 ) demonstrate a fidelity-stability trade-off. Low ( τ=0.0 ) recovers sharp boundaries but lacks global consistency, whereas high ( τ=0.8 ) causes detail loss ( e.g. , blur). An optimal anchor ( τ=0.5 ) balances these regimes, achieving a trade-off between detail recovery and metric accuracy. More detailed quantitative analyses are shown in Figure 10 .
Figure 4: Timestep embedding similarity. Cosine similarity matrix of timestep embeddings ( t∈[0,1] , stride 0.1 ). While embeddings are broadly consistent, mid-range timesteps exhibit high similarity with a wider range of states.
Figure 5: LMR mitigates mean collapse. Naive regression ( 2nd Row ) exhibits mean collapse, losing high-frequency details. In contrast, our LMR ( 3rd Row ) enforces differential constraints to rectify the latent manifold, recovering both sharp spatial boundaries and temporal coherence. Quantitative analyses are placed in Figure 11 .
Figure 6: Inter-window overlap consistency ( Geiger et al., 2012 ; Palazzolo et al., 2019 ; Dai et al., 2017 ) . Unlike generative baselines with high alignment error and variance, our deterministic regression yields minimal MSE and zero variance, validating our global affine coherence that bounds inter-window discrepancies to linear transformations.
Method
Train Frames
KITTI
ScanNet
Bonn
Sintel
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
DAv2-L ( Yang et al., 2024c )
-
10.9
0.913
6.4
0.967
6.9
0.957
50.0
0.557
Marigold v1.1 ( Ke et al., 2025c )
-
9.5
0.936
7.6
0.940
9.5
0.936
65.1
0.411
RollingDepth ( Ke et al., 2025a )
-
9.8
0.912
5.8
0.964
5.9
0.966
43.7
0.500
ChoronDepth ( Shao et al., 2025 )
381K
15.2
0.775
17.1
0.818
16.8
0.901
52.8
0.504
DepthCrafter ( Hu et al., 2025 )
∼ 30M
9.9
0.907
7.1
0.960
5.9
0.959
37.1
0.664
Table 1: Zero-shot video depth estimation results . The best and the second best results are highlighted.
Method
Paradigm
Bonn
ScanNet
KITTI
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
DAv2-L ( Yang et al., 2024c )
Diff.+D
8.7
0.952
9.5
0.940
11.9
0.879
Marigold v1.1 ( Ke et al., 2025c )
Diff.+G
11.6
0.890
12.0
0.870
24.7
0.569
RollingDepth ( Ke et al., 2025a )
Diff.+G
7.2
0.966
7.5
0.957
11.1
0.911
ChronoDepth ( Shao et al., 2025 )
Diff.+G
17.3
0.859
21.2
0.715
13.0
0.846
DepthCrafter ( Hu et al., 2025 )
Diff.+G
8.5
0.962
11.4
0.866
12.0
0.858
Table 2: Zero-shot long video depth estimation results. Paradigm denotes the backbone type and inference paradigm, where ”Diff.”, ”ViT”, ”D”, and ”G” denote Diffusion-based, ViT-based, Discriminative, and Generative, respectively.
Method
Bonn
ScanNet
KITTI
B-Recall ↑
B-F1 ↑
B-Recall ↑
B-F1 ↑
B-Recall ↑
B-F1 ↑
ChronoDepth ( Shao et al., 2025 )
0.221
0.319
0.144
0.204
0.049
0.090
DepthCrafter ( Hu et al., 2025 )
0.282
0.185
0.115
0.173
0.082
0.044
VDA ( Chen et al., 2025d )
0.223
0.325
0.147
0.210
0.047
0.088
DVD(Ours)
0.336
0.422
0.208
0.259
0.217
0.285
Table 3: Zero-shot boundary metrics, i.e. , Recall and F1. Higher values indicate sharper boundaries and finer details.
Figure 7: Qualitative comparison on indoor and outdoor scenes. DVD consistently produces higher fidelity depth with noticeably sharper structural boundaries
Figure 8: ( Left ) Data scaling curve of long video performance on ScanNet. ( Middle ) Inference latency & FPS comparisons. ( Right ) Stability of long video inference on KITTI. These results demonstrate DVD ’s remarkable data efficiency, competitive inference speed, and consistent long-video stability.
Figure 9: Qualitative results on long-horizon indoor navigation. DVD leverages global affine coherence to better preserve sharp boundaries and globally coherent geometry across thousands of frames compared to prior SOTA methods.
Figure 10: Ablation of timestep τ . The structural anchor τ dictates a fidelity-stability trade-off. While outdoor scenes are more sensitive, both datasets achieve a balance at τ=0.5 .
Figure 11: ( Left ) Ablation of latent manifold rectification. ( Middle ) Ablation of the sampling strategy. ( Right ) Ablation of training schema. The results demonstrate that latent manifold rectification, deterministic adaptation, and joint image-video training effectively enhance geometric accuracy and structural stability.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12: Visualization of inter-window affine alignment. ( Left ) Joint pixel density distribution between the current and reference windows in the overlapping region. The strictly linear correlation (red line) confirms that the inter-window discrepancy is predominantly affine. ( Middle ) After applying our calculated affine transformation, the predictions tightly cluster along the ideal y=x diagonal. ( Right ) The histogram of pixel-wise residuals (Aligned minus Reference) exhibits a zero-mean, tightly bounded distribution with minimal variance, demonstrating the high precision of our global affine coherence strategy.
Setting
Hyperparameter
Value
Model Architecture & Adaptation
Backbone
pre-trained model
Wan2.1-1.3B
VAE
spatial compression
8×
temporal compression
4×
LoRA
target modules
Wq,Wk,Wv,Wo,Wffn
rank ( r ) / alpha ( α )
512/512
Appendix
Table 5: Hyperparameter configurations for DVD .
Wan2.1-1.3B
CogVideoX-5B (by τ )
Metric
DVD
0.1
0.3
0.5
0.7
0.9
AbsRel ↓
6.7
7.8
7.6
7.4
7.4
9.5
δ↑
0.967
0.931
0.934
0.938
0.935
0.898
Appendix
Table 6: Cross-backbone generalization on KITTI. We deploy DVD on CogVideoX-5B ( Hong et al., 2022 ) . Consistent with our default Wan2.1-1.3B backbone, the mid-range timestep ( τ=0.5 ) serves as the optimal structural anchor.
Figure 13: Qualitative results of DVD on CogVideoX ( Hong et al., 2022 ) . Despite employing a different foundation architecture, our deterministic paradigm still preserves significantly sharper high-frequency details (highlighted in red boxes) compared to the leading baseline.
Timestep
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
w/o τ
11.3
0.940
13.7
0.904
τ=0.1
8.6
0.957
9.2
0.918
τ=0.2
6.9
0.972
8.3
0.954
τ=0.3
6.0
0.975
7.5
0.960
τ=0.4
6.1
0.974
7.0
0.966
τ=0.5
5.5
0.974
6.7
0.967
Appendix
Table 7: Analysis of structural anchor τ , which is evaluated on ScanNet ( Left ) and KITTI ( Right ).
Regularizer
AbsRel ↓
δ1↑
B-F1 ↑
L2 only (Eq.( 2 ))
8.5
0.966
0.210
+ RGB reconstruction
10.5
0.951
0.174
+ Edge-aware smoothness
7.5
0.978
0.193
+ Multi-scale gradient matching
8.2
0.969
0.257
+ LMR (Ours)
7.3
0.977
0.259
Appendix
Table 8: Analysis of regularization strategies. All variants use the same backbone, training, and inference settings.
Overlap Size
AbsRel ↓
δ1↑
Rel. Time ↓
O=3
7.9
0.937
1.00 ×
O=6
7.7
0.941
1.04 ×
O=9
7.3
0.945
1.17 ×
O=14
7.2
0.948
1.34 ×
O=19
7.1
0.947
1.55 ×
Appendix
Table 9: Analysis of overlap size on KITTI. Increasing the number of overlapping frames ( O ) improves geometric accuracy but incurs diminishing returns and computational overhead (Rel. Time).
LoRA Rank
AbsRel ↓
δ1↑
256
7.7
0.974
512
7.3
0.977
1024
7.3
0.979
Appendix
Table 10: Analysis of LoRA ranks.
Figure 14: Failure case analysis on 1100 -frame sequence with massive scene transitions. While both DVD and VDA ( Chen et al., 2025d ) inevitably suffer from global scale drift across disjointed scenes ( e.g. , contrasting the absolute depth representations between Frame # 1 and # 500 ), DVD consistently maintains significantly sharper local structural fidelity ( e.g. , hands in Frames # 800 –# 900 ).
Figure 15: More results demonstrations on short videos.
Figure 16: More results demonstrations on short videos.
Figure 17: More results demonstrations on short videos.
Figure 18: More results demonstrations on long videos.
Figure 19: More results demonstrations on long videos.
Figure 20: More results demonstrations on long videos.
Monocular video depth estimation requires temporal consistency, geometric accuracy, and generalization across diverse scenarios, yet existing methods struggle to achieve all three simultaneously. Discriminative models excel at per-frame accuracy but suffer from temporal drift due to limited context windows, while generative methods improve consistency and generalization at the cost of extensive training data (10M+ samples) and lack of geometric precision. In response to these issues, we introduce ICDepth, a framework that adapts pre-trained text-to-video diffusion transformers for video depth estimation via In-Context Conditioning (ICC), leveraging their rich spatial-temporal priors. To address key challenges in transferring ICC from generation to dense prediction, we propose: (1)SAND-Attention, which ensures precise spatial-temporal alignment via shared RoPE and enforces unidirectional attention to prevent noise contamination; (2)SRFM, which injects DINOv2 semantic and resolution priors to enhance geometric precision. ICDepth achieves state-of-the-art results on multiple benchmarks with remarkable data efficiency, trained on only 0.8M frames (6--13× less than competing generative methods), while demonstrating strong zero-shot generalization to diverse domains.
Xuanhua He, Jiaxin Xie, Mingzhe Zheng +1
The Hong Kong University of Science and Technology, Hong Kong SAR, China
Video depth estimation is essential for providing 3D scene structure in applications ranging from autonomous driving to mixed reality. Current end-to-end video depth models have established state-of-the-art performance. Although current end-to-end (E2E) models have achieved state-of-the-art performance, they function as tightly coupled systems that suffer from a significant adaptation lag whenever superior single-image depth estimators are released. To mitigate this issue, post-processing methods such as NVDS offer a modular plug-and-play alternative to incorporate any evolving image depth model without retraining. However, existing post-processing methods still struggle to match the efficiency and practicality of E2E systems due to limited speed, accuracy, and RGB reliance. In this work, we revitalize the role of post-processing by proposing VDPP (Video Depth Post-Processing), a framework that improves the speed and accuracy of post-processing methods for video depth estimation. By shifting the paradigm from computationally expensive scene reconstruction to targeted geometric refinement, VDPP operates purely on geometric refinements in low-resolution space. This design achieves exceptional speed (>43.5 FPS on NVIDIA Jetson Orin Nano) while matching the temporal coherence of E2E systems, with dense residual learning driving geometric representations rather than full reconstructions. Furthermore, our VDPP's RGB-free architecture ensures true scalability, enabling immediate integration with any evolving image depth model. Our results demonstrate that VDPP provides a superior balance of speed, accuracy, and memory efficiency, making it the most practical solution for real-time edge deployment. Our project page is at https://github.com/injun-baek/VDPP
Depth Anything has achieved remarkable success in monocular depth estimation with strong generalization ability. However, it suffers from temporal inconsistency in videos, hindering its practical applications. Various methods have been proposed to alleviate this issue by leveraging video generation models or introducing priors from optical flow and camera poses. Nonetheless, these methods are only applicable to short videos (< 10 seconds) and require a trade-off between quality and computational efficiency. We propose Video Depth Anything for high-quality, consistent depth estimation in super-long videos (over several minutes) without sacrificing efficiency. We base our model on Depth Anything V2 and replace its head with an efficient spatial-temporal head. We design a straightforward yet effective temporal consistency loss by constraining the temporal depth gradient, eliminating the need for additional geometric priors. The model is trained on a joint dataset of video depth and unlabeled images, similar to Depth Anything V2. Moreover, a novel key-frame-based strategy is developed for long video inference. Experiments show that our model can be applied to arbitrarily long videos without compromising quality, consistency, or generalization ability. Comprehensive evaluations on multiple video benchmarks demonstrate that our approach sets a new state-of-the-art in zero-shot video depth estimation. We offer models of different scales to support a range of scenarios, with our smallest model capable of real-time performance at 30 FPS.