Authors: Mengke Zhang, Tianyi Yang, Xinzhuo Li, Anlong Ming
Organizations: International School, Beijing University of Posts and Telecommunications, Beijing, China · International School, Queen Mary University of London, Beijing, China · School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China
Recently, the integration of generative priors has revolutionized Monocular Depth Estimation (MDE), achieving state-of-the-art (SOTA) performance with remarkably high data efficiency. However, although methods based on Text-to-Image (T2I) models excel in general scenes, their efficacy degrades sharply when confronted with open-world environmental perturbations, such as illumination variations and adverse weather. Since T2I priors are optimized for semantic synthesis rather than preserving pixel-level structure, they struggle to decouple geometry from appearance under severe visual shifts, easily triggering structural hallucinations. Therefore, our core insight is to model open-world depth estimation as consistent geometric inference across multi-style images, fully leveraging the powerful structure-preserving editing priors of Image-to-Image (I2I) models. Based on this, we propose OpenDepth, a robust open-world MDE framework that introduces "Intra-Domain Awareness" -- the capability to map diverse stylistic inputs sharing identical geometry into a unified latent space. To achieve this, we first align shallow features within the Diffusion Transformer (DiT) to anchor degraded observations to unified geometric representations. Additionally, we introduce a joint RGB latent reconstruction task, requiring the model to recover clean RGB information from degraded inputs. This multi-stage strategy effectively decouples environmental style interference, compelling the model to focus strictly on underlying geometric recovery. Extensive evaluations show that OpenDepth sets new SOTA records for robust zero-shot depth estimation across standard and extreme weather benchmarks, demonstrating exceptional structural integrity and adaptability.
Figures & tables
Figure 1. Standalone comparison of T2I and I2I priors under perturbations (zero-shot). In a separate controlled experiment, both models are trained on the same 59K samples from Hypersim ( Roberts et al., 2021 ) and Virtual KITTI ( Cabon et al., 2020 ) . The T2I model (Lotus-D ( He et al., 2024 ) ) suffers from severe structural hallucinations under visual degradations, while the I2I baseline better preserves scene geometry, suggesting stronger inherent robustness for depth estimation.
Figure 2. Overview of the OpenDepth framework. To harness editing priors for robust open-world depth estimation, we introduce Latent Feature Alignment ( Lfeat ) and Unified RGB Reconstruction ( Lrecon ). This dual-constraint strategy projects diverse degraded inputs into a unified latent domain, strictly decoupling environmental styles from geometric structures.
Figure 3. Cosine similarity analysis in the feature space across normalized network depths. (Left) Comparison between T2I (Lotus-D) and I2I (OpenDepth) priors. The I2I baseline exhibits significantly higher and more stable similarity than the T2I model, demonstrating inherent intra-domain awareness. (Right) Effect of our Latent Feature Alignment.
Figure 4. t-SNE visualization of deep features across diverse environmental styles. (Left) The Baseline exhibits scattered feature clusters dominated by environmental styles. (Right) Our RGB Reconstruction perfectly aligns these diverse perturbation clusters into a unified latent domain , demonstrating true style-invariance for consistent geometric perception.
Method
Encoder
KITTI ( Geiger et al., 2012 )
NYUv2 ( Silberman et al., 2012 )
Sintel ( Butler et al., 2012 )
ETH3D ( Schops et al., 2017 )
DIODE(indoor) ( Vasiljevic et al., 2019 )
Avg Rank ↓
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
General Depth Estimation Model
DepthPro ( Bochkovskii et al., 2024 )
ViT-S
6.8
94.7
5.2
97.2
24.0
68.2
5.7
96.3
6.0
95.2
4.3
DepthAnything V1 ( Yang et al., 2024a )
ViT-S
8.2
93.6
5.3
97.2
21.1
73.6
6.4
96.6
7.8
94.0
5.2
DepthAnything V2 ( Yang et al., 2024b )
ViT-S
8.3
93.4
5.1
97.3
24.0
71.2
6.4
96.9
7.3
94.3
4.9
Marigold v1.1∗ ( Ke et al., 2024 )
VAE
9.9
91.6
5.5
96.4
42.5
53.5
6.5
95.9
31.0
87.5
10.2
Table 1. Zero-shot relative depth estimation results on general benchmarks. Comparison with general depth estimation models and robust depth estimation models. Best results are in bold , and second best are underlined . * indicates models based on pre-trained Stable Diffusion.
Method
Encoder
NuScenes-night ( Caesar et al., 2020 )
RobotCar-night ( Maddern et al., 2017 )
DS-rain ( Yang et al., 2019 )
DS-cloud ( Yang et al., 2019 )
DS-fog ( Yang et al., 2019 )
Avg Rank ↓
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
General Depth Estimation Model
DepthPro ( Bochkovskii et al., 2024 )
ViT-S
21.8
66.9
23.7
53.4
12.4
84.1
15.8
77.9
10.2
89.2
5.1
DepthAnything V1 ( Yang et al., 2024a )
ViT-S
23.2
67.9
23.9
51.8
13.3
81.9
15.0
80.1
9.8
89.1
5.6
DepthAnything V2 ( Yang et al., 2024b )
ViT-S
20.0
72.5
23.9
51.8
12.5
84.0
15.1
79.8
10.3
89.0
5.4
MoGe ( Wang et al., 2025c )
ViT-L
32.9
41.5
37.9
27.6
23.7
53.2
24.5
52.7
24.4
47.9
13.4
Table 2. Zero-shot relative depth estimation results on open-world benchmarks. Comparison with general depth estimation models and robust depth estimation models. * indicates models based on pre-trained Stable Diffusion.
Figure 5. Qualitative comparison on zero-shot depth estimation. OpenDepth demonstrates superior detail preservation and structural consistency over Lotus-D ( He et al., 2024 ) , FE2E ( Wang et al., 2025b ) , and DepthAnything-AC ( Sun et al., 2025 ) across both general and open-world conditions, as highlighted by white arrows.
Method
Encoder
Dark
Snow
Motion
Gaussian
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
DynaDepth ( Zhang et al., 2022 )
ResNet
16.3
75.2
33.8
39.3
23.4
60.9
27.4
50.1
STEPS ( Zheng et al., 2023 )
ResNet
23.0
63.1
24.2
62.2
29.1
50.8
20.4
69.2
DepthPro ( Bochkovskii et al., 2024 )
ViT-S
14.5
79.3
19.7
68.5
17.0
74.6
17.0
74.5
DepthAnything V2 ( Yang et al., 2024b )
ViT-S
13.0
83.2
11.5
87.2
12.7
84.0
15.7
78.5
DepthAnything-AC ( Sun et al., 2025 )
ViT-S
13.0
83.4
11.4
87.3
12.6
84.1
15.3
79.3
Table 3. Zero-shot relative depth estimation results on synthetic KITTI-C ( Kong et al., 2023 ) benchmarks.
Baseline
Modules
DS-Rain ( Yang et al., 2019 )
DS-Fog ( Yang et al., 2019 )
NuScenes-night ( Caesar et al., 2020 )
RobotCar-night ( Maddern et al., 2017 )
P
Lfeat
Lrecon
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
T2I (Lotus-D) ( He et al., 2024 )
12.2
83.8
10.2
88.3
22.1
66.8
24.4
53.4
✓
12.0
84.6
10.0
88.9
20.1
69.8
22.5
57.3
OpenDepth(G)
12.1
85.1
10.2
88.8
19.3
71.9
21.2
60.4
OpenDepth(D)
11.9
85.5
10.3
89.4
18.0
73.7
20.4
62.5
✓
11.8
85.9
9.7
89.7
16.6
77.0
19.6
66.1
Table 4. Ablation study of different components. T2I (Lotus-D ( He et al., 2024 ) ) and Ours(G) are included as baselines for comparison. (G) and (D) denote generative and discriminative models. P : perturbation, Lfeat : feature alignment, Lrecon : RGB reconstruction.
Figure 6. Qualitative comparison of multi-scale edge responses under severe low-light (Dark) perturbations.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Perturbation
Prob.
Sev.
Process
Light
Dark
100%
1∼5
Brightness is decreased nonlinearly ( γ=2 ), with brighter regions decreasing more and darker regions less. Poisson noise is added to model random photon noise due to reduced photon counts in low-light environments, additional sensor read noise is added to model the read noise of the camera sensor.
Color
Contrast
20%
1∼3
A centered scaling transform is applied to pixel intensities, reducing the overall image contrast using severity-dependent coefficients.
Weather
Fog ∗
20%
1∼3
A 2D self-similar noise field is generated using the Diamond-Square algorithm, where the severity parameter controls the fog density and texture amplitude, leading to reduced detail retention.
Snow ∗
20%
1∼2
A Gaussian random field is generated as the snow distribution, with snowflake size and density adjusted via nonlinear transformations. Directional motion blur ( θ∈[−135∘,−45∘] ) is applied to simulate falling snow.
Blur
Motion Blur †
10%
1∼3
A directional blur kernel is constructed via 1D Gaussian sampling with small displacements along a randomly selected motion direction angle θ∈[−45∘,45∘] , weighted by Gaussian coefficients.
Zoom Blur †
10%
1∼3
Scaling layers with incremental scaling factors are generated using cropping and scaling techniques, and the blurring effect is synthesized by averaging them with equal weights.
Appendix
Table B1. Summary of image perturbations applied during training. Mutually exclusive groups within a category are denoted by matching symbols ( ∗ for weather, † for blur).
Baseline
Sampling ratio (H:V:C)
DS-rain ( Yang et al., 2019 )
DS-fog ( Yang et al., 2019 )
NuScenes-night ( Caesar et al., 2020 )
RobotCar-night ( Maddern et al., 2017 )
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
OpenDepth(D)
1:1:1
10.4
88.4
9.1
91.0
15.5
79.6
18.0
71.8
4:1:1
10.0
88.8
8.7
91.6
14.9
80.7
17.5
72.6
8:1:1
9.7
89.3
8.5
92.0
14.5
81.2
17.0
73.3
12:1:1
9.9
89.0
8.6
91.8
14.8
80.9
17.4
72.8
8:2:1
9.8
89.1
8.6
91.8
14.8
80.9
17.3
72.9
Appendix
Table B2. Ablation study on training-set sampling probabilities. All variants are built upon OpenDepth(D) with perturbation training, feature alignment, and RGB reconstruction enabled; only the batch-level dataset sampling ratio is changed. H/V/C denote Hypersim ( Roberts et al., 2021 ) , Virtual KITTI ( Cabon et al., 2020 ) , and Cityscapes ( Cordts et al., 2016 ) , respectively.
Baseline
Loss weights ( α : β : γ )
DS-rain ( Yang et al., 2019 )
DS-fog ( Yang et al., 2019 )
NuScenes-night ( Caesar et al., 2020 )
RobotCar-night ( Maddern et al., 2017 )
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
OpenDepth(D)
0.5:1:1
10.5
88.1
9.1
90.9
15.9
79.4
18.5
71.1
1:1:1
9.7
89.3
8.5
92.0
14.5
81.2
17.0
73.3
2:1:1
10.1
88.7
8.7
91.4
15.1
80.3
17.8
72.1
1:0.5:1
10.4
88.3
8.9
91.0
15.7
79.5
18.4
71.2
1:2:1
10.0
88.8
8.7
91.6
15.2
80.1
17.9
72.0
Appendix
Table C1. Ablation study on loss weights. All variants are built upon OpenDepth(D) with perturbation training, feature alignment, and cross-image reconstruction enabled; only the loss weights in the final objective are changed.
Target
DS-rain ( Yang et al., 2019 )
DS-fog ( Yang et al., 2019 )
NuScenes-night ( Caesar et al., 2020 )
RobotCar-night ( Maddern et al., 2017 )
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
Fit Clean
10.3
88.5
9.0
91.1
15.3
79.8
17.8
71.9
Fit GT (ours)
9.7
89.3
8.5
92.0
14.5
81.2
17.0
73.3
Appendix
Table C2. Ablation on the supervision target for the perturbed branch. “Fit GT” regresses ground-truth depth; “Fit Clean” regresses the stopped-gradient clean-branch prediction.
Baseline
Aligned layers
DS-rain ( Yang et al., 2019 )
DS-fog ( Yang et al., 2019 )
NuScenes-night ( Caesar et al., 2020 )
RobotCar-night ( Maddern et al., 2017 )
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
OpenDepth(D)
First 1
10.0
88.8
8.8
91.5
14.9
80.5
17.7
72.3
First 2
9.8
89.1
8.6
91.8
14.7
80.9
17.4
72.9
First 4
9.7
89.3
8.5
92.0
14.5
81.2
17.0
73.3
First 8
9.9
89.0
8.6
91.8
14.8
80.7
17.3
72.8
Middle 4
10.0
88.8
8.7
91.6
15.0
80.2
17.6
72.0
Appendix
Table C3. Ablation study on the depth and position of feature alignment. All variants are built upon OpenDepth(D) with perturbation training, feature alignment, and RGB reconstruction enabled; only the aligned Single Stream layers are changed. First N denotes aligning the first N Single Stream blocks. Middle 4 and Last 4 denote aligning the middle four and last four Single Stream blocks, respectively.
Model
Dataset
Error ↓
Accuracy (%) ↑
AbsRel (%)
SqRel
RMSE
RMSElog
SILog
log 10
δ1
δ2
δ3
Discriminative
NYUv2
3.88
0.0179
0.1871
0.0599
5.9807
0.0170
98.04
99.42
99.75
KITTI
6.30
0.2924
3.1318
0.0986
9.8265
0.0276
96.20
99.22
99.84
Generative
NYUv2
3.90
0.0203
0.1974
0.0616
6.1434
0.0171
97.85
99.35
99.74
KITTI
6.54
0.3885
3.3800
0.1041
10.3756
0.0281
95.88
99.07
99.74
Appendix
Table C4. Quantitative comparison of generative and discriminative depth estimation models on NYUv2 (cap 10 m, Eigen crop) and KITTI (cap 80 m, no crop). Alignment: disparity-space least-squares. ↓ : lower is better; ↑ : higher is better. Bold indicates the better result per metric per dataset.
Method
DS-rain ( Yang et al., 2019 )
DS-cloud ( Yang et al., 2019 )
KITTI-C Gaussian ( Kong et al., 2023 )
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
DepthAnything-AC ( Sun et al., 2025 )
12.5
84.0
14.9
80.1
15.3
79.3
OpenDepth (Ours)
9.7
89.3
12.5
84.2
14.1
81.3
Appendix
Table C5. Generalization to unseen perturbations. Neither model was exposed to rain, cloud, or Gaussian noise during training.
Figure D1. Representative case with remaining challenges on RobotCar-night ( Maddern et al., 2017 ) (max distance = 80 m). From left to right: RGB image, DepthAnything-AC ( Sun et al., 2025 ) , and OpenDepth. White arrows highlight that OpenDepth preserves clearer near-range boundaries and more coherent local structure under severe nighttime illumination, while very distant regions remain relatively less separated. In a few extremely dark scenes, sky regions may also be assigned a finite depth rather than the farthest depth.
Model
Resolution
Latency (ms)
Throughput (img/s)
Memory (GB)
AbsRel ↓
δ1↑
Lotus-D ( He et al., 2024 )
768x576
230.3
4.34
6.44
5.2
96.8
FE2E ( Wang et al., 2025b )
768x576
2714.6
0.38
25.20
4.1
97.7
OpenDepth(D)
768x576
704.4
1.42
25.05
3.9
98.0
Appendix
Table D1. Inference efficiency and performance comparison on the whole NYUv2 ( Silberman et al., 2012 ) dataset (batch size 1) measured on a single NVIDIA GeForce RTX 4090 GPU. All methods are evaluated at the same internal resolution of 768x576.
Figure F1. Comparison between DepthAnything-AC ( Sun et al., 2025 ) and our OpenDepth
Figure F2. Comparison between DepthAnything-AC ( Sun et al., 2025 ) and our OpenDepth
Recent monocular depth estimators achieve strong zero-shot generalization, yet often struggle to preserve fine-grained structures and object boundaries. We attribute this limitation to the prevalent combination of large-patch ViT encoders and convolutional decoders, as coarse tokenization can weaken pixel-level cues that upsampling cannot fully recover. To address this issue, we propose PXDepth, a discriminative monocular depth model that separates global context modeling from pixel-level depth prediction. Specifically, a large-patch ViT captures global scene context, while a pixel-space predictor composed of Context-Modulated Pixel Transformer blocks maintains high-resolution spatial representations throughout depth estimation. This design preserves fine structures and sharp boundaries without sacrificing global depth consistency. Across diverse zero-shot benchmarks, PXDepth combines faithful local geometry with competitive global depth accuracy while remaining efficient at inference. Our code and model are available at https://yuanzhy29.github.io/PXDepth-Page/.
Zhiyuan Yuan, Guanying Chen, Lingteng Qiu +3
1Sun Yat-sen University · 3CUHKSZ · 2Shenzhen-FNii
In this paper, we propose Iris, a deterministic framework for Monocular Depth Estimation (MDE) that integrates real-world priors into the diffusion model. Conventional feed-forward methods rely on massive training data, yet still miss details. Previous diffusion-based methods leverage rich generative priors yet struggle with synthetic-to-real domain transfer. Iris, in contrast, preserves fine details, generalizes strongly from synthetic to real scenes, and remains efficient with limited training data. To this end, we introduce a two-stage Priors-to-Geometry Deterministic (PGD) schedule: the prior stage uses Spectral-Gated Distillation (SGD) to transfer low-frequency real priors while leaving high-frequency details unconstrained, and the geometry stage applies Spectral-Gated Consistency (SGC) to enforce high-frequency fidelity while refining with synthetic ground truth. The two stages share weights and are executed with a high-to-low timestep schedule. Extensive experimental results confirm that Iris achieves significant improvements in MDE performance with strong in-the-wild generalization.
Xinhao Cai, Gensheng Pei, Zeren Sun +3
Nanjing University of Science and Technology · State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery · Department of Electrical and Computer Engineering, Sungkyunkwan University +2
Monocular depth estimation is a ubiquitous yet highly ill-posed computer vision task, with downstream applications in scene reconstruction, computational photography, and robotics, among others. Despite the field's maturity, recent models still struggle to generalize to out-of-distribution inputs and to produce sharp and detailed depth maps. In this paper, we revisit Marigold, a set of techniques for repurposing modern image generation and editing models, powered by the diffusion transformer (DiT) architecture, into state-of-the-art monocular depth estimators. Our recipes target single-step inference from pretrained multi-step flow-matching models, with quantization where needed, preserving model capacity while remaining cheap to run. We analyze the artifacts of naive training and identify two effective remedies: aligning the model's internal representations with semantic features extracted from ground-truth, and adopting a 2-stage fine-tuning protocol built around a novel Sinkhorn-based loss. The results are crisper, cleaner depth maps that generalize well out-of-distribution, with 16-26% improvement in AbsRel over the previous best on KITTI and ETH3D. Qualitatively, our model resolves fur, foliage, and hair-thin edges that have eluded prior models. Furthermore, Marigold V2 achieves state-of-the-art results when applied to other dense regression tasks, such as surface normals estimation and intrinsic image decomposition. Project website: https://hf.co/spaces/huawei-bayerlab/marigold-v2-web
Igor Pavlovic, Thiemo Wandel, Anton Obukhov +6
EPFL, HUAWEI Bayer Lab, Switzerland · HUAWEI Bayer Lab, Switzerland · University of Bologna, Italy +1