OpenDepth: Repurposing Image Editing Priors for Robust Open-World Monocular Depth Estimation

Mengke Zhang; Tianyi Yang; Xinzhuo Li; Anlong Ming

cs.AIOct 13, 2026

OpenDepth: Repurposing Image Editing Priors for Robust Open-World Monocular Depth Estimation

Authors: Mengke Zhang, Tianyi Yang, Xinzhuo Li, Anlong Ming

Organizations: International School, Beijing University of Posts and Telecommunications, Beijing, China · International School, Queen Mary University of London, Beijing, China · School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China

Abstract

Recently, the integration of generative priors has revolutionized Monocular Depth Estimation (MDE), achieving state-of-the-art (SOTA) performance with remarkably high data efficiency. However, although methods based on Text-to-Image (T2I) models excel in general scenes, their efficacy degrades sharply when confronted with open-world environmental perturbations, such as illumination variations and adverse weather. Since T2I priors are optimized for semantic synthesis rather than preserving pixel-level structure, they struggle to decouple geometry from appearance under severe visual shifts, easily triggering structural hallucinations. Therefore, our core insight is to model open-world depth estimation as consistent geometric inference across multi-style images, fully leveraging the powerful structure-preserving editing priors of Image-to-Image (I2I) models. Based on this, we propose OpenDepth, a robust open-world MDE framework that introduces "Intra-Domain Awareness" -- the capability to map diverse stylistic inputs sharing identical geometry into a unified latent space. To achieve this, we first align shallow features within the Diffusion Transformer (DiT) to anchor degraded observations to unified geometric representations. Additionally, we introduce a joint RGB latent reconstruction task, requiring the model to recover clean RGB information from degraded inputs. This multi-stage strategy effectively decouples environmental style interference, compelling the model to focus strictly on underlying geometric recovery. Extensive evaluations show that OpenDepth sets new SOTA records for robust zero-shot depth estimation across standard and extreme weather benchmarks, demonstrating exceptional structural integrity and adaptability.

Figures & tables

Figure 1. Standalone comparison of T2I and I2I priors under perturbations (zero-shot). In a separate controlled experiment, both models are trained on the same 59K samples from Hypersim ( Roberts et al., 2021 ) and Virtual KITTI ( Cabon et al., 2020 ) . The T2I model (Lotus-D ( He et al., 2024 ) ) suffers from severe structural hallucinations under visual degradations, while the I2I baseline better preserves scene geometry, suggesting stronger inherent robustness for depth estimation.

Figure 2. Overview of the OpenDepth framework. To harness editing priors for robust open-world depth estimation, we introduce Latent Feature Alignment ( $\mathcal{L}_{feat}$ ) and Unified RGB Reconstruction ( $\mathcal{L}_{recon}$ ). This dual-constraint strategy projects diverse degraded inputs into a unified latent domain, strictly decoupling environmental styles from geometric structures.

Figure 3. Cosine similarity analysis in the feature space across normalized network depths. (Left) Comparison between T2I (Lotus-D) and I2I (OpenDepth) priors. The I2I baseline exhibits significantly higher and more stable similarity than the T2I model, demonstrating inherent intra-domain awareness. (Right) Effect of our Latent Feature Alignment.

Figure 4. t-SNE visualization of deep features across diverse environmental styles. (Left) The Baseline exhibits scattered feature clusters dominated by environmental styles. (Right) Our RGB Reconstruction perfectly aligns these diverse perturbation clusters into a unified latent domain , demonstrating true style-invariance for consistent geometric perception.

Method	Encoder	KITTI ( Geiger et al., 2012 )		NYUv2 ( Silberman et al., 2012 )		Sintel ( Butler et al., 2012 )		ETH3D ( Schops et al., 2017 )		DIODE(indoor) ( Vasiljevic et al., 2019 )		Avg Rank $\downarrow$
Method	Encoder	AbsRel $\downarrow$	$\delta 1\uparrow$	AbsRel $\downarrow$	$\delta 1\uparrow$	AbsRel $\downarrow$	$\delta 1\uparrow$	AbsRel $\downarrow$	$\delta 1\uparrow$	AbsRel $\downarrow$	$\delta 1\uparrow$	Avg Rank $\downarrow$
General Depth Estimation Model
DepthPro ( Bochkovskii et al., 2024 )	ViT-S	6.8	94.7	5.2	97.2	24.0	68.2	5.7	96.3	6.0	95.2	4.3
DepthAnything V1 ( Yang et al., 2024a )	ViT-S	8.2	93.6	5.3	97.2	21.1	73.6	6.4	96.6	7.8	94.0	5.2
DepthAnything V2 ( Yang et al., 2024b )	ViT-S	8.3	93.4	5.1	97.3	24.0	71.2	6.4	96.9	7.3	94.3	4.9
Marigold ${}_{v1.1}^{*}$ ( Ke et al., 2024 )	VAE	9.9	91.6	5.5	96.4	42.5	53.5	6.5	95.9	31.0	87.5	10.2

Table 1. Zero-shot relative depth estimation results on general benchmarks. Comparison with general depth estimation models and robust depth estimation models. Best results are in bold , and second best are underlined . * indicates models based on pre-trained Stable Diffusion.

Method	Encoder	NuScenes-night ( Caesar et al., 2020 )		RobotCar-night ( Maddern et al., 2017 )		DS-rain ( Yang et al., 2019 )		DS-cloud ( Yang et al., 2019 )		DS-fog ( Yang et al., 2019 )		Avg Rank $\downarrow$
Method	Encoder	AbsRel $\downarrow$	$\delta 1\uparrow$	AbsRel $\downarrow$	$\delta 1\uparrow$	AbsRel $\downarrow$	$\delta 1\uparrow$	AbsRel $\downarrow$	$\delta 1\uparrow$	AbsRel $\downarrow$	$\delta 1\uparrow$	Avg Rank $\downarrow$
General Depth Estimation Model
DepthPro ( Bochkovskii et al., 2024 )	ViT-S	21.8	66.9	23.7	53.4	12.4	84.1	15.8	77.9	10.2	89.2	5.1
DepthAnything V1 ( Yang et al., 2024a )	ViT-S	23.2	67.9	23.9	51.8	13.3	81.9	15.0	80.1	9.8	89.1	5.6
DepthAnything V2 ( Yang et al., 2024b )	ViT-S	20.0	72.5	23.9	51.8	12.5	84.0	15.1	79.8	10.3	89.0	5.4
MoGe ( Wang et al., 2025c )	ViT-L	32.9	41.5	37.9	27.6	23.7	53.2	24.5	52.7	24.4	47.9	13.4

Table 2. Zero-shot relative depth estimation results on open-world benchmarks. Comparison with general depth estimation models and robust depth estimation models. * indicates models based on pre-trained Stable Diffusion.

Figure 5. Qualitative comparison on zero-shot depth estimation. OpenDepth demonstrates superior detail preservation and structural consistency over Lotus-D ( He et al., 2024 ) , FE2E ( Wang et al., 2025b ) , and DepthAnything-AC ( Sun et al., 2025 ) across both general and open-world conditions, as highlighted by white arrows.

Method	Encoder	Dark		Snow		Motion		Gaussian
Method	Encoder	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$
DynaDepth ( Zhang et al., 2022 )	ResNet	16.3	75.2	33.8	39.3	23.4	60.9	27.4	50.1
STEPS ( Zheng et al., 2023 )	ResNet	23.0	63.1	24.2	62.2	29.1	50.8	20.4	69.2
DepthPro ( Bochkovskii et al., 2024 )	ViT-S	14.5	79.3	19.7	68.5	17.0	74.6	17.0	74.5
DepthAnything V2 ( Yang et al., 2024b )	ViT-S	13.0	83.2	11.5	87.2	12.7	84.0	15.7	78.5
DepthAnything-AC ( Sun et al., 2025 )	ViT-S	13.0	83.4	11.4	87.3	12.6	84.1	15.3	79.3

Table 3. Zero-shot relative depth estimation results on synthetic KITTI-C ( Kong et al., 2023 ) benchmarks.

Baseline	Modules			DS-Rain ( Yang et al., 2019 )		DS-Fog ( Yang et al., 2019 )		NuScenes-night ( Caesar et al., 2020 )		RobotCar-night ( Maddern et al., 2017 )
Baseline	$P$	$L_{feat}$	$L_{recon}$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$
T2I (Lotus-D) ( He et al., 2024 )				12.2	83.8	10.2	88.3	22.1	66.8	24.4	53.4
	✓			12.0	84.6	10.0	88.9	20.1	69.8	22.5	57.3
OpenDepth(G)				12.1	85.1	10.2	88.8	19.3	71.9	21.2	60.4
OpenDepth(D)				11.9	85.5	10.3	89.4	18.0	73.7	20.4	62.5
	✓			11.8	85.9	9.7	89.7	16.6	77.0	19.6	66.1

Table 4. Ablation study of different components. T2I (Lotus-D ( He et al., 2024 ) ) and Ours(G) are included as baselines for comparison. (G) and (D) denote generative and discriminative models.

P

: perturbation,

L_{feat}

: feature alignment,

L_{recon}

: RGB reconstruction.

Figure 6. Qualitative comparison of multi-scale edge responses under severe low-light (Dark) perturbations.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Category	Perturbation	Prob.	Sev.	Process
Light	Dark	100%	$1\sim 5$	Brightness is decreased nonlinearly ( $\gamma=2$ ), with brighter regions decreasing more and darker regions less. Poisson noise is added to model random photon noise due to reduced photon counts in low-light environments, additional sensor read noise is added to model the read noise of the camera sensor.
Color	Contrast	20%	$1\sim 3$	A centered scaling transform is applied to pixel intensities, reducing the overall image contrast using severity-dependent coefficients.
Weather	Fog ∗	20%	$1\sim 3$	A 2D self-similar noise field is generated using the Diamond-Square algorithm, where the severity parameter controls the fog density and texture amplitude, leading to reduced detail retention.
Weather	Snow ∗	20%	$1\sim 2$	A Gaussian random field is generated as the snow distribution, with snowflake size and density adjusted via nonlinear transformations. Directional motion blur ( $\theta\in[-135^{\circ},-45^{\circ}]$ ) is applied to simulate falling snow.
Blur	Motion Blur †	10%	$1\sim 3$	A directional blur kernel is constructed via 1D Gaussian sampling with small displacements along a randomly selected motion direction angle $\theta\in[-45^{\circ},45^{\circ}]$ , weighted by Gaussian coefficients.
Blur	Zoom Blur †	10%	$1\sim 3$	Scaling layers with incremental scaling factors are generated using cropping and scaling techniques, and the blurring effect is synthesized by averaging them with equal weights.

Appendix

Table B1. Summary of image perturbations applied during training. Mutually exclusive groups within a category are denoted by matching symbols ( ∗ for weather, † for blur).

Baseline	Sampling ratio (H:V:C)	DS-rain ( Yang et al., 2019 )		DS-fog ( Yang et al., 2019 )		NuScenes-night ( Caesar et al., 2020 )		RobotCar-night ( Maddern et al., 2017 )
Baseline	Sampling ratio (H:V:C)	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$
OpenDepth(D)	1:1:1	10.4	88.4	9.1	91.0	15.5	79.6	18.0	71.8
	4:1:1	10.0	88.8	8.7	91.6	14.9	80.7	17.5	72.6
	8:1:1	9.7	89.3	8.5	92.0	14.5	81.2	17.0	73.3
	12:1:1	9.9	89.0	8.6	91.8	14.8	80.9	17.4	72.8
	8:2:1	9.8	89.1	8.6	91.8	14.8	80.9	17.3	72.9

Appendix

Table B2. Ablation study on training-set sampling probabilities. All variants are built upon OpenDepth(D) with perturbation training, feature alignment, and RGB reconstruction enabled; only the batch-level dataset sampling ratio is changed. H/V/C denote Hypersim ( Roberts et al., 2021 ) , Virtual KITTI ( Cabon et al., 2020 ) , and Cityscapes ( Cordts et al., 2016 ) , respectively.

Baseline	Loss weights ( $\alpha$ : $\beta$ : $\gamma$ )	DS-rain ( Yang et al., 2019 )		DS-fog ( Yang et al., 2019 )		NuScenes-night ( Caesar et al., 2020 )		RobotCar-night ( Maddern et al., 2017 )
Baseline	Loss weights ( $\alpha$ : $\beta$ : $\gamma$ )	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$
OpenDepth(D)	0.5:1:1	10.5	88.1	9.1	90.9	15.9	79.4	18.5	71.1
	1:1:1	9.7	89.3	8.5	92.0	14.5	81.2	17.0	73.3
	2:1:1	10.1	88.7	8.7	91.4	15.1	80.3	17.8	72.1
	1:0.5:1	10.4	88.3	8.9	91.0	15.7	79.5	18.4	71.2
	1:2:1	10.0	88.8	8.7	91.6	15.2	80.1	17.9	72.0

Appendix

Table C1. Ablation study on loss weights. All variants are built upon OpenDepth(D) with perturbation training, feature alignment, and cross-image reconstruction enabled; only the loss weights in the final objective are changed.

Target	DS-rain ( Yang et al., 2019 )		DS-fog ( Yang et al., 2019 )		NuScenes-night ( Caesar et al., 2020 )		RobotCar-night ( Maddern et al., 2017 )
Target	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$
Fit Clean	10.3	88.5	9.0	91.1	15.3	79.8	17.8	71.9
Fit GT (ours)	9.7	89.3	8.5	92.0	14.5	81.2	17.0	73.3

Appendix

Table C2. Ablation on the supervision target for the perturbed branch. “Fit GT” regresses ground-truth depth; “Fit Clean” regresses the stopped-gradient clean-branch prediction.

Baseline	Aligned layers	DS-rain ( Yang et al., 2019 )		DS-fog ( Yang et al., 2019 )		NuScenes-night ( Caesar et al., 2020 )		RobotCar-night ( Maddern et al., 2017 )
Baseline	Aligned layers	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$
OpenDepth(D)	First 1	10.0	88.8	8.8	91.5	14.9	80.5	17.7	72.3
	First 2	9.8	89.1	8.6	91.8	14.7	80.9	17.4	72.9
	First 4	9.7	89.3	8.5	92.0	14.5	81.2	17.0	73.3
	First 8	9.9	89.0	8.6	91.8	14.8	80.7	17.3	72.8
	Middle 4	10.0	88.8	8.7	91.6	15.0	80.2	17.6	72.0

Appendix

Table C3. Ablation study on the depth and position of feature alignment. All variants are built upon OpenDepth(D) with perturbation training, feature alignment, and RGB reconstruction enabled; only the aligned Single Stream layers are changed. First N denotes aligning the first

N

Single Stream blocks. Middle 4 and Last 4 denote aligning the middle four and last four Single Stream blocks, respectively.

Model	Dataset	Error $\downarrow$						Accuracy (%) $\uparrow$
Model	Dataset	AbsRel (%)	SqRel	RMSE	RMSElog	SILog	log 10	$\delta_{1}$	$\delta_{2}$	$\delta_{3}$
Discriminative	NYUv2	3.88	0.0179	0.1871	0.0599	5.9807	0.0170	98.04	99.42	99.75
Discriminative	KITTI	6.30	0.2924	3.1318	0.0986	9.8265	0.0276	96.20	99.22	99.84
Generative	NYUv2	3.90	0.0203	0.1974	0.0616	6.1434	0.0171	97.85	99.35	99.74
Generative	KITTI	6.54	0.3885	3.3800	0.1041	10.3756	0.0281	95.88	99.07	99.74

Appendix

Table C4. Quantitative comparison of generative and discriminative depth estimation models on NYUv2 (cap 10 m, Eigen crop) and KITTI (cap 80 m, no crop). Alignment: disparity-space least-squares.

\downarrow

: lower is better;

\uparrow

: higher is better. Bold indicates the better result per metric per dataset.

Method	DS-rain ( Yang et al., 2019 )		DS-cloud ( Yang et al., 2019 )		KITTI-C Gaussian ( Kong et al., 2023 )
Method	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$	AbsRel $\downarrow$	$\delta_{1}\uparrow$
DepthAnything-AC ( Sun et al., 2025 )	12.5	84.0	14.9	80.1	15.3	79.3
OpenDepth (Ours)	9.7	89.3	12.5	84.2	14.1	81.3

Appendix

Table C5. Generalization to unseen perturbations. Neither model was exposed to rain, cloud, or Gaussian noise during training.

Figure D1. Representative case with remaining challenges on RobotCar-night ( Maddern et al., 2017 ) (max distance = 80 m). From left to right: RGB image, DepthAnything-AC ( Sun et al., 2025 ) , and OpenDepth. White arrows highlight that OpenDepth preserves clearer near-range boundaries and more coherent local structure under severe nighttime illumination, while very distant regions remain relatively less separated. In a few extremely dark scenes, sky regions may also be assigned a finite depth rather than the farthest depth.

Model	Resolution	Latency (ms)	Throughput (img/s)	Memory (GB)	AbsRel $\downarrow$	$\delta_{1}\uparrow$
Lotus-D ( He et al., 2024 )	768x576	230.3	4.34	6.44	5.2	96.8
FE2E ( Wang et al., 2025b )	768x576	2714.6	0.38	25.20	4.1	97.7
OpenDepth(D)	768x576	704.4	1.42	25.05	3.9	98.0

Appendix

Table D1. Inference efficiency and performance comparison on the whole NYUv2 ( Silberman et al., 2012 ) dataset (batch size 1) measured on a single NVIDIA GeForce RTX 4090 GPU. All methods are evaluated at the same internal resolution of 768x576.

Figure F1. Comparison between DepthAnything-AC ( Sun et al., 2025 ) and our OpenDepth

Figure F2. Comparison between DepthAnything-AC ( Sun et al., 2025 ) and our OpenDepth

Monocular Depth Estimation

Monocular depth estimation infers scene geometry and distance from a single camera image, enabling 3D perception without stereo or depth sensors.

See all 132 papers

PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimation
Aug 17, 2026Zhiyuan Yuan, Guanying Chen, Lingteng Qiu +3Depth Estimation Vision Transformer
1Sun Yat-sen University · 3CUHKSZ · 2Shenzhen-FNii
Iris: Bringing Real-World Priors into Diffusion Model for Monocular Depth Estimation
Mar 17, 2026Xinhao Cai, Gensheng Pei, Zeren Sun +3Monocular Depth Estimation Diffusion Models
Nanjing University of Science and Technology · State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery · Department of Electrical and Computer Engineering, Sungkyunkwan University +2
Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation
Sep 8, 2026Igor Pavlovic, Thiemo Wandel, Anton Obukhov +6Diffusion Transformer OOD Generalization
EPFL, HUAWEI Bayer Lab, Switzerland · HUAWEI Bayer Lab, Switzerland · University of Bologna, Italy +1

Aug 17, 2026J/K move · Enter open · S save

PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimation

Zhiyuan Yuan, Guanying Chen, Lingteng Qiu, Ruimao Zhang+2

1Sun Yat-sen University · 3CUHKSZ · 2Shenzhen-FNii

Depth Estimation Vision Transformer Monocular Depth Estimation

Open paper AskPDF arXiv

Recent monocular depth estimators achieve strong zero-shot generalization, yet often struggle to preserve fine-grained structures and object boundaries. We attribute this limitation to the prevalent combination of large-patch ViT encoders and convolutional decoders, as coarse tokenization can weaken pixel-level cues that upsampling cannot fully recover. To address this issue, we propose PXDepth, a discriminative monocular depth model that separates global context modeling from pixel-level depth prediction. Specifically, a large-patch ViT captures global scene context, while a pixel-space predictor composed of Context-Modulated Pixel Transformer blocks maintains high-resolution spatial representations throughout depth estimation. This design preserves fine structures and sharp boundaries without sacrificing global depth consistency. Across diverse zero-shot benchmarks, PXDepth combines faithful local geometry with competitive global depth accuracy while remaining efficient at inference. Our code and model are available at https://yuanzhy29.github.io/PXDepth-Page/.

Abstract

Monocular Depth Estimation

Explore similar work

PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimation

Iris: Bringing Real-World Priors into Diffusion Model for Monocular Depth Estimation

Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation