Monocular Depth Estimation From the Perspective of Feature Restoration: A Diffusion Enhanced Depth Restoration Approach

Huibin Bai; Shuai Li; Hanxiao Zhai; Yanbo Gao; Chong Lv; Yibo Wang; Haipeng Ping; Wei Hua

cs.CVApr 9, 2026

Monocular Depth Estimation From the Perspective of Feature Restoration: A Diffusion Enhanced Depth Restoration Approach

Authors: Huibin Bai, Shuai Li, Hanxiao Zhai, Yanbo Gao, Chong Lv, Yibo Wang, Haipeng Ping, Wei Hua,

Organizations: School of Software, Shandong University, Jinan 250100, China · Shandong University-WeiHai Research Institute of Industrial Technology, Weihai 264209, China · School of Control Science and Engineering, Shandong University, Jinan, China · Shandong Institute of Information Technology Industry Development, Jinan, China ·

Abstract

Monocular Depth Estimation (MDE) is a fundamental computer vision task with important applications in 3D vision. The current mainstream MDE methods employ an encoder-decoder architecture with multi-level/scale feature processing. However, the limitations of the current architecture and the effects of different-level features on the prediction accuracy are not evaluated. In this paper, we first investigate the above problem and show that there is still substantial potential in the current framework if encoder features can be improved. Therefore, we propose to formulate the depth estimation problem from the feature restoration perspective, by treating pretrained encoder features as degraded features of an assumed ground truth feature that yields the ground truth depth map. Then an Invertible Transform-enhanced Indirect Diffusion (InvT-IndDiffusion) module is developed for feature restoration. Due to the absence of direct supervision on feature, only indirect supervision from the final sparse depth map is used. During the iterative procedure of diffusion, this results in feature deviations among steps. The proposed InvT-IndDiffusion solves this problem by using an invertible transform-based decoder under the bi-Lipschitz condition. Finally, a plug-and-play Auxiliary Viewpoint-based Low-level Feature Enhancement module (AV-LFE) is developed to enhance local details with auxiliary viewpoint when available. Experiments demonstrate that the proposed method achieves better performance than the state-of-the-art methods on various datasets. Specifically on the KITTI benchmark, compared with the baseline, the performance is improved by 4.09% and 37.77% under different training settings in terms of RMSE. Code is available at https://github.com/whitehb1/IID-RDepth.

Figures & tables

Fig. 1: Illustration of the feature deviation problem in the diffusion process due to the indirect supervision and iterative inference. It can be mitigated with the proposed InvT-IndDiffusion based on the invertible decoder under bi-Lipschitz condition.

Fig. 2: Effects of different levels of features on the final depth prediction performance in terms of RMSE (the smaller the better).

Fig. 3: Overview of the proposed IID-RDepth. The input image is first processed through a frozen encoder to extract the multi-level features. Then the high-level semantic features are combined as mutual conditional maps and fed into the InvT-IndDiffusion for restoration. For low-level detail features, AV-LFE is used as a plug-and-play module to enhance the identity connection when auxiliary viewpoint is available.

Type	Method	RMSE $\downarrow$	$\Delta$ RMSE $\uparrow$	Abs Rel $\downarrow$	Sq Rel $\downarrow$	RMESlog $\downarrow$	$\zeta_{1}\uparrow$	$\zeta_{2}\uparrow$	$\zeta_{3}\uparrow$
Regression	Eigen et al. [ 51 ]	6.307	-203.07%	0.203	1.548	-	-	-	-
	P3Depth [ 53 ]	2.842	-36.56%	0.071	0.270	0.103	0.953	0.993	0.998
	DORN [ 54 ]	2.727	-31.04%	0.072	0.307	0.120	0.932	0.984	0.995
	NeWCRFs [ 33 ]	2.129	-2.31%	0.052	0.155	0.079	0.974	0.997	0.999
	URCDC [ 55 ]	2.032	2.35%	0.050	0.142	0.076	0.977	0.997	0.999
	EVP [ 29 ]	2.015	3.17%	0.048	0.136	0.073	0.980	0.999	1.000

TABLE I: Quantitative results of the proposed method against the existing methods, using the Eigen split on the KITTI [ 52 ] dataset. ’

\downarrow

’ indicates smaller is better, while ’

\uparrow

’ denotes larger is better.

\Delta

RMSE is calculated by comparing with the baseline PixelFormer [ 32 ] . † refers to results obtained with both InvT-IndDiffusion and AV-LFE in compatible mode. ‡ refers to results obtained with both InvT-IndDiffusion and AV-LFE in fully trainable mode.

Fig. 4: Qualitative results on the KITTI dataset comparing our IID-RDepth with existing methods. IID-RDepth utilizes the proposed InvT-IndDiffusion, while IID-RDepth † combines both InvT-IndDiffusion and AV-LFE in a compatible mode. With the proposed method, results of both fine-grained details and distant objects are improved. Some examples are zoomed such as the gaps between trees in the first row, the geometric structure of distant road signs in the second row, the edges of nearby white stone monuments along the road in the third row, the roadside signs in the fourth row.

Fig. 5: Visual illustration of the predicted depth maps and their differences when using the InvT-IndDiffusion with different interference steps, where the blue to red colors represent improvement from small to large. The residual map displays the differences of results obtained at different steps. The improvement map further illustrates the reduction in prediction error towards the ground truth depth map, derived from the residuals between the results and the ground truth depth map at different steps, with blue to red indicating the magnitude of the improvement.

DDAD Benchmark
Method	RMSE $\downarrow$	$\Delta$ RMSE $\uparrow$	Abs Rel $\downarrow$	Sq Rel $\downarrow$	RMESlog $\downarrow$	$\delta_{1}$ $\uparrow$	$\delta_{2}$ $\uparrow$	$\delta_{3}$ $\uparrow$
NeWCRFs [ 33 ]	5.21	9.23%	0.111	0.930	0.141	0.891	0.982	0.995
PixelFormer [ 32 ]	5.74	$\backslash$	0.127	0.134	0.157	0.861	0.972	0.993
Ours-P	5.26	8.36%	0.121	0.988	0.146	0.881	0.977	0.994
Ours-N	4.82	16.03%	0.095	0.793	0.127	0.914	0.985	0.995

TABLE II: Quantitative result comparison on the DDAD [ 61 ] dataset. ’

\downarrow

’indicates smaller is better, while ’

\uparrow

’ denotes larger is better.

\Delta

RMSE is calculated by comparing with the baseline PixelFormer [ 32 ] . ours-N (-P) means using NeWCRFs (PixelFormer) as the backbone.

Fig. 6: Qualitative Results of Ablation Studies on the AV-LFE Module in the KITTI [48] Dataset. A different pseudo color scheme, changing the colormap by replacing the default Magma colormap to Jet, is used to better illustrate the close-range depth result, with blue to red denoting increasing depth.

Method	RMSE $\downarrow$	$\Delta$ RMSE $\uparrow$	Sq Rel $\downarrow$	$\delta_{1}\uparrow$
w/o Diffusoin	2.074	-	0.148	0.976
Diffusion-1	2.020	2.60%	0.142	0.978
Diffusion-6	1.996	3.76%	0.140	0.979

TABLE III: Ablation experiment on the InvT-IndDiffusion module. Diffusion-1 (-6) represents diffusion with 1 (6) inference steps.

Method	RMSE $\downarrow$	$\Delta$ RMSE $\uparrow$	Sq Rel $\downarrow$	$\delta_{1}\uparrow$
TF	2.054	-	0.147	0.976
Conv	2.017	1.80%	0.144	0.977
Inv	1.996	2.82%	0.140	0.979

TABLE IV: Results using different types of decoders for the restored features by diffusion.

Methods	Range	RMSE $\downarrow$	$\Delta$ RMSE $\uparrow$	Sq Rel $\downarrow$	$\delta_{1}\uparrow$
Pixelformer	0-20m	0.803	-	0.056	0.988
IID-RDepth	0-20m	0.781	2.74%	0.053	0.989
Pixelformer	20-50m	3.514	-	0.433	0.935
IID-RDepth	20-50m	3.477	2.41%	0.428	0.938
Pixelformer	50-80m	9.128	-	1.528	0.828
IID-RDepth	50-80m	8.393	9.02%	1.307	0.862

TABLE V: Results in terms of different depth ranges, including near (0-20m), medium (20-50m) and far (50-80m).

Method	Total number	$P<0.05$	$P<0.01$
Ours / NewCRFs [ 33 ]	652	636	620
Ours / Pixelformer [ 32 ]	652	634	627
Ours / idisc [ 60 ]	652	632	623

TABLE VI: Paired t-test for improvement significance evaluation. The numerical values listed under each criterion indicate the number of pixels that our method outperforms the respective competitor.

Depth Estimation

Depth estimation infers scene distance from images and sensors, enabling 3D reconstruction, navigation, and spatial understanding.

See all 198 papers

Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation
Sep 8, 2026Igor Pavlovic, Thiemo Wandel, Anton Obukhov +6Diffusion Transformer OOD Generalization
EPFL, HUAWEI Bayer Lab, Switzerland · HUAWEI Bayer Lab, Switzerland · University of Bologna, Italy +1
Monocular Depth Estimation from a Single Image: Progress and Opportunities
Sep 1, 2026Muxin Liu, Xiaoyang Lyu, Yang-Tian Sun +4Metric Depth Estimation Depth Estimation
Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong, 999077.
LiftFormer: Lifting and Frame Theory Based Monocular Depth Estimation Using Depth and Edge Oriented Subspace Representation
Apr 8, 2026Shuai Li, Huibin Bai, Yanbo Gao +5Depth Estimation Monocular Depth Estimation
School of Control Science and Engineering, Shandong University, and Key Laboratory of Machine Intelligence and System Control, Ministry of Education, Jinan 250100, China. · School of Software, Shandong University, Jinan 250100, China · Shandong University-WeiHai Research Institute of Industrial Technology, Weihai 264209, China +2

Sep 8, 2026J/K move · Enter open · S save

Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation

Igor Pavlovic, Thiemo Wandel, Anton Obukhov, Luca Bartolomei+5

EPFL, HUAWEI Bayer Lab, Switzerland · HUAWEI Bayer Lab, Switzerland · University of Bologna, Italy+1

Diffusion Transformer OOD Generalization Monocular Depth Estimation Surface Normal Estimation Diffusion Model Fine-Tuning

Open paper AskPDF arXiv

Monocular depth estimation is a ubiquitous yet highly ill-posed computer vision task, with downstream applications in scene reconstruction, computational photography, and robotics, among others. Despite the field's maturity, recent models still struggle to generalize to out-of-distribution inputs and to produce sharp and detailed depth maps. In this paper, we revisit Marigold, a set of techniques for repurposing modern image generation and editing models, powered by the diffusion transformer (DiT) architecture, into state-of-the-art monocular depth estimators. Our recipes target single-step inference from pretrained multi-step flow-matching models, with quantization where needed, preserving model capacity while remaining cheap to run. We analyze the artifacts of naive training and identify two effective remedies: aligning the model's internal representations with semantic features extracted from ground-truth, and adopting a 2-stage fine-tuning protocol built around a novel Sinkhorn-based loss. The results are crisper, cleaner depth maps that generalize well out-of-distribution, with 16-26% improvement in AbsRel over the previous best on KITTI and ETH3D. Qualitatively, our model resolves fur, foliage, and hair-thin edges that have eluded prior models. Furthermore, Marigold V2 achieves state-of-the-art results when applied to other dense regression tasks, such as surface normals estimation and intrinsic image decomposition. Project website: https://hf.co/spaces/huawei-bayerlab/marigold-v2-web

Abstract

Depth Estimation

Explore similar work

Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation

Monocular Depth Estimation from a Single Image: Progress and Opportunities

LiftFormer: Lifting and Frame Theory Based Monocular Depth Estimation Using Depth and Edge Oriented Subspace Representation