Metric Depth Estimation from Arbitrarily Degraded Low-Resolution Depth Prompts
Authors: Kun Wang, Yun Zhu, Pan Zhou, Na Zhao
Organizations: IMPL, Singapore University of Technology and Design · PCA Lab, Nanjing University of Science and Technology · LV-Lab, Singapore Management University
We propose AdaDS, an Adaptive Diffusion Sampling framework for prompted metric depth estimation, which estimates high-resolution metric depth from an image and an arbitrarily degraded low-resolution depth prompt. This setting is commonly studied as depth super-resolution, where existing methods typically regress depth directly without an explicit prior over high-quality depth, and thus often produce artifacts under severe or unseen degradations. In contrast, AdaDS exploits the contraction property of Gaussian smoothing: as noise accumulates in the forward diffusion process, the noisy distributions of degraded prompts and their high-quality counterparts converge toward an isotropic Gaussian prior, while the prompt content is gradually erased. Accordingly, AdaDS refines the prompt and estimates its uncertainty to adaptively select a starting timestep, and injects learned noise to place the intermediate sample in a high-density region of the depth distribution learned by a pre-trained diffusion model. Denoising from this sample yields high-quality depth with fine-grained details and corrects residual errors, even under imperfect prompt refinement. Experiments on real-world and synthetic benchmarks demonstrate its superior zero-shot generalization and robustness to diverse degradations.
Figures & tables
Figure 1: Given an RGB image and an arbitrarily degraded low-resolution depth prompt, AdaDS adaptively places the intermediate noisy sample in a high-density region of a pre-trained depth diffusion prior, enabling zero-shot estimation of fine-grained high-resolution metric depth.
Figure 2: The distribution of the degraded depth prompt (with predicted mean and variance from coarse refinement) progressively aligns with the ground-truth distribution due to Gaussian smoothing contraction. Dots indicate the means.
Figure 3: Overall framework of AdaDS. (a) The first stage produces a refined depth latent along with an estimated uncertainty map reflecting the refinement reliability. (b) The second stage blends the coarse refinement with estimated noise, placing it in a high-density region of the high-quality depth distribution. The resulting intermediate noisy latent is denoised by a pre-trained relative depth diffusion model to recover the final depth map. Ep , El and Dl denote the patch embedding, latent encoder and latent decoder, respectively.
Figure 4: As t increases, the reconstructed depth maps exhibit sharper structural details but deviate further from the input depth prompt, so that the error first decreases and then increases. All depth maps share a unified color palette, with RMSE reported in centimeters.
Method
Ref.
RGB-D-D
TOFDSR
192×144
64×48
32×24
192×144
64×48
32×24
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
DA v2-L
NeurIPS-24
0.1082
0.0745
0.1082
0.0745
0.1082
0.0745
0.1035
0.0690
0.1035
0.0690
0.1035
0.0690
MG-LCM
CVPR-24
0.1306
0.1007
0.1306
0.1007
0.1306
0.1007
0.1219
0.0835
0.1219
0.0835
0.1219
0.0835
DA v3-NGL
ICLR-26
0.2835
0.2528
0.2835
0.2528
0.2835
0.2528
0.5229
0.3806
0.5229
0.3806
0.5229
0.3806
SGNet
AAAI-24
0.0726
0.0388
0.0723
0.0391
0.0740
0.0416
0.0726
0.0400
0.0726
0.0404
0.0767
0.0433
Table 1: Quantitative zero-shot comparison on real-world benchmarks. Input resolutions for the low-resolution depth prompt are provided in the second row. For each metric, the best and second-best results are highlighted in bold and underline , respectively.
Method
Ref.
ScanNet
NYUv2
16× DS.
DS. + GN.
DS. + GB.
DS. + SM.
DS. + LC.
16× DS.
DS. + GN.
DS. + GB.
DS. + SM.
DS. + LC.
DA v2-L
NeurIPS-24
0.1416
0.1416
0.1416
0.1416
0.1416
0.2075
0.2075
0.2075
0.2075
0.2075
MG-LCM
CVPR-24
0.1986
0.1986
0.1986
0.1986
0.1986
0.2587
0.2587
0.2587
0.2587
0.2587
DA v3-NGL
ICLR-26
0.2186
0.2186
0.2186
0.2186
0.2186
0.3095
0.3095
0.3095
0.3095
0.3095
PromptDA
CVPR-25
0 .0927
0.0983
0 .0935
0 .0953
0.0950
0 .1326
0 .1352
0 .1349
0 .1369
0 .1338
OMNI-DC
ICCV-25
0.1173
0.1224
0.1127
0.1188
0.1206
0.1966
0.2011
0.1894
0.2003
0.1980
Table 2: Quantitative zero-shot comparison on synthetic benchmarks. DS., GN., GB., SM. and LC. denote downsampling, Gaussian noise, Gaussian blur, sparse measurement and low-bit compression, respectively. RMSE metric is reported in this table.
Figure 5: Comparison on real-world benchmarks with arbitrary upsampling factors. The δ1.05↑ metric is reported in this figure.
Figure 6: Qualitative comparison on real-world benchmarks. The first two samples are from RGB-D-D and the last two from TOFDSR. Blue boxes mark regions enlarged in the bottom-left inset of each result, and error maps are shown in the bottom-right inset, with warmer colors denoting larger errors.
Figure 7: Qualitative comparison on synthetic benchmarks. The first two samples are from NYUv2 and the last two from ScanNet. To facilitate 3D structural evaluation, the input and estimated depth maps are back-projected into point clouds and colorized with the corresponding RGB images.
Scaling has powered recent advances in vision foundation models, yet extending this paradigm to metric depth estimation remains challenging due to heterogeneous sensor noise, camera-dependent biases, and metric ambiguity in noisy cross-source 3D data. We introduce Metric Anything, a simple and scalable pretraining framework that learns metric depth from noisy, diverse 3D sources without manually engineered prompts, camera-specific modeling, or task-specific architectures. Central to our approach is the Sparse Metric Prompt, created by randomly masking depth maps, which serves as a universal interface that decouples spatial reasoning from sensor and camera biases. Using about 20M image-depth pairs spanning reconstructed, captured, and rendered 3D data across 10000 camera models, we demonstrate-for the first time-a clear scaling trend in the metric depth track. The pretrained model excels at prompt-driven tasks such as depth completion, super-resolution and Radar-camera fusion, while its distilled prompt-free student achieves state-of-the-art results on monocular depth estimation, camera intrinsics recovery, single/multi-view metric 3D reconstruction, and VLA planning. We also show that using pretrained ViT of Metric Anything as a visual encoder significantly boosts Multimodal Large Language Model capabilities in spatial intelligence. These results show that metric depth estimation can benefit from the same scaling laws that drive modern foundation models, establishing a new path toward scalable and efficient real-world metric perception. We open-source MetricAnything at http://metric-anything.github.io/metric-anything-io/ to support community research.
This work presents Lapis, a linear-attention-based pixel-space generative framework that achieves efficient and high-fidelity depth estimation with one-step diffusion. While generative frameworks have significantly advanced monocular depth estimation with superior detail fidelity, the O(N2) complexity of standard attention and the multi-step denoising process introduce prohibitive computational costs when scaling them to high-resolution image applications. Although linear attention and one-step prediction are intuitively viable, directly applying them leads to poor structural consistency, detail loss, and noise. Lapis rectifies these limitations through a coarse-to-fine hierarchy. Specifically, a Patch-level Consistency Module restores structural coherence by integrating semantic and spatial priors. Subsequently, a Pixel-level Refinement Module recovers sharp geometric boundaries via skip-connection-based pixel correspondence. Furthermore, to mitigate sampling noise inherent in one-step diffusion, we leverage the manifold assumption and adopt a direct x-prediction strategy to target the clean data manifold. Extensive evaluations on multiple benchmarks demonstrate that Lapis consistently achieves state-of-the-art (SOTA) accuracy and boundary sharpness across various resolutions, reducing inference latency by up to 7.6× at 1080P and 10.9× at 1440P resolution compared to previous SOTA generative models.
Bingde Liu, Wu Ran, Jinglei Zhang +2
MoE Key Lab of Artificial Intelligence, Institute of AI, Shanghai Jiao Tong University, Shanghai, China · Zhiyuan College, Shanghai Jiao Tong University, Shanghai, China
We introduce Marigold-SSD, a single-step, late-fusion depth completion framework that leverages strong diffusion priors while eliminating the costly test-time optimization typically associated with diffusion-based methods. By shifting computational burden from inference to finetuning, our approach enables efficient and robust 3D perception under real-world latency constraints. Marigold-SSD achieves significantly faster inference with a training cost of only 4.5 GPU days. We evaluate our method across four indoor and two outdoor benchmarks, demonstrating strong cross-domain generalization and zero-shot performance compared to existing depth completion approaches. Our approach significantly narrows the efficiency gap between diffusion-based and discriminative models. Finally, we challenge common evaluation protocols by analyzing performance under varying input sparsity levels. Page: https://dtu-pas.github.io/marigold-ssd/
Jakub Gregorek, Paraskevas Pegios, Nando Metzger +3
DTU - Technical University of Denmark · Pioneer Centre for AI · ETH Zürich