Any to Full: Prompting Depth Anything for Depth Completion in One Stage

Zhiyuan Zhou; Ruofeng Liu; Taichi Liu; Weijian Zuo; Shanshan Wang; Zhiqing Hong; Desheng Zhang

cs.CVMar 5, 2026

Any to Full: Prompting Depth Anything for Depth Completion in One Stage

Authors: Zhiyuan Zhou, Ruofeng Liu, Taichi Liu, Weijian Zuo, Shanshan Wang, Zhiqing Hong, Desheng Zhang

Organizations: Rutgers University, USA · Michigan State University, USA · JD Logistics, China · HKUST (Guangzhou), China

Abstract

Accurate, dense depth estimation is crucial for robotic perception, but commodity sensors often yield sparse or incomplete measurements due to hardware limitations. Existing RGBD-fused depth completion methods learn priors jointly conditioned on training RGB distribution and specific depth patterns, limiting domain generalization and robustness to various depth patterns. Recent efforts leverage monocular depth estimation (MDE) models to introduce domain-general geometric priors, but current two-stage integration strategies relying on explicit relative-to-metric alignment incur additional computation and introduce structured distortions. To this end, we present Any2Full, a one-stage, domain-general, and pattern-agnostic framework that reformulates completion as a scale-prompting adaptation of a pretrained MDE model. To address varying depth sparsity levels and irregular spatial distributions, we design a Scale-Aware Prompt Encoder. It distills scale cues from sparse inputs into unified scale prompts, guiding the MDE model toward globally scale-consistent predictions while preserving its geometric priors. Extensive experiments demonstrate that Any2Full achieves superior robustness and efficiency. It outperforms OMNI-DC by 32.2% in average AbsREL and delivers a 1.4 $\times$ speedup over PriorDA with the same MDE backbone, establishing a new paradigm for universal depth completion. Codes and checkpoints are available at https://github.com/zhiyuandaily/Any2Full.

Figures & tables

Figure 1: Depth completion recovers dense depth maps from raw measurements and RGB guidance. (a) Traditional two-stage methods predict coarse depth to bridge the gap between sparse input and dense output. (b) Recent approaches integrate monocular depth estimation (MDE) to generate relative depth and explicitly align it with sparse depth, disrupting MDE’s geometric priors. (c) Our framework employs a lightweight prompting mechanism that tightly integrates MDE’s geometric priors, achieving domain-general and pattern-agnostic depth completion in one stage.

Figure 2: Overview of Any2Full. (a) Our framework reformulates depth completion as a scale-prompting adaptation of a pretrained MDE model. The normalized raw depth is encoded into scale prompts to modulate MDE features for scale-consistent relative depth prediction, followed by a non-parametric least-squares fit to recover the dense metric depth. (b) The Scale-aware Prompt Encoder (SAPE) transforms raw depth into unified scale prompts through two hierarchical modules: Local Enrichment anchors scale cues into the MDE latent space; and Global Propagation leverages MDE geometric features as guidance to diffuse scale cues across patches. Finally, Scale Prompt Fusion injects the resulting prompts into the MDE decoder for final prediction.

	Avg	AVG						NYU-Depth V2						IBims-1
	Rank	Hole		Range		Sparse		Hole		Range		Sparse-Random		Hole		Range		Sparse-Random
Method	$\downarrow$	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE
CompFormer	7.9	0.316	9.682	0.355	5.499	0.067	8.020	0.342	1.098	0.324	1.050	0.011	0.089	0.371	1.372	0.372	1.511	0.013	0.197
DepthPrompt ‡★	5.1	0.018	0.816	0.121	3.558	0.098	2.032	0.008	0.097	0.051	0.338	0.014	0.104	0.008	0.145	0.091	0.647	0.017	0.202
OMNI-DC †‡	3.4	0.014	0.472	0.082	2.805	0.025	0.648	0.019	0.108	0.073	0.415	0.014	0.112	0.014	0.152	0.076	0.592	0.009	0.148
Depth Anything †★	6.4	0.069	2.361	0.079	2.472	0.074	2.207	0.062	0.313	0.074	0.342	0.061	0.299	0.038	0.329	0.042	0.303	0.038	0.337

Table 1: Quantitative comparison on large-scale zero-shot benchmarks (Metrics are AbsREL (↓) and RMSE (↓); bold and underlined denote the best and second-best results). “Any2Full (DA-L/B/S)” denotes our model using Depth Anything-Large/Base/Small backbones. Our method Any2Full (DA-L) achieves the lowest average rank (2.3) across all scenarios , consistently performing among the top competitors against strong domain-generalization and pattern-agnostic approaches, demonstrating robust generalization across diverse depth patterns and domains.

Figure 3: Qualitative comparison under various depth sampling patterns. Black shows missing depths, and red boxes mark key areas. Any2Full demonstrates accurate global geometry, structural consistency, and fine-grained detail preservation across all patterns.

Model	MDE	MDE Params.	Add. Params.	Latency (s)	AbsREL ( $\downarrow$ )
Marigold-DC	Marigold	1.29B	0M	112.47	0.016
TestPromptDC	UniDepth	363.21M	0.9M	92.40	0.013
PriorDA	DA-L	335.3M	96.4M	0.68	0.011
Depth Anything	DA-L	335.3M	0M	0.48	0.038
Ours	DA-L	335.3M	60.6M	0.49	0.007
	DA-B	97.5M	34.2M	0.21	0.008

Table 2: Model efficiency comparison on IBims-1. Average runtime is computed on 100 640×480 images and hole patterns using a single NVIDIA RTX P40 GPU.

Method	Average			NYU-Depth V2			IBims-1			DIODE			KITTI DC			ETH3D			Logistic-Black
Method	Hole	Range	Sparse	Hole	Range	Random	Hole	Range	Random	Hole	Range	Random	Hole	Range	LiDAR	Hole	Range	SfM	Mixed
MoGe-2-B	0.059	0.066	0.084	0.060	0.072	0.061	0.036	0.044	0.036	0.085	0.087	0.086	0.067	0.072	0.079	0.049	0.052	0.158	0.030
Any2Full (MoGe-2-B)	0.012	0.051	0.034	0.010	0.043	0.018	0.009	0.038	0.014	0.020	0.066	0.025	0.015	0.060	0.021	0.008	0.048	0.090	0.011
Any2Full (DA-B)	0.010	0.046	0.028	0.008	0.035	0.017	0.008	0.031	0.013	0.018	0.063	0.022	0.011	0.058	0.017	0.005	0.041	0.069	0.009

Table 3: Cross-backbone generalization of SAPE . AbsREL (

\downarrow

) is evaluated across six benchmarks and multiple depth patterns. “Any2Full (MoGe-2-B)” denotes our model using MoGe-2-Base as the MDE backbone, while “Any2Full (DA-B)” denotes our model using Depth Anything-Base.

Method	AVG		VOID-1500		VOID-500		VOID-150		KITTI-64L		KITTI-16L		KITTI-4L
Method	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE
DepthPrompt	0.122	3.187	0.046	0.623	0.080	0.648	0.115	0.755	0.098	5.005	0.106	4.701	0.283	7.391
OMNI-DC	0.032	1.220	0.026	0.555	0.039	0.551	0.057	0.650	0.013	1.230	0.020	1.703	0.039	2.630
TestPromptDC	0.172	1.638	0.039	0.683	0.068	0.655	0.850	3.218	0.020	1.350	0.023	1.699	0.032	2.221
Ours	0.031	1.203	0.030	0.561	0.037	0.538	0.045	0.620	0.017	1.356	0.023	1.760	0.035	2.380

Table 4: Robustness comparison on varying depth sparsity levels. We evaluate on the VOID dataset with 1500/500/150 sparse points sampled from a VIO system, and the KITTI dataset with 64/16/4-Line LiDAR scans.

Figure 4: Robustness comparison on varying depth range. Evaluation is conducted using AbsREL (

\downarrow

) on NYU-Depth V2 and KITTI DC.

Model	SP	LE	GP	IBims-1			KITTI DC			IBims-1			KITTI DC
Model	SP	LE	GP	Hole	Range	Sparse	Hole	Range	Sparse	10%	1%	0.1%	64L	16L	4L
DA-B				0.047	0.048	0.047	0.079	0.087	0.079	0.044	0.044	0.044	0.086	0.087	0.089
Any2Full	✓			0.010	0.068	0.022	0.017	0.089	0.039	0.011	0.020	0.036	0.021	0.032	0.062
Any2Full	✓	✓		0.009	0.066	0.020	0.015	0.092	0.044	0.011	0.017	0.039	0.021	0.032	0.065
Any2Full	✓	✓	✓	0.008	0.031	0.013	0.011	0.046	0.021	0.010	0.012	0.018	0.017	0.022	0.034

Table 5: Ablation study of Any2Full based on DA-B. SP : scale prompt fusion, LE : local enrichment, GP : global propagation. Metric is AbsREL (

\downarrow

) evaluated on IBims-1 and KITTI DC across different depth patterns and sparsity levels.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Visualization of scale inconsistency. (a) The backbone MDE (Depth Anything) shows varying regional scale factors when aligned to metric depth (red = larger, blue = smaller), while our prediction yields nearly uniform scales consistent with the ground truth. (b) The bottom row shows the region-partitioned RGB input, sparse depth, and our final result, illustrating how region-wise scales are computed.

Figure 6: Two-stage methods struggle to correct noisy coarse depths. We showcase the intermediate and final results on NYU-Depth V2 from two categories of two-stage pipelines: traditional RGBD fused methods (e.g., CompFormer [ 75 ] ) and MDE-integrated methods (e.g., DepthPrompt [ 41 ] , PriorDA [ 67 ] ). Both categories fail to correct noise and structural distortion in coarse depths, resulting in blurred details, disrupted structures, and visible artifacts in the final output. In contrast, our Any2Full framework directly bridges sparse and dense domains in a single stage, achieving superior efficiency, accuracy, and robustness.

Panel of Figure 7 — Figure 7: Top: 3D point-cloud reconstruction comparison. Any2Full yields fewer outliers and cleaner reconstructions than TestPromptDC. Bottom: Qualitative results on highly irregular sparse inputs, including ETH3D-SfM and VOID-150.

	NYU-Depth V2		IBims-1		KITTI DC
Method	AbsREL	RMSE	AbsREL	RMSE	AbsREL	RMSE
Depth Anything+LS	0.061	0.302	0.038	0.346	0.078	3.534
Marigold+LS	0.080	0.319	0.058	0.349	0.198	6.044
PromptDA	0.030	0.221	0.016	0.158	0.036	2.745
Marigold-DC	0.018	0.127	0.014	0.174	0.050	2.442
PriorDA	0.012	0.095	0.011	0.138	0.026	1.952

Appendix

Table 6: Zero-shot depth super-resolution results. Quantitative comparison under

\times 8

downsampling setting.

Figure 8: Visualization of attention propagation from different query points (yellow dots). Each row corresponds to a distinct query location shown in the left column (RGB & query points). The middle and right columns visualize attention maps from two representative heads of the global propagation module in layer 2, illustrating how the propagation extends from nearby to distant regions. Head 1 focuses on refining nearby patches with similar geometry, whereas Head 2 captures long-range scale consistency. Together, they achieve a balanced propagation of scale cues, enhancing both local scale recovery and global scale consistency .

	IBims-1 (AbsREL)			KITTI DC (AbsREL)
Method	Hole	Range	Sparse	Hole	Range	Sparse
Cross-attention	0.009	0.057	0.016	0.015	0.057	0.029
Ours	0.007	0.030	0.013	0.011	0.046	0.021

Appendix

Table 7: Comparison with cross-attention-based propagation. We replace our MDE-guided propagation with standard cross-attention to test robustness under different depth patterns. Our method consistently achieves lower AbsREL across all settings.

$\mathcal{L}_{\text{ssi}}$	$\mathcal{L}_{\text{gm}}$	$\mathcal{L}_{\text{r-ssim}}$	$\mathcal{L}_{\text{anchor}}$	IBims-1 (AbsREL)			KITTI DC (AbsREL)
				Hole	Range	Sparse	Hole	Range	Sparse
✓				0.0086	0.0336	0.0149	0.0132	0.0661	0.0189
✓	✓			0.0083	0.0338	0.0148	0.0132	0.0600	0.0191
✓	✓	✓		0.0083	0.0348	0.0148	0.0136	0.0632	0.0192
✓	✓	✓	✓	0.0077	0.0306	0.0131	0.0115	0.0587	0.0173

Appendix

Table 8: Loss ablation of Any2Full based on DA-B. Each loss term is incrementally added:

\mathcal{L}_{\text{ssi}}

(global alignment),

\mathcal{L}_{\text{gm}}

(fine geometry),

\mathcal{L}_{\text{r-ssim}}

(relative structure), and

\mathcal{L}_{\text{anchor}}

(sparse anchoring).

Figure 9: Robotic warehouse grasping setup and depth results. The table reports grasping success rates across package sizes. Black indicates successful grasps without damage, while red denotes successful but deformed packages due to poor grasp points.

Figure 10: Examples of black packages used in the grasping evaluation. From left to right: large (smallest edge $>$ 30 cm), medium (10–30 cm), and small ( $<$ 10 cm).

Figure 11: Additional qualitative results on DIODE under different Hole patterns (semantic/small-block masking). We compare PriorDA and our method (Any2Full) in predicted depth and error maps (AbsREL). Any2Full consistently preserves global geometry and achieves higher accuracy than PriorDA.

Figure 12: Additional qualitative results on KITTI DC under different Sparse-LiDAR patterns (4L/16L).

Figure 13: Additional qualitative results on KITTI DC under the Range depth patterns.

Figure 14: Additional qualitative results on Logistic-Black under the Mixed depth pattern.

Figure 15: Additional qualitative results on IBims-1 under the Range depth pattern.

Figure 16: Additional qualitative results on IBims-1 under the Hole and Sparse-Random depth patterns.

Explore similar work

Mar 11, 2026

Need for Speed: Zero-Shot Depth Completion with Single-Step Diffusion

We introduce Marigold-SSD, a single-step, late-fusion depth completion framework that leverages strong diffusion priors while eliminating the costly test-time optimization typically associated with diffusion-based methods. By shifting computational burden from inference to finetuning, our approach enables efficient and robust 3D perception under real-world latency constraints. Marigold-SSD achieves significantly faster inference with a training cost of only 4.5 GPU days. We evaluate our method across four indoor and two outdoor benchmarks, demonstrating strong cross-domain generalization and zero-shot performance compared to existing depth completion approaches. Our approach significantly narrows the efficiency gap between diffusion-based and discriminative models. Finally, we challenge common evaluation protocols by analyzing performance under varying input sparsity levels. Page: https://dtu-pas.github.io/marigold-ssd/

Jakub Gregorek, Paraskevas Pegios, Nando Metzger +3

DTU - Technical University of Denmark · Pioneer Centre for AI · ETH Zürich

Depth Completion Diffusion Models Diffusion Model Inference Acceleration

Ask PDF

Jan 29, 2026

MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources

Scaling has powered recent advances in vision foundation models, yet extending this paradigm to metric depth estimation remains challenging due to heterogeneous sensor noise, camera-dependent biases, and metric ambiguity in noisy cross-source 3D data. We introduce Metric Anything, a simple and scalable pretraining framework that learns metric depth from noisy, diverse 3D sources without manually engineered prompts, camera-specific modeling, or task-specific architectures. Central to our approach is the Sparse Metric Prompt, created by randomly masking depth maps, which serves as a universal interface that decouples spatial reasoning from sensor and camera biases. Using about 20M image-depth pairs spanning reconstructed, captured, and rendered 3D data across 10000 camera models, we demonstrate-for the first time-a clear scaling trend in the metric depth track. The pretrained model excels at prompt-driven tasks such as depth completion, super-resolution and Radar-camera fusion, while its distilled prompt-free student achieves state-of-the-art results on monocular depth estimation, camera intrinsics recovery, single/multi-view metric 3D reconstruction, and VLA planning. We also show that using pretrained ViT of Metric Anything as a visual encoder significantly boosts Multimodal Large Language Model capabilities in spatial intelligence. These results show that metric depth estimation can benefit from the same scaling laws that drive modern foundation models, establishing a new path toward scalable and efficient real-world metric perception. We open-source MetricAnything at http://metric-anything.github.io/metric-anything-io/ to support community research.

Baorui Ma, Jiahui Yang, Donglin Di +5

Li Auto Inc

Vision Foundation Models Metric Depth Estimation 3D Reconstruction Depth Estimation Depth Completion Monocular Depth Estimation

Ask PDF

May 12, 2026

The Midas Touch for Metric Depth

Recent advances have markedly improved the cross-scene generalization of relative depth estimation, yet its practical applicability remains limited by the absence of metric scale, local inconsistencies, and low computational efficiency. To address these issues, we present Midas Touch for Depth (MTD), a mathematically interpretable approach that converts relative depth into metric depth using only extremely sparse 3D data. To eliminate local scale inconsistencies, it applies a segment-wise recovery strategy via sparse graph optimization, followed by a pixel-wise refinement strategy using a discontinuity-aware geodesic cost. MTD exhibits strong generalization and achieves substantial accuracy improvements over previous depth completion and depth estimation methods. Moreover, its lightweight, plug-and-play design facilitates deployment and integration on diverse downstream 3D tasks. Project page is available at https://mias.group/MTD.

Yu Ma, Zizhan Guo, Zuyi Xiong +5

College of Electronic and Information Engineering, Tongji University · 2Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University · 3National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University

Metric Depth Estimation Depth Estimation Monocular Depth Estimation

Ask PDF