Few-shot depth completion aims to recover dense metric depth from sparse measurements using only a small number of target-domain annotations, offering a favorable balance between annotation cost and model adaptation. We propose LIFT, a framework that harnesses both depth and feature priors from a frozen pretrained monocular depth estimation (MDE) foundation model for few-shot depth completion. Specifically, LIFT first estimates local scale and shift parameters on a coarse grid and interpolates them into smooth, dense parameter fields, providing a spatially adaptive metric depth initialization. It then progressively refines this initialization through multi-scale depth propagation guided by foundation features. Finally, a lightweight residual decoder combines multi-level foundation features with propagation correction to refine the metric depth. Extensive experiments on NYUv2 and KITTI-DC demonstrate that our proposed method achieves state-of-the-art few-shot performance. The source code will soon be released.
Figures & tables
Fig. 1: Overview of the proposed LIFT framework. Given an RGB image and sparse depth, the frozen MDE foundation model provides relative depth and multi-level features. The relative depth is first converted into a coarse metric depth by the Grid-based Local Alignment (GLA) module, then refined by the Multi-scale Depth Propagation (MDP) module guided by foundation features, and finally enhanced by a lightweight residual decoder. The final metric depth is obtained by combining the coarse metric depth, propagated depth, and predicted residual.
Fig. 2: Illustration of the scale and shift fields estimated by pixel-wise local fitting and our grid-based local alignment method.
Dataset
Method
1-shot
10-shot
100-shot
1-Sequence Training
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
NYUv2
CSPN [ 30 ]
1.4827
1.2058
0.3166
0.1961
0.2854
0.1307
0.3166
0.1961
NLSPN [ 32 ]
1.9358
1.6132
1.5995
0.8261
0.5501
0.4150
0.8881
0.6421
DySPN [ 33 ]
1.5474
1.2851
0.4102
0.2817
0.3079
0.1706
0.2584
0.1320
CompletionFormer [ 36 ]
1.8218
1.5539
1.1583
1.0162
0.9914
0.8164
0.6779
0.5356
BPNet [ 37 ]
0.3573
0.2077
0.2392
0.1120
0.1757
0.0793
0.2220
0.1040
TABLE I: Few-shot depth completion results on NYUv2 and KITTI-DC. The results of the first six methods are taken from [ 23 , 24 ] , while the remaining baseline results are reported from their respective original papers. RMSE and MAE are reported in meters. The best and second-best results for each dataset are highlighted in bold and underlined , respectively.
Fig. 3: Qualitative comparison on KITTI-DC under 1-shot and 100-shot supervision. The first column shows the RGB image and ground-truth depth, while the remaining columns show predictions from BPNet, UniDC, OASIS-DC, and our method. The top and bottom rows of predictions correspond to 1-shot and 100-shot training, respectively.
Alignment
Propagation
Decoder
RMSE ↓
MAE ↓
Global LS
3.7918
1.7075
Global LS
✓
✓
2.2355
0.6183
Pixel LS
1.8138
0.3753
Pixel LS
✓
✓
1.5530
0.3314
GLA
1.4293
0.4119
GLA
✓
1.4111
0.3821
TABLE II: Ablation study of the proposed components under 100-shot supervision on KITTI-DC. Global LS and Pixel LS denote global and pixel-wise least-squares fitting, respectively.
Alignment Methods
KITTI-DC
NYUv2
RMSE ↓
MAE ↓
T(s) ↓
RMSE ↓
MAE ↓
T(s) ↓
DA-V2
65.3760
64.3420
–
7.3239
7.2311
–
Global LS
3.7918
1.7075
0.005
0.3362
0.2148
0.002
Poisson OASIS-DC [ 24 ]
1.5918
0.4184
0.199
0.2170
0.1269
0.062
Poisson LDCM [ 26 ]
1.5735
0.4765
0.290
0.2067
0.1098
0.199
Pixel LS PriorDA [ 13 ]
1.8138
0.3753
0.254
0.1839
0.0607
0.029
TABLE III: Accuracy and runtime of different metric initialization methods on KITTI-DC and NYUv2. Global LS and Pixel LS denote global and pixel-wise least-squares fitting, respectively. Poisson denotes the Poisson-based reconstruction used in OASIS-DC [ 24 ] and LDCM [ 26 ] .
Fig. 4: Qualitative comparison on NYUv2 under 1-shot and 100-shot supervision. The first column shows the RGB image and ground-truth depth; the remaining columns compare BPNet, UniDC, OASIS-DC, and our method. The top and bottom prediction rows correspond to 1-shot and 100-shot training.
Fig. 5: Visualization of the outputs throughout the proposed pipeline. From top to bottom: RGB images, sparse depth, relative depth estimated by the MDE model, coarse metric depth aligned by GLA, final depth predictions, and the residual and fusion gate maps predicted by the residual decoder.
Settings
10-shot
100-shot
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
Full model
1.2767
0.3415
1.1650
0.2992
Residual propagation
1.3003
0.3408
1.2004
0.3044
RGB propagation
1.2945
0.3427
1.1804
0.3023
Single scale
1.2963
0.3565
1.1770
0.3022
TABLE IV: Propagation design ablations under 10-shot and 100-shot supervision on KITTI-DC.
Decoder Conditioning
10-shot
100-shot
{Z(l)}
Dout(L)−D0
M
RMSE ↓
MAE ↓
RMSE ↓
MAE ↓
✓
✓
✓
1.2767
0.3415
1.1650
0.2992
✓
1.3613
0.3794
1.3086
0.3776
✓
✓
1.3288
0.3640
1.2489
0.3386
✓
✓
1.2831
0.3443
1.1809
0.3121
TABLE V: Ablation of decoder conditioning inputs under 10-shot and 100-shot supervision on KITTI-DC.
Dense metric depth from cameras is essential to real-world 3D applications, yet achieving accuracy, faithful surface geometry, and fast inference simultaneously remains challenging. Monocular foundation models provide rich, transferable geometric priors but lack reliable metric scale, while depth-completion networks recover metric depth at the cost of geometric fidelity, cross-domain robustness, or speed. We present FounRef, a training-free method that aligns a frozen monocular foundation prior with sparse metric anchors to produce dense metric depth. FounRef is modular by design: its depth prior, anchor source, and refinement solver can each be replaced independently. We instantiate FounRef with MoGe-2 and LiDAR anchors. FounRef validates each anchor against the prior's dense depth prediction, rejecting inconsistencies caused by cross-sensor misalignment that geometry-only filters cannot detect. It then applies global and local metric corrections through a structure-preserving solver, retaining the prior's fine-grained geometry. FounRef requires no task-specific training and operates out of the box across unfamiliar cameras and scenes. On out-of-domain data, it delivers up to 24% lower depth error, 92% lower surface-normal noise, and almost 15x faster inference than DMD3C, a state-of-the-art depth-completion network. By decoupling metric alignment from geometry prediction, FounRef provides an accurate, geometrically faithful, and efficient approach to dense metric depth that can directly benefit from future advances in foundation models and metric sensors.
This work presents the Large Depth Completion Model (LDCM), a simple, effective, and robust framework for single-view metric depth estimation with sparse observations. Without relying on complex architectural designs, LDCM generates metric-accurate dense depth maps using a transformer. It outperforms existing approaches across diverse datasets and sparse observations. We achieve this from two key perspectives: (1) leveraging existing monocular foundation models to improve the quality of sparse depth inputs, and (2) reformulating training objectives to better capture geometric structure and metric consistency. Specifically, a Poisson-based depth initialization strategy is first introduced to generate a uniform coarse dense depth map from diverse sparse observations, providing a strong structural prior for the network. Regarding the training objective, we replace the conventional depth head with a point map head that regresses per-pixel 3D coordinates in camera space, enabling the model to directly learn the underlying 3D scene structure instead of performing pixel-wise depth map restoration. Moreover, this design eliminates the need for camera intrinsic parameters, allowing LDCM to naturally produce metric-scaled 3D point maps. Extensive experiments demonstrate that LDCM consistently outperforms state-of-the-art methods across multiple benchmarks and varying sparsity levels in both depth completion and point map estimation, showcasing its effectiveness and strong generalization to unseen data distributions.
Zhu Yu, Zhengyi Zhao, Runmin Zhang +7
Zhejiang University · Internship at Tongyi Lab · Tongyi Lab, Alibaba Group +4
We introduce CAPA, a parameter-efficient test-time optimization framework that adapts pre-trained 3D foundation models (FMs) for depth completion, using sparse geometric cues. Unlike prior methods that train task-specific encoders for auxiliary inputs, which often overfit and generalize poorly, CAPA freezes the FM backbone. Instead, it updates only a minimal set of parameters using Parameter-Efficient Fine-Tuning (e.g. LoRA or VPT), guided by gradients calculated directly from the sparse observations available at inference time. This approach effectively grounds the foundation model's geometric prior in the scene-specific measurements, correcting distortions and misplaced structures. For videos, CAPA introduces sequence-level parameter sharing, jointly adapting all frames to exploit temporal correlations, improve robustness, and enforce multi-frame consistency. CAPA is model-agnostic, compatible with any ViT-based FM, and achieves state-of-the-art results across diverse condition patterns on both indoor and outdoor datasets. Project page: research.nvidia.com/labs/dvl/projects/capa.