Test-Time Adaptation for Height Completion via Self-Supervised ViT Features and Monocular Foundation Models
Authors: Osher Rafaeli, Tal Svoray, Ariel Nahlieli
Organizations: Department of Environmental, Geoinformatics and Urban Planning Sciences, Ben-Gurion University of the Negev, Israel · Department of Psychology, Ben-Gurion University of the Negev, Israel
Accurate digital surface models (DSMs) are essential for many geospatial applications, including urban monitoring, environmental analyses, infrastructure management, and change detection. However, large-scale DSMs frequently contain incomplete or outdated regions due to acquisition limitations, reconstruction artifacts, or changes in the built environment. Traditional height completion approaches primarily rely on spatial interpolation or which assume spatial continuity and therefore fail when objects are missing. Recent learning-based approaches improve reconstruction quality but typically require supervised training on sensor-specific datasets, limiting their generalization across domains and sensing conditions. We propose Prior2DSM, a training-free framework for metric DSM completion that operates entirely at test time by leveraging foundation models. Unlike previous height completion approaches that require task-specific training, the proposed method combines self-supervised Vision Transformer (ViT) features from DINOv3 with monocular depth foundation models to propagate metric information from incomplete height priors through semantic feature-space correspondence. Test-time adaptation (TTA) is performed using parameter-efficient low-rank adaptation (LoRA) together with a lightweight multilayer perceptron (MLP), which predicts spatially varying scale and shift parameters to convert relative depth estimates into metric heights. Experiments demonstrate consistent improvements over interpolation based methods, prior-based rescaling height approaches, and state-of-the-art monocular depth estimation models. Prior2DSM reduces reconstruction error while preserving structural fidelity, achieving up to a 46% reduction in RMSE compared to linear fitting of MDE, and further enables DSM updating and coupled RGB-DSM generation.
Figures & tables
Method
Sensor-Agnostic
Object-Agnostic
Missing Objects
Completion Strategy
Training
Global Rescaling ( Yang et al., 2024 )
✓
✓
×
Global linear rescaling
×
LWLR ( Xu et al., 2022 )
✓
✓
∘
Local weighted regression
×
PriorDA ( Wang et al., 2025b )
×
×
×
kNN coarse completion + MDE
✓
Song et al. ( Song et al., 2026 )
×
✓
✓
Random Forest regression
✓
TomoSAR2Height ( Chen et al., 2026b )
×
×
∘
CNN-based reconstruction
✓
Prior2DSM (Proposed)
✓
✓
✓
DINOv3 + MLP
×
Table 1: Comparison of representative height completion approaches. ✓ indicates full support, × indicates no support, and ∘ indicates limited support.
Figure 1: Overview of the proposed height completion framework. Given an RGB image, a monocular depth estimation (MDE) model produces a relative height map, while a self-supervised DINOv3 encoder extracts dense semantic features. Incomplete metric priors are used together with a change mask to identify regions that require completion. A lightweight MLP predicts spatially varying scale and shift parameters that transform relative height predictions into metric heights. During test-time adaptation (TTA), LoRA is applied to the DINOv3 attention layers and the MLP is optimized using the available metric priors, enabling local calibration of the relative height prediction. The final output is a refined metric DSM where missing regions are reconstructed while preserving structural consistency.
Figure 2: Comparison of semantic feature representations extracted from DINOv3. Left: input RGB aerial image. Middle: native DINOv3 patch tokens obtained using the standard patch grid. Right: dense strided ViT tokens obtained using overlapping patch extraction, significantly increasing spatial sampling density and yielding smoother, higher-resolution semantic feature maps that better preserve fine spatial structures.
Figure 3: The experimental dataset consists of two domains: (1) the Denver NAIP dataset and (2) the Worldview-3 Fresno dataset. Both datasets include high-resolution RGB imagery, ground-truth normalized DSM (nDSM), and three levels of prior incompleteness (25%, 50%, and 75%) for evaluation.
25% Completion
50% Completion
75% Completion
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
Global Rescaling (Baseline)
5.08
9.01
0.82
4.81
8.45
0.82
4.74
8.57
0.82
Bilinear Interpolation
4.30
8.42
0.80
4.72
9.16
0.78
5.06
9.75
0.75
LWLR Rescaling
4.93
8.97
0.80
4.70
8.63
0.81
4.57
8.42
0.82
Spatial kNN Rescaling
3.88
7.75
0.84
4.25
8.49
0.82
4.45
8.64
0.80
RS3DAda
4.07
7.80
0.87
3.85
7.20
0.88
3.80
7.16
0.88
Table 2: Mean metrics of NAIP dataset against GT for 25%, 50%, and 75% completion.
25% Completion
50% Completion
75% Completion
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
Global Rescaling (Baseline)
2.06
2.88
0.91
2.11
2.91
0.91
2.20
3.02
0.90
Bilinear Interpolation
2.08
2.91
0.90
2.14
2.99
0.89
2.18
3.03
0.89
LWLR Rescaling
2.81
3.91
0.84
2.98
4.09
0.82
3.05
4.23
0.82
Spatial kNN Rescaling
2.07
3.29
0.91
2.09
3.28
0.91
2.15
3.39
0.91
RS3DAda
1.95
2.77
0.92
1.98
2.76
0.92
2.03
2.81
0.91
Table 3: Mean metrics of WV3 dataset against GT for 25%, 50%, and 75% completion.
Method
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
w/o MDE (MLP → Height)
2.71
5.52
0.91
w/o LoRA (Frozen DINOv3)
2.66
5.32
0.91
Prior2DSM
2.40
4.97
0.92
Table 4: Ablation study of Prior2DSM (mean results across both datasets for 50% completion).
Figure 4: Comparison of DSM completion methods. From left to right: RGB image, Bilinear interpolation, Global affine fitting, Locally Weighted Linear Regression (LWLR), Spatial kNN propagation, DINOv3 feature-space kNN, and the proposed Prior2DSM. Each block corresponds to different levels of missing prior data (25%, 50%, and 75%). For each case, the ground truth (GT) DSM and the degraded prior input are shown on the left, followed by the reconstruction results and the corresponding absolute error maps. While simple interpolation and global fitting produce oversmoothed surfaces and spatial artifacts, semantic feature-based approaches better preserve structural details. The proposed framework consistently achieves more accurate reconstructions with lower errors, particularly under severe missing data conditions. Height: black = low, yellow = high. Error: blue = low, red = high.
Method
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
None (DSM 2008)
1.89
5.35
0.92
Prior2DSM
1.52
3.83
0.94
Table 5: DSM updating results for the NAIP dataset.
Figure 5: Prior-based DSM completion results: From left to right: RGB image, Marigold-DC, PriorDA, and the proposed framework. Each row corresponds to increasing levels of missing data (25%, 50%, and 75%). For each case, the ground truth (GT) DSM and the degraded prior input are shown on the left. The second row in each block shows the corresponding absolute error maps. The framework Prior2DSM produces sharper building structures and lower reconstruction errors compared to existing prior-based approaches, particularly under severe missing data conditions. Height: black = low, yellow = high. Error: blue = low, red = high.
Figure 6: Visual comparison of relative height predictions from three models, Depth Anything V2 (DAV2), Depth Pro, and MoGe-2. DAV2 provides a smoother representation that better preserves flat regions such as ground and roads, while Depth Pro and MoGe-2 produce sharper and more detailed predictions with enhanced structural boundaries. Height: black = low, yellow = high. Error: blue = low, red = high.
Figure 7: Coupled RGB–DSM editing using generative remote sensing models: Text2Earth generates modified RGB imagery from textual prompts describing new urban structures. The proposed framework then predicts the corresponding metric DSM for the edited scene. Red boxes indicate the regions where new buildings are introduced and reconstructed in the generated height map.
We introduce CAPA, a parameter-efficient test-time optimization framework that adapts pre-trained 3D foundation models (FMs) for depth completion, using sparse geometric cues. Unlike prior methods that train task-specific encoders for auxiliary inputs, which often overfit and generalize poorly, CAPA freezes the FM backbone. Instead, it updates only a minimal set of parameters using Parameter-Efficient Fine-Tuning (e.g. LoRA or VPT), guided by gradients calculated directly from the sparse observations available at inference time. This approach effectively grounds the foundation model's geometric prior in the scene-specific measurements, correcting distortions and misplaced structures. For videos, CAPA introduces sequence-level parameter sharing, jointly adapting all frames to exploit temporal correlations, improve robustness, and enforce multi-frame consistency. CAPA is model-agnostic, compatible with any ViT-based FM, and achieves state-of-the-art results across diverse condition patterns on both indoor and outdoor datasets. Project page: research.nvidia.com/labs/dvl/projects/capa.
This work presents the Large Depth Completion Model (LDCM), a simple, effective, and robust framework for single-view metric depth estimation with sparse observations. Without relying on complex architectural designs, LDCM generates metric-accurate dense depth maps using a transformer. It outperforms existing approaches across diverse datasets and sparse observations. We achieve this from two key perspectives: (1) leveraging existing monocular foundation models to improve the quality of sparse depth inputs, and (2) reformulating training objectives to better capture geometric structure and metric consistency. Specifically, a Poisson-based depth initialization strategy is first introduced to generate a uniform coarse dense depth map from diverse sparse observations, providing a strong structural prior for the network. Regarding the training objective, we replace the conventional depth head with a point map head that regresses per-pixel 3D coordinates in camera space, enabling the model to directly learn the underlying 3D scene structure instead of performing pixel-wise depth map restoration. Moreover, this design eliminates the need for camera intrinsic parameters, allowing LDCM to naturally produce metric-scaled 3D point maps. Extensive experiments demonstrate that LDCM consistently outperforms state-of-the-art methods across multiple benchmarks and varying sparsity levels in both depth completion and point map estimation, showcasing its effectiveness and strong generalization to unseen data distributions.
Zhu Yu, Zhengyi Zhao, Runmin Zhang +7
Zhejiang University · Internship at Tongyi Lab · Tongyi Lab, Alibaba Group +4
Accurate Digital Surface Model (DSM) reconstruction from satellite imagery is critical for applications such as disaster response, urban planning, and large-scale geographic mapping. Existing approaches face a fundamental trade-off: optimization-based methods achieve strong accuracy but require hours of per-scene computation, while generalizable geometry foundation models offer near-instant inference but fail to generalize to satellite imagery due to the domain gap introduced by the Rational Polynomial Camera (RPC) model and mismatched depth scale distributions. We present Sat3R, a feed-forward framework that bridges this gap via RPC-aware metric depth fine-tuning of Depth Anything V2 using the Scale-Invariant Logarithmic (SiLog) loss. By constructing physically consistent pseudo depth supervision from RPC geometry, Sat3R adapts a monocular depth foundation model to the satellite domain without per-scene optimization. Experiments on the DFC2019 benchmark demonstrate that Sat3R reduces MAE by 38% over zero-shot feed-forward baselines and achieves competitive accuracy against optimization-based methods, while delivering over 300x speedup. Sat3R demonstrates that feed-forward models, when properly adapted to the satellite domain, can match optimization-based accuracy at a fraction of the computational cost, paving the way for practical large-scale satellite DSM reconstruction.
Qiaoyi Yang, Chaoyi Zhou, Xi Liu +9
1Clemson University · University of Washington · University of Tennessee