Test-Time Adaptation for Height Completion via Self-Supervised ViT Features and Monocular Foundation Models
Authors: Osher Rafaeli, Tal Svoray, Ariel Nahlieli
Organizations: Department of Environmental, Geoinformatics and Urban Planning Sciences, Ben-Gurion University of the Negev, Israel · Department of Psychology, Ben-Gurion University of the Negev, Israel
Accurate digital surface models (DSMs) are essential for many geospatial applications, including urban monitoring, environmental analyses, infrastructure management, and change detection. However, large-scale DSMs frequently contain incomplete or outdated regions due to acquisition limitations, reconstruction artifacts, or changes in the built environment. Traditional height completion approaches primarily rely on spatial interpolation or which assume spatial continuity and therefore fail when objects are missing. Recent learning-based approaches improve reconstruction quality but typically require supervised training on sensor-specific datasets, limiting their generalization across domains and sensing conditions. We propose Prior2DSM, a training-free framework for metric DSM completion that operates entirely at test time by leveraging foundation models. Unlike previous height completion approaches that require task-specific training, the proposed method combines self-supervised Vision Transformer (ViT) features from DINOv3 with monocular depth foundation models to propagate metric information from incomplete height priors through semantic feature-space correspondence. Test-time adaptation (TTA) is performed using parameter-efficient low-rank adaptation (LoRA) together with a lightweight multilayer perceptron (MLP), which predicts spatially varying scale and shift parameters to convert relative depth estimates into metric heights. Experiments demonstrate consistent improvements over interpolation based methods, prior-based rescaling height approaches, and state-of-the-art monocular depth estimation models. Prior2DSM reduces reconstruction error while preserving structural fidelity, achieving up to a 46% reduction in RMSE compared to linear fitting of MDE, and further enables DSM updating and coupled RGB-DSM generation.
Figures & tables
Method
Sensor-Agnostic
Object-Agnostic
Missing Objects
Completion Strategy
Training
Global Rescaling ( Yang et al., 2024 )
✓
✓
×
Global linear rescaling
×
LWLR ( Xu et al., 2022 )
✓
✓
∘
Local weighted regression
×
PriorDA ( Wang et al., 2025b )
×
×
×
kNN coarse completion + MDE
✓
Song et al. ( Song et al., 2026 )
×
✓
✓
Random Forest regression
✓
TomoSAR2Height ( Chen et al., 2026b )
×
×
∘
CNN-based reconstruction
✓
Prior2DSM (Proposed)
✓
✓
✓
DINOv3 + MLP
×
Table 1: Comparison of representative height completion approaches. ✓ indicates full support, × indicates no support, and ∘ indicates limited support.
Figure 1: Overview of the proposed height completion framework. Given an RGB image, a monocular depth estimation (MDE) model produces a relative height map, while a self-supervised DINOv3 encoder extracts dense semantic features. Incomplete metric priors are used together with a change mask to identify regions that require completion. A lightweight MLP predicts spatially varying scale and shift parameters that transform relative height predictions into metric heights. During test-time adaptation (TTA), LoRA is applied to the DINOv3 attention layers and the MLP is optimized using the available metric priors, enabling local calibration of the relative height prediction. The final output is a refined metric DSM where missing regions are reconstructed while preserving structural consistency.
Figure 2: Comparison of semantic feature representations extracted from DINOv3. Left: input RGB aerial image. Middle: native DINOv3 patch tokens obtained using the standard patch grid. Right: dense strided ViT tokens obtained using overlapping patch extraction, significantly increasing spatial sampling density and yielding smoother, higher-resolution semantic feature maps that better preserve fine spatial structures.
Figure 3: The experimental dataset consists of two domains: (1) the Denver NAIP dataset and (2) the Worldview-3 Fresno dataset. Both datasets include high-resolution RGB imagery, ground-truth normalized DSM (nDSM), and three levels of prior incompleteness (25%, 50%, and 75%) for evaluation.
25% Completion
50% Completion
75% Completion
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
Global Rescaling (Baseline)
5.08
9.01
0.82
4.81
8.45
0.82
4.74
8.57
0.82
Bilinear Interpolation
4.30
8.42
0.80
4.72
9.16
0.78
5.06
9.75
0.75
LWLR Rescaling
4.93
8.97
0.80
4.70
8.63
0.81
4.57
8.42
0.82
Spatial kNN Rescaling
3.88
7.75
0.84
4.25
8.49
0.82
4.45
8.64
0.80
RS3DAda
4.07
7.80
0.87
3.85
7.20
0.88
3.80
7.16
0.88
Table 2: Mean metrics of NAIP dataset against GT for 25%, 50%, and 75% completion.
25% Completion
50% Completion
75% Completion
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
Global Rescaling (Baseline)
2.06
2.88
0.91
2.11
2.91
0.91
2.20
3.02
0.90
Bilinear Interpolation
2.08
2.91
0.90
2.14
2.99
0.89
2.18
3.03
0.89
LWLR Rescaling
2.81
3.91
0.84
2.98
4.09
0.82
3.05
4.23
0.82
Spatial kNN Rescaling
2.07
3.29
0.91
2.09
3.28
0.91
2.15
3.39
0.91
RS3DAda
1.95
2.77
0.92
1.98
2.76
0.92
2.03
2.81
0.91
Table 3: Mean metrics of WV3 dataset against GT for 25%, 50%, and 75% completion.
Method
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
w/o MDE (MLP → Height)
2.71
5.52
0.91
w/o LoRA (Frozen DINOv3)
2.66
5.32
0.91
Prior2DSM
2.40
4.97
0.92
Table 4: Ablation study of Prior2DSM (mean results across both datasets for 50% completion).
Figure 4: Comparison of DSM completion methods. From left to right: RGB image, Bilinear interpolation, Global affine fitting, Locally Weighted Linear Regression (LWLR), Spatial kNN propagation, DINOv3 feature-space kNN, and the proposed Prior2DSM. Each block corresponds to different levels of missing prior data (25%, 50%, and 75%). For each case, the ground truth (GT) DSM and the degraded prior input are shown on the left, followed by the reconstruction results and the corresponding absolute error maps. While simple interpolation and global fitting produce oversmoothed surfaces and spatial artifacts, semantic feature-based approaches better preserve structural details. The proposed framework consistently achieves more accurate reconstructions with lower errors, particularly under severe missing data conditions. Height: black = low, yellow = high. Error: blue = low, red = high.
Method
MAE (m) ↓
RMSE (m) ↓
SSIM ↑
None (DSM 2008)
1.89
5.35
0.92
Prior2DSM
1.52
3.83
0.94
Table 5: DSM updating results for the NAIP dataset.
Figure 5: Prior-based DSM completion results: From left to right: RGB image, Marigold-DC, PriorDA, and the proposed framework. Each row corresponds to increasing levels of missing data (25%, 50%, and 75%). For each case, the ground truth (GT) DSM and the degraded prior input are shown on the left. The second row in each block shows the corresponding absolute error maps. The framework Prior2DSM produces sharper building structures and lower reconstruction errors compared to existing prior-based approaches, particularly under severe missing data conditions. Height: black = low, yellow = high. Error: blue = low, red = high.
Figure 6: Visual comparison of relative height predictions from three models, Depth Anything V2 (DAV2), Depth Pro, and MoGe-2. DAV2 provides a smoother representation that better preserves flat regions such as ground and roads, while Depth Pro and MoGe-2 produce sharper and more detailed predictions with enhanced structural boundaries. Height: black = low, yellow = high. Error: blue = low, red = high.
Figure 7: Coupled RGB–DSM editing using generative remote sensing models: Text2Earth generates modified RGB imagery from textual prompts describing new urban structures. The proposed framework then predicts the corresponding metric DSM for the edited scene. Red boxes indicate the regions where new buildings are introduced and reconstructed in the generated height map.