FSKD: Monocular Forest Structure Inference via LiDAR-to-RGBI Knowledge Distillation

Taimur Khan; Hannes Feilhauer; Muhammad Jazib Zafar

cs.CVApr 2, 2026

FSKD: Monocular Forest Structure Inference via LiDAR-to-RGBI Knowledge Distillation

Authors: Taimur Khan, Hannes Feilhauer, Muhammad Jazib Zafar

Organizations: Helmholtz Centre for Environmental Research – UFZ, Halle (Saale), Germany · Leipzig University, Leipzig, Germany · Georg-August University of Göttingen, Göttingen, Germany

Abstract

Very High Resolution (VHR) forest structure data at individual-tree scale is essential for carbon, biodiversity, and ecosystem monitoring. Still, airborne LiDAR remains costly and infrequent despite being the reference for forest structure metrics like Canopy Height Model (CHM), Plant Area Index (PAI), and Foliage Height Diversity (FHD). We propose FSKD: a LiDAR-to-RGB-Infrared (RGBI) knowledge distillation (KD) framework in which a multi-modal teacher fuses RGBI imagery with LiDAR-derived planar metrics and vertical profiles via cross-attention, and an RGBI-only SegFormer student learns to reproduce these outputs. Trained on 384 $km^2$ of forests in Saxony, Germany (20 cm ground sampling distance (GSD)) and evaluated on eight geographically distinct test tiles, the student achieves state-of-the-art (SOTA) zero-shot CHM performance (MedAE 4.17 m, $R^2$ =0.51, IoU 0.87), outperforming HRCHM/DAC baselines by 29--46% in MAE (5.81 m vs. 8.14--10.84 m) with stronger correlation coefficients (0.713 vs. 0.166--0.652). Ablations show that multi-modal fusion improves performance by 10--26% over RGBI-only training, and that asymmetric distillation with appropriate model capacity is critical. The method jointly predicts CHM, PAI, and FHD, a multi-metric capability not provided by current monocular CHM estimators, although PAI/FHD transfer remains region-dependent and benefits from local calibration. The framework also remains effective under temporal mismatch (winter LiDAR, summer RGBI), removing strict co-acquisition constraints and enabling scalable 20 cm operational monitoring for workflows such as Digital Twin Germany and national Digital Orthophoto programs.

Figures & tables

Figure 1: Multi-modal data acquisition and forest structure derivation framework. (a) RGBI orthophotos and LiDAR are acquired at different times. (b) Inputs are spatially aligned in 2 km $\times$ 2 km tiles. (c) LiDAR provides privileged structural supervision (CHM, PAI, FHD, PAD) for teacher training, while the deployed student uses RGBI only.

Figure 2: Study area and dataset split. Spatial distribution of the 96 RGBI–LiDAR tile pairs in Saxony and neighboring areas with geographically separated train/validation/test regions.

Metric	Equation
CHM - planar	$\text{H}_{\text{canopy}}=\max(\text{HAG}_{\text{points}})$
PAD - vertical	$\text{PAD}_{i-1,i}=\ln\left(\frac{S_{e}}{S_{t}}\right)\frac{1}{k\Delta z}$
PAI - planar	$\text{PAI}=\sum_{i=1}^{n}\text{PAD}_{i-1,i}$
FHD - planar	$\text{FHD}=-\sum_{i=1}^{n}p_{i}\ln(p_{i})$
P05, P50, P95	$P_{xx}$ is the height below which $xx\%$ of all points lie.

Table 1: Common canopy structure metrics and their equations [ 22 , 32 ] .

Figure 3: Preprocessing and alignment overview. RGBI and LiDAR products are transformed into pixel-aligned multi-modal training tensors for distillation.

Figure 4: Teacher–student architecture. A LiDAR-aware teacher supervises an RGBI-only student for CHM/PAI/FHD prediction.

Figure 5: Teacher feature flow. Multi-modal feature extraction, fusion, and decoding in the teacher network.

Figure 6: Student feature flow. Multi-scale RGBI encoding and dense regression in the RGBI-only student.

Study	What to remove/test	Why
No distillation loss	Remove KD loss $\to$ only RGBI supervision.	Does distillation actually help?
Asymmetric learning	1. Train teacher for fewer epochs than student. 2. Train teacher for more epochs than student.	Does symmetry matter in learning representations?
Without multimodal fusion (no cross-attention)	Remove multi-modal fusion.	Does multi-stage feature fusion via cross-attention matter?
Batch size sensitivity	Try batch size 8 vs 64.	Important for generalization vs. memory.
Backbone size	Try MiT-B2 vs B5.	Smaller vs larger (impact on capacity).

Table 2: Ablation studies for model architecture and training parameters

Figure 7: Quantitative agreement on held-out test tiles. CHM shows the strongest correlation; PAI and FHD remain harder under zero-shot transfer.

Model	MAE (m)	MedAE (m)	RMSE (m)	Bias (m)	R	IoU	F1
FSKD (MiT-B2/b64)	$5,81\pm 1,44$	$4,17\pm 1,62$	7,90	-2,57	0,713	0,870	0,930
HRCHM-aerial (Full Res)	$9,88\pm 2,49$	$8,83\pm 3,93$	12,89	-8,55	0,452	0,733	0,842
HRCHM-aerial (60cm)	$8,14\pm 2,64$	$7,46\pm 3,01$	10,34	-6,94	0,652	0,834	0,907
DAC-B	$10,84\pm 2,38$	$10,99\pm 3,09$	12,56	-6,49	0,166	0,801	0,888

Table 3: Comparison of evaluation metrics with state-of-the-art methods on CHM prediction.

Figure 8: Representative tile. Normalised (for comparison) student CHM/PAI/FHD maps are spatially coherent with LiDAR-derived references, with strongest agreement for CHM.

Figure 9: Broadleaved patch comparison. Ground truth (top), teacher (middle), and student (bottom) show close CHM agreement with clear crown delineation; PAI/FHD preserve major spatial gradients but appear smoother.

Figure 10: Coniferous patch comparison. Ground truth (top), teacher (middle), and student (bottom) show strong CHM transfer; PAI/FHD remain directionally correct with compressed amplitude.

Figure 11: Metric-wise ITC masking. Tree-crown masks derived with DeepTrees [ 23 ] can be applied consistently to CHM, PAI, and FHD outputs for per-tree summaries.

Remote Sensing

Remote sensing uses satellite, aerial, and ground-based observations to measure and map Earth and other planetary surfaces.

See all 528 papers

Jun 30, 2026

FLORA: A deep learning approach to predict forest attributes from heterogeneous LiDAR data

Forest attributes are essential for national-scale resource monitoring. Airborne LiDAR metrics are among the auxiliary variables most strongly correlated with forest attributes used in National Forest Inventory (NFI) estimates. However, producing wall-to-wall predictions remains challenging when LiDAR data are acquired under heterogeneous conditions. As national LiDAR programs expand across Europe, variability in sensors, flight parameters, seasons, and scan angles limits the robustness of existing models, which are often calibrated for local conditions. We present FLORA (Forest LiDAR Octree Regression with Auxiliary Data), a deep learning framework that predicts six forest attributes: dominant height, total volume, deciduous volume, coniferous volume, basal area, and stem density from heterogeneous LiDAR point clouds. FLORA combines an octree-based backbone with ecological and spatiotemporal auxiliary variables through a late-fusion gating mechanism. Models are trained and evaluated on 32,052 National Forest Inventory plots across mainland France using data from the French LiDAR HD program. A single model trained on both leaf-on and leaf-off acquisitions outperforms season-specific models and improves cross-season robustness. Auxiliary variables provide modest overall gains but contribute more strongly to species-specific volume prediction. FLORA achieves an rRMSE of about 12.3% (R2 = 0.88) for dominant height and 39% (R2 = 0.74) for total volume, providing a robust baseline for large-scale forest attribute estimation from heterogeneous national LiDAR programs.

Emilie Vautier, Clément Mallet, Cédric Vega

Univ Gustave Eiffel, Géodata Paris, IGN, LASTIG, F-77454 Marne-la-Vallée, France · Univ Gustave Eiffel, Géodata Paris, IGN, LIF, F-54000, Nancy, France · Université de Lorraine, Géodata Paris, IGN, LIF, F-54000 Nancy, France

Point Cloud Learning Remote Sensing Airborne Laser Scanning

Ask PDF

Mar 6, 2026

CHMv2: Improvements in Global Canopy Height Mapping using DINOv3

Accurate canopy height information is essential for quantifying forest carbon, monitoring restoration and degradation, and assessing habitat structure, yet high-fidelity measurements from airborne laser scanning (ALS) remain unevenly available globally. Here we present CHMv2, a global, meter-resolution canopy height map derived from high-resolution optical satellite imagery using a depth-estimation model built on DINOv3 and trained against ALS canopy height models. Compared to existing products, CHMv2 substantially improves accuracy, reduces bias in tall forests, and better preserves fine-scale structure such as canopy edges and gaps. These gains are enabled by a large expansion of geographically diverse training data, automated data curation and registration, and a loss formulation and data sampling strategy tailored to canopy height distributions. We validate CHMv2 against independent ALS test sets and against tens of millions of GEDI and ICESat-2 observations, demonstrating consistent performance across major forest biomes.

John Brandt, Seungeun Yi, Jamie Tolan +9

World Resources Institute, 10 G St NE #800, Washington, DC 20002, USA · Fundamental AI Research (FAIR), Meta, 75002 Paris, France · Meta, 1 Hacker Way, Menlo Park, CA 94025, USA +1

Remote Sensing Remote Sensing Image Understanding

Ask PDF

Jun 17, 2026

Viking Hill Dataset: A Lidar-Radar-Camera Dataset for Detection and Segmentation in Forest Scenes

Autonomous robots operating under forest canopies need robust perception of trees and surrounding vegetation across varying seasonal conditions. Existing forestry datasets provide lidar or camera data with per-tree annotations, but none include co-registered 4D imaging radar -- a modality of growing interest for its resilience to visual degradation, surface contamination, and vegetation occlusion. We introduce a multi-sensor forest dataset collected by a mobile robot equipped with a high-resolution FMCW imaging radar, lidar, RGB camera, IMU, and RTK-GNSS. The site was recorded in two sessions under contrasting vegetation states, and 3D cuboid annotations -- including per-tree diameter estimates -- provide shared semantic labels across all three perception modalities. Furthermore, we provide baseline results for semantic segmentation of the radar and lidar point clouds using MinkowskiUNet. Radar achieves IoU scores competitive with lidar for dominant classes (ground 91%, canopy 86%) while lagging on geometrically fine structures such as tree trunks (56% vs. 74%). A cross-modality analysis further compares lidar and radar trunk segmentation against an RGB detection model, and a diameter-stratified evaluation reveals how trunk segmentation quality varies with tree size. Beyond segmentation, the co-registered multi-modal data and RTK-GNSS-aided reference positioning support research in mapping, localization, and sensor fusion under canopy. The dataset and annotation tools are publicly available.

Vladimír Kubelka, Oleksandr Kotlyar, Unal Artan +1

Örebro University, AASS research centre, Robot Navigation and Perception Lab, Örebro, Sweden

Millimeter-Wave Radar mmWave Radar Sensing 3D Semantic Segmentation Robotic Perception Multimodal Sensor Fusion

Ask PDF

Abstract

Remote Sensing

Explore similar work

FLORA: A deep learning approach to predict forest attributes from heterogeneous LiDAR data

CHMv2: Improvements in Global Canopy Height Mapping using DINOv3

Viking Hill Dataset: A Lidar-Radar-Camera Dataset for Detection and Segmentation in Forest Scenes