Synthetic Aperture Imaging (SAI) enables drones to observe forest canopies by warping and averaging multiple displaced views at a common depth. However, occlusion is suppressed rather than eliminated, causing poorly observed regions to appear occupied and leading to systematic biomass overestimation. The Deep Forest project cannot distinguish reconstruction artifacts from real vegetation because its training data relies on measured reflectance points, excluding empty space and preventing the network from learning explicit occupancy boundaries. This thesis addresses the problem by modifying training labels rather than the imaging process. Each simulated scene is rendered twice: as a 9x9 grid of top-down captures for focal-stack generation and as 550 depth-clipped slabs derived from scene geometry to provide occupancy labels, explicitly representing void space as a negative class. A Gazebo pipeline generates procedural forest stands containing 80-200 trees across three species. A pretrained Pix2Pix model translates rendered RGB images into a synthetic near-infrared band, and Light Field Rendering produces aligned focal stacks of 550 planes across three channels. Four architectures were evaluated on a held-out scene. The plain 2D U-Net achieved an IoU of 0.6290, outperforming the volumetric 3D U-Net (0.5707). Established reconstruction methods performed substantially worse: COLMAP achieved 0.3446 and DepthAnything3 0.2959, both after their respective optimal dilation sweeps. An RGB-input variant of the volumetric architecture achieved the highest IoU of 0.6514 without dilation or per-scene calibration. These results highlight the limitations of conventional reconstruction methods, which recover visible surfaces but struggle to infer occupancy within closed canopies.
Figures & tables
Approach
Input
Output
Interior
Needs labels
SfM / MVS [ 18 ]
Multi-view
Sparse points
Partial
No
Learned depth prediction
One or more views
Depth map
No
No
LiDAR [ 5 ]
Active scan
Dense points
Partial
No
DeepForest [ 22 ]
Focal stack
Reflectance
Yes
Yes
This work
Focal stack
Occupancy
Yes
Yes
Table 2.1 : Comparison of reconstruction approaches for self-occluding forest volumes. “Interior” means the method produces an answer for positions below the outermost visible surface.
Figure 3.1 : The pipeline as used for v3, marked accordingly. It shows the data generation from the Gazebo simulator, translation via NIR-GAN, processing into volumetric stacks via LFR, and training the 3D U-Net on the RED, NIR and NDVI channels. Figure 3.3 shows the simpler pipeline v1 and v3b use instead.
Group
Parameter
Value
Forest
Patch size
70×70 units
Trees per scene
80 to 200 (16-value cycle)
Random seed
Scene number
Ground texture index
0
Direct spawning
Enabled
Species
Birch
40 % of trees
Table 3.1 : Simulator configuration used to generate the dataset on Github (training-02). Tree counts cycle through the listed values as scenes are generated, so the dataset covers a range of canopy densities rather than one fixed density. Mostly the parameters are the same as in the original DeepForest implementation, but the tree variety and ground textures have been diversified.
Figure 3.2 : Visual comparison of the data distributions. Left: The simplified DeepForest dataset. Center: The improved simulator with diverse trees and floor textures. Right: Real-world forest data from an October field experiment.
Figure 3.3 : The pipeline as used for v1 and v3b. Neither reads a synthesized channel, so NIR-GAN and the red and NDVI extraction steps of Figure 3.1 drop out entirely, and training runs straight off the rendered RGB stack. The dashed branch from v3b to inference marks that v3b was only ever scored on synthetic scenes in this thesis; v1 is the only variant of the two that was run on the real captures shown in section 4.5 .
Figure 3.4 : Comparison of the original synthetic NIR (left), the mean-variance matched synthetic NIR (center), and the real-world NIR ground truth (right). Matching the average brightness and its spread aligns the synthetic distribution with the real-world domain.
Parameter
Value
Render resolution
512×512
Field of view
35∘
Input views per plane
81
Focal planes
550
Near focal depth
−16.5 units
Far focal depth
+7.8 units
Table 3.2 : Light Field Rendering parameters. The same settings are applied to the red, near-infrared, and NDVI channels, so the three resulting stacks are voxel-aligned.
Role
Scenes
Samples
Training
All remaining Scene_*
550 per scene, ×8 augmented
Validation
Scene_202 , Scene_204 , Scene_205
1,650
Test
Scene_201
550
Table 3.3 : Dataset split. Scene_201 is excluded from both training and validation and used only for the final evaluation, so no weight and no hyperparameter was ever fitted on it.
Version
Type
Input
Depth reasoning
Parameters
Channels
v1
2D U-Net
3×4482
None
1,885,441
RGB
v2
2D U-Net, 12 channels
12×2562
Channel-wise
1,890,625
RED, NIR, NDVI
v3
3D U-Net, from scratch
3×4×4402
Volumetric
5,638,481
RED, NIR, NDVI
v4
Dual-head 3D CNN
1×D×22
Per-voxel patch
2,931,362 per layer
GRAY
Table 3.4 : Overview of the four evaluated architectures. Parameter counts are for the configuration actually trained. The v4 count is given for an axial resolution of 21 input channels, which varies per layer in that model.
Figure 3.5 : Architecture of the proposed 3D U-Net (v3), shown in the deployed configuration with four encoder levels starting at 16 features. The first two pooling stages use a (1,2,2) kernel and leave the depth axis intact. The final max projection along depth collapses the output volume into a single probability map for the target plane.
Parameter
Type
Range or value
Batch size
Categorical
{1,2,4}
Learning rate
Log-uniform
10−5 to 10−3
Initial features
Categorical
{16,32}
Encoder depth
Integer
2 to 4
Input size
Fixed
440×440
Focal planes
Fixed
4 of a 16-plane window
Table 3.5 : Hyperparameter search space for the v3 architecture. Fixed settings are given for reference. The search ran 20 trials, each for four epochs on a deterministic 10 % subsample of the training data.
Trial
Val. loss
Batch
Features
Depth
Bottleneck
Learning rate
16
0.3382
4
16
4
256
3.05×10−4
13
0.3405
4
16
4
256
2.06×10−4
7
0.3426
4
16
3
128
3.21×10−4
18
0.3396
2
32
4
512
4.60×10−5
0
0.3502
4
16
4
256
8.27×10−5
Table 3.6 : The five best trials of the v3 hyperparameter study, ranked by validation loss. Trial 16 was selected. The bottleneck width is the derived quantity f⋅2d .
Model
301 (150)
302 (175)
303 (200)
Pooled
v1 (2D U-Net)
0.6033
0.6069
0.5903
0.5993
v2 (multi-channel)
0.5632
0.5710
0.6145
0.5857
v3 (3D U-Net)
0.5669
0.5522
0.5883
0.5706
v3 before search
0.5407
0.5362
0.5597
0.5466
v3b (RGB)
0.6528
0.6540
0.6874
0.6668
Table 4.1 : Voxel-level IoU on three scenes excluded from training, validation and the hyperparameter search, at 150, 175 and 200 trees (ground-truth occupied fraction 0.326, 0.366 and 0.462). Pooled accumulates the confusion matrix over all three scenes before computing IoU. All 550 planes were evaluated per scene.
Figure 4.1 : Layer 200, the only layer v4 ever produced. Shown in the viridis spectrum with no contrast stretch, so the printed appearance matches what the following stages receive. The layer spans 0.13 to 0.22 of the scale, stored as 34 to 57 of 255. The faint grid is an aliasing artifact of the PSF implementation, not structure.
Metric
Before search
After search
Change
Precision
0.8052
0.7800
−0.0252
Recall
0.6296
0.6802
+0.0506
F1 / Dice
0.7067
0.7267
+0.0200
Specificity
0.9360
0.9194
−0.0166
Accuracy
0.8453
0.8486
+0.0033
Balanced accuracy
0.7828
0.7998
+0.0170
Table 4.2 : The v3 architecture before and after the hyperparameter search, evaluated on Scene_201 at threshold 0.5. The search traded precision for recall and gained 2.4 points of IoU.
Threshold (8-bit)
Probability
IoU
Dice
50
0.20
0.5951
0.7462
65
0.25
0.6062
0.7549
70
0.27
0.6077
0.7560
75
0.29
0.6082
0.7564
80
0.31
0.6079
0.7562
85
0.33
0.6069
0.7554
Table 4.3 : Threshold sweep of the v3 predictions on Scene_201 . Values are given on both the 8-bit storage scale and the normalized probability scale. The maximum sits at 75, well below the default cutoff of 128.
Method
Optimal Dilation Iterations
Peak IoU
Peak Dice
3D U-Net (v3b)
0
0.6514
0.7889
COLMAP (SfM)
40
0.3446
0.5126
DepthAnything3
79
0.2959
0.4567
Table 4.4 : Quantitative evaluation of volumetric reconstruction accuracy across different methodologies on Scene 201. For the point cloud baselines (COLMAP and DepthAnything3), the reported metrics reflect the absolute global maximum achieved after sweeping the optimal number of 3D morphological dilation iterations, so both are tuned on the scene they are reported on. v3b, v3’s architecture retrained on the RGB focal stack ( subsection 4.1.2 ), outputs a dense volume directly, requiring zero morphological expansion, and is entered at the default cutoff of 0.5 without any per-scene calibration. All three values come from the same metric implementation and the same nearest-neighbor resampling onto the label grid ( subsection 3.7.1 ).
Figure 4.2 : Volumetric structures for Scene 201. One method per column, raw output in the top row and processed volume in the bottom row. The ground truth has no raw counterpart, so the fourth slot of the top row holds the color scale. Occupancy runs from 0 for empty space to 1 for a fully occupied voxel, and the second axis gives the opacity the renderer maps that occupancy to. All panels share the same camera pose. Both baselines are shown after seven dilation steps, a middle setting chosen for readability rather than the best step count per method from Table 4.4 . The v3b volume is binarized at the default 0.5 boundary, the same setting Table 4.4 reports. The opacity axis holds for every panel except (b) and (c). Those two raw baseline outputs are so sparse that they are close to invisible at a peak opacity of 0.05, so they are rendered at 0.2 instead. Apparent density should therefore not be compared between those two panels and the rest.
Campaign
Date
Canopy state
v3 density
v3b density
October
2024-10-25
fall
0.143
0.011
March
2025-03-03
leaf-off
0.084
0.012
May
2025-05-14
full leaf-on
0.022
0.010
Table 4.5 : Mean predicted occupancy of v3 and v3b, for all three seasons of the same flight. Both models were run over the full 440-layer stack of each campaign.
Figure 4.3 : Predicted volumes for v3 and v3b on Real_Scene_Oct , 440×440×440 voxels each. Left column: v3, reading the measured red and near-infrared bands. Right column: v3b, v3’s architecture reading the measured RGB band instead. Top row is an oblique view, bottom row looks straight down the depth axis. Both columns come from the same October 2024 flight; the only difference is which measured channels each model reads. All panels use the same camera pose and the color scale on the right, where occupancy runs from 0 for empty space to 1 for a fully occupied voxel and the second axis gives the matching opacity, which peaks at 0.05. Bottom right, below the scale: the raw nadir photograph closest to the center of the flight grid, given for a plausibility check against the reconstructed structure above it. No ground truth exists for this capture, so no metric is reported.
Figure 4.4 : Predicted volumes for v3 and v3b on Real_Scene_Mar , same layout, same camera pose and color scale as Figure 4.3 . Both columns come from the same March 2025 flight, the leaf-off winter canopy. Bottom right, below the scale: the raw nadir photograph closest to the center of the flight grid. No ground truth exists for this capture, so no metric is reported.
Figure 4.5 : Predicted volumes for v3 and v3b on Real_Scene_May , same layout, same camera pose and color scale as Figure 4.3 . Both columns come from the same May 2025 flight, the full leaf-on canopy. Bottom right, below the scale: the raw nadir photograph closest to the center of the flight grid. No ground truth exists for this capture, so no metric is reported.
Ground robots operating in temperate forests suffer from systematic LiDAR occlusions caused by tree trunks, creating persistent blind regions that degrade traversability estimation. In this paper, we propose FOSCo-Net, a self-supervised voxel completion framework that learns to reconstruct occluded geometry from a single scan using temporally accumulated observations as automatic supervision, and we evaluate cylindrical and Cartesian ego-centric grids with multiple decoder designs. FOSCo-Net consistently improves reconstruction quality and traversal safety, with a gated skip decoder achieving the most reliable performance and significantly reducing false obstacle predictions compared to baselines.
Iman Esfandiyar, Dominik Belter
Poznań University of Technology, Institute of Robotics and Machine Intelligence, Poznań, Poland · Łukasiewicz Research Network – Poznań Institute of Technology, Center of Digital Transformation, Department of Artificial Intelligence, Poznań, Poland
Reliable evaluation of visual 3D reconstruction and depth estimation in vegetation-rich outdoor environments is limited by the scarcity of geometric ground-truth data and the cost of complete scanner surveys. We present a LiDARbased reference-generation pipeline for evaluating the zero-shot performance of feed-forward visual-geometry models alongside conventional structure-from-motion (SfM) methods. The pipeline reuses camera images, LiDAR scans, and reference poses from existing multi-sensor localization datasets to generate per-view depth maps and visible reference point clouds. To address visibility errors in accumulated LiDAR scans, we introduce the LargeDepth-Gap Visibility Score (LDGVS) for point-wise visibility filtering and automatic scale selection in Hidden Point Removal(HPR). Experiments on TartanAirV2 and TartanGround show that, compared with Poisson surface reconstruction, which is an approach suited to completed scan surveys, the combined HPR and LDGVS filtering retains approximately three times the point density at comparable absolute relative error. We also apply the pipeline to the BotanicGarden dataset to systematically compare depth estimation and 3D reconstruction methods. Feedforward methods produce dense depth predictions efficiently, but their 3D reconstructions can contain duplicated structures. Conventional SfM generally produces more coherent geometry, but its reconstructions remain sparse and more prone to failure in challenging vegetated scenes. The benchmark toolkit and datasets will be released.
Zhipeng Zhong, Matthew Coombes, Cunjia Liu
Department of Aeronautical and Automotive Engineering, Loughborough University, Leicestershire, UK, LE11 3TU
Occlusion-robust scene recovery remains a major challenge in computational imaging, particularly in natural environments where dense foreground vegetation severely limits visibility. We propose a vision-reasoning-guided light field occlusion removal framework that combines the visibility recovery capability of light field integration (LFI) with the semantic reasoning capacity of vision-language models (VLMs). Multi-view observations are first integrated via LFI to suppress foreground occlusions and produce an initial visibility-enhanced representation. A VLM is then incorporated as a conditional semantic prior to restore degraded structures and recover fine details, guided by the observed measurements. To improve recovery consistency and reduce hallucination artifacts, we introduce a multi-sample fusion strategy that aggregates multiple generated hypotheses into a unified estimate. Experimental results on synthetic and real-world datasets demonstrate state-of-the-art performance, achieving the highest average SSIM across four synthetic light field benchmark scenes (4-Syn) and strong generalization across structured and unstructured acquisition settings. These results highlight the effectiveness of combining physical imaging constraints with vision-language reasoning for robust perception under severe occlusion, with applicability to search-and-rescue and exploratory robotic navigation.
Mohamed Youssef, Oliver Bimber
Department of Computer Science, Johannes Kepler University, Altenbergerstr. 65, Linz, 4040, Austria