cs.CVNov 3, 2026
SavePredicting Void Points in Self-Occluding Forest Volumes
Organizations: Institute of Computer Graphics, Johannes Kepler University Linz, Austria
Abstract
Synthetic Aperture Imaging (SAI) enables drones to observe forest canopies by warping and averaging multiple displaced views at a common depth. However, occlusion is suppressed rather than eliminated, causing poorly observed regions to appear occupied and leading to systematic biomass overestimation. The Deep Forest project cannot distinguish reconstruction artifacts from real vegetation because its training data relies on measured reflectance points, excluding empty space and preventing the network from learning explicit occupancy boundaries. This thesis addresses the problem by modifying training labels rather than the imaging process. Each simulated scene is rendered twice: as a 9x9 grid of top-down captures for focal-stack generation and as 550 depth-clipped slabs derived from scene geometry to provide occupancy labels, explicitly representing void space as a negative class. A Gazebo pipeline generates procedural forest stands containing 80-200 trees across three species. A pretrained Pix2Pix model translates rendered RGB images into a synthetic near-infrared band, and Light Field Rendering produces aligned focal stacks of 550 planes across three channels. Four architectures were evaluated on a held-out scene. The plain 2D U-Net achieved an IoU of 0.6290, outperforming the volumetric 3D U-Net (0.5707). Established reconstruction methods performed substantially worse: COLMAP achieved 0.3446 and DepthAnything3 0.2959, both after their respective optimal dilation sweeps. An RGB-input variant of the volumetric architecture achieved the highest IoU of 0.6514 without dilation or per-scene calibration. These results highlight the limitations of conventional reconstruction methods, which recover visible surfaces but struggle to infer occupancy within closed canopies.
Figures & tables
| Approach | Input | Output | Interior | Needs labels |
|---|---|---|---|---|
| SfM / MVS [ 18 ] | Multi-view | Sparse points | Partial | No |
| Learned depth prediction | One or more views | Depth map | No | No |
| LiDAR [ 5 ] | Active scan | Dense points | Partial | No |
| DeepForest [ 22 ] | Focal stack | Reflectance | Yes | Yes |
| This work | Focal stack | Occupancy | Yes | Yes |
Table 2.1 : Comparison of reconstruction approaches for self-occluding forest volumes. “Interior” means the method produces an answer for positions below the outermost visible surface.
Figure 3.1 : The pipeline as used for v3, marked accordingly. It shows the data generation from the Gazebo simulator, translation via NIR-GAN, processing into volumetric stacks via LFR, and training the 3D U-Net on the RED, NIR and NDVI channels. Figure 3.3 shows the simpler pipeline v1 and v3b use instead.
| Group | Parameter | Value |
| Forest | Patch size | units |
| Trees per scene | 80 to 200 (16-value cycle) | |
| Random seed | Scene number | |
| Ground texture index | 0 | |
| Direct spawning | Enabled | |
| Species | Birch | 40 % of trees |
Table 3.1 : Simulator configuration used to generate the dataset on Github (training-02). Tree counts cycle through the listed values as scenes are generated, so the dataset covers a range of canopy densities rather than one fixed density. Mostly the parameters are the same as in the original DeepForest implementation, but the tree variety and ground textures have been diversified.
Figure 3.2 : Visual comparison of the data distributions. Left: The simplified DeepForest dataset. Center: The improved simulator with diverse trees and floor textures. Right: Real-world forest data from an October field experiment.
Figure 3.3 : The pipeline as used for v1 and v3b. Neither reads a synthesized channel, so NIR-GAN and the red and NDVI extraction steps of Figure 3.1 drop out entirely, and training runs straight off the rendered RGB stack. The dashed branch from v3b to inference marks that v3b was only ever scored on synthetic scenes in this thesis; v1 is the only variant of the two that was run on the real captures shown in section 4.5 .
Figure 3.4 : Comparison of the original synthetic NIR (left), the mean-variance matched synthetic NIR (center), and the real-world NIR ground truth (right). Matching the average brightness and its spread aligns the synthetic distribution with the real-world domain.
| Parameter | Value |
|---|---|
| Render resolution | |
| Field of view | |
| Input views per plane | 81 |
| Focal planes | 550 |
| Near focal depth | units |
| Far focal depth | units |
Table 3.2 : Light Field Rendering parameters. The same settings are applied to the red, near-infrared, and NDVI channels, so the three resulting stacks are voxel-aligned.
| Role | Scenes | Samples |
|---|---|---|
| Training | All remaining Scene_* | 550 per scene, augmented |
| Validation | Scene_202 , Scene_204 , Scene_205 | 1,650 |
| Test | Scene_201 | 550 |
Table 3.3 : Dataset split. Scene_201 is excluded from both training and validation and used only for the final evaluation, so no weight and no hyperparameter was ever fitted on it.
| Version | Type | Input | Depth reasoning | Parameters | Channels |
|---|---|---|---|---|---|
| v1 | 2D U-Net | None | 1,885,441 | RGB | |
| v2 | 2D U-Net, 12 channels | Channel-wise | 1,890,625 | RED, NIR, NDVI | |
| v3 | 3D U-Net, from scratch | Volumetric | 5,638,481 | RED, NIR, NDVI | |
| v4 | Dual-head 3D CNN | Per-voxel patch | 2,931,362 per layer | GRAY |
Table 3.4 : Overview of the four evaluated architectures. Parameter counts are for the configuration actually trained. The v4 count is given for an axial resolution of 21 input channels, which varies per layer in that model.
Figure 3.5 : Architecture of the proposed 3D U-Net (v3), shown in the deployed configuration with four encoder levels starting at 16 features. The first two pooling stages use a kernel and leave the depth axis intact. The final max projection along depth collapses the output volume into a single probability map for the target plane.
| Parameter | Type | Range or value |
|---|---|---|
| Batch size | Categorical | |
| Learning rate | Log-uniform | to |
| Initial features | Categorical | |
| Encoder depth | Integer | 2 to 4 |
| Input size | Fixed | |
| Focal planes | Fixed | 4 of a 16-plane window |
Table 3.5 : Hyperparameter search space for the v3 architecture. Fixed settings are given for reference. The search ran 20 trials, each for four epochs on a deterministic 10 % subsample of the training data.
| Trial | Val. loss | Batch | Features | Depth | Bottleneck | Learning rate |
|---|---|---|---|---|---|---|
| 16 | 0.3382 | 4 | 16 | 4 | 256 | |
| 13 | 0.3405 | 4 | 16 | 4 | 256 | |
| 7 | 0.3426 | 4 | 16 | 3 | 128 | |
| 18 | 0.3396 | 2 | 32 | 4 | 512 | |
| 0 | 0.3502 | 4 | 16 | 4 | 256 |
Table 3.6 : The five best trials of the v3 hyperparameter study, ranked by validation loss. Trial 16 was selected. The bottleneck width is the derived quantity .
| Model | 301 (150) | 302 (175) | 303 (200) | Pooled |
|---|---|---|---|---|
| v1 (2D U-Net) | 0.6033 | 0.6069 | 0.5903 | 0.5993 |
| v2 (multi-channel) | 0.5632 | 0.5710 | 0.6145 | 0.5857 |
| v3 (3D U-Net) | 0.5669 | 0.5522 | 0.5883 | 0.5706 |
| v3 before search | 0.5407 | 0.5362 | 0.5597 | 0.5466 |
| v3b (RGB) | 0.6528 | 0.6540 | 0.6874 | 0.6668 |
Table 4.1 : Voxel-level IoU on three scenes excluded from training, validation and the hyperparameter search, at 150, 175 and 200 trees (ground-truth occupied fraction 0.326, 0.366 and 0.462). Pooled accumulates the confusion matrix over all three scenes before computing IoU. All 550 planes were evaluated per scene.
Figure 4.1 : Layer 200, the only layer v4 ever produced. Shown in the viridis spectrum with no contrast stretch, so the printed appearance matches what the following stages receive. The layer spans 0.13 to 0.22 of the scale, stored as 34 to 57 of 255. The faint grid is an aliasing artifact of the PSF implementation, not structure.
| Metric | Before search | After search | Change |
| Precision | 0.8052 | 0.7800 | |
| Recall | 0.6296 | 0.6802 | |
| F1 / Dice | 0.7067 | 0.7267 | |
| Specificity | 0.9360 | 0.9194 | |
| Accuracy | 0.8453 | 0.8486 | |
| Balanced accuracy | 0.7828 | 0.7998 |
Table 4.2 : The v3 architecture before and after the hyperparameter search, evaluated on Scene_201 at threshold 0.5. The search traded precision for recall and gained 2.4 points of IoU.
| Threshold (8-bit) | Probability | IoU | Dice |
|---|---|---|---|
| 50 | 0.20 | 0.5951 | 0.7462 |
| 65 | 0.25 | 0.6062 | 0.7549 |
| 70 | 0.27 | 0.6077 | 0.7560 |
| 75 | 0.29 | 0.6082 | 0.7564 |
| 80 | 0.31 | 0.6079 | 0.7562 |
| 85 | 0.33 | 0.6069 | 0.7554 |
Table 4.3 : Threshold sweep of the v3 predictions on Scene_201 . Values are given on both the 8-bit storage scale and the normalized probability scale. The maximum sits at 75, well below the default cutoff of 128.
| Method | Optimal Dilation Iterations | Peak IoU | Peak Dice |
|---|---|---|---|
| 3D U-Net (v3b) | 0 | 0.6514 | 0.7889 |
| COLMAP (SfM) | 40 | 0.3446 | 0.5126 |
| DepthAnything3 | 79 | 0.2959 | 0.4567 |
Table 4.4 : Quantitative evaluation of volumetric reconstruction accuracy across different methodologies on Scene 201. For the point cloud baselines (COLMAP and DepthAnything3), the reported metrics reflect the absolute global maximum achieved after sweeping the optimal number of 3D morphological dilation iterations, so both are tuned on the scene they are reported on. v3b, v3’s architecture retrained on the RGB focal stack ( subsection 4.1.2 ), outputs a dense volume directly, requiring zero morphological expansion, and is entered at the default cutoff of 0.5 without any per-scene calibration. All three values come from the same metric implementation and the same nearest-neighbor resampling onto the label grid ( subsection 3.7.1 ).
Figure 4.2 : Volumetric structures for Scene 201. One method per column, raw output in the top row and processed volume in the bottom row. The ground truth has no raw counterpart, so the fourth slot of the top row holds the color scale. Occupancy runs from 0 for empty space to 1 for a fully occupied voxel, and the second axis gives the opacity the renderer maps that occupancy to. All panels share the same camera pose. Both baselines are shown after seven dilation steps, a middle setting chosen for readability rather than the best step count per method from Table 4.4 . The v3b volume is binarized at the default 0.5 boundary, the same setting Table 4.4 reports. The opacity axis holds for every panel except (b) and (c). Those two raw baseline outputs are so sparse that they are close to invisible at a peak opacity of 0.05, so they are rendered at 0.2 instead. Apparent density should therefore not be compared between those two panels and the rest.
| Campaign | Date | Canopy state | v3 density | v3b density |
|---|---|---|---|---|
| October | 2024-10-25 | fall | 0.143 | 0.011 |
| March | 2025-03-03 | leaf-off | 0.084 | 0.012 |
| May | 2025-05-14 | full leaf-on | 0.022 | 0.010 |
Table 4.5 : Mean predicted occupancy of v3 and v3b, for all three seasons of the same flight. Both models were run over the full 440-layer stack of each campaign.
Figure 4.3 : Predicted volumes for v3 and v3b on Real_Scene_Oct , voxels each. Left column: v3, reading the measured red and near-infrared bands. Right column: v3b, v3’s architecture reading the measured RGB band instead. Top row is an oblique view, bottom row looks straight down the depth axis. Both columns come from the same October 2024 flight; the only difference is which measured channels each model reads. All panels use the same camera pose and the color scale on the right, where occupancy runs from 0 for empty space to 1 for a fully occupied voxel and the second axis gives the matching opacity, which peaks at 0.05. Bottom right, below the scale: the raw nadir photograph closest to the center of the flight grid, given for a plausibility check against the reconstructed structure above it. No ground truth exists for this capture, so no metric is reported.
Figure 4.4 : Predicted volumes for v3 and v3b on Real_Scene_Mar , same layout, same camera pose and color scale as Figure 4.3 . Both columns come from the same March 2025 flight, the leaf-off winter canopy. Bottom right, below the scale: the raw nadir photograph closest to the center of the flight grid. No ground truth exists for this capture, so no metric is reported.
Figure 4.5 : Predicted volumes for v3 and v3b on Real_Scene_May , same layout, same camera pose and color scale as Figure 4.3 . Both columns come from the same May 2025 flight, the full leaf-on canopy. Bottom right, below the scale: the raw nadir photograph closest to the center of the flight grid. No ground truth exists for this capture, so no metric is reported.