Mapping and Classification of Trees Outside Forests using Deep Learning

Moritz Lucas; Hamid Ebrahimy; Viacheslav Barkov; Ralf Pecenka; Kai-Uwe Kühnberger; Björn Waske

cs.CVOct 29, 2025

Mapping and Classification of Trees Outside Forests using Deep Learning

Authors: Moritz Lucas, Hamid Ebrahimy, Viacheslav Barkov, Ralf Pecenka, Kai-Uwe Kühnberger, Björn Waske

Organizations: Osnabrück University, Institute for Computer Science, Remote Sensing Osnabrück, Germany · Leibniz Institute for Agricultural Engineering and Bioeconomy, System Process Engineering, Germany · Osnabrück University, Institute of Cognitive Science, Artificial Intelligence, Germany · Osnabrück University, Joint Lab Artificial Intelligence and Data Science, Germany

Abstract

Trees Outside Forests (TOF) play an important role in agricultural landscapes by supporting biodiversity, sequestering carbon, and regulating microclimates. Yet, most studies have treated TOF as a single class or relied on rigid rule-based thresholds, limiting ecological interpretation and adaptability across regions. To address this, we evaluate deep learning for TOF classification using a newly generated dataset and high-resolution aerial imagery from four agricultural landscapes in Germany. Specifically, we compare convolutional neural networks (CNNs), vision transformers, and hybrid CNN-transformer models across six semantic segmentation architectures (ABCNet, LSKNet, FT-UNetFormer, DC-Swin, BANet, and U-Net) to map four categories of woody vegetation: Forest, Patch, Linear, and Tree, derived from previous studies and governmental products. Overall, the models achieved good classification accuracy across the four landscapes, with the FT-UNetFormer performing best (mean Intersection-over-Union 0.74; mean F1 score 0.84), underscoring the importance of spatial context understanding in TOF mapping and classification. Our results show good results for Forest and Linear class and reveal challenges particularly in classifying complex structures with high edge density, notably the Patch and Tree class. Our generalization experiments highlight the need for regionally diverse training data to ensure reliable large-scale mapping. The dataset and code are openly available at https://github.com/Moerizzy/TOFMapper

Figures & tables

Figure 1: Location of the study areas in Germany with detailed insets for each region. The four study areas—Brandenburg (BB), North Rhine-Westphalia North (NRW_N), North Rhine-Westphalia South (NRW_S), and Schleswig-Holstein (SH) are outlined in different colors. White boxes indicate the locations of the representative testing tiles.

State	Acquisition	Tree	Forest	TOF	Patch	Linear	Tree
State	Date	Cover(%)	(%)	(%)	(%)	(%)	(%)
BB	19.07.2022	19.4	16.5	2.9	0.6	1.7	0.6
NRW_N	15.06.2022	13.1	7.4	5.7	1.5	2.9	1.3
NRW_S	11.08.2023	44.0	38.2	5.8	1.2	3.1	1.5
SH	14.06.2021	14.2	7.7	6.5	1.2	3.9	1.4
Overall	–	22.7	17.5	5.2	1.1	2.9	1.2

Table 1: Metadata of the study areas used for training, validation, and testing, including acquisition date, overall tree cover, and the distribution into Forest and TOF into Patch , Linear , and Tree classes.

	Area (m²)		Width (m)		Elongation
Class	Mean	Median	Mean	Median	Mean	Median
Forest	103502.40	11435.58	188.54	111.98	2.03	1.76
Patch	981.93	556.70	30.87	24.79	1.78	1.70
Linear	1688.04	759.24	31.98	19.03	4.52	3.72
Tree	44.44	20.24	5.90	4.71	1.51	1.28

Table 2: Geometric properties of TOF classes in the reference data.

Figure 2: Examples of the structural classes: Forest, Patch, Linear, and Tree.

Model	Parameters	Architecture
ABCNet	13.6 M	Bilateral CNN with spatial and contextual paths ( Li et al., 2021 )
BANet	12.7 M	Parallel Transformer and CNN paths ( Wang et al., 2021 )
DC-Swin	66.9 M	Swin Transformer encoder with dense decoder ( Wang et al., 2022a )
FT-UNetFormer	96.0 M	Full Transformer with Swin encoder ( Wang et al., 2022b )
LSKNet	14.4 M	CNN with Large Selective Kernel modules ( Li et al., 2023b )
U-Net	44.6 M	Encoder-decoder CNN with skip connections ( Ronneberger et al., 2015 )

Table 3: Comparison of the tested models, including the number of parameters and a brief description of their architectures.

Section	Hyper-parameter	Value / Setting
Hardware	GPUs	4 × NVIDIA A100 (40 GB)
Hardware	Iterations / epoch	6 750
Augment	Photometric	Random brightness, contrast saturation, hue
	Geometric	H/V flip, rotation (±15°)
	Regularisation	Coarse dropout
Optimise	Optimiser	AdamW + Lookahead

Table 4: Training configuration and data-augmentation settings.

Dataset	Training, Validation	Test
Combination 1	SH, NRW_N, NRW_S	BB
Combination 2	SH, BB, NRW_S	NRW_N
Combination 3	SH, NRW_N, BB	NRW_S
Combination 4	BB, NRW_N, NRW_S	SH

Table 5: Dataset configurations used to evaluate the spatial generalization capability of the method, with different study areas assigned to training, validation, and testing.

Figure 3: Comparison of TOF mapping and classification results for all tested models in selected subregions.

	Mean		Forest		Patch		Linear		Tree
Model	mIoU	mF1	IoU	F1	IoU	F1	IoU	F1	IoU	F1
ABCNet	0.709 $\pm$ 0.037	0.821 $\pm$ 0.025	0.944 $\pm$ 0.032	0.971 $\pm$ 0.058	0.532 $\pm$ 0.067	0.695 $\pm$ 0.056	0.739 $\pm$ 0.063	0.850 $\pm$ 0.042	0.622 $\pm$ 0.011	0.767 $\pm$ 0.009
BANet	0.656 $\pm$ 0.025	0.777 $\pm$ 0.022	0.937 $\pm$ 0.058	0.967 $\pm$ 0.032	0.422 $\pm$ 0.081	0.593 $\pm$ 0.082	0.689 $\pm$ 0.027	0.816 $\pm$ 0.019	0.575 $\pm$ 0.025	0.730 $\pm$ 0.021
DC-Swin	0.714 $\pm$ 0.019	0.824 $\pm$ 0.018	0.949 $\pm$ 0.037	0.974 $\pm$ 0.020	0.570 $\pm$ 0.075	0.726 $\pm$ 0.065	0.751 $\pm$ 0.017	0.858 $\pm$ 0.011	0.586 $\pm$ 0.023	0.727 $\pm$ 0.019
FT-UNetFormer	0.739 $\pm$ 0.016	0.843 $\pm$ 0.012	0.952 $\pm$ 0.045	0.975 $\pm$ 0.024	0.606 $\pm$ 0.055	0.754 $\pm$ 0.044	0.774 $\pm$ 0.020	0.872 $\pm$ 0.013	0.626 $\pm$ 0.019	0.770 $\pm$ 0.015
LSKNet	0.697 $\pm$ 0.016	0.812 $\pm$ 0.013	0.943 $\pm$ 0.047	0.971 $\pm$ 0.026	0.545 $\pm$ 0.058	0.705 $\pm$ 0.052	0.718 $\pm$ 0.030	0.836 $\pm$ 0.020	0.582 $\pm$ 0.016	0.736 $\pm$ 0.013

Table 6: Per-class IoU and F1 scores across all study areas. mIoU and mF1 are calculated as unweighted averages over all classes. Standard deviations indicated by (

\pm

) show variation across the four study areas. The highest values for each metric are highlighted in bold.

Ground Truth	Classification Results
	Background	Forest	Patch	Linear	Tree
Background	99.17	0.37	0.07	0.24	0.16
Forest	1.34	97.94	0.23	0.40	0.09
Patch	5.61	8.11	73.19	9.16	3.93
Linear	5.05	4.70	1.37	86.34	2.54
Tree	11.78	1.65	3.27	5.01	78.28

Table 7: Normalized error matrix for the best-performing model (FT-UNetFormer). Rows represent ground truth classes. Columns show the predicted classification results.

Figure 4: Qualitative results of FT-UNetFormer on selected examples (a–e), showing the RGB orthophoto, ground truth, classification, and a difference map. The difference map marks each pixel as Correct, Omission, Commission, or Class mismatch.

Class	GT Area (ha)	Pred. Area (ha)	Bias (%)
Background	1549.72	1548.02	$-0.11$
Forest	347.97	359.40	$+3.28$
Patch	22.95	15.30	$-33.35$
Linear	70.07	64.20	$-8.38$
Tree	17.16	20.95	$+22.14$

Table 8: Area comparison between ground truth and prediction per class.

Metric	Class	SH	BB	NRW_N	NRW_S
IoU	Forest	0.97	0.96	0.88	0.96
	Patch	0.59	0.51	0.61	0.64
	Linear	0.79	0.80	0.77	0.75
	Tree	0.62	0.59	0.63	0.64
F1	Forest	0.99	0.98	0.93	0.98
	Patch	0.74	0.68	0.76	0.78

Table 9: Comparison of IoU, F1, precision, and recall of FT-UNetFormer for different classes across study areas.

Figure 5: Comparison of the SWF (left) and the FT-UNetFormer prediction (right) over example areas a and b. SWF represents structural classes as Linear and Patch only, whereas FT-UNetFormer additionally maps individual Tree objects and Forest .

Class	SWF (ha)	Pred. (ha)	Pred. vs SWF	Intersection (ha)	IoU
Linear	85.39	63.93	$-25.1\%$	31.62	$26.9\%$
Patch	4.05	15.23	$+276.0\%$	0.91	$5.0\%$

Table 10: Area-based comparison of SWF and the prediction (ftunetformer) for the linear and patch classes, aggregated across all study areas.

Metric	Class	SH	BB	NRW_N	NRW_S
IoU	Forest	0.94 (-0.03)	0.91 (-0.05)	0.86 (-0.02)	0.89 (-0.07)
	Patch	0.36 (-0.23)	0.38 (-0.13)	0.48 (-0.13)	0.47 (-0.17)
	Linear	0.71 (-0.08)	0.59 (-0.21)	0.74 (-0.03)	0.53 (-0.22)
	Tree	0.44 (-0.18)	0.46 (-0.13)	0.52 (-0.11)	0.47 (-0.17)
F1	Forest	0.97 (-0.02)	0.95 (-0.03)	0.92 (-0.01)	0.94 (-0.04)
	Patch	0.52 (-0.22)	0.55 (-0.13)	0.65 (-0.11)	0.64 (-0.14)

Table 11: Comparison of spatial generalization of the FT-UNetFormer. The table presents IoU, F1 scores, precision, and recall for different classes across study areas. Values in parentheses indicate the difference between the model trained on three study areas and the model trained on all study areas.

Figure 6: The first row shows results from the FT-UNetFormer trained on three study areas and tested on an unseen region. The second row presents results from the model trained on all study areas. The third row displays the ground truth.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Training loss and validation mean Intersection over Union (mIoU) for the tested models (ABCNet, LSKNet, FT-UNetFormer, DC-Swin, BANet, and U-Net). Solid lines represent training loss, while dashed lines indicate validation mIoU.

Figure 8: Results of the best-performing model (FT-UNetFormer) in four testing tiles.

Input	mIoU	mF1	mPrecision	mRecall
RGB	0.732	0.838	0.840	0.833
RGB+NIR	0.735	0.841	0.838	0.839
RGB+nDSM	0.764	0.863	0.857	0.866
RGB+NIR+nDSM	0.754	0.856	0.862	0.847

Appendix

Table 12: Mean test metrics of FT-UNetFormer trained with additional input modalities. NIR denotes the additional aerial near-infrared band and nDSM the normalised digital surface model. Values in parentheses give the relative change from the RGB baseline.

Class	IoU	F1	Precision	Recall
Forest	0.904 ( $-0.050$ )	0.949 ( $-0.027$ )	0.977 ( $+0.006$ )	0.922 ( $-0.059$ )
Patch	0.623 ( $+0.028$ )	0.774 ( $+0.026$ )	0.727 ( $-0.066$ )	0.825 ( $+0.127$ )
Linear	0.779 ( $+0.006$ )	0.875 ( $+0.004$ )	0.922 ( $+0.046$ )	0.830 ( $-0.038$ )
Tree	0.674 ( $+0.077$ )	0.816 ( $+0.060$ )	0.785 ( $+0.035$ )	0.850 ( $+0.086$ )
Mean	0.750 ( $+0.015$ )	0.856 ( $+0.016$ )	0.851 ( $+0.005$ )	0.863 ( $+0.029$ )

Appendix

Table 13: Per-class metrics applying patch oversampling and the change to the baseline in parenthesis.

Explore similar work

SilvaScenes: Tree Detection and Species Classification from Under-Canopy Images in Natural Forests
Oct 10, 2025David-Alexandre Duclos, William Guimont-Martin, Gabriel Jeanson +7Forests Species Identification
Northern Robotics Laboratory, Universit´e Laval, Qu´ebec, QC, G1V 0A6, Canada · D´epartement des sciences du bois et de la forˆet, Universit´e Laval, Qu´ebec, QC, G1V 0A6, Canada · Institut des Sciences de la Forˆet temp´er´ee, Universit´e du Qu´ebec en Outaouais, Ripon, QC, J0V 1V0, Canada
Label-Efficient 3D Forest Mapping: Self-Supervised and Transfer Learning for Instance Segmentation, Semantic Segmentation, and Species Classification
Nov 9, 2025Aldino Rizaldy, Fabian Ewald Fassnacht, Ahmed Jamal Afifi +3Forests Point Clouds
Helmholtz-Zentrum Dresden-Rossendorf (HZDR), Helmholtz Institute Freiberg for Resource Technology (HIF), 09599 Freiberg, Germany. · Institute of Geomatics, BOKU University, 1190 Vienna, Austria · Remote Sensing and Geoinformatics, Freie Universität Berlin, 12249 Berlin, Germany +1
SegmentAnyTreeV2: Scaling Transformer-Based Tree Instance Segmentation Across Sensors, Platforms, and Forests
Jun 6, 2026Maciej Wielgosz, Stefano Puliti, Rasmus AstrupForests Voxel
Norwegian Institute of Bioeconomy Research (NIBIO), Høgskoleveien 7, 1433 Ås, Norway

Oct 10, 2025, cs.CVJ/K move · Enter open · S save

SilvaScenes: Tree Detection and Species Classification from Under-Canopy Images in Natural Forests

David-Alexandre Duclos, William Guimont-Martin, Gabriel Jeanson, Arthur Larochelle-Tremblay+6

Northern Robotics Laboratory, Universit´e Laval, Qu´ebec, QC, G1V 0A6, Canada · D´epartement des sciences du bois et de la forˆet, Universit´e Laval, Qu´ebec, QC, G1V 0A6, Canada · Institut des Sciences de la Forˆet temp´er´ee, Universit´e du Qu´ebec en Outaouais, Ripon, QC, J0V 1V0, Canada

Forests Species Identification Vision Datasets

Open paper AskPDF arXiv

Interest in forestry automation is growing alongside rapid advances in deep learning. In particular, tree detection and taxonomic classification are seen as core tasks required for automating field surveys and forestry equipment. These operations must often be performed in under-canopy settings, which pose challenging conditions for perception systems, including heavy occlusion, variable lighting, and dense vegetation. Despite this necessity, current work has yet to properly establish the feasibility of simultaneously executing tree detection and taxonomic classification in natural forests, as available datasets primarily focus on urban settings or on a limited number of species. To address this gap, we present SilvaScenes, a benchmark dataset for instance segmentation of tree species from under-canopy images in natural forests. Collected across five bioclimatic domains in Quebec, Canada, our dataset features 1421 trees from 28 species, with segmentation masks for pixel-precise tree trunk detection and fine-grained species annotations from forestry experts. We demonstrate the relevance and difficult nature of SilvaScenes by evaluating modern deep learning approaches, showing that while trunk segmentation is feasible, with a top mean average precision (mAP) of 69.9% and mean average recall (mAR) of 76.4%, species-aware segmentation remains a significant challenge with an mAP and an mAR of only 39.2% and 68.6%, respectively. Alongside additional experiments, we highlight key challenges, namely that species imbalance and tree occlusion figure among the most pressing issues for precise segmentation and identification. Meanwhile, higher image resolutions contribute to significant performance gains and will likely prove fundamental to these tasks moving forward. Our dataset, source code, and models will be made available at https://github.com/norlab-ulaval/SilvaScenes.

Abstract

Explore similar work

SilvaScenes: Tree Detection and Species Classification from Under-Canopy Images in Natural Forests

Label-Efficient 3D Forest Mapping: Self-Supervised and Transfer Learning for Instance Segmentation, Semantic Segmentation, and Species Classification

SegmentAnyTreeV2: Scaling Transformer-Based Tree Instance Segmentation Across Sensors, Platforms, and Forests