Three LiDARs, Twenty-Five Datasets: Cross-Source Generalization in Tree-Species Recognition
Organizations: Purdue University, Department of Computer Science 305 N University St, West Lafayette, IN 47907, USA
Abstract
FOR-species20K is the largest public LiDAR benchmark for tree-species classification, pooling 25 independently collected terrestrial, mobile, and airborne laser-scanning datasets across 33 species. We introduce a multi-resolution, multi-view ConvNeXtV2 model that improves official hidden-test accuracy from 79.5% to 85.0%, ranking first at the time of submission. We then examine what this score measures. Because the hidden test is species-balanced, its overall accuracy is more comparable to a macro metric than to accuracy under the public set's long-tailed distribution. Moreover, the official random split places every source dataset in both training and evaluation. A logistic probe recovers source identity from out-of-fold model probabilities with 69% accuracy, while geometric features identify the source above 90% within two widely shared species; 18 of the 33 species occur in only one source. Under a source-species-cell holdout, evaluated on 5,566 trees from eight eligible species that retain the same LiDAR-type training support in other sources, the unweighted species mean falls from 82% to 36%, and tree-weighted accuracy falls from 95% to 60%. Count-matched retraining and experiments with two alternative backbones indicate that this gap is not explained by reduced training support alone, while an external PureForest experiment shows a smaller but consistent grouped-split effect. These results establish the 85% leaderboard score as strong in-distribution performance, but not as an estimate of source-disjoint generalization, motivating balanced metrics and source-aware evaluation. The code is available at: https://github.com/JaeLee18/FOR20K-lidar-tree-species
Figures & tables
| Method | Input | OA | Prec. | Rec. | F1 |
| DetailView [ 11 ] | Img | 79.5 | 82.3 | 76.7 | 78.0 |
| YOLOv5 | Img | 77.9 | 84.2 | 75.0 | 77.3 |
| SimpleView | Img | 76.2 | 76.9 | 75.5 | 75.6 |
| Ensemble PointNet++ [ 38 ] | PC | 75.6 | 78.2 | 73.5 | 74.9 |
| MinkNet [ 7 ] | PC | 73.7 | 79.9 | 70.6 | 72.3 |
| PointMixer | PC | 71.1 | 74.4 | 65.5 | 71.1 |
| Backbone | Rend. | OA | macro-F1 | TLS | MLS | ULS |
|---|---|---|---|---|---|---|
| DenseNet-201 3 (baseline) | 256 | 0.818 | 0.761 | 0.814 | 0.860 | 0.816 |
| DINOv2-Base 3 | 224 | 0.819 | 0.758 | 0.805 | 0.894 | 0.836 |
| ConvNeXtV2-Base 3 (ours) | 256 | 0.886 | 0.843 | 0.879 | 0.934 | 0.894 |
| ConvNeXtV2-Base 3 (ours) | 512 | 0.894 | 0.855 | 0.888 | 0.928 | 0.900 |
| + multi-resolution ensemble | 256 512 | 0.899 | 0.860 | 0.893 | 0.936 | 0.903 |
| Species | random | LSO | ||
|---|---|---|---|---|
| Acer_pseudoplatanus | 24 | 0.79 | 0.00 | +0.79 |
| Betula_pendula | 210 | 0.91 | 0.25 | +0.66 |
| Fagus_sylvatica | 44 | 0.93 | 0.34 | +0.59 |
| Quercus_robur | 182 | 0.92 | 0.36 | +0.57 |
| Picea_abies | 1983 | 0.95 | 0.55 | +0.40 |
| Pinus_sylvestris | 3048 | 0.96 | 0.69 | +0.27 |
| Hyperparameter | Value |
|---|---|
| Backbone | ConvNeXtV2-Base (FCMAE pretrained) |
| Backbones | 3 (one per view group) |
| Total params | 264.7M |
| Loss | Logit-adjusted CE ( ) |
| Optimizer | AdamW |
| Backbone LR |
| Model | OA | Bal. acc. | Macro-F1 | OA Macro-F1 |
|---|---|---|---|---|
| ConvNeXtV2 (image) | 0.886 | 0.846 | 0.843 | +0.043 |
| DenseNet-201 (image) | 0.818 | 0.819 | 0.761 | +0.057 |
| DINOv2-Base (image) | 0.819 | 0.759 | 0.758 | +0.061 |
| PointNet++ (point cloud) | 0.573 | 0.433 | 0.460 | +0.113 |
| DGCNN (point cloud) | 0.626 | 0.391 | 0.427 | +0.199 |
| Source dataset | Plat. | Source acc. | |
| Saarinen_2021 | TLS | 1958 | 0.999 |
| Xi_2020b | TLS | 278 | 0.993 |
| FORinstance_SCION | ULS | 97 | 0.990 |
| Luck | TLS | 280 | 0.982 |
| Xi_2020a | TLS | 488 | 0.980 |
| BlueCat_a | TLS | 3474 | 0.973 |
| Fold | OA | macro-F1 | TLS | MLS | ULS | |
|---|---|---|---|---|---|---|
| 0 | 3542 | 0.8905 | 0.8488 | 0.8783 | 0.9486 | 0.9076 |
| 1 | 3542 | 0.8862 | 0.8466 | 0.8805 | 0.9301 | 0.8890 |
| 2 | 3541 | 0.8870 | 0.8386 | 0.8775 | 0.9233 | 0.9048 |
| 3 | 3541 | 0.8820 | 0.8387 | 0.8804 | 0.9156 | 0.8735 |
| 4 | 3541 | 0.8865 | 0.8405 | 0.8770 | 0.9532 | 0.8929 |
| Mean | – | 0.8864 | 0.8427 | 0.8787 | 0.9342 | 0.8936 |
| Species | Support | F1 | ||
|---|---|---|---|---|
| DenseNet | ConvNeXtV2 | |||
| P. sylvestris | 3296 | 0.895 | 0.964 | +0.069 |
| F. sylvatica | 2482 | 0.826 | 0.923 | +0.098 |
| P. abies | 1983 | 0.924 | 0.951 | +0.027 |
| C. betulus | 1243 | 0.648 | 0.720 | +0.072 |
| A. campestre | 1240 | 0.743 | 0.791 | +0.048 |
| Cells | Patches | Random | Grouped | Drop | |
|---|---|---|---|---|---|
| 0 | 40 | 22993 | 0.936 | 0.815 | +0.121 |
| 100 | 39 | 22276 | 0.942 | 0.841 | +0.101 |
| 200 | 38 | 21974 | 0.941 | 0.846 | +0.095 |
| 400 | 37 | 21877 | 0.942 | 0.848 | +0.094 |
| 800 | 27 | 21211 | 0.946 | 0.854 | +0.092 |
| Species | Random | Grouped | Drop | |
|---|---|---|---|---|
| A. alba | 811 | 0.691 | 0.004 | +0.687 |
| Q. robur | 447 | 0.902 | 0.441 | +0.461 |
| P. nigra | 3198 | 0.873 | 0.694 | +0.179 |
| Q. petraea | 3071 | 0.967 | 0.788 | +0.179 |
| P. menziesii | 530 | 0.930 | 0.781 | +0.149 |
| P. abies | 4074 | 0.931 | 0.840 | +0.091 |
Tree Species Classification
Tree species classification identifies tree taxa from images, spectral measurements, 3D structure, and satellite or aerial observations.
See all 18 papers