cs.LGSep 25, 2026
SaveSAGE: A sampling-aware global evaluation benchmark for species distribution modeling
Organizations: EcoVision Lab, Department of Mathematical Modeling and Machine Learning, University of Zurich, Zurich, Switzerland · ECEO, École Polytechnique Fédérale de Lausanne, Sion, Switzerland · Swiss Federal Institute for Forest, Snow and Landscape Research, WSL Birmensdorf, Switzerland · Ecosystems and Landscape Evolution, Institute of Terrestrial Ecosystems, Department of Environmental Systems Science, ETH Zürich, Zürich, Switzerland ·
Abstract
Knowing where species occur is fundamental for biodiversity research and conservation. Species distribution models (SDMs) link species observations to environmental conditions to estimate their spatial distribution. However, accuracy varies with the underlying data and models, making it essential to know for which species models can be trusted. Deep-learning-based SDMs ("DeepSDMs") now jointly model thousands of species, drawing on hundreds of millions of community-science records. At this scale, averaging performance hides substantial species-level variability, particularly for rare species, often of greatest conservation concern. Records are also strongly biased, making occurrence counts misleading. Accounting for these factors is essential for a reliable and informative evaluation of multi-species SDMs. Here, we introduce a Sampling-Aware Global Evaluation (SAGE) benchmark, combining GBIF records for training with sPlotOpen vegetation plots for presence-absence evaluation across 5771 plant species. We propose an evaluation framework that groups species based on two properties, sampling effort and relative prevalence, which describe how densely a species' range is sampled and how frequently the species is recorded. Evaluating single-species SDMs and multi-species DeepSDMs, we find that Random Forests and DeepSDMs perform best overall, but neither dominates: DeepSDMs outperform single-species SDMs for infrequently recorded species while offering no consistent advantage for well-sampled ones. Crucially, this advantage emerges only when established bias-correction practices, such as spatial thinning and reweighting, are carried over to the deep-learning setting. SAGE helps identify the species and data conditions for which a given approach is beneficial, thereby supporting the development of more transparent and ecologically credible SDMs. Data and code: https://earens.github.io/sage/
Figures & tables
Figure 1: Overview of SAGE. Dataset: presence-only (PO) occurrences from GBIF paired with expert-curated presence-absence (PA) vegetation plots from sPlotOpen ( Sabatini et al.,, 2021 ) , covering non-anonymized plant species. Evaluation framework: each species is characterized by two properties, sampling effort and relative prevalence, defining a two-dimensional space partitioned into sparse versus dense sampling effort and infrequently versus frequently recorded species. Model comparison: models are compared across this space, with performance differences reported per quadrant rather than as a single average.
| Benchmark | Geographic Scope | # Species | Species Identity | Evaluation Data |
| Elith et al., (2020) | Regional | No | Plot data | |
| Cole et al., (2023) | Global | † | Yes | Expert ranges |
| SatBird ( Teng et al.,, 2023 ) | Regional | Yes | Community science | |
| GeoPlant ( Picek et al.,, 2024 ) | Europe | No | Plot data | |
| SAGE (ours) | Global | Yes | Plot data |
Table 1: Comparison of existing SDM benchmark datasets. Color indicators reflect the degree to which each criterion supports transparent, species-resolved evaluation: fully, partially, not fulfilled. † Training data only; evaluation covers up to species.
Figure 2: Dataset statistics. (a) Presence-only training data from GBIF (full variant: occurrence records for species) and (b) presence-absence evaluation data from sPlotOpen vegetation plots ( plots for species). In each panel, the map shows the geographic distribution of records, colored by occurrence density (log scale), and the histogram below shows the distribution of per-species occurrence counts. Color scales and histogram axes differ between panels, reflecting the much larger volume of GBIF data.
Figure 3: Conceptual illustration of species data properties. From the distribution of the observations in range (a) , we derive two complementary properties. (b) Sampling Effort ( ): the fraction of grid cells within a species’ range that contain at least one occurrence record (from any species). (c) Relative Prevalence ( ): the fraction of visited cells within the species’ range in which the species was recorded. (d) Property Space : the resulting two-dimensional space, where species are grouped into four quadrants defined by the median of each property. Species in the lowest decile of either property (i.e., the sparsest or most infrequent taxa; “ ”) are also highlighted.
Figure 4: Examples of species representing the four quadrants of the property space , with their sampling effort (SE in blue) and relative prevalence (RP in orange), and their corresponding percentile ranking (P): (a) Chryselium gnaphalioides sparse and infrequent; (b) Castilleja fissifolia sparse and frequent; (c) Aspidistra attenuata dense and infrequent; (d) Staphylea formosana dense and frequent. Species photos: Chryselium gnaphalioides © Fabien Anthelme, Pl@ntNet, CC BY-SA 2.0 (observed April 3, 2019), Castilleja fissifolia © Fabien Anthelme, Pl@ntNet, CC BY-SA 2.0 (observed November 25, 2021), Aspidistra attenuata © Jacy Chen, iNaturalist, CC BY 4.0 (observed December 9, 2021), Staphylea formosana © Cheng-Te Hsu, iNaturalist, CC BY 4.0 (observed March 31, 2026).
| AUROC (%) | ||||||
| Sparse | Dense | |||||
| Model | Overall | Infrequent | Frequent | Infrequent | Frequent | Time (min) |
| Single-species SDMs | ||||||
| GLM | 84.0 ± 0.0 | 84.5 | 81.9 | 86.3 | 83.2 | 1667 ± 25 |
| BRT | 85.2 ± 0.0 | 85.5 | 83.5 | 87.2 | 84.6 | 1152 ± 29 |
| GAM | 83.8 ± 0.1 | 83.9 | 82.0 | 85.9 | 83.4 | 2073 ± 49 |
Table 2: Models performance comparison on SAGE . Overall and group-level AUROC ( ) are reported as the mean ± standard deviation over 5 random seeds. Training times are given in minutes (see Section 2.3.3 for details). For the corresponding AUPRG performance, see Appendix D.2 .
Figure 5: Performance differences ( AUROC, %) across species groups in the data-property space , with sampling effort on the horizontal axis and relative prevalence on the vertical axis. The thin dashed L-strips along the bottom and left edges isolate the tails of the distribution (bottom: the 10% of species with the most sparsely sampled ranges; left: the 10% of species most infrequently observed). (a–c) Pairwise model comparisons: (a) Random Forest – MaxEnt, (b) Optimized DeepSDM – RF, (c) Optimized DeepSDM – Baseline DeepSDM. Positive values (green) indicate that the model named first in each panel title outperforms the one named second; negative values (purple) indicate the opposite. (d–g) Incremental effect of each design choice with respect to the previous configuration considered for the DeepSDM setup: (d) aggregation, (e) subsampling, (f) architecture tuning, (g) loss function. Positive values (green) indicate that the added design choice outperforms the previous configuration, while negative values (purple) indicate the reverse.
| (a) Subsampling | (b) Architecture | (c) Loss | ||||||
| Threshold | AUROC | Model | AUROC | Pseudo-absence | Weighting | AUROC | ||
| None (full) | 84.0 | MLP | 512 | 4 | 84.3 | Both | None | 84.5 |
| 84.0 | ResNet | 512 | 4 | 84.4 | Random | None | 80.1 | |
| 84.1 | ResNet | 512 | 8 | 84.5 | Target-group | None | 85.5 | |
| 84.3 | ResNet | 1024 | 8 | 84.5 | Target-group | Tuned | 85.7 | |
| 83.7 | Transf. | 512 | 4 | 84.5 | Target-group | Freq. | 85.7 | |
Table 3: Sensitivity analyses of the design choices. Average AUROC ( ) across all species for different (a) subsampling thresholds, (b) model architectures, and (c) loss configurations. Here, denotes the hidden dimension, the number of blocks, and the loss weighting can be based on the frequency of species presences and/or fine-tuned via its hyperparameters.
Figure 6: Prediction maps for four species representing the four quadrants of the property space , comparing Random Forest (b) and the optimized DeepSDM (c) . Predicted probabilities are rescaled for each species and model through min-max normalization; colorbars are therefore not comparable. Training and test data are shown in the top row for reference. Species photos: Quercus ilex © Guillaume Martin, iNaturalist, CC BY-NC 4.0 (observed February 7, 2026), Azorella pedunculata © David Torres, Pl@ntNet, CC BY-NC 4.0 (observed January 18, 2026), Aspidistra attenuata © Jacy Chen, iNaturalist, CC BY 4.0 (observed December 9, 2021), Artemisia mesatlantica © Abdelmonaim Homrani Bakali, teline.fr (Biodiversité végétale du sud-ouest marocain), CC BY-NC 4.0 (observed October 2018).
Figure 7: Geographic distribution of model performance and data properties. Values are obtained by averaging over all species whose ranges overlap each country polygon, or finer administrative units for larger countries. Differences are computed between the optimized DeepSDM and the baseline DeepSDM (a) , and between the optimized DeepSDM and the Random Forest (b) . The two species data properties, sampling effort (c) and relative prevalence (d) , are computed over the same regions. Note : the color scale differs between the two performance panels.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Geographic distribution of occurrence records across the two other GBIF training dataset variants : aggregated to a 1 km grid ( records; samples), and subsampled with occurrences per species ( records; samples). Color indicates observation density per H3 hexagonal cell (resolution 3) on a logarithmic scale. All variants retain the same species.
Figure 9: Distribution of per-species occurrence counts across dataset variants. Top row : full, aggregated (1 km), and subsampled ( ). Bottom row : additional subsampling thresholds ( ). Each panel shows the number of species as a function of occurrence count.
| Sparse | Dense | ||||
| Statistic | Overall | Infreq. | Freq. | Infreq. | Freq. |
| # species | |||||
| Presence-only (training) | |||||
| # occurrences (full) | |||||
| # occurrences (1 km) | |||||
| avg. / species | |||||
Table 4: Detailed dataset statistics for SAGE , overall and stratified by the four species groups defined by the median of sampling effort (sparse/dense) and relative prevalence (infrequent/frequent). PO occurrence counts are reported both for the full training set and the 1 km-aggregated variant used to compute the species properties; PA presences are recorded across sPlotOpen plots. Per-species averages and medians are computed over the species in each group.
| Acronym | Name | Description | Unit |
| bio1 | Annual mean temperature | Mean annual near-surface air temperature calculated as the average of mean monthly temperatures over the year | ∘ C |
| bio2 | Mean diurnal range | Mean diurnal near-surface air temperature range computed as the average of monthly (tasmax tasmin) | ∘ C |
| bio3 | Isothermality | Isothermality: ; compares day–night variability to annual temperature range | unitless |
| bio4 | Temperature seasonality | Standard deviation of mean monthly temperatures | ∘ C |
| bio5 | Max temp. warmest month | Highest monthly mean of daily maximum temperatures across the year | ∘ C |
| bio6 | Min temp. coldest month | Lowest monthly mean of daily minimum temperatures across the year | ∘ C |
Table 5: Bioclimatic predictors derived from CHELSA v2.1 at 1 km spatial resolution ( Brun et al.,, 2022 ) .
| Acronym | Name | Description | Unit |
| ORCDRC | Soil organic carbon | Organic carbon content of the fine earth fraction | g kg -1 |
| PHIHOX | Soil pH (H 2 O) | Soil pH measured in water (stored as pH ) | pH |
| SNDPPT | Sand content | Sand fraction of the fine earth fraction | w% |
| SLTPPT | Silt content | Silt fraction of the fine earth fraction | w% |
| CLYPPT | Clay content | Clay fraction of the fine earth fraction | w% |
| BLDFIE | Bulk density | Bulk density of the fine earth fraction | kg m -3 |
Table 6: Soil property predictors derived from SoilGrids at 250 m spatial resolution ( Hengl et al.,, 2017 ) .
| Acronym | Name | Description | Unit |
| elevation | Elevation | Terrain elevation | m |
| roughness | Roughness | Largest local elevation difference | m |
| tri | Terrain Ruggedness Index | Mean local elevation difference | m |
| tpi | Topographic Position Index | Elevation relative to local neighbourhood | m |
| vrm | Vector Ruggedness Measure | Local dispersion of surface-normal vectors | unitless |
| aspectcosine | Aspect cosine | Cosine-transformed aspect | unitless |
Table 7: Topographic predictors derived from global terrain products at 1 km spatial resolution ( Amatulli et al.,, 2018 ) .
| Acronym | Name | Pressure Description | Unit |
| Built2009 | Built environments | Urban and built-up areas | score |
| Popdensity2010 | Population density | Human population density | score |
| Lights2009 | Night-time lights | Night-time light emissions | score |
| Croplands2005 | Croplands | Intensive cropland agriculture | score |
| Pasture2009 | Pasturelands | Grazing and pasture lands | score |
| Roads | Major roadways | Roads and associated access | score |
Table 8: Human footprint predictors describing anthropogenic pressures. Variables are derived following the Human Footprint methodology ( Venter et al.,, 2016 ) at 1 km spatial resolution.
Figure 10: Distribution of species in the two-dimensional property space , defined by sampling effort (SE) and relative prevalence (RP). (a) Joint distribution shown as a 2D hexbin plot on the original scale, with marginal histograms along each axis. (b) Same distribution shown in percentile-rank space, with percentile thresholds ( , , ) indicated. (c) The resulting species grouping scheme: four quadrants defined by the median of each property and the lowest-decile groups.
Table 9: Hyperparameter grids tested for each standard SDM , along with the resulting overall test AUROC (%; single reference run, seed 0).
Figure 11: Fine-grained performance differences ( AUROC, %) across the property space. Same panel layout as Figure 5 , but computed on a smoothed rank grid (window size 2000, step 50) rather than eight discrete regions. Panels (a–c) show pairwise model comparisons; panels (d–g) show the incremental effect of each design choice.
| AUPRG ( ) | ||||||
| Sparse | Dense | |||||
| Model / Configuration | Overall | Infrequent | Frequent | Infrequent | Frequent | Time (min) |
| Non-DL models | ||||||
| GLM | 79.3 ± 0.2 | 76.6 | 77.6 | 82.0 | 80.3 | 1667 ± 25 |
| BRT | 84.5 ± 0.2 | 83.9 | 82.1 | 87.7 | 84.0 | 1152 ± 29 |
| GAM | 81.0 ± 0.2 | 78.4 | 78.8 | 84.2 | 81.7 | 2073 ± 49 |
Table 10: Models performance comparison on SAGE. Overall and group-level AUPRG ( ) are reported as median ± standard deviation over 5 random seeds.
Figure 12: Performance differences ( AUPRG, %) across species groups defined by sampling effort and relative prevalence , mirroring Figure 5 with AUPRG in place of AUROC.
| Unweighted | -weighted | Area-weighted | ||||
| Model | AUROC | Rank | AUROC | Rank | AUROC | Rank |
| Single-species SDMs | ||||||
| GLM | 84.0 | 5 | 83.8 | 5 | 83.8 | 5 |
| BRT | 85.3 | 3 | 85.1 | 3 | 85.1 | 3 |
| GAM | 83.7 | 6 | 83.5 | 6 | 83.5 | 6 |
| MaxEnt | 84.5 | 4 | 84.3 | 4 | 84.4 | 4 |
Table 11: Model performance under area-weighted evaluation. AUROC (%) averaged over all species, with each evaluation cell weighted uniformly, by the square root of its surveyed area, or proportionally to it. Cells without a recorded area are assigned the mean recorded area ( m 2 ). Scores are computed from the seed-0 run of each model. Down-weighting small plots lowers every score slightly and leaves the ranking unchanged.