Species Identification

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 6Week of Sep 21

Latest papers 34

Sep 28, 2026cs.CV

Automated Species Identification in Camera Trap Images for Wildlife Conservation

Wildlife conservation involves protecting, preserving, and managing wildlife species and their habitats. With today's rapid pace of human development, climate change, and other unsustainable practices, the need for wildlife conservation has heightened. Despite significant progress in species identification using deep-learning models, significant challenges still remain in effectively detecting small animals in low-contrast trap images due to limited feature extraction capabilities. This thesis presents a novel end-to-end framework integrating a self-attention mechanism to address these limitations. The proposed architecture involves a Swin-BiFPN backbone integrated in a Faster RCNN detection network, coupled with a visual semantic extraction module driven by the LLaVA v1.5 (13B) multimodal large language model. The detection framework, capable of extracting crucial features in challenging trap images, demonstrates consistently high results and robust generalization capabilities. Furthermore, the visual semantic extraction module provides zero-shot detection capability, as well as providing valuable insights and emergent cues of the animal's behavior, further supporting the conservation effort. The MLLM evaluation was conducted using both traditional NLP metrics (precision, recall, F1, and SBERT similarity) and subjective scoring by LLM-based judges (GPT-4.1 and GROK 3.0), across five MLLMs, demonstrating the model's strong performance in visual description generation. The proposed framework improves detection accuracy across low-contrast trap images and small animals while also demonstrating zero-shot detection capability leveraging the MLLM.

Nowshin Amin, Nafisa Tabassum Oyshi, Tahmid Abrar Zidan +2

Department of Computer Science and Engineering Brac University

Species Identification Roadside Cameras Species Multimodal Large Language Models

Sep 21, 2026cs.CV

Vision Transformers versus convolutional neural networks for fine-grained orchid genus identification in a species-rich, data-poor flora: a controlled benchmark on the Orchidaceae of New Guinea

New Guinea is the world's richest island flora (~2,856 orchid species), yet most species are represented by only a handful of photographs, far fewer than direct species-level classification requires. Methods for fine-grained identification in such species-rich, data-poor floras are needed, and it remains unclear which backbone architecture and pretraining strategy best support them. We built a two-stage system that first predicts the genus of a query photograph, then retrieves visually similar reference images of candidate species using FAISS. We compared four pretrained backbones -- two Vision Transformers (ViTs; DINOv2, BioCLIP 2) and two CNNs (ConvNeXt V2-L, EfficientNetV2-L) -- fine-tuned under an identical protocol on a fixed, species-stratified partition of 16,701 photographs spanning 120 genera and 1,350 species, assessing accuracy, calibration, error structure, species retrieval, and open-set detection of novel genera. DINOv2 attained the best genus performance (macro top-1 66.9%, 95% CI 63.7-70.6; global top-1 88.9%); both ViTs outranked both CNNs, and general-purpose self-supervised pretraining (DINOv2) outperformed domain-matched biological pretraining (BioCLIP 2) by 7.1 points of macro top-1. Errors concentrated on two abundant genera acting as error attractors. DINOv2 embeddings achieved species Recall@5 of 86.6% and genus Recall@5 of 98.7%; temperature scaling reduced every backbone's Expected Calibration Error to about 0.03; and a distance-based open-set gate flagged unseen genera (mean AUROC 0.958). A self-supervised Vision-Transformer backbone combined with embedding retrieval is an effective, deployable strategy for fine-grained identification in species-rich, data-poor floras. The system is released as an open web application (the New Guinea Orchid Identifier), offering a practical template for other hyperdiverse, under-documented taxa.

Reza Saputra, Diah Harnoni Apriyanti, André Schuiteman +4

College of Science and Engineering, James Cook University. McGregor Rd, Smithfield, Cairns, QLD 4878, Australia · Southwest Papua Natural Resources Conservation Agency, Ministry of Forestry, Indonesia. Jalan Klamono KM 16, Sorong, Southwest Papua Province, Indonesia · The Directorate of Scientific Collection Management, National Research and Innovation Agency (BRIN), Republic of Indonesia, Gedung Kehati, KST Soekarno BRIN, Jl. Raya Jakarta - Bogor KM 46, Cibinong, Kabupaten Bogor, Jawa Barat, 16911, Indonesia. +5

Species Identification Leaf Disease Classification Dinov3 Self-Supervised Vision Transformers Convolutional Neural Networks Vision Transformer

Sep 15, 2026cs.CV

InfoTaxa: Information-Calibrated Label-Free Clustering for Fine-Grained Visual Taxonomy

Label-free clustering of frozen pretrained visual embeddings offers a scalable route to biodiversity monitoring, but image-only fine-grained taxonomy exhibits a consistent coarse-to-fine failure mode: clusters recover broad taxonomic structure yet plateau at species level. We study this behaviour on BIOSCAN-5M through an information-calibrated clustering analysis. BioCLIP~2 features with UMAP and HDBSCAN reach

0.79

AMI at family and

0.67

at genus, substantially improving over the prior image baseline and remaining competitive with oracle-

K

, graph-based, and learned clustering heads on the same frozen features. To diagnose whether the remaining plateau is method-limited or information-limited, we introduce InfoTaxa, which combines clustering efficiency---the fraction of probe-estimated image information recovered by an unsupervised partition---with paired DNA as an audit signal only, not an inference input. The density pipeline recovers approximately

0.90

and

0.81

of the image-available information at order and family, respectively. Held-out late-fusion probes show that adding DNA to the image embedding reduces species-level prediction error by approximately two bits. Robustness analyses cover multiple image encoders, described-species and rare-class subsets, probe diagnostics, and held-out-species coarse-rank generalisation and same-species retrieval. Thus, in the tested setting, species-level label-free clustering is both clustering-limited and representation-limited: improved clustering may recover additional image-exposed structure, but cannot close the DNA-audited information gap alone.

David Ahmedt-Aristizabal, Mohammad Ali Armin, Lars Petersson

Imaging and Computer Vision Group, CSIRO, Australia

Species Identification Biodiversity Monitoring Clustering Life Sciences Research Taxonomy Fine-Grained Perception

Sep 14, 2026cs.CV

CatchMonitor: a machine learning system for automated fish discard quantification

We report on the continued development of CatchMonitor, resulting in a prototype computer vision system designed to automatically quantify discarded fish from video footage collected from Remote Electronic Monitoring (REM) systems on fishing trawlers. The analysis of trawler surveillance footage is a challenging problem due to the real-world conditions on board fishing vessels. Building on our prior work we improve the accuracy of species identification through the application of semi-supervised learning. We utilise a simple and robust object tracking approach, upon which we build our prototype discard quantification system. Finally we analyse the variability of manual discard quantification performed by multiple expert human analysts, using it as a benchmark against which we compare the performance of our system.

Geoff French, Michal Mackiewicz, Mark Fisher +2

Fish Species Identification Computer Vision Quantifying

Sep 12, 2026cs.AI

Can Edge-Deployable Vision-Language Models Identify Species?

Camera traps often run in the field on edge hardware with limited or no connectivity, making small, locally-deployable vision-language models (VLMs) -- not frontier-scale ones -- the practically relevant class to evaluate for species identification. We test whether models in this deployment-relevant 2--8B range carry genuine taxonomic knowledge, evaluating four such VLMs (Qwen3-VL 2B/4B/8B, Gemma3 4B) against the domain-specific specialist BioCLIP (300M parameters) on a 96-species task, comparing clean iNaturalist photographs against camera-trap imagery from 6 LILA.science collections, on two independently-sampled evaluation sets. All models identify species far above chance, but every model -- general-purpose or specialist -- degrades sharply on field imagery (domain gaps of 9.6--26.6 percentage points, consistent across taxonomic levels and both evaluation sets), indicating the degradation reflects general image legibility rather than fine-grained discrimination failure. BioCLIP substantially outperforms every VLM tested (by 33.2--59.2 percentage points across an expanded 200-image sample for every model) despite its far smaller size, suggesting the gap reflects specialized training data rather than model scale; yet BioCLIP's own domain gap (18.0 points) is statistically indistinguishable from the best VLM's (22.3 points), suggesting the clean-to-field degradation itself is a property of the image-quality shift rather than a general-purpose-model weakness. Under open-set prompting, 5.9--9.6% of responses are syntactically valid but taxonomically nonexistent species names; the relative fabrication-rate ranking across models replicates exactly across both evaluation sets, a more robust finding than any single point estimate.

William Zhou, Mayukha Siripuram, Xiao Yan +2

Plano West Senior High School

Species Identification Recent Vision-Language Models Roadside Cameras Species Life Sciences Research

Sep 7, 2026cs.CV

BarkNet-Lite: A Lightweight Texture and Colour Network with the BarkBD Benchmark for Bark-Based Tree Species Recognition in Bangladesh

Tree species recognition supports forest inventory and biodiversity monitoring but still depends on scarce taxonomic expertise. Bark is visible year-round at ground level, yet bark recognition has concentrated on temperate floras and on large ImageNet-pre-trained backbones. We address both gaps. First, we release BarkBD, a bark dataset for Bangladesh: 14,258 uncropped smartphone photographs of 20 native species across four districts and three weather conditions, with a fixed stratified split. Second, we propose BarkNet-Lite, a 2.96M-parameter network trained from random initialisation, pairing a multi-scale texture pathway with a parallel colour-aware pathway. Over five seeds it reaches 96.64+-0.66%accuracyunderstrict single-image inference, within 2.3 points of nine ImageNet-pre-trained backbones fine-tuned under an identical protocol and within one seed-level standard deviation of the smallest ofthem, andtransfers to public benchmarks (95.86% on BarkVN-50, 92.85% on BarkNet 1.0). Grad-CAM, validated by faithfulness and weight-randomisation checks, confirms its decisions rest on bark structure rather than background. The exported single-precision model classifies one photograph in 15.34ms on a commodity smartphone.

Aroshi Ali, Saad Ahmed, Md. Khalid Syfullah

Department of Computer Science and Engineering, Khulna University of Engineering & Technology, Khulna, 9203, Bangladesh · Department of Computer Science and Engineering, Bangladesh Army University of Science & Technology, Saidpur, Bangladesh

Species Identification Benchmark Datasets Frozen Backbone Bangla Texture

Sep 3, 2026cs.CV

IchthyoNoma: Nomenclature and Context Sensitivity of Zero-Shot Biological Vision--Language Models for Bangladeshi Freshwater Fish Recognition

Zero-shot vision-language models (VLMs) are increasingly used as training-free species recognizers, but reported accuracy can reflect more than visual species knowledge. We audit CLIP, BioCLIP, BioCLIP2, and a multilingual Jina CLIP v2 control on seven freshwater-fish categories from two Bangladeshi sources (10,321 images). BioCLIP2 reaches 72.36% on BFF-15 with English common names and 68.91% on SylFishBD with scientific names, versus 25.15% and 14.40% for generic CLIP. BioCLIP2 Bengali prompts are near chance in balanced accuracy (14.22-14.29%); Jina partially recovers Bengali discrimination to 21.89% and 16.36%, but bare Bengali names return to 14.29% on both sources. Paired SylFishBD interventions show no significant weak-blur effect, modest losses from stronger blur/gray masking, a larger white-mask artifact, and strong species dependence. Zero-shot biological VLM scores therefore jointly reflect biological specialization, multilingual alignment, nomenclature, prompt formulation, and context.

Nazim-E-Alam, Tarek Rahman, Md Kishor Morol

American International University Bangladesh · United International University · ELITE Research Lab, Queens, New York, USA

Species Identification Fish Life Sciences Research Zero-Shot

Sep 3, 2026cs.CV

Counting Animals in Camera-Traps Image Sequences without Count Labels: Winning Solution to the iWildCam 2021 Challenge

Camera traps have become an essential tool for wildlife monitoring, motivating the development of computer vision methods for the automated extraction of information from these data. While most prior work has focused on species identification, many ecological applications also require estimating the number of unique individuals appearing across short image sequences. This task is particularly challenging because camera traps typically acquire bursts of images at approximately one frame per second, creating large temporal discontinuities that may make conventional multi-object tracking methods unreliable, and because manually collecting individual count annotations is prohibitively expensive. In this work, we describe the winning solution to the iWildCam 2021 Challenge, which introduced a benchmark for counting animals at the sequence level under realistic annotation constraints where count annotations are unavailable for training. Our approach, MaxBoxCount, combines a strong species classification pipeline with a simple yet effective counting heuristic based on MegaDetector detections to estimate the number of unique individuals without requiring count annotations. Code is available at https://github.com/alcunha/iwildcam2021ufam.

Fagner Cunha, Juan G. Colonna, Eulanda M. dos Santos

Federal University of Amazonas · Manaus, Amazonas, Brazil

Roadside Cameras Species Identification Counting In-The-Wild Videos Multi-Object Tracking

Aug 31, 2026cs.CV

Camera trap classification with deep learning under ground truth uncertainty

Supervised deep learning methods enable the rapid processing of ecological image data, but depend on a costly annotation process. Consequently, training labels are commonly derived from volunteer citizen science projects. However, disagreement among volunteers introduces uncertainty in the "ground truth" data that are assumed to be correct for model training and validation. Using two datasets containing camera trap images with associated volunteer and expert classifications, we investigated the effects of training under higher ground truth uncertainty. We observed improved overall test accuracy, particularly for images that were more difficult for volunteers. Species-level accuracy also generally improved, but generalisation to a different dataset did not. The benefits of ground truth uncertainty were enhanced by pre-training on ImageNet. Pre-training also reduced the number of training epochs required; further reductions in computational cost, but not gains in accuracy, resulted from additional pre-training on other camera trap images. With unbalanced training data, we still observed a clear benefit of increased ground truth uncertainty for overall accuracy, especially on difficult images. Class imbalance improved accuracy for common species, reduced rare species accuracy, and changed patterns of misclassification to more closely resemble mistakes made by volunteers. Our findings have implications for applying deep learning across ecological image types with multiple labels. Practitioners can improve accuracy, especially on difficult examples, by including moderate levels of label disagreement during training and using models pre-trained on general image data. In addition to improving the use of citizen science-derived labels in model training, our study suggests avenues for more effectively integrating human and deep learning classifications in combined workflows. (abridged)

Leonard Hockerts, Peter S. Stewart, Sarthak Arora +1

School of Mathematics & Statistics, University of Glasgow, Glasgow, G12 8QQ,2026 United Kingdom. · 2Collaborative Earth, Berkeley, California, United States

Species Identification Roadside Cameras Ground Truth Uncertainty-Aware Classification Deep Learning Uncertainty

Aug 9, 2026cs.CV

On-Device Multi-Species Malaria Detection with Uncertainty-Calibrated Slide-Level Aggregation

Malaria remains a leading cause of mortality in resource-limited settings, where expert microscopists are scarce. Automated diagnosis based on microscopy images thus has strong potential to improve care delivery. But for an algorithm to deploy, a necessary requirement is that it meet a suite of non-obvious (from a machine learning (ML) perspective) clinical constraints. Therefore, in close consultation with a national health center we developed a malaria diagnosis pipeline which addresses key requirements listed by the health care center but typically ignored in the ML malaria literature. In particular, it includes: (i) stopping criteria (to reduce image acquisition and time-to-result); (ii) human-in-the-loop functionality (for review and accountability); (iii) multi-species discrimination (since treatment varies by species); (iv) thick film detection (standard for microscopy); (v) computationally-efficient uncertainty calculations (to aid clinician review); and (vi) an edge device platform (since internet can be spotty in this catchment area). The mobile system performs all inference on-device using YOLOv13n deployed via TensorFlow Lite. It detects four species and white blood cells from Giemsa-stained thick blood smear images, aggregating per-image detections into slide-level parasitemia with World Health Organization (WHO)-standard quantification. This paper highlights these various clinical constraints and offers methods to address them. Evaluated on 2,739 annotated images across all four species, the system achieves mAP@0.5 of 0.863, per-image parasite count correlation of r = 0.812, slide-level r = 0.951 (soft counting, 10 images/slide), and runs entirely offline with a pipeline time of 10.27 +- 1.65 s per image.

Idaya Seidu, Ahmed Tahiru Issah, Charles B. Delahunt +1

Carnegie Mellon University Africa, Kigali Innovation City, Kigali, Rwanda · University of Washington, Seattle, Washington, USA

Accurate Dengue Forecasting Acute Myeloid Leukemia Species Identification Microscopy

Jul 27, 2026cs.LG

Calibrated Tree-Neural Fusion for Fine-Grained Vegetation Community Classification

Accurate vegetation-community classification is essential for ecological monitoring, habitat assessment, and evidence-based environmental management in heterogeneous landscapes. Existing studies often rely on standalone tree ensembles or generic neural networks, although fine-grained ecological classes frequently exhibit overlapping spectral, topographic, and structural characteristics. Many frameworks also provide limited protection against stacking leakage, insufficient probability calibration, weak minority-class evaluation, and little evidence of stability across repeated data splits. To address these limitations, this study proposes Calibrated EcoTreeFuseNet-Plus, a tree-neural probability-fusion framework that combines out-of-fold tree probabilities, EcoFuseNet-V2 outputs, validation-selected meta-learning, and post-hoc temperature scaling. Raster values from six LiDAR-derived terrain and canopy variables and two hyperspectral vegetation indices were extracted at coordinate-based reference locations. Quality control removed 26 samples with missing elevation and one sample with non-finite NDWI, producing 1,833 complete records across 29 vegetation and non-vegetation classes. On the held-out test set, the proposed model achieved an accuracy of 0.8000, a macro F1-score of 0.7768, a balanced accuracy of 0.7903, and an MCC of 0.7903. Calibration reduced the expected calibration error from 0.3866 to 0.0651 without changing class predictions. Five-seed evaluation yielded a macro F1-score of 0.7717 +/- 0.0112, indicating stable performance across repeated splits. The results demonstrate a reliable discrimination-calibration trade-off for small-sample, fine-grained ecological classification.

Dristi Datta, Md Khalid Hasan Sakib, Manoranjan Paul

School of Information Technology and Engineering, Sydney Metropolitan Institute of Technology (SydneyMet), Sydney, NSW, 2000, Australia · Department of Electrical & Electronic Engineering, Rajshahi University of Engineering & Technology, Rajshahi, Bangladesh · School of Computing, Mathematics and Engineering, Charles Sturt University, Bathurst, NSW, 2795, Australia

Forest Biomass Normalized Difference Vegetation Index Land Cover Species Identification

Jul 20, 2026cs.CV

Benchmarking NACTI Species Recognition in Long-Tailed Regimes

As with most ``in the wild'' collections of the natural world, the North America Camera Trap Images (NACTI) dataset exhibits long-tailed class imbalance, with the largest class covering over 50% of its 3.7M images. Building on the PyTorch Wildlife model, we systematically evaluate Long-Tail Recognition (LTR) methodologies to benchmark species recognition performance, including specialised loss functions and LTR-sensitive regularisation. Our optimised configuration achieves state-of-the-art 99.40% Top-1 accuracy on the NACTI test split, significantly outperforming standard baselines and previously reported top performances. To assess robustness under domain shifts (e.g., night-time captures, occlusion, motion-blur), we extend our evaluation across three independent reduced-bias test sets (including ENA-Detection, Caltech Camera Traps and Missouri Camera Traps). Across these out-of-distribution (OOD) evaluations, our LTR-enhanced model consistently demonstrates substantially stronger generalisation capabilities compared to standard cross-entropy approaches. However, qualitative and quantitative analyses underline that current LTR optimisations cannot fully overcome representational bottlenecks, resulting in catastrophic predictive breakdown for rare `Tail' classes under severe domain shift. For maximum reproducibility, all dataset splits, key code, and network weights are published with this paper at https://github.com/ZehuaLiuY/Species-Classification.

Zehua Liu, Tilo Burghardt

School of Computer Science, University of Bristol, MVB Woodland Rd, BS8 1UB, Bristol, UK

Species Identification Roadside Cameras Imbalanced Classification Pytorch

Jul 16, 2026cs.CV

Multi-Scale ViT Inference with Habitat-Fit Priors and kNN Retrieval for Multi-Species Plant Identification

This paper describes DS@GT ARC's third-place solution to the PlantCLEF 2026 challenge on multi-species plant identification in vegetation quadrat images, where systems must predict every species present in high-resolution (~3000 x 3000 pixel) plot photographs while training only on single-label images of individual plants. The pipeline is built around a fine-tuned DINOv2 ViT-L/14 classifier applied over a multi-scale tile decomposition of each quadrat, with per-tile predictions blended with a FAISS kNN retriever and post-processed by source-aware temporal fusion across repeated plot visits, a habitat-fit demotion that injects geographic and altitude priors from the training data, and a South-Western Europe geographic mask. Habitat-fit demotion and multi-scale aggregation are the largest individual contributors in the ablations. Two complementary training-centric directions, a cross-region transformer with noisy-student distillation on the LUCAS dataset and a label-as-query transformer decoder over synthetic CLS-domain pseudo-quadrats, yielded null results. An inference-time augmentation with instance-aware segmentation crops also did not improve performance. The selected submission reaches a private-leaderboard macro-F1 of 0.43902 (third place; public 0.51096); an unselected configuration of the same pipeline scored above 0.45 on the private set. Code: https://github.com/dsgt-arc/plantclef-2026.

Alper Erten, Murilo Gustineli, Adrian Cheung

Georgia Institute of Technology, North Ave NW, Atlanta, GA 30332

Species Identification Dinov3 Self-Supervised Vision Transformers Prior Knowledge Integration Approximate Nearest-Neighbor Search

Jul 7, 2026cs.CV

EcoVision: AI-Powered Drone Imaging for Salt Marsh Vegetation Monitoring and Dominance Mapping

High-resolution RGB imagery acquired from low-altitude UAV surveys was processed through a modular pipeline incorporating transformer-based semantic segmentation, connected-component vegetation extraction, fine-grained species classification using a ConvNeXt architecture, and grid-based dominance scoring at 2x2m resolution. The framework targeted two ecologically significant halophytic grasses, Spartina maritima and Puccinellia maritima, and was trained using a curated and manually annotated UAV imagery, along with biodiversity imagery sourced from publicly accessible datasets. In order to identify these plants from the imagery, our segmentation yielded reliable species masks (mean IoU = 0.56; pixel-level accuracy = 0.96), while object-level classification achieved very good discrimination (F1 = 0.99). Dominance estimates closely matched quadrat-based field surveys, with mean absolute differences below 8%, preserving fine-scale spatial structure under realistic survey conditions. The developed system, named EcoVision, establishes a practical foundation for scalable, high-resolution salt marsh monitoring, demonstrating how AI-driven workflows can translate pixel-level predictions into ecologically interpretable metrics.

Innocent Onyenonachi, Peter J. Lawerance, Nadia Kanwal

School of Computer Science and Mathematics, Keele University, ST5 3BG, United Kingdom · School of Life Science, Keele University, ST5 3BG, United Kingdom

Aerial Imagery Canopy Species Identification Text Dominance

Jul 6, 2026cs.LG

Reliability and Identifiability in Persona-Trained Monte Carlo: Variance Decomposition, Stability Bounds, and the Identifiability of Heterogeneous News Reaction

Persona-Trained Monte Carlo (PTMC) estimates distributions of market-outcome functionals by repeatedly simulating limit-order-book interaction among

K

neural policy bots whose behavioral personas are drawn from a learned heterogeneity distribution

\mathcal{P}

. This paper develops the statistical theory that makes the word "reliable" precise for such estimators. We decompose estimator variance into a persona-draw component

σ_P^2

and a within-run component

σ_w^2

, give unbiased ANOVA estimators of both, and derive the variance-optimal allocation of a fixed compute budget between outer persona draws and inner replications. A coupling-based stability bound quantifies how misestimation of

\mathcal{P}

and error in the trained policy propagate into the estimand, yielding a three-term total-error budget whose terms are separately estimable; a uniform-in-horizon version holds under a Doeblin condition on the market chain. The main contribution is an identification theory for heterogeneous news reaction: under a fixed response nonlinearity, the aggregate impact curve

A(z)=\mathbb{E}_Q[g(ηz)]

detects heterogeneous news sensitivity through a strict Jensen gap and identifies the distribution

Q

locally via odd moments and Hausdorff determinacy, with sharp failure when the response family is unknown. We provide

\sqrt{n}

-consistent estimators and a boundary-corrected test of homogeneous news reaction. Two separation theorems delimit when PTMC is provably preferable to homogeneous-population simulators and reduced-form forecasters, formalizing an irreducible Jensen bias floor and the Lucas critique as a minimax limit on intervention extrapolation. All proofs are given in full; guarantees are classified as unconditional (Monte Carlo convergence), conditional worst-case (the error budget), or open (the large-

K

mean-field limit).

Salavat Ishbulatov

Independent researcher

Monte Carlo Bias-Variance Decomposition Identifiability Species Identification

Jul 3, 2026eess.AS

Mixture-Constrained Max Pooling Improves Separation-Based Bird Species Classification

Bird species classification from field recordings remains challenging due to overlapping vocalizations and incomplete species labels. We study source separation as a preprocessing for bird species classification to improve multi-species detection. Specifically, we employ an ensemble of two separators, FTRNN and TF-Locoformer, both trained with mixture invariant training (MixIT). To address the false positive gain caused by separation errors in separated outputs, we propose mixture-constrained max pooling (MCM), which clips the predicted probability from each separated channel based on the corresponding species probability in the original mixture. The classifier is applied to each separated output and the original mixture independently, and MCM aggregates the predictions into a final per-species probability. Experiments on two real-world datasets show that the ensemble outperforms individual separators and MCM outperforms standard max pooling across multiple metrics, and reveal that separation leads to both true positive gain for present species and false positive gain for absent species.

Yuzhu Wang, Kalle Lahtinen, Patrik Lauha +4

Signal Processing Research Center, Tampere University, Tampere, Finland · Faculty of Biological and Environmental Sciences, University of Helsinki, Helsinki, Finland · Department of Biological and Environmental Science, University of Jyväskylä, Jyväskylä, Finland

Species Identification Animal Vocalizations Speech Separation Separation Mixture Models Global Average Pooling

Jun 22, 2026cs.CV

PHOEBI: An Open-World Benchmark for Bacterial Identification in Phase-Contrast Microscopy

Optical microscopy (OM) enables rapid, label-free imaging of live bacteria and is the standard instrument for species identification across clinical, environmental, and industrial microbiology. Real samples, however, are routinely polymicrobial and may contain organisms never seen during training, and no computer-vision benchmark evaluates multi-label species identification from phase-contrast microscopy (PCM) of such mixtures. We introduce Phase-contrast Optical bEnchmark for Bacterial Identification (

\textbf{PHOEBI}

), a wet-lab-prepared dataset of

120{,}000

PCM images covering

40

combinations of six rod-shaped species, together with a leave-combinations-out (LCO) protocol that holds out entire species combinations, mirroring a model trained on catalogued mixtures that must recognise new ones. Under LCO, gradient-trained per-image classifiers, from fine-tuned backbones to attention-based multiple-instance learning, collapse on unseen combinations despite high in-distribution accuracy, and the failure lies in how per-image predictions are aggregated rather than in the visual representation. We propose three lightweight

\textbf{anchor-based}

decoders that read each species' presence against fixed geometric prototypes over a shared frozen tile-feature pool, and they remain stable under the same shift. Without additional training, the same features also support open-set rejection of unseen species and the discovery of a new class from unlabeled test images, with negligible disruption to the known classes.

Aaditya Baranwal, Md Jahid Hasan, Shruti Vyas

University of Central Florida

Bacterial Colony Counting Species Identification Microscopy Optical Open-World

Jun 19, 2026cs.CV

Cross-Modal Corroboration for Annotation-Free Wildlife Monitoring

Scaling wildlife monitoring for real-world conservation deployments requires automated analysis of smart sensors that operate under severe annotation scarcity. We propose leveraging expert knowledge of species activity patterns as an annotation-free validation signal for multimodal monitoring pipelines. We operationalize agreement as the alignment of independently derived hourly activity curves both with each other and with published behavioral priors-a three-way convergence that rules out shared-data confounds and dataset-internal correlation as alternative explanations. Our vision pipeline combines zero-shot species detection via BioCLIP 2, sliced inference to handle deployment-constrained camera positioning, and geometry-based geographic localization from camera trap imagery. Our acoustic pipeline detects species vocalizations via a fine-tuned classifier. We validate the pipeline on a breeding herd of Milu deer and demonstrate that both modalities independently recover activity patterns consistent with known deer behavioral ecology with minimal manual annotation. The framework applies to species detectable in both visual and acoustic modalities for which behavioral priors are documented in the literature, suggesting a practical path toward self-validating wildlife-monitoring pipelines at conservation scale.

Bharath Pillai, Varun Viswapriyan, Christopher Stewart +2

The Ohio State University

Species Identification Multi-Modal Sensing Cross-Modal Computer Vision Prior Knowledge Integration

Jun 11, 2026cs.LG

Decoding Insect Song: A Multitask Semisupervised Orthoptera Bioacoustic Classifier

Passive acoustic monitoring holds great promise for ecological inference, yet existing automated tools are typically narrowly trained and non-transferable. We address these limitations with PULSE, a semi-supervised, multi-task framework for Orthoptera bioacoustics, combining weakly-supervised species classification, self-supervised learning on unlabelled field audio, and knowledge distillation from a general-purpose bioacoustic model. Our domain-adapted specialist model outperforms a state-of-the-art general model across all metrics (macro F1: 0.21 vs. 0.07; AUC: 0.74 vs. 0.45; AP: 0.32 vs. 0.19), with active learning further raising F1 to 0.34 and AUC to 0.84. Beyond classification, the learned embeddings encode ecologically meaningful structure, exposed through an interactive visualisation tool for ecological discovery.

Olga Isupova, Danil Kuzin, Ella Browning +2

Leverhulme Centre for Nature Recovery, School of Geography and the Environment, University of Oxford, Oxford, UK

Passive Acoustic Monitoring Species Identification Classifier Audio Understanding Ecosystems New Tasks

Jun 9, 2026cs.CV

Advancing Wood Identification in the Philippines: Utilizing the Xylorix Platform for Efficient AI Model Development and Deployment for Five Key Species

Illegal logging and timber trade continue to pose significant challenges in the Philippines, where accurate wood species identification is essential for enforcement but limited by the need for specialised equipment and expertise. This study aims to evaluate whether AI models for macroscopic wood identification can be developed and deployed by wood scientists without programming expertise using the Xylorix platform, focusing on five Philippine hardwood species: Mangium (Acacia mangium Willd.), Rain Tree [Samanea saman (Jacq.) Merr.], Banuyo (Wallaceodendron celebicum Koord.), Tindalo [Afzelia rhomboidea (Blanco) Vidal], and Ipil [Intsia bijuga (Colebr.) O. Kuntze]. Binary classifiers were trained on 10,663 verified cross-section images from 260 specimens and evaluated using specimen-level mean scoring to mirror operational field conditions. Area Under the ROC Curve (AUC) values ranged from 0.969 (Ipil) to 1.000 (Mangium), and Average Precision (AP) values ranged from 0.589 (Samanea) to 1.000 (Mangium). Four of five species achieved AA grade (AUC and AP both \geq 0.90); Rain Tree received AE (AUC \geq 0.90, AP < 0.60) due to AP compression from its small positive test set (3 specimens). All five classifiers rank their target specimens above non-target specimens with near-perfect fidelity. Specimen-level error analysis revealed 9 false negatives from Ipil, primarily stemming from localized image artifacts and 3 false positives for Rain Tree and 1 false positive for Tindalo caused by shared tribal-level anatomical traits. These findings demonstrate that Xylorix non-programmers can leverage the Xylorix platform to construct operationally reliable wood identification models suitable for field deployment at supply chain checkpoints.

Rosalie C. Mendoza, Vivian C. Daracan, Arlene D. Romano +4

College of Forestry and Natural Resources, University of the Philippines Los Banos · Agritix

Species Identification Trees Species Scientific Agents

Jun 6, 2026math.OC

Latent Structural Categorical Matrix Completion with Application to Quasispecies Analysis

Matrix completion has been extensively studied for real-valued data, but existing methods are often limited in handling categorical variables. We propose LCMC, a double-loop optimization framework for categorical matrix completion via latent factorization based on a binary tensor representation. In this setting, each categorical entry is encoded as a one-hot vector along a third tensor mode, thereby preserving its discrete, non-ordinal nature. The outer loop adaptively estimates the latent dimension by iteratively updating it with feedback from the inner loop, while the inner loop reconstructs the categorical matrix through tensor factorization, supported by a corresponding theoretical analysis. To further improve scalability and robustness, we introduce enhancements including a split-merge-refine strategy and an adaptive data reduction technique. Experiments on synthetic and real-world datasets in viral quasispecies reconstruction, demonstrate that LCMC achieves superior accuracy and efficiency compared to existing methods.

Qian Zhang, Meixia Lin

Engineering Systems and Design, Singapore University of Technology and Design, Singapore · Institute of Statistics and Big Data, Renmin University of China, P.R. China

Tensor Completion Species Identification

May 31, 2026cs.CL

A Registry-Bound LLM Pipeline for Evidence-Grounded Trait Extraction across Tropical Plants, Aquatic Species, and Exotic Pets

We describe a registry-bound large-language-model extraction pipeline producing evidence-grounded structured trait records at scale, on cultivated tropical plant, aquatic, and pet species. Four mechanisms render LLM-derived rows auditable: a versioned 39-key closed-vocabulary trait registry constraining every admitted value to a typed schema; a per-row verbatim evidence quote tying each value to source text; a per-row confidence label (high or medium; low dropped pre-persist); and multi-version preservation. Applied to 409,880 publishable species from the Tropical Species Encyclopedia, the pipeline executed 706,220 runs and persisted 5,489,881 trait records across 409,820 species (99.985%), 81.57% at high confidence. We report three validation layers in descending evidentiary strength: at full population, 90.12% of 5,427,588 evidence-bearing rows have their quote as a verbatim source substring (93.49% excluding one compliance meta-trait); a quote-supports-value audit on n=100 stratified non-red-zone rows yielded 100/100 (lower bound 96.30%); face-validity on n=50 red-zone rows yielded 50/50 Accept (lower bound 92.86%). Per-record correctness is not claimed; 100% pending human curation. The contribution is the four-mechanism framework.

Jeff Wang

NEXLY LLC, United States

Species Identification Large Language Model Pipelines Species Rdf

May 15, 2026cs.CV

Multi-Object Tracking Consistently Improves Wildlife Inference

Camera traps have become a common tool for wildlife monitoring efforts in ecological research and biodiversity conservation. Wildlife classification models have benefited from the increase in wildlife visual data. These models reach high levels of accuracy on curated, high-quality datasets. However, their performance remains sensitive to real-world environmental constraints. They often produce inconsistent predictions when performing inference on temporally coherent sequences. The predicted label for a single individual shifts rapidly between frames. This study exploits the temporal nature of camera-trap data to augment inferred predictions from a wildlife classification model. Specifically, we adopt several standard Multi-Object Tracking (MOT) models to link detections across consecutive frames. The curated trajectories are used to fuse the softmax class probabilities. The fused probability score produces a single consensus class label estimate that overrides misclassifications caused by noise. The analysis of the experimental results shows that our proposed strategy improves over a standalone classifier over all datasets and for each metric. Specifically, the best-performing MOT models gain a weighted F1-Score of 5.1%, 3.1% and 2.0% over the classifier across three MOT datasets.

Mufhumudzi Muthivhi, Jiahao Huo, Fredrik Gustafsson +1

Academy of Computer Science and Software Engineering University of Johannesburg Johannesburg, South Africa · Department of Electrical Engineering Linköping University Linköping, Sweden

Roadside Cameras Species Identification Multi-Object Tracking Species

May 7, 2026cs.LG

Transformer-Based Wildlife Species Classification from Daily Movement Trajectories

Inferring the identity of wildlife species from daily movement data alone is a challenging task. We train sequence models on large-scale, 7-species GPS trajectories from the Movebank platform. Trajectories models are evaluated using a protocol in which entire telemetry studies or regions are heldout during testing. We compare Transformer-based sequence models to LSTM, CNN, and Temporal Convolutional Networks, and find that Transformers consistently achieve higher balanced accuracy with gains of approximately 8 to 22 percentage points, depending on the species and experimental setting. In an elephant binary classification task with 1-hour resolution, the Transformer achieves a balanced accuracy of 0.83 and an AUC of 0.92, substantially outperforming all baseline models. We examine, under data-limited conditions, feature representations by analyzing the differences between a basic displacement-based encoding and an expanded range of movement descriptors that include speed, direction, and turning behavior. With feature augmentation, we see clear performance gains, especially for underrepresented and sparsely represented species, such as large carnivores, lions, and Zebras. Finally, experiments comparing 1-hour and 30-minutetemporal resolutions show that while finer sampling can capture short-term movement patterns for some species, a unified 1-hour resolution yields more promising performance across studies by reducing missing data and ensuring consistent temporal coverage.

Obed Irakoze, Prasenjit Mitra

Department of Electrical & Computer Engineering Carnegie Mellon University Africa Kigali, Rwanda

Species Identification Species Temporal Convolutional Networks Transformer Architectures

May 7, 2026cs.CV

A Novel Graph-Regulated Disentangling Mamba Model with Sparse Tokens for Enhanced Tree Species Classification from MODIS Time Series

Although tree species classification from Moderate Resolution Imaging Spectroradiometer (MODIS) time series data is critical for supporting various environmental applications, it is a challenging task due to several key difficulties: the subtle signature differences among tree species, strong spatial-spectral-temporal information coupling, and the difficulty of modeling large-scale topological context information. To better address these challenges, this paper presents a novel Graph-regulated Disentangled Sparse Mamba model (GDS-Mamba) for enhanced tree species classification, with the following contributions. (1) First, to improve large-scale context modeling, we design a mini-batch graph-regulated approach that explicitly explores topological correlation effects among input images. (2) Second, to disentangle the high-dimensional spatial-spectral-temporal information coupling for improved feature extraction, we propose a novel disentangling Mamba architecture tailored for capturing independent spatial patterns, spectral signatures, and temporal phenology behaviors in MODIS time series. (3) Third, to improve efficiency and subtle feature learning, we design novel sparse token approaches that adaptively learn the optimum subset of tokens to better address the correlation decay problem that bottlenecks standard Mamba models. Extensive experiments using large-scale annual MOD13Q1 data across two Canadian provinces (i.e., Alberta and Saskatchewan) achieved an overall accuracy of 93.94% in Alberta and 80.19% in cross-provincial evaluations, outperforming twelve state-of-the-art classification models.

Motasem Alkayid, Zhengsen Xu, Saeid Taleghanidoozdoozan +7

Department of Geomatics Engineering, University of Calgary, Canada

Species Identification Time-Series Classification Mamba-Based Models Trees Hyperspectral Image

May 5, 2026cs.LG

Probabilistic Classification and Uncertainty Quantification of Sahara Desert Climate Using Feedforward Neural Networks

Climate classification plays a vital role in agricultural planning, hydrological studies, and climate science. One of the most widely used systems for classifying global climate zones is the Köppen-Trewartha (KT) classification. However, the KT classification is fundamentally deterministic, offering discrete labels to spatial locations without accounting for uncertainties in classification. In this paper, we provide a framework for probabilistic modeling of climatic zones. We implement a feedforward artificial neural network (ANN) for classification, allowing for efficient, uncertainty-aware categorization of climatic regions, thereby offering a more nuanced understanding of transitional climate zones compared to traditional deterministic methods. We apply this method to the Sahara Desert region over the 30-year period of 1960 - 1989, using data at more than 400,000 space-time locations from the first 11 years to train our model. We assess the model's short- and long-term classification capabilities to evaluate its stability and accuracy over time. We also compare the probabilistic classification from our model with the traditional KT classification. In addition, we use fluctuation analysis methods to highlight the temporal evolution of climatic zones across the Sahara region and identify areas undergoing significant flux of probabilities of their climate classes, providing insights into broader trends in desertification.

Stephen Tivenan, Indranil Sahoo, Yanjun Qian

Department of Statistical Sciences and Operations Research, Virginia Commonwealth University

Climate Species Identification Neural Network Uncertainty

Apr 27, 2026cs.CV

Multispectral airborne laser scanning dataset for tree species classification: MS-ALS-SPECIES

The shift from stand-level to individual-tree-level forest assessments supports improved species mapping and biodiversity monitoring, particularly in boreal ecosystems where tree species like aspen (Populus tremula L.) play a keystone role. Airborne laser scanning (ALS) is the standard for such inventories, but a major limitation for developing improved species classification methods is the small number of publicly available ALS datasets containing high-quality, field-validated reference data. Recently, multispectral ALS data has shown promise for tree species classification, but the progress is hindered by the lack of open multispectral ALS datasets with high-quality field reference data. This paper presents and details an open multispectral ALS dataset for tree species classification that was used before its public release for an international benchmarking study of machine learning and deep learning classification methods in a related publication by Taher et al.,(2026). The dataset comprises 6326 segment-level point clouds of individual trees representing nine species in southern Finland. The point cloud data has been acquired using two multispectral laser scanning systems each operating at three laser wavelengths: a helicopter-borne system (HeliALS) with a point density exceeding 1000 points\m2 and an Optech Titan system with approximately 35 points\m2. Furthermore, we present a crowdsourcing application that facilitates the collection of high-quality field reference data of tree species in an efficient and scalable manner. Our article showcases the versatility of the open dataset by presenting new analyses on species classification using multispectral data building upon the initial findings of Taher et al.,(2026).

Matti Hyyppä, Klaara Salolahti, Eric Hyyppä +9

Department of Remote Sensing and Photogrammetry, Finnish Geospatial Research Institute FGI, The National Land Survey of Finland, Vuorimiehentie 5, Espoo, FI-02150, Finland

Forest Biomass Species Identification Trees Biodiversity Monitoring Lidar

Apr 27, 2026cs.CV

DeepTaxon: An Interpretable Retrieval-Augmented Multimodal Framework for Unified Species Identification and Discovery

Identifying species in biology among tens of thousands of visually similar taxa while discovering unknown species in open-world environments remains a fundamental challenge in biodiversity research. Current methods treat identification and discovery as separate problems, with classification models assuming closed sets and discovery relying on threshold-based rejection. Here we present DeepTaxon, a retrieval-augmented multimodal framework that unifies species identification and discovery through interpretable reasoning over retrieved visual evidence. Given a query image, DeepTaxon retrieves the top-

k

candidate species with

n

exemplar images each from a retrieval index and performs chain-of-thought comparative reasoning. Critically, we redefine discovery as an explicit, retrieval-based decision problem rather than an implicit parametric memory problem. A sample is novel if and only if the retrieval index lacks sufficient evidence for identification, so each retrieval naturally yields a classification or discovery label without manual annotation, thereby providing automatic supervision for both tasks. We train the framework via supervised fine-tuning on synthetic retrieval-augmented data, followed by reinforcement learning on hard samples, converting high-recall retrieval into high-precision decisions that scale to massive taxonomic vocabularies. Extensive experiments on a large-scale in-distribution benchmark and six out-of-distribution datasets demonstrate consistent improvements in both identification and discovery. Ablation studies further reveal effective test-time scaling with candidate count

k

and exemplar count

n

, strong zero-shot transfer to unseen domains, and consistent performance across retrieval encoders, establishing an interpretable solution for biodiversity research.

Jiawei Wang, Ming Lei, Yaning Yang +8

National University of Singapore · Hunan University · Xiangtan University +4

Species Identification Taxonomy Multimodal Retrieval Discovery Query Image

Apr 24, 2026cs.CV

Understanding Representation Gaps Across Scales in Tropical Tree Species Classification from Drone Imagery

Accurate classification of tropical tree species from unoccupied aerial vehicle (UAV) imagery remains challenging due to high species diversity and strong visual similarity among species at typical image resolutions (centimeters per pixel). In contrast, models trained on close-up citizen science photographs captured with smartphones achieve strong plant species classification performance. Recent advances in UAV data acquisition now enable the collection of close-up images that are spatially registered with top-view aerial imagery and approach the level of visual detail found in smartphone photographs, with the trade-off that such high-resolution photos cannot be acquired for many trees. In this work, we evaluate the performance of existing methods using paired top-view and close-up UAV imagery collected in a species-rich tropical forest. Through fine-tuning experiments, we quantify the performance gap between vision foundation models and in-domain generalist plant recognition models across both image types (high-resolution close-up versus coarser-resolution top-view imagery). We show that classification performance is consistently higher on close-up images than on top-view aerial imagery, and that this performance gap widens for rare species. Finally, we propose that self-supervised representation alignment across these two spatial scales offers a promising approach for integrating fine-grained visual information into canopy-level species classification models based on top-view UAV imagery. Leveraging high-resolution close-up UAV imagery to enhance canopy-level species classification could substantially improve large-scale monitoring of tropical forest biodiversity.

Sulagna Saha, Arthur Ouaknine, Etienne Laliberté +7

Mila – Quebec AI Institute · McGill University · Université de Montréal +4

Aerial Imagery Species Identification Leaf Disease Classification Trees Smartphone Photographs

Apr 17, 2026cs.CV

Automated Palynological Analysis System: Integrating Deep Metric Learning, Detection and Classification in Bright Field Microscopy

Traditional melissopalynology is a time-consuming and subjective process, often taking 4-6 hours per sample. We present an automated, high-throughput microscopy system that integrates H_\infty robust mechanical control with advanced deep learning pipelines for the precise counting, classification, and morphological analysis of pollen grains from Bio Bio region in south central territory in Chile. Our system employs U^2-Net for salient object detection and a DINOv2 Vision Transformer backbone trained via Deep Metric Learning for classification. By integrating Gradient-Weighted Attention, the model provides human-interpretable texture and diagnostic feature annotations. The system achieves a 95.8% classification recall and at least 6x processing speedup compared to manual expert analysis.

J. Staforelli-Vivanco, R. Jofré, P. Coelho +12

Facultad de Ingeniería, Universidad Andres Bello, Talcahuano, CCP-THNO 7100 Chile · Universidad Arturo Prat, Sede Victoria, Chile · Laboratory of Palynology and Plant Ecology, School of Sciences and Technologies, University of Concepción, Los Angeles Campus

Microscopy Species Identification Dinov3 Morphology Infinities

Apr 17, 2026cs.CV

From Articles to Canopies: Knowledge-Driven Pseudo-Labelling for Tree Species Classification using LLM Experts

Hyperspectral tree species classification is challenging due to limited and imbalanced class labels, spectral mixing (overlapping light signatures from multiple species), and ecological heterogeneity (variability among ecological systems). Addressing these challenges requires methods that integrate biological and structural characteristics of vegetation, such as canopy architecture and interspecific interactions, rather than relying solely on spectral signatures. This paper presents a biologically informed, semi-supervised deep learning method that integrates multi-sensor Earth observation data, specifically hyperspectral imaging (HSI) and airborne laser scanning (ALS), with expert, ecological knowledge. The approach relies on biologically inspired pseudo-labelling over a precomputed canopy graph, yielding accurate classification at low training cost. In addition, ecological priors on species cohabitation are automatically derived from reliable sources using large language models (LLMs) and encoded as a cohabitation matrix with likelihoods of species occurring together. These priors are incorporated into the pseudo-labelling strategy, effectively introducing expert knowledge into the model. Experiments on a real-world forest dataset demonstrate 5.6% improvement over the best reference method. Expert evaluation of cohabitation priors reveals high accuracy with differences no larger than 15%.

Michał Romaszewski, Dominik Kopeć, Michał Cholewa +6

Institute of Theoretical and Applied Informatics, Polish Academy of Sciences, Bałtycka 5, Gliwice, 44-100, Poland · University of Lodz, Narutowicza 68, 90-136, Lodz, Poland · MGGP Aero, Kaczkowskiego 6, 33-100, Tarnów, Poland +1

Species Identification Canopy Hyperspectral Image Semi-Supervised Learning Trees

Dec 10, 2025cs.LG

Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors

Visual Species Recognition (VSR) is a fundamental task in scientific disciplines that require species-level identification, including ecology, palynology, evolutionary biology, systematics, and phylogenetics. Automating VSR through machine learning can significantly accelerate these efforts. However, species-level annotation requires extensive domain expertise, making large-scale labeled datasets difficult to obtain. Consequently, few-shot learning (FSL) is a practical paradigm, where an expert model is trained using only a few labeled examples. Meanwhile, Large Multimodal Models (LMMs) have demonstrated unprecedented zero-shot visual recognition capabilities, raising the question of whether they can serve as an alternative to FSL expert models for VSR. We start this work with a systematic comparison between FSL expert models and LMMs, revealing that, despite advanced prompting strategies, contemporary LMMs significantly underperform FSL expert models. Interestingly, we find that LMMs possess a complementary strength: given an image and a shortlist of candidate species generated by an expert model, LMMs can often recover the correct label when the expert model's top prediction is incorrect. Motivated by this, we propose Post-hoc Correction (POC), a simple training-free framework that leverages an LMM to post-process an expert model's top predictions. We develop a multimodal prompting strategy to enable POC to improve FSL expert models by 6.4 accuracy points, averaged over five VSR benchmarks. We show that POC generalizes across diverse FSL methods, visual encoders, and LMMs, making it a practical and effective framework for VSR.

Tian Liu, Anwesha Basu, James Caverlee +1

Department of Computer Science and Engineering, Texas A&M University · Department of AI, University of Macau

Species Identification Large Multimodal Models Post-Hoc

Nov 9, 2025cs.CV

Label-Efficient 3D Forest Mapping: Self-Supervised and Transfer Learning for Instance Segmentation, Semantic Segmentation, and Species Classification

Detailed structural and species information on individual tree level is increasingly important to support precision forestry, biodiversity conservation, and provide reference data for biomass and carbon mapping. Point clouds from airborne and ground-based laser scanning are currently the most suitable data source to rapidly derive such information at scale. Recent advancements in deep learning improved segmenting and classifying individual trees and identifying semantic tree components. However, deep learning models typically require large amounts of annotated training data which limits further improvement. Producing dense, high-quality annotations for 3D point clouds, especially in complex forests, is labor-intensive and challenging to scale. We explore strategies to reduce dependence on large annotated datasets using self-supervised and transfer learning. Our objective is to improve performance across three tasks: instance segmentation, semantic segmentation, and tree classification using realistic and operational training sets. We observe improvements across all tasks, compared to training from scratch, evaluated with their respective metrics. For instance segmentation, self-supervised learning combined with domain adaptation improves AP50 by 16.98%. For semantic segmentation, self-supervised learning alone improves mIoU by 1.79%. For tree classification, hierarchical transfer learning improves mean Jaccard by 6.07%. To simplify use and encourage uptake, we integrated the tasks into a unified framework, streamlining the process from raw point clouds to tree delineation, structural analysis, and species classification. Pretrained models reduce energy consumption and carbon emissions by ~21%. This open-source contribution aims to accelerate operational extraction of individual tree information from laser scanning point clouds to support forestry, biodiversity, and carbon mapping.

Aldino Rizaldy, Fabian Ewald Fassnacht, Ahmed Jamal Afifi +3

Helmholtz-Zentrum Dresden-Rossendorf (HZDR), Helmholtz Institute Freiberg for Resource Technology (HIF), 09599 Freiberg, Germany. · Institute of Geomatics, BOKU University, 1190 Vienna, Austria · Remote Sensing and Geoinformatics, Freie Universität Berlin, 12249 Berlin, Germany +1

Forests Point Clouds Species Identification Semantic Segmentation 3D Mapping Delineation+1

Oct 10, 2025cs.CV

SilvaScenes: Tree Detection and Species Classification from Under-Canopy Images in Natural Forests

Interest in forestry automation is growing alongside rapid advances in deep learning. In particular, tree detection and taxonomic classification are seen as core tasks required for automating field surveys and forestry equipment. These operations must often be performed in under-canopy settings, which pose challenging conditions for perception systems, including heavy occlusion, variable lighting, and dense vegetation. Despite this necessity, current work has yet to properly establish the feasibility of simultaneously executing tree detection and taxonomic classification in natural forests, as available datasets primarily focus on urban settings or on a limited number of species. To address this gap, we present SilvaScenes, a benchmark dataset for instance segmentation of tree species from under-canopy images in natural forests. Collected across five bioclimatic domains in Quebec, Canada, our dataset features 1421 trees from 28 species, with segmentation masks for pixel-precise tree trunk detection and fine-grained species annotations from forestry experts. We demonstrate the relevance and difficult nature of SilvaScenes by evaluating modern deep learning approaches, showing that while trunk segmentation is feasible, with a top mean average precision (mAP) of 69.9% and mean average recall (mAR) of 76.4%, species-aware segmentation remains a significant challenge with an mAP and an mAR of only 39.2% and 68.6%, respectively. Alongside additional experiments, we highlight key challenges, namely that species imbalance and tree occlusion figure among the most pressing issues for precise segmentation and identification. Meanwhile, higher image resolutions contribute to significant performance gains and will likely prove fundamental to these tasks moving forward. Our dataset, source code, and models will be made available at https://github.com/norlab-ulaval/SilvaScenes.

David-Alexandre Duclos, William Guimont-Martin, Gabriel Jeanson +7

Northern Robotics Laboratory, Universit´e Laval, Qu´ebec, QC, G1V 0A6, Canada · D´epartement des sciences du bois et de la forˆet, Universit´e Laval, Qu´ebec, QC, G1V 0A6, Canada · Institut des Sciences de la Forˆet temp´er´ee, Universit´e du Qu´ebec en Outaouais, Ripon, QC, J0V 1V0, Canada

Forests Species Identification Vision Datasets