cs.SDMar 23, 2026

AnimalCLAP: Taxonomy-Aware Language-Audio Pretraining for Species Recognition and Trait Inference

Authors: Risa Shinoda, Kaede Shiohara, Nakamasa Inoue, Hiroaki Santo, Fumio Okura

Organizations: The University of Osaka · The University of Tokyo · Institute of Science Tokyo

Abstract

Animal vocalizations provide crucial insights for wildlife assessment, particularly in complex environments such as forests, aiding species identification and ecological monitoring. Recent advances in deep learning have enabled automatic species classification from their vocalizations. However, classifying species unseen during training remains challenging. To address this limitation, we introduce AnimalCLAP, a taxonomy-aware language-audio framework comprising a new dataset and model that incorporate hierarchical biological information. Specifically, our vocalization dataset consists of 4,225 hours of recordings covering 6,823 species, annotated with 22 ecological traits. The AnimalCLAP model is trained on this dataset to align audio and textual representations using taxonomic structures, improving the recognition of unseen species. We demonstrate that our proposed model effectively infers ecological and biological attributes of species directly from their vocalizations, achieving superior performance compared to CLAP. Our dataset, code, and models will be publicly available at https://dahlian00.github.io/AnimalCLAP_Page/.

Figures & tables

Explore similar work

CardsList
  1. ChiroEcho: extending automated bat vocalisation classification beyond the learned taxonomy

    Aug 18, 2026Burooj Ghani, Welmoed Eversteijn, Milan van Hirtum +4ClassificationBioacoustics

  2. BioVITA: Biological Dataset, Model, and Benchmark for Visual-Textual-Acoustic Alignment

    Mar 25, 2026Risa Shinoda, Kaede Shiohara, Nakamasa Inoue +3Cross-Modal AlignmentMultimodal Embedding

  3. Localized time-frequency representation learning for bioacoustic classification in complex soundscapes

    Feb 19, 2025Simen Hexeberg, Mandar Chitre, Matthias Hoffmann-Kuhnt +1BioacousticsAudio Classification