cs.LGApr 13, 2026
Savebacpipe: a Python package to make bioacoustic deep learning models accessible
Organizations: Naturalis Biodiversity Center, Leiden, The Netherlands · Department of Intelligent Systems, Tilburg University, Tilburg, The Netherlands · Muséum Nationale d’Histoire Naturelle, Paris, France
Abstract
- Natural sounds have been recorded for millions of hours over the previous decades using passive acoustic monitoring. Improvements in deep learning models have vastly accelerated the analysis of large portions of this data. While new models advance the state-of-the-art, accessing them using tools to harness their full potential is not always straightforward. Here we present bacpipe, a collection of bioacoustic deep learning models and evaluation pipelines accessible through a graphical and programming interface, designed for both ecologists and computer scientists. Bacpipe is a modular software package intended as a point of convergence for bioacoustic models. 2. Bacpipe streamlines the usage of state-of-the-art models on custom audio datasets, generating acoustic feature vectors (embeddings) and classifier predictions. A modular design allows evaluation and benchmarking of models through interactive visualizations, clustering and probing. 3. We believe that access to new deep learning models is important. By designing bacpipe to target a wide audience, researchers will be enabled to answer new ecological and evolutionary questions in bioacoustics. 4. In conclusion, we believe accessibility to developments in deep learning to a wider audience benefits the ecological questions we are trying to answer.
Figures & tables
Figure 1: Processing workflow of bacpipe . From left to right, audio data (Data) is passed into deep neural networks (Models), which generate acoustic feature vectors (Embeddings). Embeddings are then used for a number of tasks which are applied to all selected models for comparability (Evaluation). Evaluation outputs are saved to a standardized folder structure (see Fig. 3 ). By default, evaluations include classification (outputs produce lists of predicted species), dimensionality reduction, (used for visualizations) and clustering (used to measure shared mutual information), but can be expanded for other tasks.
Figure 2: A large unlabeled dataset from a noisy urban soundscape in the Netherlands is visualized in different ways. A : Bacpipe dashboard view of a 2d UMAP visualization of BirdNET embeddings from the entire dataset. The embeddings ( ) were processed from 104 recordings ( 100 hrs of audio). Little boxes indicate what purpose different sections of the dashboard serve. The points are coloured by generated time-of-day timestamps. While the view of the entire dataset ( A ) shows some structure, the grey rectangle corresponds to the zoomed-in view ( B ). In C a spectrogram is shown visualizing the audio data corresponding to a selected embedding point in B . Using the inbuilt functions, a handful of unknown bird vocalizations could be found and isolated, listened to and exported as a .csv file to isolate them from the otherwise anthropogenic noise dominated soundscape. Without any ground truth, these processing and interactive capabilities make evaluating large noise-dominated datasets more feasible.
| name | clfier | training | architecture | dim | trained on | sr [kHz] | size [ ] | ref. |
|---|---|---|---|---|---|---|---|---|
| AudioMAE | no | ssl + ft | ViT | 768 | general | 16.0 | 86.0 | [ 17 ] |
| AudioProtoPNet | yes | supl | ConvNeXt | 768 | birds | 32.0 | 98.0 | [ 15 ] |
| AvesEcho | yes | supl | PaSST | 1024 | birds | 32.0 | 86.3 | [ 10 ] |
| AVES | no | ssl + ft | HuBERT | 768 | general, biotic | 16.0 | 94.2 | [ 13 ] |
| BEATs | no | ssl + ft | HuBERT | 768 | general | 16.0 | 91.0 | [ 7 ] |
| BioLingual | no | supl | CLAP | 512 | animals, birds | 48.0 | 190.0 | [ 34 ] |
Table 1: List of models currently available in bacpipe. The column headings correspond to the following: clfier lists if a classifier is included for the model, training shows the training setup, i.e. ssl for self-supervised learning, supl for supervised learning, sup cl for supervised contrastive learning and ft for fine-tuning. architecture lists the model backbone and dim shows the dimension of the embeddings. trained on shows the model’s training data, sr lists the sample rate in kHz, size shows the model size in millions of parameters and ref. lists the respective publication.
Figure 3: Output folder structure ( A ) and graphical user interface ( B ) of bacpipe . In A, the standardized folder structure is shown. This is how the processed files are saved, showing that the saved embeddings mirror the input structure from Figure 1 . Furthermore, evaluations, dimensionality reduced embeddings and processing logs are saved. In B, the graphical interface that is produced by default is shown. The interface enables evaluating the results in an accessible way.
Figure 4: Case study of a large annotated dataset using AnuraSet which consists of recordings from the Amazonian rainforest [ 5 ] . In A a bacpipe dashboard is shown with a side-by-side comparison of 2d UMAP visualizations of embeddings generated using two different models: AvesEcho_Passt [ 11 ] and NatureBEATs [ 33 ] . Little boxes indicate what purpose different sections of the dashboard serve. The points are coloured by the parent directory of the audio file they were processed from, which corresponds to the recording site. The embedding plot for NatureBEATs shows fewer points, as only the annotated portion of the dataset was chosen to be displayed. For B the model was retrained using the probing procedure (see Section 2.5 ) on species which were annotated in the AnuraSet dataset. With the linear probe, the model is now able to classify frog species and the results of the class-wise performance are shown in a bar plot. In C an automatically generated presence heatmap is shown of the frog species Sphaenorhynchus surdus using the model AvesEcho_Passt with the trained linear probe. These heatmaps can be generated for all species present in the annotations. This workflow demonstrates that bacpipe can be used to retrain models on classes they were not originally trained on and visualize species presence immediately.
| name | brief description |
|---|---|
| config , settings | bacpipe config and bacpipe settings |
| supported_models | list all supported models |
| integrated pipelines | |
| play | runs the entire bacpipe pipeline, for all specified models and evaluation tasks |
| benchmark | calculates precision, recall and f1 score for a classifier based on provided ground truth annotations |
| Loader | class to handle file loading and saving |
Table 2: Selection of API functions of bacpipe . All pipelines can be used with the supported models or models that are passed during runtime.
More in this topic
Bioacoustics
Bioacoustics uses sound to study animal activity, communication, biodiversity, and ecological change.
See all 64 papers