Effects of interpulse-interval variation on deep-learning classification of bat vocalizations
Authors: Welmoed R. Eversteijn, Burooj Ghani, A. Leonie Baier, Dan Stowell
Organizations: Naturalis Biodiversity Centre, Darwinweg 2, Leiden, 2333 CR, The Netherlands · Department of Cognitive Science and Artificial Intelligence, Tilburg University, Warandelaan 2, Tilburg, 5037 AB, The Netherlands
Temporal context may aid automated bat-species classification, but the contribution of specific features remains unclear. We investigated whether variation in the interpulse interval (IPI)-the time between consecutive call onsets-provides species-discriminative information and whether transformer-based models are more sensitive to this information than convolutional neural networks. We created two matched datasets from European bat recordings: a natural-IPI condition retaining the original call timing and a normalized-IPI condition in which call onsets were spaced at 50-ms intervals. EfficientNet-B0 and PaSST were fine-tuned and evaluated within each condition. In an additional experiment, each architecture was trained separately on natural-IPI and normalized-IPI recordings, and evaluated on the same natural-IPI test set. Finally, the pretrained classifiers BatDetect2 and BAT were evaluated on both conditions. Within-condition IPI normalization had model-dependent effects. PaSST accuracy differed little between the natural-IPI (71±2.3%) and normalized-IPI (70±6.3%) conditions, whereas EfficientNet accuracy increased from 47±4.7% to 57±3.9%. PaSST exceeded EfficientNet under both conditions. In the cross-condition evaluation, models trained on natural-IPI recordings outperformed those trained on normalized-IPI recordings on the natural-IPI test set: accuracy decreased from 54% to 50% for EfficientNet and from 65% to 57% for PaSST. BatDetect2 and BAT differed little between IPI conditions. Overall, we found limited support for the hypotheses that natural IPI variation contributes substantially to bat-species classification and that it is used more effectively by transformer-based than CNN-based models. Nevertheless, the cross-condition performance decrease shows that results obtained under normalized conditions may not transfer fully to natural recordings.
No additional training; pretrained models were used
Models were evaluated on natural-IPI and matched normalized-IPI recordings. Performance was calculated for all 16 species and for only the species represented in each model’s original training data.
Table 1: Overview of the experimental design. Natural IPI refers to recordings retaining the original temporal spacing between calls; normalized IPI refers to recordings in which variation in the intervals between call onsets was reduced.
Species
Number of recordings
Total audio duration (sec)
BatDetect2 training set
BAT training set
Miniopterus schreibersii
21
42
–
✓
Barbastella barbastellus
20
40
✓
–
Eptesicus serotinus
20
40
✓
✓
Hypsugo savii
20
40
–
✓
Myotis capaccinii
20
40
–
✓
Myotis dasycneme
20
40
–
✓
Table 2: Number and total duration of recordings per species. Checkmarks indicate that the species was represented in the original training data of BatDetect2 or BAT.
Parameter
EfficientNet
PaSST
Backbone
CNN
Transformer
Model version
EfficientNet-B0
passt_s_f128_20sec_p16 _s10_ap.474_swa.pt
Pretraining
ImageNet
AudioSet
Input size
3×128×1024
1×128×2000
Clip duration
2 sec
20 sec
Sampling rate
250 kHz
32 kHz
Table 3: Model, preprocessing, and training configurations.
Figure 2: Classification accuracy across the three experiments. (A) Within-condition performance of EfficientNet-B0 and PaSST under the natural-IPI and normalized-IPI conditions. In each case, models were trained and validated on recordings from the same IPI condition using five-fold stratified cross-validation. (B) Cross- and matched-condition performance in Experiment 2. EfficientNet-B0 and PaSST were trained on normalized-IPI (left) and natural-IPI (right) recordings and evaluated on the same fixed test set with natural-IPI variation. Connected points show the change from training on normalized-IPI recordings (normalized → natural) to training on their natural-IPI counterparts (natural → natural). Experiment 2 was the only experiment in which models were evaluated under an IPI condition different from that used for training. (C) Performance of the existing pretrained classifiers BatDetect2 and BAT when evaluated without additional training on the natural-IPI and normalized-IPI datasets. Bars show results for the complete dataset and each model’s known-species subset.
Figure 3: Effect of IPI normalization on species-level F1-scores. Species-level F1-scores under the natural-IPI and normalized-IPI conditions are shown for EfficientNet-B0, PaSST, BatDetect2, and BAT (from left to right). For EfficientNet-B0 and PaSST, scores were obtained when each model was trained and evaluated within the same IPI condition. BatDetect2 and BAT were evaluated without additional training on both dataset conditions. Arrows indicate the change from the natural-IPI score to the normalized-IPI score for each species. Green right-pointing arrows denote an increase following IPI normalization, whereas red left-pointing arrows denote a decrease. Numerical labels report the corresponding change in F1-score in percentage points, calculated as normalized IPI minus natural IPI. In the BatDetect2 and BAT panels, an asterisk indicates a species that was not represented in the respective model’s original training data; asterisks do not indicate statistical significance.
Experiment / Model
Dataset Condition
Accuracy (%)
Macro F1-Score
Experiment 1: Within-Condition Cross-Validation
PaSST
Natural IPI
71.0±2.3
0.68
PaSST
Normalized IPI
70.0±6.3
0.67
EfficientNet-B0
Natural IPI
47.0±4.7
0.43
EfficientNet-B0
Normalized IPI
57.0±3.9
0.52
Experiment 2: Cross-Condition Generalization
Table 4: Classification performance across experimental conditions for custom-trained models and pretrained benchmark classifiers. Values represent accuracy or macro F1-score, as indicated. For Experiment 1, values are reported as the mean ( ± SD) across the five validation folds. Experiments 2 and 3 used single evaluation sets and are therefore reported without variability estimates.
Naturalis Biodiversity Center · Department of Cognitive Science and Artificial Intelligence, Tilburg University · People and Nature Lab, University College London +1