Overview

Audio classification studies how to assign sounds to labels, from spoken words and music genres to bird calls, machine faults, and respiratory conditions. Systems commonly use spectrograms, mel-frequency cepstral coefficients, or raw waveforms with convolutional networks, transformers, and pretrained or self-supervised audio encoders; limited-label settings also use transfer learning, few-shot methods, and audio-language models. Benchmarks such as AudioSet, ESC-50, UrbanSound8K, and ICBHI test different tasks, but strong benchmark scores may not carry over across recording devices, locations, speakers, or species. A central challenge is building reliable models from scarce or weak labels while avoiding data leakage and shortcuts, and keeping models efficient enough for clinical and on-device use.

Latest papers

154 papers12 papers in the last four weeks, down 29% on the four weeks before. 0.1% of all new papers.

Nov 13, 2026

FEND: Far-End Detection in Outbound Voice AI Calls

When a conversational Voice AI agent places an outbound call, the far end may be a live person, a voicemail greeting, an interactive voice response (IVR) system, or increasingly another AI system answering on the callee's behalf. Standard Answering Machine Detection (AMD) reports only whether a human or a machine picked up, a design meant for human call centres, where one machine is as uninteresting as another. That single bit does not tell a Voice AI agent what to do, since leaving a voicemail message, navigating an IVR and holding a short exchange with another AI are three different actions and a two-valued label cannot distinguish them. In 1,000 randomly sampled calls from our own outbound deployment, a live person answered 48.1%, a voicemail system 44.6%, an AI 6.9% and an IVR 0.4%. Our proposed framework, FEND, reads the far end's audio and its streaming transcript and runs one detector for each automated class (voicemail, IVR, AI callee), with human the default when nothing fires, returning a single label within 20 seconds. On a public benchmark the voicemail components reach 100% accuracy without ever misclassifying a live human, where Asterisk's AMD misclassifies 30 to 66% of them. For the AI callee, acoustic features do not transfer across recording conditions, but speech timing does, separating AI from human at 0.952 ROC-AUC, because synthesised speech leaves far shorter gaps between words. On production traffic, separating a live person from any automated far end reaches F1 0.979, and only 2.9% of live people are given an automated label, the error that matters most for an agent acting on the result.
Nov 6, 2026

SAS: Scene-Annotated AudioSet Strong for Joint Scene-Event Analysis

Context is central to human auditory perception, as listeners interpret sounds within their acoustic environment to resolve ambiguity. However, large-scale datasets that jointly capture sound events and acoustic scenes remain limited. We address this by augmenting AudioSet Strong with clip-level scene annotations under two taxonomies for joint sceneevent modeling. A corpus-scale analysis shows that scene identity is better captured by events distinctive to a scene than by those that merely occur frequently. Reweighting event co-occurrences toward scene-specific associations produces representations that better align with human-derived taxonomies, while also exposing scenes where the acoustic and semantic views diverge. This geometric divergence predicts classification errors, while supervised validation confirms that the proposed scene annotations provide meaningful complementary information for developing scalable context-aware machine listening systems.
Nov 2, 2026

A Human-Verified AI Workflow for Annotating Environmental Sound Data

Large-scale annotation of environmental sound recordings presents significant challenges under time and resource constraints, particularly when sound categories are imbalanced and only part of a dataset has been labeled. This paper describes the development and evaluation of a human-AI workflow used during annotation of the Kenya Forest Sounds Dataset (KFSD), using a project-stage working collection of 7,531 audio recordings across 21 categories. We investigated zero-shot classification, a feature-based ensemble, and a fine-tuned Audio Spectrogram Transformer (AST). The fine-tuned AST achieved 89.36% accuracy, 85.98% precision, 89.24% recall, and 87.58% F1 on held-out test data for binary classification of Person talking. Because this result remained below the predefined 95% threshold for autonomous annotation, model predictions were used as suggestions and subjected to human verification. Of the 5,026 initially unlabeled recordings, 2,337 were routed to a speech-verification queue and 2,689 remained for ordinary manual annotation. The verifier reviewed all 2,337 machine-labeled files and detected zero false positives, yielding 100% verification accuracy. The observed average verification time was approximately 4.2 seconds per file. At an average rate of 35 seconds per file for manual annotation, the hybrid workflow reduced work on the remaining queue from approximately 48.9 to 28.9 person-hours, saving about 20.0 hours (41%). The study documents the model-selection process, provenance-aware file organization, verification procedure, observed errors, practical limitations, and opportunities for extending AI assistance beyond one category.
Oct 30, 2026

Self-Supervised Object-Centric Representation for Multi-Task Audio Analysis

This paper studies the capability of an object-centric audio representation to support multiple downstream tasks within a unified framework. The object-centric model is trained in a self-supervised setup using slot attention by reconstructing Audio Spectrogram Transformer (AST) patch embeddings from a set of latent slots. It is trained on synthetic audio mixtures, which provides controlled multi-source audio scenes. These slot representations are evaluated for the classification, detection, and separation of sound events by training task specific output layers. On the classification task, the model achieves an F1-score of 0.8, comparable to clip-level AST embeddings which achieve 0.7. For sound event detection, it reaches a macro F1-score of 0.7, outperforming clip-level AST embeddings at 0.4 and matching AST patch embeddings at 0.7, which encode finer time-frequency information. In source separation, it achieves an SI-SDR improvement (SI-SDRi) of 18.4 dB. Furthermore, in a zero-shot source separation setting, the model achieves an SI-SDRi of 14.5 dB. These results demonstrate that slot-based representations capture transferable and compositional structure in audio scenes, enabling generalization across tasks and datasets.
Oct 29, 2026

Bridging Neural Representations and Symbolic Semantics in Audio Classification

Deep audio classifiers can achieve strong predictive performance, but their hidden representations remain difficult to interpret. Existing audio explainability methods often focus on input-level attribution, such as waveform segments, spectrogram regions, or acoustic features, and provide limited insight into what hidden neurons encode. Interpreting hidden neurons at the concept level can provide a more meaningful view of what an audio model learns internally. This paper investigates whether hidden neurons in audio classifiers can be associated with human-understandable symbolic acoustic concepts. A model-agnostic neurosymbolic framework is presented for analyzing the final hidden layer of audio classifiers, containing 32 neurons, through symbolic concept induction. The framework is evaluated on two audio classification settings: music genre classification using GTZAN dataset and musical instrument classification using IRMAS dataset. Results show that several hidden neurons can be associated with meaningful induced concepts, and most confirmed neuron-concept pairs are statistically supported using the Mann-Whitney U test. These findings suggest that symbolic concept induction is a promising direction for explaining latent audio representations beyond input-level attribution.
Oct 29, 2026

Lightweight Underwater Acoustic Vessel Classification for Embedded Deployment

We propose a Raspberry Pi-based underwater vessel classification system for edge deployment under limited connectivity, bandwidth and power in offshore acoustic surveillance. Using ShipsEar, we compare six classifiers on a Raspberry Pi 5 through a complete audio-to-prediction pipeline. We assess classification performance, processing latency, CPU utilisation, memory consumption and monitored power. Supporting experiments examine acoustic feature selection, statistical pooling and Transformer patch configurations. The two-layer convolutional neural network (CNN) achieved a mean macro-F1 of (99.14 \textpm{} 0.32)% across five random seeds. The radial basis function support vector machine (RBF-SVM) achieved 98.83%, with a median end-to-end latency of 28.109ms compared with 120.655ms for the CNN. This favourable trade-off motivated RBF-SVM deployment on the Raspberry Pi Zero 2 W. It maintained the same classification performance with a median end-to-end latency of 349.09~ms and a sustained throughput of 2.962 classifications/s, or 14.8 times the rate required for consecutive five-second audio segments.
Oct 26, 2026

Geometric Consistency Based Label-Free Validation for Checkpoint Selection in Audio Active Learning

Audio active learning (AL) trains a classifier at low annotation cost by labeling only an informative fraction of an unlabeled collection over successive rounds. Each round picks one of the checkpoints saved during training. Mainstream AL protocols do so with an assumed validation set, but in practice its labels also count toward the annotation cost, shrinking the advantage of an already small annotation budget. We propose geometric consistency, a label-free validation rule for checkpoint selection. Both classifiers, nearest-class-mean and k-nearest-neighbor, are built from the labeled set in a frozen pretrained embedding space. They then predict the unlabeled data to model its geometry. The rule scores each checkpoint by its agreement with these pseudo-labels and its cross-entropy on the labeled set. It works with any query strategy, has one hyperparameter, insensitive in practice, and adds only one batched forward pass over the unlabeled data per round. At matched label budgets across datasets, encoders, and query strategies, it attains the best mean rank in area under the labeling curve (AULC) among methods without extra labels and is on par with an out-of-budget validation set. To our knowledge, this is the first systematic study of label-free validation for checkpoint selection in audio AL.
Oct 26, 2026

A Hierarchical Method for Environmental Sounds Recognition in Complex Acoustic Scenes

Environmental noise identification is essential for noise control, yet conventional single-stage classifiers suffer from class confusion under imbalanced multiclass conditions. This paper proposes a lightweight hierarchical model integrating a Frequency-Time Adaptive Network (FTANet) with a two-stage classification mechanism. FTANet employs parallel asymmetric one-dimensional convolutions to capture temporal dynamics and frequency-band distributions, while adaptive fusion attention (AFA) enhances discriminative acoustic features. In the first stage, FTANet classifies sounds into five coarse-grained categories covering 25 fine-grained classes. In the second stage, samples are routed to the corresponding category-specific classifiers for fine-grained recognition. On a self-constructed dataset, FTANet achieves 95.45% coarse-grained accuracy with only 0.2004M parameters. On UrbanSound8K, it achieves 79.24% mean accuracy and 80.37% mean F1-score in ten-fold evaluation, outperforming compared convolutional models. Field tests on the Tianjin Online Acoustic Environment Monitoring Platform yield 87.94% coarse-grained accuracy and 97.21% conditional fine-grained accuracy for natural sounds correctly routed to C2. The complete hierarchical model achieves 87.18% overall accuracy and 87.40% weighted-average F1-score, demonstrating its feasibility for urban environmental noise monitoring.
Oct 25, 2026

STFT Frontend Mismatch in Pretrained Audio Spectrogram Transformers: Analysis and Adaptation

Pretrained audio spectrogram transformers are widely used as backbones of machine listening systems for sound event classification. However, they are typically trained using spectrograms with a fixed Short-Time Fourier Transform (STFT) configuration, implicitly binding the model to the time-frequency resolution seen during pretraining. In practice, adapting these systems to new domains or computational constraints often requires different STFT frontend parameters. In this work, we conduct a controlled study of the sensitivity of the foundational Audio Spectrogram Transformer to variations in STFT window and hop size, showing that audio classification performance degrades substantially as frontend parameters deviate from pretraining settings. As a mitigation strategy, we introduce a lightweight conditioning mechanism using FiLM that enables a pretrained model to adapt to variable input configurations using limited additional training data. Our method generalizes well to unseen STFT configurations, improving model robustness without sacrificing performance in the original setting.
Oct 21, 2026

MarineSet: Large-Scale Underwater Acoustic Data Curation and evaluation

The scarcity of labeled underwater acoustic data limits the development of data-driven models for marine acoustic monitoring, motivating the use of self-supervised learning to exploit the abundance of unlabeled recordings. However, large-scale data alone does not guarantee effective representation learning, because model performance is strongly influenced by the distribution of the training data. To facilitate such training data, this study introduces a scalable curation pipeline that curates 3,000 hours from 41 years of raw NOAA hydrophone recordings, producing MarineSet, a large-scale and diverse dataset designed for underwater acoustic learning. A baseline model, MarineNet, trained on MarineSet achieves competitive performance on marine mammal call and ship-type classification, highlighting the value of systematic data curation for developing self-supervised learning models in underwater acoustics.
Oct 20, 2026

MiCLA: Mixture-to-Constituent Latent Alignment for Polyphonic Audio Representation Learning

Real-world audio typically contains concurrent sound events, yet standard self-supervised methods treat each clip as an isolated training unit without explicitly modeling mixture--constituent relationships. We propose Mixture-to-Constituent Latent Alignment (MiCLA), a two-stage self-supervised pre-training framework that uses synthetic log-mel mixing to align mixture representations with their constituent clips. Stage 1 structures single-clip representations via masked self-distillation and contrastive regularization. Stage 2 introduces bidirectional alignment: for a synthetic mixture, MiCLA suppresses the component aligned with one constituent and aligns the residual with the other, using projective residualization on patch tokens and sample-adaptive soft residualization on the global class token. In downstream evaluations, MiCLA achieves 43.2% mean average precision (mAP) on AS-20K and 50.3% mAP on AS-2M under fine-tuning, and achieves the highest mAP among evaluated models on both AudioSet settings under frozen Convex Gated Probing (CGP). Furthermore, a Joint Semantic Retrieval diagnostic shows that MiCLA improves simultaneous retrieval of both constituent classes over target-averaging baselines. These results indicate that directional mixture-to-constituent alignment provides an effective pre-training signal for polyphonic audio representation learning.
Oct 19, 2026

Multilingual Voicemail and Answering Machine Detection via Pretrained Audio Embeddings and Gradient Boosting Machines

Answering machine detection (AMD) is a fundamental task in telephony that classifies whether an answered call is picked up by a human or a voicemail (machine). Despite its widespread use across cloud communication platforms, existing approaches rely on proprietary systems with undisclosed architectures, or require computationally expensive models that cannot operate in real time. We propose a lightweight AMD system that combines pretrained audio embeddings with gradient boosting machines, trained to prioritize the correct detection of human-answered calls. We evaluate seven models across two datasets: three models that we propose, a proprietary industry model, two prior academic approaches, and a large language model baseline that relies on automatic speech recognition. Furthermore, we release OpenAMD, the first publicly available AMD dataset comprising of more than 500 recordings in 7 languages. Our best model achieves the highest precision across both datasets while maintaining low inference latency. We assess the robustness of models to additive noise and packet loss, and perform tests of statistical significance to discover per-language and per-gender model performance. Finally, we evaluate our model on an out-of-domain task of detecting robocalls, where it achieves 94.34% recall without any additional training, which is comparable to the industry standard.
Oct 15, 2026

Hierarchical Interpretable Multitask Audio Representation Learning: An Analysis of Event-Scene Relationships through the Information Bottleneck

Joint Acoustic Scene Classification (ASC) and Audio Tagging (AT) models traditionally rely on unconstrained multihead architectures. However, this design ignores the natural hierarchical dependency between sounds and acoustic scenes. In this work, we investigate a hierarchical (sequential) conditioning paradigm that predicts scene categories directly from intermediate event probabilities. Through a large-scale empirical benchmark with 66 full training runs across three state-of-the-art backbones (PaSST, PANNs, and BEATs) and different loss-weighting settings (beta in [0.0, 1.0]), we show that hierarchical conditioning achieves competitive classification performance while altering multitask optimization dynamics. Unlike multihead baselines, which sequentially solve the easier ASC task before optimizing AT, the hierarchical architecture enables synchronized dual-task learning for specific loss-weighting settings (beta in [0.6, 0.9]). Furthermore, we propose a multi-perspective interpretability framework that combines an analysis of the information bottleneck, manifold geometry, and Formal Concept Analysis (FCA) on a Term-Frequency Inverse-Document-Frequency (TF-IDF) event representation, providing complementary insights into event-scene relationships. Our findings reveal that hierarchical conditioning systematically yields lower conditional entropy, higher mutual information, and superior latent cluster separability (higher Silhouette scores), establishing a more structured and semantically faithful intermediate feature space.
Oct 10, 2026

FAM-gate: field-aware metadata attention with prior-constrained vector gating for audio classification

User-generated audio collections frequently contain textual metadata such as titles, tags, and descriptions. Effectively leveraging these complementary fields requires accounting for both field het erogeneity and varying semantic relevance relative to the acoustic content. We propose FAM-Gate, a two-stage framework for metadata-enriched audio classification. In the first stage, field-aware metadata-view attention dynamically aggregates multiple metadata views using audio-guided weighting, explicit field-identity embeddings, a shared identity-initialized projection, and mean-anchored regularized aggregation. In the second stage, a prior-constrained vector gate performs bounded, dimension-wise fusion of audio and metadata representations centered around an acoustic prior. Ex periments on BSD10K, BSD35K, and FSD50K show consistent improvements in both hierarchical single-label and multi-label classification. Compared with matched random-view scalar-fusion base lines, FAM-Gate improves the hierarchical F-score by 2.53 and 5.72 percentage points on BSD10K and BSD35K, respectively, while increasing macro mAP and macro F1 by 5.07 and 10.25 percentage points on FSD50K.
Oct 9, 2026

Speech event detector: Timestamped detection of non-verbal vocalizations and background sounds for rich speech transcription

Automatic speech recognition produces words but drops the non-verbal content of a recording: the breaths, coughs, sighs and laughter of the speaker, and the horns, music or barking around them. We describe Speech Event Detector, a 5.8-million-parameter convolutional-Transformer model that outputs probabilities every 40 ms for 54 event classes (20 speaker vocalizations and 34 background sounds), together with post-processing that converts these probabilities into timestamped events and a procedure for merging the events into a Whisper transcript. The model is trained on material drawn from seven public datasets whose labels come in incompatible forms: single detected regions, free-text tags with timestamps, temporally strong ontology labels, symbols embedded in transcripts, isolated recordings and environmental recordings. We unify them using rule-based label mapping, recover timestamps for transcript-embedded labels through CTC forced alignment, locating 6,360 of 8,620 sounds, and combine about 168 hours of real annotated training recordings with synthetic mixtures and augmentation using real background noise. Per-class thresholds for two operating profiles are tuned on real held-out recordings, and performance is estimated using two-fold cross-validation. On whole held-out recordings from four datasets, considering the 48 classes with at least ten held-out events, the sensitive profile detects an average of 49 percent of a class's events, covering 67 percent of all 9,367 events, at a macro precision of 0.30. The balanced profile achieves a macro recall of 0.37 at a precision of 0.36, with macro event F1 scores of 0.32 and 0.33, respectively. The detector runs about 39 times faster than real time on eight CPU threads.
Oct 9, 2026

Revisiting Accent Classification with DistilHuBERT: Speaker-Disjoint Evaluation and Cross-Corpus Generalization

Recent accent classification systems can achieve high accuracy in standard in-corpus evaluations. However, when speakers overlap between the training and test sets, such results may overestimate model robustness compared with real-world deployment scenarios. This study revisits the robustness of DistilHuBERT-based accent classification by using L2-ARCTIC and multiple external English accent datasets. The comparison covers file-level and speaker-disjoint splits, along with balanced external-data training and teacher-student pseudo-labeling. The large performance gap between in-corpus and external evaluations suggests that file-level evaluation substantially overestimates performance, while speaker-disjoint evaluation can provide a more realistic estimate of the generalization to unseen speakers. In this study, models trained on external data alone do not reliably generalize to L2-ARCTIC. Among the tested configurations, balanced external-data training offers the most promising supervised setting. The pseudo-labeling experiments further indicate that confidence-based filtering alone is insufficient under domain mismatch. Overall, this study suggests that robust accent classification evaluation should simultaneously include speaker-disjoint evaluation and cross-corpus testing.
Oct 8, 2026

Lightweight Systems for Spatial Semantic Segmentation of Sound Scenes

Spatial semantic segmentation of sound scenes (S5) jointly performs audio source separation and tagging on a multichannel audio mixture. Therefore, an S5 system can be decomposed into an audio tagging model and a separation model. The baseline S5 system in task~4 of the DCASE 2025 challenge uses M2D and ResUNet models for tagging and separation, respectively. These are large in overall size and therefore not appropriate for use in embedded devices. The main aim of this paper is to assess the performance loss in an S5 system when using lightweight tagging and separation models instead. To this end, we first compare the tagging performance of lightweight audio tagging models, namely PANNs and DyMNs, in terms of label prediction accuracy against the baseline M2D model. Secondly, we assess separation performance in terms of the scale-invariant signal-to-distortion ratio of the baseline ResUNet separator relative to that of a convolutional-based separator using SuDoRMRF. Finally, we combine the tagging and separation models across various configurations and assess the overall performance of the S5 system using the class-aware signal-to-distortion ratio.
Oct 7, 2026

Mosquito acoustic identification at unseen recording sites: practical limits of common transfer remedies for vector surveillance

Passive acoustic monitoring could widen mosquito surveillance, but a classifier trained under one set of recording conditions often fails when moved to a new site, device or enclosure. On a nine-species, five-condition corpus of about 271,000 recordings, the published baseline identifies species at 0.881 balanced accuracy under conditions it has seen and 0.175 under held-out ones. Using that corpus's leave-domain protocol, we asked which transfer strategy is most reliable when a program brings a new site online. Using fixed features from a pre-trained bioacoustic model and a small classifier improved held-out-condition balanced accuracy to 0.305, without updating backbone weights and in minutes of training per site. Four widely used additions did not improve transfer: rebalancing training across recording conditions, fine-tuning the backbone and fusing a weaker second modality each reduced accuracy, and a six-term objective reduced to a single alignment term. Rebalancing reversed direction across four encoders, helping weak representations and harming strong ones. Genus-level aggregation was more accurate than species-level classification (about 0.50 against 0.30), suggesting genus-level screening may be the more realistic operating point here. These results come from one corpus under one protocol, and are dataset-bounded guidance rather than validated general rules.
Oct 5, 2026

AccentCL: Robust Accent Classification with Incremental Expansion

Accent classifiers are typically trained with a fixed label inventory and cannot accommodate new accent categories as new data becomes available. Moreover, accented speech corpora often exhibit substantial class imbalance and/or domain shift due to differences in recording conditions across corpora. We present AccentCL, a class-incremental learning framework for English accent classification that is robust to class imbalance and cross-corpus domain shift. AccentCL extracts multi-layer representations from a frozen Whisper-Large-v3 encoder, optimized with an imbalance-aware cross-entropy loss to reduce bias toward the majority accent classes and a domain mean alignment loss that minimizes distributional mean shift across training corpora. The label space is then expanded via replay-based continual learning, using the frozen base model for knowledge retention and an old-to-new margin loss to reduce overprediction on newly added classes. On a five-class accent classification task, AccentCL achieves 77.1% balanced accuracy and a 76.9% macro-averaged F1 score. We further evaluate the model's ability to incrementally incorporate two new accent categories: Spanish-accented and Chinese-accented English. When adding Spanish-accented English to the pretrained model, AccentCL attains an F1 of 83.3% on the new class while retaining 77.3% balanced accuracy on the base classes. When subsequently adding Chinese-accented English, it achieves 61.8% F1 on the new class while preserving 77.6% balanced accuracy on the previously learned classes. These results show that AccentCL enables robust regional accent classification while allowing new accent categories to be added without full retraining.
Oct 5, 2026

Neural Representations, Natural Connections: What Transfers From Human Speech Foundation Models to Animal Vocalizations?

Speech-pretrained models have shown promise in animal bioacoustics, but the factors governing their cross-species transfer remain poorly understood. We evaluate 15 frozen encoders spanning monolingual and multilingual speech, speaker verification, animal bioacoustics, and general audio on caller identification across four species and call-type classification across three. Differences between the strongest speech- and animal-pretrained representations range from −0.027-0.027 to +0.119+0.119 UAR; speech is significantly better in three of seven settings (p<0.01p<0.01) and not significantly different in the remaining four. Controlled comparisons show no systematic advantage from increased language coverage or animal-domain pretraining, while frozen speaker-verification embeddings transfer poorly. The results also show that transfer is strongly layer-dependent: raw-waveform models peak early, whereas patch-spectrogram models peak deeper.
Oct 5, 2026

Do Speech Representations Preserve Regional Accent Across Read and Spontaneous Speech?

Regional accent cues can be captured under matched conditions, but it remains unclear whether they persist between read and spontaneous speech. We study RVG1, with 500 German speakers from nine regions, comparing ten speech representations on regional classification and continuous geolocation under matched conditions and speaker-independent read--spontaneous transfer. Whisper performs best under matched conditions, reaching 0.489 nine-way UAR and 148 km median geolocation error, but drops to 0.11/0.18 UAR across transfer directions and 363 km geolocation error. Self-supervised models show a similar degradation, whereas speaker embeddings are less discriminative in-domain but more robust under transfer. This contrast is consistent across classification and geolocation. Across representations, robustness is associated with how little a representation shifts between styles (style-invariance), for which crossstyle speaker retrieval is an interpretable proxy. Age, sex, sentence-overlap, and duration controls do not account for the gap, although channel characteristics contribute. These results show that strong matched-condition performance does not indicate robust regional information.
Oct 4, 2026

Learning from imperfect teachers for low-resource acoustic generalization

Knowledge distillation (KD) improves low-resource acoustic learning by enriching one-hot supervision with the softened predictive distribution of a fixed teacher network. However, a teacher trained with limited or imbalanced annotations may produce a biased distribution whose components are not uniformly reliable. Although this distribution can still encode useful knowledge, direct full-distribution matching may also transfer teacher-induced biases, thereby distorting the student's decision boundary and degrading its generalization performance. To address this limitation, we propose Boundary-Anchored Mass-Partitioned Distillation (BA-MPD), a logit-based distillation objective composed of Boundary-Anchored Correction (BAC) and Mass-Partitioned Distillation (MPD). BAC addresses missing ground-truth labels in the set of the teacher's top predictions by swapping the true label for the lowest-ranked entry of the set, thus keeping the mass and uncertainty of the set unchanged. MPD then distills this corrected distribution through separate losses that enforce relational consistency within the set, balance the mass between high- and low-confidence groups, and weight lower-confidence dependencies. Ultimately, BAC and MPD together suppress harmful ranking errors and noisy low-confidence details, while retaining all useful teacher information. Experiments on two acoustic benchmarks under multiple label budgets show that BA-MPD consistently improves over supervised-learning baselines and vanilla KD while remaining competitive with strong logit-based KD baselines. Cross-budget results further show that BA-MPD remains effective when the teacher and student models use mismatched label budgets, demonstrating its ability to exploit imperfect teachers across supervision gaps. Implementation available at https://github.com/ShuanglinLi/BA-MPD.
Oct 2, 2026

Aggregate accuracy conceals concentrated temporal vulnerability in a spiking speech classifier

Aggregate accuracy cannot reveal which utterances are locally vulnerable or how internal activity changes when labels remain stable. We retain every prediction for 725,070 adjacent-bin, one-count changes around 100 validation utterances of a frozen SpikeSCR-based classifier. The canonical native-horizon GPU singleton path reaches 86.0836% validation accuracy. Equal-source expected accuracy under a uniformly chosen neighbor rises from 84.00% to 84.54%, although 13 of 84 initially correct sources admit adverse neighbors. Five sources carry 93.21% of adverse moves. Margin-guided and surrogate-gradient searches miss sparse cases at fixed query budgets; a post hoc gradient prefix finds all 13 at 73.45% of the census of initially correct sources. Source-matched traces and clean-state interventions distinguish internal change from harmful direction and recoverability. Two count-readout replicas have similar validation accuracies but a 5.21-fold class-change gap concentrated in four sources. Controlled query/key source isolation eliminates 531 batch-order label changes and restores bitwise order invariance across all 9,981 validation score vectors. Isolated batch predictions reproduce padding-matched singleton labels on 9,980 of 9,981 inputs. Paired CPU/GPU replay of all 25,820 class-changing neighbors gives 99.8993% label agreement and preserves all 13 vulnerable sources and their fixed witnesses. Complete source-conditioned maps distinguish vulnerability incidence, concentration, internal change, and execution dependence that aggregate accuracy leaves unresolved.
Oct 1, 2026

LAST: Looped Audio Spectrogram Transformer

Increasing depth of transformer models improves recognition, but it comes at a substantial cost. Each additional layer requires more parameters, which makes the process computationally inefficient. We ask whether additional processing can focus on integrating features already computed. Looped Audio Spectrogram Transformer (LAST) first processes all tokens, then reuses the same blocks to refine only the class token over fixed audio features, thereby making later passes inexpensive. On AudioSet, ten-pass LAST achieves 0.345 mean average precision, exceeding a twelve-layer sequential transformer by 2.1% relative with 49.4% fewer parameters, 42% fewer multiply-accumulate operations, and 9.8% higher measured throughput. Across separately trained models, increasing the pass count from two to ten improves accuracy while adding only 1.2% computation. Further evaluations show improved robustness to temporal masking and various other auditory augmentations, with better generalization on classification tasks with music, environmental, and event sounds.
Sep 29, 2026

Rate-Agnostic Bioacoustics: Heterogeneous Multi-Taxa Classification with Continuous Filterbanks and Fourier Neural Operators

Conventional bioacoustic classification models rely on fixed-rate spectral representations, requiring recordings acquired at heterogeneous sampling rates to be resampled before analysis. We propose a Sampling-Frequency-Independent (SFI) frontend that processes each recording directly at its native sampling rate, coupled with a Fourier Neural Operator (FNO) backbone featuring progressive temporal-scale fusion. This framework avoids fixed-rate resampling and high-frequency information loss while producing fixed-size representations across sampling rates. Mild training-time sampling-rate (sr) augmentation further improves robustness to unseen rate variations. Evaluated on a multi-taxa corpus comprising 84 classes and 60 sampling rates, the proposed SFI-FNO configuration outperforms fixed-rate and corpus-maximum-rate baselines, achieving .906 accuracy, .921 balanced accuracy, and a Macro-F1 score of .899.
Sep 28, 2026

OpenWhistle: A Large-Scale Longitudinal Dataset and Benchmark of Bottlenose Dolphin Vocalizations

Recent advances in bioacoustics have been driven by large-scale corpora and standardized benchmarks, yet existing resources are overwhelmingly bird-centric and shallow per species, limiting their use for studying the structure of a single species' communication system. This gap is particularly acute for cetaceans: despite bottlenose dolphins (Tursiops truncatus) being a compelling case of complex vocal communication among non-human mammals, existing dolphin datasets are small, fragmented, and largely closed. We introduce OpenWhistle, the largest publicly available dataset of dolphin vocalizations. It comprises approximately 180,000 whistles (114 hours) recorded over five years from a stable pod of five individuals in a semi-natural environment, paired with a curated subset of 8,354 expert-annotated whistles and reproducible evaluation protocols for whistle-type detection and classification. We further release the full processing pipeline for whistle detection, segmentation, and categorization. To demonstrate its utility, we pretrain a Wav2Vec2.0 model adapted to dolphin acoustics on the OpenWhistle corpus and show that it learns effective representations, outperforming general-purpose bioacoustic models such as AVES and BioLingual on both tasks while leaving meaningful headroom for future work. By releasing the dataset, pipeline, and evaluation protocol, we provide the first open dolphin whistle dataset tailored for training self-supervised models, laying the groundwork for advancing dolphin communication research and developing models that capture fine-grained acoustic structure within species.
Sep 27, 2026

Uncovering shortcut learning in audio classifiers by discovering recurring concepts in temporal explanations

Correlations between events in machine learning datasets may result in shortcut learning, where models learn to predict the target event based on the presence of a correlated event. When these correlations are spurious -- arising from data collection artifacts -- models are likely to perform poorly in practice. We propose a pipeline to uncover shortcut learning in audio classifiers by discovering recurring concepts in their temporal explanations. Specifically, we isolate audio segments that explain classifier decisions, caption them with an ensemble of Large Audio-Language Models, and use a Large Language Model to extract recurring concepts. The resulting concepts can be audited by humans to uncover potential shortcut learning. We evaluate our framework using datasets curated from AudioSet Strong, controlling for the presence or absence of spurious correlations. Results show that this approach reliably uncovers learned shortcuts, such as the model relying on the presence of "laughter" to predict "applause".
Sep 24, 2026

Towards Deployable Underwater Vessel Classification

We propose a compact underwater acoustic classification framework combining multi-representation feature engineering, temporal statistical pooling, and compact convolutional architectures designed for acoustic time-frequency and cochlear representations. We investigate multiple conventional and auditory-inspired representations and first evaluate lightweight classifiers and Conventional Neural Networks (CNNs) on ShipsEar dataset. On the provided split, a two-layer CNN achieves a macro F1 of 0.9918, while a Radial Basis Function Support Vector Machine (RBF-SVM) reaches 0.9883. However, source-recording provenance cannot be reconstructed, preventing verification of recording-independent generalisation. We therefore evaluate on DeepShip dataset using recording-level partitioning before segmentation. Under this protocol, a 157K-parameter compact CNN achieves a test macro F1 of 0.7226, while an 11.17M-parameter ResNet18 provides no improvement in validation performance under the matched setting. These results demonstrate the importance of representation-aware feature and model design, together with rigorous recording-level evaluation, for classification performance and deployability in compact underwater acoustic systems.
Sep 23, 2026

Physiologically Informed Digital Auscultation for Pneumonia Detection in Long-term Care Residents

Pneumonia is difficult to diagnose in older long-term care residents; multimorbidity and atypical presentations obscure signs, motivating operationally efficient objective testing. We analyzed multi-channel digital stethoscope recordings from 185 Japanese residents (73 pneumonia, 112 symptomatic without), using radiologist-confirmed chest X-rays and clinician diagnoses as supervisory signals that train convolutional neural networks, multimodal fusion, and channel-based variants with time-domain Grad-CAM interpretability. Models were evaluated with repeated patient-level cross-validation showing models with X-ray supervision outperformed clinician supervision (F1 0.729, accuracy 0.783 vs. F1 0.637, accuracy 0.711). Additionally, a three-channel selection protocol maintained performance (F1 0.736; accuracy 0.803), with two mid-thoracic sites ranking highest and Grad-CAM attention overlapping adventitious sounds. These findings indicate automated multi-channel lung-sound analysis can aid long-term care pneumonia diagnosis, with X-ray supervision being more reliable than clinical, and fewer channels preserving performance while lowering acquisition times.
Sep 20, 2026

Long-Tail Rebalancing for Non-Verbal Vocalization-Aware ASR: A Track 1 System for the NVVSpeech Challenge

Non-verbal vocalizations (NVVs) carry important paralinguistic information but are often omitted by conventional automatic speech recognition (ASR) systems. The ISCSLP NVVSpeech Challenge requires joint transcription of lexical content and 16 NVV categories under limited and highly imbalanced supervision. We present a data-centric NVV-aware ASR pipeline based on cross-dataset label harmonization and a two-stage sampling schedule. We map heterogeneous source labels to the official taxonomy and exclude samples without a reliable mapping. Our schedule first uses square-root category sampling to moderate the long-tailed distribution and then applies uniform-category fine-tuning. On a fixed local validation split, square-root category sampling performs best among the tested single-stage settings. The final two-stage system obtains an official score of 63.86 and ranks fourth in Track 1.