Uncovering shortcut learning in audio classifiers by discovering recurring concepts in temporal explanations

Cecilia Bolaños; Luciana Ferrer; Magdalena Fuentes

cs.SDSep 27, 2026

Uncovering shortcut learning in audio classifiers by discovering recurring concepts in temporal explanations

Authors: Cecilia Bolaños, Luciana Ferrer, Magdalena Fuentes

Organizations: Departamento de Computación, FCEyN, UBA · ICC, CONICET-UBA, Buenos Aires, Argentina · Music and Audio Research Lab, New York University, USA · Integrated Design & Media, New York University, USA

Abstract

Correlations between events in machine learning datasets may result in shortcut learning, where models learn to predict the target event based on the presence of a correlated event. When these correlations are spurious -- arising from data collection artifacts -- models are likely to perform poorly in practice. We propose a pipeline to uncover shortcut learning in audio classifiers by discovering recurring concepts in their temporal explanations. Specifically, we isolate audio segments that explain classifier decisions, caption them with an ensemble of Large Audio-Language Models, and use a Large Language Model to extract recurring concepts. The resulting concepts can be audited by humans to uncover potential shortcut learning. We evaluate our framework using datasets curated from AudioSet Strong, controlling for the presence or absence of spurious correlations. Results show that this approach reliably uncovers learned shortcuts, such as the model relying on the presence of "laughter" to predict "applause".

Figures & tables

Target ( $x$ )	Context ( $y$ )	Recall
		CLN $\rightarrow$ CLN	CLN $\rightarrow$ SPU	SPU $\rightarrow$ SPU	SPU $\rightarrow$ CLN
Applause	Laughter	0.899 $\pm$ 0.056	0.801 $\pm$ 0.063	0.943 $\pm$ 0.039	0.868 $\pm$ 0.061
Dog	Clock	0.856 $\pm$ 0.061	0.821 $\pm$ 0.063	0.791 $\pm$ 0.066	0.797 $\pm$ 0.066
Bird	Wind	0.762 $\pm$ 0.069	0.845 $\pm$ 0.069	0.889 $\pm$ 0.047	0.713 $\pm$ 0.075
Car	M. Speech	0.807 $\pm$ 0.073	0.813 $\pm$ 0.062	0.897 $\pm$ 0.056	0.767 $\pm$ 0.072
Liquid	F. Speech	0.829 $\pm$ 0.064	0.829 $\pm$ 0.067	0.847 $\pm$ 0.062	0.579 $\pm$ 0.056

Table 1: Recall and 95% confidence intervals (CI) for each target class

x

across CLN and SPU evaluation settings.

Pair ( $x$ + $y$ )	CLN Model Concepts	SPU Model Concepts
Applause+	Beep (4/6), Applause (3/6),	Laughter (5/6),
Laughter	Crowd Noise (3/6), Explosion (3/6)	Applause (4/6)
Dog+	Dog Barking (6/6),	Dog Barking (6/6), Sneezing (4/6)
Clock	Animal Sounds (3/6)	Grunting (4/6), Whimpering (4/6)
Bird+	Bird Chirping (6/6)	Bird Chirping (4/6),
Wind		Continuous Sound (3/6)

Table 2: Validated concepts (

\geq

3/6 LALM agreement) for the target class generated by CLN vs. SPU models (90th percentile, zero-masking). Correlation-related concepts are highlighted in bold.

Explore similar work

SpurAudio: A Benchmark for Studying Shortcut Learning in Few-Shot Audio Classification
May 13, 2026Giries Abu Ayoub, Morad Tukan, Loay MualemFew-Shot Learning Audio Understanding
Independent Researcher · University of Stuttgart, Germany · IMPRS-IS, Germany
Controlling Implicit Shortcut Reliance in L2 Spoken English Auto-markers
Jul 17, 2026Shilin Gao, Mark J. F. Gales, Kate M. KnillLinguistics Shortcut Learning
ALTA Institute, Machine Intelligence Laboratory, Cambridge University Engineering Department Cambridge, UK
APEX: Audio Prototype EXplanations for Classification Tasks
May 11, 2026Piotr Kawa, Kornel Howil, Piotr Borycki +3Audio Understanding Audio Editing
Department of Artificial Intelligence, Wroclaw University of Science and Technology, Poland · IDEAS Research Institute, Poland · Faculty of Mathematics and Computer Science, Jagiellonian University, Poland +1

May 13, 2026, cs.CVJ/K move · Enter open · S save

SpurAudio: A Benchmark for Studying Shortcut Learning in Few-Shot Audio Classification

Giries Abu Ayoub, Morad Tukan, Loay Mualem

Independent Researcher · University of Stuttgart, Germany · IMPRS-IS, Germany

Few-Shot Learning Audio Understanding Shortcut Learning Private Signals

Open paper AskPDF arXiv

Few-shot classification (FSC) is widely used for learning from limited labeled data, yet most evaluations implicitly assume that target concepts are independent of contextual cues. In real-world settings, however, examples often appear within rich contexts, allowing models to exploit spurious correlations between foreground content and background signals. While such effects have been studied in few-shot image classification, their role in few-shot audio classification remains largely unexplored, and existing audio benchmarks offer limited control over contextual structure. We introduce SpurAudio, a benchmark that leverages the natural separability of foreground events and background environments in audio to enable controlled, multi-level evaluation of contextual shifts across support and query sets. Using this benchmark, we show that many state-of-the-art few-shot methods suffer severe performance degradation when background correlations are disrupted, despite achieving similar accuracy under standard evaluation protocols. Crucially, this vulnerability persists even in large pretrained audio foundation models, ruling out limited backbone capacity as an explanation. Moreover, methods that appear comparable under conventional benchmarks can exhibit markedly different sensitivity to spurious correlations, revealing systematic algorithmic strengths and vulnerabilities tied to how feature representations interact with classifier heads at inference time. These findings provide new insight into the behavior of few-shot methods in audio and highlight the need for benchmarks that explicitly probe context dependence when evaluating FSC models.

Abstract

Explore similar work

SpurAudio: A Benchmark for Studying Shortcut Learning in Few-Shot Audio Classification

Controlling Implicit Shortcut Reliance in L2 Spoken English Auto-markers

APEX: Audio Prototype EXplanations for Classification Tasks