A Toolkit for Detecting Spurious Correlations in Speech Datasets

Lara Gauder; Pablo Riera; Andrea Slachevsky; Gonzalo Forno; Adolfo M. García; Luciana Ferrer

cs.SDApr 29, 2026

A Toolkit for Detecting Spurious Correlations in Speech Datasets

Authors: Lara Gauder, Pablo Riera, Andrea Slachevsky, Gonzalo Forno, Adolfo M. García, Luciana Ferrer

Organizations: Instituto de Investigaci´on en Ciencias de la Computaci´on, UBA-CONICET, Argentina · Departamento de Computaci´on, Facultad de Ciencias Exactas y Naturales, UBA, Argentina · Facultad de Medicina, Universidad de Chile, Chile · Centro de Neurociencias Cognitivas, Universidad de San Andr´es, Argentina

Abstract

We introduce a toolkit for uncovering spurious correlations between recording characteristics and target class in speech datasets. Spurious correlations may arise due to heterogeneous recording conditions, a common scenario for health-related datasets. When present both in the training and test data, these correlations result in an overestimation of the system performance -- a dangerous situation, specially in high-stakes application where systems are required to satisfy minimum performance requirements. Our toolkit implements a diagnostic method based on the detection of the target class using only the non-speech regions in the audio. Better than chance performance at this task indicates that information about the target class can be extracted from the non-speech regions, flagging the presence of spurious correlations. The toolkit is publicly available for research use.

Explore similar work

Cleaner Speech, Weaker Generalization: Revisiting Pitt-Derived Benchmarks for Alzheimer's Disease Detection
Aug 31, 2026Luqi Sun, Shreeram Suresh Chandra, Lin Zhang +5Librispeech Alzheimer
Ethical and Technical Limits of Deepfake Speech Datasets
Jun 9, 2026Vojtěch Staněk, Eva Trnovská, Kamil Malinka +1Audio Deepfake Detection Algorithm Auditing
SpeechDx: A Multi-Task Benchmark for Clinical Speech AI
Jun 15, 2026Sejal Bhalla, Larry Kieu, Aina Merchant +2Librispeech

Aug 31, 2026 · cs.SDJ/K move · Enter open · S save

Cleaner Speech, Weaker Generalization: Revisiting Pitt-Derived Benchmarks for Alzheimer's Disease Detection

Luqi Sun, Shreeram Suresh Chandra, Lin Zhang, You-Jin Li+4

Center for Language and Speech Processing (CLSP), Johns Hopkins University · Research Center for Information Technology Innovation, Academia Sinica · University of Michigan, Ann Arbor · Carnegie Mellon University

Speech-based Alzheimer's disease (AD) detection increasingly relies on speech-enhanced and curated versions of the Pitt Corpus, where speech enhancement, sample selection, and demographic balancing are often treated as beneficial preprocessing steps. However, whether these transformations improve real-world AD detection or instead affect model generalization and prediction behavior remains unclear. In this work, we revisit the role of speech preprocessing and dataset curation across widely used benchmarks for speech-based AD detection. We evaluate the speech quality of different datasets, the cross-dataset generalization of multiple deep learning models under matched and mismatched enhancement settings, and the behavior of several recent large audio-language models (LALMs). Experimental results show that across multiple supervised speech models, speech-enhanced datasets often improve in-domain performance while reducing robustness in cross-domain evaluation. Matched enhancement between training and test data alleviates, but does not eliminate, this degradation. LALMs show a similar sensitivity: enhanced datasets induce stronger class imbalance and prediction shifts than unprocessed data. These results suggest that speech preprocessing and dataset curation can substantially influence downstream AD detection behavior, indicating that ``cleaner'' speech datasets are not necessarily more reliable for real-world AD detection.

Librispeech Alzheimer Disease Detection Preprocessing

Open paper Ask about it PDF

Abstract

Explore similar work

Cleaner Speech, Weaker Generalization: Revisiting Pitt-Derived Benchmarks for Alzheimer's Disease Detection

Ethical and Technical Limits of Deepfake Speech Datasets

SpeechDx: A Multi-Task Benchmark for Clinical Speech AI