Speech-Based Dementia Detection

Ask about this topic

Overview

The field studies how changes in word choice, sentence structure, pauses, fluency, and vocal acoustics can signal dementia or milder cognitive impairment. Methods range from interpretable linguistic features and acoustic models to automatic speech recognition, self-supervised speech representations, large language models, and systems that combine audio with transcripts or other clinical data. Researchers evaluate these approaches on conversational and task-based speech benchmarks, including ADReSS, DementiaBank, and newer multilingual or clinical corpora. Strong benchmark results remain difficult to carry across datasets, languages, and patient groups, so fairness, clinical validation, faithful explanations, and robustness to recording differences are central open challenges.

Latest papers

CardsList

44 papers9 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

Jun 10, 2026

Unifying Acoustic Features and Text with Multimodal LLMs for Neurodegenerative Screening

Voice-based screening offers a scalable and non-invasive way to assess neurodegenerative diseases such as Alzheimer's disease (AD) and Parkinson's disease (PD), but their staging remains challenging due to the difficulty of integrating heterogeneous data. This paper presents NeurMLLM, an efficient multimodal generative framework for neurodegenerative disease staging. NeurMLLM first encodes the spectrograms and Mel-frequency cepstral coefficients of audio data with vision transformers and projects their representations into the embedding space of a large language model (LLM), where they are concatenated with transcript and demographic instruction tokens as a single unified sequence. The LLM is then instruction-tuned via Low-Rank Adaptation using task prompts to autoregressively predict a constrained label token, enabling a generative classification. By evaluating on the Bridge2AI-Voice dataset for fine-grained staging of AD and PD, we observe that NeurMLLM achieves strong performance, consistently outperforming classical machine learning methods and existing LLM-based approaches. The results show the high potential of multimodal LLMs in neurodegenerative disease staging, improving staging accuracy and supporting accessible deployment.

Qingfeng Zhang, Yuanxiong Guo, Yanmin Gong

Dept. of Information Systems and Cybersecurity The University of Texas at San Antonio San Antonio, TX, USA · School of Engineering Medicine Texas A&M University Houston, TX, USA

Alzheimer's DiseaseMultimodal Large Language ModelsSpeech-Based Dementia DetectionMultimodal ClassificationAlzheimer's Disease Classification
AskPDF
Jun 4, 2026

Multilingual Detection of Alzheimer's Disease from Speech: A Cross-Linguistic Transfer Learning Approach

The development of multilingual Alzheimer's Disease Dementia (AD) detection models presents significant challenges due to the resource-intensive and time-consuming nature of language-specific model training. We propose a novel solution using cross-language training to detect AD in languages beyond those used for model training. This study investigates multilingual deep learning models for detecting AD across different languages and cognitive impairment levels. Using datasets in English, Chinese, Arabic, and Hindi, we developed transformer-based models for binary AD classification. Our approach achieved F1 scores of 82% across all languages, demonstrating strong cross-linguistic generalization. The rapid inference time (0.5 seconds) supports potential real-time screening applications, while consistent performance across languages indicates feasibility for global deployment.

Nadine Yasser Abdelhalim, Emmanuel Akinrintoyo, Nicole Salomons

Department of Computing, Imperial College London, United Kingdom

Speech-Based Dementia DetectionSpeech ProcessingAlzheimer's Disease ClassificationCross-Lingual Transfer
AskPDF
May 25, 2026

Forgotten Words: Benchmarking NeoBERT for Dementia Detection in Low-Resource Conversational Filipino and English Speech

Dementia detection from spontaneous speech offers a scalable approach to cognitive screening, yet NLP systems remain predominantly English-centric. This limitation is especially acute in the Philippines, where Filipino-English code-switching is pervasive and no prior work has addressed NLP-based dementia detection. We present the first systematic evaluation of transformer-based dementia detection in Filipino speech and the first assessment of NeoBERT in a clinical NLP setting. To separate language from domain effects, we construct a parallel bilingual dataset of 4,000 DementiaBank-derived transcripts, with Filipino translations produced manually to preserve discourse-level markers of cognitive decline. We evaluate five model families, TF-IDF + LogReg, BERT, NeoBERT, XLM-R, and RoBERTa-Tagalog, under monolingual, zero-shot cross-lingual, and bilingual fine-tuning settings. We find that in-domain performance does not transfer across languages, with English-trained BERT dropping to Macro-F1 = 0.455 on Filipino, and that architectural modernization alone does not improve robustness. Bilingual fine-tuning, however, eliminates cross-lingual degradation across all transformer models, converging to Macro-F1 = 0.969-0.973. These results suggest that multilingual clinical NLP performance is driven primarily by linguistic coverage during training rather than model scale or architecture.

Rez Samantha Z. Floresca, Edric Castel C. Hao, Hannah Grachiella Buñales +3

Ateneo de Manila Senior High School · Analog Devices, Inc.

Multilingual Language Model EvaluationSpeech-Based Dementia DetectionLow-Resource Language ProcessingClinical NLPCross-Lingual Transfer
AskPDF
May 25, 2026

A Multimodal Framework for Dementia Detection via Linguistic and Acoustic Representation Learning

Alzheimer's disease (AD) is a progressive neurodegenerative disorder and the leading cause of dementia, affecting memory, reasoning, communication, and daily functioning. Early diagnosis is particularly important, as timely intervention may help slow cognitive decline and improve patient care. Recent studies have demonstrated that spontaneous speech contains valuable linguistic and acoustic biomarkers associated with dementia. However, existing approaches often rely on independently trained modality-specific models, feature concatenation strategies, ensemble methods, or attention-based fusion mechanisms that do not explicitly maximize the dependency between speech and transcript representations. In this work, we propose a multimodal deep learning framework for automatic dementia detection that jointly exploits speech and transcript information in an end-to-end trainable manner. Specifically, speech recordings are divided into 10-second segments and passed through a pre-trained HuBERT model to extract contextualized acoustic representations. To better capture informative temporal speech characteristics, attentive statistics pooling is employed to aggregate frame-level acoustic embeddings. For the textual modality, transcripts are encoded using a pre-trained BERT model, where the [CLS] token representation is used as the linguistic embedding. The acoustic and textual representations are subsequently combined using an attention-based Audio-Text Fusion (AT-Fusion) mechanism. In addition, we introduce a MINE objective to maximize the mutual information between modalities and improve multimodal representation alignment. The fused multimodal representation is finally used for dementia classification. Experiments conducted on the publicly available ADReSS Challenge and PROCESS-2 dataset demonstrate the effectiveness and robustness of the proposed approach for speech-based dementia assessment.

Loukas Ilias, Dimitris Askounis

Decision Support Systems Laboratory, School of Electrical and Computer Engineering, National Technical University of Athens, 15780 Athens, Greece

Speech-Based Dementia DetectionSpeech ProcessingAlzheimer's Disease ClassificationMutual Information MaximizationMultimodal Fusion
AskPDF
May 15, 2026

Can Large Language Models Imitate Human Speech for Clinical Assessment? LLM-Driven Data Augmentation for Cognitive Score Prediction

Accurate assessment of cognitive decline from spontaneous speech remains challenging due to limited dataset size and class imbalance. In this work, we propose a large language model (LLM)-driven data augmentation framework to improve the prediction of cognitive scores from speech. Experiments are conducted on a Japanese corpus in which each participant provides both a spontaneous oral narrative and a written response to the same clinical prompt. The written responses serve as semantic anchors to generate multiple oral-like monologues in different styles using GPT-5. We then predict Hasegawa Dementia Scale scores, a widely used cognitive screening tool in Japan, using a Partial Least Squares regression model trained on Sentence-BERT speech embeddings. We investigate two augmentation strategies: random class-balanced selection, which yields moderate but unstable improvements, and similarity-guided class-balanced selection. The latter prioritizes semantically close synthetic samples, leading to more consistent improvements and substantially reducing prediction error for minority low-score participants while maintaining performance for the majority group. Overall, our findings demonstrate the potential of semantically guided LLM-driven augmentation as a principled approach for addressing class imbalance and improving data efficiency in clinical speech analysis.

Si-Belkacem Yamine Ketir, Lenard Paulo Tamayo, Shohei Hisada +3

Télécom SudParis, France · Nara Institute of Science and Technology, Japan

Class-Imbalanced LearningSynthetic Data AugmentationData AugmentationSpeech-Based Dementia DetectionSpeech ProcessingLarge Language Model-Based Data Augmentation+1
AskPDF
May 14, 2026

PROCESS-2: A Benchmark Speech Corpus for Early Cognitive Impairment Detection

Speech-based analysis offers a scalable and non-invasive approach for detecting cognitive decline, yet progress has been constrained by the limited availability of clinically validated datasets collected under realistic conditions. We introduce PROCESS-2, a large-scale speech dataset designed to support research on automatic assessment of cognitive impairment from spontaneous and task-oriented speech. The dataset comprises recordings from 200 healthy controls, 150 mild cognitive impairment, and 50 dementia diagnoses collected using the CognoMemory digital assessment platform. Each participant completed a single assessment session, including picture description and verbal fluency tasks, accompanied by manually verified transcripts and participant-level metadata. PROCESS-2 contains approximately 21 hours of speech audio with predefined train/test partitions. Comprehensive technical validation evaluated demographic balance, clinical consistency, recording stability, embedding-space structure, and reproducible baseline modelling performance, demonstrating clinically meaningful group separation and stable performance across modelling approaches while preserving real-world conversational variability. PROCESS-2 is released under controlled access via Hugging Face to enable responsible reuse while protecting participant privacy, providing a reproducible benchmark resource for speech-based cognitive assessment research.

Madhurananda Pahar, Caitlin H. Illingworth, Bahman Mirheidari +7

School of Computer Science, University of Sheffield, Sheffield, S1 4DP, UK · Sheffield Institute for Translational Neuroscience (SITraN), University of Sheffield, Sheffield, S10 2HQ, UK

Speech-Based Dementia DetectionSpeech Processing
AskPDF
Apr 29, 2026

ASR-Agnostic Multimodal Spectrotemporal Modeling for Early Dementia Detection

Speech recruits the same executive, attentional, and working memory processes underlying instrumental activities of daily living, or IADLs, providing a non-invasive proxy for cognitive assessment. Yet most speech-based dementia detection systems depend on transcription, discard within-recording temporal structure, and are validated on a single English corpus with known recording artifacts. We propose an ASR-agnostic framework operating directly on Mel spectrograms. Our key contribution is extracting spectrotemporal displacement fields from consecutive spectrogram frames, capturing shifting spectral energy patterns as digital biomarkers of cognitive decline. These features are fused with CNN-ConvGRU acoustic embeddings via a learned cross-attention mechanism and aggregated using a Transformer encoder with learnable query pooling. A composite temporal loss enforces smoothness and contrastive coherence across segments. We train independent models on English DementiaBank, Slovak EWA-DB, and Spanish Ivanova corpora, using clinical elicitation protocols taxing IADL-relevant cognitive domains. The Slovak model achieves 83.9% accuracy, and Spanish achieves, while the English baseline yields 53.2%, confirming known artifacts. Cross-lingual ablation studies reveal distinct fusion regimes: removing cross-attention collapses Spanish performance to 53.7%, below unimodal models, while the Slovak audio encoder alone outperforms the full model, 93.7% vs. 83.9%, and all English configurations remain near chance. Thus, multimodal fusion's value is corpus-dependent: essential when signal is distributed across modalities, counterproductive when one dominates, and irrelevant when no signal exists. Auxiliary temporal losses converge to language-invariant values, indicating cross-lingual architectural stability.

Chukwuemeka Ugwu, Oluwafemi Richard Oyeleke

Speech-Based Dementia DetectionSpeech ProcessingAudio UnderstandingMultimodal Classification
AskPDF
Apr 8, 2026

Multilingual Cognitive Impairment Detection in the Era of Foundation Models

We evaluate cognitive impairment (CI) classification from transcripts of speech in English, Slovene, and Korean. We compare zero-shot large language models (LLMs) used as direct classifiers under three input settings -- transcript-only, linguistic-features-only, and combined -- with supervised tabular approaches trained under a leave-one-out protocol. The tabular models operate on engineered linguistic features, transcript embeddings, and early or late fusion of both modalities. Across languages, zero-shot LLMs provide competitive no-training baselines, but supervised tabular models generally perform better, particularly when engineered linguistic features are included and combined with embeddings. Few-shot experiments focusing on embeddings indicate that the value of limited supervision is language-dependent, with some languages benefiting substantially from additional labelled examples while others remain constrained without richer feature representations. Overall, the results suggest that, in small-data CI detection, structured linguistic signals and simple fusion-based classifiers remain strong and reliable signals.

Damar Hoogland, Boshko Koloski, Jaya Caporusso +5

Jožef Stefan Institute, Ljubljana, Slovenia · Jožef Stefan International Postgraduate School, Ljubljana, Slovenia · Institute of Synergetics and Psychotherapy Research, Paracelsus Medical University, Salzburg, Austria +2

Speech-Based Dementia DetectionClinical Language Model EvaluationText ClassificationFeature FusionMultilingual Language Model Evaluation
AskPDF
Mar 25, 2026

HASS: Hierarchical Simulation of Logopenic Aphasic Speech for Scalable PPA Detection

Building a diagnosis model for primary progressive aphasia (PPA) has been challenging due to the data scarcity. Collecting clinical data at scale is limited by the high vulnerability of clinical population and the high cost of expert labeling. To circumvent this, previous studies simulate dysfluent speech to generate training data. However, those approaches are not comprehensive enough to simulate PPA as holistic, multi-level phenotypes, instead relying on isolated dysfluencies. To address this, we propose a novel, clinically grounded simulation framework, Hierarchical Aphasic Speech Simulation (HASS). HASS aims to simulate behaviors of logopenic variant of PPA (lvPPA) with varying degrees of severity. To this end, semantic, phonological, and temporal deficits of lvPPA are systematically identified by clinical experts, and simulated. We demonstrate that our framework enables more accurate and generalizable detection models.

Harrison Li, Kevin Wang, Cheol Jun Cho +13

UC Berkeley, USA · UCSF, USA · Zhejiang University, China +2

Speech-Based Dementia DetectionClinical Speech Processing
AskPDF
Mar 18, 2026

Impact of automatic speech recognition quality on Alzheimer's disease detection from spontaneous speech: a reproducible benchmark study with lexical modeling and statistical validation

Early detection of Alzheimer's disease from spontaneous speech has emerged as a promising non-invasive screening approach. However, the influence of automatic speech recognition (ASR) quality on downstream clinical language modeling remains insufficiently understood. In this study, we investigate Alzheimer's disease detection using lexical features derived from Whisper ASR transcripts on the ADReSSo 2021 diagnosis dataset. We evaluate interpretable machine-learning models, including Logistic Regression and Linear Support Vector Machines, using TF-IDF text representations under repeated 5x5 stratified cross-validation. Our results demonstrate that transcript quality has a statistically significant impact on classification performance. Models trained on Whisper-small transcripts consistently outperform those using Whisper-base transcripts, achieving balanced accuracy above 0.7850 with Linear SVM. Paired statistical testing confirms that the observed improvements are significant. Importantly, classifier complexity contributes less to performance variation than ASR transcription quality. Feature analysis reveals that cognitively normal speakers produce more semantically precise object- and scene-descriptive language, whereas Alzheimer's speech is characterized by vagueness, discourse markers, and increased hesitation patterns. These findings suggest that high-quality ASR can enable simple, interpretable lexical models to achieve competitive Alzheimer's detection performance without explicit acoustic modeling. The study provides a reproducible benchmark pipeline and highlights ASR selection as a critical modeling decision in clinical speech-based artificial intelligence systems.

Himadri S Samanta

Independent Researcher, Austin, Texas, USA

Speech-Based Dementia DetectionAlzheimer's Disease ClassificationASR Evaluation
AskPDF
Mar 3, 2026

Bias and Fairness in Self-Supervised Acoustic Representations for Cognitive Impairment Detection

Speech-based detection of cognitive impairment (CI) offers a promising non-invasive approach for early diagnosis, yet performance disparities across demographic and clinical subgroups remain underexplored, raising concerns around fairness and generalizability. This study presents a systematic bias analysis of acoustic-based CI and depression classification using the DementiaBank Pitt Corpus. We compare traditional acoustic features (MFCCs, eGeMAPS) with contextualized speech embeddings from Wav2Vec 2.0 (W2V2), and evaluate classification performance across gender, age, and depression-status subgroups. For CI detection, higher-layer W2V2 embeddings outperform baseline features (UAR up to 80.6%), but exhibit performance disparities; specifically, females and younger participants demonstrate lower discriminative power (AUCAUC: 0.769 and 0.746, respectively) and substantial specificity disparities (ΔspecΔ_{spec} up to 18% and 15%, respectively), leading to a higher risk of misclassifications than their counterparts. These disparities reflect representational biases, defined as systematic differences in model performance across demographic or clinical subgroups. Depression detection within CI subjects yields lower overall performance, with mild improvements from low and mid-level W2V2 layers. Cross-task generalization between CI and depression classification is limited, indicating that each task depends on distinct representations. These findings emphasize the need for fairness-aware model evaluation and subgroup-specific analysis in clinical speech applications, particularly in light of demographic and clinical heterogeneity in real-world applications.

Kashaf Gulzar, Korbinian Riedhammer, Elmar Nöth +2

Zentrum für Künstliche Intelligenz (KIZ), Technische Hochschule Nürnberg, Nürnberg, Germany · Pattern Recognition Lab, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Germany

Speech-Based Dementia DetectionClinical Speech ProcessingDepression DetectionSelf-Supervised Speech Representation LearningDemographic Bias in ASR
AskPDF
Feb 27, 2026

MINT: Multimodal Imaging-to-Speech Knowledge Transfer for Early Alzheimer's Screening

Alzheimer's disease is a progressive neurodegenerative disorder in which mild cognitive impairment (MCI) precedes dementia. Structural MRI provides biomarkers but requires costly infrastructure, limiting population-scale deployment. Speech offers a non-invasive alternative, yet speech-only classifiers are developed independently of neuroimaging and lack biological grounding for CN-versus-MCI classification. We propose MINT (Multimodal Imaging-to-Speech Knowledge Transfer), a three-stage framework that transfers MRI-derived biomarker structure to speech during training. An MRI teacher defines a compact embedding space for CN-versus-MCI classification, while a residual projection head aligns speech representations to this space using a combined geometric loss. The frozen MRI classifier enables imaging-free inference. On ADNI-4, aligned speech achieves performance comparable to speech baselines, while multimodal fusion improves over MRI alone. Ablations identify dropout regularization and self-supervised pretraining as important design choices. To our knowledge, MINT is the first demonstration of MRI-to-speech knowledge transfer for early Alzheimer's screening without imaging at inference.

Vrushank Ahire, Yogesh Kumar, Anouck Girard +1

Department of Computer Science and Engineering, Indian Institute of Technology Ropar, India · Embry-Riddle Aeronautical University, Florida, USA

Cross-Modal Knowledge DistillationAlzheimer's DiseaseSpeech-Based Dementia DetectionClinical PredictionAlzheimer's Disease Classification
AskPDF
Feb 16, 2026

Breaking Data Efficiency Dilemma: A Federated and Augmented Learning Framework For Alzheimer's Disease Detection via Speech

Early diagnosis of Alzheimer's Disease (AD) is crucial for delaying its progression. While AI-based speech detection is non-invasive and cost-effective, it faces a critical data efficiency dilemma due to medical data scarcity and privacy barriers. Therefore, we propose FAL-AD, a novel framework that synergistically integrates federated learning with data augmentation to systematically optimize data efficiency. Our approach delivers three key breakthroughs: First, absolute efficiency improvement through voice conversion-based augmentation, which generates diverse pathological speech samples via cross-category voice-content recombination. Second, collaborative efficiency breakthrough via an adaptive federated learning paradigm, maximizing cross-institutional benefits under privacy constraints. Finally, representational efficiency optimization by an attentive cross-modal fusion model, which achieves fine-grained word-level alignment and acoustic-textual interaction. Evaluated on ADReSSo, FAL-AD achieves a state-of-the-art multi-modal accuracy of 91.52%, outperforming all centralized baselines and demonstrating a practical solution to the data efficiency dilemma. Our source code is publicly available at https://github.com/smileix/fal-ad.

Xiao Wei, Bin Wen, Yuqin Lin +5

Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University, Tianjin, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · College of Computer and Data Science, Fuzhou University, Fuzhou, China +1

Speech-Based Dementia DetectionData-Efficient LearningFederated LearningSynthetic Data AugmentationMultimodal Learning
AskPDF
Feb 13, 2026

False positive bias in AI-powered speech-based cognitive screening for multilingual English speakers in the UK

Conversational speech reveals early signs of cognitive decline, including dementia and mild cognitive impairment (MCI). AI models show promise for speech-based screening, yet most research focuses on monolingual groups. In the UK, dementia is projected to rise fastest among Black and Asian communities, where multilingualism is common, making equity assessment critical. We recruited 1,395 participants (monolingual English speakers and multilingual speakers from Sheffield/Bradford) and collected over 263 hours of speech via the CognoMemory agent. Multilingual participants spoke English alongside Somali, Chinese, or South Asian languages (Hindi, Urdu, Punjabi, Mirpuri, Arabic). We evaluated ASR (Whisper, Wav2Vec 2.0, NeMo) and downstream AI models for cognitive classification and MMSE regression. ASR accuracy showed no significant differences across groups. However, downstream models exhibited systematic disparities: multilingual speakers were more often misclassified as impaired, especially in memory, fluency, and reading tasks. False-positive rates were substantially higher for multilingual (28 to 37%) than monolingual (12 to 16%) speakers, meaning multilingual individuals were approximately 2.5 times more likely to receive incorrect impairment labels. These biases worsened when models were trained on DementiaBank. This is the first large-scale analysis of false-positive bias in speech-based AI cognitive screening for UK multilingual ethnic minorities. Despite strong overall performance, current models show measurable disparities affecting multilingual speakers. Addressing these biases is essential for safe, equitable deployment in diverse healthcare settings.

Madhurananda Pahar, Caitlin Illingworth, Dorota Braun +4

School of Computer Science, University of Sheffield, Sheffield, S1 4DP, UK · Sheffield Institute for Translational Neuroscience (SITraN), University of Sheffield, Sheffield, S10 2HQ, UK · NIHR Devices for Dignity HTC, Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, S10 2JF, UK

Speech-Based Dementia Detection
AskPDF