Gated Multi-Graph Fusion via Graph Attention Networks for Alzheimer's Disease Detection

Jinyu Li; Xiao Wei; Bin Wen; Kai Li; Yuqin Lin; Xiaobao Wang; Longbiao Wang; Jianwu Dang

cs.CLJun 30, 2026

Gated Multi-Graph Fusion via Graph Attention Networks for Alzheimer's Disease Detection

Authors: Jinyu Li, Xiao Wei, Bin Wen, Kai Li, Yuqin Lin, Xiaobao Wang, Longbiao Wang, Jianwu Dang

Organizations: School of Future Technology, Tianjin University, Tianjin, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · College of Computer and Data Science, Fuzhou University, Fuzhou, China · Huiyan Technology (Tianjin) Co., Ltd, Tianjin, China

Abstract

Spontaneous speech is a vital non-invasive biomarker for Alzheimer's Disease (AD), yet many systems overlook non-linear structural disruptions and clinical heterogeneity in pathological language. We propose a Multi-View Gated Graph Attention Network that transcribes audio via Automatic Speech Recognition (ASR) to construct semantic, dependency, and co-occurrence graphs, characterizing speech through a "content-structure-flow" framework. Notably, the co-occurrence graph leverages Pointwise Mutual Information (PMI) from a normative corpus to quantify narrative logic and linguistic deviation. To address symptomatic diversity, an adaptive gated fusion mechanism dynamically integrates these views. Evaluated on the ADReSSo dataset, our model achieves 90.00% accuracy. Ablation results confirm that the PMI-based graph and heterogeneity-aware gating are essential for robust classification across diverse clinical populations. Our source code is publicly available at https://github.com/opeacc/AD.

Explore similar work

A Multimodal Framework for Dementia Detection via Linguistic and Acoustic Representation Learning
May 25, 2026Loukas Ilias, Dimitris AskounisAlzheimer Discrete Speech Representations
LLM-Anchored Paralinguistic Enrichment for Alzheimer's Disease Detection
Sep 9, 2026Xiao Wei, Yuqin Lin, Yaru Cao +6Paralinguistic Cues Alzheimer
Robust Rank Aggregation for Multimodal Speech-Based Alzheimer's Disease Detection
Sep 9, 2026Zemin Jin, Tomoko MatsuiAlzheimer Disease Detection

May 25, 2026 · cs.SDJ/K move · Enter open · S save

A Multimodal Framework for Dementia Detection via Linguistic and Acoustic Representation Learning

Loukas Ilias, Dimitris Askounis

Alzheimer's disease (AD) is a progressive neurodegenerative disorder and the leading cause of dementia, affecting memory, reasoning, communication, and daily functioning. Early diagnosis is particularly important, as timely intervention may help slow cognitive decline and improve patient care. Recent studies have demonstrated that spontaneous speech contains valuable linguistic and acoustic biomarkers associated with dementia. However, existing approaches often rely on independently trained modality-specific models, feature concatenation strategies, ensemble methods, or attention-based fusion mechanisms that do not explicitly maximize the dependency between speech and transcript representations. In this work, we propose a multimodal deep learning framework for automatic dementia detection that jointly exploits speech and transcript information in an end-to-end trainable manner. Specifically, speech recordings are divided into 10-second segments and passed through a pre-trained HuBERT model to extract contextualized acoustic representations. To better capture informative temporal speech characteristics, attentive statistics pooling is employed to aggregate frame-level acoustic embeddings. For the textual modality, transcripts are encoded using a pre-trained BERT model, where the [CLS] token representation is used as the linguistic embedding. The acoustic and textual representations are subsequently combined using an attention-based Audio-Text Fusion (AT-Fusion) mechanism. In addition, we introduce a MINE objective to maximize the mutual information between modalities and improve multimodal representation alignment. The fused multimodal representation is finally used for dementia classification. Experiments conducted on the publicly available ADReSS Challenge and PROCESS-2 dataset demonstrate the effectiveness and robustness of the proposed approach for speech-based dementia assessment.

Alzheimer Discrete Speech Representations Linguistics

Open paper Ask about it PDF

Abstract

Explore similar work

A Multimodal Framework for Dementia Detection via Linguistic and Acoustic Representation Learning

LLM-Anchored Paralinguistic Enrichment for Alzheimer's Disease Detection

Robust Rank Aggregation for Multimodal Speech-Based Alzheimer's Disease Detection