Dual Task Vector Merging: Leveraging Emotion and ASR Knowledge for Low-Resource Alzheimer's Disease Detection

Yuan-Chao Li; Yin-Long Liu; Liu He; Sizhe Liu; Ji-Xuan Deng; Yin Zhang; Zhen-Hua Ling; Jiahong Yuan

eess.ASOct 17, 2026

Dual Task Vector Merging: Leveraging Emotion and ASR Knowledge for Low-Resource Alzheimer's Disease Detection

Authors: Yuan-Chao Li, Yin-Long Liu, Liu He, Sizhe Liu, Ji-Xuan Deng, Yin Zhang, Zhen-Hua Ling, Jiahong Yuan

Organizations: University of Cambridge, UK · University of Science and Technology of China, China

Abstract

Alzheimer's Disease (AD) detection from speech benefits from both paralinguistic and linguistic cues, yet model training is frequently constrained by limited data. Fine-tuning a large pretrained speech encoder can easily overfit, while a frozen probe leaves the encoder's task-relevant knowledge underutilized. Therefore, we propose Dual Task Vector Merging, a parameter-efficient method that adapts a frozen wav2vec 2.0 backbone using task knowledge learned outside the AD task. Specifically, we inject two task vectors, derived as weight deltas from emotion- and ASR-fine-tuned wav2vec 2.0 models, into each transformer layer. During AD training, only layer-wise merging coefficients and a linear classification head are optimized, updating less than 0.1% of the parameters required for full fine-tuning. Speaker-independent experiments on ADReSS and ADReSSo show that dual merging improves over frozen probes, LoRA, and full fine-tuning, in the main binary settings, and gives the best results when combined with LoRA. Evaluation on PROCESS further shows that our advantages transfer to three-class HC/MCI/dementia classification when paired with LoRA. Experiments under reduced training data (20%, 40%, and 80%) confirm reliable gains in low-resource settings.

Figures & tables

Figure 1: Overview of Dual Task Vector Merging. Emo and ASR task vectors are computed once by subtracting the base wav2vec 2.0 weights from task-fine-tuned checkpoints. During AD training, the backbone and task vectors are frozen. Only the layer-wise coefficients $\lambda_{e}^{(l)}$ , $\lambda_{a}^{(l)}$ and the linear head are updated.

ADReSS
Method	#Trainable	Acc $\uparrow$	Macro-F1 $\uparrow$	AUC $\uparrow$
Base (frozen probe)	2.1K	66.2 $\pm$ 2.1	65.3 $\pm$ 2.7	64.5 $\pm$ 2.8
+LoRA	3.5M	67.3 $\pm$ 1.1	66.7 $\pm$ 1.1	63.4 $\pm$ 2.6
+Full FT	311M	62.9 $\pm$ 1.1	61.3 $\pm$ 2.3	60.5 $\pm$ 1.8
Merge
+Emo	2.1K	67.1 $\pm$ 2.3	66.0 $\pm$ 3.4	66.3 $\pm$ 1.2

Table 1: Speaker-independent 5-fold CV on ADReSS and ADReSSo. Results are mean

\pm

std over paired seed-fold runs. Best scores within each dataset are bolded. Parameter counts denote trainable parameters.

\uparrow

: higher the better.

Panel of Figure 2 — Figure 2: Accuracy versus trainable parameters. Dual merging occupies a favorable low-capacity region, improving over frozen probe while remaining far smaller than fine-tuning.

Configuration	Acc $\uparrow$
Dual merge	69.6
Prune
-front 2 layers	65.1
-front 4 layers	64.5
-back 2 layers	69.1
-back 4 layers	69.1

Table 2: Layer-pruning analysis on ADReSS. Layers are removed before training, and the classifier reads the midpoint of the remaining hidden-state sequence.

\uparrow

: higher the better. Std omitted for brevity.

Dataset	Method	20%	40%	80%	Full
ADReSS	Base	60.4	64.3	66.5	66.2
ADReSS	Dual	63.3	66.3	67.4	69.6
ADReSSo	Base	61.0	66.8	74.6	75.0
ADReSSo	Dual	69.5	73.7	78.6	80.8

Table 3: Low-resource accuracy on ADReSS and ADReSSo. Values are mean accuracy over seeds. Std omitted for brevity.

Merge
Method	Macro-F1 $\uparrow$	UAR $\uparrow$
Base	42.1 $\pm$ 9.9	46.2 $\pm$ 10.9
+LoRA	46.3 $\pm$ 9.0	48.8 $\pm$ 8.7
+Full FT	39.6 $\pm$ 9.3	43.8 $\pm$ 7.4
+Emo	43.0 $\pm$ 11.4	46.8 $\pm$ 11.7
+ASR	42.2 $\pm$ 10.6	45.7 $\pm$ 10.7

Table 4: PROCESS three-class classification (HC/MCI/dementia), speaker-independent cross validation.

\uparrow

: higher the better.

Speech-Based Dementia Detection

Speech-based dementia detection uses spoken language and vocal patterns to identify cognitive impairment and support accessible screening.

See all 44 papers

LLM-Anchored Paralinguistic Enrichment for Alzheimer's Disease Detection
Sep 9, 2026Xiao Wei, Yuqin Lin, Yaru Cao +6Speech-Based Dementia Detection Speech Prosody
Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University, Tianjin, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · College of Computer and Data Science, Fuzhou University, Fuzhou, China +1
Breaking Data Efficiency Dilemma: A Federated and Augmented Learning Framework For Alzheimer's Disease Detection via Speech
Feb 16, 2026Xiao Wei, Bin Wen, Yuqin Lin +5Speech-Based Dementia Detection Data-Efficient Learning
Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University, Tianjin, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · College of Computer and Data Science, Fuzhou University, Fuzhou, China +1
LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features
Jun 26, 2026Jonghyeon Park, Olivier Jiyoun Jung, Myungwoo OhMulti-View Learning Speech-Based Dementia Detection
NAVER Cloud, South Korea · Division of Communication and Media, Ewha Womans University, South Korea

Sep 9, 2026J/K move · Enter open · S save

LLM-Anchored Paralinguistic Enrichment for Alzheimer's Disease Detection

Xiao Wei, Yuqin Lin, Yaru Cao, Jinyu Li+5

Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University, Tianjin, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · College of Computer and Data Science, Fuzhou University, Fuzhou, China+1

Speech-Based Dementia Detection Speech Prosody Alzheimer's Disease Classification Speech Language Models

Open paper AskPDF arXiv

Speech-based automatic detection of Alzheimer's disease (AD) provides a non-invasive and scalable approach to early cognitive screening. AD affects both lexical-semantic organization and speech production, including atypical pauses and word elongations. However, existing methods have yet to fully integrate these paralinguistic cues with linguistic content. We propose LLM-Anchored Paralinguistic Enrichment (LAPE), which enriches LLM-derived linguistic representations with paralinguistic cues through three coordinated innovations. The first is prosodic event textualization, which enables the LLM to model pauses and elongations jointly with lexical content by encoding them as explicit markers with bounded duration-aware repetition. The second is lexico-prosodic unitization and chunking, which preserves event identity and magnitude in both modalities by pooling only consecutive word units. The third is text-anchored paralinguistic fusion, which integrates local and utterance-level speech features by using NormGate to normalize and dynamically scale them relative to text. We evaluate LAPE on ADReSS and ADReSSo using participant-level cross-validation and leave-one-subject-out evaluation. LAPE achieves state-of-the-art performance across all four primary settings. Code will be released upon acceptance.

Abstract

Speech-Based Dementia Detection

Explore similar work

LLM-Anchored Paralinguistic Enrichment for Alzheimer's Disease Detection

Breaking Data Efficiency Dilemma: A Federated and Augmented Learning Framework For Alzheimer's Disease Detection via Speech

LoRA-Tuned Large Language Models for Dementia Detection via Multi-View Speech-Derived Features