cs.LGJul 28, 2026

DynaBridge: Dynamic Summary-Guided Cross-Task Multimodal Fusion for DASS-Structured Mental Health Assessment

Authors: Shiyu TengHaichen YuJiaqing LiuHao SunYu SongShurong ChaiRuibo HouLanfen Lin+1 more

Abstract

Multimodal behavioral analysis offers a scalable approach to assessing depression, anxiety, and stress, yet generic fusion models often ignore the psychometric structure of questionnaire labels. In DASS-21, risk labels are derived from ordered symptom items through fixed item-to-subscale mappings. We propose \textbf{DynaBridge}, a dynamic summary-guided cross-task multimodal framework for DASS-structured mental health assessment. DynaBridge encodes acoustic, visual, and textual cues across multiple sessions and augments them with frozen-LLM-generated DASS-aware summaries as participant-level semantic evidence. It predicts ordinal item distributions, reconstructs depression, anxiety, and stress risk evidence from item-level soft scores, and fuses this evidence with direct multimodal risk predictions. A confidence-aware refinement strategy further incorporates high-confidence semantic cues conservatively. On the official AdoDAS validation split, DynaBridge outperforms the official baseline and representative multimodal methods, achieving 0.5012 mean F1 for D/A/S risk prediction and 0.3216 mean QWK for DASS-21 item prediction. These results show the value of bridging multimodal cues, semantic summaries, and DASS-21 psychometric structure.

Explore similar work

Jun 22, 2026cs.AI

PsyBridge: A Hybrid Intelligent Framework for Multi-Dimensional Mental Health Assessment and Decision Support

Mental health assessment commonly relies on isolated screening instruments or data-driven models that often lack interpretability and multi-dimensional integration. Existing approaches frequently focus on individual indicators such as depression or anxiety while providing limited support for comprehensive and explainable decision-making. To address this limitation, this study proposes PsyBridge, a hybrid intelligent decision-support framework designed for multi-dimensional mental health assessment through the integration of clinically validated screening tools, cognitive evaluation, and personality profiling within a unified architecture. The proposed framework incorporates PHQ-9 and GAD-7 assessments alongside cognitive and behavioural indicators using a modular design and a weighted aggregation mechanism to generate interpretable mental health risk classifications and recommendations. To evaluate the framework, a semi-synthetic dataset consisting of 500 patient profiles representing varying severity levels was constructed based on clinically grounded score distributions. Experimental results demonstrate that PsyBridge achieves an overall accuracy of 0.84, outperforming standalone PHQ-9 and GAD-7 assessments while improving precision, recall, and F1-score. Sensitivity analysis and ablation studies further indicate that integrating cognitive and personality components contributes to more stable classification performance and reduces inconsistencies in moderate-risk prediction. The findings suggest that PsyBridge provides a scalable and interpretable approach for AI-assisted mental health decision support, particularly within digital healthcare and telehealth environments.
Sunil Wanjari, Manish Thakre, Aayushi Asole +4
Aug 31, 2026cs.CV

TAMI: Temporally Aligned, Missingness-Aware, and Interpretable Multimodal Fusion for Mental Health Assessment in Older Adults with Mild Cognitive Impairment

Depression and anxiety in older adults with Mild Cognitive Impairment (MCI) are frequently underdiagnosed due to limited access to care. Multimodal analysis of remote clinical interviews is a scalable screening approach, but existing methods have three limitations. First, they do not correct temporal misalignment across multimodal features extracted at different resolutions, inducing spurious cross-modal associations. Second, remote recordings exhibit uneven modality dropout, but missing values are often zero-filled, making them indistinguishable from valid near-zero measurements. Finally, they do not jointly attribute predictions to modalities, questions, and interview moments, limiting fine-grained clinical interpretation. We propose a Temporally-Aligned, Missingness-Aware, Interpretable (TAMI) multimodal fusion framework. TAMI aligns speech, language, facial, and physiological features within question-answer segments on a shared timeline, encodes modality-level missingness over time, and conditions fusion on question context. In interviews with 49 older adults with MCI, TAMI achieved area under the receiver operating characteristic curve (AUROC) scores of 0.68 (depression) and 0.69 (anxiety). Fine-grained temporal alignment of multimodal features produced the largest performance gain (Δ0.1Δ{\geq}0.1). Multi-level interpretability analysis revealed that depression classification relied on eyegaze and open-ended questions, while anxiety classification depended on eyegaze and head pose, with attribution uniformly distributed across questions. Using only responses to the open-ended questions (5.1min), the depression model achieved an AUROC score of 0.67, which was not significantly different from using the full interview (19min) (p>0.05p>0.05). Our findings support designing interview protocols centered on open-ended questions for depression screening in older adults with MCI.
Merna Bibars, Bolaji Omofojoye, Allan I. Levey +3
Apr 17, 2026cs.LG

EviDep: Uncertainty-Aware Multimodal Depression Estimation via Disentangled Evidential Learning

Audio--visual recordings provide complementary cues for estimating depression severity, but their informativeness varies across time and modalities. Point predictions alone do not express the uncertainty associated with these estimates. We present EviDep, a multimodal evidential regression framework that integrates multi-scale temporal modeling and shared--private representation learning for uncertainty-aware depression estimation. Frequency-aware Feature Extraction decomposes behavioral feature sequences into multiple frequency bands and refines them with scale-specific experts. Disentangled Evidential Learning encourages the disentanglement of cross-modal shared and modality-specific information in the refined features. Multi-branch Evidential Regression maps the resulting shared and private representations to three Normal-Inverse-Gamma (NIG) outputs and uses evidence-weighted aggregation to estimate depression severity and quantify aleatoric and epistemic uncertainty. Experiments on AVEC 2013, AVEC 2014, DAIC-WOZ, and E-DAIC show competitive prediction accuracy, with ablation studies supporting the contributions of frequency-aware refinement and shared--private disentanglement. Further analyses show that estimated epistemic uncertainty helps identify higher-error predictions, while both uncertainty estimates generally increase under controlled feature degradation.
Fangyuan Liu, Sirui Zhao, Yangsong Zhang +5