cs.AINov 17, 2026

EDA: Observe, Verify, and Assess for Evidence-Grounded Multimodal Depression Analysis

Authors: Changzeng Fu, Ming Shang, Xinyi Zhang, Jiadong Wang, Runhao Zeng, Shiqi Zhao

Organizations: Northeastern University · Northeastern University at Qinhuangdao · Technical University of Munich · Shenzhen MSU-BIT University

Abstract

Existing multimodal depression assessment methods often summarize long recordings into global behavioral descriptions. This leaves the source and decision relevance of individual observations difficult to inspect. We propose EDA, an evidence-grounded framework organized around Observe, Verify, and Assess. In Observe, literature-informed acoustic and visual cues are localized into source-linked Evidence Records with temporal intervals and behavioral measurements. In Verify, frozen multimodal agents examine whether the cited media support each observation. The simplified Support admission policy retains positively verified records and their review grounds. In Assess, statistical descriptors and temporal events support reference-based behavioral comparison, while PHQ-9-guided text analysis provides complementary symptom evidence. Cross-modal disagreement triggers review of the original text before final selection. Experiments on D-Vlog and LMVD evaluate human-referenced cue agreement and downstream assessment utility separately. AI cue annotations achieve 91.02% accuracy against human consensus labels on shared source segments. With a frozen 7B backbone, EDA Support improves F1 over direct prediction by 7.61 and 9.88 percentage points on D-Vlog Leave-Query-Out and LMVD, respectively. These findings support explicit, source-linked behavioral evidence for inspectable multimodal depression assessment.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Aug 22, 2026

HiMA-MDD: A Hierarchical Multi-Agent Harness for Interpretable Multimodal Depression Detection in Clinical Interviews

Depression assessment from multimodal clinical interviews requires integrating dispersed evidence from multiple symptoms into a coherent PHQ-8 profile. This process is hierarchical: relevant evidence is often sparse and context-dependent within local question-answer exchanges, multiple exchanges jointly support symptom-level judgments, and the final assessment depends on the coherence of the complete symptom profile. Existing LLM systems either process interviews holistically or distribute work across generic agent roles; neither design necessarily provides an explicit orchestration mechanism that coordinates evidence access, item-score authority, bounded feedback, and state recording across these levels. To address this gap, we introduce HiMA-MDD, a hierarchical multi-agent harness that aligns this assessment hierarchy with three agent layers. After non-agentic preprocessing constructs context-preserving multimodal QA units, Layer 1 identifies candidate QA-to-item relations and supports bounded item-grounded evidence routing. Layer 2 assigns symptom groups to operational factor specialists, with one specialist responsible for each provisional item score. Layer 3 audits the complete provisional profile, requests at most one round of targeted revision, and reconstructs the verified PHQ-8 profile. This layered design naturally yields a Hierarchical Evidence Trace, preserves all intermediate evidence, judgments, and revisions for auditability. The final item scores then deterministically produce the total score and screening decision. Using Qwen2.5-72B-Instruct as the harness backbone, our experiments on E-DAIC demonstrate that HiMA-MDD outperforms the compared state-of-the-art methods.
Oct 8, 2026

From Sparse Representations to Behavioral Insights for Multimodal Depression Assessment

Multimodal depression assessment offers a promising approach to analyzing behavioral patterns associated with depression. However, existing methods often rely on dense and opaque multimodal representations, making it difficult to interpret the behavioral patterns underlying their predictions. In this work, we introduce BehavDep, a sparse factor-based framework that decomposes multimodal behavioral representations into sparse latent factors and associates them with behaviorally meaningful concepts through a semantic bridge. To address the mismatch between user-level annotations and heterogeneous video-level behaviors, BehavDep further learns video-level depression tendency scores under weak supervision and aggregates information across multiple observations for user-level assessment. Extensive experiments demonstrate that BehavDep achieves the best overall assessment performance while revealing complementary modality contributions, heterogeneous behavioral patterns across observations, and prediction responses to concept-level editing. These results show that BehavDep provides a structured and interpretable approach to analyzing multimodal behavioral representations for depression assessment.
Apr 17, 2026

EviDep: Uncertainty-Aware Multimodal Depression Estimation via Disentangled Evidential Learning

Audio--visual recordings provide complementary cues for estimating depression severity, but their informativeness varies across time and modalities. Point predictions alone do not express the uncertainty associated with these estimates. We present EviDep, a multimodal evidential regression framework that integrates multi-scale temporal modeling and shared--private representation learning for uncertainty-aware depression estimation. Frequency-aware Feature Extraction decomposes behavioral feature sequences into multiple frequency bands and refines them with scale-specific experts. Disentangled Evidential Learning encourages the disentanglement of cross-modal shared and modality-specific information in the refined features. Multi-branch Evidential Regression maps the resulting shared and private representations to three Normal-Inverse-Gamma (NIG) outputs and uses evidence-weighted aggregation to estimate depression severity and quantify aleatoric and epistemic uncertainty. Experiments on AVEC 2013, AVEC 2014, DAIC-WOZ, and E-DAIC show competitive prediction accuracy, with ablation studies supporting the contributions of frequency-aware refinement and shared--private disentanglement. Further analyses show that estimated epistemic uncertainty helps identify higher-error predictions, while both uncertainty estimates generally increase under controlled feature degradation.