HiMA-MDD: A Hierarchical Multi-Agent Harness for Interpretable Multimodal Depression Detection in Clinical Interviews

Ao Chen; Xiaojiang Peng

cs.AIAug 22, 2026

HiMA-MDD: A Hierarchical Multi-Agent Harness for Interpretable Multimodal Depression Detection in Clinical Interviews

Authors: Ao Chen, Xiaojiang Peng

Organizations: School of Artificial Intelligence, Shenzhen Technology University

Abstract

Depression assessment from multimodal clinical interviews requires integrating dispersed evidence from multiple symptoms into a coherent PHQ-8 profile. This process is hierarchical: relevant evidence is often sparse and context-dependent within local question-answer exchanges, multiple exchanges jointly support symptom-level judgments, and the final assessment depends on the coherence of the complete symptom profile. Existing LLM systems either process interviews holistically or distribute work across generic agent roles; neither design necessarily provides an explicit orchestration mechanism that coordinates evidence access, item-score authority, bounded feedback, and state recording across these levels. To address this gap, we introduce HiMA-MDD, a hierarchical multi-agent harness that aligns this assessment hierarchy with three agent layers. After non-agentic preprocessing constructs context-preserving multimodal QA units, Layer 1 identifies candidate QA-to-item relations and supports bounded item-grounded evidence routing. Layer 2 assigns symptom groups to operational factor specialists, with one specialist responsible for each provisional item score. Layer 3 audits the complete provisional profile, requests at most one round of targeted revision, and reconstructs the verified PHQ-8 profile. This layered design naturally yields a Hierarchical Evidence Trace, preserves all intermediate evidence, judgments, and revisions for auditability. The final item scores then deterministically produce the total score and screening decision. Using Qwen2.5-72B-Instruct as the harness backbone, our experiments on E-DAIC demonstrate that HiMA-MDD outperforms the compared state-of-the-art methods.

Figures & tables

Figure 1: Comparison between holistic interview-level assessment and our HiMA-MDD.

Figure 2: Overview of the HiMA-MDD assessment framework.

Group	Method	MAE $\downarrow$	RMSE $\downarrow$	Acc. $\uparrow$	Screening $\kappa\uparrow$	F1[C] $\uparrow$	F1[D] $\uparrow$	Macro F1 $\uparrow$
Reported Results	MLlm-DR	3.57	4.83	—	—	—	—	0.69
	GPT-5.5	—	—	0.70	—	0.75	0.61	0.68
	Gemini-3.1-Pro	—	—	0.73	—	0.77	0.68	0.73
	Claude-Opus-4.6	—	—	0.71	—	0.76	0.65	0.71
	DeepSeek-V4	—	—	0.71	—	0.78	0.60	0.69
	Dep-LLM (Gemma3-12B- Instruct)	—	—	0.75	—	0.80	0.67	0.73

Table 1: Comparison of HiMA-MDD with baselines on E-DAIC.Results marked with * are better than Zero-Shot (

p<0.05

) based on metric-specific one-tailed paired tests.

Method	Total MAE $\downarrow$	Acc. $\uparrow$	Screening $\kappa\uparrow$	Macro F1 $\uparrow$
Zero-Shot	3.9143	0.7143	0.2081	0.5536
3-Shot	3.7714	0.7143	0.2457	0.5949
CoT	3.4286	0.7429	0.3046	0.6182
MDAgents	3.0571	0.8000	0.4842	0.7281
AgentMental	3.0571	0.7714	0.5122	0.7552
HiMA-MDD (raw)	3.9714	0.8571	0.6765	0.8381

Table 2: Results on the DAIC-WOZ development split. Bold indicates the best performance. Following the established DAIC-WOZ development-cohort protocol, all rows report raw outputs using ASR-derived transcripts.

Variant	Acc. $\uparrow$	$\kappa\uparrow$	F1[C] $\uparrow$	F1[D] $\uparrow$	Macro-F1 $\uparrow$
HiMA-MDD	0.8036	0.5686	0.8493	0.7179	0.7836
w/o ①	0.7679	0.4902	0.8219	0.6667	0.7443
w/o ②	0.7857	0.5248	0.8378	0.6842	0.7610
w/o ③	0.7857	0.5340	0.8333	0.7000	0.7667

Table 3: Component ablations of our HiMA-MDD. All rows report raw outputs w/o calibration. ① Cross-factor audit and targeted revision. ② Centralized verified-profile reconstruction. ③ Acoustic descriptor augmentation.

Figure 3: Hierarchical Evidence Trace for E-DAIC participant 716, showing evidence retention after domain review for Depressed Mood and multi-evidence aggregation for Feelings of Failure.

Reasoning Configuration	Total MAE $\downarrow$	Total RMSE $\downarrow$	Screening $\kappa\uparrow$	Macro F1 $\uparrow$
Single Agent	4.3036	5.2627	0.4667	0.7333
Two-Factor	4.3571	5.3352	0.5000	0.7499
Four-Factor (Default)	4.0714	5.0533	0.4902	0.7443
Item-Specific	4.3393	5.3802	0.4563	0.7278
Four-Factor w/ Shared Evidence	4.2500	5.0815	0.4340	0.7169

Table 4: Effect of reasoning granularity and evidence access. Configurations share candidate QA-to-item grounding, backbone, rubric, acoustic descriptors, and test cases, and the comparison ends after Layer 2 symptom reasoning.

Partition	Participants	Role in this study
Train	163	Post-hoc calibration fitting and final refit
Development	56	Post-hoc calibration selection and final refit; three-shot pool
Test	56	Final evaluation only

Table 5: Official E-DAIC partitions and their use in this study. The test partition is reserved for final reporting.

Descriptor	Low/moderate boundary	Moderate/high boundary
Pitch variability	0.15	0.45
Mean loudness	0.20	0.80
Local jitter	0.015	0.050
Local shimmer (dB)	0.40	1.20
HNR (dB)	5.0	20.0

Table 6: Engineering thresholds used only to verbalize selected eGeMAPSv02 functionals. They do not define depression severity.

System	Metric	System value	Zero-Shot value	Test	$p$	Star
HiMA-MDD (raw)	Total MAE	3.9643	4.5357	Paired $t$ -test	0.1347	No
HiMA-MDD (raw)	Total RMSE	4.9497	6.1296	Paired $t$ -test on squared error	0.0161	Yes
HiMA-MDD (raw)	Accuracy	0.8036	0.7321	Exact McNemar	0.1719	No
HiMA-MDD (raw)	F1[C]	0.8493	0.8193	Paired approximate randomization	0.2113	No
HiMA-MDD (raw)	F1[D]	0.7179	0.4828	Paired approximate randomization	0.0154	Yes
HiMA-MDD + Post-hoc Calibration	Total MAE	3.4107	4.5357	Paired $t$ -test	0.0122	Yes

Table 7: Tests underlying the star markers in the main comparison. All tests use Zero-Shot as the reference and report metric-specific one-tailed

p

-values. The HiMA-MDD + Post-hoc Calibration condition uses the selected supervised Bayesian Ridge ordinal score-correction model. These exploratory values are nominal and unadjusted for multiple comparisons.

Explore similar work

May 4, 2026

ADAPTS: Agentic Decomposition for Automated Protocol-agnostic Tracking of Symptoms

Modeling latent clinical constructs from unconstrained clinical interactions is a unique challenge in affective computing. We present ADAPTS (Agentic Decomposition for Automated Protocol-agnostic Tracking of Symptoms), a framework for automated rating of depression and anxiety severity using a mixture-of-agents LLM architecture. This approach decomposes long-form clinical interviews into symptom-specific reasoning tasks, producing auditable justifications while preserving temporal and speaker alignment. Generalization was evaluated across two independent datasets (

N=204

) with distinct interview structures. On high-discrepancy interviews, automated ratings approximated expert benchmarks (

\text{absolute error}=22

) more closely than original human ratings (

\text{absolute error}=26

). Implementing an ``extended'' protocol that incorporates qualitative clinical conventions significantly stabilized ratings, with absolute agreement reaching

\text{ICC(2,1)} = 0.877

. These findings suggest that the ADAPTS framework enables promising evaluations of psychiatric severity. While the current implementation is purely text-based, the underlying architecture is readily extensible to multimodal inputs, including acoustic and visual features. By approximating expert-level precision in a protocol-agnostic manner, this framework provides a foundation for objective and scalable psychiatric assessment, especially in resource-limited settings.

Alexandria K. Vail, Marcelo Cicconet, Katie Aafjes-van Doorn +2

Deliberate Solutions, Inc. New York, New York, USA

Depression Detection Mental Health Depression Severity Estimation Clinical Language Model Evaluation Clinical NLP

Ask PDF

Jul 30, 2026

The MADRS Pipeline: Supporting Depression Assessment in Clinical Trials

Depression is a major mental disorder for which diagnosis relies primarily on clinical assessments. Automated methods to support its detection via the psychiatric MADRS scale are getting more and more attention. While existing solutions primarily focus on detecting the disorder from different text sources (e.g., online text, social media), there is still limited support for clinical trials, where clinical assessments are conducted through structured interviews based on standard guidelines such as SIGMA. In this work, we develop a LLM pipeline specifically designed to support clinicians in supporting the assessment of depression in patients enrolled in clinical trials. Our pipeline converts audio interviews into transcripts, maps them into the ten MADRS symptom items, estimates their severity, and identify problematic clinical ratings associated with them. Evaluation on real clinical interviews shows a strong overall correlation of 0.867 with expert ratings, providing interpretable support for future assessments in clinical trials.

Mila Fodor, Katalin Ócsai, Francesco Periti +2

Clario, part of Thermo Fisher Scientific

Depression Detection Clinical Decision Support Depression Severity Estimation Medical LLMs

Ask PDF

Apr 13, 2026

Dynamic Summary Generation for Interpretable Multimodal Depression Detection

Depression remains widely underdiagnosed and undertreated because stigma and subjective symptom ratings hinder reliable screening. To address this challenge, we propose a coarse-to-fine, multi-stage framework that leverages large language models (LLMs) for accurate and interpretable detection. The pipeline performs binary screening, five-class severity classification, and continuous regression. At each stage, an LLM produces progressively richer clinical summaries that guide a multimodal fusion module integrating text, audio, and video features, yielding predictions with transparent rationale. The system then consolidates all summaries into a concise, human-readable assessment report. Experiments on the E-DAIC and CMDC datasets show significant improvements over state-of-the-art baselines in both accuracy and interpretability.

Shiyu Teng, Jiaqing Liu, Hao Sun +6

College of Information Science and Engineering, Ritsumeikan University, Shiga, Japan · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · Faculty of Engineering, University of the Ryukyus, Japan

Depression Detection Depression Severity Estimation Multimodal Model Interpretability Medical Report Generation

Ask PDF