MC-TRCM: Observation-Aware Recursive Fusion for Incomplete Mobile and Wearable Mental-Health Feature Views

Wentao Wang; Lifeng Han; Zining Ren; Hengyu Zhong; Guangyu Zou

cs.LGOct 8, 2026

MC-TRCM: Observation-Aware Recursive Fusion for Incomplete Mobile and Wearable Mental-Health Feature Views

Authors: Wentao Wang, Lifeng Han, Zining Ren, Hengyu Zhong, Guangyu Zou

Organizations: Dalian University of Technology, Dalian, China · The Hong Kong Polytechnic University, Hong Kong, China · Southwest University, Chongqing, China

Abstract

Public mobile and wearable mental-health datasets often provide summarized feature tables rather than synchronized raw sensor streams. In these releases, each anchor corresponds to a survey or label time and may combine phone or wearable summaries, prior symptom scores, demographics, clinical variables, and source-availability indicators. We propose the Modality-Conditioned Temporal Recursive Context Model (MC-TRCM), which preserves each feature source as a separate token and incorporates missingness as part of the input context. Observed sources are encoded with values and missingness summaries, absent sources use learned absence tokens, dataset and task embeddings condition fusion, and a recursive prediction head refines each output over validation-selected steps. We evaluated MC-TRCM on six predefined endpoints from DepreST-CAT and Prediction of Severity Change-Depression (PSYCHE-D) using participant-level splits and validation-only model selection. MC-TRCM achieved the lowest mean absolute error on DepreST-CAT Patient Health Questionnaire-9 (PHQ-9) and Generalized Anxiety Disorder-7 (GAD-7) severity, improving over the best tabular reference by 0.181 and 0.217 scale points. Classification endpoints showed task-dependent behavior: MC-TRCM matched the best rounded GAD-7 category balanced accuracy, was numerically highest by 0.002 balanced-accuracy points on PSYCHE-D multiclass prediction, and remained close to the strongest references on PHQ-9 category and PSYCHE-D binary prediction. Ablations support Feature-wise Linear Modulation, absence tokens, missingness projections, and recursive refinement, while calibration and feature-source controls characterize endpoint behavior. Our code is available at https://github.com/Botwwt/MC-TRCM.

Figures & tables

Figure 1: Endpoint construction and evaluation protocol. Public cohorts are harmonized into modality-specific feature views, aligned to participant anchors, and evaluated with training-only preprocessing and validation-locked model selection.

Figure 2: MC-TRCM architecture. Source tokens carry observed values, missingness context, and modality identity; FiLM-conditioned Transformer fusion and recursive decoding produce the endpoint-specific prediction.

Model	PHQ-9 MAE $\downarrow$	GAD-7 MAE $\downarrow$	PHQ-9 BA $\uparrow$	GAD-7 BA $\uparrow$	Psy. bin. BA $\uparrow$	Psy. multi. BA $\uparrow$
Elastic Net	7.582 $\pm$ 0.000	6.791 $\pm$ 0.000	0.178 $\pm$ 0.000	0.245 $\pm$ 0.000	0.582 $\pm$ 0.001	0.499 $\pm$ 0.000
LightGBM	6.328 $\pm$ 0.090	5.297 $\pm$ 0.084	0.239 $\pm$ 0.043	0.343 $\pm$ 0.031	0.574 $\pm$ 0.004	0.488 $\pm$ 0.002
XGBoost	5.918 $\pm$ 0.066	5.318 $\pm$ 0.096	0.242 $\pm$ 0.024	0.297 $\pm$ 0.019	0.587 $\pm$ 0.005	0.505 $\pm$ 0.005
EBM	5.817 $\pm$ 0.085	5.028 $\pm$ 0.088	0.220 $\pm$ 0.050	0.308 $\pm$ 0.030	0.589 $\pm$ 0.007	0.512 $\pm$ 0.003
MLP	6.138 $\pm$ 0.100	5.449 $\pm$ 0.106	0.193 $\pm$ 0.041	0.278 $\pm$ 0.035	0.566 $\pm$ 0.003	0.463 $\pm$ 0.015
GRU	6.118 $\pm$ 0.031	5.364 $\pm$ 0.003	0.212 $\pm$ 0.040	0.250 $\pm$ 0.016	0.582 $\pm$ 0.004	0.509 $\pm$ 0.004

Table 1: Main test performance across pre-specified comparator families (mean and standard deviation).

Dataset	Endpoint	Metric	Test $n$	MC-TRCM	Best ref.	Ref. value
StudentLife	PHQ-9 severity	MAE $\downarrow$	7	3.812 $\pm$ 0.302	Transformer	4.916 $\pm$ 0.229
StudentLife	PHQ-9 category	BA $\uparrow$	7	0.350 $\pm$ 0.152	XGBoost	0.250 $\pm$ 0.000
Depresjon	Depr. status	BA $\uparrow$	11	0.911 $\pm$ 0.124	EBM	1.000 $\pm$ 0.000
Depresjon	MADRS severity	MAE $\downarrow$	4	2.001 $\pm$ 0.622	EBM	2.493 $\pm$ 0.198
OBF	Clin. vs. Ctrl.	BA $\uparrow$	33	1.000 $\pm$ 0.000	LightGBM	1.000 $\pm$ 0.000

Table 2: Auxiliary dataset test results. These endpoints are outside the primary benchmark because held-out sets are small or saturated.

Figure 3: Feature-source and calibration controls. a Balanced accuracy (BA) across feature-view subsets. Symptom/static/clinical context provides strong categorical signal, while sensor and missingness views contribute endpoint-dependent information. b Calibration deltas (MC minus reference); negative values favor MC-TRCM. Abbreviations: S+M (sensor + missingness), SSC (symptom/static/clinical context), and ref. (matched tabular reference).

Dataset	$K=1$	$K=2$	$K=4$	$K=6$	$K=8$	Selected
DepreST-CAT	0.589	0.592	0.599	0.598	0.606	8
PSYCHE-D	0.548	0.552	0.563	0.555	0.553	4

Table 3: Validation-selected recursive depth. Values are within-dataset validation selection scores used for recursive-depth tuning: severity endpoints contribute

1-\mathrm{MAE}/R_{\tau}

, and classification or ordinal endpoints contribute BA. Higher is better. Scores are not intended for comparison across datasets or as test metrics.

Variant	$\Delta$ MAE (2 severity)	$\Delta$ BA (4 category)
Full MC-TRCM	0.000	0.000
No FiLM	+0.136	-0.017
No absence token	+0.125	-0.012
No missingness projection	+0.041	-0.008
No PCGrad	+0.008	-0.003
No task conditioning	+0.067	+0.002

Table 4: Five-seed component ablations by metric family.

Explore similar work

TAMI: Temporally Aligned, Missingness-Aware, and Interpretable Multimodal Fusion for Mental Health Assessment in Older Adults with Mild Cognitive Impairment
Aug 31, 2026Merna Bibars, Bolaji Omofojoye, Allan I. Levey +3Missing-Modality Learning Depression Detection
Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA · Department of Systems and Biomedical Engineering, Cairo University, Giza, Egypt · Department of Biomedical Informatics, Emory University, Atlanta, GA, USA +2
Beyond Semantics: An Evidential Reasoning-Aware Multi-View Learning Framework for Trustworthy Mental Health Prediction
May 6, 2026Yucheng Ruan, Ling Huang, Qika Lin +2Evidential Deep Learning Uncertainty Quantification
Saw Swee Hock School of Public Health, National University of Singapore, Singapore
VCR: Learning Valid Contextual Representation for Incomplete Wearable Signals
May 13, 2026Yuxuan Weng, Wenhan Luo, Qijia ShaoMissing-Modality Learning Disentangled Representation Learning
The Hong Kong University of Science and Technology (HKUST)

Aug 31, 2026, cs.CVJ/K move · Enter open · S save

TAMI: Temporally Aligned, Missingness-Aware, and Interpretable Multimodal Fusion for Mental Health Assessment in Older Adults with Mild Cognitive Impairment

Merna Bibars, Bolaji Omofojoye, Allan I. Levey, Rachel Hershenberg+2

Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology and Emory University, Atlanta, GA, USA · Department of Systems and Biomedical Engineering, Cairo University, Giza, Egypt · Department of Biomedical Informatics, Emory University, Atlanta, GA, USA+2

Missing-Modality Learning Depression Detection Cross-Modal Alignment Multimodal Model Interpretability Multimodal Fusion

Open paper AskPDF arXiv

Depression and anxiety in older adults with Mild Cognitive Impairment (MCI) are frequently underdiagnosed due to limited access to care. Multimodal analysis of remote clinical interviews is a scalable screening approach, but existing methods have three limitations. First, they do not correct temporal misalignment across multimodal features extracted at different resolutions, inducing spurious cross-modal associations. Second, remote recordings exhibit uneven modality dropout, but missing values are often zero-filled, making them indistinguishable from valid near-zero measurements. Finally, they do not jointly attribute predictions to modalities, questions, and interview moments, limiting fine-grained clinical interpretation. We propose a Temporally-Aligned, Missingness-Aware, Interpretable (TAMI) multimodal fusion framework. TAMI aligns speech, language, facial, and physiological features within question-answer segments on a shared timeline, encodes modality-level missingness over time, and conditions fusion on question context. In interviews with 49 older adults with MCI, TAMI achieved area under the receiver operating characteristic curve (AUROC) scores of 0.68 (depression) and 0.69 (anxiety). Fine-grained temporal alignment of multimodal features produced the largest performance gain (

Δ{\geq}0.1

). Multi-level interpretability analysis revealed that depression classification relied on eyegaze and open-ended questions, while anxiety classification depended on eyegaze and head pose, with attribution uniformly distributed across questions. Using only responses to the open-ended questions (5.1min), the depression model achieved an AUROC score of 0.67, which was not significantly different from using the full interview (19min) (

p>0.05

). Our findings support designing interview protocols centered on open-ended questions for depression screening in older adults with MCI.

Abstract

Explore similar work

TAMI: Temporally Aligned, Missingness-Aware, and Interpretable Multimodal Fusion for Mental Health Assessment in Older Adults with Mild Cognitive Impairment

Beyond Semantics: An Evidential Reasoning-Aware Multi-View Learning Framework for Trustworthy Mental Health Prediction

VCR: Learning Valid Contextual Representation for Incomplete Wearable Signals