CogDrift-4D: Hierarchical Multimodal Modeling of Temporal Behavioral Decay for MCI Detection
Organizations: Department of Computer Science, University of Massachusetts Boston, MA, USA · School of Medicine, University of North Carolina at Chapel hill, NC, USA
Abstract
Early detection of mild cognitive impairment remains a critical bottleneck in dementia care, demanding scalable tools capable of capturing subtle behavioral decay. To capture the subtle temporal changes of cognitive decline, we introduce a behavioral dataset collected across seven unscripted multimodal interactions over an 18-month horizon. This dataset formalizes continuous, multi-session interactions into a dense 4D tensor (Batch * Session * Time * Feature) that explicitly preserves concurrent local dynamics alongside long-term changes. To effectively process these complex behavioral patterns, we propose CogDrift-4D: a novel, hierarchical 4D Transformer framework. To capture the complex trajectory of cognitive decline, CogDrift-4D implements a dual-level behavioral fatigue and drift modeling architecture. At the intra-session level, the model computes localized least-squares regression slopes to quantify the acute ``fading'' of frame-level facial action units and acoustic energy. At the inter-session level, a recursive cross-attention feedback encoder acts as a cumulative fatigue accumulator, mapping the patient's continuous behavioral drift across successive sessions. To mitigate environmental noise and track failures across sessions, a localized Quality Gate dynamically filters low-integrity features before final fusion, also an early fusion that integrates semantic insights with raw physical dynamics. Evaluated on structured verbal and linguistic interactions, our architecture successfully detects MCI with a classification accuracy of 0.733 +- 0.13 and an F1-score of 0.716 +- 0.14, outperforming advanced baselines. The framework provides a mathematically rigorous, scalable, and robust foundation for home-based, real-time dementia screening.
Figures & tables
| Model Variant | Accuracy | F1-score | AUC |
|---|---|---|---|
| Baseline 1 | 0.5429 0.09 | 0.5429 0.1 | 0.4990 0.00 |
| Baseline 2 | 0.5524 0.09 | 0.5413 0.09 | 0.5528 0.07 |
| AAMLA | 0.5714 0.07 | 0.5580 0.06 | 0.5712 0.04 |
| PRA-PoE | 0.5810 0.07 | 0.5665 0.07 | 0.5588 0.06 |
| M-BEHRT | 0.6571 0.13 | 0.6364 0.17 | 0.6549 0.11 |
| CogDrift-4D | 0.733 0.13 | 0.716 0.15 | 0.709 0.14 |
| Model Variant | Accuracy | F1-score | AUC |
|---|---|---|---|
| Without Fatigue | 0.714 0.14 | 0.693 0.16 | 0.687 0.16 |
| Without Divergence | 0.705 0.14 | 0.689 0.15 | 0.685 0.14 |
| Without quality | 0.722 0.15 | 0.712 0.15 | 0.703 0.15 |
| Video + Audio + Text | 0.705 0.16 | 0.688 0.18 | 0.681 0.18 |
| Full Model | 0.742 0.17 | 0.731 0.18 | 0.724 0.18 |