cs.CLNov 14, 2026

Leakage-aware calibrated decision-level fusion for mental-health text classification

Authors: Wadii Boulila, Giuseppe Varone, Bilel Benjdira, Maha Driss

Organizations: Robotics and Internet-of-Things Laboratory, Prince Sultan University, Riyadh 11586, Saudi Arabia · Department of Psychiatry, Harvard Medical School, Boston, MA 02115, United States · College of Engineering and Advanced Computing, Alfaisal University, Riyadh 11533, Saudi Arabia

Abstract

Decision-level fusion benefits from complementary errors, calibrated probabilities, and leakage-aware evaluation. We evaluated a heterogeneous transformer fusion framework on 262,276 proxy-labeled Reddit records covering anxiety, depression, schizophrenia, and suicidal ideation. In a stratified random split, MinHash-LSH identified 29.1% of test records as lexical near-duplicates of training data. Augmentation-lineage auditing found 50.2% lineage overlap with training, including 30.0% of records retained after lexical filtering. We therefore used lineage-disjoint partitions. Twelve encoders, comprising three seeds each of Bio-ClinicalBERT, MentalBERT, MentalRoBERTa, and RoBERTa-base, underwent cross-fitted validation for calibration and ensemble selection. A class-specific Complementarity Index (CI), defined as the Jaccard distance between members' correctly classified validation sets, was assessed against independent fusion gain. CI correlated with gain for depression (r=0.50) and suicidal ideation (r=0.68), but not anxiety; results for schizophrenia were mixed. The selected MentalBERT, MentalRoBERTa, and RoBERTa-base triad achieved macro-F1 0.844 [0.823, 0.863], MCC 0.779, and ECE 0.0033 after post-fusion recalibration, improving macro-F1 by 0.021 [0.007, 0.037] over the validation-selected best single encoder. Integrated-Gradients attribution diversity was 2.3 times greater in fusion-rescued cases. Removing divergence-driving words destroyed 58% of rescues, versus 12% under random removal. These findings show that lineage auditing detects dependencies missed by lexical filtering, while class-specific complementarity and post-fusion calibration reveal ensemble behavior obscured by aggregate accuracy.

Figures & tables

Explore similar work

Mar 14, 2026

Enhancing Mental Health Classification with Layer-Attentive Residuals and Contrastive Feature Learning

The classification of mental health is challenging for a variety of reasons. For one, there is overlap between the mental health issues. In addition, the signs of mental health issues depend on the context of the situation, making classification difficult. Although fine-tuning transformers has improved the performance for mental health classification, standard cross-entropy training tends to create entangled feature spaces and fails to utilize all the information the transformers contain. We present a new framework that focuses on representations to improve mental health classification. This is done using two methods. First, layer-attentive residual aggregation which works on residual connections to to weigh and fuse representations from all transformer layers while maintaining high-level semantics. Second, supervised contrastive feature learning uses temperature-scaled supervised contrastive learning with progressive weighting to increase the geometric margin between confusable mental health problems and decrease class overlap by restructuring the feature space. With a score of 74.36%, the proposed method is the best performing on the SWMH benchmark and outperforms models that are domain-specialized, such as MentalBERT and MentalRoBERTa by margins of (3.25% - 2.2%) and 2.41 recall points over the highest achieving model. These findings show that domain-adaptive pretraining for mental health text classification can be surpassed by carefully designed representation geometry and layer-aware residual integration, which also provide enhanced interpretability through learnt layer importance.
May 18, 2026

FedMental: Evaluating Federated Learning for Mental Health Detection from Social Media Data

Social media text data are often used to train Machine Learning (ML) models to identify users exhibiting high-risk mental health behaviors. However, sharing this sensitive data poses privacy risks and limits the growth of benchmark datasets. We comprehensively evaluate whether privacy-preserving ML techniques can enable safer data sharing while preserving performance. Specifically, we apply federated learning (FL) and Differentially Private FL for two widely-studied mental health prediction tasks: depression detection on X (Twitter) and suicide crisis detection on Reddit. We simulate realistic data-sharing scenarios by treating each user as a client in a non-IID setting, evaluating across different client fractions, aggregation strategies, and privacy budgets. While FL achieves comparable performance to centralized training (centralized F1 = 85.63; best FL model F1 = 83.16) on depression identification, we find that Differentially Private FL has a large performance-privacy trade-off (up to F1 = 27.01 drop) even with low levels of noise (epsilon = 50). This is due to the distortion of highly informative yet sparse mental health linguistic markers related to mental health, like health topics and emotion words. This research empirically demonstrates the potential and limitations of current privacy preservation techniques for mental health inference tasks.
Jun 19, 2026

Evaluating Document-Tuned Transformer Representations for Person-level Mental Health Assessment

Person-level psychological assessment requires aggregating meaning across many messages from the same individual, a task that document-level training objectives were not explicitly designed for. We present a systematic, empirical comparison between architecturally matched traditional (a) base-transformers and (b) document-tuned-transformers (further contrastively fine-tuned at the document-level, sometimes referred to as "sentence transformers") under otherwise identical conditions. Comparing layer-wise and overall performance across two longitudinal mental health and psychological datasets, we find document-tuned models demonstrated a consistent improvement over base representations (increase in Pearson r of 13.4%, p=.015). Robustness analyses revealed document-tuned models remained more accurate under perturbations to word deletion, synonym replacement, typo injection, and back translation. Further, hedged language (e.g., usually') was more characteristic of outcomes in document-tuned embeddings while abundance (e.g., lot') was more characteristic of base-transformers, suggesting document-tuned models may better capture uncertainty. These results suggest representation choice impacts mental health prediction, document-tuned models often being more adept.