cs.CLNov 14, 2026

Leakage-aware calibrated decision-level fusion for mental-health text classification

Authors: Wadii Boulila, Giuseppe Varone, Bilel Benjdira, Maha Driss

Organizations: Robotics and Internet-of-Things Laboratory, Prince Sultan University, Riyadh 11586, Saudi Arabia · Department of Psychiatry, Harvard Medical School, Boston, MA 02115, United States · College of Engineering and Advanced Computing, Alfaisal University, Riyadh 11533, Saudi Arabia

Abstract

Decision-level fusion benefits from complementary errors, calibrated probabilities, and leakage-aware evaluation. We evaluated a heterogeneous transformer fusion framework on 262,276 proxy-labeled Reddit records covering anxiety, depression, schizophrenia, and suicidal ideation. In a stratified random split, MinHash-LSH identified 29.1% of test records as lexical near-duplicates of training data. Augmentation-lineage auditing found 50.2% lineage overlap with training, including 30.0% of records retained after lexical filtering. We therefore used lineage-disjoint partitions. Twelve encoders, comprising three seeds each of Bio-ClinicalBERT, MentalBERT, MentalRoBERTa, and RoBERTa-base, underwent cross-fitted validation for calibration and ensemble selection. A class-specific Complementarity Index (CI), defined as the Jaccard distance between members' correctly classified validation sets, was assessed against independent fusion gain. CI correlated with gain for depression (r=0.50) and suicidal ideation (r=0.68), but not anxiety; results for schizophrenia were mixed. The selected MentalBERT, MentalRoBERTa, and RoBERTa-base triad achieved macro-F1 0.844 [0.823, 0.863], MCC 0.779, and ECE 0.0033 after post-fusion recalibration, improving macro-F1 by 0.021 [0.007, 0.037] over the validation-selected best single encoder. Integrated-Gradients attribution diversity was 2.3 times greater in fusion-rescued cases. Removing divergence-driving words destroyed 58% of rescues, versus 12% under random removal. These findings show that lineage auditing detects dependencies missed by lexical filtering, while class-specific complementarity and post-fusion calibration reveal ensemble behavior obscured by aggregate accuracy.

Figures & tables

Explore similar work

CardsList
  1. Enhancing Mental Health Classification with Layer-Attentive Residuals and Contrastive Feature Learning

    Mar 14, 2026Menna Elgabry, Ali Hamdi, Khaled ShabanMental HealthTransformer

  2. FedMental: Evaluating Federated Learning for Mental Health Detection from Social Media Data

    May 18, 2026Nuredin Ali Abdelkadir, Anjali Ratnam, Zeerak Talat +1Depression DetectionMental Health

  3. Evaluating Document-Tuned Transformer Representations for Person-level Mental Health Assessment

    Jun 19, 2026Aaron Marker, Oscar Kjell, Vasudha Varadarajan +1Sentence EmbeddingsMental Health