Pedagogical AI in Mental Health: A Tri-Stream Fine-Tuned LLM Framework for Automated Clinical Supervision and Risk Triage
Organizations: Senior Software Engineer, Microsoft · Lead Software Engineer, BigCommerce, IEEE Senior Member · University at Buffalo, The State University of New York, Buffalo, NY, USA · Senior Software Engineer, Amazon
Abstract
Modern mental healthcare faces a critical shortage of senior supervisory oversight, leading to a "supervision gap" where novice therapists manage high-stakes risks with delayed professional feedback. This paper proposes a new framework utilizing a fine-tuned Mistral-7B-instruct model as an automated "Supervisor-in-the-Loop" system. By leveraging 106 sessions from the DAIC-WOZ dataset, the model performs a tri-stream analysis: (1) Therapeutic Alliance tracking via semantic adherence, (2) Latent risk prediction using attention-weighted analytics, and (3) Supervisory Triage via a Dynamic Clinical Urgency Index (D-CUI). Our multi-modal VAL (Visual-Acoustic-Linguistic) framework achieves 95% technique identification accuracy [95% CI: 75.1%-99.9%], alliance assessment MAE of 0.105 on a 5-point scale [95% CI: 0.059-0.151], therapeutic fidelity alpha = 0.423, and mean D-CUI of 0.370 [95% CI: 0.322-0.419]. Training converged in 105 steps with 85.2% loss reduction on a single Tesla T4 GPU. The system reduces supervisory triage latency from 72 hours to real time (~10 seconds per session), enabling proactive intervention in high-risk cases. The system addresses the cold-start problem through Bayesian priors and implements timestamp-based modality synchronization for robust multi-modal fusion.
Figures & tables
| Metric | Score | 95% CI |
|---|---|---|
| Technique Identification Accuracy | 95.5% | [75.1%, 99.9%] |
| Risk Identification Accuracy | 63.6% | [40.8%, 84.6%] |
| Alliance Assessment MAE | 0.105 | [0.059, 0.151] |
| Therapeutic Fidelity ( ) | 0.423 | — |
| Mean D-CUI | 0.370 | [0.322, 0.419] |
| Profile | R | E | D-CUI | Triage | |
|---|---|---|---|---|---|
| Low risk, expert | 0.10 | 0.80 | 0.0 | 0.125 | Routine |
| Moderate, mid-level | 0.50 | 0.50 | 0.0 | 0.500 | Elevated |
| High risk, novice | 0.90 | 0.20 | 0.3 | 1.000 | Immediate |
| Low risk + incongruent | 0.21 | 0.50 | 0.4 | 0.395 | Routine |