MindGuard: Guardrail Classifiers for Multi-Turn Mental Health Support
Organizations: Sword Health · Instituto de Telecomunicações · Instituto Superior Técnico
Abstract
Large language models are increasingly used for mental health support, yet their conversational coherence alone does not ensure clinical appropriateness. Existing general-purpose safeguards often fail to distinguish therapeutic disclosures from genuine clinical crises, and progress on developing better ones is bottlenecked by a lack of clinically grounded evaluation resources. To address this gap, we introduce a risk taxonomy, developed in collaboration with PhD-level clinical psychologists, that identifies actionable harm (self-harm and harm to others) while preserving space for safe, non-crisis therapeutic content. We release MindGuard-testset, a dataset of multi-turn conversations annotated at the turn level by clinical experts, which is, to our knowledge, the first public benchmark with turn-level clinical risk labels for multi-turn mental health support. We also release an automated red-teaming (ART) framework designed to measure how safety classifiers affect downstream model behavior in adversarial multi-turn interactions. Using synthetic dialogues generated via a controlled two-agent setup, we train MindGuard, a family of lightweight safety classifiers (with 4B and 8B parameters). Our classifiers reduce false positives at high-recall operating points and, when paired with clinician language models, help achieve lower attack success and harmful engagement rates compared to general-purpose safeguards. We release all models, training and human evaluation data, and the ART framework.
Figures & tables
| Turn-level | Detection (n=17) | ||||
| Model | AUROC | FPR 90 | FPR 95 | Det. | At/Before |
| LG 1B | 59% | 53% | |||
| LG 8B | 71% | 41% | |||
| GPT 20B | 88% | 65% | |||
| GPT 120B | 94% | 82% | |||
| MindGuard 4B | 100% | 82% | |||
Appendix figures & tables22 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | AUROC | FPR@90TPR | FPR@95TPR |
| PolyGuard-Qwen-Smol | 0.839 | 0.508 | 0.535 |
| PolyGuard-Ministral | 0.921 | 0.214 | 0.289 |
| PolyGuard-Qwen | 0.934 | 0.219 | 0.323 |
| Qwen3Guard-Gen-4B (loose) | 0.889 | 0.401 | 0.569 |
| Qwen3Guard-Gen-4B (strict) | 0.946 | 0.164 | 0.247 |
| Qwen3Guard-Gen-8B (loose) | 0.913 | 0.238 | 0.535 |
| GLM-4.6 | Qwen3-235B-Instruct | Qwen3-235B-Thinking | ||||
| Model | ASR | EHR | ASR | EHR | ASR | EHR |
| No Classifier | 25.1 | 13.7 | 54.1 | 33.2 | 20.4 | 20.6 |
| Llama Guard 3 1B | 21.5 | 12.8 | 40.8 | 27.6 | 14.9 | 16.1 |
| Llama Guard 3 8B | 18.6 | 8.4 | 41.7 | 24.0 | 15.9 | 14.8 |
| gpt-oss-safeguard 20B | 14.2 | 6.6 | 29.7 | 20.1 | 11.4 | 13.1 |
| gpt-oss-safeguard 120B | 14.1 | 6.9 | 26.8 | 17.7 | 11.7 | 13.4 |
LLM Guardrails
Large language model guardrails detect or constrain harmful outputs, prompt attacks, and unsafe agent actions before they cause harm.
See all 205 papers