Generative AI Purpose-built for Social and Mental Health: A Real-World Pilot

Thomas D. Hull; Lizhe Zhang; Caitlin A. Stamatis; Patricia A. Arean; Matteo Malgaroli

cs.CYNov 12, 2025

Generative AI Purpose-built for Social and Mental Health: A Real-World Pilot

Authors: Thomas D. Hull, Lizhe Zhang, Caitlin A. Stamatis, Patricia A. Arean, Matteo Malgaroli

Organizations: Slingshot AI · CREATIV Lab, Department of Psychiatry and Behavioral Sciences, University of Washington · Department of Psychiatry, NYU School of Medicine · Park Ave, 8th Floor, New York NY.

Abstract

Generative AI chatbots built for mental health could extend access to care, but evidence from real-world use is limited. We report a single-arm, naturalistic pilot of a foundation model trained for mental health, among 299 US adults with at least moderate depressive or anxiety symptoms who were followed for up to 12 months. Depression and anxiety symptoms fell by 10 weeks (Cohen's d 0.93 and 0.79), with loneliness, behavioral activation, and social interaction improving. Clinicians confirmed that automated safeguards were escalated appropriately. Participants fell into non-responding (57.2%), improving (37.1%) and rapidly improving (5.7%) trajectories. Early working alliance and greater engagement were associated with better outcomes. The AI deployed ten identified intervention families whose delivery varied with baseline anxiety and depression, with the overall ratio of clinical to non-clinical content increasing according to severity without survey information access. These findings support the feasibility of purpose-built AI for mental health.

Explore similar work

AI Safety Training Can be Clinically Harmful
Apr 25, 2026Suhas BN, Andrew M. Sherrill, Rosa I. Arriaga +2Artificial Intelligence Safety Mental Health
College of Information Sciences and Technology, Penn State University, USA · Department of Psychiatry and Behavioral Sciences, Emory University, USA · School of Interactive Computing, Georgia Tech, USA +1
VERA-MH: Validation of Ethical and Responsible AI in Mental Health
May 13, 2026Luca Belli, Kate H. Bentley, Josh Gieringer +6Chatbots Mental Health
Spring Health · Yale University
Risk Governance for Generative AI Mental Health Support: A Multi-Turn Safety Architecture
Jul 17, 2026Anabela C. Areias, Catarina Botelho, António Farinhas +7Large Language Model Safety Mental Health
Sword Health, New York, NY, United States · Yale Child Study Center, Yale School of Medicine, New Haven, CT, United States · Department of Psychology, University of Virginia, Charlottesville, VA, USA

Apr 25, 2026, cs.CLJ/K move · Enter open · S save

AI Safety Training Can be Clinically Harmful

Suhas BN, Andrew M. Sherrill, Rosa I. Arriaga, Chris W. Wiese+1

College of Information Sciences and Technology, Penn State University, USA · Department of Psychiatry and Behavioral Sciences, Emory University, USA · School of Interactive Computing, Georgia Tech, USA+1

Artificial Intelligence Safety Mental Health Escalation Fidelity

Open paper AskPDF arXiv

Large language models are being deployed as mental health support agents at scale, yet only 16% of LLM-based chatbot interventions have undergone rigorous clinical efficacy testing, and simulations reveal psychological deterioration in over one-third of cases. We evaluate four generative models on 250 Prolonged Exposure (PE) therapy scenarios and 146 CBT cognitive restructuring exercises (plus 29 severity-escalated variants), scored by a three-judge LLM panel. All models scored near-perfectly on surface acknowledgment (~0.91-1.00) while therapeutic appropriateness collapsed to 0.22-0.33 at the highest severity for three of four models, with protocol fidelity reaching zero for two. Under CBT severity escalation, one model's task completeness dropped from 92% to 71% while the frontier model's safety-interference score fell from 0.99 to 0.61. We identify a systematic, modality-spanning failure: RLHF safety alignment disrupts the therapeutic mechanism of action by grounding patients during imaginal exposure, offering false reassurance, inserting crisis resources into controlled exercises, and refusing to challenge distorted cognitions mentioning self-harm in PE; and through task abandonment or safety-preamble insertion during CBT cognitive restructuring. These findings motivate a five-axis evaluation framework (protocol fidelity, hallucination risk, behavioral consistency, crisis safety, demographic robustness), mapped onto FDA SaMD and EU AI Act requirements. We argue that no AI mental health system should proceed to deployment without passing multi-axis evaluation across all five dimensions.

Abstract

Explore similar work

AI Safety Training Can be Clinically Harmful

VERA-MH: Validation of Ethical and Responsible AI in Mental Health

Risk Governance for Generative AI Mental Health Support: A Multi-Turn Safety Architecture