cs.CLNov 8, 2026

The Average Patient: Do Simulated Psychiatric Patients Answer Like Real People?

Authors: Weixiang Sun, Zehong Wang, Kaiwen Shi, Hong Huang, Colby Nelson, Yiyang Li, Yanfang Ye

Organizations: University of Notre Dame · Simon Fraser University

Abstract

Language models are used as simulated psychiatric patients to train clinicians and evaluate AI therapists, and are judged by plausibility or total scores. We give a framework for simulated respondents: an information budget bounds how much simulated patients can differ, and an average-patient hypothesis, that a model answers each item at the level typical of the named condition and severity, implies little variation, a full profile of the condition's typical symptoms and totals at or above the top of the stated severity band. We examine these implications at matched totals on item-level data from eight human samples, including a clinical and an instructed-feigning sample. Across 17 models, simulated patients vary 1.5 to 11 times less than real respondents with the same total and on most questionnaires endorse nearly every symptom at a middling level; on the PHQ-9 their symptoms covary less like real ones. People, honest or feigning, answer "not at all" about as often as genuine patients with the same totals, and more often than every model. Single answers are highly concentrated, and prompt changes, published personas, larger models and reasoning reduce the pattern without removing it. In interviews, patients told a severity are rated as having nearly every symptom, so an assessment agent's recognition of absent symptoms goes untested, and scored against the stated severity the agent places a quarter of them in their band, against about half of patients given a real symptom profile. Having the model word such a profile restores half to two thirds of the variation.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. PSI-Bench: Interpretable and Clinically Meaningful Evaluation of Depression Patient Simulators

    Apr 28, 2026Nguyen Khoi Hoang, Shuhaib Mehri, Tse-An Hsu +4Benchmark DesignClinical Language Model Evaluation

  2. PatientAct: Theory-Grounded Mental Health Client Simulation

    Aug 13, 2026Sahand Sabour, TszYam NG, Yaqian Chen +3Mental Health CounselingUser Simulation

  3. Patients With Personality: Realistic Patient Simulation through Controlled Diversity and Selective Disclosure

    May 13, 2026Moritz Schlager, Friederike Jungmann, Samuel Schmidgall +15Personality Modeling in Language ModelsPersona-Conditioned Generation