cs.CLNov 8, 2026

The Average Patient: Do Simulated Psychiatric Patients Answer Like Real People?

Authors: Weixiang Sun, Zehong Wang, Kaiwen Shi, Hong Huang, Colby Nelson, Yiyang Li, Yanfang Ye

Organizations: University of Notre Dame · Simon Fraser University

Abstract

Language models are used as simulated psychiatric patients to train clinicians and evaluate AI therapists, and are judged by plausibility or total scores. We give a framework for simulated respondents: an information budget bounds how much simulated patients can differ, and an average-patient hypothesis, that a model answers each item at the level typical of the named condition and severity, implies little variation, a full profile of the condition's typical symptoms and totals at or above the top of the stated severity band. We examine these implications at matched totals on item-level data from eight human samples, including a clinical and an instructed-feigning sample. Across 17 models, simulated patients vary 1.5 to 11 times less than real respondents with the same total and on most questionnaires endorse nearly every symptom at a middling level; on the PHQ-9 their symptoms covary less like real ones. People, honest or feigning, answer "not at all" about as often as genuine patients with the same totals, and more often than every model. Single answers are highly concentrated, and prompt changes, published personas, larger models and reasoning reduce the pattern without removing it. In interviews, patients told a severity are rated as having nearly every symptom, so an assessment agent's recognition of absent symptoms goes untested, and scored against the stated severity the agent places a quarter of them in their band, against about half of patients given a real symptom profile. Having the model word such a profile restores half to two thirds of the variation.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Apr 28, 2026

PSI-Bench: Interpretable and Clinically Meaningful Evaluation of Depression Patient Simulators

Patient simulators are gaining traction in mental health training by providing scalable exposure to complex and sensitive patient interactions. Simulating depressed patients is challenging, as safety constraints and high patient variability complicate simulations and underscore the need for simulators that capture diverse and realistic patient behaviors. However, existing evaluations heavily rely on LLM-judges with poorly specified prompts and do not assess behavioral diversity. We introduce PSI-Bench, an automatic evaluation framework that provides interpretable, clinically meaningful diagnostics of depression patient simulator behavior across turn-, dialogue-, and population-level dimensions. Using PSI-Bench, we benchmark seven LLMs across two simulator frameworks and find that simulators produce overly long, lexically diverse responses, show reduced variability, and move through therapeutic stages and toward positive valence too quickly. We also show that the simulation framework has a larger impact on fidelity than the model scale. Results from a human study demonstrate that our benchmark is strongly aligned with judgments of mental health professionals. Our work reveals key limitations of current depression patient simulators and provides an interpretable, extensible benchmark to guide future simulator design and evaluation.
Aug 13, 2026

PatientAct: Theory-Grounded Mental Health Client Simulation

LLM-based simulated clients are increasingly used to train novice counselors, evaluate LLM therapists, and generate synthetic data. However, current simulators produce overly cooperative clients that disclose too readily, accept therapeutic reframes without resistance, and resolve core issues within a single session. We trace these issues to profiles that lack causal depth and behavioral mechanisms that treat all content as equally accessible. We present PatientAct, a framework for client simulation grounded in established clinical theories. Our profiles integrate the 5Ps clinical case formulation, providing causal depth without tying the design to any single therapeutic modality. During simulation, profiles include a dynamic memory layer in which items carry trust thresholds (e.g., symptoms are available early, whereas formative memories require a sustained therapeutic alliance). At each turn, the client's emotional reaction and behavior are modeled before generating a response. If the therapist approaches gated content, PatientAct expresses resistance in terms of quantity, content, and style rather than defaulting to cooperation or a single resistance pattern. We evaluate our framework on 40 clinical situations and demonstrate that it generates diverse profiles with high clinical plausibility. Moreover, PatientAct significantly outperforms the baselines, yielding substantial gains in resistance quality and behavioral realism. Our code and data are publicly available via github.com/Sahandfer/PatientHub.
May 13, 2026

Patients With Personality: Realistic Patient Simulation through Controlled Diversity and Selective Disclosure

Realistic patient simulation is a key requirement for developing and testing clinical LLMs at scale without costly user studies. Existing simulators lack realism and controllability: they overshare information and miss the variability of real patient behavior. Here, we introduce PatientsWithPersonality (PWP), a patient simulation framework that generates realistic yet diverse patient responses through explicit personality parametrization over a latent patient state. Grounded in the six-dimensional HEXACO personality model, it gives fine-grained control over conversational style, cooperativeness, and information disclosure. We evaluate PWP in the first direct comparison of different patient simulators under a shared protocol, anchored in transcripts of real physician-patient encounters. Clinicians judge PWP conversations as realistic at least as often as the recorded encounters themselves and far more often than prior simulators, and almost never flag them as "too informative". Our simulator also reproduces the disclosure pacing of real patients, and its HEXACO-conditioned personality traits can be recovered by clinicians and an autorater. PWP further shows a substantially wider behavioral range than the closest baseline. In a downstream benchmark, LLMs interviewing patients simulated by PWP perform closest to how they perform on the real encounters, whereas prior simulators make the task easier. PWP thus provides a realistic, steerable, and diverse patient population for improving and benchmarking clinical LLMs.