Synthetic or Authentic? Building Mental Patient Simulators from Longitudinal Evidence

Baihan Li; Bingrui Jin; Kunyao Lan; Ming Wang; Mengyue Wu

cs.CLMar 24, 2026

Synthetic or Authentic? Building Mental Patient Simulators from Longitudinal Evidence

Authors: Baihan Li, Bingrui Jin, Kunyao Lan, Ming Wang, Mengyue Wu

Organizations: SJTU Paris Elite Institute of Technology, Shanghai Jiao Tong University, China · X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · School of Computer Science and Engineering, Northeastern University, China · School of Computing and Information Systems, Singapore Management University, Singapore

Abstract

Patient simulation is essential for developing and evaluating mental health dialogue systems. As most existing approaches rely on snapshot-style prompts with limited profile information, homogeneous behaviors and incoherent disease progression in multi-turn interactions have become key chellenges. In this work, we propose DEPROFILE, a data-grounded patient simulation framework that constructs unified, multi-source patient profiles by integrating demographic attributes, standardized clinical symptoms, counseling dialogues, and longitudinal life-event histories from real-world data. We further introduce a Chain-of-Change agent to transform noisy longitudinal records into structured, temporally grounded memory representations for simulation. Experiments across multiple large language model (LLM) backbones show that with more comprehensive profile constructed by DEPROFILE, the dialogue realism, behavioral diversity, and event richness have consistently improved and exceed state-of-the-art baselines, highlighting the importance of grounding patient simulation in verifiable longitudinal evidence.

Figures & tables

Figure 1: The Deprofile Framework Overview

Figure 2: Illustrative workflow of the LLM-based Chain of Change (CoC) Agent.

Profile	MentalBERT		G-Eval
Profile	Sim $\uparrow$	Q-Cent $\uparrow$	Pers	Evt	Sym
basic	0.931	0.054	4.71	4.60	3.99
basic+T	0.944	0.059	4.63	4.84	3.89
basic+S	0.936	0.061	4.94	4.80	4.84
basic+ST	0.941	0.068	4.80	4.90	4.71
full w/o T	0.941	0.066	4.97	4.57	4.86

Table 1: Automatic evaluation across profile configurations. Sim=Similarity; Q-Cent=Q-Centroid; Pers=Persona Faithfulness; Evt=Event Richness; Sym=Symptom Consistency.

Backbone Model	Realism $\uparrow$			Diversity(Q-centroid) $\uparrow$			Event Richness $\uparrow$
Backbone Model	P- $\Psi$	Eeyore	Deprofile	P- $\Psi$	Eeyore	Deprofile	P- $\Psi$	Eeyore	Deprofile
Llama-3.1-8B-Instruct	0.888	0.896	0.936	0.085	0.085	0.085	3.74	3.99	4.24
Qwen-3-4B-Instruct	0.893	0.898	0.913	0.024	0.020	0.039	4.76	4.83	4.84
GPT-4o-mini	0.927	0.925	0.940	0.026	0.029	0.050	2.95	3.08	4.39
DeepSeek-V3.2-Exp	0.923	0.922	0.939	0.046	0.044	0.063	4.76	3.77	4.85
Average	0.908	0.910	0.932	0.045	0.044	0.059	4.05	3.92	4.58

Table 2: Main results comparing Deprofile with Patient-

\Psi

( P-

\Psi

) and Eeyore across different LLM backbones. Metrics include dialogue realism (MentalBERT similarity), response diversity (Q-Centroid), and event richness.

Model	Realism	Persona Faith	Event Richness	Symptom Const	Overall	Win Rate	Adjacent%
basic	3.02 $\pm$ 0.48 †	3.55 $\pm$ 0.60	2.55 $\pm$ 0.63 †	3.35 $\pm$ 0.77 ∗	3.09 $\pm$ 0.45 †	80.0%	60.0%
basic+S	3.20 $\pm$ 0.62 ∗	3.90 $\pm$ 0.77	2.82 $\pm$ 0.66 ∗	3.75 $\pm$ 0.85	3.39 $\pm$ 0.37 ∗	63.3%	43.3%
P- $\Psi$	2.22 $\pm$ 0.81 †	2.54 $\pm$ 0.58 †	3.33 $\pm$ 1.04 ∗	2.68 $\pm$ 0.97 †	2.73 $\pm$ 0.55 †	86.7%	30.0%
Eeyore	2.16 $\pm$ 0.94 †	2.67 $\pm$ 0.93 †	3.06 $\pm$ 0.86 ∗	2.27 $\pm$ 0.90 †	2.58 $\pm$ 0.62 †	90.0%	46.7%
Deprofile	3.67 $\pm$ 0.59	3.75 $\pm$ 0.75	3.49 $\pm$ 0.85	3.86 $\pm$ 0.65	3.73 $\pm$ 0.41	—	76.7%

Table 3: Human evaluation results. Mean

\pm

std across profiles. Faith: Persona Faithfulness; Const: Symptom Consistency. Win Rate: proportion of profiles where Deprofile is rated higher than the row model. Adjacent%: proportion of profile

\times

model pairs where the three raters’ scores differ by at most 1 point. Wilcoxon signed-rank test (Deprofile vs. that baseline): †

p<0.001

, ∗

p<0.05

. Significance shown only for baselines.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Attribute	Value
Demographics	24-year-old male; single; employed
Personality	Openness: 5; Conscientiousness: 4; Extraversion: 3; Agreeableness: 6; Neuroticism: 6
Longitudinal Alignment	Matched social media timeline (similarity = 0.96; symptom similarity = 0.67)
Positive Symptoms	Suicidal ideation; hopelessness; low self-worth; fatigue; sleep disturbance; appetite dysregulation; psychomotor agitation/retardation; social withdrawal; indecisiveness
Negative Symptoms	No suicide attempt; no severe work impairment; no persistent sleep disorder
Clinical Summary	Persistent depressed mood, anhedonia, low energy, and cognitive difficulties over the past month, indicating a moderate-to-severe depressive condition

Appendix

Table 4: An example patient profile constructed by the Deprofile framework.

Category	Topic / Label	Question (Translated)
Profile	Intro	How old are you?
	Bio	What is your gender?
	Profession	Are you currently working or studying? What is your major or field?
	Marital Status	What is your current relationship status? Are you married or in a relationship?
Big Five	Extraversion (Talkativeness)	Tell me, are you generally a talkative person, or do you prefer to stay quiet?
	Agreeableness (Fault Finding)	When interacting with others, do you tend to find fault or pick at their mistakes?

Appendix

Table 5: The structured question list used for dialogue generation and assessment.

Model Variant	Realism (95% CI)	Diversity (95% CI)	Event	Persona	Symptom
basic	[0.9252, 0.9369]	[0.0495, 0.0585]	4.60 $\pm$ 0.51	4.71 $\pm$ 0.64	3.99 $\pm$ 0.94
basic+T	[0.9396, 0.9484]	[0.0515, 0.0661]	4.84 $\pm$ 0.40	4.63 $\pm$ 0.65	3.89 $\pm$ 0.87
basic+S	[0.9304, 0.9414]	[0.0556, 0.0661]	4.80 $\pm$ 0.43	4.94 $\pm$ 0.31	4.84 $\pm$ 0.42
basic+ST	[0.9363, 0.9448]	[0.0577, 0.0781]	4.90 $\pm$ 0.32	4.80 $\pm$ 0.51	4.71 $\pm$ 0.55
full w/o T	[0.9364, 0.9461]	[0.0597, 0.0715]	4.57 $\pm$ 0.55	4.97 $\pm$ 0.23	4.86 $\pm$ 0.38
Deprofile	[0.9337, 0.9445]	[0.0570, 0.0687]	4.86 $\pm$ 0.37	4.87 $\pm$ 0.43	4.79 $\pm$ 0.45

Appendix

Table 6: Ablation study under the DeepSeek-V3.2-Exp backbone. We report 95% confidence intervals (CI) for semantic realism and inter-patient diversity, and mean

\pm

standard deviation for G-Eval dimensions.

\uparrow

indicates higher is better.

Realism (95% CI)
Metric / Method	Llama-3.1-8B	Qwen-3-4B	GPT-4o-mini	DeepSeek-V3.2
Patient- $\Psi$	[0.8777, 0.8977]	[0.8828, 0.9016]	[0.9206, 0.9328]	[0.9147, 0.9310]
Eeyore	[0.8869, 0.9040]	[0.8901, 0.9053]	[0.9175, 0.9317]	[0.9148, 0.9285]
Deprofile	[0.9308, 0.9415]	[0.9057, 0.9189]	[0.9345, 0.9444]	[0.9337, 0.9445]
Diversity (95% CI)
Patient- $\Psi$	[0.0792, 0.0903]	[0.0227, 0.0247]	[0.0207, 0.0321]	[0.0424, 0.0513]

Appendix

Table 7: Transposed view of Table 2 . Rows correspond to evaluation metrics and methods, while columns correspond to backbone models. All values are identical to those reported in Table 2 .

Large Language Model-Based Role-Play Simulation

Large language model-based role-play simulation creates interactive agents that portray people, characters, or users for training, testing, and research.

See all 106 papers

PatientAct: Theory-Grounded Mental Health Client Simulation
Aug 13, 2026Sahand Sabour, TszYam NG, Yaqian Chen +3Mental Health Counseling User Simulation
The CoAI Group, DCST, Institute for Artificial Intelligence, Tsinghua University, Beijing, China · Department of Psychology, Beijing Normal University, Beijing, China · Counseling and Psychological Development Guidance Center, Tsinghua University, Beijing, China
Synthesis and Evaluation of Long-term History-aware Medical Dialogue
May 19, 2026Hebin Hu, Renke Dai, Ah-Hwee Tan +1Healthcare AI Agent Benchmarks
South-Central Minzu University Wuhan, China · Singapore Management University Singapore, Singapore
PSI-Bench: Interpretable and Clinically Meaningful Evaluation of Depression Patient Simulators
Apr 28, 2026Nguyen Khoi Hoang, Shuhaib Mehri, Tse-An Hsu +4Benchmark Design Clinical Language Model Evaluation
University of Illinois Urbana-Champaign · VinUniversity

Aug 13, 2026J/K move · Enter open · S save

PatientAct: Theory-Grounded Mental Health Client Simulation

Sahand Sabour, TszYam NG, Yaqian Chen, Guanqun Bi+2

The CoAI Group, DCST, Institute for Artificial Intelligence, Tsinghua University, Beijing, China · Department of Psychology, Beijing Normal University, Beijing, China · Counseling and Psychological Development Guidance Center, Tsinghua University, Beijing, China

Mental Health Counseling User Simulation

Open paper AskPDF arXiv

LLM-based simulated clients are increasingly used to train novice counselors, evaluate LLM therapists, and generate synthetic data. However, current simulators produce overly cooperative clients that disclose too readily, accept therapeutic reframes without resistance, and resolve core issues within a single session. We trace these issues to profiles that lack causal depth and behavioral mechanisms that treat all content as equally accessible. We present PatientAct, a framework for client simulation grounded in established clinical theories. Our profiles integrate the 5Ps clinical case formulation, providing causal depth without tying the design to any single therapeutic modality. During simulation, profiles include a dynamic memory layer in which items carry trust thresholds (e.g., symptoms are available early, whereas formative memories require a sustained therapeutic alliance). At each turn, the client's emotional reaction and behavior are modeled before generating a response. If the therapist approaches gated content, PatientAct expresses resistance in terms of quantity, content, and style rather than defaulting to cooperation or a single resistance pattern. We evaluate our framework on 40 clinical situations and demonstrate that it generates diverse profiles with high clinical plausibility. Moreover, PatientAct significantly outperforms the baselines, yielding substantial gains in resistance quality and behavioral realism. Our code and data are publicly available via github.com/Sahandfer/PatientHub.

Abstract

Large Language Model-Based Role-Play Simulation

Explore similar work

PatientAct: Theory-Grounded Mental Health Client Simulation

Synthesis and Evaluation of Long-term History-aware Medical Dialogue

PSI-Bench: Interpretable and Clinically Meaningful Evaluation of Depression Patient Simulators