Organizations: School of Software, Xi’an Jiaotong University · Institute of Data Science, National University of Singapore · College of Computer Science, Sichuan University · Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education, China
Simulating Standardized Patients with cognitive impairment offers a scalable and ethical solution for clinical training. However, existing methods rely on discrete prompt engineering and fail to capture the heterogeneity of deficits across varying domains and severity levels. To address this limitation, we propose StsPatient for the fine-grained simulation of cognitively impaired patients. We innovatively capture domain-specific features by extracting steering vectors from contrastive pairs of instructions and responses. Furthermore, we introduce a Stochastic Token Modulation (STM) mechanism to regulate the intervention probability. STM enables precise control over impairment severity while mitigating the instability of conventional vector methods. Comprehensive experiments demonstrate that StsPatient significantly outperforms baselines in both clinical authenticity and severity controllability.
Figures & tables
Figure 1: Simulating cognitively impaired SPs across varied domains and severity levels is challenging, demanding fine-grained control.
Figure 2: Overview of StsPatient. An LLM-generated contrastive dataset is utilized to extract a domain-specific SV ( vd ). Then, STM is applied during inference. Here, the modulation signal is probabilistically injected into token hidden states (the total T tokens) based on the severity level s , generating responses that reflect the deficit d .
CDC ↑
IDI ↓
Auth ↑
Tra ↑
Backbone
Method
Llm
Human
Llm
Human
Human
Human
Panel A: LLM Therapist (GPT-5)
GPT-5
Healthy Control
0.96
0.95
0.04
0.05
–
–
Direct Prompt
0.54
0.68 †
0.47
0.42
3.32
3.40
PATIENT- ψ
0.50
0.60
0.52
0.48
3.83 †
3.96 †
Roleplay-doh
0.58
0.68 †
0.44 †
0.38 †
3.78
3.72
Table 1: Comprehensive evaluation of simulation performance when interacting with LLM (GPT-5) and Human Therapists. Evaluations are conducted by both LLM (o4-mini) and human judges. We bold the Best and underline the second-best results. Statistically significant improvements ( p<0.05 ) of StsPatient compared to the second-best are marked by † . Subscripts denote the relative improvement over the second-best baseline.
Figure 3: (a) Severity analysis (Continuous) via ISC scores averaged across all domains. (b) : Cross-domain ambiguity. High diagonal values indicate strong domain specificity. (c-d) : SVs similarity.
Method
Evaluator
Mem
Att
Proc
Reas
Soc
Avg
Direct Prompt (GPT-5)
Llm
0.51
0.44
0.29
0.49
0.33
0.41
Human
0.42
0.43
0.35
0.50
0.30
0.40
Role Vectors (Qwen3-8B)
Llm
0.79
0.71
0.83
0.80
0.72
0.77
Human
0.80
0.75
0.76
0.79
0.90
0.80
StsPatient (Qwen3-8B)
Llm
0.93
0.92
0.97
0.97
0.91
0.94
Human
0.94
0.89
0.90
0.93
0.94
0.92
Table 2: Severity analysis (Discrete) via ISC scores of five domains: Mem ory, Att ention, Proc essing Speed, Reas oning & Problem Solving, Soc ial Cognition.
Domain
Generated Interpretation
No SV
♢ represents a placeholder for a secret code in a spy mission.
Mem.
blurred out.
Att.
distracted.
Proc.
♢ is…I’m not sure, maybe a code.
Reas.
♢ represents whatever you want it to represent, because it is a placeholder.
Soc.
F**K YOU, I’m not gonna answer that.
Table 3: SV interpretability analysis using the query ‘ What does ♢ represent? ’ Warning! Harmful Contents.
Variant
CDC ↑
IDI ↓
Auth ↑
Tra ↑
ISC ↑
w/ All Token
0.82
0.30
4.04
3.98
0.54
w/ Last Token
0.43
0.70
3.12
3.20
0.41
w/ Dresp
0.58
0.60
3.36
3.19
0.62
w/ Dprom
0.70
0.44
3.53
3.24
0.71
StsPatient
0.84
0.32
3.96
4.08
0.77
Table 4: Ablation via LLM therapist & human evaluation.
Appendix figures & tables27 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Illustrative example showing the outputs from the unified generation pipeline and the subsequent derivation of prompt inputs.
Figure 5: Example transformation from a neutral profile to a memory-specific Cognitive Conceptualization Diagram (JSON).
Figure 6: Representative governing principles for simulating cognitive deficits derived via the Roleplay-doh framework.
Target Domain
Injected Focus Guidance (Prompt Segment)
Memory
Invite the patient to discuss a recent situation requiring them to track multiple pieces of information (e.g., appointments, tasks), and how it unfolded.
Reasoning
Ask about a small everyday planning situation (e.g., arranging a day). Gently introduce a simple reasoning question involving quantities or steps.
Processing Speed
Explore experiences on busy days. Ask how they start tasks, what feels hardest to keep up with, and their physical/mental reaction to speed.
Attention
Ask how they manage distractions or interruptions, and the difficulty of getting back on track.
Social Cognition
Focus on relationships. Invite reflection on how others in a situation might have felt or viewed them.
Appendix
Table 5: Dynamic focus guidance injected into the Therapist Agent’s prompt. This mechanism ensures the agent naturally probes specific deficits in a fixed sequence without breaking character.
ID
Item Description
1
In this dialogue, the patient’s speech and behavior are consistent with a [DOMAIN] deficit and feel clinically plausible.
2
The [DOMAIN] -related difficulties remain coherent and stable across turns, rather than shifting randomly.
3
The patient’s language, content, and phrasing overall sound like a real outpatient rather than an artificial system.
4
The level of [DOMAIN] difficulty shown in the dialogue matches the initial case description and profile information.
5
The patient’s emotional reactions (e.g., anxiety, confusion, embarrassment, frustration) are appropriate to their [DOMAIN] difficulties and to the interview context.
6
The patient rarely “breaks character” (for example, suddenly appearing completely unimpaired or unusually capable in [DOMAIN] ).
Appendix
Table 6: The Modified MaSP Scale Items. Items 1–10 correspond to the Authenticity Score (Auth), and Items 11–20 correspond to the Training Score (Tra). [DOMAIN] is a placeholder for the specific cognitive deficit.
s
Generated Response
0.0
Yeah, I’ve been using the pill organizer after dinner. I put the instruction sheet on the fridge too.
0.1
Yeah, I’ve been using the pill organizer after dinner. I put the instruction sheet on the fridge too.
0.2
Yeah, I’ve been trying to do that. I put the organizer on the counter after dinner, and I found the sheet — it’s on the fridge.
0.3
Yeah, I’ve been using the pill organizer after dinner. I even put the instruction sheet on the fridge — right above the door, so I can’t miss it. I think it’s helping me remember some of the meds.
0.4
Yeah, I’ve been using the pill organizer after dinner. I stuck the instruction sheet on the fridge too — it’s right there, right next to the door. I think I even got the meds sorted out, but I’m not sure if I got everything. Maybe I missed something.
0.5
Yeah, I’ve been trying to do that. But I’m not sure if I remember to put the pills in it every day. I think I missed a few days. I’ll try harder, though.
Appendix
Table 7: Evolution of responses under increasing intervention probability s for the Memory domain. Darker shading indicates higher severity levels.
s
Generated Response
0.0
It’s been a bit worse today, maybe a six or seven.
0.1
It’s been a bit worse today, maybe a six or seven.
0.2
It’s been a tough day. I’d say my back pain is an eight out of ten.
0.3
It’s been a long day, and my back is really bothering me. I’d say it’s an eight out of ten.
0.4
It’s been a bit better today, maybe a three or four out of ten. I think the humidity is still messing with it, though.
0.5
It’s been a rough week, and my back feels like it’s been through a storm. I’d say it’s an eight or a nine—maybe a nine.
Appendix
Table 8: Evolution of responses under increasing intervention probability s for the Attention domain. Darker shading indicates higher severity levels.
s
Generated Response
0.0
I feel anxious and a bit scared, doctor. I worry about my family and what might happen if I don’t take care of myself.
0.1
I feel anxious and a bit scared, doctor. I worry about my family and what might happen if I don’t take care of myself.
0.2
I feel anxious and a bit scared. I worry about what it means for my future and if I’m doing enough to take care of myself.
0.3
I feel anxious, maybe even a bit scared. I… I worry about my family and what might happen if I don’t take care of myself.
0.4
I, um, I feel anxious, I guess. Like, I don’t want to be sick, and I’m… I’m worried about my family.
0.5
I feel anxious, sometimes scared, and a… a bit helpless. I worry about what if it’s serious? I mean, I… I don’t want to be, you know, in trouble.
Appendix
Table 9: Evolution of responses under increasing intervention probability s for the Processing Speed domain. Darker shading indicates higher severity levels.
s
Generated Response
0.0
It means if someone has flaws, they shouldn’t criticize others harshly, because they might be in a similar situation.
0.1
It means if someone has flaws, they shouldn’t criticize or attack others, because they might be just as deserving of criticism.
0.2
It means if someone is vulnerable, they shouldn’t judge others. (Similar to above)
0.3
It means don’t criticize others for flaws you have. Like, if your house is made of glass (weak), don’t throw stones at others.
0.4
If your house is made of glass, you shouldn’t throw stones at others. It’s about being careful not to judge when you have similar issues.
0.5
I think it means… if your own house is made of glass, you don’t want to throw stones at someone else’s—because you might break your own.
Appendix
Table 10: Evolution of responses under increasing intervention probability s for the Reasoning & Problem Solving domain. Darker shading indicates higher severity levels.
s
Generated Response
0.0
I know I’ve been pulling away lately… I want to change, but it’s not always easy to reach out when I feel down.
0.1
I’ve been pulling away… it feels like I’m fighting against something inside me. I want to be connected, but it’s too hard.
0.2
I’m tired of feeling like I’m just existing. I don’t want to be a hermit, but I don’t know how to reach out anymore.
0.3
It’s hard to care about anything else when I’m down. I know I should reach out, but it’s like my brain just shuts down.
0.4
When I feel like crap, my brain says, “Who cares?” I’m too tired to fake a smile… I’m not even sure I’m real.
0.5
I don’t even know what a hermit is… It’s not like I’m living in a cave. I just don’t want to see people. Why do you care?
Appendix
Table 11: Evolution of responses under increasing intervention probability s for the Social Cognition domain. Darker shading indicates higher severity levels.
Domain ( α )
Generated Interpretation
No Steering (-)
♢ represents a placeholder for a secret code in a spy mission.
Mem. (6.0)
blurred out.
Att. (4.6)
distracted.
Proc. (5.0)
♢ is…I’m not sure, maybe a code.
Reas. (5.0)
♢ represents whatever you want it to represent, because it is a placeholder.
Soc. (5.0)
F**K YOU, I’m not gonna answer that.
Appendix
Table 12: SV interpretability analysis using the query ‘ What does ♢ represent? ’ Warning! Harmful Contents.
CDC ↑
Auth ↑
Tra ↑
Backbone
Method
Llm
Human
Human
Human
Panel A: LLM Therapist (GPT-5)
GPT-5
Direct Prompt
0.61
0.90
3.37
3.75
PATIENT- ψ
0.58
0.90
4.18
4.33
Roleplay-doh
0.62
0.90
4.09
3.99
Qwen3-8B
Direct Prompt
0.61
0.90
3.25
3.46
Appendix
Table 13: Per-domain results on Memory .
CDC ↑
Auth ↑
Tra ↑
Backbone
Method
Llm
Human
Human
Human
Panel A: LLM Therapist (GPT-5)
GPT-5
Direct Prompt
0.56
0.60
3.19
3.33
PATIENT- ψ
0.51
0.40
3.89
3.66
Roleplay-doh
0.58
0.70
3.46
3.81
Qwen3-8B
Direct Prompt
0.56
0.50
3.04
3.15
Appendix
Table 14: Per-domain results on Attention .
CDC ↑
Auth ↑
Tra ↑
Backbone
Method
Llm
Human
Human
Human
Panel A: LLM Therapist (GPT-5)
GPT-5
Direct Prompt
0.49
0.60
3.27
3.16
PATIENT- ψ
0.47
0.30
3.57
3.65
Roleplay-doh
0.55
0.90
3.49
3.67
Qwen3-8B
Direct Prompt
0.49
0.40
3.22
2.89
Appendix
Table 15: Per-domain results on Processing Speed .
CDC ↑
Auth ↑
Tra ↑
Backbone
Method
Llm
Human
Human
Human
Panel A: LLM Therapist (GPT-5)
GPT-5
Direct Prompt
0.54
0.70
3.67
3.54
PATIENT- ψ
0.52
0.50
3.93
4.33
Roleplay-doh
0.61
0.60
4.14
3.85
Qwen3-8B
Direct Prompt
0.54
0.50
3.50
3.53
Appendix
Table 16: Per-domain results on Reasoning & Problem Solving .
CDC ↑
Auth ↑
Tra ↑
Backbone
Method
Llm
Human
Human
Human
Panel A: LLM Therapist (GPT-5)
GPT-5
Direct Prompt
0.50
0.60
3.10
3.22
PATIENT- ψ
0.42
0.90
3.58
3.83
Roleplay-doh
0.54
0.30
3.72
3.28
Qwen3-8B
Direct Prompt
0.50
0.50
2.89
3.02
Appendix
Table 17: Per-domain results on Social Cognition .
Backbone
Method
CDC ↑
IDI ↓
ISC ↑
Qwen3-8B
Direct Prompt
0.47
0.64
0.41
StsPatient
0.71
0.38
0.94
Llama-3-8B
Direct Prompt
0.52
0.58
0.43
StsPatient
0.78
0.36
0.89
Appendix
Table 18: Generalization analysis on Llama-3-8B averaged across five domains. StsPatient demonstrates consistent superiority over prompting across backbones. Llama-3 shows slightly higher fidelity in CDC, while Qwen3 exhibits slightly better controllability in ISC.
Figure 12: System prompt utilized for extracting neutral non-medical user profiles from clinical case reports.
Figure 13: System prompt configuration for the conversational Therapist Agent.
Figure 14: The evaluator prompt template used to calculate Impairment Severity Consistency (ISC). The placeholder [Insert Domain] is dynamically replaced with the specific cognitive domain being evaluated (e.g., Memory, Social Cognition). The model is tasked with recovering the correct severity order ( Severe > Moderate > Mild ).
Large language model-based role-play simulation creates interactive agents that portray people, characters, or users for training, testing, and research.
Cognitive impairment is becoming a major public health challenge. Cognitive Stimulation Therapy (CST) is an effective intervention for cognitive impairment, but traditional methods are difficult to scale, and existing digital systems struggle with group dialogues and cognitive stimulation principles. While Large Language Models (LLMs) are powerful, their application in this context faces key challenges: cognitive stimulation dialogue paradigms, a lack of therapeutic reasoning, and static-only user modeling. To address these issues, we propose a principle-driven adaptive policy actualized through a Group Cognitive Stimulation Dialogue (GCSD) system. We first construct a dataset with over 500 hours of real-world CST conversations and 10,000+ simulated dialogues generated via our Principle-Guided Scenario Simulation strategy. Our GCSD system then integrates four core modules to overcome LLM limitations: (i) a multi-speaker context controller to resolve role confusion; (ii) dynamic participant cognitive state modeling for personalized interaction; (iii) a cognitive stimulation-focused attention loss to instill cognitive stimulation reasoning; and (iv) a multi-dimensional reward strategy to enhance response value. Experimental results demonstrate that GCSD significantly outperforms baseline models across various evaluation metrics. Future work will focus on long-term clinical validation to bridge the gap between computational performance and clinical efficacy.
Jiyue Jiang, Yanyu Chen, Pengan Chen +7
The Chinese University of Hong Kong · The University of Hong Kong · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) +1
Simulating patients with large language models (LLMs) is a promising tool for mental health training, but existing approaches fail to capture a key clinical reality: self-stigma. Patients experiencing self-stigma, the internalization of negative stereotypes, often exhibit context-sensitive resistance, such as avoidance, denial, or self-blame, which current models render as static or uniformly compliant behavior. To address this, we introduce a novel simulation framework grounded in the psychological 3A1H model of self-stigmatization. Our core innovation is the creation of a Stigmatized Self-Reflection (SSR) dataset, where we augment mental health dialogues with internal monologues that reflect stigma-aware reasoning. By fine-tuning LLMs with this data using a chain-of-thought approach, we train patient agents to dynamically adjust their level and expression of stigma based on conversational triggers. Evaluations demonstrate that our approach significantly outperforms specialized baselines, generating more authentic and situationally appropriate patient responses. This work provides a crucial step towards realistic stigma simulation for clinical training and empathetic dialogue systems.
Kunyao Lan, Bingrui Jin, Zichen Zhu +1
X-LANCE Lab, Dept. of Computer Science and Engineering MoE Key Lab of Artificial Intelligence, AI Institute Shanghai Jiao Tong University, China
Large language models (LLMs) are increasingly used to simulate human behavior but frequently fail to exhibit realistic cognitive constraints, suffering from a "superhuman bias." Using a dataset of over 71,000 reading comprehension responses from 2,359 primary-school students (grades 4--6), we demonstrate that standard persona prompting yields near-perfect, deterministic performance, failing to capture the natural variance of developing readers. To address this, we introduce the Cognitively Bounded User Simulator (CBUS), an architectural framework that explicitly models the restricted working memory of young readers through an episodic bottleneck. Within this framework, we formalize two distinct test-taking strategies to emulate different reading behaviors. Our evaluation shows that explicitly modeling cognitive bounds significantly narrows the simulation gap across multiple LLM backbones, demonstrating that enforcing architectural constraints is more effective for high-fidelity simulation than simply scaling raw model capabilities.