Comparing Human Role-Players and LLM-Simulated Clients in Online Counselling Training: An Analysis of Counselling Patterns

Eric Rudolph; Philipp Steigerwald; Jens Albrecht

cs.AIOct 29, 2026

Comparing Human Role-Players and LLM-Simulated Clients in Online Counselling Training: An Analysis of Counselling Patterns

Authors: Eric Rudolph, Philipp Steigerwald, Jens Albrecht

Organizations: Technische Hochschule Nürnberg Georg Simon Ohm, Nürnberg, Germany

Abstract

This study investigates the capabilities of Large Language Models to simulate counselling clients in educational role-plays in comparison to human role-players. Initially, we recorded role-playing sessions, where novice counsellors interacted with human peers acting as clients, followed by role-plays between humans and clients simulated by Mistrals Mixtral 8x7b using 4-bit quantization. These interactions were analysed with a counselling communication pattern system at sentence level. We investigated two key questions: (1) to what extent LLM-generated responses replicate authentic conversational dynamics and (2) whether counsellors' communication behaviour differs when interacting with human versus LLM-simulated clients. The findings highlight both similarities and differences in the application of counselling patterns across scenarios, showing the potential of LLM-based role-playing exercises to enhance counselling competencies and to identify areas for further refinement in virtual client simulations.

Figures & tables

Statistic	Human-Human	Human-LLM
Number of Conversations	64	74
Number of Messages ( $N$ )	2411	2892
Total Sentences	5383	7040
Avg. Sentences per Msg	2.23	2.43
Total Words	59805	82587
Avg. Words per Msg	24.81	28.56

Table 1: Comparison of Human-Human and Human-LLM Counselling Messages

Figure 1: Pattern Distribution for Client and Counselor Patterns across all conversations

Figure 2: Frequency of Predicted Counsellor Labels along Conversation Progression

Figure 3: Frequency of Predicted Client Labels along conversation progression

Figure 4: t-SNE Projection of Client Sentences

Figure 5: t-SNE Projection of Counsellor Sentences

Large Language Model-Based Role-Play Simulation

Large language model-based role-play simulation creates interactive agents that portray people, characters, or users for training, testing, and research.

See all 111 papers

PsyCLIENT: Client Simulation via Conversational Trajectory Modeling for Trainee Practice and Model Evaluation in Mental Health Counseling
Jan 12, 2026Huachuan Qiu, Zhaoming Chen, Yuqian Chen +3Large Language Model-Based Role-Play Simulation User Simulation
Westlake University · The University of Utah
Therapy as an NLP Task: Comparing LLMs and Human Peers Behaviors in CBT Sessions
Sep 3, 2024Zainab Iftikhar, Sean Ransom, Amy Xiao +2Mental Health Counseling Mental Health Conversational Agents
Brown University, USA · Department of Psychiatry, Louisiana State University Health Sciences, USA · Alpert Medical School of Brown University, Brown University, USA
Structured Prompting and Automated Evaluation in Fixed Synthetic Japanese-Language Counseling Dialogues
Jun 28, 2025Keita Kiuchi, Yoshikazu Fujimoto, Hideyuki Goto +5Mental Health Counseling LLM-as-a-Judge
Japan National Institute of Occupational Safety and Health, Kanagawa, Japan · Kaze To Taiyo, Tokyo, Japan · Saga Occupational Health Association, Saga, Japan +5

Jan 12, 2026J/K move · Enter open · S save

PsyCLIENT: Client Simulation via Conversational Trajectory Modeling for Trainee Practice and Model Evaluation in Mental Health Counseling

Huachuan Qiu, Zhaoming Chen, Yuqian Chen, Yuan Xie+2

Westlake University · The University of Utah

Large Language Model-Based Role-Play Simulation User Simulation Persona-Conditioned Generation

Open paper AskPDF arXiv

LLM-based client simulation provides a scalable approach to novice counselor training, counseling-dialogue synthesis, and interactive evaluation of automated counseling systems. However, existing approaches are limited by insufficient profile diversity, weak behavioral grounding, and the lack of open Chinese-language resources for simulated counseling clients. We propose PsyCLIENT, a framework that conditions simulated-client responses on client profiles, dialogue histories, and conversational trajectories specifying target behaviors and content constraints at each client turn. We also construct PsyCLIENT-CP, a dataset of 120 Chinese client profiles spanning 60 counseling topics. Evaluations involving 24 professional counselors show that PsyCLIENT receives higher ratings of perceived authenticity and training utility than the comparison methods. In a separate source-identification study, PsyCLIENT dialogues were more frequently misclassified as human-client interactions than those generated by the baselines. These results suggest that conversational trajectory modeling can transform static client profiles into behavior-guided, dynamically unfolding counseling interactions.

Abstract

Large Language Model-Based Role-Play Simulation

Explore similar work

PsyCLIENT: Client Simulation via Conversational Trajectory Modeling for Trainee Practice and Model Evaluation in Mental Health Counseling

Therapy as an NLP Task: Comparing LLMs and Human Peers Behaviors in CBT Sessions

Structured Prompting and Automated Evaluation in Fixed Synthetic Japanese-Language Counseling Dialogues