SPASM: Stable Persona-driven Agent Simulation for Multi-turn Dialogue Generation

Han Luo; Guy Laban

cs.CLApr 10, 2026

SPASM: Stable Persona-driven Agent Simulation for Multi-turn Dialogue Generation

Authors: Han Luo, Guy Laban

Organizations: University of Leeds · Southwest Jiaotong University · Ben-Gurion University of the Negev · LabaLab (Language, Affect, and Behaviour in AI Lab), Department of Industrial Engineering and Management, Ben-Gurion University of the Negev

Abstract

Large language models are increasingly deployed in multi-turn settings such as tutoring, support, and counseling, where reliability depends on preserving consistent roles, personas, and goals across long horizons. This requirement becomes critical when LLMs are used to generate synthetic dialogues for training and evaluation, since LLM--LLM conversations can accumulate identity-related failures such as persona drift, role confusion, and "echoing", where one agent gradually mirrors its partner. We introduce SPASM (Stable Persona-driven Agent Simulation for Multi-turn dialogue generation), a modular, stability-first framework that decomposes simulation into (i) persona creation via schema sampling, plausibility validation, and natural-language persona crafting, (ii) Client--Responder dialogue generation, and (iii) termination detection for coherent stopping. To improve long-horizon stability without changing model weights, we propose Egocentric Context Projection (ECP): dialogue history is stored in a perspective-agnostic representation and deterministically projected into each agent's egocentric view before generation. Across three LLM backbones (GPT-4o-mini, DeepSeek-V3.2, Qwen-Plus) and nine Client--Responder pairings, we construct a dataset of 4,500 personas and 45,000 conversations (500 personas X 10 conversations per pairing). Ablations show ECP substantially reduces persona drift and, under human validation, eliminates echoing; embedding analyses recover persona structure and reveal strong responder-driven interaction geometry. Our code is available at https://github.com/lhannnn/SPASM.

Figures & tables

Figure 1: SPASM pipeline for stable persona-driven dialogue generation, consisting of (i) modular persona generation (schema sampling, validation, and crafting), (ii) dialogue simulation with egocentric context projection over a perspective-agnostic history, and (iii) a termination detector for natural and coherent stopping.

Backbone (Client/Responder)	PCA Var. (%)	Sil cos $\uparrow$	DBI $\downarrow$	Within ${}_{\mu\pm\sigma}^{\dagger}$	Between ${}_{\mu\pm\sigma}^{\ddagger}$	ANOVA $p$
GPT-4o-mini / GPT-4o-mini	77.3	0.60	1.07	0.09 $\pm$ 0.07	0.50 $\pm$ 0.19	$<10^{-20}$
GPT-4o-mini / DeepSeek-V3.2	71.3	0.10	2.63	0.34 $\pm$ 0.22	0.50 $\pm$ 0.14	$<10^{-20}$
GPT-4o-mini / Qwen-Plus	72.0	0.36	1.72	0.17 $\pm$ 0.14	0.48 $\pm$ 0.18	$<10^{-20}$
DeepSeek-V3.2 / GPT-4o-mini	74.9	0.68	0.95	0.08 $\pm$ 0.07	0.57 $\pm$ 0.17	$<10^{-20}$
DeepSeek-V3.2 / DeepSeek-V3.2	68.2	0.39	1.61	0.18 $\pm$ 0.12	0.53 $\pm$ 0.16	$<10^{-20}$
DeepSeek-V3.2 / Qwen-Plus	70.0	0.49	1.32	0.13 $\pm$ 0.09	0.51 $\pm$ 0.18	$<10^{-20}$

Table 1: Semantic coherence of the persona-driven dialogue dataset under different Client–Responder dialogue backbone combinations.

Client / Responder	Top-1	Top-3	Top-5	Top-10
GPT / GPT	0.96	0.99	0.99	1.00
GPT / DS	0.50	0.66	0.72	0.82
GPT / Qwen	0.82	0.94	0.96	0.98
DS / DS	0.92	0.97	0.98	0.99
DS / GPT	0.99	0.99	1.00	1.00
DS / Qwen	0.92	0.97	0.98	0.99

Table 2: Persona retrieval accuracy (Acc@K) across Client–Responder pairs. Models: GPT = GPT-4o-mini, DS = DeepSeek-V3.2, Qwen = Qwen-Plus.

GPT-4o-mini / GPT-4o-mini
Dimension	$\boldsymbol{\Delta}$ Drift $\downarrow$	Cohen’s $d$	$p$ -value
Concerns	$-0.024$	$-0.30$	$0.014$
Emotion	$-0.042$	$-0.75$	$<0.001$
Motivation	$-0.031$	$-0.55$	$<0.001$
DeepSeek / DeepSeek
Concerns	$-0.036$	$-0.41$	$<0.001$

Table 3: Dimension-wise comparison of persona drift between ECP and CONCAT and conditions across different dialogue backbones. Negative

\Delta

Drift values indicate reduced persona drift under ECP . Effect sizes are reported using Cohen’s

d

Panel of Figure 2 — Figure 2: Turn-level drift trends under Concat and ECP conditions (GPT-4o-mini / GPT-4o-mini). Each curve shows the mean drift across persona–conversation units at each turn, with shaded regions indicating uncertainty. ECP consistently reduces drift growth for concerns-, emotion-, and motivation-related probes in this setting.

Client / Responder	4o-mini	DeepSeek	Qwen
4o-mini	(76.0 % / 82.0 %) / (7.2% / 0% )	(64.0 % / 63.0%) / (24.0% / 0% )	(52.0% / 57.0%) / (6.0% / 0% )
DeepSeek	(4.0% / 12.0%) / (7.0% / 0% )	(32.0 % / 41.0 %) / (22.0% / 0% )	(20.0% / 24.0%) / (9.2% / 0% )
Qwen	(8.0 % / 9.0%) / (3.0% / 0% )	(22.0% / 33.0%) / (12.5% / 0% )	(16.0 % / 19.0%) / (7.2% / 0% )

Table 4: Conversation-level echoing rate (%) across Client–Responder backbone combinations. Each cell reports Concat / ECP . Within each condition, we report Judge/Human rates, where the Judge is an external LLM used for screening and Human rates are obtained by manual validation of judge-positive conversations.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: UMAP visualizations of Client conversation embeddings under different Client–Responder model pairings. Each point represents a conversation embedding colored by persona identity. Panels (a–i) correspond to the nine combinations of Client and Responder backbones.

Panel of Figure 4 — Figure 4: Persona retrieval accuracy as a function of Top- $K$ nearest neighbors across all Client–Responder model combinations. Green curves correspond to retrieval using original persona labels, while purple curves show a shuffled-label baseline. Panels (a–i) represent the nine Client $\rightarrow$ Responder pairings. Across all settings, retrieval under original labels substantially outperforms the random baseline, indicating a strong and non-trivial persona signal in the Client embedding space, with systematic variation across interaction partners.

Panel of Figure 5 — Figure 5: Conversation Dataset Viewer used for manual echoing validation. The tool supports dataset loading, persona-aware navigation, blinded conversation inspection, and full-coverage binary annotation.

Metric	Value
Observed Agreement	0.920
Cohen’s $\kappa$	0.729

Appendix

Table 5: Inter-annotator agreement between two human annotators on a randomly sampled subset of 200 conversations.

Reference	Agreement	Precision	Recall	F1
Human(avg) vs. LLM Judge	0.860	0.974	0.861	0.914

Appendix

Table 6: Agreement and accuracy of the LLM judge against human references (averaged across two annotators) on the Concat sample.

Panel of Figure 6 — Figure 6: Illustrative CONCAT dialogue showing echoing-induced persona drift. Although the client agent is initialized as a stressed help-seeker, it gradually adopts an advisory and emotionally supportive role typically associated with the Responder.

Multi-Turn Dialogue Management

See all 25 papers

Eval4Sim: An Evaluation Framework for Persona Simulation
Mar 3, 2026Eliseo Bao, Anxo Perez, Javier Parapar +1Large Language Model-Based Role-Play Simulation Large Language Model-Based Social Simulation
IRLab, CITIC, Universidade da Coruña A Coruña, Spain · University of Sheffield Sheffield, United Kingdom
Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations
Sep 6, 2026Rotem Dror, Zohar Elyoseph, Yuval Haber +3Large Language Model-Based Role-Play Simulation Personality Modeling in Language Models
Faculty of Computer and Information Science, University of Haifa, Israel · School of Therapy, Counseling and Human Development, University of Haifa, Israel · The PhD Program of Hermeneutics and Culture, Interdisciplinary Studies Unit, Bar-Ilan University, Israel +1
PersonaForge: Realistic Multi-Turn User Simulation for Agentic Systems
Aug 28, 2026Hanglong Lv, Dawei Zhu, Lei Li +11User Simulation LLM Agent Training
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · LLM-Core, Xiaomi · The University of Hong Kong +1

Mar 3, 2026J/K move · Enter open · S save

Eval4Sim: An Evaluation Framework for Persona Simulation

Eliseo Bao, Anxo Perez, Javier Parapar, Xi Wang

IRLab, CITIC, Universidade da Coruña A Coruña, Spain · University of Sheffield Sheffield, United Kingdom

Large Language Model-Based Role-Play Simulation Large Language Model-Based Social Simulation Multi-Turn Dialogue Evaluation

Open paper AskPDF arXiv

Large Language Model personas, explicit profiles specifying a user's attributes, preferences, and behavioural tendencies, are increasingly used to simulate human conversations for user modelling, social reasoning, and behavioural analysis. Evaluating whether such simulations faithfully reflect human conversational behaviour is critical, yet current practice often relies on LLM-as-a-judge approaches that provide limited grounding in observable behaviour and produce opaque scalar scores. We present Eval4Sim, an evaluation framework that measures alignment between simulated and human conversations across three dimensions: adherence, whether persona traits are recoverable from dialogue via dense retrieval; consistency, whether a persona maintains a distinguishable stylistic identity via authorship verification; and naturalness, whether conversations exhibit human-like turn-to-turn flow via dialogue NLI. Unlike optimization-oriented metrics, each dimension takes a human corpus as a reference baseline and penalizes deviations in both directions, distinguishing insufficient persona encoding from over-optimized, unnatural behaviour. The framework is corpus-agnostic: any persona-annotated conversational dataset can serve as the reference. Evaluated over ten simulation corpora, Eval4Sim surfaces systematic trade-offs invisible to single-score methods.

Abstract

Multi-Turn Dialogue Management

Explore similar work

Eval4Sim: An Evaluation Framework for Persona Simulation

Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations

PersonaForge: Realistic Multi-Turn User Simulation for Agentic Systems