cs.CLApr 10, 2026

SPASM: Stable Persona-driven Agent Simulation for Multi-turn Dialogue Generation

Authors: Han Luo, Guy Laban

Organizations: University of Leeds · Southwest Jiaotong University · Ben-Gurion University of the Negev · LabaLab (Language, Affect, and Behaviour in AI Lab), Department of Industrial Engineering and Management, Ben-Gurion University of the Negev

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language models are increasingly deployed in multi-turn settings such as tutoring, support, and counseling, where reliability depends on preserving consistent roles, personas, and goals across long horizons. This requirement becomes critical when LLMs are used to generate synthetic dialogues for training and evaluation, since LLM--LLM conversations can accumulate identity-related failures such as persona drift, role confusion, and "echoing", where one agent gradually mirrors its partner. We introduce SPASM (Stable Persona-driven Agent Simulation for Multi-turn dialogue generation), a modular, stability-first framework that decomposes simulation into (i) persona creation via schema sampling, plausibility validation, and natural-language persona crafting, (ii) Client--Responder dialogue generation, and (iii) termination detection for coherent stopping. To improve long-horizon stability without changing model weights, we propose Egocentric Context Projection (ECP): dialogue history is stored in a perspective-agnostic representation and deterministically projected into each agent's egocentric view before generation. Across three LLM backbones (GPT-4o-mini, DeepSeek-V3.2, Qwen-Plus) and nine Client--Responder pairings, we construct a dataset of 4,500 personas and 45,000 conversations (500 personas X 10 conversations per pairing). Ablations show ECP substantially reduces persona drift and, under human validation, eliminates echoing; embedding analyses recover persona structure and reveal strong responder-driven interaction geometry. Our code is available at https://github.com/lhannnn/SPASM.

Figures & tables

Figure 1: SPASM pipeline for stable persona-driven dialogue generation, consisting of (i) modular persona generation (schema sampling, validation, and crafting), (ii) dialogue simulation with egocentric context projection over a perspective-agnostic history, and (iii) a termination detector for natural and coherent stopping.
Backbone (Client/Responder)PCA Var. (%)Sil cos ↑\uparrowDBI ↓\downarrowWithin μ±σ†{}_{\mu\pm\sigma}^{\dagger}Between μ±σ‡{}_{\mu\pm\sigma}^{\ddagger}ANOVA pp
GPT-4o-mini / GPT-4o-mini77.30.601.070.09 ±\pm 0.070.50 ±\pm 0.19<10−20<10^{-20}
GPT-4o-mini / DeepSeek-V3.271.30.102.630.34 ±\pm 0.220.50 ±\pm 0.14<10−20<10^{-20}
GPT-4o-mini / Qwen-Plus72.00.361.720.17 ±\pm 0.140.48 ±\pm 0.18<10−20<10^{-20}
DeepSeek-V3.2 / GPT-4o-mini74.90.680.950.08 ±\pm 0.070.57 ±\pm 0.17<10−20<10^{-20}
DeepSeek-V3.2 / DeepSeek-V3.268.20.391.610.18 ±\pm 0.120.53 ±\pm 0.16<10−20<10^{-20}
DeepSeek-V3.2 / Qwen-Plus70.00.491.320.13 ±\pm 0.090.51 ±\pm 0.18<10−20<10^{-20}
Table 1: Semantic coherence of the persona-driven dialogue dataset under different Client–Responder dialogue backbone combinations.
Client / ResponderTop-1Top-3Top-5Top-10
GPT / GPT0.960.990.991.00
GPT / DS0.500.660.720.82
GPT / Qwen0.820.940.960.98
DS / DS0.920.970.980.99
DS / GPT0.990.991.001.00
DS / Qwen0.920.970.980.99
Table 2: Persona retrieval accuracy (Acc@K) across Client–Responder pairs. Models: GPT = GPT-4o-mini, DS = DeepSeek-V3.2, Qwen = Qwen-Plus.
DimensionΔ\boldsymbol{\Delta} Drift ↓\downarrowCohen’s ddpp -value
GPT-4o-mini / GPT-4o-mini
Concerns−0.024-0.024−0.30-0.300.0140.014
Emotion−0.042-0.042−0.75-0.75<0.001<0.001
Motivation−0.031-0.031−0.55-0.55<0.001<0.001
DeepSeek / DeepSeek
Concerns−0.036-0.036−0.41-0.41<0.001<0.001
Table 3: Dimension-wise comparison of persona drift between ECP and CONCAT and conditions across different dialogue backbones. Negative Δ\Delta Drift values indicate reduced persona drift under ECP . Effect sizes are reported using Cohen’s dd .
Figure 2: Turn-level drift trends under Concat and ECP conditions (GPT-4o-mini / GPT-4o-mini). Each curve shows the mean drift across persona–conversation units at each turn, with shaded regions indicating uncertainty. ECP consistently reduces drift growth for concerns-, emotion-, and motivation-related probes in this setting.
Client / Responder4o-miniDeepSeekQwen
4o-mini(76.0 % / 82.0 %) / (7.2% / 0% )(64.0 % / 63.0%) / (24.0% / 0% )(52.0% / 57.0%) / (6.0% / 0% )
DeepSeek(4.0% / 12.0%) / (7.0% / 0% )(32.0 % / 41.0 %) / (22.0% / 0% )(20.0% / 24.0%) / (9.2% / 0% )
Qwen(8.0 % / 9.0%) / (3.0% / 0% )(22.0% / 33.0%) / (12.5% / 0% )(16.0 % / 19.0%) / (7.2% / 0% )
Table 4: Conversation-level echoing rate (%) across Client–Responder backbone combinations. Each cell reports Concat / ECP . Within each condition, we report Judge/Human rates, where the Judge is an external LLM used for screening and Human rates are obtained by manual validation of judge-positive conversations.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: UMAP visualizations of Client conversation embeddings under different Client–Responder model pairings. Each point represents a conversation embedding colored by persona identity. Panels (a–i) correspond to the nine combinations of Client and Responder backbones.
Figure 4: Persona retrieval accuracy as a function of Top- KK nearest neighbors across all Client–Responder model combinations. Green curves correspond to retrieval using original persona labels, while purple curves show a shuffled-label baseline. Panels (a–i) represent the nine Client →\rightarrow Responder pairings. Across all settings, retrieval under original labels substantially outperforms the random baseline, indicating a strong and non-trivial persona signal in the Client embedding space, with systematic variation across interaction partners.
Figure 5: Conversation Dataset Viewer used for manual echoing validation. The tool supports dataset loading, persona-aware navigation, blinded conversation inspection, and full-coverage binary annotation.
MetricValue
Observed Agreement0.920
Cohen’s κ\kappa0.729
Appendix
Table 5: Inter-annotator agreement between two human annotators on a randomly sampled subset of 200 conversations.
ReferenceAgreementPrecisionRecallF1
Human(avg) vs. LLM Judge0.8600.9740.8610.914
Appendix
Table 6: Agreement and accuracy of the LLM judge against human references (averaged across two annotators) on the Concat sample.
Figure 6: Illustrative CONCAT dialogue showing echoing-induced persona drift. Although the client agent is initialized as a stressed help-seeker, it gradually adopts an advisory and emotionally supportive role typically associated with the Responder.
More in this topic

Multi-Turn Dialogue Management

See all 25 papers

Explore similar work

CardsList
Mar 3, 2026

Eval4Sim: An Evaluation Framework for Persona Simulation

Large Language Model personas, explicit profiles specifying a user's attributes, preferences, and behavioural tendencies, are increasingly used to simulate human conversations for user modelling, social reasoning, and behavioural analysis. Evaluating whether such simulations faithfully reflect human conversational behaviour is critical, yet current practice often relies on LLM-as-a-judge approaches that provide limited grounding in observable behaviour and produce opaque scalar scores. We present Eval4Sim, an evaluation framework that measures alignment between simulated and human conversations across three dimensions: adherence, whether persona traits are recoverable from dialogue via dense retrieval; consistency, whether a persona maintains a distinguishable stylistic identity via authorship verification; and naturalness, whether conversations exhibit human-like turn-to-turn flow via dialogue NLI. Unlike optimization-oriented metrics, each dimension takes a human corpus as a reference baseline and penalizes deviations in both directions, distinguishing insufficient persona encoding from over-optimized, unnatural behaviour. The framework is corpus-agnostic: any persona-annotated conversational dataset can serve as the reference. Evaluated over ten simulation corpora, Eval4Sim surfaces systematic trade-offs invisible to single-score methods.

Eliseo Bao, Anxo Perez, Javier Parapar +1

IRLab, CITIC, Universidade da Coruña A Coruña, Spain · University of Sheffield Sheffield, United Kingdom

Large Language Model-Based Role-Play SimulationLarge Language Model-Based Social SimulationMulti-Turn Dialogue Evaluation
AskPDF
Sep 6, 2026

Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations

Existing approaches to persona simulation with Large Language Models (LLMs) mostly rely on shallow character descriptions that fail to sustain coherent character behavior across extended interactions. We introduce Deep Persona, a psychologically grounded, three-layered architecture that organizes personas into hierarchical levels of observable expression, latent beliefs, and core motivational drives, for constructing highly convincing role-playing agents. Governed by the principles of scripted determinism and bounded agency, the architecture restricts the model to a reactive engine guided by a structured internal script. We further propose a reference-free evaluation framework that benchmarks dialogue naturalness against empirical human distributions using established psychological clinical instruments and adversarial stress-tests. Empirical evaluation reveals that while LLMs achieve high pragmatic fluency, they exhibit systematic limitations in emotional expression and joint attention. In addition, we present a case study of two Deep Personas and evaluate them using the proposed framework, demonstrating that structured personas can produce interactions that more closely align with human conversational behavior.

Rotem Dror, Zohar Elyoseph, Yuval Haber +3

Faculty of Computer and Information Science, University of Haifa, Israel · School of Therapy, Counseling and Human Development, University of Haifa, Israel · The PhD Program of Hermeneutics and Culture, Interdisciplinary Studies Unit, Bar-Ilan University, Israel +1

Large Language Model-Based Role-Play SimulationPersonality Modeling in Language ModelsReference-Free EvaluationEmotion Representation in Language Models
AskPDF
Aug 28, 2026

PersonaForge: Realistic Multi-Turn User Simulation for Agentic Systems

Large language models are increasingly used as agentic workflow executors, yet existing training data and benchmarks largely assume informationally complete, single-turn queries. Our analysis of 16K real-world sessions shows that 75.9% of interactions are multi-turn, revealing a substantial gap between how users interact with agents and how such systems are trained and evaluated. We introduce PersonaForge, a user simulation framework for synthesizing realistic multi-turn user--agent interactions. PersonaForge combines a four-dimensional persona space, SOUL-driven behavioral control calibrated to real-user statistics, and Reverse Deep Construction grounded in authentic seed queries. Using PersonaForge, we construct a 6.3K-record training dataset and PersonaForge-Bench, a manually annotated 138-task benchmark spanning over 20 professional domains with four-dimensional scoring. Experiments on Qwen3.5-27B show that PersonaForge training improves the composite score by +4.1%, with gains across all four dimensions and the largest improvements in Task Completion (+6.0%) and Response Quality (+6.8%). Further analyses show that PersonaForge-trained agents use fewer turns and tool calls, suggesting improved interaction efficiency, while ablations confirm the contribution of SOUL components and adaptive simulation. Together, PersonaForge and PersonaForge-Bench establish a foundation for training and evaluating agents under realistic multi-turn user interaction.

Hanglong Lv, Dawei Zhu, Lei Li +11

State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · LLM-Core, Xiaomi · The University of Hong Kong +1

User SimulationLLM Agent TrainingAI Agent BenchmarksMulti-Turn Dialogue EvaluationMulti-Turn Dialogue ManagementLarge Language Model-Based Social Simulation
AskPDF