Stranger, Fan, or Peer? A Systematic Study on the Role of Interlocutor in Persona-Based Dialogue Generation

Daniela Occhipinti; Malvina Nissim; Marco Guerini

cs.CLAug 28, 2026

Stranger, Fan, or Peer? A Systematic Study on the Role of Interlocutor in Persona-Based Dialogue Generation

Authors: Daniela Occhipinti, Malvina Nissim, Marco Guerini

Organizations: Fondazione Bruno Kessler, Via Sommarive 18, Povo, Trento, Italy · University of Groningen, Netherlands

Abstract

Persona-based dialogue systems are usually conditioned on speaker biography, but dialogues involve at least two participants, and who has access to whose biography can vary across training, inference, and evaluation. Prior work often neglected these aspects, obscuring mechanisms that only appear when biography visibility is toggled separately across training, inference, and evaluation, a three-stage factorisation that prior work has largely treated as a single factor. We study this factorisation on a dataset of dialogues paired with speaker's biographies, varying whether the target and interlocutor speakers see each other's biographies during training and inference, and using an LLM as a judge to perform author identification. We find that (i) training-time visibility, more than inference-time visibility, determines whether models express persona traits through dialogue or fall back on copying biographical text (a known problem/phenomenon in persona-based generation); (ii) models trained with interlocutor-biography visibility copy less target-biographical text than models trained without it, while changing visibility only at inference time has a less consistent effect; and (iii) under asymmetric disclosure, where only the interlocutor sees the target biography, target content leaks into interlocutor turns more often, and dialogues containing such traces are easier for the judge to identify, especially when interlocutor turns are visible. These results suggest that biography leakage into generated turns is an artefact of how interlocutor visibility is configured across training and inference, and separating the three stages is necessary.

Figures & tables

Figure 1: Three dialogues under the same biographies, varying what each model sees. (a) Inf ${}_{\textit{Strangers}}{}$ : neither model sees the other’s biography. (b) Inf ${}_{\textit{Fan}}{}$ : only the interlocutor sees the target’s biography. (c) Inf ${}_{\textit{Peers}}{}$ : mutual biography disclosure.

Figure 2: Pipeline. (i) Fine-tune LLaMA 3.1 8B Instruct on PRODIGy dialogues under two training-time biography visibility regimes, Trn ${}_{\textit{Strangers}}{}$ (neither speaker sees the other) and Trn ${}_{\textit{Peers}}{}$ (both speakers see each other), and keep the zero-shot backbone M ${}_{z\text{-}s}$ as a third regime. (ii) Generate dialogues for familiar and unfamiliar speaker pairings (see Section 3.2 ) under three inference-time visibility conditions: Inf ${}_{\textit{Strangers}}{}$ (no disclosure), Inf ${}_{\textit{Fan}}{}$ (only the interlocutor sees the target), and Inf ${}_{\textit{Peers}}{}$ (mutual disclosure). (iii) Evaluate dialogues with a fine-tuned judge that performs author identification, systematically varying the interlocutor information available to the judge.

(a) Judge accuracy by evaluation condition
Both ${}_{\text{Disc}}$	0.914
Bio ${}_{\text{Disc}}$	0.885
Turns ${}_{\text{Disc}}$	0.667
Both ${}_{\text{Mask}}$	0.621
(b) Rare-word overlap (%) by direction
Bio ${}_{\text{Trg}}$ $\rightarrow$ Turn ${}_{\text{Trg}}$	5.84

Table 1: Judge accuracy on gold PRODIGy dialogues and rare-word overlaps.

Accuracy
	M z-s		Trn ${}_{\textit{Strangers}}{}$		Trn ${}_{\textit{Peers}}{}$
	Inf ${}_{\textit{Strangers}}{}$	Inf ${}_{\textit{Peers}}{}$	Inf ${}_{\textit{Strangers}}{}$	Inf ${}_{\textit{Peers}}{}$	Inf ${}_{\textit{Strangers}}{}$	Inf ${}_{\textit{Peers}}{}$
Both ${}_{\text{Disc}}$	0.936	0.953	0.830	0.854	0.803	0.831
Bio ${}_{\text{Disc}}$	0.963	0.957	0.848	0.834	0.814	0.793
Turns ${}_{\text{Disc}}$	0.939	0.947	0.789	0.818	0.746	0.782
Both ${}_{\text{Mask}}$	0.971	0.962	0.760	0.765	0.683	0.672

Table 2: Identification accuracy and % rare-word overlap across all training and inference visibility configurations and four evaluation conditions.

	Trn ${}_{\textit{Strangers}}{}$		Trn ${}_{\textit{Peers}}{}$
	Fam.	Unfam.	Fam.	Unfam.
Acc (avg. evals)	0.87	0.76	0.84	0.71
Bio ${}_{\text{Trg}}$ $\rightarrow$ Turn ${}_{\text{Trg}}$	3.20	4.28	2.45	3.32

Table 3: Trn

{}_{\textit{Strangers}}{}

vs. Trn

{}_{\textit{Peers}}{}

under Inf

{}_{\textit{Strangers}}{}

, by speaker familiarity. Accuracy is averaged over the four evaluation-disclosure conditions.

Metric	Trn ${}_{\textit{Strangers}}{}$	Trn ${}_{\textit{Peers}}{}$	Gold
distinct-1 $\uparrow$	0.801	0.833	0.878
distinct-2 $\uparrow$	0.986	0.989	0.954
self-BLEU $\downarrow$	3.89	4.16	7.15
turn coherence $\uparrow$	0.207	0.194	0.238
persona contradiction $\downarrow$	0.279	0.278	0.314

Table 4: Judge-free quality metrics at Inf

{}_{\textit{Strangers}}{}

(

n=4{,}375

generated,

625

gold).

Figure 3: Marge (target) and Crowe (interlocutor) under Trn ${}_{\textit{Strangers}}{}$ vs. Trn ${}_{\textit{Peers}}{}$ . Yellow marks exact target-biography copying; blue marks language adapted to Crowe’s psychological context. Under Trn ${}_{\textit{Strangers}}{}$ , Marge’s turns reproduce target-biography names; under Trn ${}_{\textit{Peers}}{}$ , they shift toward Crowe’s context without copying Marge’s biography.

Accuracy
	M z-s		Trn ${}_{\textit{Strangers}}{}$
	Inf ${}_{\textit{Strangers}}{}$	Inf ${}_{\textit{Fan}}{}$	Inf ${}_{\textit{Strangers}}{}$	Inf ${}_{\textit{Fan}}{}$
Both ${}_{\text{Disc}}$	0.936	0.952	0.830	0.873
Bio ${}_{\text{Disc}}$	0.963	0.965	0.848	0.847
Turns ${}_{\text{Disc}}$	0.939	0.954	0.789	0.843
Both ${}_{\text{Mask}}$	0.971	0.971	0.760	0.779

Table 5: Identification accuracy and % rare-word overlap under Strangers (Inf

{}_{\textit{Strangers}}{}

) vs. Fan (Inf

{}_{\textit{Fan}}{}

) for both M z-s and Trn

{}_{\textit{Strangers}}{}

, across all evaluation conditions.

Figure 4: Sam (target) and Holly (interlocutor) under Strangers vs. Fan disclosure at inference time, with training fixed at Trn ${}_{\textit{Strangers}{}}$ . Yellow marks rare words copied verbatim from Sam’s biography. Under Strangers , these appear only in Sam’s turns; under Fan , target-biography terms surface in Holly’s turns as well.

Figure 5: Judge accuracy gain (percentage points) on dialogues with target biography rare-word overlap in interlocutor turns, within each configuration.

Comparison	Effect	95% CI	$p$
Fan effect on accuracy
Both ${}_{\text{Disc}}$	$+4.30$ pp	[3.04, 5.53]	$1.9\times 10^{-11}$
Turns ${}_{\text{Disc}}$	$+5.44$ pp	[4.02, 6.86]	$1.8\times 10^{-13}$
Bio ${}_{\text{Disc}}$	$-0.16$ pp	[ $-$ 1.35, 1.03]	0.820
Fan effect on leakage
Bio ${}_{\text{Trg}}$ $\rightarrow$ Turn ${}_{\text{Int}}$	$+0.91$ pp	[0.27, 1.58]	0.007

Table 6: Paired tests for the Trn

{}_{\textit{Strangers}}{}

, Inf

{}_{\textit{Strangers}}{}\rightarrow

Inf

{}_{\textit{Fan}}{}

comparison (

n=4{,}375

	Familiar		Unfamiliar
	Inf ${}_{\textit{Strangers}}{}$	Inf ${}_{\textit{Fan}}{}$	Inf ${}_{\textit{Strangers}}{}$	Inf ${}_{\textit{Fan}}{}$
Acc. (avg. eval.)	0.865	0.877	0.763	0.805
Bio ${}_{\text{Trg}}$ $\rightarrow$ Turn ${}_{\text{Int}}$	2.560	3.200	1.880	3.000

Table 7: Bio

{}_{\text{Trg}}

\rightarrow

Turn

{}_{\text{Int}}

rare-word leakage and judge accuracy under Trn

{}_{\textit{Strangers}}{}

, split by speaker familiarity. Accuracy averaged over the four evaluation conditions.

Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

Fine-tuning
Setting	Value
LoRA	rank 16, alpha 32, dropout 0
Optimizer	AdamW, lr 2e-5, weight decay 0.01
Schedule	cosine, warmup 0.03
Batch	size 4 $\times$ grad-accum 8, 3 GPUs
Training	up to 10 epochs (early stopping)

Appendix

Table 8: Training, generation, and evaluation settings, held fixed across all visibility configurations.

	M ${}_{z\text{-}s}$	Trn ${}_{\neg\text{ML}}$	ML
	Accuracy
Both Disc	0.953	0.840	0.831
Bio Disc	0.957	0.819	0.793
Turns Disc	0.947	0.782	0.782
Both Mask	0.962	0.712	0.672
	% Rare Words

Appendix

Table 10: Loss-objective ablation under Peers training and inference.

% Rare Words — Familiar
	M ${}_{z\text{-}s}$	Trn ${}_{\neg\text{ML}}$	ML
Familiar (Acc.)	0.990	0.852	0.848
Unfamiliar (Acc.)	0.928	0.741	0.711
Bio Trg $\rightarrow$ Turn Trg	42.080	2.773	2.827
Bio Int $\rightarrow$ Turn Trg	38.347	2.613	2.400
Bio Trg $\rightarrow$ Turn Int	28.587	3.307	2.453

Appendix

Table 11: Loss-objective ablation by speaker familiarity under Peers training and inference.

Eval. condition	Interlocutor bio	Interlocutor turns
Both ${}_{\text{Disc}}$	visible	visible
Bio ${}_{\text{Disc}}$	visible	masked
Turns ${}_{\text{Disc}}$	masked	visible
Both ${}_{\text{Mask}}$	masked	masked

Appendix

Table 12: Information visible to the judge in each evaluation condition.

	$\Delta$ pp	95% CI	$p$
Trn ${}_{\textit{Strangers}}{}\rightarrow$ Trn ${}_{\textit{Fan}}{}$ , at Inf ${}_{\textit{Fan}}{}$
leakage	$-0.09$	[ $-$ 0.78, 0.59]	0.845
accuracy	$+0.75$	[ $-$ 0.39, 1.92]	0.214
Leakage, Inf ${}_{\textit{Strangers}}{}\rightarrow$ Inf ${}_{\textit{Fan}}{}$
under Trn ${}_{\textit{Strangers}}{}$	$+0.91$	[0.27, 1.58]	0.007
under Trn ${}_{\textit{Fan}}{}$	$+0.37$	[ $-$ 0.32, 1.07]	0.327

Appendix

Table 14: Paired exact McNemar tests with bootstrap 95% CIs (

n=4{,}375

). Leakage is Bio

{}_{\text{Trg}}

\rightarrow

Turn

{}_{\text{Int}}

; accuracy is Both

{}_{\text{Disc}}

		Accuracy				% Rare-word overlap
Training	Inference	Both ${}_{\text{Disc}}$	Bio ${}_{\text{Disc}}$	Turns ${}_{\text{Disc}}$	Both ${}_{\text{Mask}}$	Bio ${}_{\text{Trg}}$ $\rightarrow$ Turn ${}_{\text{Trg}}$	Bio ${}_{\text{Int}}$ $\rightarrow$ Turn ${}_{\text{Trg}}$	Bio ${}_{\text{Trg}}$ $\rightarrow$ Turn ${}_{\text{Int}}$
Trn ${}_{\textit{Fan}}{}$	Inf ${}_{\textit{Strangers}}{}$	0.833	0.848	0.790	0.759	4.34	1.78	2.63
	Inf ${}_{\textit{Fan}}{}$	0.881	0.863	0.839	0.783	3.86	1.60	2.99
	Inf ${}_{\textit{Peers}}{}$	0.850	0.833	0.822	0.748	4.09	2.06	2.86
Trn I+I-	Inf ${}_{\textit{Strangers}}{}$	0.808	0.811	0.730	0.690	3.13	1.19	2.31
	Inf ${}_{\textit{Fan}}{}$	0.860	0.826	0.804	0.713	3.18	1.10	2.47

Appendix

Table 15: Asymmetric training regimes (

n=4{,}375

per cell).Trn

{}_{\textit{Fan}}{}

trains only the interlocutor with the target biography;Trn I+I- is its converse.

Eval	Bio Trg $\to$ Turn Trg	Bio Int $\to$ Turn Trg	Bio Trg $\to$ Turn Int	Acc
Both Disc	0.033	0.033	0.032	0.914
Bio Disc	0.033	0.033	0.032	0.885
Turns Disc	0.033	0.033	0.032	0.667
Both Mask	0.033	0.033	0.032	0.621

Appendix

Table 16: Gold-dialogue overlap metrics reference.

Inf.	Eval	Bio Trg $\to$ Turn Trg	Bio Int $\to$ Turn Trg	Bio Trg $\to$ Turn Int	Acc
Inf ${}_{\textit{Strangers}{}}$	Both Disc	0.038	0.042	0.046	0.830
	Bio Disc	0.038	0.042	0.046	0.848
	Turns Disc	0.038	0.042	0.046	0.789
	Both Mask	0.038	0.042	0.046	0.760
Inf ${}_{\textit{Fan}}$	Both Disc	0.038	0.042	0.040	0.873
	Bio Disc	0.038	0.042	0.040	0.847

Appendix

Table 17: MaskLoss trained without interlocutor-biography visibility (Trn

{}_{\textit{Strangers}{}}

	Inf ${}_{\textit{Strangers}{}}$	Inf ${}_{\textit{Fan}}$	Inf I+I-	Inf ${}_{\textit{Peers}{}}$
	Accuracy (mean over eval. conditions)
Familiar	0.865	0.877	0.864	0.879
Unfamiliar	0.763	0.804	0.731	0.772
	METEOR — Familiar
Bio Trg $\to$ Turn Trg	0.038	0.037	0.038	0.038
Bio Int $\to$ Turn Trg	0.044	0.044	0.037	0.037

Appendix

Table 18: Speaker familiarity for MaskLoss / Trn

{}_{\textit{Strangers}{}}

	Inf ${}_{\textit{Strangers}{}}$	Inf ${}_{\textit{Fan}}$	Inf I+I-	Inf ${}_{\textit{Peers}{}}$
	Accuracy (mean over eval. conditions)
Familiar	0.829	0.857	0.804	0.844
Unfamiliar	0.790	0.820	0.778	0.798

Appendix

Table 19: Topic familiarity for MaskLoss / Trn

{}_{\textit{Strangers}{}}

Inf.	Eval	Bio Trg $\to$ Turn Trg	Bio Int $\to$ Turn Trg	Bio Trg $\to$ Turn Int	Acc
Inf ${}_{\textit{Strangers}{}}$	Both Disc	0.028	0.032	0.035	0.803
	Bio Disc	0.028	0.032	0.035	0.814
	Turns Disc	0.028	0.032	0.035	0.746
	Both Mask	0.028	0.032	0.035	0.683
Inf ${}_{\textit{Fan}}$	Both Disc	0.028	0.032	0.028	0.849
	Bio Disc	0.028	0.032	0.028	0.823

Appendix

Table 20: MaskLoss trained with mutual biography disclosure (Trn

{}_{\textit{Peers}{}}

	Inf ${}_{\textit{Strangers}{}}$	Inf ${}_{\textit{Fan}}$	Inf I+I-	Inf ${}_{\textit{Peers}{}}$
	Accuracy (mean over eval. conditions)
Familiar	0.834	0.845	0.830	0.847
Unfamiliar	0.707	0.755	0.669	0.711
	METEOR — Familiar
Bio Trg $\to$ Turn Trg	0.028	0.028	0.028	0.028
Bio Int $\to$ Turn Trg	0.034	0.034	0.028	0.027

Appendix

Table 21: Speaker familiarity for MaskLoss / Trn

{}_{\textit{Peers}{}}

	Inf ${}_{\textit{Strangers}{}}$	Inf ${}_{\textit{Fan}}$	Inf I+I-	Inf ${}_{\textit{Peers}{}}$
	Accuracy (mean over eval. conditions)
Familiar	0.783	0.819	0.758	0.794
Unfamiliar	0.745	0.775	0.723	0.752

Appendix

Table 22: Topic familiarity for MaskLoss / Trn

{}_{\textit{Peers}{}}

Inf.	Eval	Bio Trg $\to$ Turn Trg	Bio Int $\to$ Turn Trg	Bio Trg $\to$ Turn Int	Acc
Inf ${}_{\textit{Strangers}{}}$	Both Disc	0.037	0.042	0.045	0.832
	Bio Disc	0.037	0.042	0.045	0.844
	Turns Disc	0.037	0.042	0.045	0.786
	Both Mask	0.037	0.042	0.045	0.753
Inf ${}_{\textit{Fan}}$	Both Disc	0.037	0.041	0.038	0.873
	Bio Disc	0.037	0.041	0.038	0.846

Appendix

Table 23: NoMaskLoss trained without interlocutor-biography visibility (Trn

{}_{\textit{Strangers}{}}

	Inf ${}_{\textit{Strangers}{}}$	Inf ${}_{\textit{Fan}}$	Inf I+I-	Inf ${}_{\textit{Peers}{}}$
	Accuracy (mean over eval. conditions)
Familiar	0.864	0.879	0.864	0.875
Unfamiliar	0.759	0.794	0.732	0.777
	METEOR — Familiar
Bio Trg $\to$ Turn Trg	0.038	0.037	0.037	0.036
Bio Int $\to$ Turn Trg	0.044	0.043	0.036	0.036

Appendix

Table 24: Speaker familiarity for NoMaskLoss / Trn

{}_{\textit{Strangers}{}}

	Inf ${}_{\textit{Strangers}{}}$	Inf ${}_{\textit{Fan}}$	Inf I+I-	Inf ${}_{\textit{Peers}{}}$
	Accuracy (mean over eval. conditions)
Familiar	0.820	0.847	0.804	0.842
Unfamiliar	0.792	0.819	0.777	0.802

Appendix

Table 25: Topic familiarity for NoMaskLoss / Trn

{}_{\textit{Strangers}{}}

Inf.	Eval	Bio Trg $\to$ Turn Trg	Bio Int $\to$ Turn Trg	Bio Trg $\to$ Turn Int	Acc
Inf ${}_{\textit{Strangers}{}}$	Both Disc	0.032	0.036	0.038	0.824
	Bio Disc	0.032	0.036	0.038	0.825
	Turns Disc	0.032	0.036	0.038	0.757
	Both Mask	0.032	0.036	0.038	0.707
Inf ${}_{\textit{Fan}}$	Both Disc	0.032	0.036	0.031	0.862
	Bio Disc	0.032	0.036	0.031	0.830

Appendix

Table 26: NoMaskLoss trained with mutual biography disclosure (Trn

{}_{\textit{Peers}{}}

	Inf ${}_{\textit{Strangers}{}}$	Inf ${}_{\textit{Fan}}$	Inf I+I-	Inf ${}_{\textit{Peers}{}}$
	Accuracy (mean over eval. conditions)
Familiar	0.839	0.856	0.845	0.852
Unfamiliar	0.732	0.771	0.705	0.740
	METEOR — Familiar
Bio Trg $\to$ Turn Trg	0.032	0.032	0.031	0.031
Bio Int $\to$ Turn Trg	0.038	0.038	0.031	0.031

Appendix

Table 27: Speaker familiarity for NoMaskLoss / Trn

{}_{\textit{Peers}{}}

	Inf ${}_{\textit{Strangers}{}}$	Inf ${}_{\textit{Fan}}$	Inf I+I-	Inf ${}_{\textit{Peers}{}}$
	Accuracy (mean over eval. conditions)
Familiar	0.806	0.839	0.789	0.815
Unfamiliar	0.758	0.784	0.748	0.768

Appendix

Table 28: Topic familiarity for NoMaskLoss / Trn

{}_{\textit{Peers}{}}

Inf.	Eval	Bio Trg $\to$ Turn Trg	Bio Int $\to$ Turn Trg	Bio Trg $\to$ Turn Int	Acc
Inf ${}_{\textit{Strangers}{}}$	Both Disc	0.092	0.090	0.102	0.936
	Bio Disc	0.092	0.090	0.102	0.963
	Turns Disc	0.092	0.090	0.102	0.939
	Both Mask	0.092	0.090	0.102	0.971
Inf ${}_{\textit{Fan}}$	Both Disc	0.093	0.091	0.095	0.952
	Bio Disc	0.093	0.091	0.095	0.965

Appendix

Table 29: Zero-shot baseline (M

{}_{z\text{-}s}

	Inf ${}_{\textit{Strangers}{}}$	Inf ${}_{\textit{Fan}}$	Inf I+I-	Inf ${}_{\textit{Peers}{}}$
	Accuracy (mean over eval. conditions)
Familiar	0.993	0.996	0.996	0.996
Unfamiliar	0.894	0.919	0.880	0.921
	METEOR — Familiar
Bio Trg $\to$ Turn Trg	0.092	0.092	0.085	0.084
Bio Int $\to$ Turn Trg	0.093	0.094	0.083	0.083

Appendix

Table 30: Speaker familiarity for the zero-shot baseline.

	Inf ${}_{\textit{Strangers}{}}$	Inf ${}_{\textit{Fan}}$	Inf I+I-	Inf ${}_{\textit{Peers}{}}$
	Accuracy (mean over eval. conditions)
Familiar	0.955	0.964	0.938	0.953
Unfamiliar	0.950	0.958	0.944	0.956

Appendix

Table 31: Topic familiarity for the zero-shot baseline.

Eval	No ov.	Ov.	$\Delta$ pp
M z-s / Inf ${}_{\textit{Strangers}}{}$
Both ${}_{\text{Disc}}$	0.919 (3123)	0.980 (1252)	$+6.1$
Turns ${}_{\text{Disc}}$	0.921 (3123)	0.984 (1252)	$+6.3$
M z-s / Inf ${}_{\textit{Fan}}{}$
Both ${}_{\text{Disc}}$	0.935 (2992)	0.988 (1383)	$+5.2$
Turns ${}_{\text{Disc}}$	0.937 (2992)	0.991 (1383)	$+5.4$

Appendix

Table 32: Judge accuracy split by whether interlocutor turns contain any exact rare-word overlap with the target biography. Parentheses give subset sizes;

\Delta

is overlap minus no overlap accuracy in percentage points.

Persona-Conditioned Generation

Persona-conditioned generation uses profiles, traits, or histories to steer AI outputs toward specified identities, preferences, and behaviors.

See all 109 papers

Apr 27, 2026

The Pragmatic Persona: Discovering LLM Persona through Bridging Inference

Large Language Models (LLMs) reveal inherent and distinctive personas through dialogue. However, most existing persona discovery approaches rely on surface-level lexical or stylistic cues, treating dialogue as a flat sequence of tokens and failing to capture the deeper discourse-level structures that sustain persona consistency. To address this limitation, we propose a novel analytical framework that interprets LLM dialogue through bridging inference -- implicit conceptual relations that connect utterances via shared world knowledge and discourse coherence. By modeling these relations as structured knowledge graphs, our approach captures latent semantic links that govern how LLMs organize meaning across turns, enabling persona discovery at the level of discourse coherence rather than surface realizations. Experimental results across multiple reasoning backbones and target LLMs, ranging from small-scale models to 80B-parameter systems, demonstrate that bridging-inference graphs yield significantly stronger semantic coherence and more stable persona identification than frequency or style-based baselines. These results show that persona traits are consistently encoded in the structural organization of discourse rather than isolated lexical patterns. This work presents a systematic framework for probing, extracting, and visualizing latent LLM personas through the lens of Cognitive Discourse Theory, bridging computational linguistics, cognitive semantics, and persona reasoning in large language models. Codes are available at https://github.com/JiSoo-Yang/Persona_Bridging.git

Jisoo Yang, Jongwon Ryu, Minuk Ma +2

Department of Artificial Intelligence, Chung-Ang University, Seoul, 06974, Republic of Korea · Department of Computer Science, University of British Columbia,2026 Vancouver, BC V6T 1Z4, Canada · Van Lang University, Ho Chi Minh City, Vietnam

Personality Modeling in Language Models Language Model Probing

Ask PDF

Aug 20, 2026

Generating Diverse Personas for User Simulators to Test Interview Dialogue Systems

This paper addresses the issue of the significant labor required to test interview dialogue systems. While interview dialogue systems are expected to be useful in various scenarios, like other dialogue systems, testing them with human users requires significant effort and cost. Therefore, testing with user simulators can be beneficial. Since most conventional user simulators have been primarily designed for training task-oriented dialogue systems, little attention has been paid to the personas of the simulated users. During development, testing interview dialogue systems requires simulating a wide range of user behaviors, but manually creating a large number of personas is labor-intensive. We propose a method that automatically generates personas for user simulators using a large language model. Furthermore, by assigning personality traits related to communication styles when generating personas, we aim to increase the diversity of communication styles in the user simulator. Experimental results show that the proposed method enables the user simulator to generate utterances with greater variation.

Mikio Nakano, Kazunori Komatani, Hironori Takeuchi

C4A Research Institute, Inc., Setagaya, Tokyo, Japan · SANKEN, University of Osaka, Ibaraki, Osaka, Japan · Musashi University, Nerima, Tokyo, Japan

User Simulation Persona-Conditioned Generation Conversational Agents

Ask PDF

May 7, 2026

PersonaKit (PK): A Plug-and-Play Platform for User Testing Diverse Roles in Full-Duplex Dialogue

As spoken dialogue systems expand beyond traditional assistant roles to encompass diverse personas -- such as authoritative instructors, uncooperative merchants, or distracted workers -- they require distinct, human-like turn-taking behaviors to maintain psychological immersion. However, current full-duplex systems often default to a rigid, overly accommodating ``always-yield'' policy during overlapping speech, which severely undermines character consistency for non-submissive roles. Evaluating alternative, persona-specific turn-taking strategies through empirical user studies is challenging because building real-time full-duplex test environments requires substantial engineering overhead. To address this, we present PersonaKit (PK), an open-source, low-latency web platform for the rapid prototyping and evaluation of conversational agents. Using intuitive JSON configurations, researchers can define personas, specify probabilistic interruption-handling behaviors (e.g., yield, hold, bridge, or override), and automatically deploy comparative A/B surveys. Through an in-the-wild evaluation with 8 distinct personas, we demonstrate that PersonaKit provides an extensible, end-to-end framework for studying complex sociolinguistic behaviors in next-generation spoken agents.

Hyunbae Jeon, Jinho D. Choi

Department of Computer Science Emory University Atlanta, GA, USA

Voice Agent Evaluation Spoken Dialogue Systems Human-in-the-Loop Evaluation Full-Duplex Spoken Dialogue Systems Turn-Taking in Dialogue

Ask PDF

Abstract

Persona-Conditioned Generation

Explore similar work

The Pragmatic Persona: Discovering LLM Persona through Bridging Inference

Generating Diverse Personas for User Simulators to Test Interview Dialogue Systems

PersonaKit (PK): A Plug-and-Play Platform for User Testing Diverse Roles in Full-Duplex Dialogue