Realistic user simulation is crucial for training and evaluating multi-turn dialogue systems, yet creating simulators that accurately replicate human behavior remains a significant challenge. An effective simulator must expose the failure modes of the systems under evaluation. This work introduces Direct Iterative Adversarial Learning (DIAL), an adversarial framework that iteratively enhances user simulator realism through a competitive dynamic between a generator (user simulator) and a discriminator. When applied to mental health support, a domain characterized by diverse failure types and a critical dependence on realistic user behavior for failure detection, DIAL restores lexical diversity diminished by supervised fine-tuning and drastically reduces discriminator accuracy. The resulting simulator exhibits a strong correlation between simulated and real failure occurrence rates while maintaining low distributional divergence of failure modes. These findings indicate that DIAL is a promising method for developing realistic user simulators in multi-turn dialogue, facilitating reliable and cost-effective system evaluation prior to deployment.
Figures & tables
Figure 1: Overview of the DIAL training process. The simulator generates sessions that are evaluated by the discriminator to distinguish real from simulated conversations. Discriminator feedback guides preference dataset generation, which is then used to refine the simulator through DPO, creating an iterative competitive dynamic that improves realism.
Simulator
H
TTR
D-2
D-3
MAUVE
LSM
Human
6.53
.499
.872
.965
–
.714
Zero-shot
GPT-4o
7.12
.408
.837
.956
.128
.729
GPT-5.2
7.73
.266
.738
.922
.015
.674
Kimi-K2
7.65
.416
.851
.965
.118
.672
Qwen-2.5-7B
7.07
.390
.791
.914
.158
.726
Table 1: Linguistic diversity features comparing real users with simulators. Best fine-tuned values are in bold , second best are underlined ; for LSM, best means closest to the human value.
Figure 2: Issue rate comparison across simulated and real sessions. Error bars represent standard error.
Turn 1
Turn 5
Turn 10
Turn 25
Overall
Human
Discriminator
Acc
F1
MCC
Acc
F1
MCC
Acc
F1
MCC
Acc
F1
MCC
Acc
F1
MCC
Acc
SFT
62.0
80.1
24.3
82.3
81.6
64.7
90.2
88.6
80.1
96.1
93.9
91.1
87.9
85.9
75.4
67.0
DIAL-it1
52.8
53.8
5.5
70.3
69.3
40.6
79.5
76.0
58.3
87.4
77.4
68.7
76.5
71.4
51.5
64.0
DIAL-it2
53.8
51.3
8.2
67.3
65.3
34.7
76.2
71.6
51.3
90.1
86.5
79.2
74.0
69.1
47.2
56.0
DIAL-it3
52.5
60.6
5.5
64.0
64.4
28.0
68.1
59.5
33.9
84.5
78.9
67.3
69.5
63.9
37.6
55.0
Table 2: Discriminator performance at different conversation turns, and success rate with human judgment on the same conversations. All values are in percentage (%). MCC (Matthews Correlation Coefficient) is a measure of the correlation between the observed and predicted classifications. Best results are in bold , second best are underlined .
Simulator
r
p
KL Div.
Zero-shot
GPT-4o
.484
.079
1.566
Kimi-K2
.564
.036
.453
Llama-3.3-70B
.629
.016
.542
UserSim
160K-SFT
.528
.052
.671
Table 3: Correlation with chatbot issue rates and KL divergence of issue category distributions, computed across 14 diverse chatbot configurations. Aggregate combines UserSim-160K-SFT, DIAL-it1, it2, and it3. Best values are in bold , second best are underlined .
Method
TTR
D-2
D-3
MAUVE
Success (%)
Human
.037
.229
.470
–
–
RulePolicy
.004
.015
.026
.039
50.2
TUS
.007
.028
.065
.158
48.8
GenTUS
.010
.039
.086
.175
59.1
Zero-shot
.028
.150
.316
.071
99.1 ∗
SFT
.022
.106
.228
.137
98.3 ∗
Table 4: MultiWOZ 2.1 user simulator evaluation. Diversity metrics computed on user utterances against the human test set. Success rates are reported using the ConvLab-3 evaluator. Best fine-tuned values are in bold .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: t-SNE visualization of session embeddings. Production sessions are marked with stars.
Figure 4: User action distributions comparing simulators to real data. Error bars represent standard error.
Simulator
All Actions
Bad
Good
Moderators
Features
Zero-shot
GPT-4o
.1727
.1399
.1804
.1048
.2124
Kimi-K2
.1678
.1075
.1811
.1020
.2250
Llama-3.3-70B
.1884
.1112
.2440
.1031
.2233
UserSim
160K-SFT
.1388
.1031
.1168
.1035
.2015
Appendix
Table 5: BCE loss from real user action distributions by category. Best values are in bold , second best are underlined .
Figure 5: Correlation between simulated and real issue rates for UserSim-160K-DIAL-it2 across 14 chatbot model configurations spanning multiple model families (Qwen-based, Llama-based, GPT-based). Error bars represent standard error.
Simulation (UserSim-160K-SFT)
Production
Model
Control
Treatment
z
p
Control
Treatment
z
p
Model A (Qwen-based)
2.65%
3.25%
2.83
0.005*
2.71%
3.56%
3.27
0.001*
Model B (Llama-based)
2.42%
2.87%
2.23
0.026*
2.32%
2.64%
1.35
0.178
Model C (GPT-based)
1.78%
2.13%
2.02
0.044*
1.54%
1.42%
-0.66
0.511
*Statistically significant at α=0.05
Appendix
Table 6: Issue rate comparison between simulated and real evaluations with two-tailed significance tests ( α=0.05 ). The treatment (fine-tuned with DPO on synthetic issue data to reduce issues) failed and actually increased issue rates. Both simulation and real data show consistent directional trends, validating the simulator’s predictive utility.
Hyperparameter
SFT
DPO
Epochs
1
1
Batch size
1
1
DPO beta ( β )
–
1
LoRA rank
8
8
LoRA alpha
16
16
LoRA trainable modules
all-linear
all-linear
Appendix
Table 7: Hyperparameters for supervised fine-tuning (SFT) and Direct Preference Optimization (DPO).
Model
Agreement (%)
Omission (%)
Detected
GPT-5
45.2
14.3
93
Claude-Sonnet-4-5
63.6
42.9
44
Gemini-3-Pro
38.0
22.4
100
Appendix
Table 8: Human validation of LLM-based issue analyzers. Detected denotes the number of issues flagged by each model. Best values are in bold , second best are underlined .
Per-Annotator Fool Rate
Annotator
SFT
DIAL-it1
DIAL-it2
DIAL-it3
Mean
A1
.24
.56
.40
.44
.41
A2
.28
.28
.40
.36
.33
A3
.44
.36
.44
.48
.43
A4
.36
.24
.52
.52
.41
Mean
.33
.36
.44
.45
.395
Appendix
Table 9: Per-annotator fool rate by simulator, with marginal means across simulators (last column) and across annotators (last row). The bottom-right cell reports the overall mean fool rate computed across all 400 annotations.
Faithful user simulation is fundamental to building, evaluating, and improving interactive AI at scale. Yet current simulators often produce plausible individual responses without reproducing the intent evolution and outcomes observed in real interactions. We propose TRACER, a multi-turn user simulator that models evolving user intent and aligns simulated trajectories with real ones. TRACER is trained in two stages: supervised fine-tuning on real user dialogues, followed by multi-turn reinforcement learning. The RL stage combines hierarchical outcome- and trajectory-level rewards with deviation-aware advantage modulation, jointly addressing reward sparsity and credit assignment challenges in long dialogues. On real customer-service sessions organized into reference cohorts, TRACER-7B surpasses the strongest baseline by 11.4 conversion F1 points, while outperforming all baselines on group-level conversion-rate error and semantic trajectory distance and generalizing to out-of-distribution scenarios. In human Turing tests, annotators identified TRACER conversations at near-chance accuracy. Building on this simulator, we further introduce the Dynamic Marketing Benchmark, which jointly evaluates persuasion and response quality via simulated interactions, revealing that higher response quality does not necessarily correspond to higher conversion rates.
Geng Chen, Ruotong Pan, Zhirui Yang +9
Peking University · Kuaishou Technology · Xi’an Jiaotong University +3
Multi-turn agents are commonly evaluated with LLM-based simulated users, which can be overly cooperative and can therefore overestimate an agent's performance. We here introduce StressSim, an intent-preserving adversarial user simulator framework for stress-testing agents with challenging yet valid user interactions. StressSim optimises the simulator's high-level adversarial plan for each task instance. Candidate planner prompts are evolved with a genetic prompt-optimisation loop that uses rollout traces, granular task-completion signals, and validity feedback to favour conversations that reduce task success without changing the user's underlying goal. We evaluate the approach on the MultiWOZ dataset and the retail and airline domains from tau-bench with three target LLM agents, comparing against standard simulators and hand-designed non-collaborative user behaviours. We demonstrate the effectiveness of StressSim in exposing robustness gaps across frontier LLMs, with tau-bench pass^4 scores decreasing by up to 69% on gemini-3.8-flash and 62% on gpt-5.6-sol. StressSim maintains goal preservation performance (which decreases by at most 10%), using an optimisation budget of 20 rollouts per task. Sensitivity analysis suggests that increasing the budget further decreases task success. Overall our results demonstrate that StressSim exposes robustness gaps missed by standard and hand-designed user simulators while maintaining conversation realism. This is achieved through lightweight prompt optimisation alone, without model fine-tuning.
Atri Vivek Sharma, Brian Formento, Alessio Lomuscio
There is growing interest in exploring user simulation as an alternative to gathering and scoring real user-chatbot interactions for AI chatbot evaluation. For this purpose, it is important to ensure the realism of the simulation, i.e., the extent to which simulated dialogues reflect real dialogues users have with chatbots. Most existing methods evaluating simulation realism produce coarse quality signal and remain solely at the level of individual dialogues. To support more rigorous evaluation in this area, we propose realsim, an evaluation framework that enables practitioners to take a distributional view of real vs. simulated dialogues along 8 dimensions, covering attributes related to the communicative functions of the interaction, user states, and the surface form of user messages. We then instantiate the framework with a curated dataset of 1K multi-turn task-focused real user-chatbot dialogues that cover 16 domains of chatbot applications. Overall, we find that simulated users tend to struggle at capturing communication frictions that real users introduce to interactions, which could make evaluations based on such simulations overly optimistic. We also observe variability in performance across different domains, which may indicate a need for domain-specific user simulators.