Realistic user simulation is crucial for training and evaluating multi-turn dialogue systems, yet creating simulators that accurately replicate human behavior remains a significant challenge. An effective simulator must expose the failure modes of the systems under evaluation. This work introduces Direct Iterative Adversarial Learning (DIAL), an adversarial framework that iteratively enhances user simulator realism through a competitive dynamic between a generator (user simulator) and a discriminator. When applied to mental health support, a domain characterized by diverse failure types and a critical dependence on realistic user behavior for failure detection, DIAL restores lexical diversity diminished by supervised fine-tuning and drastically reduces discriminator accuracy. The resulting simulator exhibits a strong correlation between simulated and real failure occurrence rates while maintaining low distributional divergence of failure modes. These findings indicate that DIAL is a promising method for developing realistic user simulators in multi-turn dialogue, facilitating reliable and cost-effective system evaluation prior to deployment.
Figures & tables
Figure 1: Overview of the DIAL training process. The simulator generates sessions that are evaluated by the discriminator to distinguish real from simulated conversations. Discriminator feedback guides preference dataset generation, which is then used to refine the simulator through DPO, creating an iterative competitive dynamic that improves realism.
Simulator
H
TTR
D-2
D-3
MAUVE
LSM
Human
6.53
.499
.872
.965
–
.714
Zero-shot
GPT-4o
7.12
.408
.837
.956
.128
.729
GPT-5.2
7.73
.266
.738
.922
.015
.674
Kimi-K2
7.65
.416
.851
.965
.118
.672
Qwen-2.5-7B
7.07
.390
.791
.914
.158
.726
Table 1: Linguistic diversity features comparing real users with simulators. Best fine-tuned values are in bold , second best are underlined ; for LSM, best means closest to the human value.
Figure 2: Issue rate comparison across simulated and real sessions. Error bars represent standard error.
Turn 1
Turn 5
Turn 10
Turn 25
Overall
Human
Discriminator
Acc
F1
MCC
Acc
F1
MCC
Acc
F1
MCC
Acc
F1
MCC
Acc
F1
MCC
Acc
SFT
62.0
80.1
24.3
82.3
81.6
64.7
90.2
88.6
80.1
96.1
93.9
91.1
87.9
85.9
75.4
67.0
DIAL-it1
52.8
53.8
5.5
70.3
69.3
40.6
79.5
76.0
58.3
87.4
77.4
68.7
76.5
71.4
51.5
64.0
DIAL-it2
53.8
51.3
8.2
67.3
65.3
34.7
76.2
71.6
51.3
90.1
86.5
79.2
74.0
69.1
47.2
56.0
DIAL-it3
52.5
60.6
5.5
64.0
64.4
28.0
68.1
59.5
33.9
84.5
78.9
67.3
69.5
63.9
37.6
55.0
Table 2: Discriminator performance at different conversation turns, and success rate with human judgment on the same conversations. All values are in percentage (%). MCC (Matthews Correlation Coefficient) is a measure of the correlation between the observed and predicted classifications. Best results are in bold , second best are underlined .
Simulator
r
p
KL Div.
Zero-shot
GPT-4o
.484
.079
1.566
Kimi-K2
.564
.036
.453
Llama-3.3-70B
.629
.016
.542
UserSim
160K-SFT
.528
.052
.671
Table 3: Correlation with chatbot issue rates and KL divergence of issue category distributions, computed across 14 diverse chatbot configurations. Aggregate combines UserSim-160K-SFT, DIAL-it1, it2, and it3. Best values are in bold , second best are underlined .
Method
TTR
D-2
D-3
MAUVE
Success (%)
Human
.037
.229
.470
–
–
RulePolicy
.004
.015
.026
.039
50.2
TUS
.007
.028
.065
.158
48.8
GenTUS
.010
.039
.086
.175
59.1
Zero-shot
.028
.150
.316
.071
99.1 ∗
SFT
.022
.106
.228
.137
98.3 ∗
Table 4: MultiWOZ 2.1 user simulator evaluation. Diversity metrics computed on user utterances against the human test set. Success rates are reported using the ConvLab-3 evaluator. Best fine-tuned values are in bold .
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: t-SNE visualization of session embeddings. Production sessions are marked with stars.
Figure 4: User action distributions comparing simulators to real data. Error bars represent standard error.
Simulator
All Actions
Bad
Good
Moderators
Features
Zero-shot
GPT-4o
.1727
.1399
.1804
.1048
.2124
Kimi-K2
.1678
.1075
.1811
.1020
.2250
Llama-3.3-70B
.1884
.1112
.2440
.1031
.2233
UserSim
160K-SFT
.1388
.1031
.1168
.1035
.2015
Appendix
Table 5: BCE loss from real user action distributions by category. Best values are in bold , second best are underlined .
Figure 5: Correlation between simulated and real issue rates for UserSim-160K-DIAL-it2 across 14 chatbot model configurations spanning multiple model families (Qwen-based, Llama-based, GPT-based). Error bars represent standard error.
Simulation (UserSim-160K-SFT)
Production
Model
Control
Treatment
z
p
Control
Treatment
z
p
Model A (Qwen-based)
2.65%
3.25%
2.83
0.005*
2.71%
3.56%
3.27
0.001*
Model B (Llama-based)
2.42%
2.87%
2.23
0.026*
2.32%
2.64%
1.35
0.178
Model C (GPT-based)
1.78%
2.13%
2.02
0.044*
1.54%
1.42%
-0.66
0.511
*Statistically significant at α=0.05
Appendix
Table 6: Issue rate comparison between simulated and real evaluations with two-tailed significance tests ( α=0.05 ). The treatment (fine-tuned with DPO on synthetic issue data to reduce issues) failed and actually increased issue rates. Both simulation and real data show consistent directional trends, validating the simulator’s predictive utility.
Hyperparameter
SFT
DPO
Epochs
1
1
Batch size
1
1
DPO beta ( β )
–
1
LoRA rank
8
8
LoRA alpha
16
16
LoRA trainable modules
all-linear
all-linear
Appendix
Table 7: Hyperparameters for supervised fine-tuning (SFT) and Direct Preference Optimization (DPO).
Model
Agreement (%)
Omission (%)
Detected
GPT-5
45.2
14.3
93
Claude-Sonnet-4-5
63.6
42.9
44
Gemini-3-Pro
38.0
22.4
100
Appendix
Table 8: Human validation of LLM-based issue analyzers. Detected denotes the number of issues flagged by each model. Best values are in bold , second best are underlined .
Per-Annotator Fool Rate
Annotator
SFT
DIAL-it1
DIAL-it2
DIAL-it3
Mean
A1
.24
.56
.40
.44
.41
A2
.28
.28
.40
.36
.33
A3
.44
.36
.44
.48
.43
A4
.36
.24
.52
.52
.41
Mean
.33
.36
.44
.45
.395
Appendix
Table 9: Per-annotator fool rate by simulator, with marginal means across simulators (last column) and across annotators (last row). The bottom-right cell reports the overall mean fool rate computed across all 400 annotations.