Multi-turn agents are commonly evaluated with LLM-based simulated users, which can be overly cooperative and can therefore overestimate an agent's performance. We here introduce StressSim, an intent-preserving adversarial user simulator framework for stress-testing agents with challenging yet valid user interactions. StressSim optimises the simulator's high-level adversarial plan for each task instance. Candidate planner prompts are evolved with a genetic prompt-optimisation loop that uses rollout traces, granular task-completion signals, and validity feedback to favour conversations that reduce task success without changing the user's underlying goal. We evaluate the approach on the MultiWOZ dataset and the retail and airline domains from tau-bench with three target LLM agents, comparing against standard simulators and hand-designed non-collaborative user behaviours. We demonstrate the effectiveness of StressSim in exposing robustness gaps across frontier LLMs, with tau-bench pass^4 scores decreasing by up to 69% on gemini-3.8-flash and 62% on gpt-5.6-sol. StressSim maintains goal preservation performance (which decreases by at most 10%), using an optimisation budget of 20 rollouts per task. Sensitivity analysis suggests that increasing the budget further decreases task success. Overall our results demonstrate that StressSim exposes robustness gaps missed by standard and hand-designed user simulators while maintaining conversation realism. This is achieved through lightweight prompt optimisation alone, without model fine-tuning.
Figures & tables
Figure 1: Adversarial user-simulator workflow. Task instructions initialise a goal-state tracker; an adversarial planner selects turn-level user policies; and a generator produces user utterances. Planner prompts are optimised to reduce task success while preserving goal alignment.
gpt-5.4-nano
kimi-k2.6
gemini-3.8-flash
Dataset
User
pass@1
pass@4
pass4
IGA
pass@1
pass@4
pass4
IGA
pass@1
pass@4
pass4
IGA
MultiWOZ
standard
0.800
0.956
0.532
0.920
0.924
0.980
0.820
0.940
0.940
0.944
0.916
0.920
NC emotional
0.812
0.980
0.512
0.880
0.912
0.980
0.760
0.960
0.920
0.960
0.840
0.880
NC fragment
0.724
0.960
0.380
0.980
0.876
0.980
0.640
0.960
0.952
0.960
0.920
0.920
NC tangential
0.772
0.960
0.448
0.920
0.900
0.980
0.780
0.960
0.940
0.980
0.860
0.860
NC unavail.
0.416
0.820
0.044
0.800
0.604
0.952
0.208
0.840
0.652
0.960
0.160
0.980
Table 1: Results on MultiWOZ and the τ -bench retail and airline datasets. We evaluate 50 tasks per benchmark with n=5 trials per task and k=4 . Initial Goal Alignment (IGA) is the fraction of trajectories judged as valid on the first attempt. “NC” refer to the non-collaborative hand-written baselines introduced by Shim et al. (2026) .
Figure 2: Attack strength versus goal alignment, aggregated from Table 1 . Each point represents the mean of the evaluated metrics over the 9 dataset–LLM combinations; the x axis is the drop in pass4 relative to the standard user, (pass4standard−pass4sim)/pass4standard , and the y axis is the mean IGA.
Natural
Self-consist.
Progressive
Non-repet.
Goal-seeking
Goal-info. cov.
Final validity
Judge vs. Ann. 1
0.729
0.852
0.919
0.908
0.787
0.343
0.679
Judge vs. Ann. 2
0.968
1.000
0.948
1.000
0.928
0.736
0.840
Inter-annotator
0.768
0.852
0.921
0.908
0.812
0.525
0.839
Table 2: Cohen’s κ agreement on the human validation study of the automated validity judge over 100 sampled conversations. The first two rows report agreement between the automated judge and each human annotator; the last row reports inter-annotator agreement between the two human annotators.
Agent LLM
Optimised on
pass@1
pass@4
pass4
IGA
gemini-3.8-flash
none (standard)
0.588
0.788
0.320
0.960
gpt-5.4-nano
0.456
0.668
0.184
0.880
gemini-3.8-flash (self)
0.384
0.668
0.100
1.000
gpt-5.6-sol
none (standard)
0.480
0.704
0.296
0.960
gpt-5.4-nano
0.488
0.736
0.264
0.940
gemini-3.8-flash
0.524
0.696
0.304
0.920
Table 3: Transferability of the optimised planner prompts on τ -bench retail. “Optimised on” gives the source agent the planner prompt was optimised against.
Strategy family
Description
MultiWOZ
τ -bench
Conditional and staged disclosure
Delays a fallback, constraint, identifier, or later goal until a specified condition or dialogue stage is reached.
0.48
0.34
Relational and cross-turn memory pressure
Expresses information through prior dialogue state rather than repeating the literal value.
0.50
0.00
Boundary, ranked, or relative option selection
Requires a boundary-optimal, ranked, non-default, or relationally described option.
0.36
0.00
Evidence and identifier-state pressure
Tests identifier roles and distinctions among availability, authorisation, and completed-action evidence.
0.24
0.08
Multi-item and multi-action coordination pressure
Requires mappings, grouping, or separation across several items, orders, or actions.
0.00
0.18
Consent and confirmation-stage intent pressure
Changes or withholds intent at the boundary between discussion and commitment.
0.06
0.14
Table 4: Adversarial strategy families discovered by the genetic optimiser, and their prevalence among optimised planner prompts per dataset. Prompts may exhibit multiple strategies.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Asset
Type / Source
License / Governing Terms
MultiWOZ
Dataset; official repository
MIT License
τ -bench
Dataset; official repository
MIT License
gpt-5.4- nano
Model; OpenRouter API (target agent) and OpenAI API (user simulator)
OpenRouter Terms of Service and OpenAI service terms
gpt-5.6- sol
Model; OpenRouter API
OpenRouter Terms of Service and OpenAI service terms
gemini- 3.8-flash
Model; OpenRouter API
OpenRouter Terms of Service and Google Gemini API Terms of Service
gpt-5.5
Model; OpenAI API (strategy taxonomy construction only)
OpenAI service terms
Appendix
Table 5: Licenses and governing terms for datasets and models used in the experiments.
Figure 3: Sensitivity of the optimisation to its hyperparameters, with the other hyperparameter fixed at its default ( β=5 , 20 rollouts per task). (a) Increasing β improves first-attempt goal alignment (IGA) but weakens adversarial pressure (higher passˆ4 ), while pass@4 is unaffected. (b) Increasing the rollout budget increases adversarial pressure, with varying impact on the different metrics, while IGA stays at 1.000 throughout.
Judge vs. annotator
Validity axis
Ann.
Acc.
Prec.
Rec.
F1
κ
IAA κ
Natural
1
0.920
0.737
0.824
0.778
0.729
0.768
2
0.990
1.000
0.950
0.974
0.968
Self-consistent
1
0.990
1.000
0.750
0.857
0.852
0.852
2
1.000
1.000
1.000
1.000
1.000
Progressive
1
0.970
0.920
0.958
0.939
0.919
0.921
Appendix
Table 6: Agreement between the automated validity judge and two human annotators on 100 sampled conversations, computed with invalid conversations as the positive class. The final column reports inter-annotator agreement (IAA) between the two human annotators.
Interactive agent benchmarks and multi-turn reinforcement learning increasingly place a second language model in the role of the user. This simulated user controls what information the agent receives and when, yet current benchmarks score only the agent and do not directly measure whether the user correctly executed its assigned role. We introduce UserProxyBench, an evaluation layer over the tau-bench family, and the User Fidelity Score (UFS), which measures adherence to the benchmark's private user instructions using task-grounded rubric criteria scored independently of agent success. Holding the agent fixed at GPT-5.5 and varying only the user proxy across 375 enterprise tasks changes mean task reward by 15.2 points, while 24.4% of successful episodes contain a user-specification violation. The dominant failure is premature disclosure: users provide information before it is requested. This behavior has little effect on task reward, yet among successful episodes it causes the agent to make 1.06 fewer tool calls on average, changing the interaction being evaluated while preserving the reward. Finally, across seven proxies we identify an empirical cost-fidelity frontier, enabling practitioners to select the least expensive simulator that satisfies a required fidelity level.
As NLP evaluation shifts from static benchmarks to multi-turn interactive settings, LLM-based simulators have become widely used as user proxies, serving two roles: generating user turns and providing evaluation signals. Yet, these simulations are frequently assumed to be faithful to real human behaviors, often without rigorous verification. We formalize the Sim2Real gap in user simulation and present the first study running the full τ-bench protocol with real humans (451 participants, 165 tasks), benchmarking 31 LLM simulators across proprietary, open-source, and specialized families using the User-Sim Index (USI), a metric we introduce to quantify how well LLM simulators resemble real user interactive behaviors and feedback. Behaviorally, LLM simulators are excessively cooperative, stylistically uniform, and lack realistic frustration or ambiguity, creating an "easy mode" that inflates agent success rates above the human baseline. In evaluations, real humans provide nuanced judgments across eight quality dimensions while simulated users produce uniformly more positive feedback; rule-based rewards are failing to capture rich feedback signals generated by human users. Overall, higher general model capability does not necessarily yield more faithful user simulation. These findings highlight the importance of human validation when using LLM-based user simulators in the agent development cycle and motivate improved models for user simulation.
Xuhui Zhou, Weiwei Sun, Qianou Ma +8
Carnegie Mellon University, Language Technologies Institute
LLM-based user simulation is the primary mechanism for end-to-end agent evaluation, yet simulated users are poor proxies for real humans: unconstrained LLM defaults produce a Formalism Ceiling (style match rates of 6-8% against real users), while hand-crafted behavioral directives trigger Directive Amplification, where models hyper-interpret instructions into unnatural behavioral extremes that vary dramatically across simulator models. We present RealUserSim, the first user simulation framework grounded in real behavioral data. From 14,000+ authentic human-LLM conversations (WildChat), we extract 7,275 executable behavioral profiles and use them to ground LLM simulators. A fidelity benchmark (PT3) on 600 conversations across 71+ domains with anti-leakage controls shows that grounded simulation raises match rate from 24.2% to 45.3% across five behavioral dimensions. Agent evaluation on TauBench with 6 simulator models and extensive analysis shows that grounded simulation acts as a realistic stress test, surfacing three failure mechanisms invisible to cooperative simulators (mean -3.2% to -3.5% task success degradation), while Directive Amplification in existing benchmarks produces unrealistic behavior that compromises the validity of agent evaluation.