ARGUS: An Agenda-Driven Framework for Reliable User Simulation
Authors: Hang Zheng, Nan Jiang, Zichen Zhu, Zixu Yang, Zhiyang Tang, Shuai Fan, Lu Chen, Xiaobao Wu, Kai Yu
Organizations: X-LANCE Lab, School of Computer Science, MoE Key Lab of Artificial Intelligence SJTU AI Institute, Shanghai Jiao Tong University, Shanghai, China · Jiangsu Key Lab of Language Computing, Suzhou, China · AISpeech Co., Ltd., Suzhou, China · Suzhou Laboratory, Suzhou, China
High-fidelity user simulators are essential for evaluating and training agents in interactive environments. However, current LLM-based simulators struggle with reliability issues, such as goal omission and policy non-compliance. We propose ARGUS (Agenda-dRiven Goal-oriented User Simulator), a neuro-symbolic framework that maintains the user's goal state using a dynamic agenda stack manipulated via symbolic operations. By rigorously decoupling state tracking from natural language generation, ARGUS facilitates strict goal progression and constraint-aligned interactions. For cost-efficient deployment, we distill ARGUS into compact models using a unified pipeline: evaluator-guided SFT followed by online RL with dense step-level rewards. To rigorously evaluate behavioral fidelity, we introduce a rubric-based metric Composite Behavioral Score (CBS). Evaluations on τ3-Bench and MultiWOZ demonstrate the significant superiority of the ARGUS architecture and the efficacy of our training pipeline. Code and data will be released soon.
Figures & tables
Figure 1: Stateless vs. stateful user simulation. Stateless generation may lose constraints or pending goals, while Argus preserves them through an explicit agenda stack.
Figure 2: Overview of ARGUS. (1) Agenda Initialization: structured user goals are parsed into a LIFO agenda stack with conditional behaviors. (2) Neuro-Symbolic User Simulation: a state controller updates the stack via symbolic operations ( Push , Pop , Modify , Wait ), and a response generator produces user utterances. (3) Two-Stage Training pipeline: teacher rollouts filtered by a turn-level evaluator (Soundness, Faithfulness, Adherence) yield SFT data, followed by online RL with step-level rewards.
Method
Retail
MultiWOZ
Airline
CBS
TSR
CBS
TSR
CBS
TSR
Prompt-based
Vanilla (8B)
50.3
44.4
76.5
56.8
59.8
40.4
ARGUS (8B)
72.5
51.6
76.0
57.4
69.0
42.4
Vanilla (235B)
73.1
50.4
88.4
63.2
75.1
47.2
ARGUS (32B)
83.9
57.5
80.4
58.6
78.9
34.4
Table 2: CBS (%) and TSR (%) for trained simulators. Airline is the held-out OOD environment. Unless marked, all rows use Qwen3-8B. Prompt-based rows are included for comparisons.
Figure 4
Figure 5: Error rate (%) across 12 CBS subcriteria for six simulators on Retail. Subcriteria are grouped by their parent dimension. Darker cells indicate higher error rates.
Figure 6
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Mean step-score bucket
N
Mean step score
Mean CBS (%)
(0.2,0.3]
8
0.2751
27.30
(0.3,0.4]
12
0.3584
46.70
(0.4,0.5]
30
0.4625
62.18
(0.5,0.6]
39
0.5618
70.86
(0.6,0.7]
51
0.6480
73.42
(0.7,0.8]
54
0.7552
82.37
Appendix
Table 3: Alignment between trajectory-aggregated step scores and final CBS. The 300 trajectories are grouped into eight non-empty score buckets; no trajectory falls in [0,0.2] .
Domain or layer
Supported operations
Hotel
Search by static attributes and dynamic constraints; check room availability and price; create, modify, cancel, and retrieve bookings.
Restaurant
Search by area, food type, price range, and reservation support; check table availability; create, modify, cancel, and retrieve reservations.
Train
Search by route, date, time, class, seat preference, and availability; assign seats; create, modify, cancel, and retrieve bookings.
Taxi
Search by route, time, capacity, vehicle attributes, and reservation support; estimate fares; create, modify, cancel, and retrieve taxi bookings.
Attraction
Search by area, type, name, and reservation support; check reservation feasibility; create, modify, cancel, and retrieve reservations when supported.
Geo and utility
Compute distances between places, retrieve nearby entities within a target domain, and perform simple arithmetic calculations.
Appendix
Table 4: Supported operations in the interactive MultiWOZ environment.
Dimension
Subcriterion
Description
D1: Faithfulness & Consistency
Hallucination
Simulator fabricates information in the response, introducing new details not present in the scenario or conversation facts, such as fictitious order numbers, account information, device status, or other business details.
State Confusion
Simulator confuses the conversation state, mixing up attributes of different entities, different orders, different slots, or different check results, which leads to incorrect factual correspondences.
Ungrounded Confirm
Simulator makes ungrounded confirmations, directly saying "yes" or "that’s right" without verification when the Agent proposes candidate information that is unprovided or incorrect.
Cooperativeness
Simulator fails to cooperate truthfully, baselessly pretending not to know, shifting responsibility, or glossing over information that it already knows in the scenario and should have answered.
D2: Policy Adherence
Constraint Violation
Simulator violates constraints by prematurely leaking information it should not say, exposing meta-instructions, or accepting options and out-of-boundary solutions explicitly prohibited by the scenario.
Cond. Logic Failure
Simulator fails to execute conditional logic correctly, meaning the if -> then rules in the scenario are either not triggered when they should be (missed trigger) or triggered when they should not be (false trigger).
Appendix
Table 5: The 12 binary subcriteria of the CBS evaluation framework, grouped by dimension.
Metric
Agreement
Cell CBS Pearson r
0.873
Cell CBS Spearman ρ
0.886
Cell CBS Kendall τb
0.746
Protocol Failure agreement
93.81%
Protocol Failure Cohen’s κ
0.624
Method-pair ranking agreement
91.67%
Appendix
Table 6: Agreement between the Qwen and DeepSeek CBS judges on Retail domain. Cell-level correlations are computed over 24 method cells.
Dimension
CBS Subcriterion
Human IAA
Human–Qwen
Human–GPT
Qwen–GPT
AC2
AC1
AC1
AC1
Faithfulness & Consistency
Hallucination
0.896
0.807
0.528
0.705
State Confusion
1.000
0.846
1.000
0.901
Ungrounded Confirm.
0.928
0.935
0.539
0.638
Cooperativeness
0.939
0.929
0.916
0.887
Policy Adherence
Constraint Violation
0.828
0.603
0.701
0.739
Appendix
Table 7: Consolidated CBS reliability results across human inter-annotator agreement, human–LLM agreement, and Qwen–GPT model agreement. Human IAA uses Gwet’s AC2 because it is a multi-rater setting; pairwise comparisons use Gwet’s AC1.
Method
Retail
MultiWOZ
CBS
TSR
CBS
TSR
Naive-SFT (trajectory)
62.3
47.0
84.5
64.0
Naive-SFT (step-level)
66.8
43.8
84.6
65.7
UGST-SFT (trajectory)
54.3
48.6
78.2
67.1
UGST-SFT (step-level)
62.6
47.2
81.4
64.9
Argus -SFT (trajectory)
81.9
51.9
84.5
63.6
Appendix
Table 9: Ablation of architecture and SFT data filtering. All variants use Qwen3-8B. “Trajectory” keeps samples only from task-successful trajectories, while “step-level” applies the evaluator in Appendix § B . For Naive-SFT and UGST-SFT, we design step evaluators tailored to their respective requirements.
Method
Qwen3-MoE-30B
Gemma3-27B
CBS
TSR
CBS
TSR
Retail
Vanilla
65.1
44.6
68.1
32.6
+ Structured goal
76.4
48.4
71.6
39.1
+ Prompt constraints
68.8
47.5
67.6
43.7
+ Both
78.0
53.5
76.1
42.5
Appendix
Table 10: Input-representation and prompt controls for prompt-based simulators (%). “Structured goal” supplies the same parsed goal representation used by Argus ; “Prompt constraints” adds matched behavioral rules; “Both” applies both controls.
Method
Qwen3-MoE-30B
Gemma3-27B
CBS
TSR
CBS
TSR
Retail
UGST (GPT-4o extractor)
60.7
52.3
74.7
41.8
UGST (GPT-5.1 extractor)
69.9
34.9
74.7
44.7
Argus
85.6
61.1
85.6
53.8
Airline
Appendix
Table 11: Control for the structured-goal extractor used by UGST (%). The original UGST setup uses GPT-4o, whereas the controlled variant uses GPT-5.1, matching the extractor model used by Argus .
Method
Retail
Airline
MultiWOZ
CBS
TSR
CBS
TSR
CBS
TSR
Naive-SFT
53.7
43.3
59.6
32.0
69.6
53.2
UGST-SFT
49.7
46.1
57.5
31.6
69.4
59.6
UGST-RL
53.0
52.5
58.1
30.0
72.6
63.1
ARGUS-SFT
76.0
50.5
72.8
34.4
78.6
55.4
ARGUS-RL
82.6
57.7
80.4
35.6
83.0
58.3
Appendix
Table 12: Performance of trained Qwen3-4B user simulators.
Simulator
Easy
Medium
Hard
Expert
Overall
TSR
CBS
TSR
CBS
TSR
CBS
TSR
CBS
TSR
CBS
Vanilla
77.1
70.7
47.6
68.8
38.2
71.9
35.0
69.9
48.8
70.3
React
72.5
68.4
54.4
69.6
42.5
67.6
41.0
66.3
52.2
68.0
Verify
72.3
72.1
45.8
71.2
42.3
73.8
36.6
67.2
48.6
71.0
Reflection
74.2
70.6
47.6
67.3
41.2
69.9
36.6
62.5
49.3
67.5
UGST
72.3
68.7
49.2
67.1
44.3
67.1
32.4
55.4
49.0
64.5
Appendix
Table 13: Performance of user simulators across task difficulty levels on the Retail domain and Qwen series. Complexity is defined as ncond+nkey , where ncond counts conditional behaviors and nkey counts database-modifying key actions ( cancel , modify , exchange , and return ). Bold indicates the best result per column.
Case 1: Skipping a Conditional Preference
Task and intended interaction. The user should first request a double room for two people in a centre guesthouse on January 8, 2026. If no centre guesthouse is available, the user may relax the type constraint and accept any centre accommodation. If multiple options remain, the user should request expensive properties. If there are still several expensive options, the user should ask the agent to choose the lowest total price among them.
Abridged dialogue. Agent: No centre guesthouse is available. Would you like me to search for other accommodation types? User: Okay, please look for any type of accommodation in the centre that can offer a double room for two people for one night. Agent: I found two centre hotels: Gonville Hotel (£147, expensive) and University Arms Hotel (£237, expensive). Would you like to proceed with one of these? User: Yes, please book the Gonville Hotel. It is the cheaper one, so that works for me.
Final-state result. The final booking is correct: the simulator books Gonville Hotel for the requested date, room type, party size, and stay length. The final-state check therefore marks the task as successful (TSR = 1.0).
CBS judgment. The judge preserves credit for faithfulness and format, since the simulator does not hallucinate unsupported facts and follows the output protocol. However, it removes credit for policy adherence and goal completion/progression. After the agent presents two options, the user immediately selects the cheaper hotel. The intended behavior is to first express the expensive-property preference and then use lowest price only as a tie-break among expensive options. Although the final hotel happens to be correct, the simulated user exposes the final choice through a shortcut rather than through the specified conditional goal progression.
Case 2: Skipping a Required Search Refinement
Task and intended interaction. The user should book standard-class train tickets for two people from Cambridge to Bishops Stortford on January 7, 2026. The initial request should only consider available trains departing after 23:30. If no train is found, the user should ask the agent to search again without the time restriction. If this produces too many results, the user should add a new constraint requiring departures after 05:28. Only then should the user choose the option with the earliest arrival time, using the lowest total price as a tie-break.
Appendix
Table 14: Representative shortcut cases where the final-state check succeeds, but CBS judgment identifies violations of the intended user-side interaction process. Dialogues are abridged for readability.
Method
Qwen3-8B
Qwen3-32B
Qwen3-MoE-30B
Qwen3-MoE-235B
T
D
T
D
T
D
T
D
Vanilla
0.113
0.842
0.341
1.96
0.149
0.879
0.412
2.43
React
0.180
1.17
0.634
3.83
0.260
1.52
0.697
4.26
Verify
0.308
2.26
0.820
4.75
0.383
2.32
0.874
5.20
Reflection
0.317
2.68
0.848
4.88
0.362
2.15
0.956
5.71
UGST
1.09
9.79
3.88
29.5
2.00
11.8
5.00
26.0
Appendix
Table 15: Inference cost of prompt-based user simulators across model sizes. All values in ×10−3 USD (1 CNY = $0.147); T = cost/turn, D = cost/dialogue; averaged over Retail and MultiWOZ.
Method
4B
8B
T
D
T
D
Naive-SFT
0.066
0.418
0.098
0.609
UGST-SFT
0.079
0.452
0.127
0.752
UGST-RL
0.075
0.397
0.125
0.784
ARGUS-SFT
0.123
0.820
0.144
0.950
ARGUS-RL
0.088
0.661
0.138
0.974
Appendix
Table 16: Inference cost of trained user simulators. Same unit convention as Table 15 . All costs computed from actual token-usage records; models run locally via vLLM and priced at official API rates.
Figure 9: Per-dimension CBS comparison between SFT (at varying data scales) and RL. RL shows clear advantages on Faithfulness & Consistency and Policy Adherence, while SFT is stronger on Goal Completion. Format & Protocol is saturated for both.
As NLP evaluation shifts from static benchmarks to multi-turn interactive settings, LLM-based simulators have become widely used as user proxies, serving two roles: generating user turns and providing evaluation signals. Yet, these simulations are frequently assumed to be faithful to real human behaviors, often without rigorous verification. We formalize the Sim2Real gap in user simulation and present the first study running the full τ-bench protocol with real humans (451 participants, 165 tasks), benchmarking 31 LLM simulators across proprietary, open-source, and specialized families using the User-Sim Index (USI), a metric we introduce to quantify how well LLM simulators resemble real user interactive behaviors and feedback. Behaviorally, LLM simulators are excessively cooperative, stylistically uniform, and lack realistic frustration or ambiguity, creating an "easy mode" that inflates agent success rates above the human baseline. In evaluations, real humans provide nuanced judgments across eight quality dimensions while simulated users produce uniformly more positive feedback; rule-based rewards are failing to capture rich feedback signals generated by human users. Overall, higher general model capability does not necessarily yield more faithful user simulation. These findings highlight the importance of human validation when using LLM-based user simulators in the agent development cycle and motivate improved models for user simulation.
Xuhui Zhou, Weiwei Sun, Qianou Ma +8
Carnegie Mellon University, Language Technologies Institute
Large Language Model (LLM) agents are increasingly deployed in settings where they interact with diverse users, including those who are unclear, impatient, or reluctant to share information. However, collecting real interaction data at scale remains expensive. The field has turned to LLM-based user simulators as stand-ins, but these simulators inherit the behavior of their underlying models: cooperative and homogeneous. As a result, agents that appear strong in simulation often fail in real human interactions. To narrow this gap, we introduce Persona Policies (PPol), a plug-and-play control layer that induces realistic behavioral variation in user simulators while preserving original task goals. Rather than hand-crafting personas, we employ an evolutionary coding agent to discover persona generation programs optimized for human-likeness and behavioral coverage over real user conversations. The evolved program generates diverse, human-like personas for any task in the domain. Across 4 benchmarks--including τ2-bench Retail and Airline, ColBench, and WildChat--evolved PPol yield 28-72% absolute gains in fitness score over the baseline simulator. In blinded evaluations, annotators judged PPol users as 'human' 80.4% of the time, nearly 2x more than the baseline simulators. Training agents with PPol also improves real-world performance: our user study with live human-agent interactions showed that fine-tuning with our method boosted task success by +23% over default baselines. PPol thus offers a novel approach to strengthen simulator-based evaluation and training without changing underlying tasks.
Users increasingly rely on AI systems for extended, multi-turn tasks, yet evidence indicates that LLM-based system performance degrades over prolonged interaction. To evaluate and improve AI assistants in such settings, we need user simulators that faithfully follow complex user intents while exhibiting realistic conversational behavior. Existing approaches manifest a trade-off: prompted assistants can follow complex user intents but lack behavioral fidelity, while specialized user simulators better approximate conversational behavior but struggle to remain faithful to complex intents. In this paper, we propose a reinforcement learning (RL) recipe that jointly rewards task intent adherence and behavioral naturalness. A crucial contribution of the method is operationalizing naturalness with a lightweight discriminator trained in an alternating, adversarial loop with the user simulator. We release the resulting model, UserLM-2.0, which produces valid simulated conversations (i.e., meeting both task-faithfulness and naturalness criteria) 46.2% of the time, over three times the rate of the strongest prior user simulator (14.1%).