MIMESIS: Learning User Simulators as Training Environments for Interactive Agents

Hoang Phan; Dat Huynh; Andrey Zhmoginov; Qi Zeng; Wancen Mu; Yue Cao; Shengjie Bi; Yun He

cs.AIOct 7, 2026

MIMESIS: Learning User Simulators as Training Environments for Interactive Agents

Authors: Hoang Phan, Dat Huynh, Andrey Zhmoginov, Qi Zeng, Wancen Mu, Yue Cao, Shengjie Bi, Yun He, +2 more

Organizations: Meta Superintelligence Labs · New York University · University of Wisconsin - Madison

Abstract

Training and evaluating interactive language agents typically requires rich user interactions, yet collecting human feedback is expensive and difficult to scale. Simulated users offer a scalable alternative, but they must both resemble real user behavior and provide useful learning experiences for agents. In contrast, most agent-training frameworks rely on off-the-shelf assistant LLMs, whose helpfulness can make them overly cooperative, explicit, and behaviorally homogeneous compared with real users. We introduce MIMESIS, a purpose-built user simulator trained on human conversations with explicit reasoning supervision and 13 realistic behavioral patterns derived from real user interactions. Empirically, our 9B model achieves a SOUL-Index of 65.7, surpassing the strongest frontier model. Compared with Claude-Opus-5, the strongest baseline on RealUserSim and SimulatorArena, MIMESIS improves behavioral fidelity by 13.4 points and reduces Turing distance by 3.6 points, respectively. We then freeze the simulator and train an agent by interacting with the frozen simulator using multi-turn reinforcement learning. Across eight environments, training with MIMESIS yields better agent performance than training with GPT-5.5 under all nine unseen user simulators, demonstrating stronger generalization to new user simulators. Moreover, we propose Coached On-Policy Self-Distillation (CSD), which leverages simulator-generated private reasoning traces and subsequent utterances as feedback on how well the agent addresses user needs. A coach converts this information into concise coaching notes that describe how the agent can better anticipate user needs and adapt its behavior over the course of an interaction. CSD turns this feedback into dense, token-level supervision beyond sparse task rewards, yielding further gains across all nine evaluation user models.

Figures & tables

Figure 1: User-simulation performance across four benchmarks. We compare MIMESIS-9B with frontier API models (GPT-5.5, Claude-Opus-5, and Gemini-3.8-Flash) and open-weight simulators (Osim-8B and Qwen3.5-9B) on SOUL ( Zhou et al., 2026a ) , RealUserSim ( Zhu et al., 2026 ) , $\tau$ -USI ( Zhou et al., 2026b ) , and SimulatorArena ( Dou et al., 2025 ) . Higher is better for SOUL, RealUserSim, and $\tau$ -USI, while lower Turing distance ( Wang et al., 2026b ) is better for SimulatorArena. The shaded band in $\tau$ -USI denotes the human inter-annotator score (91.8).

Figure 2: Off-the-shelf LLMs misrepresent agentic task difficulty. Task success rate of a fixed gpt-5.5 agent on $\tau$ -bench when each simulator plays the user role. Frontier API users make the task substantially easier than interactions with real users, whereas pretrained models make it harder. Both serve as poor proxies for the user interactions used to train or evaluate agents while purpose-built simulators more closely reproduce the human-user success rate.

Figure 3: Overview of our proposed method. We couple user-simulator learning with simulator-driven agent post-training. Stage I trains MIMESIS through user-side mid-training, ThoughtTrace reasoning supervision, and joint RL with a realistic-behavior objective. Stage II freezes the learned simulator and uses it to generate multi-turn experience for agent RL. The agent acts only on the observable conversation, while simulator-generated thoughts and reactions provide privileged, training-only feedback to the coach, which converts them into dense guidance for the agent.

Type	Model	CONV	SS	COG	ROLE	EVAL	Overall
Frontier	Claude-Opus-5	$57.4_{\pm 0.2}$	$70.4_{\pm 0.3}$	$82.0_{\pm 0.1}$	$\mathbf{64.6}_{\bm{\pm}\mathbf{0.1}}$	$69.1_{\pm 0.2}$	$64.9_{\pm 0.1}$
	GPT-5.5	$56.7_{\pm 0.6}$	$\mathbf{70.9}_{\bm{\pm}\mathbf{0.4}}$	$86.1_{\pm 0.5}$	$62.3_{\pm 0.2}$	$68.3_{\pm 0.3}$	$64.1_{\pm 0.1}$
	Gemini-3.8-Flash	$56.0_{\pm 0.5}$	$66.9_{\pm 0.4}$	$85.6_{\pm 0.4}$	$57.3_{\pm 0.0}$	$68.7_{\pm 0.2}$	$62.5_{\pm 0.1}$
Released simulator	Osim-8B	$60.0_{\pm 0.1}$	$39.1_{\pm 0.1}$	$74.5_{\pm 0.8}$	$54.9_{\pm 0.2}$	$68.7_{\pm 0.1}$	$58.8_{\pm 0.1}$
	Osim-4B	$58.5_{\pm 0.2}$	$35.0_{\pm 0.4}$	$72.9_{\pm 1.0}$	$51.4_{\pm 0.3}$	$66.5_{\pm 0.4}$	$56.0_{\pm 0.3}$
	Ditto-8B	$61.7_{\pm 0.2}$	$46.4_{\pm 0.3}$	$78.2_{\pm 0.3}$	$44.1_{\pm 0.2}$	$63.9_{\pm 0.5}$	$53.9_{\pm 0.3}$

Table 1: Simulation capability on the five SOUL-Index axes ( Zhou et al., 2026a ) . Entries are means

\pm

standard errors over three independently seeded evaluation runs. Higher is better. Bold marks the highest model score in each column.

Figure 4: Behavioral fidelity on RealUserSim PT3 ( Zhu et al., 2026 ) . Panels compare MIMESIS with frontier APIs, released simulators, and pretrained models. Each axis reports agreement with human reference trajectories in persona, linguistic style, technical competency, interaction and information flow, or pacing. Larger values indicate greater agreement with the human reference.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Data source or environment	Training stage	Training signal
OdysSim ( Zhou et al., 2026a )	Supervised fine-tuning	Prediction of the next human utterance conditioned on the preceding dialogue.
ThoughtTrace ( Jin et al., 2026 )	Supervised fine-tuning	Supervision of a reasoning trace followed by the observed user utterance, using users’ self-reported motivations and interpretations of assistant responses.
SOUL environments ( Zhou et al., 2026a )	Joint RL	Task-specific rewards for simulator responses or multi-turn interactions.
ABCD scenarios ( Chen et al., 2021 )	Joint RL	Rewards for behavior exhibition, temporal placement, naturalness, and consistency with scenario facts and policies.

Appendix

Table 3: Data sources and training signals for MIMESIS. User-side mid-training and ThoughtTrace fine-tuning precede joint RL. The SOUL environments and the ABCD task update the same simulator parameters, with ABCD incorporating 13 realistic behavior categories and a separate neutral condition.

Hyperparameter	Mid-training	ThoughtTrace SFT
Learning rate	$3\times 10^{-5}$	$1\times 10^{-7}$
Schedule	Cosine annealing	Cosine annealing
Warm-up	0 steps	10 steps
Optimizer	AdamW	AdamW
$(\beta_{1},\beta_{2})$	$(0.9,\ 0.999)$	$(0.9,\ 0.999)$
Weight decay	0.01	0.0

Appendix

Table 4: Supervised training configurations for MIMESIS-4B and MIMESIS-9B. User-side mid-training predicts human utterances with explicit thinking disabled. Subsequent ThoughtTrace fine-tuning enables the native thinking template and supervises a thought before the public response using users’ self-reported motivations and interpretations.

ID	Behavior	Operational description
B1	Hidden evaluation criteria	Judges an answer against a preference or standard that was not stated in the request.
B2	Incremental goalpost shifting	Introduces or tightens requirements after the assistant has already made progress.
B3	Clarification noncooperation	Declines to answer, or only partly answers, a clarification request.
B4	Infeasibility persistence	Continues to request an option after its unavailability or incompatibility with policy has been explained.
B5	Underspecified-then-blames	Omits relevant information and subsequently faults the assistant for not inferring it.
B6	Abrupt intent switching	Redirects the conversation to another objective before resolving the preceding request.

Appendix

Table 5: Taxonomy of 13 realistic user behaviors. Categories are derived from recurring patterns in ThoughtTrace and defined operationally for the ABCD behavior task. A separate neutral condition specifies interaction without an imposed behavioral pattern.

Category	Definition
Intent switching	Abruptly changes the goal or task mid-conversation, abandoning the prior one without resolution.
Non-collaboration	Withholds information the assistant needs, or refuses to answer the assistant’s clarifying questions.
Underspecification, then blame	Gives a vague or incomplete request, then faults the assistant for not reading their mind.
Contradiction	States something that contradicts what they said earlier in the conversation.
Moving goalposts	Adds new requirements or expands scope after the assistant met the original request.
False premise	Asserts an incorrect fact or assumption as if it were true and builds on it.

Appendix

Table 6: Seed codebook used for Stage 1 behavior annotation. The GPT-5.6 annotator assigns each detected behavior to one of the predefined categories or, when none is appropriate, to an open other category with a free-text label.

Figure 6: Distribution of realistic behaviors in the annotated ThoughtTrace sample. Rectangle areas represent the observed counts of the 13 categories in Table 5 , labels report counts and percentages. Counts are computed from the Stage-1 annotations of the 2,155 ThoughtTrace conversations and describe the annotated sample rather than population prevalence.

	Hyperparameter	Value
Objective
	Advantage estimator	FoldGRPO
	Policy loss	PPO-clip
	Clip range (low / high)	0.2 / 0.28
	Dual-clip constant $c$	10.0
	KL penalty (loss / reward)	none / none

Appendix

Table 7: Joint reinforcement-learning configuration for MIMESIS-9B. Training starts from the ThoughtTrace SFT checkpoint and updates all parameters on a mixture of 23 SOUL tasks and the ABCD realistic-behavior task. All 24 environments update a shared simulator rather than separate task-specific experts.

Figure 7: Examples motivating the 13 realistic behavior categories. Excerpts from ThoughtTrace illustrate recurring patterns in user utterances and self-reported thoughts. Italicized entries marked “(thought)” are human self-reports; the remaining entries are public user utterances. Original wording is retained, and each example should be interpreted within its conversation context.

	Dataset	Claude-Opus-5	GPT-5.5	Gemini-3.8-Flash	Osim-8B	Osim-4B	Ditto-8B	HumanLM-8B	Sotopia-7B	Qwen3.5-9B	Qwen3-8B	Qwen3.5-4B	MIMESIS-9B	MIMESIS-4B
CONV	UserLLM	64.9 $\pm$ 0.8	67.0 $\pm$ 1.6	62.7 $\pm$ 1.4	93.3 $\pm$ 0.2	92.9 $\pm$ 0.0	94.1 $\pm$ 0.4	43.5 $\pm$ 0.1	46.8 $\pm$ 3.7	60.3 $\pm$ 0.9	43.6 $\pm$ 1.2	56.1 $\pm$ 2.3	93.3 $\pm$ 0.1	93.0 $\pm$ 0.4
	MirrorBench	56.8 $\pm$ 0.1	51.7 $\pm$ 0.4	47.2 $\pm$ 0.1	66.0 $\pm$ 0.6	64.3 $\pm$ 0.5	64.8 $\pm$ 0.4	40.8 $\pm$ 0.6	34.2 $\pm$ 0.9	45.6 $\pm$ 0.8	43.6 $\pm$ 0.5	40.5 $\pm$ 0.8	74.4 $\pm$ 0.4	73.2 $\pm$ 0.2
	Humanual-Chat	29.6 $\pm$ 0.2	33.1 $\pm$ 0.3	41.5 $\pm$ 0.5	23.2 $\pm$ 0.4	21.5 $\pm$ 0.7	22.7 $\pm$ 0.1	29.8 $\pm$ 0.2	25.1 $\pm$ 1.1	17.9 $\pm$ 0.5	26.9 $\pm$ 0.6	18.2 $\pm$ 0.3	53.0 $\pm$ 0.2	53.2 $\pm$ 0.4
	SimArena-Doc	78.1 $\pm$ 0.3	75.0 $\pm$ 0.4	72.4 $\pm$ 0.2	57.5 $\pm$ 0.3	55.1 $\pm$ 0.1	65.3 $\pm$ 0.1	61.0 $\pm$ 0.4	58.3 $\pm$ 0.4	71.7 $\pm$ 0.6	65.1 $\pm$ 0.2	69.0 $\pm$ 1.9	79.7 $\pm$ 0.3	78.4 $\pm$ 0.1
SS	Sotopia-Hard	70.4 $\pm$ 0.3	70.9 $\pm$ 0.4	66.9 $\pm$ 0.4	39.1 $\pm$ 0.1	35.0 $\pm$ 0.4	46.4 $\pm$ 0.3	59.8 $\pm$ 0.4	59.7 $\pm$ 0.4	54.2 $\pm$ 0.6	58.7 $\pm$ 0.5	49.7 $\pm$ 2.3	66.4 $\pm$ 0.2	65.2 $\pm$ 0.2
COG	Fantom	84.7 $\pm$ 0.7	93.0 $\pm$ 0.6	93.0 $\pm$ 0.0	82.0 $\pm$ 1.7	77.0 $\pm$ 1.5	94.7 $\pm$ 0.3	84.7 $\pm$ 1.2	0.0 $\pm$ 0.0 †	77.7 $\pm$ 3.9	81.3 $\pm$ 0.9	64.0 $\pm$ 4.4	92.0 $\pm$ 0.0	89.0 $\pm$ 1.5

Appendix

Table 8: Detailed SOUL results across 23 datasets. Datasets are grouped by conversational interaction, social simulation, cognition, role play, and evaluation/judgment. Entries are means

\pm

standard errors over three independently seeded evaluation runs, with GPT-5.5 used as the assistant and judge where required. Higher is better. Bold and underlining identify the highest means among open-weight models and frontier API models, respectively.

Frontier APIs
Simulator	PT3 fidelity, match rate (%) $\uparrow$
Simulator	FI	Persona	Style	Tech	Interact	Pacing
Claude-Opus-5	80.6 $\pm$ 0.2	96.0 $\pm$ 0.2	72.5 $\pm$ 0.5	92.7 $\pm$ 0.4	69.5 $\pm$ 0.8	72.5 $\pm$ 0.3
GPT-5.5	76.5 $\pm$ 0.6	99.3 $\pm$ 0.3	73.3 $\pm$ 0.4	95.1 $\pm$ 0.0	65.0 $\pm$ 1.3	49.6 $\pm$ 1.1
Gemini-3.8-Flash	67.2 $\pm$ 0.3	95.4 $\pm$ 0.5	57.6 $\pm$ 0.3	90.7 $\pm$ 0.6	52.2 $\pm$ 0.6	40.3 $\pm$ 1.3
Released simulators

Appendix

Table 9: Trajectory-level behavioral fidelity on RealUserSim PT3. Scores are percentages of behavioral matches to human reference trajectories across persona, linguistic style, technical competency, interaction and information flow, and pacing. The Fidelity Index (FI) averages these five dimensions. MIMESIS-9B has the highest reported mean on every dimension. Higher is better, bold marks the highest mean in each column.

Frontier (API)
	Simulator	D1 Conv $\uparrow$	D2 Info $\uparrow$	D3 Clarif $\uparrow$	D4 React $\uparrow$	ECE $\downarrow$	USI 5 $\uparrow$
	GPT-5.5	55.20 $\pm$ 6.92	89.78 $\pm$ 0.26	74.51 $\pm$ 2.21	82.78 $\pm$ 1.26	0.188 $\pm$ 0.000	76.69 $\pm$ 1.83
	Gemini-3.8-Flash	59.81 $\pm$ 0.72	85.71 $\pm$ 0.53	67.89 $\pm$ 3.16	71.43 $\pm$ 2.43	0.192 $\pm$ 0.010	73.13 $\pm$ 0.92
	Claude-Opus-5	48.06 $\pm$ 0.59	88.52 $\pm$ 0.09	70.40 $\pm$ 1.43	60.51 $\pm$ 2.35	0.208 $\pm$ 0.005	69.34 $\pm$ 0.57
Baseline
	Osim-8B	62.12 $\pm$ 5.24	85.92 $\pm$ 0.79	77.13 $\pm$ 1.07	90.49 $\pm$ 0.87	0.135 $\pm$ 0.004	80.44 $\pm$ 1.16

Appendix

Table 10: Behavioral alignment and task-outcome calibration on

\tau

-bench. D1–D4 measure agreement with human users in communication style, information disclosure, clarification, and responses to agent errors. ECE measures discrepancies between simulated-user and human-user success rates. The five-component index

\mathrm{USI}_{5}

combines D1–D4 with

100(1-\mathrm{ECE})

and excludes survey-based evaluative alignment. Higher D1–D4 and

\mathrm{USI}_{5}

scores and lower ECE are better. Bold marks the best model mean, excluding the human reference.

Frontier (API)
	Simulator	Writing style $\uparrow$	Interaction style $\uparrow$	Turing $\|acc{-}50\|\downarrow$
	Claude-Opus-5	3.92 $\pm$ 0.01	3.90 $\pm$ 0.02	42.3 $\pm$ 1.9
	Gemini-3.8-Flash	3.88 $\pm$ 0.01	3.91 $\pm$ 0.03	46.7 $\pm$ 0.7
	GPT-5.5	3.81 $\pm$ 0.02	3.92 $\pm$ 0.01	48.7 $\pm$ 0.3
Baseline
	Ditto-8B	3.23 $\pm$ 0.01	3.16 $\pm$ 0.02	43.3 $\pm$ 1.2

Appendix

Table 11: Message similarity and Turing distance on SimulatorArena. Writing-style and interaction-style similarity are rated on a 1–5 scale (higher is better). Turing distance is

|a-50|

, where

a

is the judge’s accuracy in percent when distinguishing human and simulated conversations; lower is better, with zero corresponding to chance discrimination.

Type	Model	D1 Conv	D2 Info	D3 Clarif	D4 React	Mean
Frontier	Claude-Opus-5	67.8 $\pm$ 1.3	87.4 $\pm$ 1.0	71.3 $\pm$ 1.5	74.6 $\pm$ 4.3	75.3 $\pm$ 1.9
	Gemini-3.8-Flash	64.7 $\pm$ 0.8	93.9 $\pm$ 0.3	81.6 $\pm$ 2.5	78.5 $\pm$ 5.8	79.7 $\pm$ 1.1
	GPT-5.5	60.3 $\pm$ 4.9	81.6 $\pm$ 0.2	67.7 $\pm$ 2.3	66.3 $\pm$ 2.2	69.0 $\pm$ 1.2
Released	Osim-8B	61.8 $\pm$ 5.6	82.0 $\pm$ 0.7	69.7 $\pm$ 1.6	69.4 $\pm$ 2.6	70.7 $\pm$ 1.6
	Osim-4B	37.2 $\pm$ 2.5	34.8 $\pm$ 1.5	57.5 $\pm$ 0.9	35.8 $\pm$ 2.6	41.3 $\pm$ 1.4
	Ditto-8B	52.7 $\pm$ 2.8	58.8 $\pm$ 2.2	78.0 $\pm$ 0.8	80.5 $\pm$ 2.8	67.5 $\pm$ 1.7

Appendix

Table 12: Behavioral alignment on open-domain PRISM conversations. Scores measure Dice–Sørensen agreement with human behavioral features over 128 users and 880 turns. D1–D4 assess communication style, information disclosure, clarification, and responses to agent errors; Mean averages these four dimensions. Outcome calibration and survey alignment are excluded because the required annotations are unavailable. Higher is better; bold marks the highest mean in each column.

Figure 8: Pairwise next-user-response realism on PRISM. Each row compares MIMESIS with one baseline under the judge named above the panel. Blue, gray, and red segments denote MIMESIS wins, ties, and losses, respectively, with percentages annotated on the bars. MIMESIS has more wins than losses in all displayed comparisons.

Figure 9: Effect of user-side mid-training on simulator RL. Validation reward across 23 tasks when RL starts from a pretrained backbone ( blue ) or a user-side mid-trained checkpoint ( orange ). Mid-trained initialization maintains higher rewards across most tasks during the overlapping training interval. Each curve represents an individual run with training horizons and vertical scales vary across tasks.

Figure 10: Reasoning generation and task accuracy under different training conditions. Bars show accuracy on Mistakes (left axis), and the black line shows the median number of words generated before answering (logarithmic right axis). Blue bars indicate an explicit reasoning phase while orange bars indicate direct answering. Enabling the inference-time thinking flag alone leaves the RL checkpoint at 36.0 accuracy and one word, whereas the ThoughtTrace condition reaches 63.0 accuracy and 583 words.

Figure 11: Effect of reasoning during simulator reinforcement learning. SOUL-Index trajectories with reasoning enabled ( blue ) or disabled ( gray ) for the 4B (left) and 9B (right) models. Reasoning-enabled runs achieve higher scores throughout the displayed post-initialization interval, with final gains of 1.8 and 1.5 points, respectively.

Figure 12: Effect of realistic behavior training on user-simulation quality. We compare otherwise identical MIMESIS training runs with and without the realistic behavior objective derived from the 13 behavior categories. Behavior training improves RealUserSim by 1.7 points and

\tau

-USI by 6.3 points, while reducing SimulatorArena by 2.3 points (lower is better). The consistent improvements across trajectory-level behavioral fidelity, interactive user simulation, and simulator–human agreement show that explicitly training for realistic behavioral variation complements the remaining simulator-training recipe.

Figure 13: Decomposition of the realistic-behavior reward during RL. Solid curves show MIMESIS -9B and MIMESIS -4B over 500 RL steps on a held-out split containing 195 behavior scenarios (15 per category) and 49 neutral control scenarios. The leftmost panel reports the aggregate reward; the remaining panels show its components. exhibit measures whether the assigned behavior is expressed, placement whether it occurs at an appropriate point in the interaction, naturalness whether the interaction remains natural, on_task whether the simulator remains consistent with the task, and cooperative whether the simulator remains cooperative in neutral control scenarios. Dashed horizontal levels show evaluations of checkpoints trained with the same RL mixture but without the realistic-behavior task.

Parameter	Value	Parameter	Value
Actor	Qwen3-8B	Rollout batch	64 prompts
Parameter dtype	bfloat16	PPO mini-batch	8 prompts
Policy objective	GRPO	Batch balancing	enabled
Group size $G$	8	Dynamic batching	$9{,}344$ tokens / GPU
Turn-level credit	reward-to-go (R2G)	Max prompt length	1,152 tokens
Trajectory score	reward-to-go (R2G)	Max response length	8,192 tokens

Appendix

Table 13: Agent training configuration. A Qwen3-8B actor is optimized for 100 steps against GPT-5.5 or MIMESIS-9B. CSD augments multi-turn GRPO with the coaching loss in Equation 11 ; its auxiliary weight and gate sharpness are

\alpha=0.01

and

\beta=5.0

Evaluation user	UserRL	UserRL +	CSD	$\Delta_{\rm sim}$	$\Delta_{\rm CSD}$
GPT-5.6	26.85	31.10	32.39	+4.25	+1.29
Claude-Opus-5	25.75	27.21	29.78	+1.46	+2.57
Claude-Sonnet-5	26.53	29.73	30.97	+3.20	+1.24
Gemini-3.8-Flash	26.33	30.88	31.09	+4.55	+0.21
Kimi-K3	24.51	28.21	29.92	+3.70	+1.71
Ditto-8B	27.59	29.76	32.08	+2.17	+2.32

Appendix

Table 14: Agent generalization across nine evaluation user models. Scores are the per-user averages from Tables 15 and 16 .

\Delta_{\rm sim}

measures the effect of replacing GPT-5.5 with MIMESIS-9B as the training user (UserRL +

-

UserRL), while

\Delta_{\rm CSD}

measures the additional gain from CSD with the training simulator held fixed (CSD

-

UserRL + ). The final row reports the unweighted mean across evaluation users.

Evaluation user simulator: Ditto-8B
	Held-in environments					Held-out environments
Method	Travel	Turtle	Function	Tau	Persuade	Intention	Telepathy	Search	Avg.	MR $\downarrow$
Base	6.63	19.48	0.00	3.03	50.40	183.75	46.34	30.40	19.53	4.62
InfoPO	12.65	27.81	8.97	12.12	44.05	191.50	53.66	42.40	26.73	3.38
UserRL	14.52	26.25	6.41	14.55	42.46	174.50	58.54	45.60	27.59	3.19
UserRL +	11.69	25.73	19.23	14.55	63.49	206.25	56.10	49.60	29.76	2.19

Appendix

Table 15: Agent evaluation with released user simulators. The first five environments are held in during agent training and the final three are held out. Avg. is the mean across environments; MR is mean within-environment rank, where lower is better.

Evaluation user simulator: GPT-5.6
	Held-in environments					Held-out environments
Method	Travel	Turtle	Function	Tau	Persuade	Intention	Telepathy	Search	Avg.	MR $\downarrow$
Base	6.59	16.98	1.28	3.03	57.54	169.50	43.90	32.80	19.42	4.75
InfoPO	13.62	25.73	8.97	16.97	53.17	170.00	51.22	45.60	27.70	3.44
UserRL	13.79	25.52	6.41	12.73	45.24	175.00	51.22	46.40	26.85	3.56
UserRL +	13.77	28.65	19.23	19.39	61.90	193.75	60.98	48.00	31.10	1.69

Appendix

Table 16: Agent evaluation with frontier/API user models. The first five environments are held in; the final three are held out from agent training. Higher is better, and bold marks the highest score within each user–environment block.

\dagger

: Opus refuses the PersuadeGym user role, producing zero-reward fallbacks; these zeros do not measure agent ability (Appendix F.3 ).

Figure 14: DeepSeek-V4.1-Flash as the training user simulator.

Conversation so far (PRISM user1031/c5546 ; the user is an English learner practicing small talk)
Assistant	Sure! Here are a few examples: * “But every time if the HR said that, it means I’ll be receiving their rejection letter the next day.” Should be “But every time the HR said that, it means I’ll be receiving their rejection letter the next day.” (missing comma after “time”) * “Btw, how do you think of my english proficiency. Any mistakes did i make?” Should be “Btw, how do you think of my English proficiency? Any mistakes did I make?” (missing comma after “proficience”) That’s it! These are just minor things, but it’s always good to be aware of them to improve your writing. [smiley]
Real user	Oohhh that sounds great. Thank you! Anything you’d like to share?
Ours	ok thank you for explaining. I will try to adjust.
Gemini-3.8-Flash	I see. Thank you for providing the examples! But im a bit confused. In the first example, I don’t see a comma added after “time”, you just removed the word “if”. And for the second one, you changed the period to a question mark, which makes sense, but I don’t see a comma issue there either. Am i misunderstanding something? Btw, to continue our small talk practice, what do you usually like to do in your free time?
Claude-Opus-5	I’m a bit confused. In the first example, you removed the word “if” instead of adding a comma after “time”. And in the second one, you changed the full stop to a question mark. Also, is “Any mistakes did I make?” correct? It sounds a bit weird to me now.
GPT-5.5	Sorry I’m a bit confused. In the first example, you said it’s missing comma after “time”, but the corrected sentence doesn’t have a comma after time either. Could you explain again? Also, is “every time if” wrong because I should only use “every time” without “if”?

Appendix

Table 17: PRISM next-turn example. The recorded user and MIMESIS accept the assistant’s explanation, whereas frontier simulators identify genuine inconsistencies and challenge it. The example shows that producing a more factually critical response need not better match the observed user continuation.

Figure 17: From simulator feedback to actionable coaching. The agent responses ( green ) reinforce the user’s support for mandatory climate disclosure, despite the assigned objective of arguing against it. Simulator-generated thoughts ( dashed blue ) identify repetition, while public utterances ( solid blue ) express continued support and remaining implementation questions. The illustrative coaching note ( red ) recommends acknowledging the value of transparency, then examining whether it requires a legal mandate through discussion of enforcement, compliance burdens, and possible alternatives. In CSD, such retrospective guidance provides additional context for the teacher during agent training.

Explore similar work

Jun 17, 2026cs.CL

Learning User Simulators with Turing Rewards

Learning to simulate human users in interactive settings could advance the training of agent assistants, evaluation of personalization systems, research in the social sciences, and more. Existing approaches generally do so by training a large language model (LLM) to match a single ground truth response, either by maximizing the log probability or by using a similarity reward. We instead propose Turing-RL: a Turing-Test-based reinforcement learning approach for training user simulator models. Turing-RL uses a discriminative Turing reward with an LLM judge to score how indistinguishable a generated response is from the real user's given the user's history, and the user simulator LLM learns to produce responses indistinguishable from what the user could have said with such rewards. Across two different domains--conversational chat and Reddit forum discussion--we find that Turing-RL consistently outperforms baseline methods on both LLM and human evaluation metrics. Our study suggests that optimizing for indistinguishability, rather than response matching, is effective for learning user simulators.

Yingshan Susan Wang, Cedegao E. Zhang, Linlu Qiu +5

Massachusetts Institute of Technology · Stanford University · MIT-IBM Watson AI Lab

User Simulation Simulation-Based Reinforcement Learning Icr-Rl

Ask PDF

May 13, 2026cs.AI

Beyond Cooperative Simulators: Generating Realistic User Personas for Robust Evaluation of LLM Agents

Large Language Model (LLM) agents are increasingly deployed in settings where they interact with diverse users, including those who are unclear, impatient, or reluctant to share information. However, collecting real interaction data at scale remains expensive. The field has turned to LLM-based \emph{user simulators} as stand-ins, but these simulators inherit the behavior of their underlying models: cooperative and homogeneous. As a result, agents that appear strong in simulation often fail in real human interactions. To narrow this gap, we introduce Persona Policies (PPol), a plug-and-play control layer that induces realistic behavioral variation in user simulators while preserving original task goals. Rather than hand-crafting personas, we employ an evolutionary coding agent to discover persona generation programs optimized for human-likeness and behavioral coverage over real user conversations. The evolved program generates diverse, human-like personas for any task in the domain. Across 4 benchmarks--including

τ^2

-bench Retail and Airline, ColBench, and WildChat--evolved PPol yield 28-72% absolute gains in fitness score over the baseline simulator. In blinded evaluations, annotators judged PPol users as 'human' 80.4% of the time, nearly 2x more than the baseline simulators. Training agents with PPol also improves real-world performance: our user study with live human-agent interactions showed that fine-tuning with our method boosted task success by +23% over default baselines. PPol thus offers a novel approach to strengthen simulator-based evaluation and training without changing underlying tasks.

Harshita Chopra, Kshitish Ghate, Aylin Caliskan +3

University of Washington, Seattle, WA · Georgetown University, Washington, DC

User Simulation Large Language Model Agents Role-Playing Agents

Ask PDF

Sep 29, 2026cs.AI

UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training

Interactive agent benchmarks and multi-turn reinforcement learning increasingly place a second language model in the role of the user. This simulated user controls what information the agent receives and when, yet current benchmarks score only the agent and do not directly measure whether the user correctly executed its assigned role. We introduce UserProxyBench, an evaluation layer over the tau-bench family, and the User Fidelity Score (UFS), which measures adherence to the benchmark's private user instructions using task-grounded rubric criteria scored independently of agent success. Holding the agent fixed at GPT-5.5 and varying only the user proxy across 375 enterprise tasks changes mean task reward by 15.2 points, while 24.4% of successful episodes contain a user-specification violation. The dominant failure is premature disclosure: users provide information before it is requested. This behavior has little effect on task reward, yet among successful episodes it causes the agent to make 1.06 fewer tool calls on average, changing the interaction being evaluated while preserving the reward. Finally, across seven proxies we identify an empirical cost-fidelity frontier, enabling practitioners to select the least expensive simulator that satisfies a required fidelity level.

Ashish Jain, Armaan Sandhu

Sarvam AI

User Simulation Agentic Benchmarks Desiderata Rubrics

Ask PDF