User simulators are essential for the scalable training and evaluation of interactive AI systems. However, existing approaches often rely on shallow user profiling, struggle to maintain persona consistency over long interactions, and are largely limited to English or single-domain settings. We present MUSE, a multi-domain Chinese user simulation framework designed to generate human-like, controllable, and behaviorally consistent responses. First, we propose Iterative Profile Self-Evolution (IPSE), which gradually optimizes user profiles by comparing and reasoning discrepancies between simulated trajectories and real dialogue behaviors. We then apply Role-Reversal Supervised Fine-Tuning to improve local response realism and human-like expression. To enable fine-grained behavioral alignment, we further train a specialized rubric-based reward model and incorporate it into rubric-guided multi-turn reinforcement learning, which optimizes the simulator at the dialogue level and enhances long-horizon behavioral consistency. Experiments show that MUSE consistently outperforms strong baselines in both utterance-level and session-level evaluations, generating responses that are more realistic, coherent, and persona-consistent over extended interactions.
Figures & tables
Figure 1: MUSE training pipeline overview. We first employ Iterative Profile Self-Evolution to extract persona-consistent profiles from real dialogues, then apply Role-Reversal SFT for human expression alignment. A two-stage rubric-based reward model is trained for fine-grained feedback, followed by Rubric-Guided multi-turn RL to optimize behavioral alignment in multi-turn interactions.
Domain
Source Dataset
Size (Sessions)
General Chat
WildChat
2,572
Customer Service
JDDC
1,293
Medical
MedDialog
1,481
Legal
Legal QA
1,476
Technology & Education
NaturalConv
1,472
Sports & Entertainment
NaturalConv
1,482
Table 1: Statistics of the constructed User Simulation Dataset across six domains.
Panel A: Utterance-Level Evaluation
Model
Objective Metrics
LLM-Judge Metrics
AI Prob. ↓
Style Sim. ↑
AVA ↑
Contxt. Relv. ↑
Resp. Fid. ↑
Goal Contr. ↑
Ling. Nat. ↑
UserLM
69.92
0.5888
55.38
0.5111
0.5289
0.4742
0.5655
USP
43.14
0.6644
59.25
0.6384
0.6874
0.5779
0.7314
Qwen3-8B
44.45
0.7307
64.69
0.8644
0.9102
0.7888
0.8976
GPT-4o
45.50
0.7181
62.79
0.8891
0.9221
0.8304
0.9214
Table 2: Main results on utterance-level and session-level evaluation. Best results are highlighted in bold .
Method
Human Likeness
Persona Consistency
Context Coherence
Overall
Acc
EM
Acc
EM
Acc
EM
Acc
EM
Qwen3-8B
0.7087
0.4400
0.6875
0.4250
0.6725
0.4375
0.6896
0.4342
Qwen3-8B-SFT w/o CoT
0.9087
0.8200
0.8525
0.7550
0.8712
0.8050
0.8775
0.7933
Qwen3-8B-SFT w/ CoT
0.9125
0.8325
0.8725
0.7850
0.8888
0.8300
0.8912
0.8158
Qwen3-8B-RL w/o CoT
0.8975
0.8175
0.8612
0.7525
0.9012
0.8550
0.8867
0.8083
Qwen3-8B-RL w/ CoT
0.9213
0.8425
0.8862
0.7950
0.9126
0.8600
0.9067
0.8333
Table 3: Ablation results of different reward model training strategies. Best results are highlighted in bold .
Method
Extraction Model
PPL ↓
Single-pass Profile Extraction
Gemini-3-Flash
85.97
Single-pass Profile Extraction
DeepSeek-V3
77.86
IPSE Framework (Ours)
DeepSeek-V3
73.12
Table 4: Perplexity (PPL) of different profile extraction methods on the test set.