User simulators enable interactive training and evaluation of AI assistants at scale. Yet current simulators struggle to reproduce the diverse behaviors of real users, leading to assistants that generalize poorly to underrepresented users. We identify user representations as an underexplored bottleneck in user simulation. User representations are natural-language descriptions that condition simulators to reproduce a particular user's behavior. A representation's quality becomes apparent in hindsight, through the behavior it induces in a simulator. To evaluate and optimize user representations, we formulate a conversation reconstruction task: given an original conversation, we generate a representation, condition a simulator on it, simulate a conversation, and then measure fidelity by comparing user behavior in the original and simulated conversations. This comparison also yields hindsight feedback, which identifies where the representation falls short and how it can be improved. We introduce hindsight distillation, an online training procedure that distills feedback-conditioned improvements into the representation generator, enabling it to generate improved representations directly from the original conversation. We present UserRepGen-4B, a representation generator trained with hindsight distillation. Across 700 conversations spanning seven datasets, hindsight feedback helps improve user representations at test time, and UserRepGen-4B generates representations that enable more faithful simulation than those generated by GPT-5.5. Beyond conversation reconstruction, learned representations improve simulation fidelity for unseen tasks, better preserve the population-level diversity of real users, and better distinguish users across conversations.
Figures & tables
Figure 1: User representations are a bottleneck in user simulation. They capture the information a simulator needs to faithfully reproduce a user’s behavior. We train UserRepGen-4B to generate high-quality user representations and demonstrate their importance for reconstructing user conversations, simulating users in new scenarios, simulating diverse user populations, and distinguishing users across conversations.
Figure 2: Hindsight distillation for Gθ . A representation is sampled, and is improved with I rounds of hindsight feedback to get r(I) , which serves as a training target.
Figure 3: Simulation fidelity across iterations of hindsight feedback. Shaded regions show 95% bootstrap confidence intervals over conversations.
Model
Best-of-1
Best-of-3
Qwen3.5-4B
3.41
4.42
+ GRPO
3.57 ↑0.16
4.59 ↑0.17
+ SFT
4.64 ↑1.23
5.68 ↑1.26
+ OPSD
4.46 ↑1.05
5.60 ↑1.18
+ OSFT
4.95 ↑1.54
6.02 ↑1.60
Qwen3.5-9B
3.72 ↑0.31
4.68 ↑0.26
Table 1: Simulation fidelity across user representation generators. We report Best-of-1 and Best-of-3 scores. Subscripts indicate absolute improvements over the Qwen3.5-4B, and darker blue indicates stronger scores.
Model
Task
Behavior
Total
R-L (%)
Qwen3.5-4B
61
101
163
27.2
+ GRPO
61
101
162
27.8
+ SFT
58
126
185
25.8
+ OPSD
72
111
183
25.3
+ OSFT
62
148
210
23.9
Table 2: Quantitative analysis of user representations. We report the average word counts of the task, behavior and complete user representations, as well as ROUGE-L precision against the source conversation.
Figure 4: Simulation fidelity when conditioned under configurations \scriptsizeA⃝, \scriptsizeB⃝, \scriptsizeC⃝, and \scriptsizeD⃝. Each cell reports Best-of-3 scores. Subscripts indicate absolute improvement relative to the condition immediately to the left.
Task Only
Task + Behavior
Model
JS↓
SB ↓
JS↓
SB ↓
Qwen3.5-4B
0.389
0.257
0.315
0.227
+ GRPO
0.356
0.256
0.338
0.217
+ SFT
0.353
0.264
0.270
0.196
+ OPSD
0.326
0.239
0.308
0.202
+ OSFT
0.354
0.246
0.273
0.192
Table 3: Distributional gap between original and simulated user behaviors for our different representation generators when the simulator is conditioned on the task description alone or both the task and behavior descriptions. We report ( JS ) divergence and Self-BLEU (SB). Darker shades of blue and lower values indicate better performance.
Representation Generator
Accuracy (%)
Full conversation (baseline)
44.22
Qwen3.5-4B
38.78 ↓5.44
+ GRPO
41.50 ↓2.72
+ SFT
43.54 ↓0.68
+ OPSD
50.34 ↑6.12
+ OSFT
51.02 ↑6.80
Table 4: GPT-5.4 mini accuracy at distinguishing users across conversations. Subscripts indicate performance compared to the full conversation baseline, and darker blue indicates higher accuracy.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Score gap
Agreement
Rate
1
33 / 41
80.5%
2
17 / 20
85.0%
≥3
38 / 39
97.4%
Overall
88 / 100
88.0%
Appendix
Table 5: Agreement between our manually labeled preferences and the LLM judge. The score gap is the absolute difference between the two simulated conversations’ overall fidelity scores.
Hyperparameter
GRPO
OPSD
SFT
OSFT
Learning rate
5×10−6
5×10−6
5×10−6
5×10−6
Effective batch size
32
32
32
32
Training steps
100
100
100
100
LoRA rank ( r )
32
64
64
64
LoRA alpha ( α )
64
128
128
128
Maximum prompt length
8,192
8,192
8,192
8,192
Appendix
Table 6: Training hyperparameters for GRPO, OPSD, SFT, and OSFT.
Question
UserRepGen-4B Selected (%)
Task description
Captures constraints introduced in later turns
63.6
Captures shifts in direction or goal
61.0
Captures multiple objectives or subtasks
58.3
Has less repetition
33.1
Behavior description
Appendix
Table 7: Comparing user representations generated by our trained and untrained models along several dimensions. We report the percentage of times that GPT-5.4 mini selects the representation generated by UserRepGen-4B over the one generated by Qwen3.5-4B.