On the Web, user profiling builds compact user representations from interaction histories for efficient personalization, and dynamic user profiling updates them as users' behavior and preferences change. However, new user behavior can be ambiguous, and its interpretation differs across users. To address this, we introduce Uncertainty-aware Adaptive Profile Manager (UAPM), which separates the claims in a user profile from interpretations of new behavior that are not yet confirmed, through a Profile, a Buffer, and a Rulebook. UAPM records user-specific interpretation and update rules in the Rulebook, keeps ambiguous evidence in the Buffer outside the Profile as evidence for later decisions, and writes supported general and situational characteristics to the Profile. We train UAPM with Privileged On-Policy Distillation with Dual Specialization (POPD-DS), which learns from recorded future interactions during training without access to them at deployment. Its first stage, Dual Specialization, distills a stronger model into a current-only Student and a future-informed model that becomes the Teacher. In the second stage, privileged on-policy distillation (POPD), only the fixed Teacher receives future interactions and guides the Student's own management decisions alongside task rewards. Experiments on three Web personalization tasks show that UAPM outperforms multiple baselines under the same downstream input budget, improving on the best same-budget baseline by up to 5%.
Figures & tables
Figure 1. A new interaction can often be read in more than one way when it arrives. Committing one reading at once can leave a wrong claim in the Profile that misleads later predictions, whereas holding it back limits this contamination. A fantasy fan rates a fantasy drawing guide 3/5. Committing at once claims the user is cooling on fantasy, so a later fantasy novel rated 5/5 is predicted lower. Holding the reading in the Buffer, under a user rule to wait for a second similar title, lets a later drawing guide rated 3/5 support that drawing guides are rated lower.
Figure 2. UAPM on a user stream. Left: chunks arrive in time order. Right: one management step at turn t . The downstream task model predicts the responses in Ct from the previous Profile Pt−1 . UAPM reads the previous state, observed outcomes, and task feedback to update the Profile, Buffer, and Rulebook. The updated Profile Pt is used on the next chunk Ct+1 . Management loop with the Profile, Buffer, and Rulebook, where only the Profile reaches the downstream task model.
Figure 3. Dual Specialization followed by Privileged On-Policy Distillation (POPD). Stage 1 distills a stronger model’s trajectories without and with future interactions into two models, a current-only Student and a future-informed model, respectively. Stage 2 updates the Student with the task reward and guidance from the future-informed model as the fixed Teacher. Two stages: specialization of a current-only Student and a future-informed model, followed by Student updates using the future-informed model as a fixed Teacher, token-distribution matching, and task reward.
Dataset
Task
Unit
DS
POPD
DTM
Val
Test
LP
Amazon Books
rating
users
400
400
133
50
50
128
instances
42,000
42,000
13,965
5,250
5,250
LaMP-2
tagging
users
129
129
43
51
77
128
instances
3,759
3,794
1,170
1,380
2,222
LaMP-QA
generation
users
400
400
133
50
50
256
instances
10,000
10,000
3,325
1,250
1,250
Table 1. Dataset statistics. DS, POPD, and DTM are the disjoint training subsets for Dual Specialization, POPD, and the downstream task model; LP is the Profile budget in tokens.
Method
Amazon Books
LaMP-2
LaMP-QA
MAE ↓
RMSE ↓
Acc ↑
Macro-F1 ↑
Score ↑
No Personalization
0.7431 ± 0.000
0.9730 ± 0.000
0.4614 ± 0.000
0.3746 ± 0.000
0.3976 ± 0.000
Recent History
0.6213 ± 0.000
0.9199 ± 0.000
0.4909 ± 0.000
0.4202 ± 0.000
0.3619 ± 0.000
Static user profiling
Prompting
0.6629 ± 0.005
1.0073 ± 0.005
0.5006 ± 0.002
0.4429 ± 0.002
0.3673 ± 0.002
RLPF
0.6550 ± 0.005
0.9937 ± 0.005
0.5017 ± 0.001
0.4456 ± 0.002
0.4023 ± 0.002
Table 2. Main results. Methods above the double rule use the same backbone and Profile budget. Values with subscripts are means and standard deviations over three sampling runs. † denotes a statistically significant improvement over the best baseline above the double rule (paired t-test over test users on run-averaged metrics, p<0.05 ).
Method
Amazon Books
LaMP-2
LaMP-QA
MAE ↓
RMSE ↓
Acc ↑
Macro-F1 ↑
Score ↑
Stage 1 target generator with current input (not ranked)
Stronger Model (GPT-6.1-sol)
0.5889
0.9297
0.5205
0.4698
0.4038
Student with current input
Student Prompting
0.7075
0.9342
0.4670
0.4142
0.3695
Student Distillation
0.5884
0.9236
0.5165
0.4543
0.3919
Table 3. Training stage and role specialization ablation. Future-informed rows see future input and are not ranked.
Condition
Amazon Books
LaMP-2
LaMP-QA
MAE ↓
RMSE ↓
Acc ↑
Macro-F1 ↑
Score ↑
UAPM
0.5801
0.9152
0.5313
0.4742
0.4163
UAPM w/o Buffer and Rulebook
0.5961
0.9432
0.4818
0.4150
0.3834
UAPM w/o Rulebook
0.6044
0.9347
0.4852
0.4169
0.3965
UAPM w/o Buffer
0.5904
0.9080
0.4864
0.4188
0.3902
Table 4. State component ablation. Best and second-best results are in bold and underlined.
Setting
Amazon Books
LaMP-2
LaMP-QA
MAE ↓
RMSE ↓
Acc ↑
Macro-F1 ↑
Score ↑
Divergence
Forward KL
0.5915
0.9227
0.5250
0.4627
0.3993
JSD
0.6004
0.9357
0.5290
0.4755
0.3937
Reverse KL ( UAPM )
0.5801
0.9152
0.5313
0.4742
0.4163
Mixing weight α
Table 5. POPD analysis: divergence, mixing weight, and Teacher input. Best and second-best are marked per group.
Table 9
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Method
Calls
Amazon Books
LaMP-2
LaMP-QA
In
Out
In
Out
In
Out
Dynamic Profile Prompting
1
466
186
353
74
1,316
353
TRIPLE
5
2,676
350
1,919
291
6,873
339
MemGPT
5.6–6.3
20,227
980
18,062
849
21,139
1,096
UAPM
1
2,702
308
2,817
185
3,337
114
Appendix
Table 8. Manager cost per turn on the test streams.