Web agents increasingly serve the same user across repeated online tasks such as booking travel, shopping, and scheduling, so their long-term usefulness depends on how well they know the user's preferences. Passive memory and personalization methods cannot obtain preferences the user never reveals, and clarification methods that value a question by its benefit to the current task overlook the benefit of reusing the answer in later tasks. We study Active Preference Acquisition (APA): which missing preferences are worth the cost of asking? We define the Prospective Personalization Value (PPV) of a missing preference as its expected cumulative utility gain over future tasks minus the effort and disclosure-risk cost of asking, and select questions under a per-interaction budget. We characterize when cumulative reuse justifies acquisition despite insufficient single-task benefit, and evaluate this regime with an initial-profile diagnostic. APA computes the counterfactual gain of each candidate preference on public task prototypes with the agent's own task solver and aggregates it by the expected reuse of each task family. We evaluate it on ActiveWebPref, a controlled longitudinal benchmark with hidden but queryable user profiles and exact task utilities. At a horizon of 20 tasks, 34.4% of budget-feasible valuable missing preferences are amortization-dependent; APA acquires 80.3% of them, whereas current-task value of information acquires 5.1%. Under per-user asking-cost caps matched to APA's realized expenditure, APA achieves higher net utility than four heuristic selection baselines. In our cost setting, asking about every unknown preference yields lower net utility than not asking, while APA recovers 84.2% of its utility gain with 27.6% of its questions. The value of acquiring a user preference should account for its reuse across future tasks, not only its benefit to the task at hand.
Figures & tables
Figure 1. A single use does not justify asking, but cumulative reuse does. In session t the user asks for a hotel, and the seat preference m is unknown and useless for this task. Each later seat selection gains from knowing m , but with illustrative numbers, a gain of 0.05 per use against an asking cost of 0.15, no single use justifies asking, so a criterion that values only the task at hand does not ask even when a flight is booked. Four uses gain 0.20, so acquiring m once and reusing it pays off. Timeline from session t to t+3. In session t the user books a hotel; a missing seat preference has no immediate value. In sessions t+1 to t+3 flight, conference-travel and business-trip bookings each require selecting a seat; each use alone gains less than the cost of asking, but repeated reuse gains more.
Figure 2. Overview of APA. ① Gap discovery proposes preferences Gt missing from the profile Pt . ② Future reuse modeling estimates how often each may be reused, from a coarse distribution over task families. ③ Prospective value estimation scores each by relevance and impact. ④ Cost-aware acquisition subtracts asking cost and disclosure risk and chooses Ask , Defer , or Skip . Here, the seat preference is asked; the dining preference, rarely reused because restaurant tasks are rare, is deferred for later re-scoring, as is the hotel budget once the seat question has used this interaction’s budget; and a previously refused political preference, which never enters the candidate set, is skipped. Answers update the profile for future tasks. Four-stage pipeline: gap discovery, future reuse modeling, a relevance-impact matrix for value estimation, and a cost-aware policy that asks about the seat preference, defers the rarely reused dining preference (and the hotel budget once the interaction budget is spent), and skips a previously refused political preference; answers update the persistent profile.
Statistic
Travel
Dining
Shopping
Scheduling
Work
# users (train/dev/test)
18/18/36
18/18/36
18/18/36
18/18/36
18/18/36
# task families
4
3
3
3
3
# attributes / user
27
27
27
27
27
% hidden at t=0
53.1
53.1
53.1
53.1
53.1
% high-sensitivity
11.1
11.1
11.1
11.1
11.1
mean uses / used field
1.687
2.000
1.862
1.780
2.178
Table 1. ActiveWebPref statistics by primary-user domain. Test combines 90 IID + 90 unseen users; each cell aggregates its 36 test users unless train/dev counts are shown. Attribute inventories are shared across domains; hidden fractions use all 27 fields. Structural use is counted over the first T=10 future tasks.
Policy
CTU ↑
AC ↓
RC ↓
NU ↑
#Q ↓
Unn. ↓
Regret ↓
Ask-Nothing
6.289
0.000
0.000
6.289
0.000
–
0.769
Ask-Everything
7.263
3.977
1.428
4.918
13.472
0.728
2.147
LLM-Importance
7.080
2.455
0.883
5.631
11.294
0.741
1.427
Heuristics with per-user asking cost capped at APA’s realized expenditure
Random- k (cost-matched)
6.585
0.654
0.096
6.234
3.696
0.698
0.824
Frequency (cost-matched)
6.748
0.659
0.072
6.400
3.783
0.592
0.658
Table 2. Main results on ActiveWebPref: Overall = 90 IID + 90 unseen users, T=10 , B=0.5 , λ=0.5 , β=0.25 , K=10 . CTU: cumulative task utility; AC/RC: asking/disclosure risk cost; NU: net utility; #Q: questions; Unn.: unnecessary query rate. Cost-matched rows obey both the per-interaction budget and a per-user cap on total asking cost (Appendix B ); Random- k averages five seeds. Pointwise 95% CIs appear in Table 9 ; paired tests are in Table 10 .
Policy
NU ↑
Δ NU
pHolm
Acq.
Ask-Nothing
5.706
+0.679
0.004
0%
Ask-Everything
4.534
+1.852
0.004
–
LLM-Importance
5.347
+1.039
0.004
–
Random- k (cost-matched)
5.815
+0.570
0.004
–
Frequency (cost-matched)
6.020
+0.365
0.004
–
Impact-only (cost-matched)
6.271
+0.114
0.109
–
Table 3. Amortization-enriched test set (60 users, T=10 , B=0.5 ): NU, paired difference APA minus each policy with Holm p over the eight non-oracle baselines, and acquisition of its 202 amortization-dependent pairs.
T=10 : 91 pairs, 71 users (36 IID + 35 unseen)
T=20 : 198 pairs, 120 users (63 IID + 57 unseen)
Policy
Acquired
Before use
Attr. gain
Acquired
Before use
Attr. gain
Ask-Nothing
0.0% (0)
0.0% (0)
0.000
0.0% (0)
0.0% (0)
0.000
Ask-Everything
100.0% (91)
100.0% (91)
0.254
100.0% (198)
100.0% (198)
0.576
LLM-Importance
100.0% (91)
35.2% (32)
0.204
100.0% (198)
49.0% (97)
0.431
Current-task VoI (public)
3.3% (3)
0.0% (0)
-0.001
5.1% (10)
0.0% (0)
0.017
APA
70.3% (64)
33.0% (30)
0.227
80.3% (159)
46.5% (92)
0.307
Table 4. Initial-profile amortization diagnostic: feasible pairs with cm≤B=0.5 , λ=0.5 , β=0.25 , K=10 . Overall cohorts contain at least one such pair. Rates divide by all designated pairs, including unacquired pairs; counts appear in parentheses.
T=10 : 71 users (36 IID + 35 unseen)
T=20 : 120 users (63 IID + 57 unseen)
Policy
MVMR ↓
Δ CTU ↑
NU ↑
MVMR ↓
Δ CTU ↑
NU ↑
Ask-Nothing
1.000
0.000
6.291
1.000
0.000
12.437
Ask-Everything
0.000
1.044
4.988
0.000
2.098
12.153
LLM-Importance
0.027
0.903
5.726
0.041
1.864
12.817
Current-task VoI (public)
0.459
0.516
6.639
0.463
1.231
13.462
APA
0.188
0.868
6.787
0.150
1.862
13.725
Table 5. Whole-sequence outcomes on the budget-feasible initial-profile amortization cohorts: each user has at least one designated pair. B=0.5 , λ=0.5 , β=0.25 , K=10 . MVMR is the missed valuable rate (Appendix B ); Δ CTU is the paired-cohort mean gain over Ask-Nothing.
Figure 3. Acquisition cost versus CTU: full range (a) and detail (b). Overall = 90 IID + 90 unseen users, T=10 , K=10 , β=0.25 ; all B∈{0.25,0.5,0.75,1} and selection λ∈{0,0.1,0.25,0.5,1,2} settings. Points retain the saved mean coordinates; the APA step line is its observed configuration envelope. Two side-by-side panels show acquisition cost versus CTU, with the full range at left and a magnified central region at right.
Figure 4. Acquisition cost versus NU: full range (a) and detail (b). Overall = 90 IID + 90 unseen users, T=10 , K=10 , β=0.25 ; all B∈{0.25,0.5,0.75,1} and selection λ∈{0,0.1,0.25,0.5,1,2} settings. Points retain the saved mean coordinates; the APA step line is its observed configuration envelope. NU uses common evaluation prices λ=0.5 , β=0.25 . Two side-by-side panels show acquisition cost versus NU, with the full range at left and a magnified central region at right.
Figure 5. Horizon scaling: paired NU(APA) minus NU(Current-task VoI (public)) on the IID + unseen test set (180 users) and the amortization-enriched test set (60 users); B=0.5 , λ=0.5 , β=0.25 , K=10 , nested saved prefixes. Bands and error bars are pointwise 95% paired-user bootstrap CIs. Paired net-utility differences over five horizons.
Variant
NU ↑
Unn. ↓
APA (full)
6.755
0.387
No FutureDist
6.760
0.342
Next-Task Only
6.683
0.315
No Cost ( λ=0 )
6.498
0.557
No Risk ( β=0 )
6.727
0.397
Gold Candidate Schema
6.753
0.378
Table 6. Ablations: Overall 180 users (IID + unseen), T=10 , B=0.5 , K=10 . NU always uses evaluation λ=0.5 , β=0.25 ; No Cost/No Risk change only the selection coefficient. No FutureDist uses uniform task-family weights. Next-Task Only values the actual next task instance, a privileged lookahead without hidden answers. Gold Candidate Schema bypasses K with the complete publicly eligible missing-field pool.
Figure 6. What each policy chooses to learn: successful acquisitions divided by initially unknown fields in each class, pooled over 90 IID + 90 unseen users, T=10 , B=0.5 , λ=0.5 , β=0.25 , K=10 . Classes overlap; no ephemeral fields occur (N/A). High-sensitivity fields are refused. Acquisition fractions for stable, semi-stable, ephemeral, distractor and high-sensitivity fields.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Policy
CTU ↑
AC ↓
RC ↓
NU ↑
#Q ↓
Unn. ↓
Regret ↓
Ask-Nothing
6.289
0.000
0.000
6.289
0.000
–
0.776
Ask-Everything
7.263
3.977
1.428
4.918
13.472
0.728
2.147
Random- k
6.605
0.684
0.086
6.241
3.981
0.700
0.824
Frequency
7.248
3.977
1.428
4.903
13.472
0.736
2.162
Impact-only
7.250
3.977
1.428
4.904
13.472
0.742
2.161
LLM-Importance
7.206
3.977
1.428
4.860
13.472
0.739
2.204
Appendix
Table 7. Full-policy comparison at B=1 : Overall = 90 IID + 90 unseen users, B=1 , λ=0.5 , β=0.25 , T=10 , K=10 . Ask-Everything and Oracle Future-VoI (unlimited) have no budget cap; Random- k uses five seeds. These measurements use B=1 , unlike Table 2 ( B=0.5 ).
Policy
CTU ↑
AC ↓
RC ↓
NU ↑
#Q ↓
Unn. ↓
Regret ↓
Random- k (cost-matched)
6.585
0.654
0.096
6.234
3.696
0.698
0.824
Frequency (cost-matched)
6.748
0.659
0.072
6.400
3.783
0.592
0.658
Impact-only (cost-matched)
7.037
0.666
0.083
6.683
3.744
0.462
0.376
LLM-Importance (cost-matched)
6.603
0.660
0.339
6.188
3.011
0.704
0.870
APA (ours)
7.109
0.680
0.058
6.755
3.717
0.387
0.304
Appendix
Table 8. Cost-matched comparison: Overall = 90 IID + 90 unseen users; B=0.5 , λ=0.5 , β=0.25 , T=10 , K=10 . Each control pays every attempted asking cost and obeys both the per-arrival cap and its user's realized APA total-cost cap. Random- k uses five seeds.
Metric
Ask- Nothing
Ask- Everything
LLM- Importance
Current-task VoI (public)
APA (ours)
Oracle Future-VoI
Random- k (cost-matched)
Frequency (cost-matched)
Impact-only (cost-matched)
LLM- Importance (cost-matched)
CTU ↑
6.289 [6.112, 6.455]
7.263 [7.142, 7.382]
7.080 [6.947, 7.205]
6.853 [6.717, 6.984]
7.109 [6.985, 7.233]
7.265 [7.148, 7.380]
6.585 [6.426, 6.739]
6.748 [6.592, 6.895]
7.037 [6.901, 7.170]
6.603 [6.448, 6.751]
AC ↓
0.000 [0.000, 0.000]
3.977 [3.851, 4.101]
2.455 [2.382, 2.528]
0.379 [0.351, 0.410]
0.680 [0.642, 0.720]
0.400 [0.366, 0.436]
0.654 [0.614, 0.692]
0.659 [0.622, 0.698]
0.666 [0.629, 0.706]
0.660 [0.622, 0.698]
RC ↓
0.000 [0.000, 0.000]
1.428 [1.339, 1.514]
0.883 [0.808, 0.956]
0.000 [0.000, 0.000]
0.058 [0.036, 0.081]
0.028 [0.011, 0.047]
0.096 [0.057, 0.137]
0.072 [0.047, 0.103]
0.083 [0.058, 0.111]
0.339 [0.294, 0.386]
NU ↑
6.289 [6.112, 6.455]
4.918 [4.772, 5.067]
5.631 [5.491, 5.776]
6.664 [6.524, 6.793]
6.755 [6.626, 6.880]
7.058 [6.939, 7.175]
6.234 [6.071, 6.394]
6.400 [6.240, 6.549]
6.683 [6.547, 6.819]
6.188 [6.029, 6.341]
#Q ↓
0.000 [0.000, 0.000]
13.472 [13.183, 13.750]
11.294 [11.044, 11.550]
2.044 [1.878, 2.206]
3.717 [3.522, 3.917]
2.122 [1.950, 2.311]
3.696 [3.451, 3.936]
3.783 [3.572, 4.000]
3.744 [3.544, 3.961]
3.011 [2.822, 3.206]
Unn. ↓
–
0.728 [0.713, 0.743]
0.741 [0.722, 0.758]
0.314 [0.263, 0.366]
0.387 [0.350, 0.426]
0.029 [0.015, 0.044]
0.698 [0.667, 0.728]
0.592 [0.552, 0.633]
0.462 [0.420, 0.503]
0.704 [0.664, 0.743]
Appendix
Table 9. Overall main results with pointwise 95% user-bootstrap CIs: 180 pooled IID/unseen users, T=10 , B=0.5 , λ=0.5 , β=0.25 , K=10 ; 2,000 replicates, seed 20261003. No-question Unn. is undefined. Cost-matched controls follow Table 8 ; Random- k CIs resample five seeds, and Unn. uses 179 defined users.
Experiment
Comparator
Δ NU
95% CI
p
pHolm
Main comparison
Ask-Nothing
0.466
[0.349, 0.583]
0.0005
0.0075
Main comparison
Ask-Everything
1.837
[1.750, 1.922]
0.0005
0.0075
Main comparison
LLM-Importance
1.123
[1.049, 1.197]
0.0005
0.0075
Main comparison
Current-task VoI (public)
0.091
[0.019, 0.174]
0.0205
0.0615
Main comparison
Oracle Future-VoI
-0.304
[-0.342, -0.265]
0.0005
0.0075
Main comparison
Random- k (cost-matched)
0.521
[0.425, 0.635]
0.0005
0.0020
Appendix
Table 10. Paired Overall NU contrasts (APA minus comparator). Holm families comprise 15 main and 15 ablation contrasts across IID, unseen and Overall (IID + unseen). The amortization family has two Overall contrasts: T=10 (71 paired users) and T=20 (120 paired users), each using 2,000 paired resamples (seed 20261003). Settings match the corresponding main, amortization subset or ablation experiment. The cost-matched rows form a separate four-test Overall Holm family (180 paired users; 2,000 replicates, seed 20261003).
T
Δ NU
95% CI
pHolm
1
-0.011
[-0.021, -0.003]
0.2664
3
-0.021
[-0.049, 0.007]
1.0000
5
-0.004
[-0.042, 0.036]
1.0000
10
0.091
[0.019, 0.174]
0.2664
20
0.175
[0.047, 0.327]
0.2474
Appendix
Table 11. Overall horizon contrasts, APA minus Current-task VoI (public); B=0.5 , λ=0.5 , β=0.25 , nested prefixes, 180 users. Pointwise CIs; Holm adjustment across 15 horizon contrasts.
Main comparison ( n=180 )
Amortization ( T=10 , n=71 )
Amortization ( T=20 , n=120 )
Policy
NDCG
Spearman
NDCG
Spearman
NDCG
Spearman
Ask-Nothing
–
–
–
–
–
–
Ask-Everything
–
–
–
–
–
–
LLM-Importance
0.608
0.020
0.625
0.032
0.657
0.023
Current-task VoI (public)
0.550
0.072
0.538
0.057
0.584
0.164
APA (ours)
0.681
0.215
0.669
0.176
0.691
0.296
Appendix
Table 12. Ranking diagnostics for the main comparison and the initial-profile amortization cohorts: main comparison, Overall 180 IID/unseen users; T=10 , B=0.5 , λ=0.5 , β=0.25 , K=10 . Amortization cohorts contain 71 users at T=10 and 120 at T=20 , with the same B , λ , β and K . Means are over users with defined diagnostics; – indicates no recorded ranking diagnostic.
T
Δ NU
95% CI
pHolm
1
−0.017
[ −0.040 , 0.002]
0.364
3
−0.015
[ −0.066 , 0.034]
1.000
5
0.030
[ −0.057 , 0.129]
1.000
10
0.354
[0.194, 0.533]
0.0025
20
1.296
[0.786, 1.834]
0.0025
Appendix
Table 13. Horizon contrasts on the amortization-enriched test set: APA minus Current-task VoI (public), 60 paired users, pointwise 95% CIs, Holm over five horizons.
Variant, test set
T=1
3
5
10
20
APA-v2, IID+unseen
0.034
0.038
0.043
−0.007
−0.013
APA-v2 (2 × cost), IID+unseen
0.005
0.031
0.001
−0.003
0.068
APA-v2, amortization
0.023
0.021
0.026
0.035
−0.013
APA-v2 (2 × cost), amortization
0.009
−0.024
−0.004
−0.088
0.102
Appendix
Table 14. Mean paired NU difference of APA-v2 variants to APA by horizon (180 IID + unseen or 60 amortization-enriched users; main settings). Means only.
Policy
CTU ↑
AC ↓
RC ↓
NU ↑
#Q ↓
Unn. ↓
Regret ↓
Ask-Nothing
6.430
0.000
0.000
6.430
0.000
–
0.691
Ask-Everything
7.360
3.966
1.411
5.024
13.522
0.726
2.098
LLM-Importance
7.202
2.463
0.883
5.750
11.378
0.744
1.371
Current-task VoI (public)
6.959
0.383
0.000
6.768
2.067
0.332
0.353
APA (ours)
7.191
0.694
0.056
6.830
3.800
0.387
0.291
Oracle Future-VoI
7.328
0.399
0.028
7.121
2.133
0.028
0.000
Appendix
Table 15. IID: 90 users, T=10 , B=0.5 , λ=0.5 , β=0.25 , K=10 . Definitions and regret references match Table 2 . Cost-matched rows use the per-user caps of Table 8 ; Random- k averages five seeds.
Policy
CTU ↑
AC ↓
RC ↓
NU ↑
#Q ↓
Unn. ↓
Regret ↓
Ask-Nothing
6.149
0.000
0.000
6.149
0.000
–
0.847
Ask-Everything
7.166
3.988
1.444
4.811
13.422
0.731
2.196
LLM-Importance
6.957
2.447
0.883
5.513
11.211
0.737
1.483
Current-task VoI (public)
6.747
0.376
0.000
6.560
2.022
0.295
0.436
APA (ours)
7.027
0.665
0.061
6.680
3.633
0.387
0.316
Oracle Future-VoI
7.203
0.401
0.028
6.996
2.111
0.029
0.000
Appendix
Table 16. Unseen users: 90 users, T=10 , B=0.5 , λ=0.5 , β=0.25 , K=10 . Definitions and regret references match Table 2 . Cost-matched rows use the per-user caps of Table 8 ; Random- k averages five seeds. Cost-matched Unn. excludes one no-question user (89 defined users).
Figure 7. Overall sensitivity (90 IID + 90 unseen users): performance (top: CTU, NU, regret) and behavior (bottom: AC, RC, questions). Each grid has β∈{0,0.25,0.5,1} by column and B∈{0.25,0.5,0.75,1} by line style; λ∈{0,0.1,0.25,0.5,1,2} on each horizontal axis. T=10 , K=10 . Shading shows pointwise 95% CIs. NU uses each cell's own λ and β . All saved means and CI endpoints are retained. Two compact Overall grids show all budget and coefficient combinations for APA and Oracle Future-VoI.
A long-lived LLM agent, such as OpenClaw, earns its value by acting on a user's preferences and constraints across sessions, not just the current request. Yet today's agents keep what a user volunteers but rarely ask for what stays unspoken, leaving a proactivity gap in long-lived LLM agents: an agent cannot act on a preference it never obtained. As users delegate more of their affairs to agents, the impact of this gap grows. We isolate one concrete, controllable slice of this gap as Ask-to-Remember (ATR): the agent decides whether to ask now for a reusable user preference that the current task does not need but a later session with the same user will. ATR is hard even to evaluate: the right question is underdetermined and its payoff deferred to tasks that may never arise. ATRBench, to the best of our knowledge the first ATR benchmark, makes it measurable by fixing each user's preferences as hidden ground truth, so success demands asking, not recall. Across eight frontier LLM agents, defaults fall at least 62 points below an oracle handed the relevant preference, and prompting closes little of it. Diagnostics identify acquisition as the bottleneck. ATRBench surfaces this proactivity gap in current agents and offers a diagnostic testbed for closing it.
Bin Wu, Guanyun Zou, Bingbing Wang +2
Beijing University of Posts and Telecommunications · Nanjing University of Aeronautics and Astronautics · Harbin Institute of Technology (Shenzhen) +1
Large language models have advanced web agents, yet current agents lack personalization capabilities. Since users rarely specify every detail of their intent, practical web agents must be able to interpret ambiguous queries by inferring user preferences and contexts. To address this challenge, we present Persona2Web, the first benchmark for evaluating personalized web agents on the real open web, built upon the clarify-to-personalize principle, which requires agents to resolve ambiguity based on user history rather than relying on explicit instructions. Persona2Web consists of: (1) user histories that reveal preferences implicitly over long time spans, (2) ambiguous queries that require agents to infer implicit user preferences, and (3) a reasoning-aware evaluation framework that enables fine-grained assessment of personalization. We conduct extensive experiments across various agent architectures, backbone models, history access schemes, and queries with varying ambiguity levels, revealing key challenges in personalized web agent behavior. For reproducibility, our codes and datasets are publicly available at https://serin-kimm.github.io/Persona2Web/.
Serin Kim, Sangam Lee, Dongha Lee
Department of Artificial Intelligence, Yonsei University, Seoul, Republic of Korea.
Large language model (LLM) agents have demonstrated strong performance on complex web navigation tasks, yet they remain brittle in real-world settings where user intentions are underspecified and preferences are heterogeneous. In practice, users rarely provide explicit profiles, requiring agents to infer latent preferences from implicit signals. Despite its importance for deployment, this problem setting is largely underexplored in existing benchmarks. To address this gap, we introduce AdaptArena, a benchmark for evaluating test-time personalization of web agents via implicit preference inference. AdaptArena consists of 480 tasks, featuring both single-preference and double-preference scenarios. Each evaluation task must be solved by retrieving and leveraging the most relevant historical user trajectory that implicitly encodes the target preference. In addition, we introduce AdaptiveAgent, a retrieval-based framework for standardized evaluation of implicit preference inference. Experiments reveal a substantial performance gap: while oracle agents with access to ground-truth preferences achieve an 82.92% success rate, the evaluated LLM agents using our framework reach at most 15.62%. Furthermore, we find that correctly inferring user preferences is necessary but not sufficient for task success, as execution failures in downstream web interactions remain a significant bottleneck even when agents align with the target preference. These findings highlight implicit preference inference and robust action grounding as key challenges for deploying reliable, user-facing web agents. We release our code: https://github.com/McGill-NLP/web-agents-test-time-adaptations
Dongchan Shin, Xing Han Lù, Jiaqi Deng +7
Mila - Quebec AI Institute · McGill University · ServiceNow Research +2