Agentic-Priors: Using Large Language Models to Address the Cold Start Problem in Preference Elicitation

Bruno Arcanjo; Zhaoxing Li; Sebastian Stein

cs.MAOct 15, 2026

Agentic-Priors: Using Large Language Models to Address the Cold Start Problem in Preference Elicitation

Authors: Bruno Arcanjo, Zhaoxing Li, Sebastian Stein

Organizations: University of Southampton, Southampton, United Kingdom

Abstract

An agent that learns a user's preferences from their choices faces a cold start problem: before any choices are observed, it must rely solely on a prior. Informative priors are usually fitted to historical choice data, which new tasks and deployments lack. To address this, we present Agentic-Priors, which lets agentic systems use the implicit knowledge of large language models (LLMs) instead. Specifically, an LLM states a new user's trade-offs in interpretable terms, such as willingness to pay or value of time, which we map to a prior over multinomial logit coefficients. This prior is then refined by a Bayesian learner as choices are observed. Across seven choice datasets, Agentic-Priors improves cold start accuracy over an uninformed prior by 5-40 percentage points. It also outperforms AutoElicit, a state-of-the-art LLM prior-elicitation method, on three datasets and loses on one. It is up to 18 points more accurate than in-context learning, and never significantly worse. Against hierarchical Bayes models fitted to surveys of increasing size, our approach is worth roughly 6-40 surveyed users, in terms of log-loss, so its value is concentrated in a deployment's first users.

Figures & tables

Figure 1. Agentic-Priors overview. An LLM, given the choice task and (optionally) the user’s characteristics $z_{i}$ , states interpretable quantities such as willingness to pay. These are mapped to multinomial logit (MNL) coefficients that form a personalised prior $\beta_{i}\sim\mathcal{N}(\mu_{i}^{\mathrm{LLM}},\sigma^{2}I_{K})$ , which a Bayesian learner updates as choices are observed. Unlike a hierarchical Bayes prior, it needs no historical choice data. A flow diagram of the Agentic-Priors pipeline.

Panel of Figure 2 — Figure 2. LLM input for one Swissmetro user (abridged: definitions shortened). Prompts for all datasets are in Appendix A . A boxed example of the two messages sent to the LLM for one Swissmetro user: a system message setting the analyst role, and a user message with their profile and the three monetary quantities to estimate.

Dataset	Users	Tasks	Alternatives	Choice attributes	$z_{i}$
Swissmetro	1,191	9	Train, Swissmetro, Car a	Travel time, cost; Train and Car ASCs	Age, gender, income, trip purpose, travel class, luggage, who pays, rail season ticket
LPMC	1,304	6–16	Walk, cycle, public transport, drive	Mode-specific travel times, cost, PT interchanges; mode ASCs	Age, gender, travel year, driving licence, household car ownership, fare type, trip purpose
Cars / Cars CN	384 / 448	15	Three vehicles	Powertrain (HEV/PHEV/BEV vs gasoline), electric range, fast charging, price, operating cost, acceleration, brand origin	Age, gender, household income, marital status, education, household size, vehicles owned, daily mileage, home/work charging access, environmental and status attitudes; urbanicity (US) or city (China)
EV	983	6	Gasoline car, EV	EV indicator, price, range, walking time to home/work charger, fast-charge time, in-town fast-charger drive time and availability, highway charger spacing and availability	Age, gender, state, household income, education, car budget, daily mileage, home/work charging access, EV ownership
Beer	1,291	8	Two six-packs, no purchase	Packaging, water/carbon/landfill eco-labels, locally brewed, price; no-purchase ASC	Age, region, income, education, primary shopper, purchase frequency, typical price, usual beer type, novelty seeking, recycling
Bank	946	14–17	Two credit-card offers	Interest rate, reward programme, annual fee, issuing bank, rebate, credit line, grace period (levels disguised)	Age, gender, income

Table 1. Datasets. Tasks: choice tasks per user.

Dataset	Reference	Quantities stated
Swissmetro	Cost (CHF)	Value of time; Train and Car vs Swissmetro preference
LPMC	Cost (£)	Cycle, PT and drive vs walk; value of time per mode; avoiding an interchange
Cars	Price ($ / ¥)	Each powertrain vs gasoline; per unit of electric range; fast charging; lower operating cost; faster acceleration; each brand origin vs German
EV	Price ($)	EV vs gasoline; per mile of range; per minute of home/work walking, fast-charge and in-town charger time; in-town and highway charger availability; per mile of highway charger spacing
Beer	Price ($)	Bottles vs cans; each eco-label; locally brewed; highest price for a basic six-pack
Bank	Rating ( $\pm 100$ )	Desirability of each attribute level relative to its baseline

Table 2. LLM-stated quantities and their reference scale.

Accuracy
	Swissmetro	LPMC	Cars	Cars CN	EV	Beer	Bank
Uninformed a	25.8	21.6	33.6	36.3	48.5	35.1	51.0
AutoElicit	69.3	22.9	48.9	46.0	41.1	27.3	66.1
AutoElicit + $z_{i}$	68.5	45.7	51.5	44.7	41.1	35.3	65.7
Agentic-Priors	55.8	46.8	56.5	44.6	55.9	40.6	65.9
Agentic-Priors + $z_{i}$	62.5	61.9	52.3	43.6	53.5	48.2	62.7

Table 3. Cold start (

T=0

) accuracy (%, higher is better) and log-loss (lower is better) on the test users (

n=100

per dataset). Methods above the line use no historical choice data; “+

z_{i}

” methods are also given the user’s information. Best no-data method per column in bold.

Figure 3. Accuracy and log-loss as choices are observed. Dashed lines with open markers: methods given the user’s information

z_{i}

. Shaded: steps reached by only some users, whose points average a smaller, self-selected group. Fourteen line plots in two rows, accuracy above and log-loss below, one column per dataset, showing each method's performance as the number of observed choices grows from zero. The HBM curves are highest in accuracy and lowest in log-loss almost throughout; Agentic-Priors lies between them and the uninformed prior at the start, and in-context learning falls further behind on four datasets as choices are observed.

Figure 4. Cold start performance of HBM +

z_{i}

fitted to

N

surveyed users (

N=0

: uninformed prior; grey band: 95% interval).

N^{*}

: survey size at which the HBM overtakes Agentic-Priors +

z_{i}

(blue band: bootstrap interval;

N^{*}<n

: already ahead at the smallest survey that can be fitted). Arrows: values beyond the plotted range. Fourteen line plots in two rows, accuracy above and log-loss below, one column per dataset. Each shows the cold start performance of an HBM fitted to a growing number of surveyed users, from zero to one hundred, with horizontal lines for Agentic-Priors with and without user information and for AutoElicit. In log-loss, HBMs fitted on the smallest surveys lie far above the plotted range, marked by a black arrow at the top edge. The HBM curve overtakes the Agentic-Priors line within about 40 users on every dataset except Beer in accuracy (about 65).

Method	Historical choices	$z_{i}$
Uninformed prior	No	No
AutoElicit	No	No
AutoElicit + $z_{i}$	No	Yes
In-Context Learning + $z_{i}$	No	Yes
HBM	Yes	No
HBM + $z_{i}$	Yes	Yes

Table 4. Information required by each method.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

	Swissmetro	LPMC	Cars	Cars CN	EV	Beer	Bank
Uninformed	$-36.7^{*}$	$-40.3^{*}$	$-18.7^{*}$	$-7.3^{*}$	$-5.0^{*}$	$-13.1^{*}$	$-11.7^{*}$
	( $<$ 0.001)	( $<$ 0.001)	( $<$ 0.001)	(0.005)	(0.018)	( $<$ 0.001)	( $<$ 0.001)
AutoElicit	$+6.8^{*}$	$-38.9^{*}$	$-3.4$	$+2.4$	$-12.3^{*}$	$-20.9^{*}$	$+3.3$
	(0.049)	( $<$ 0.001)	(0.324)	(1.000)	(0.009)	( $<$ 0.001)	(0.091)
AutoElicit + $z_{i}$	$+5.9$	$-16.1^{*}$	$-0.8$	$+1.1$	$-12.3^{*}$	$-12.9^{*}$	$+2.9$
	(0.080)	( $<$ 0.001)	(0.723)	(1.000)	(0.009)	( $<$ 0.001)	(0.091)

Appendix

Table 5. Cold start (

T=0

) paired differences from Agentic-Priors +

z_{i}

: Accuracy (percentage points). Method minus Agentic-Priors +

z_{i}

, test users (

n=100

per dataset); in brackets, paired

t

-test

p

-values Holm-corrected over the 14 tests of each dataset (7 methods

\times

2 metrics); ∗ :

p<0.05

	Swissmetro	LPMC	Cars	Cars CN	EV	Beer	Bank
Uninformed	$+0.239^{*}$	$+0.278^{*}$	$+0.111^{*}$	$-0.049$	$+0.008^{*}$	$+0.050^{*}$	$+0.020^{*}$
	( $<$ 0.001)	(0.001)	( $<$ 0.001)	(1.000)	(0.008)	( $<$ 0.001)	( $<$ 0.001)
AutoElicit	$-0.032$	$+0.455^{*}$	$+0.231^{*}$	$+0.163$	$+0.587^{*}$	$+0.266^{*}$	$+0.085$
	(0.103)	( $<$ 0.001)	( $<$ 0.001)	(0.350)	( $<$ 0.001)	( $<$ 0.001)	(0.072)
AutoElicit + $z_{i}$	$-0.053^{*}$	$+0.325^{*}$	$+0.135^{*}$	$+0.015$	$+0.621^{*}$	$+0.197^{*}$	$+0.090$
	(0.049)	( $<$ 0.001)	(0.026)	(1.000)	( $<$ 0.001)	( $<$ 0.001)	(0.091)

Appendix

Table 6. Cold start (

T=0

) paired differences from Agentic-Priors +

z_{i}

: Log-loss (positive: worse than Agentic-Priors +

z_{i}

). Method minus Agentic-Priors +

z_{i}

, test users (

n=100

per dataset); in brackets, paired

t

-test

p

-values Holm-corrected over the 14 tests of each dataset (7 methods

\times

2 metrics); ∗ :

p<0.05

Prior	$T=0$	$T=1$	$T=3$	$T=6$
Stated ranges $\times 1$	66.5 / 0.883	67.1 / 0.877 ∗	68.5 / 0.862 ∗	70.1 / 0.842 ∗
Stated ranges $\times 4$	65.7 / 0.908 ∗	68.5 ∗ / 0.848 ∗	71.3 / 0.769 ∗	72.9 / 0.703 ∗
Stated ranges $\times 8$	63.3 / 0.939 ∗	63.7 / 0.830 ∗	70.1 / 0.713 ∗	72.9 / 0.634
Stated ranges $\times 16$	54.7 ∗ / 0.975 ∗	62.1 ∗ / 0.841 ∗	70.3 / 0.699	72.8 / 0.618
Stated ranges $\times 32$	46.2 ∗ / 1.007 ∗	61.1 ∗ / 0.879 ∗	69.6 / 0.733 ∗	72.1 / 0.652 ∗
Stated ranges $\times 64$	42.3 ∗ / 1.029 ∗	60.7 ∗ / 0.924 ∗	67.8 / 0.803 ∗	71.6 / 0.724 ∗

Appendix

Table 7. Using the LLM’s stated uncertainty as the prior’s spread. The LLM states a plausible range for each quantity; its implied standard deviation is multiplied by the factor shown. Swissmetro test users (

n=100

); accuracy (%) / log-loss after

T

observed choices. ∗ : paired difference from the single-elicitation prior with

\sigma=1

(last row) significant at

p<0.05

Prior	$T=0$	$T=1$	$T=3$	$T=6$
8 elicitations, $\sigma=1$	61.7 / 0.904 ∗	62.3 ∗ / 0.817 ∗	68.2 / 0.709 ∗	72.2 / 0.636 ∗
8 elicitations, own spread	70.1 ∗ / 0.821	70.4 ∗ / 0.809	71.5 / 0.786 ∗	72.1 / 0.765 ∗
Single elicitation, $\sigma=1$ (ours)	62.5 / 0.853	64.7 / 0.783	68.8 / 0.689	72.4 / 0.624

Appendix

Table 8. Repeated elicitation: eight elicitations per user (varied phrasing, temperature and seed), averaged, with either the same

\sigma=1

or the eight answers’ own spread as the covariance. Swissmetro test users (

n=100

); accuracy (%) / log-loss after

T

observed choices. ∗ : paired difference from the single-elicitation prior with

\sigma=1

(last row) significant at

p<0.05

Elicitor	Swissmetro		Cars		Bank
	Acc.	LL	Acc.	LL	Acc.	LL
Primary (Qwen3-32B, 4-bit, private)	62.5	0.853	52.3	0.989	62.7	0.672
Uninformed prior	25.8	1.092	33.6	1.101	51.0	0.693
mistralai/mistral-small-24b-instruct-2501	$-0.2$	$+0.017$	$-2.9$ ∗	$+0.033$ ∗	$+1.7$	$-0.005$ ∗
qwen/qwen3-30b-a3b-instruct-2507	$+0.5$	$+0.064$ ∗	$-0.3$	$-0.005$	$-1.5$	$+0.002$
qwen/qwen3.8-27b	$-0.9$	$+0.043$ ∗	–	–	–	–

Appendix

Table 9. Cold start (

T=0

) results with other elicitors, using identical prompts and no per-model tuning (test users,

n=100

per dataset). First two rows: accuracy (%) and log-loss of the primary elicitor and of the uninformed prior. Other rows: paired difference from the primary elicitor (accuracy in percentage points; positive log-loss is worse). ∗ :

p<0.05

, paired

t

-test, unadjusted. –: not run.

$T=0$
	Ours	Ours + HBM cov.	HBM + our cov.	HBM
Swissmetro	62.5 / 0.853	64.3 / 0.898	68.8 / 0.845	68.5 / 0.753
LPMC	61.9 / 1.128	60.8 / 1.208	72.3 / 0.915	71.1 / 0.762
Cars	52.3 / 0.989	53.4 / 1.010	64.6 / 0.806	64.0 / 0.797
Cars CN	43.6 / 1.150	43.4 / 1.159	49.9 / 0.998	50.5 / 0.985
EV	53.5 / 0.689	54.7 / 0.687	64.9 / 0.631	65.8 / 0.629

Appendix

Table 10. Covariance swap between Agentic-Priors +

z_{i}

(ours) and the HBM without

z_{i}

: accuracy (%) / log-loss of each prior, of our mean with the HBM’s covariance, and of the HBM’s mean with our covariance

\sigma^{2}I

(test users,

n=100

per dataset).

Log-loss
	$N=10$	$N=15$	$N=25$	$N=50$	$N=100$	$N=200$	$N=400$	All
Swissmetro	$-0.088^{*}$	$-0.117^{*}$	$-0.127^{*}$	$-0.146^{*}$	$-0.137^{*}$	$-0.137^{*}$	$-0.139^{*}$	$-0.145^{*}$
LPMC	$+0.208$	$-0.018$	$-0.297^{*}$	$-0.380^{*}$	$-0.353^{*}$	$-0.380^{*}$	$-0.430^{*}$	$-0.446^{*}$
Cars	$-0.170^{*}$	$-0.197^{*}$	$-0.217^{*}$	$-0.217^{*}$	$-0.214^{*}$	$-0.217^{*}$	–	$-0.214^{*}$
Cars CN	$-0.089$	$-0.110$	$-0.129$	$-0.146^{*}$	$-0.154^{*}$	$-0.164^{*}$	–	$-0.174^{*}$
EV	$+0.047^{*}$	$+0.004$	$-0.014$	$-0.044^{*}$	$-0.046^{*}$	$-0.055^{*}$	$-0.055^{*}$	$-0.058^{*}$

Appendix

Table 11. What a survey of

N

users adds to the LLM’s mean: our prior’s mean combined with the covariance of an HBM without

z_{i}

fitted on

N

users, minus that HBM, at

T=0

(positive: our mean is better; averaged over survey draws; ∗ :

p<0.05

, paired, unadjusted; –:

N

not below the pool).

	Log-loss $N^{*}$	Accuracy $N^{*}$	Smallest fit	Rejected	Pool
Swissmetro	9.6 [5.8, 23]	5.9 [5.0, 19]	2	12	1091
LPMC	18 [13, 21]	$<$ 6 [ $<$ 6, 6.2]	6	82	1204
Cars	7.1 [5.5, 8.4]	$<$ 2 [ $<$ 2, 3.2]	2	1	284
Cars CN	5.7 [3.9, 8.2]	3.3 [ $<$ 2, 10.0]	2	0	348
EV	18 [13, 33]	$<$ 3 [ $<$ 3, 6.1]	3	24	883
Beer	38 [23, 65]	65 [19, 432]	2	8	1191

Appendix

Table 12. Exchange rate

N^{*}

of Agentic-Priors +

z_{i}

against HBM +

z_{i}

: the survey size at which the HBM overtakes the prior, with its 95% user-bootstrap interval.

<

n

: the HBM is already ahead at the smallest survey that could be fitted. Smallest fit: smallest

N

with at least one successful fit; rejected: fits bayesm rejected across all

N

and draws; pool: non-test users.

Preference Elicitation

Preference elicitation studies how systems learn people’s values and trade-offs from questions, choices, rankings, and feedback.

See all 67 papers

Cold-Start Personalization via Training-Free Priors from Structured World Models
Feb 16, 2026Avinandan Bose, Shuyue Stella Li, Faeze Brahman +6Preference Elicitation User Preference Modeling
Meta · University of Washington · Allen Institute for AI
Jump Start or False Start? A Theoretical and Empirical Evaluation of LLM-initialized Bandits
Apr 2, 2026Adam Bayley, Xiaodan Zhu, Raquel Aoki +2Contextual Bandits Large Language Model-Based Recommendation
Department of Electrical and Computer Engineering, Queen’s University · RBC Borealis
LLM-Derived Priors for Thompson Sampling in Cold-Start Comment Recommendation
Aug 4, 2026Eugene Lee, Oseong Choi, Byungsoo Kang +1Thompson Sampling Cold-Start Recommendation
NAVER WEBTOON South Korea

Feb 16, 2026J/K move · Enter open · S save

Cold-Start Personalization via Training-Free Priors from Structured World Models

Avinandan Bose, Shuyue Stella Li, Faeze Brahman, Pang Wei Koh+5

Meta · University of Washington · Allen Institute for AI

Preference Elicitation User Preference Modeling Preference Modeling

Open paper AskPDF arXiv

Cold-start personalization requires inferring user preferences through interaction when no user-specific historical data is available. The core challenge is a routing problem: each task admits dozens of preference dimensions, yet individual users care about only a few, and which ones matter depends on who is asking. With a limited question budget, asking without structure will miss the dimensions that matter. Reinforcement learning is the natural formulation, but in multi-turn settings its terminal reward fails to exploit the factored, per-criterion structure of preference data, and in practice learned policies collapse to static question sequences that ignore user responses. We propose decomposing cold-start elicitation into offline structure learning and online Bayesian inference. Pep (Preference Elicitation with Priors) learns a structured world model of preference correlations offline from complete profiles, then performs training-free Bayesian inference online to select informative questions and predict complete preference profiles, including dimensions never asked about. The framework is modular across downstream solvers and requires only simple belief models. Across medical, mathematical, social, and commonsense reasoning, Pep achieves 80.8% alignment between generated responses and users' stated preferences versus 68.5% for RL, with 3-5x fewer interactions. When two users give different answers to the same question, Pep changes its follow-up 39-62% of the time versus 0-28% for RL. It does so with ~10K parameters versus 8B for RL, showing that the bottleneck in cold-start elicitation is the capability to exploit the factored structure of preference data.

Abstract

Preference Elicitation

Explore similar work

Cold-Start Personalization via Training-Free Priors from Structured World Models

Jump Start or False Start? A Theoretical and Empirical Evaluation of LLM-initialized Bandits

LLM-Derived Priors for Thompson Sampling in Cold-Start Comment Recommendation