Entropy Guided Diversification and Preference Elicitation in Agentic Recommendation Systems

Dat Tran; Yongce Li; Hannah Clay; Negin Golrezaei; Sajjad Beygi; Amin Saberi

cs.AIMar 12, 2026

Entropy Guided Diversification and Preference Elicitation in Agentic Recommendation Systems

Authors: Dat Tran, Yongce Li, Hannah Clay, Negin Golrezaei, Sajjad Beygi, Amin Saberi

Organizations: Stanford University · Massachusetts Institute of Technology · Amazon

Abstract

Users on e-commerce platforms can be uncertain about their preferences early in their search. Queries to recommendation systems are frequently ambiguous, incomplete, or weakly specified. Agentic systems are expected to proactively reason, ask clarifying questions, and act on the user's behalf, which makes handling such ambiguity increasingly important. In existing platforms, ambiguity led to excessive interactions and question fatigue or overconfident recommendations prematurely collapsing the search space. We present an Interactive Decision Support System (IDSS) that addresses ambiguous user queries using entropy as a unifying signal. IDSS maintains a dynamically filtered candidate product set and quantifies uncertainty over item attributes using entropy. This uncertainty guides adaptive preference elicitation by selecting follow-up questions that maximize expected information gain. When preferences remain incomplete, IDSS explicitly incorporates residual uncertainty into downstream recommendations through uncertainty-aware ranking and entropy-based diversification, rather than forcing premature resolution. We evaluate IDSS using review-driven simulated users grounded in real user reviews, enabling a controlled study of diverse shopping behaviors. Our evaluation measures both interaction efficiency and recommendation quality. Results show that entropy-guided elicitation reduces unnecessary follow-up questions, while uncertainty-aware ranking and presentation yield more informative, diverse, and transparent recommendation sets under ambiguous intent. These findings demonstrate that entropy-guided reasoning provides an effective foundation for agentic recommendation systems operating under uncertainty.

Figures & tables

Figure 1: IDSS system overview. The conversational loop uses LLM parsing and entropy-guided question selection, followed by candidate ranking and entropy-based grid diversification.

Figure 2: Example of LLM-based semantic parsing that maps a free-form user query into structured hard constraints ( $\mathcal{F}$ ), soft preference cues ( $\mathcal{P}^{+}$ ), user disliked features ( $\mathcal{P}^{-}$ ), and user’s patience level ( $s$ ) which are subsequently used for downstream retrieval and ranking.

Figure 3: Example of review-driven simulation. A single seed review is rewritten into multiple stylistically consistent variants targeting different vehicles while preserving constraints and priorities. Each review is enriched into a structured record, which yields a persona and an initial query for the interactive evaluation.

Query Data Type	Method	Config	Prec@9 $\uparrow$	NDCG@9 $\uparrow$	Sat@9 $\uparrow$	ILD $\uparrow$
Short	ES	Full	.903 $\pm$ .025	.941 $\pm$ .011	.772 $\pm$ .032	.779 $\pm$ .043
		$-$ MMR	.918 $\pm$ .020	.952 $\pm$ .007	.764 $\pm$ .019	.279 $\pm$ .011
		$-$ EntropyQ	.880 $\pm$ .056	.927 $\pm$ .032	.732 $\pm$ .051	.611 $\pm$ .015
		$-$ MMR and EntropyQ	.902 $\pm$ .019	.931 $\pm$ .023	.778 $\pm$ .016	.231 $\pm$ .004
	CR	Full	.887 $\pm$ .025	.900 $\pm$ .025	.758 $\pm$ .037	.319 $\pm$ .008
		$-$ MMR	.883 $\pm$ .012	.920 $\pm$ .005	.719 $\pm$ .024	.179 $\pm$ .003

Table 1: Ablation study of ranking and diversity components under short and long interaction settings. We evaluate the impact of removing MMR-based diversification and entropy-based follow-up questioning (EntropyQ) on recommendation quality for Embedding Similarity (ES) and Coverage-Risk (CR) methods. Results report mean

\pm

standard deviation over three runs. Bold indicates best and underline indicates second best per section.

Data	Config	Relevance $\uparrow$	Newness $\uparrow$
Short	w/ EntropyQ	1.00 $\pm$ .000	.946 $\pm$ .056
Short	w/o EntropyQ	.967 $\pm$ .058	.602 $\pm$ .047
Long	w/ EntropyQ	1.00 $\pm$ .000	.976 $\pm$ .032
Long	w/o EntropyQ	1.00 $\pm$ .000	.980 $\pm$ .021

Table 2: Ablation study of follow-up question selection under short and long query-verbosity settings. We evaluate entropy-based question selection (EntropyQ) using Relevance and Newness metrics. Results report mean

\pm

standard deviation over three runs.

Recommender Systems

Recommender systems predict and rank items for people using interaction histories, content, context, and feedback.

See all 514 papers

Beyond expert users: agents should help users construct preferences, not just elicit them
Jun 29, 2026Irena Saracay, Ludwig Schmidt, Carlos GuestrinAI Agent Benchmarks Recommender Systems
Stanford University
Ask to Be Sure: Informative Interactions for Confident Multi-Turn LLM Recommendation
Aug 16, 2026Cedar Site Bai, Zhenyu Liao, Duanshun Li +6Information Gain Recommender Systems
Amazon · Sunnyvale, CA, USA · Seattle, WA, USA +1
Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems
Jul 5, 2026Xinyu Lin, Yashar Deldjoo, Sunhao Dai +7LLM Agent Evaluation Recommender Systems
National University of Singapore · Polytechnic University of Bari · Renmin University of China +1

Jun 29, 2026J/K move · Enter open · S save

Beyond expert users: agents should help users construct preferences, not just elicit them

Irena Saracay, Ludwig Schmidt, Carlos Guestrin

Stanford University

AI Agent Benchmarks Recommender Systems Large Language Model-Based Recommendation User Preference Modeling Preference Modeling

Open paper AskPDF arXiv

Agents typically assume an expert user -- one with well-formed preferences about what they want -- and default to clarifying questions whenever the task is underspecified. We argue this assumption is unrealistic. Users often lack the domain knowledge to have completely specified preferences; if asked about their preference on some feature, the user may be unable to answer without the agent helping the user to learn some domain knowledge needed to form a preference for that feature, e.g., via examples or explanations. To formalize these principles, we draw on the Search-Experience-Credence framework from Information Economics to introduce CoPref, a model of how users construct preferences based on agent dialog actions. We then study these ideas concretely in agentic recommender systems, proposing CoShop, an interactive benchmark. In CoShop, an agent converses with and makes recommendations for a CoPref user. The agent's performance depends on whether it can help the user gain the knowledge needed to specify the task well. Evaluating five frontier models, we find that no agent exceeds 56% accuracy on CoShop despite five turns of interaction. Failures stem not from agents' ability to find items, but from how little the interaction expands what users know about what they want.

Abstract

Recommender Systems

Explore similar work

Beyond expert users: agents should help users construct preferences, not just elicit them

Ask to Be Sure: Informative Interactions for Confident Multi-Turn LLM Recommendation

Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems