Preference Elicitation

Follow this topic Ask about this topic

Preference elicitation asks how a system can infer what an individual or group values from limited answers, then use that information to recommend items, optimize decisions, or guide AI behavior. Common approaches include Bayesian active learning that chooses informative follow-up questions, pairwise-comparison and discrete-choice models that estimate utilities, and conversational systems that extract preferences from dialogue; these methods are tested in settings such as product search, portfolio choice, robotics, and human-feedback alignment. A central challenge is balancing accurate, personalized models against the time and cognitive burden of asking questions, while ensuring that minority preferences are not lost in aggregation. Elicited answers can also be inconsistent, instrument-dependent, or disconnected from later behavior, so a key open issue is when a measured preference reliably represents what someone wants in the decision at hand.

Latest papers

69 papers13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

Learning What to Ask for Tomorrow: Active Preference Acquisition for Long-Term Personalized Web Agents
Oct 22, 2026Yiqi Wang, Zizhi Zeng, Jiaqi Zhang +3Active Preference Learning Preference Elicitation
Abstract Priorities, Concrete Rewards: How Stated Preferences Relate to What People Reward in AI Responses
Oct 22, 2026Zihan Chen, Lei Nico Zheng, Di ZhuUser Preference Modeling Preference Elicitation
Stevens Institute of Technology, Hoboken, NJ, USA · University of Massachusetts Boston, Boston, MA, USA
Selecting the Best Few with Human Evaluators: Adaptively Combining Ratings and Pairwise Comparisons
Oct 19, 2026Motti Goldberger, Nils RudiHuman-in-the-Loop Evaluation Preference Elicitation
Yale School of Management
VISTA: Steering LLMs with Latent User Values Inferred from a Few Survey Answers
Oct 16, 2026Yinuo Xu, Jadie Adams, Brian Hu +3Personalized Language Model Alignment Language Model Steering
University of Michigan · Kitware Inc.
ParetoChat: An LLM-Based Interface for Interactive Multi-Objective Optimization
Oct 16, 2026Gokhan CeyhanMulti-Objective Optimization Multi-Criteria Decision Analysis
Just Eat Takeaway.com
Beyond Explicit Intent: Discovering and Shaping Artistic Intent through Human-AI Co-Creation
Oct 15, 2026Renye Yan, Jialun Zhong, Jikang Cheng +3Human-AI Co-Creation Human-in-the-Loop AI
Peking University, China
Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards
Oct 15, 2026Helena Merker, Nick Walker, Andreea BobuActive Preference Learning Preference Learning
Massachusetts Institute of Technology
Agentic-Priors: Using Large Language Models to Address the Cold Start Problem in Preference Elicitation
Oct 15, 2026Bruno Arcanjo, Zhaoxing Li, Sebastian SteinPreference Elicitation User Preference Modeling
University of Southampton, Southampton, United Kingdom
Who Should We Listen to More? Welfare-Aware Preference Acquisition for Pluralistic Alignment
Oct 13, 2026Ruomeng Ding, Tianwei Gao, Lianrui Geng +2Social Welfare Maximization Preference Alignment
University of North Carolina at Chapel Hill
SimInter: Evidence-Grounded User Simulation with Interview-Based Profile Enrichment for Conversational Recommender Systems
Oct 13, 2026Erfan Shahabi, Amirhossein Roshandel, Mohammad Sadegh Sadeghi +1User Simulation Recommender Systems
School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran · Institute for Research in Fundamental Sciences (IPM), Tehran, Iran
Rethinking Opinion Measurement Interfaces for Human-in-the-Loop Optimization
Oct 10, 2026Changkun Ou, Sven Mayer, Daniel Buschek +1Human-in-the-Loop AI Preference Elicitation
LMU Munich, Germany · University of Bayreuth, Germany
Verifiable, Articulable, and Tacit Components of Preference
Oct 2, 2026Alexander Spangher, Sheldon S. Huang, Andreas Haupt +4Human Preference Modeling Preference Learning
Stanford University · University of Toronto
Selective Elicitation as a Commercial Influence Channel: A Reproducible Synthetic Shopping-Agent Stress Test
Sep 29, 2026Jiapeng LiPreference Elicitation Large Language Model-Based Recommendation
How Much Imprecision is Enough Imprecision in my Classifier? A Practical Elicitation Procedure
Sep 27, 2026Victor F. Lopes de Souza, Sébastien Destercke, Abdelhak ImoussatenSet Prediction Classification
EuroMov Digital Health in Motion, Univ. Montpellier, IMT Mines Al`es, Montpellier, France · LIRMM, Universit´e de Montpellier, CNRS, Montpellier, France · SyCoIA, IMT Mines Al`es, Al`es, France +1
Error Bounds for Statistical Estimators in BTL Model with Parametric Multivariate Utility Functions
Sep 22, 2026Yicheng Li, Huifu XuParameter Estimation Minimax Estimation
Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong.
Bayesian Deck-of-cards-based Ordinal Regression with Sequential Preference Elicitation
Sep 19, 2026Marco Grillo, Silvano ZappalàOrdinal Regression Preference Elicitation
Institute of Computing Science, Poznań University of Technology, Piotrowo 2, 60-965, Poznań, Poland · Department of Economics and Business, University of Catania, Corso Italia 55, 95129, Catania, Italy
GUIDE: Generative Utility Inference and Decision Engine
Sep 10, 2026Anagha Tiwari, Alexander G. Gray, Nick Feamster +3Adaptive Sampling User Preference Modeling
Department of Computer Science, University of Chicago · Centaur AI Institute, University of Chicago · Heinz College & School of Computer Science, Carnegie Mellon University +1
Eliciting ESG Preferences for Reinforcement Learning-Based Portfolio Optimization
Sep 2, 2026Giovanni Dispoto, Marcello Restelli, Carmine VentrePortfolio Optimization Finance
Politecnico di Milano · King’s College London
Towards Effective Structured Context Modeling for Conversational Recommender Systems via Dual-node Monte Carlo Tree Search
Sep 1, 2026Jincheng Zhang, Chen Huang, Wenqiang Lei +2Recommender Systems Large Language Model-Based Recommendation
College of Computer Science, Sichuan University · Engineering Research Center of Machine Learning and Industry Intelligence, Ministry of Education, China · Institute of Data Science, National University of Singapore +1
How much of a measured AI preference is the model, and how much is the instrument?
Aug 24, 2026Jason HungPreference Elicitation Prompt Sensitivity
Independent With Apart Research
LLM-Derived Preference Judgments Are Not Self-Consistent
Aug 18, 2026Matthew T. Ford, Francis Bahk, Jingjing Wang +4Preference Modeling User Preference Modeling
Cornell University · Georgia Institute of Technology
AI Revealed Preferences
Aug 17, 2026Sam Wang, Sofiia Lobanova, Yonathan Arbel +2Personality Modeling in Language Models LLM Sycophancy
Supervised Program for Alignment Research (SPAR) · University of Alabama School of Law · University of Hong Kong +1
Participatory Moral AI Is Not Neutral: The Invisible Hand of Developers
Aug 14, 2026Taenyun Kim, Edyta Bogucka, Daniele QuerciaAI Alignment Preference Elicitation
Michigan State University, US · Nokia Bell Labs, Cambridge, UK · Politecnico di Torino, Italy
From Product Search to Preference Articulation: The Economics of Agentic Commerce
Aug 9, 2026Lingxiu Dong, Kaiwen Luo, Fasheng XuAgentic Search E-Commerce Search
Olin Business School, Washington University in St. Louis · School of Business, University of Connecticut
Personalized Deep Research Query Refinement with Graph-Scaffolded Evidence Grounding
Aug 6, 2026Soojin Yoon, Dongha LeeDeep Research Agents Clarification Question Generation
Department of Artificial Intelligence, Yonsei University
Compass: Continuously Aligning Social Media Feeds via In-Situ Reflections
Aug 4, 2026Aadit Barua, Leijie Wang, Amy X. ZhangSocial Media Analysis Recommender Systems
University of Washington Seattle, Washington, United States
An analysis of machine learning approaches for enhancing decision-making in complex discrete choice tasks
Jul 30, 2026Sheng Lun Christine Cao, Destenie Nock, Alex DavisDiscrete Choice Modeling User Preference Modeling
Department of Electrical and Software Engineering, University of Calgary, Calgary, AB, Canada · Department of Engineering and Public Policy, Carnegie Mellon University, Pittsburgh, PA, USA · Department of Civil Engineering, Carnegie Mellon University, Pittsburgh, PA, USA +1
Preference Elicitation for Policy Optimization and Application to Aligning Heart Transplantation with Human Values
Jul 29, 2026Itai Zilberstein, Ioannis Anagnostides, Zachary W Sollie +2Preference Elicitation Human Preference Modeling
Department of Computer Science, Carnegie Mellon University, Pittsburgh, PA · Department of Surgery, Division of Cardiothoracic Surgery, Medical University of South Carolina, Charleston, SC · Strategy Robot, Inc., Strategic Machine, Inc., Optimized Markets, Inc.
Rushes: A Human Preference Dataset for Pluralistic Alignment
Jul 22, 2026Michael Xu, Jorge Leandro, Sudha Rao +5Sequential Decision Making Human Preference Modeling
Microsoft Research
When Is Delegated Play Truthful? Within-Range Regret and the Trilemma of Aligned Delegation
Jul 15, 2026Taksch DubeMechanism Design LLM Alignment
Kent State University, Kent, OH, USA

Oct 22, 2026J/K move · Enter open · S save

Learning What to Ask for Tomorrow: Active Preference Acquisition for Long-Term Personalized Web Agents

Yiqi Wang, Zizhi Zeng, Jiaqi Zhang, Yiqun Duan+2

Active Preference Learning Preference Elicitation User Preference Modeling Web Agents

Open paper AskPDF arXiv

Web agents increasingly serve the same user across repeated online tasks such as booking travel, shopping, and scheduling, so their long-term usefulness depends on how well they know the user's preferences. Passive memory and personalization methods cannot obtain preferences the user never reveals, and clarification methods that value a question by its benefit to the current task overlook the benefit of reusing the answer in later tasks. We study Active Preference Acquisition (APA): which missing preferences are worth the cost of asking? We define the Prospective Personalization Value (PPV) of a missing preference as its expected cumulative utility gain over future tasks minus the effort and disclosure-risk cost of asking, and select questions under a per-interaction budget. We characterize when cumulative reuse justifies acquisition despite insufficient single-task benefit, and evaluate this regime with an initial-profile diagnostic. APA computes the counterfactual gain of each candidate preference on public task prototypes with the agent's own task solver and aggregates it by the expected reuse of each task family. We evaluate it on ActiveWebPref, a controlled longitudinal benchmark with hidden but queryable user profiles and exact task utilities. At a horizon of 20 tasks, 34.4% of budget-feasible valuable missing preferences are amortization-dependent; APA acquires 80.3% of them, whereas current-task value of information acquires 5.1%. Under per-user asking-cost caps matched to APA's realized expenditure, APA achieves higher net utility than four heuristic selection baselines. In our cost setting, asking about every unknown preference yields lower net utility than not asking, while APA recovers 84.2% of its utility gain with 27.6% of its questions. The value of acquiring a user preference should account for its reuse across future tasks, not only its benefit to the task at hand.