Preference Learning

Ask about this topic

Overview

Preference learning studies how to infer what people value from pairwise comparisons, rankings, choices, behavior, and other feedback. Common approaches include Bradley–Terry and Gaussian-process models, active query selection, inverse reinforcement learning, and reward-model training with methods such as direct preference optimization. These techniques support personalized recommendations, robot and agent behavior, and multi-objective decisions where no single fixed reward captures every user’s priorities. A central challenge is that preferences can be noisy, context-dependent, and heterogeneous, so researchers must balance efficient data collection and accurate prediction against robust generalization and faithful representation of differing values.

Latest papers

CardsList

147 papers19 papers in the last four weeks, up 27% on the four weeks before. 0.1% of all new papers.

  1. PerSPL: Personalized Semi-supervised Preference Learning

    Oct 21, 2026Xiaoan Xu, Tingzhou Wei, Juncheng Dong +3Personalized Reward ModelingUser Preference Modeling

    Duke University · George Washington University

  2. Preference Is Not Permission: Personalizing the Auto-Purchase Line in Agentic Commerce

    Oct 21, 2026Ryotaro Shimizu, Sai HtaungKham, Mo FanAgentic CommerceAutomated Agent Evaluation

    ZOZO Research, Tokyo, Japan

  3. Judging the Justification: Decision Certificate Supervision for Generative Reward Models

    Oct 20, 2026Guiquan Sun, Bing Bai, Martin Renqiang Min +1Reward ModelingFaithfulness of Language Model Explanations

    School of Computing, University of Connecticut · NEC Labs America

  4. Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards

    Oct 15, 2026Helena Merker, Nick Walker, Andreea BobuActive Preference LearningPreference Learning

    Massachusetts Institute of Technology

  5. Agentic-Priors: Using Large Language Models to Address the Cold Start Problem in Preference Elicitation

    Oct 15, 2026Bruno Arcanjo, Zhaoxing Li, Sebastian SteinPreference ElicitationUser Preference Modeling

    University of Southampton, Southampton, United Kingdom

  6. SPEAR: Selective Preference Elicitation with Adaptive Rewards for VLM-Guided Robotic Manipulation

    Oct 15, 2026Zewen Wu, Ruizhe Liu, Pei Zhou +2Active Preference LearningPreference Learning

    HKU Musketeers Foundation Institute of Data Science, The University of Hong Kong, Hong Kong SAR, China. · Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong SAR, China.

  7. Fully sequential Gaussian process regression for preference learning

    Oct 14, 2026Ilya O. RyzhovBayesian OptimizationGaussian Process Regression

  8. Vision: "What Don't I Know About User's Preferences?" How Uncertainty Estimation Improves Proactive Assistant Alignment

    Oct 11, 2026Zhiyu An, Ziyi Xuan, Yu Yang +1Proactive AgentsUser Preference Modeling

    University of California, Merced Merced, California, USA · Lehigh University Bethlehem, Pennsylvania USA

  9. Covariate-dependent Joint Modeling of Multivariate Ordinal Preferences and Its Connections with Comparison Models

    Oct 6, 2026Yujie Chen, Antik Chakraborty, Anindya BhadraPreference ModelingPreference Learning

    Department of Statistics Purdue University

  10. LSC-DPO: Learning-Signal-Controlled Direct Preference Optimization

    Oct 6, 2026Yang Qu, Yusheng Han, Chengjia Feng +1Direct Preference OptimizationLLM Alignment

    Northeastern University

  11. Large Language Model Orchestration under Heterogeneous Preferences via Explicit Persona Inference

    Oct 6, 2026Shuqing Shi, Ziyan Wang, Milind Tambe +1Multi-Agent CoordinationActive Preference Learning

    King’s College London, London, United Kingdom · Harvard University, Cambridge, MA, USA

  12. Bidirectional Preference Synthesis: Learning Prompt-Conditioned Preferences from Boundary Failures

    Oct 3, 2026Junbo Wang, Lidong Lu, Zhuoqun Li +4Preference LearningPreference Optimization

    Kuaishou Technology · Nanjing University

  13. Verifiable, Articulable, and Tacit Components of Preference

    Oct 2, 2026Alexander Spangher, Sheldon S. Huang, Andreas Haupt +4Human Preference ModelingPreference Learning

    Stanford University · University of Toronto

  14. Targeted Active Learning for Preference-Based Treatment Effects on Multivariate Outcomes

    Oct 2, 2026Lola Giordani, Mathieu Even, Chloé Geoffroy +3Active Preference LearningIndividualized Treatment Effect Estimation

    Theremia, Paris · Paris Brain Institute (ICM), Sorbonne Université, Inserm, CNRS, AP-HP · Inria, Inserm, Université de Montpellier, France +1

  15. From Language Priors to Field Adaptation: Preference Learning for Traversability Estimation

    Oct 2, 2026Simon Schwaiger, David Seyser, Alessandro Scherl +3Robot Traversability EstimationRobot Navigation

    Graz University of Technology, Institute of Software Engineering and Artificial Intelligence, Graz, Austria · University of Applied Sciences Technikum Wien, Research Group Automation Technology and Robotic Systems, Vienna, Austria · University of Alicante, Department of Computer Technology, Alicante, Spain +1

  16. Discrete Annotation, Continuous Preference: Rethinking Supervision for Accurate and Generalizable Aesthetic Image Cropping

    Sep 30, 2026Ziqing Zhang, Xiao Liu, Kai Liu +4Image EditingPreference Learning

    Shanghai Jiao Tong University · Institute of Media Technology and Experience Design, Huawei

  17. Uncertainty-Normalized Margins for Direct Preference Optimization

    Sep 29, 2026Sadegh Khorasani, Petrus Mikkola, Matthias GrossglauserPairwise Preference LearningDirect Preference Optimization

    School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland · University of Helsinki, Helsinki, Finland

  18. JudgeProfile: Understanding and Steering Subjectivity in LLM Judges

    Sep 29, 2026Qi Cao, Kangning Liu, Xuan Kan +10Human Preference EvaluationLLM Evaluation

    Meta · University of California San Diego · The University of Hong Kong +1

  19. From Soft Targets to Reward Signals: How Assignment and Reward Objectives Interact

    Sep 28, 2026Jiangtao Lin, Bangyang Wei, Siyi Liu +2Pairwise Preference LearningReward Modeling

    Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · Tencent Holdings Limited +1

  20. Improving LLM Collaboration via Multi-Agent Preference Learning

    Sep 26, 2026Shuo Liu, Xinzichen Li, Tianle Chen +1Multi-Agent CollaborationRL for LLM Agents

    Northeastern University, Boston

  21. Kernel-Based Steering of CLIP with Vision-Language Model Preferences

    Sep 26, 2026Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Sina Mansouri +3VLM AdaptationVisual Representation Learning

    University of California, Santa Barbara · University of California, Los Angeles · George Mason University +1

  22. Context-Continuous Preference Learning for Exoskeleton Personalization

    Sep 23, 2026Sunin Baek, Sungwoo Park, Daekyum KimGaussian ProcessesUser Preference Modeling

    School of Mechanical Engineering, Korea University, Seoul 02841, South Korea

  23. Which Terrain Is Better? Preference Learning with VLM Prototypes for Off-Road Traversability Ranking

    Sep 20, 2026Ji-Hoon Hwang, Jisung Bae, E-In Son +3Visual NavigationOff-Road Robot Navigation

  24. Learning Heterogeneous Preferences

    Sep 15, 2026Shiwali Mohan, Matt Hong, Dule Shu +3Pairwise Preference LearningHuman Preference Modeling

    Human-Centered AI Toyota Research Institute Los Altos, CA

  25. Rethinking the Implications of Human Feedback for Preference Learning in Human-Robot Collaboration

    Sep 12, 2026Qiping Zhang, Kate Candon, Debasmita Ghose +1Human Preference ModelingPreference Learning

    Department of Computer Science Yale University, New Haven, CT, United States

  26. Inverse Learning of the Altruism and Cost Level in Mixed-Individual Mean Field Games

    Sep 11, 2026Haoyang Cao, Gökçe Dayanıklı, Xiaofei ShiMean Field GamesGame Theory

    Department of Applied Mathematics and Statistics, Johns Hopkins University · Department of Statistics, University of Illinois, Urbana-Champaign · Department of Statistical Sciences, University of Toronto

  27. GUIDE: Generative Utility Inference and Decision Engine

    Sep 10, 2026Anagha Tiwari, Alexander G. Gray, Nick Feamster +3Adaptive SamplingUser Preference Modeling

    Department of Computer Science, University of Chicago · Centaur AI Institute, University of Chicago · Heinz College & School of Computer Science, Carnegie Mellon University +1

  28. Subspace Inference Enables Efficient Active Reward Learning from Preferences

    Sep 3, 2026Yutai Zhou, Erdem BıyıkBayesian RLActive Preference Learning

    Thomas Lord Department of Computer Science University of Southern California, Los Angeles, USA

  29. Toward Collective-Centric Evaluation of Preference Inference for Participatory Democracy

    Sep 2, 2026Pierre-Antoine Lequeu, Salim Hafid, Paul Lerner +6User Preference ModelingPreference Learning

    Sorbonne Université, CNRS, ISIR, Paris, France · Sciences Po, médialab, Paris, France · Sorbonne Université, STIH/CERES, Paris, France

  30. Eliciting ESG Preferences for Reinforcement Learning-Based Portfolio Optimization

    Sep 2, 2026Giovanni Dispoto, Marcello Restelli, Carmine VentrePortfolio OptimizationFinance

    Politecnico di Milano · King’s College London