Overview

Preference learning studies how to infer what people value from pairwise comparisons, rankings, choices, behavior, and other feedback. Common approaches include Bradley–Terry and Gaussian-process models, active query selection, inverse reinforcement learning, and reward-model training with methods such as direct preference optimization. These techniques support personalized recommendations, robot and agent behavior, and multi-objective decisions where no single fixed reward captures every user’s priorities. A central challenge is that preferences can be noisy, context-dependent, and heterogeneous, so researchers must balance efficient data collection and accurate prediction against robust generalization and faithful representation of differing values.

Latest papers

CardsList

162 papers19 papers in the last four weeks, up 12% on the four weeks before. 0.1% of all new papers.

  1. Learning Latent Goal Costs for Visual World-Model Planning with an Expanding Curriculum

    Nov 12, 2026Konstantinos Mylonas, Thrasyvoulos SpyropoulosWorld Model-Based PlanningGoal-Conditioned Planning

  2. Human or Synthetic? What a Fixed Budget Buys for Post-Training

    Nov 12, 2026Shourik Banerjee, Ethan Mendes, Mohit Raghavendra +2LLM Post-TrainingPreference Learning for LLMs

  3. Generate-then-Rank: Learning Symbolic Goal Interpretation for Embodied Agents

    Nov 5, 2026Jisen Lei, Huizhi LiangEmbodied AISymbolic Reasoning in Language Models

  4. TopoDPO: Dense Preference Alignment for Structure-Sensitive Aerial Referring Segmentation

    Nov 1, 2026Huaiyuan Peng, Jisheng Dang, Wencan Zhang +6Referring Image SegmentationRemote Sensing Image Segmentation

  5. Reliability-Weighted Direct Preference Optimization: Penalized Weights for Robust Preference Learning

    Oct 30, 2026Bin Luo, Param Venkat Vivek Kesireddy, Anil VallepuDirect Preference OptimizationPreference Learning

  6. SAT-ACT: Learning CDCL Branching from Outcome-Derived Action Preferences

    Oct 30, 2026Zhenchao Sun, Chongyang Tao, Ping Lu +3Boolean SatisfiabilityPreference Learning

  7. Predict the Reward, Not the World: World Reward Models for ML Agents

    Oct 28, 2026Yuxiao Lee, Zherui Yang, Fan Liu +1Preference LearningAutomated Machine Learning

  8. Same Verdict, Different Readers: Personalizing Fact-Checking Explanations from Relative Preferences

    Oct 27, 2026Haokun Wen, Xuemeng Song, Mong-Li Lee +1Preference Learning for LLMsAutomated Fact-Checking

  9. Selective Tie Learning: Mitigating Spurious Correlations in Preference Optimization

    Oct 26, 2026Christian Moya, Purav Matlia, Elliott Thornley +1Preference OptimizationSpurious Correlation Robustness

  10. Environment Edits as a Corrective Signal for Reward Learning

    Oct 26, 2026Isaac Ngui, Courtney McBeth, Nancy M. AmatoRobot LearningPreference Learning

  11. GPDPO: Learning from Reward Conflicts via Group Pareto-Dominance Policy Optimization

    Oct 26, 2026Zixuan Xu, Haoxuan Chen, Jiangxin Sun +1Multi-Objective Language Model AlignmentMulti-Objective Reinforcement Learning

  12. PerSPL: Personalized Semi-supervised Preference Learning

    Oct 21, 2026Xiaoan Xu, Tingzhou Wei, Juncheng Dong +3Personalized Reward ModelingUser Preference Modeling

  13. Judging the Justification: Decision Certificate Supervision for Generative Reward Models

    Oct 20, 2026Guiquan Sun, Bing Bai, Martin Renqiang Min +1Reward ModelingFaithfulness of Language Model Explanations

  14. Asking Smarter Questions: Targeted Active Preference Learning for Underspecified Rewards

    Oct 15, 2026Helena Merker, Nick Walker, Andreea BobuActive Preference LearningPreference Learning

  15. Agentic-Priors: Using Large Language Models to Address the Cold Start Problem in Preference Elicitation

    Oct 15, 2026Bruno Arcanjo, Zhaoxing Li, Sebastian SteinPreference ElicitationUser Preference Modeling

  16. SPEAR: Selective Preference Elicitation with Adaptive Rewards for VLM-Guided Robotic Manipulation

    Oct 15, 2026Zewen Wu, Ruizhe Liu, Pei Zhou +2Active Preference LearningPreference Learning

  17. Rank-Aware Differentially Private Release of Listwise Preferences for LLM Alignment

    Oct 13, 2026Junwei Chen, Manjiang Yu, Pengpeng Qiao +1Privacy-Preserving Language ModelsDifferential Privacy

  18. Vision: "What Don't I Know About User's Preferences?" How Uncertainty Estimation Improves Proactive Assistant Alignment

    Oct 11, 2026Zhiyu An, Ziyi Xuan, Yu Yang +1Proactive AgentsUser Preference Modeling

  19. LSC-DPO: Learning-Signal-Controlled Direct Preference Optimization

    Oct 6, 2026Yang Qu, Yusheng Han, Chengjia Feng +1Direct Preference OptimizationLLM Alignment

  20. Large Language Model Orchestration under Heterogeneous Preferences via Explicit Persona Inference

    Oct 6, 2026Shuqing Shi, Ziyan Wang, Milind Tambe +1Multi-Agent CoordinationActive Preference Learning

  21. Verifiable, Articulable, and Tacit Components of Preference

    Oct 2, 2026Alexander Spangher, Sheldon S. Huang, Andreas Haupt +4Human Preference ModelingPreference Learning

  22. Targeted Active Learning for Preference-Based Treatment Effects on Multivariate Outcomes

    Oct 2, 2026Lola Giordani, Mathieu Even, Chloé Geoffroy +3Active Preference LearningIndividualized Treatment Effect Estimation

  23. From Language Priors to Field Adaptation: Preference Learning for Traversability Estimation

    Oct 2, 2026Simon Schwaiger, David Seyser, Alessandro Scherl +3Robot Traversability EstimationRobot Navigation

  24. Discrete Annotation, Continuous Preference: Rethinking Supervision for Accurate and Generalizable Aesthetic Image Cropping

    Sep 30, 2026Ziqing Zhang, Xiao Liu, Kai Liu +4Image EditingPreference Learning