Overview

Preference learning studies how to infer what people value from pairwise comparisons, rankings, choices, behavior, and other feedback. Common approaches include Bradley–Terry and Gaussian-process models, active query selection, inverse reinforcement learning, and reward-model training with methods such as direct preference optimization. These techniques support personalized recommendations, robot and agent behavior, and multi-objective decisions where no single fixed reward captures every user’s priorities. A central challenge is that preferences can be noisy, context-dependent, and heterogeneous, so researchers must balance efficient data collection and accurate prediction against robust generalization and faithful representation of differing values.

Latest papers

CardsList

162 papers19 papers in the last four weeks, up 12% on the four weeks before. 0.1% of all new papers.

  1. What Is Missing: Interpretable Ratings for Large Language Model Outputs

    Feb 17, 2026Nicholas Stranges, Yimin YangPreference Learning for LLMsLLM Evaluation

  2. Next-token Preference: Scaling Unsupervised Reward Modeling

    Feb 11, 2026Jingxuan Fan, Yueying Li, Zhenting Qi +4Reward ModelingPreference Learning

  3. Towards Optimality in Active Preference Learning

    Feb 2, 2026Yao Zhao, Kwang-Sung JunActive Preference LearningActive Learning

  4. ReWorld: Multi-Dimensional Reward Modeling for Embodied Video Generation

    Jan 18, 2026Baorui Peng, Wenyao Zhang, Liang Xu +7Reward Modeling for Video GenerationPhysical Consistency in Video Generation

  5. Co-Evolving Agents: Learning from Failures as Hard Negatives

    Nov 27, 2025Yeonsung Jung, Trilok Padhi, Sina Shaham +4Hard Negative MiningSelf-Improving Agents

  6. Adaptive Margin RLHF via Preference over Preferences

    Sep 26, 2025Yaswanth Chittepu, Prasann Singhal, Greg Durrett +1Pairwise Preference LearningReward Modeling

  7. Interactive Multi-Objective Probabilistic Preference Learning with Soft and Hard Bounds

    Jun 27, 2025Edward Chen, Sang T. Truong, Natalie Dullerud +2Active LearningMulti-Criteria Decision Analysis

  8. MiCRo: Mixture Modeling and Context-aware Routing for Personalized Preference Learning

    May 30, 2025Jingyan Shen, Jiarui Yao, Rui Yang +5Pairwise Preference LearningReward Modeling

  9. Two Minds Better Than One: Collaborative Reward Modeling for LLM Alignment

    May 15, 2025Jiazheng Zhang, Wenqing Jing, Zizhuo Zhang +12Reward ModelingLLM Alignment

  10. Dual-Difficulty Curriculum Learning for Direct Preference Optimization

    Apr 10, 2025Mengyang Li, Haozhan Geng, Zhong Zhang +1Pairwise Preference LearningLLM Alignment

  11. Inferring the Unspoken: Aligning Embodied Agents with Implicit Preferences

    Feb 2, 2025Manjie Xu, Xinyi Yang, Wei Liang +2User Preference ModelingPreference Learning

  12. A Probabilistic Approach for Model Alignment with Human Comparisons

    Mar 16, 2024Junyu Cao, Mohsen BayatiHuman Preference ModelingStatistical Learning Theory