Pairwise Preference Learning

Ask about this topic

Overview

Pairwise preference learning studies how to infer what people value from choices between alternatives, rather than requiring absolute scores. Common approaches include Bradley–Terry reward models, direct preference optimization for language models, and active learning or dueling-bandit methods that select informative comparisons. The methods support applications from ranking and recommendations to AI alignment, robotics, and policy learning, including settings with noisy labels or limited annotation budgets. A central challenge is that preferences may be heterogeneous, context-dependent, or cyclic, so a single consistent utility score can miss important differences between people and situations.

Latest papers

CardsList

126 papers26 papers in the last four weeks, up 136% on the four weeks before. 0.2% of all new papers.

  1. Offline Preference Optimization for Rectified Flow with Noise-Tracked Pairs

    May 10, 2026Yunhong Lu, Qichao Wang, Hengyuan Cao +2T2I GenerationRectified Flow

    Zhejiang University · Shanghai Institute for Advanced Study-Zhejiang University · Shanghai Institute for Mathematics and Interdisciplinary Sciences

  2. ξξ-DPO: Direct Preference Optimization via Ratio Reward Margin

    May 9, 2026Zhengyuan Fan, Zhonghua Wu, Yuxuan Du +1Pairwise Preference LearningDirect Preference Optimization

    School of Computer Science, Northwestern Polytechnical University

  3. Beyond Pairs: Your Language Model is Secretly Optimizing a Preference Graph

    May 8, 2026Ning Liu, Chuanneng Sun, Kristina Klinkner +1Pairwise Preference LearningLLM Alignment

    Amazon

  4. Response Time Enhances Alignment with Heterogeneous Preferences

    May 7, 2026Federico Echenique, Alireza Fallah, Baihe Huang +1Pairwise Preference LearningLLM Alignment

    Department of Economics, University of California, Berkeley · Department of Computer Science and Ken Kennedy Institute, Rice University · Departments of Electrical Engineering and Computer Sciences, University of California, Berkeley +2

  5. A Unified Pair-GRPO Family: From Implicit to Explicit Preference Constraints for Stable and General RL Alignment

    May 7, 2026Hao YuPairwise Preference LearningLLM Alignment

    Department of Automation, Tsinghua University

  6. Pair2Score: Pairwise-to-Absolute Transfer for LLM-Based Essay Scoring

    May 3, 2026İbrahim Rıza Hallaç, Hasan OğulPairwise Preference LearningPairwise Comparison

    Department of Computer Science and Communication · Østfold University College, Halden 1757, Norway

  7. Multi-User Dueling Bandits: A Fair Approach using Nash Social Welfare

    May 3, 2026Maheed H. Ahmed, Mahsa GhasemiPairwise Preference LearningMulti-Armed Bandits

    Electrical and Computer Engineering Purdue University

  8. Mind the Gap: Structure-Aware Consistency in Preference Learning

    Apr 30, 2026Mehryar Mohri, Yutao ZhongPairwise Preference LearningLLM Alignment

    Google Research, New York, NY · Courant Institute of Mathematical Sciences, New York, NY

  9. Learning from Noisy Preferences: A Semi-Supervised Learning Approach to Direct Preference Optimization

    Apr 27, 2026Xinxin Liu, Ming Li, Zonglin Lyu +2Pairwise Preference LearningDiffusion Model Alignment

    University of Central Florida

  10. Preferences of a Voice-First Nation: Large-Scale Pairwise Evaluation and Preference Analysis for TTS in Indian Languages

    Apr 23, 2026Srija Anand, Ashwin Sankar, Ishvinder Sethi +10Pairwise Preference LearningHuman Preference Evaluation

    Indian Institute of Technology Madras, India · AI4Bharat, India · Josh Talks, India

  11. Vibrotactile Preference Learning: Uncertainty-Aware Preference Learning for Personalized Vibration Feedback

    Apr 22, 2026Rongtao Zhang, Xin Zhu, Masoume Pourebadi Khotbehsara +3Pairwise Preference LearningActive Preference Learning

    Department of Computer Science University of Southern California Los Angeles, California, USA

  12. Towards Disentangled Preference Optimization Dynamics: Suppress the Loser, Preserve the Winner

    Apr 20, 2026Wei Chen, Yubing Wu, Junmei Yang +5Pairwise Preference LearningLLM Alignment

    1South China University of Technology · 2AIP, RIKEN · 3Columbia University +1

  13. lmfaoooo at SemEval-2026 Task 1: Humor Is an Audience. Preference Modeling for Constrained Humor Generation

    Apr 14, 2026Alexey Tikhonov, Alexey IvanovHumor Explanation GenerationText Generation

    Inworld.AI Berlin, Germany · OpenAI Mountain View, CA

  14. From Recency Bias to Stable Convergence Block Kaczmarz Methods for Online Preference Learning in Matchmaking Applications

    Apr 11, 2026James NguyenPreference LearningRecommender Systems

    Department of Mathematics, University of California, Irvine.

  15. Efficient Personalization of Generative User Interfaces

    Apr 10, 2026Yi-Hao Peng, Jeffrey P. Bigham, Jason WuPairwise Preference LearningHuman Preference Modeling

    Massachusetts Institute of Technology · Carnegie Mellon University · Purdue University

  16. Relational Preference Encoding in Looped Transformer Internal States

    Apr 10, 2026Jan KirinPairwise Preference LearningReward Modeling

  17. Decomposing the Delta: What Do Models Actually Learn from Preference Pairs?

    Apr 9, 2026Chia-Hsuan Lee, Mingyang Zhou, Renkun Ni +6Preference OptimizationPairwise Preference Learning

    Capital One

  18. Soft Tournament Equilibrium: Differentiable Set-Valued Inference for Non-Transitive Pairwise Comparisons

    Apr 6, 2026Saad AlqithamiPairwise Preference LearningSocial Choice Theory

  19. Preference learning in shades of gray: Interpretable and bias-aware reward modeling for human preferences

    Apr 1, 2026Simona-Vasilica Oprea, Adela BâraReward ModelingPairwise Preference Learning

    Department of Economic Informatics and Cybernetics, Bucharest University of Economic Studies, Bucharest, Romania

  20. Active Preference Learning over Latent Preference Archetypes for Many-Objective Bayesian Optimization

    Mar 30, 2026Manisha Dubey, Sebastiaan De Peuter, Wanrong Wang +1Active Preference LearningBayesian Optimization

    Department of Computer Science, University of Manchester, UK · Centre for AI in Assistive Autonomy, University of Edinburgh, Scotland, UK · Department of Computer Science, Aalto University, Espoo, Finland +2

  21. Regret Bounds for Reinforcement Learning from Multi-Source Imperfect Preferences

    Mar 20, 2026Ming Shi, Yingbin Liang, Ness B. Shroff +1Preference-Based RLRL from Human Feedback

    Department of Electrical Engineering, University at Buffalo, Buffalo, NY 14260, USA · Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH 43210, USA · Department of Electrical and Computer Engineering and Computer Science and Engineering, The Ohio State University, Columbus, OH 43210, USA +1

  22. Best-of-Both-Worlds Multi-Dueling Bandits: Unified Algorithms for Stochastic and Adversarial Preferences under Condorcet and Borda Objectives

    Mar 19, 2026S Akash, Pratik Gajane, Jawar SinghDueling BanditsMulti-Armed Bandits

    Indian Institute of Technology Patna, India · University of Orléans, France

  23. From Isolated Scoring to Collaborative Ranking: A Comparison-Native Framework for LLM-Based Paper Evaluation

    Mar 18, 2026Pujun Zheng, Jiacheng Yao, Jinquan Zheng +6Automated Peer ReviewScholarly Peer Review

    School of Economics and Management, East China Normal University · School of Information Management, Wuhan University · China Academic Degrees & Graduate Education Development Center

  24. Improving through Interaction: Searching Behavioral Representation Spaces with CMA-ES-IG

    Mar 9, 2026Nathaniel Dennler, Zhonghao Shi, Yiran Tao +3Active Preference LearningPreference Learning

    CSAIL/AeroAstro, Massachusetts Institute of Technology. · Department of Computer Science, University of Southern California.

  25. CRED: Counterfactual Reasoning and Environment Design for Active Preference Learning

    Mar 9, 2026Yi-Shiuan Tung, Gyanig Kumar, Wei Jiang +2Active Preference LearningPreference Learning

    Department of Computer Science, University of Colorado Boulder

  26. When Weak LLMs Speak with Confidence, Preference Alignment Gets Stronger

    Mar 5, 2026Amirabbas Afzali, Myeongho Jeon, Maria BrbicPreference AlignmentPreference Optimization

    EPFL · Sharif University of Technology

  27. PARWiS: Winner determination under shoestring budgets using active pairwise comparisons

    Mar 1, 2026Shailendra BhandariDueling BanditsPairwise Preference Learning

    Department of Computer Science OsloMet– Oslo Metropolitan University Oslo, Norway

  28. Provably Efficient Regularized Online RLHF with Generalized Bilinear Preferences

    Feb 26, 2026Junghyun Lee, Minju Hong, Kwang-Sung Jun +2Regret Minimization in RLPreference-Based RL

    KAIST AI · KAIST EE · POSTECH CSE/AI

  29. Learning Preference from Observed Rankings

    Feb 18, 2026Yu-Chang Chen, Chen Chian Fuh, Shang En TsaiPreference ModelingUser Preference Modeling

    National Taiwan University

  30. Towards Optimality in Active Preference Learning

    Feb 2, 2026Yao Zhao, Kwang-Sung JunActive Preference LearningActive Learning

    Department of Computer Science, University of Arizona, Tucson, USA.