Pairwise Preference Learning

Overview

Pairwise preference learning studies how to infer what people value from choices between alternatives, rather than requiring absolute scores. Common approaches include Bradley–Terry reward models, direct preference optimization for language models, and active learning or dueling-bandit methods that select informative comparisons. The methods support applications from ranking and recommendations to AI alignment, robotics, and policy learning, including settings with noisy labels or limited annotation budgets. A central challenge is that preferences may be heterogeneous, context-dependent, or cyclic, so a single consistent utility score can miss important differences between people and situations.

Latest papers

CardsList

132 papers11 papers in the last four weeks, down 58% on the four weeks before. 0.1% of all new papers.

  1. Enhancing Preference Detection using Eye Movement Cascade and Multilayer Combinatorial Fusion

    Nov 12, 2026Yuanhong Wu, Mary Tsaryk, Sam Watson +3User Preference ModelingClassification

  2. Enforcing Axioms for AI Alignment under Loss-Based Rules

    Nov 9, 2026Alexandros Hollender, Sonja KraiczyLLM AlignmentReward Modeling

  3. SAT-ACT: Learning CDCL Branching from Outcome-Derived Action Preferences

    Oct 30, 2026Zhenchao Sun, Chongyang Tao, Ping Lu +3Boolean SatisfiabilityPreference Learning

  4. Counterfactual Pairwise Augmentation Improves Small Pointwise LLM Judges

    Oct 30, 2026Mohammad Zangooei, Peyvand Teymoori, Mohammad Robati Shirzad +1LLM-as-a-JudgeCounterfactual Data Augmentation

  5. Exploring the Limits of the Delta Learning Hypothesis: Single-Pair Generator Biases in Direct Preference Optimization

    Oct 27, 2026Giuseppe Curci, Stefano Simonazzi, Andrea Molinari +1Direct Preference OptimizationLLM Alignment

  6. An Interactive Evolutionary Multiobjective Optimization Algorithm Based on Interpretable Active Preference Learning

    Oct 26, 2026Thomas Quadt, Boris Čule, Roy Lindelauf +2Multi-Objective Evolutionary OptimizationActive Preference Learning

  7. Reward-Weighted Preference Optimization for Selective Combinatorial Optimization Problems

    Oct 22, 2026Haohui Zhang, Xinyu Hu, Wouter van Heeswijk +2RL for Combinatorial OptimizationCombinatorial Optimization

  8. PerSPL: Personalized Semi-supervised Preference Learning

    Oct 21, 2026Xiaoan Xu, Tingzhou Wei, Juncheng Dong +3Personalized Reward ModelingUser Preference Modeling

  9. Pairwise Learning Increases Data Efficiency in Deep Neural Networks

    Oct 20, 2026Robert Bain, Marjan Rashidi, Elizabeth ChrastilData-Efficient LearningSmall-Data Learning

  10. Statistical Learning Theory for Direct Preference Optimization

    Oct 19, 2026Joel Wills, Yuanfan Li, Yiming Ying +1Direct Preference OptimizationPairwise Preference Learning

  11. Bridging Exploration and Persona: Information-Directed Dueling for Test-Time Agent Personalization

    Oct 17, 2026Jie Mao, Pingchen Lu, Xiangyi Wang +4LLM PersonalizationUser Preference Modeling

  12. Replicable Dueling Bandits

    Oct 14, 2026Rohan Deb, Aadirupa Saha, Udaya Ghai +2Dueling BanditsMulti-Armed Bandits

  13. What Does Disagreement Mean? A Semantic Identifiability Frontier for Pairwise Preference Learning

    Oct 14, 2026Manoj Saravanan, Rohit Kumar Salla, Shrikar Reddy KotaPairwise Preference LearningAnnotator Disagreement

  14. When Pairwise Preferences Cycle: A Reliability-Anchored Preference Walk for Video GRPO

    Oct 14, 2026Xinrui Lin, Fan Li, Shumin Wang +2Pairwise Preference LearningRL for Video Generation

  15. Learning Individual Scientific Taste with Privileged Hindsight Distillation

    Oct 14, 2026Hanwei Fan, Yanqiao Wang, Yiqi Zhu +3AI-Assisted Scientific ResearchLearning Using Privileged Information

  16. Online LLM Personalization from Pairwise Preferences Across Users

    Oct 12, 2026Romain Puech, Georgy Salakhutdinov, Dimitris BertsimasEducational QALLM Personalization

  17. Can You Trust Your Preference Models? Stackelberg Games for Alignment

    Oct 10, 2026Riccardo Formenti, Aditya Agrawal, Vikas GargLLM AlignmentPreference-Based RL

  18. Uniform Information Density-Based Preference Optimization

    Oct 10, 2026Yongyuan Li, Yang XuPreference OptimizationLLM Alignment

  19. EmoGavel: Grounding-Aware Reinforcement Learning with Verifiable Rewards for Multimodal Emotion Preference Learning

    Oct 9, 2026Zilong Huang, Kong Aik Lee, Zezhong Jin +2Pairwise Preference LearningReinforcement Learning with Verifiable Rewards

  20. Groupwise Distortion Guarantees for Preference-Based Alignment

    Oct 4, 2026Jacob Brodkey, Roberto Tamez, Aaron RothPairwise Preference LearningSocial Choice Theory

  21. Targeted Active Learning for Preference-Based Treatment Effects on Multivariate Outcomes

    Oct 2, 2026Lola Giordani, Mathieu Even, Chloé Geoffroy +3Active Preference LearningIndividualized Treatment Effect Estimation

  22. On the Complexity of Preference-Based Bandits

    Sep 30, 2026Ahmed Ben Yahmed, Marc Abeille, Clément CalauzènesPairwise Preference LearningMulti-Armed Bandits

  23. Optimal Design for Active Preference Learning with Biased LLM Judges

    Sep 30, 2026Zhongman Du, Huiming Zhang, Haodong Zhu +1Pairwise Preference LearningLLM Alignment

  24. Uncertainty-Normalized Margins for Direct Preference Optimization

    Sep 29, 2026Sadegh Khorasani, Petrus Mikkola, Matthias GrossglauserPairwise Preference LearningDirect Preference Optimization