Pairwise Preference Learning

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 300% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 81

All topics

Covariate-dependent Joint Modeling of Multivariate Ordinal Preferences and Its Connections with Comparison Models
Oct 6, 2026Yujie Chen, Antik Chakraborty, Anindya BhadraPreference Learning Ordinal Regression
Department of Statistics Purdue University
Groupwise Distortion Guarantees for Preference-Based Alignment
Oct 4, 2026Jacob Brodkey, Roberto Tamez, Aaron RothPairwise Preference Learning Social Choice Theory
University of Pennsylvania
Learning in Continuous Games from Pairwise Preference Feedback
Oct 4, 2026Anas BarakatPairwise Preference Learning Nash Equilibrium Computation
Singapore University of Technology and Design
On the Complexity of Preference-Based Bandits
Sep 30, 2026Ahmed Ben Yahmed, Marc Abeille, Clément CalauzènesPairwise Preference Learning Multi-Armed Bandits
CREST, ENSAE Paris, FAIRPLAY · FAIRPLAY
The Missing Coefficients: Bayesian Pairwise Merging for Model Personalization
Sep 30, 2026Yaling Shen, Tongtong Wu, Siyuan Yan +1Pairwise Preference Learning Personalized Language Models
Monash University
Optimal Design for Active Preference Learning with Biased LLM Judges
Sep 30, 2026Zhongman Du, Huiming Zhang, Haodong Zhu +1Pairwise Preference Learning LLM Alignment
Beihang University · Zhongguancun Academy · Hangzhou Innovation Institute of Beihang University
Uncertainty-Normalized Margins for Direct Preference Optimization
Sep 29, 2026Sadegh Khorasani, Petrus Mikkola, Matthias GrossglauserPairwise Preference Learning Direct Preference Optimization
School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland · University of Helsinki, Helsinki, Finland
Using Context Is Not Enough: Test-Time Training for Personalized Reward Modeling
Sep 28, 2026Bohao Wang, Xiaoyan Zhao, Yang Zhang +4Pairwise Preference Learning Reward Modeling
Zhejiang University · National University of Singapore
From Soft Targets to Reward Signals: How Assignment and Reward Objectives Interact
Sep 28, 2026Jiangtao Lin, Bangyang Wei, Siyi Liu +2Pairwise Preference Learning Reward Modeling
Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · Tencent Holdings Limited +1
PrefLUT: Reusable and Refinable Personalized Color Editing from Pairwise Preferences
Sep 28, 2026Chuanzhi Xu, Langyi Chen, Chengkun Yue +5Pairwise Preference Learning User Preference Modeling
The University of Sydney · Indiana University · The University of Hong Kong
A Zeroth-Order Paradigm for LLM Preference Alignment
Sep 16, 2026Peter Chen, Xi Chen, Wotao Yin +1Pairwise Preference Learning LLM Alignment
Department of Electrical Engineering and Computer Sciences (EECS) University of California, Berkeley Berkeley, CA 94720, USA · Stern School of Business New York University New York, NY 10012, USA · Decision Intelligence Lab (Seattle) DAMO Academy, Alibaba Group U.S. Bellevue, WA 98004, USA +1
Learning Heterogeneous Preferences
Sep 15, 2026Shiwali Mohan, Matt Hong, Dule Shu +3Pairwise Preference Learning Human Preference Modeling
Human-Centered AI Toyota Research Institute Los Altos, CA
FlowCPO: A Unified Divergence View of Preference Alignment for Flow Models
Sep 9, 2026Yansen Han, Shengyi Liao, Peng Sun +4Pairwise Preference Learning Flow Matching
Westlake University · Zhejiang University · Kling Team, Kuaishou Technology
Low-Resource Preference Adaptation of LLMs via Activation-Based Label Propagation
Aug 31, 2026Alessio Galatolo, Meriem BeloucifPairwise Preference Learning LLM Alignment
Uppsala University
PLC-DPO: Posterior Label Correction in Noisy and Ambiguous Preference Optimization
Aug 31, 2026Boryeong Cho, Sumyeong Ahn, Se-Young YunPairwise Preference Learning Direct Preference Optimization
KAIST AI · KENTECH
Finding the Signal in the Spam: Jointly Learning Rewards and Worker Reliability from Pairwise Comparisons
Aug 10, 2026Kaustubh Shivshankar Shejole, Tanish Agarwal, Arpit Agarwal +1Pairwise Preference Learning Reward Modeling
IIT Bombay, Mumbai, India – 400076
Isotonic Bradley-Terry Model for Paired Comparison Data
Aug 3, 2026Ryoya YamasakiPairwise Preference Learning Pairwise Comparison
Hitotsubashi Institute for Advanced Study, Hitotsubashi University, 2-1 Naka, Kunitachi, 186-8601, Tokyo, Japan.
Rethinking Personalized Reward Modeling for LLMs under Preference Heterogeneity via Group-Debiased Federated Learning
Aug 3, 2026Seongyoon Kim, Boryeong Cho, Jihwan Oh +2Pairwise Preference Learning Reward Modeling
Institute of Engineering Research, Korea University Seoul, Republic of Korea · Kim Jaechul Graduate School of AI, KAIST Seoul, Republic of Korea · Industrial Management Engineering, Korea University Seoul, Republic of Korea
Ranking Infrared-Visible Fusion the Way Humans Do: A Learned Pairwise Preference Measure
Aug 2, 2026Haoran Liu, Mingzhe Liu, Peng Li +1Pairwise Preference Learning Multi-Source Image Fusion
School of Artificial Intelligence and Electronic Engineering, Sichuan Technology and Business University, Chengdu 611745, China · College of Nuclear Technology and Automation Engineering, Chengdu University of Technology, Chengdu 610059, China · National Synchrotron Radiation Laboratory, University of Science and Technology of China, Hefei 230029, China
Synchronizing Beliefs with Second-Order Theory-of-Mind in Human-Autonomy Teams (Extended Version)
Jul 29, 2026Jack Mirenzi, Henny AdmoniPairwise Preference Learning Theory of Mind
Robotics Institute, Carnegie Mellon University
BridgeAlign: Bridging Preference Alignment for Humanities and Social Sciences
Jul 29, 2026Ru Peng, Haokai Xu, Xijun Gu +11Pairwise Preference Learning Preference Optimization
Zhejiang University · Inclusion AI, Ant Group · Qwen Team, Alibaba Group +2
CSPF: A Constrained Shared-Private Fusion Method for Non-Verifiable Preference Evaluation
Jul 23, 2026Hehao Zhang, Danli Wang, Xinyuan Wang +1Pairwise Preference Learning Reward Modeling
Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
Cardinality-Decomposed Loss: Matching Training Objectives to Relation Structure in Heterogeneous Recommendation Graphs
Jul 22, 2026Parul Maheshwari, Amulya Paruchuri, Yiqing Zou +3Pairwise Preference Learning Heterogeneous GNNs
PayPal AI · PayPal Inc. · USA
Generalizing Preference-based Reinforcement Learning: a Rationality Model for Incomparability
Jul 13, 2026Simone Drago, Marco Mussi, Leonardo Bianconi +1Pairwise Preference Learning Pairwise Comparison
Politecnico di Milano, Milan, Italy
Comparison-Based Ordinal Learning for Proactive Driving Risk Assessment
Jul 13, 2026Zhuoren Li, Yi Zhong, Weiqi Zhang +4Pairwise Preference Learning Collision Avoidance
College of Automotive and Energy Engineering, Tongji University, Shanghai, China · James Watt School of Engineering, University of Glasgow, Glasgow, UK
Optimal Top- $k$ Identification from Pairwise Comparisons
Jul 9, 2026Motti Goldberger, Nils RudiPairwise Preference Learning Multi-Armed Bandits
Yale University, New Haven, CT, USA.
Attention Limited Reward Learning
Jul 6, 2026Wenqian XingPairwise Preference Learning Reward Modeling
Management Science and Engineering Department, Stanford University
Unbiased Alignment for Large Language Models with Noisy Preferences
Jul 3, 2026Jialiang Wang, Xianming Liu, Xiong Zhou +2Pairwise Preference Learning LLM Alignment
1Harbin Institute of Technology · 2City University of Hong Kong.
Internal Pluralism and the Limits of Pairwise Comparisons
Jul 2, 2026Bailey Flanigan, Michelle SiPairwise Preference Learning Pairwise Comparison
MIT, USA · Harvard, USA
Democratic ICAI: Debating Our Way to Steering Principles from Preferences
Jun 26, 2026Kevin Kingslin, Anish Natekar, Ashutosh Ranjan +3Pairwise Preference Learning Pairwise Preference Evaluation
TCS Research, India

Oct 6, 2026, stat.MLJ/K move · Enter open · S save

Covariate-dependent Joint Modeling of Multivariate Ordinal Preferences and Its Connections with Comparison Models

Yujie Chen, Antik Chakraborty, Anindya Bhadra

Department of Statistics Purdue University

Preference Learning Ordinal Regression Markov Random Fields Pairwise Preference Learning

Open paper AskPDF arXiv

Multivariate ordinal data along with covariates are commonly collected in problems ranging from alignment of language models with human preferences, as well as in recommender systems. For example, data sets such as MovieLens contain several movies rated on a scale 1--5 by human users, along with their demographic information such as age or gender. Similarly, data sets such as HelpSteer collect human feedback on several attributes such as "helpfulness" or "verbosity" of LLM response on an ordinal scale, with covariates depending on the LLM prompt--response pairs. Unfortunately, the standard approaches for modeling these data (a) look at the attributes individually rather than jointly, and (b) often convert the data into pairwise or list-wise win--loss comparisons for fitting models such as Bradley--Terry and Plackett--Luce. Both of these lead to a coarsening of what is actually observed, which we address via a joint covariate-dependent consecutive ratio Markov random field model. We also show pairwise or listwise comparison models are obtained under restrictions of our joint model, and that joint modeling improves comparisons. We also develop a maximum likelihood inference procedure even in the presence of an intractable normalizer.

Momentum

Related topics

Latest papers 81