Pairwise Preference Learning

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 300% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 81

All topics

Covariate-dependent Joint Modeling of Multivariate Ordinal Preferences and Its Connections with Comparison Models
Oct 6, 2026Yujie Chen, Antik Chakraborty, Anindya BhadraPreference Learning Ordinal Regression
Department of Statistics Purdue University
Groupwise Distortion Guarantees for Preference-Based Alignment
Oct 4, 2026Jacob Brodkey, Roberto Tamez, Aaron RothPairwise Preference Learning Social Choice Theory
University of Pennsylvania
Learning in Continuous Games from Pairwise Preference Feedback
Oct 4, 2026Anas BarakatPairwise Preference Learning Nash Equilibrium Computation
Singapore University of Technology and Design
On the Complexity of Preference-Based Bandits
Sep 30, 2026Ahmed Ben Yahmed, Marc Abeille, Clément CalauzènesPairwise Preference Learning Multi-Armed Bandits
CREST, ENSAE Paris, FAIRPLAY · FAIRPLAY
The Missing Coefficients: Bayesian Pairwise Merging for Model Personalization
Sep 30, 2026Yaling Shen, Tongtong Wu, Siyuan Yan +1Pairwise Preference Learning Personalized Language Models
Monash University
Optimal Design for Active Preference Learning with Biased LLM Judges
Sep 30, 2026Zhongman Du, Huiming Zhang, Haodong Zhu +1Pairwise Preference Learning LLM Alignment
Beihang University · Zhongguancun Academy · Hangzhou Innovation Institute of Beihang University
Uncertainty-Normalized Margins for Direct Preference Optimization
Sep 29, 2026Sadegh Khorasani, Petrus Mikkola, Matthias GrossglauserPairwise Preference Learning Direct Preference Optimization
School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland · University of Helsinki, Helsinki, Finland
Using Context Is Not Enough: Test-Time Training for Personalized Reward Modeling
Sep 28, 2026Bohao Wang, Xiaoyan Zhao, Yang Zhang +4Pairwise Preference Learning Reward Modeling
Zhejiang University · National University of Singapore
From Soft Targets to Reward Signals: How Assignment and Reward Objectives Interact
Sep 28, 2026Jiangtao Lin, Bangyang Wei, Siyi Liu +2Pairwise Preference Learning Reward Modeling
Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · Tencent Holdings Limited +1
PrefLUT: Reusable and Refinable Personalized Color Editing from Pairwise Preferences
Sep 28, 2026Chuanzhi Xu, Langyi Chen, Chengkun Yue +5Pairwise Preference Learning User Preference Modeling
The University of Sydney · Indiana University · The University of Hong Kong
A Zeroth-Order Paradigm for LLM Preference Alignment
Sep 16, 2026Peter Chen, Xi Chen, Wotao Yin +1Pairwise Preference Learning LLM Alignment
Department of Electrical Engineering and Computer Sciences (EECS) University of California, Berkeley Berkeley, CA 94720, USA · Stern School of Business New York University New York, NY 10012, USA · Decision Intelligence Lab (Seattle) DAMO Academy, Alibaba Group U.S. Bellevue, WA 98004, USA +1
Learning Heterogeneous Preferences
Sep 15, 2026Shiwali Mohan, Matt Hong, Dule Shu +3Pairwise Preference Learning Human Preference Modeling
Human-Centered AI Toyota Research Institute Los Altos, CA
FlowCPO: A Unified Divergence View of Preference Alignment for Flow Models
Sep 9, 2026Yansen Han, Shengyi Liao, Peng Sun +4Pairwise Preference Learning Flow Matching
Westlake University · Zhejiang University · Kling Team, Kuaishou Technology
Low-Resource Preference Adaptation of LLMs via Activation-Based Label Propagation
Aug 31, 2026Alessio Galatolo, Meriem BeloucifPairwise Preference Learning LLM Alignment
Uppsala University
PLC-DPO: Posterior Label Correction in Noisy and Ambiguous Preference Optimization
Aug 31, 2026Boryeong Cho, Sumyeong Ahn, Se-Young YunPairwise Preference Learning Direct Preference Optimization
KAIST AI · KENTECH
Finding the Signal in the Spam: Jointly Learning Rewards and Worker Reliability from Pairwise Comparisons
Aug 10, 2026Kaustubh Shivshankar Shejole, Tanish Agarwal, Arpit Agarwal +1Pairwise Preference Learning Reward Modeling
IIT Bombay, Mumbai, India – 400076
Isotonic Bradley-Terry Model for Paired Comparison Data
Aug 3, 2026Ryoya YamasakiPairwise Preference Learning Pairwise Comparison
Hitotsubashi Institute for Advanced Study, Hitotsubashi University, 2-1 Naka, Kunitachi, 186-8601, Tokyo, Japan.
Rethinking Personalized Reward Modeling for LLMs under Preference Heterogeneity via Group-Debiased Federated Learning
Aug 3, 2026Seongyoon Kim, Boryeong Cho, Jihwan Oh +2Pairwise Preference Learning Reward Modeling
Institute of Engineering Research, Korea University Seoul, Republic of Korea · Kim Jaechul Graduate School of AI, KAIST Seoul, Republic of Korea · Industrial Management Engineering, Korea University Seoul, Republic of Korea
Ranking Infrared-Visible Fusion the Way Humans Do: A Learned Pairwise Preference Measure
Aug 2, 2026Haoran Liu, Mingzhe Liu, Peng Li +1Pairwise Preference Learning Multi-Source Image Fusion
School of Artificial Intelligence and Electronic Engineering, Sichuan Technology and Business University, Chengdu 611745, China · College of Nuclear Technology and Automation Engineering, Chengdu University of Technology, Chengdu 610059, China · National Synchrotron Radiation Laboratory, University of Science and Technology of China, Hefei 230029, China
Synchronizing Beliefs with Second-Order Theory-of-Mind in Human-Autonomy Teams (Extended Version)
Jul 29, 2026Jack Mirenzi, Henny AdmoniPairwise Preference Learning Theory of Mind
Robotics Institute, Carnegie Mellon University
BridgeAlign: Bridging Preference Alignment for Humanities and Social Sciences
Jul 29, 2026Ru Peng, Haokai Xu, Xijun Gu +11Pairwise Preference Learning Preference Optimization
Zhejiang University · Inclusion AI, Ant Group · Qwen Team, Alibaba Group +2
CSPF: A Constrained Shared-Private Fusion Method for Non-Verifiable Preference Evaluation
Jul 23, 2026Hehao Zhang, Danli Wang, Xinyuan Wang +1Pairwise Preference Learning Reward Modeling
Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
Cardinality-Decomposed Loss: Matching Training Objectives to Relation Structure in Heterogeneous Recommendation Graphs
Jul 22, 2026Parul Maheshwari, Amulya Paruchuri, Yiqing Zou +3Pairwise Preference Learning Heterogeneous GNNs
PayPal AI · PayPal Inc. · USA
Generalizing Preference-based Reinforcement Learning: a Rationality Model for Incomparability
Jul 13, 2026Simone Drago, Marco Mussi, Leonardo Bianconi +1Pairwise Preference Learning Pairwise Comparison
Politecnico di Milano, Milan, Italy
Comparison-Based Ordinal Learning for Proactive Driving Risk Assessment
Jul 13, 2026Zhuoren Li, Yi Zhong, Weiqi Zhang +4Pairwise Preference Learning Collision Avoidance
College of Automotive and Energy Engineering, Tongji University, Shanghai, China · James Watt School of Engineering, University of Glasgow, Glasgow, UK
Optimal Top- $k$ Identification from Pairwise Comparisons
Jul 9, 2026Motti Goldberger, Nils RudiPairwise Preference Learning Multi-Armed Bandits
Yale University, New Haven, CT, USA.
Attention Limited Reward Learning
Jul 6, 2026Wenqian XingPairwise Preference Learning Reward Modeling
Management Science and Engineering Department, Stanford University
Unbiased Alignment for Large Language Models with Noisy Preferences
Jul 3, 2026Jialiang Wang, Xianming Liu, Xiong Zhou +2Pairwise Preference Learning LLM Alignment
1Harbin Institute of Technology · 2City University of Hong Kong.
Internal Pluralism and the Limits of Pairwise Comparisons
Jul 2, 2026Bailey Flanigan, Michelle SiPairwise Preference Learning Pairwise Comparison
MIT, USA · Harvard, USA
Democratic ICAI: Debating Our Way to Steering Principles from Preferences
Jun 26, 2026Kevin Kingslin, Anish Natekar, Ashutosh Ranjan +3Pairwise Preference Learning Pairwise Preference Evaluation
TCS Research, India
A Markov Chain Approach to Preference Alignment
Jun 21, 2026Takuya Koriyama, Tengyuan LiangPairwise Preference Learning Pairwise Comparison
The University of Chicago
Which Pairs to Compare for LLM Post-Training?
Jun 17, 2026Jiangze Han, Vineet Goyal, Will MaPairwise Preference Learning Pairwise Comparison
Columbia University
PrefSQA: Pairwise Preference Prediction for Speech Quality Assessment and the Critical Role of High Quality Datasets
Jun 17, 2026Junyi Fan, Donald S. WilliamsonPairwise Preference Learning Pairwise Preference Evaluation
Department of Computer Science and Engineering, The Ohio State University, USA
Learning Robust Pair Confidence for Multimodal Emotion-Cause Pair Extraction
Jun 17, 2026Zhuangzhuang Pan, Ning Dong, Yingna Su +1Pairwise Preference Learning Multimodal Robustness
Institute for Advanced Studies Universiti Malaya Kuala Lumpur 50603, Malaysia · School of Information Engineering Suqian University Suqian 223800, China · Digitization Department Suzhou University of Technology Suzhou 215500, China
Surprise-Guided MergeSort: Budget-Efficient Human-in-the-Loop Ranking via Adaptive Comparison Scheduling
Jun 14, 2026Yujin Park, Haejun Chung, Ikbeom JangPairwise Preference Learning Pairwise Comparison
Hanyang University, Seoul, Republic of Korea · Hankuk University of Foreign Studies, Yongin, Republic of Korea
Boosting Direct Preference Optimization with Penalization
Jun 10, 2026Pengwei SunPairwise Preference Learning LLM Alignment
Stanford University.
Reasoning Arena: Trace Tournaments When Verifiable Rewards Fall Short
Jun 8, 2026Han Zhou, Adam X. Yang, Laurence Aitchison +2Pairwise Preference Learning Reward Modeling
University of Cambridge · 2Mistral AI · *Work done as an AI Scientist Intern at Mistral AI.
DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivity
Jun 8, 2026Fengyuan Liu, Yongliang Miao, Zirui He +3Pairwise Preference Learning Reward Modeling
The Chinese University of Hong Kong, Shenzhen · New Jersey Institute of Technology · Institute of Computing Technology, CAS
Sparse Mixture-of-Experts Reward Models Learn Interpretable and Specialized Experts for Personalized Preference Modeling
Jun 2, 2026Yifan Wang, Jinyi Mu, Mayank Jobanputra +5Pairwise Preference Learning Reward Modeling
Saarland University · Independent Researcher · Bielefeld University +2
Follow-Your-Preference++: Rethinking Preference Alignment for Image Inpainting
Jun 2, 2026Junkun Yuan, Yutao Shen, Toru Aonishi +2Pairwise Preference Learning Reward Modeling
Zhejiang University · The University of Tokyo · Tsinghua University
Local Preferential Bayesian Optimization
Jun 1, 2026Johanna Menn, Miriam Kober, Paul Brunzema +2Pairwise Preference Learning Pairwise Preference Evaluation
Institute for Data Science in Mechanical Engineering, RWTH Aachen University, Aachen, Germany · Department of Clinical Research, University of Bern, Bern, Switzerland · Center for Reproducible Science and Research Synthesis, University of Zurich, Zurich, Switzerland +1
From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning
May 31, 2026Jun-Jie Yang, Chia-Heng Hsu, Kui-Yuan Chen +1Pairwise Preference Learning Representation Learning
Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
Enhancing LLM Metacognition via Cognitive Pairwise Training
May 30, 2026Weitao Li, Hao Zhou, Xuanyu Lei +11Pairwise Preference Learning RL for Language Model Reasoning
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China · Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China · WeChat AI, Tencent Inc., China
Reinforcement Learning with Pairwise Preferences in Long-Term Decision Problems
May 29, 2026Jonathan Colaço Carr, Prakash Panangaden, Doina Precup +1Pairwise Preference Learning Markov Decision Processes
Stanford University · School of Computer Science, McGill University, Montreal, Quebec, Canada · Mila - Quebec AI Institute, Montreal, Quebec, Canada +1
From "Weak" Signals to Strong Models: Preference Delta Aggregation with LoRA Merging
May 29, 2026Qi Sun, Siyue Zhang, Yulin Chen +3Pairwise Preference Learning Low-Rank Adaptation
NYU Shanghai · NTU · NYU +1
The Representation-Rationalizability Tradeoff in Reward Learning
May 29, 2026Jing Dong, Yaoliang Yu, Pascal PourpartPairwise Preference Learning Reward Modeling
Vector Institute · University of Waterloo & Vector Institute
Differentially Private Preference Data Synthesis for Large Language Model Alignment
May 29, 2026Fengyu Gao, Jing YangPairwise Preference Learning LLM Alignment
Department of Computer Science, University of Virginia, Charlottesville, Virginia, USA · Department of Electrical and Computer Engineering, University of Virginia, Charlottesville, Virginia, USA.
Reward Learning from Best-of- $N$ Preference Data: Targets, Tradeoffs, and Design Principles
May 28, 2026Rattana Pukdee, Maria-Florina Balcan, Pradeep RavikumarPairwise Preference Learning Reward Modeling
Machine Learning Department Carnegie Mellon University
Active Query Synthesis for Preference Learning
May 25, 2026Namrata Nadagouda, Nauman Ahad, Maegan Tucker +1Pairwise Preference Learning Active Learning
Georgia Institute of Technology · Gauss Labs
When In-Distribution Gains Fail: Evaluating Weak-to-Strong Reward Models under Preference Shift
May 25, 2026Khoi Le, Tri Cao, Phong Nguyen +5Pairwise Preference Learning Reward Modeling
National University of Singapore · VinUniversity · Nanyang Technological University
Learning Kernel-Based MDPs from Episodic Preferential Feedback
May 22, 2026Nikola Pavlovic, Sattar Vakili, Qing ZhaoPairwise Preference Learning Regret Minimization in RL
Cornell University · MediaTek Research
Convex Optimization for Alignment and Preference Learning on a Single GPU
May 22, 2026Miria Feng, Mert PilanciPairwise Preference Learning LLM Alignment
Department of Electrical Engineering, Stanford University, California, United States.
Token-weighted Direct Preference Optimization with Attention
May 21, 2026Chengyu Huang, Zhuohang Li, Sheng-Yen Chou +1Pairwise Preference Learning LLM Alignment
Cornell University · Vanderbilt University
Beyond Scalar Objectives: Expert-Feedback-Driven Autonomous Experimentation for Scientific Discovery at the Nanoscale
May 20, 2026Ralph Bulanadi, Jefferey Baxter, Arpan Biswas +5Pairwise Preference Learning Active Learning
Center for Nanophase Materials Sciences, Oak Ridge National Laboratory, Oak Ridge, TN 37831, USA · University of Tennessee-Oak Ridge Innovation Institute, University of Tennessee, Knoxville, TN 37996, USA · Department of Material Science and Engineering, School of Materials and Chemical Technology, Institute of Science Tokyo, Yokohama, 226-8502, Japan +3
Preferences Order, Ratings Anchor: From Fused Expert Aesthetic Ground Truth to Self-Distillation
May 19, 2026Yuanpei Zhao, Jie Lin, Chao Zhang +5Pairwise Preference Learning Personalized Image Aesthetics Assessment
1Sichuan University · 2NetEase Fuxi AI Lab · †Work done during an internship at NetEase Fuxi AI Lab. +1
CriterAlign: Criterion-Centric Rationale Alignment for Code Preference Judging
May 19, 2026Zhenyu Li, Aleksandar Cvejic, Zehui Chen +1Pairwise Preference Learning Human Preference Evaluation
KAUST · USTC
Pairwise Preference Reward and Group-Based Diversity Enhancement for Superior Open-Ended Generation
May 18, 2026Guining Cao, Jiaxin Peng, Chu Zeng +3Pairwise Preference Learning Open-Ended Generation
Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd · School of Software and Microelectronics, Peking University · Tsinghua University
Active Learners as Efficient PRP Rerankers
May 14, 2026Jeremías Figueiredo Paschmann, Juan Kaplan, Francisco Nattero +3Pairwise Preference Learning Pairwise Comparison
ELIAS Lab, Departamento de Ingeniería, Universidad de San Andrés
TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching
May 12, 2026Truong Nguyen, Tien-Phat Nguyen, Linh Ngo Van +3Pairwise Preference Learning LLM Alignment
Hanoi University of Science and Technology · University of Stuttgart · German Research Center for Artificial Intelligence +3
Variance-aware Reward Modeling with Anchor Guidance
May 12, 2026Shuxing Fang, Ruijian Han, Liangyu Zhang +1Pairwise Preference Learning Reward Modeling
Department of Data Science and Artificial Intelligence, Hong Kong Polytechnic University · School of Statistics and Data Science, Shanghai University of Finance and Economics

Oct 6, 2026, stat.MLJ/K move · Enter open · S save

Covariate-dependent Joint Modeling of Multivariate Ordinal Preferences and Its Connections with Comparison Models

Yujie Chen, Antik Chakraborty, Anindya Bhadra

Department of Statistics Purdue University

Preference Learning Ordinal Regression Markov Random Fields Pairwise Preference Learning

Open paper AskPDF arXiv

Multivariate ordinal data along with covariates are commonly collected in problems ranging from alignment of language models with human preferences, as well as in recommender systems. For example, data sets such as MovieLens contain several movies rated on a scale 1--5 by human users, along with their demographic information such as age or gender. Similarly, data sets such as HelpSteer collect human feedback on several attributes such as "helpfulness" or "verbosity" of LLM response on an ordinal scale, with covariates depending on the LLM prompt--response pairs. Unfortunately, the standard approaches for modeling these data (a) look at the attributes individually rather than jointly, and (b) often convert the data into pairwise or list-wise win--loss comparisons for fitting models such as Bradley--Terry and Plackett--Luce. Both of these lead to a coarsening of what is actually observed, which we address via a joint covariate-dependent consecutive ratio Markov random field model. We also show pairwise or listwise comparison models are obtained under restrictions of our joint model, and that joint modeling improves comparisons. We also develop a maximum likelihood inference procedure even in the presence of an intractable normalizer.

Momentum

Related topics

Latest papers 81