Preference Modeling

Follow this topic Ask about this topic

Latest papers

47 papers13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

Conditional independence and graphical models for rankings
Nov 3, 2026Aritz PérezProbabilistic Graphical Models Preference Modeling
Basque Center for Applied Mathematics (BCAM)
Predicting Human-Gain Curves under Local Proxy Optimization from Repeated Ratings
Oct 30, 2026Moonwon Choi, Kisung Nam, Seunggeun LeeLanguage Model Performance Prediction Human Preference Evaluation
Graduate School of Data Science, Seoul National University · Massachusetts General Hospital, Harvard Medical School · Broad Institute of MIT and Harvard
REGER: Modeling inter-item Relations with Semantic IDs for Generative Recommendation
Oct 27, 2026Yuxi Huang, Silin Zhou, Chenhao Wang +5Generative Recommendation Semantic ID Recommendation
University of Electronic Science and Technology of China
Modeling Preference Dependencies with Directed Acyclic Graphs for Narrative-Driven Recommendation
Oct 27, 2026Mingzhe Lu, Jian Qin, Ruize Qin +3Recommender Systems Preference Modeling
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Faculty of Engineering and IT, The University of Melbourne, Melbourne, Australia +1
Jointly Modeling Preferences and Imperfect Decision-Making in Large Language Models
Oct 24, 2026Axel Abels, Elias Fernández Domingos, Tom LenaertsPreference Modeling Behavioral Consistency of LLM Agents
Machine Learning Group, Université Libre de Bruxelles, Brussels, Belgium · Artificial Intelligence Lab, Vrije Universiteit Brussel, Brussels, Belgium · FARI Institute, Université Libre de Bruxelles and Vrije Universiteit Brussel, Brussels, Belgium +2
PAIR: Preference-Aligned Intent Retrieval for Premium People Recommendation at LinkedIn
Oct 22, 2026Zhongkai Shangguan, Yanbin Kang, Yihan Cao +7Recommender Systems Preference Modeling
LinkedIn, Sunnyvale, CA, USA
How Language Models Learn Taste: Brand Preference in LLMs
Oct 20, 2026Tapan ParikhLanguage Model Post-Training Preference Modeling
Cornell Tech
Auditing Safety Preference Supervision with Recorded Derivations
Oct 19, 2026Chenhan Lyu, Amir RahmaniLLM Auditing LLM Safety Alignment
University of California, Irvine Irvine, USA
Pluralistic Alignment in Large Language Models: Objectives, Human References, and Preference Representations
Oct 17, 2026Meng-Chen Wu, Umesh BodhwaniPluralistic Alignment Preference Modeling
Amazon Science
On the possibilities and limitations of LLM-based multi-agent systems for multi-criteria decision-making in healthcare operations management
Oct 16, 2026Marco Boresta, Tommaso Giovannelli, Sepehr Namirad +1Preference Modeling Multi-Criteria Decision Analysis
Istituto di Analisi dei Sistemi ed Informatica (IASI), CNR, Rome, Italy · Department of Mechanical, Materials, and Industrial Engineering, University of Cincinnati, Cincinnati, OH · Department of Computer, Control, and Management Engineering (DIAG), Sapienza University of Rome, Rome, Italy
Beyond Reward Scoring: Online Multi-Agent Preference Learning for Multimodal Response Generation
Oct 16, 2026Ziyun Zou, Jiwoo Hong, Yifan Lu +5Preference Alignment Preference-Based RL
Mohamed bin Zayed University of Artificial Intelligence · Nubank · Dartmouth College
Agentic-Priors: Using Large Language Models to Address the Cold Start Problem in Preference Elicitation
Oct 15, 2026Bruno Arcanjo, Zhaoxing Li, Sebastian SteinPreference Elicitation User Preference Modeling
University of Southampton, Southampton, United Kingdom
Online LLM Personalization from Pairwise Preferences Across Users
Oct 12, 2026Romain Puech, Georgy Salakhutdinov, Dimitris BertsimasEducational QA LLM Personalization
Massachusetts Institute of Technology
Covariate-dependent Joint Modeling of Multivariate Ordinal Preferences and Its Connections with Comparison Models
Oct 6, 2026Yujie Chen, Antik Chakraborty, Anindya BhadraPreference Modeling Preference Learning
Department of Statistics Purdue University
On the Complexity of Preference-Based Bandits
Sep 30, 2026Ahmed Ben Yahmed, Marc Abeille, Clément CalauzènesPairwise Preference Learning Multi-Armed Bandits
CREST, ENSAE Paris, FAIRPLAY · FAIRPLAY
Training LLM Judges from Language Feedback via Position-Selective Self-Distillation
Sep 30, 2026Ilgee Hong, Changlong Yu, Zhenghao Xu +5LLM-as-a-Judge Preference Modeling
Georgia Institute of Technology · Amazon · UC San Diego
JudgeProfile: Understanding and Steering Subjectivity in LLM Judges
Sep 29, 2026Qi Cao, Kangning Liu, Xuan Kan +10Human Preference Evaluation LLM Evaluation
Meta · University of California San Diego · The University of Hong Kong +1
PriceBench: A Diagnostic Benchmark for Price, Quality, and Brand Preferences in LLM Booking Agents
Sep 25, 2026Pavel KireyevLLM Agents LLM Agent Evaluation
Department of Management London School of Economics and Political Science
ZooWork-ShopRanker: An Open, Preference-Aligned E-Commerce Reranker
Sep 25, 2026Siqiao Xue, Shuxuan Liu, Ning HuE-Commerce Search LLM Reranking
ZooWork Team
Learning Heterogeneous Preferences
Sep 15, 2026Shiwali Mohan, Matt Hong, Dule Shu +3Pairwise Preference Learning Human Preference Modeling
Human-Centered AI Toyota Research Institute Los Altos, CA
VIBE-Bench: Evaluating Personalized Large Language Models When Profiles Don't Mean Preferences
Sep 1, 2026Yiwen Jiang, Yang Deng, Stephanie Fong +9LLM Evaluation LLM Personalization
1Monash University · 2Singapore Management University · University of Liverpool +1
AcCoRD: Evaluating User-Agent Collaboration Under Realistic User Preference Dynamics
Aug 28, 2026Tejas Srinivasan, Shikib Mehri, Nandita Shankar Naik +3LLM Agent Evaluation Human-AI Collaboration
University of Southern California · Contextual AI · Capital One +1
Multiple latent orderings better predict language model preferences
Aug 26, 2026Aviral Chawla, William H. W. Thompson, Jean-Gabriel YoungPreference Modeling Pairwise Preference Learning
Vermont Complex Systems Institute University of Vermont Burlington, Vermont 05403 · Vermont Complex Systems Institute Department of Statistics University of Vermont Burlington, Vermont 05403
MoPLEx: Estimating Plackett-Luce Mixture Models for Multi-Objective Alignment
Aug 25, 2026Dongyue Li, Ziniu Zhang, Lu Wang +1Learning to Rank Discrete Choice Modeling
Northeastern University, Boston, MA · University of Michigan, Ann Arbor, MI
LLM-Derived Preference Judgments Are Not Self-Consistent
Aug 18, 2026Matthew T. Ford, Francis Bahk, Jingjing Wang +4Preference Modeling User Preference Modeling
Cornell University · Georgia Institute of Technology
What do Reward Models Memorize?
Jul 27, 2026Ivo Verhoeven, Pushkar Mishra, Ekaterina ShutovaReward Modeling Pairwise Preference Evaluation
ILLC, University of Amsterdam, The Netherlands · Google DeepMind, London, United Kingdom
On the Role of Citations in Preference Data
Jul 6, 2026Yu Hou, Hal Daumé, Rachel Rudinger +1Preference Modeling
University of Maryland · Johns Hopkins University
Beyond expert users: agents should help users construct preferences, not just elicit them
Jun 29, 2026Irena Saracay, Ludwig Schmidt, Carlos GuestrinAI Agent Benchmarks Recommender Systems
Stanford University
Steerable Cultural Preference Optimization of Reward Models
Jun 17, 2026Minsik Oh, Advit Deepak, Sophie Wu +2Reward Modeling Preference Modeling
1Stanford University · University of Amsterdam
DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivity
Jun 8, 2026Fengyuan Liu, Yongliang Miao, Zirui He +3Pairwise Preference Learning Reward Modeling
The Chinese University of Hong Kong, Shenzhen · New Jersey Institute of Technology · Institute of Computing Technology, CAS

Nov 3, 2026J/K move · Enter open · S save

Conditional independence and graphical models for rankings

Aritz Pérez

Basque Center for Applied Mathematics (BCAM)

Probabilistic Graphical Models Preference Modeling Learning to Rank

Open paper AskPDF arXiv

Ranking models define probability distributions over rankings and are fundamental in applications such as preference learning and decision-making. Without structural assumptions, these models are intractable due to their factorial complexity. Although independence assumptions are key to reducing complexity and enabling interpretability, standard notions of conditional independence must account for the mutual exclusivity constraints inherent to rankings. We introduce relative rank models, a probabilistic framework based on structured coarsenings of the ranking space that relax these constraints while preserving relative order information. This representation enables a well-defined notion of conditional independence for rankings and allows rankings to be modeled using Bayesian networks. As a result, standard methods for learning and probabilistic inference can be applied to rankings, with interpretable representations based on conditional independence. We further show that two prominent ranking models based on independence assumptions tailored to rankings arise as special cases, providing a unifying framework for independence in ranking models.