Preference Alignment Learning

Follow this topic Ask about this topic

Momentum

52 papers in the last four weeks, up 333% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 271

All topics

Behavioral and Representational Evidence of Binomial Ordering Preferences in Large Language Models
Jun 19, 2026Zhiqing Yang, Yilun Liu, Yunpu Ma +2Large Language Model Bias Linguistics
Ludwig Maximilian University of Munich · Munich Center for Machine Learning
Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Users
Jun 18, 2026Haw-Shiuan Chang, Jeffrey Gomez, Mehul Patwari +2Large Language Model Alignment Large Language Model Evaluation
University of Massachusetts, Amherst, USA · York University, Canada
Through the PRISM: Preference Representation in Intermediate States of Video Diffusion Models
Jun 18, 2026Haoxuan Wu, Lai Man Po, Mengyang Liu +3Video Diffusion Models Video Foundation Models
City University of Hong Kong · Video Rebirth · The Chinese University of Hong Kong
Beyond Uniform Forgetting: A Study of Sequential Direct Preference Optimization Across Preference Settings
Jun 18, 2026Pranav Bhandari, Nicolas Fay, Amitava Datta +2Backtranslation Augmented Direct Preference Optimization Preference Alignment Learning
1Network Analysis and Social Influence Modelling (NASIM) Lab · School of Physics Maths and Computing, The University of Western Australia · School of Psychological Science, The University of Western Australia +1
Steerable Cultural Preference Optimization of Reward Models
Jun 17, 2026Minsik Oh, Advit Deepak, Sophie Wu +2Large Language Model Alignment Sequential Preference Optimization
1Stanford University · University of Amsterdam
Prompt Perturbation for Reliable LLM Evaluation over Comparison Graphs
Jun 16, 2026Dong Huang, Jianbo Sun, Pengkun YangLarge Language Model Evaluation Ranking
Department of Statistics and Data Science, Tsinghua University
Whose hotel does the AI recommend? An algorithm audit of reputation signals in LLM-assisted hotel selection
Jun 15, 2026Mirza Samad Ahmed Baig, Syeda Anshrah Gillani, Asher AliAirbnb Recommendation
Fandaqah, Al Khobar, Saudi Arabia. · Hamdard University, Karachi, Pakistan.
ChatPlanner: A Large Language Model Framework for Personalized Public Transit Routing
Jun 13, 2026Tingting Yang, Chenhao Xue, Jun ChenLarge Language Model Routing Transport
School of Engineering and Materials Science, Queen Mary University of London Mile End Road, London, E1 4NS, United Kingdom · Department of Engineering Science, University of Oxford Parks Road, Oxford, OX1 3PJ, United Kingdom
TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations
Jun 12, 2026Kevin Robledo, Matías I. Torres Galaz, Kumar Dixhant Rai +3Robot Policies Behavior Tree
Department of Computer Science California State University, Northridge · Department of Mechanical Engineering California State University, Northridge
When to Write and When to Suppress: Route-Specialized Dual Adapters for Memory-Assisted Knowledge Editing
Jun 12, 2026Baijia Zhang, Yining HuangMultimodal Knowledge Editing Peak Memory
Xinjiang University · Meta Emergence Laboratory
LLMs Contain Multitudes: How Deployment Context Reshapes Model-Level Preferences and Values
Jun 11, 2026Filip Trhlik, Aoife O'Flynn, Angela Yu +2Preference Alignment Learning Value
University of Cambridge · 2ALTA Institute · 3Leverhulme Centre for the Future of Intelligence +1
Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents
Jun 11, 2026Yujun Zhou, Kehan Guo, Haomin Zhuang +8Coding Agents Runtime Enforcement
University of Notre Dame · Tencent AI Lab · IBM Research
DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Tasks
Jun 11, 2026Jingxuan Han, Wei Liu, Mingyang Zhu +8Search Agents Agentic Benchmarks
University of Science and Technology of China · Meituan
Bandits for Efficient Experimentation: Adapting to Control Group, Preferences, and Context Drifts
Jun 8, 2026Udvas Das, Waris Radji, Debabrota Basu +1Bandits Preference Alignment Learning
Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 – CRIStAL F-59000 Lille, France
Counterfactual Transport Flows for Offline Conservative Trajectory Refinement
Jun 8, 2026Lena Krieger, Xuan Zhao, Zhuo Cao +3Model-Based Reinforcement Learning Long Refinement Trajectories
1IAS-8, Forschungszentrum J ulich, Germany · 2LMU Munich, Munich Center for Machine Learning (MCML), Germany · Department of Computer Science, Aarhus University, Denmark.
DynaCF: Mitigating Shortcut Learning in Reward Models via Dynamic Counterfactual Sensitivity
Jun 8, 2026Fengyuan Liu, Yongliang Miao, Zirui He +3Progress Reward Modeling Preference Alignment Learning
The Chinese University of Hong Kong, Shenzhen · New Jersey Institute of Technology · Institute of Computing Technology, CAS
ZIPP:Zero-shot Image Personalization from Personas
Jun 7, 2026Harini SI, Somesh Singh, Yaman Kumar Singla +2Persona Consistency Diffusion Models
Adobe Media and Data Science Research (MDSR) · SUNY at Buffalo · IIIT-Delhi
Whose Norms? Disentangling Cultural and Personal Alignment in Large Language Models
Jun 5, 2026Angana Borah, Isabelle Augenstein, Rada MihalceaLarge Language Model Alignment Large Language Model Bias
University of Michigan - Ann Arbor · University of Copenhagen
What Do People Actually Want From AI? Mapping Preference Plurality
Jun 4, 2026Julia Sepúlveda Coelho, Scott A. HaleArtificial Intelligence Alignment Preference Alignment Learning
Oxford Internet Institute, University of Oxford, United Kingdom · Meedan, United States
Statistical Priors for Implicit Preferences: Decoupling Skill Selection as a Local Harness in Personal Agents
Jun 4, 2026Zeyu Gan, Huayi Tang, Yong LiuPreference Learning Large Language Model Personalization
Gaoling School of Artificial Intelligence Renmin University of China Beijing, China
Can We Predict The Human Preference For Text-to-Image Content Prior To Generation And Is It Even Useful To Do So?
Jun 3, 2026Joong Ho Kim, Keith G. MillsText-To-Image Diffusion Models
LSU ATHENA Lab Baton Rouge, LA, U.S.A. 70803
Rethinking Sales Lead Scoring with LLM-based Hierarchical Preference Ranking
Jun 3, 2026Chenyu Zhang, Yiwen Liu, Yin Sun +4Sales Lead Preference Alignment Learning
Intelligent Business Team, Li Auto Inc.
Sparse Mixture-of-Experts Reward Models Learn Interpretable and Specialized Experts for Personalized Preference Modeling
Jun 2, 2026Yifan Wang, Jinyi Mu, Mayank Jobanputra +5Preference Alignment Learning Experts
Saarland University · Independent Researcher · Bielefeld University +2
A Dataset for Dynamic Human Preferences for Vision Language Models
Jun 2, 2026Hannah Gao, Dylan Hadfield-Menell, Rachel MaPreference Datasets Preference Alignment Learning
Massachusetts Institute of Technology · MIT Cambridge, MA
Follow-Your-Preference++: Rethinking Preference Alignment for Image Inpainting
Jun 2, 2026Junkun Yuan, Yutao Shen, Toru Aonishi +2Preference Alignment Preference Alignment Learning
Zhejiang University · The University of Tokyo · Tsinghua University
Memory Retrieval for Changing Preferences
Jun 2, 2026Yuehan Qin, Li Li, Linxin Song +4Interaction History Preference Alignment Learning
University of Southern California
Auditing Asset-Specific Preferences in Financial Large Language Models: Evidence from Bitcoin Representations and Portfolio Allocation
Jun 1, 2026Wenbin WuFinancial Strategy Research Large Language Model Decisions
Cambridge Centre for Alternative Finance, University of Cambridge
Drifting Preference Optimization for One-Step Generative Models
Jun 1, 2026Zhou Jiang, Yandong Wen, Zhen LiuDiffusion Alignment Few-Step Generation
Westlake University · The Chinese University of Hong Kong, Shenzhen
Time-Aware Diffusion based on Preference Disentanglement for Generative Recommendation
Jun 1, 2026Bangguo Zhu, Peng Huo, Yuanbo Zhao +3Real-World Content Recommendation Problem Preference Alignment Learning
Central South University Changsha, China · National Super Computing Center Tianjin, China · Renmin University of China Beijing, China +1
From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning
May 31, 2026Jun-Jie Yang, Chia-Heng Hsu, Kui-Yuan Chen +1Reinforcement Learning From Human Feedback Model-Based Reinforcement Learning
Department of Computer Science, National Yang Ming Chiao Tung University, Hsinchu, Taiwan
Large Language Models Should Learn Personalized Rather Than Aggregated Human Preferences
May 30, 2026Cristina GarbaceaLarge Language Model Personalization Preference Alignment Learning
University of Chicago, Data Science Institute, Chicago, USA.
Skill or Skip? Learning Selective Skill Invocation in Agentic Tasks via Dual-Granularity Preference Learning
May 30, 2026Chishui Chen, Jiaye Lin, Te Sun +6Skills Agentic Reinforcement Learning
Meituan · Shanghai Jiao Tong University · Nanjing University +2
Reinforcement Learning with Pairwise Preferences in Long-Term Decision Problems
May 29, 2026Jonathan Colaço Carr, Prakash Panangaden, Doina Precup +1Markov Decision Processes Optimal Policies
Stanford University · School of Computer Science, McGill University, Montreal, Quebec, Canada · Mila - Quebec AI Institute, Montreal, Quebec, Canada +1
The Representation-Rationalizability Tradeoff in Reward Learning
May 29, 2026Jing Dong, Yaoliang Yu, Pascal PourpartReinforcement Learning From Human Feedback Reward Functions
Vector Institute · University of Waterloo & Vector Institute
Federated Variational Preference Alignment with Gumbel-Softmax Prior for Personalized User Preferences
May 29, 2026Jabin Koo, Hoyoung Kim, Minwoo Jang +1Preference Learning Preference Alignment Learning
Department of CSE, POSTECH, Pohang, Republic of Korea · National AI Research Lab, Seoul, Republic of Korea · Graduate School of AI, POSTECH, Pohang, Republic of Korea
Pairwise Reference Alignment as a Model-Level Ordinal Observable
May 29, 2026Mujing LiPreference Alignment Preference Alignment Learning
Independent Researcher
In-Context Reward Adaptation for Robust Preference Modeling
May 28, 2026Zhenyu Sun, Zheng Xu, Ermin WeiReinforcement Learning From Human Feedback Preference Alignment Learning
Northwestern University · Meta Superintelligence Labs
From Blind Guess to Informed Judgment: Teaching LLMs to Evaluate Materials by Building Knowledge-Augmented Preference Signals
May 28, 2026Yeyong Yu, Wenya Hu, Xing Wu +1Data-Driven Materials Discovery Materials
School of Computer Engineering & Science, Shanghai University, Shanghai, 200444, China. · Center of Materials Informatics and Data Science, Materials Genome2026 Institute, Shanghai University, Shanghai, 200444, China. · Key Laboratory of Silicate Cultural Relics Conservation (Shanghai University), Ministry of Education, China. +1
Toward User Preference Alignment in LLM Recommendation via Explicit Context Feedback
May 27, 2026Weizhi Zhang, Wooseong Yang, Yuxin Cui +9Preference Alignment Learning Preference Alignment
University of Illinois Chicago, Chicago, USA · Meta, Menlo Park, USA
AdaDPO: Self-Adaptive Direct Preference Optimization with Balanced Gradient Updates
May 27, 2026Shaolong Chen, Madalina Ciobanu, Qingqing Mao +1Backtranslation Augmented Direct Preference Optimization Large Language Model Adaptation
Incept Labs Houston, TX · Incept Labs, Houston, TX · Titan Holdings, San Francisco, CA
Rethinking Visual Neglect: Steering via Context-Preference for MLLM Hallucination Mitigation
May 27, 2026Jingwen Wu, Xijun Zhang, Ge SongHallucination Mitigation Multimodal Large Language Models
School of Computer and Electronic Information, Nanjing Normal University, China
StepOPSD: Step-Aware Online Preference Distillation for Agent Reinforcement Learning
May 26, 2026Yanfei Zhang, Xu Lin, Chenglin WuOnline-Policy Distillation Rp-Opsd
Independent Researcher · Tencent · DeepWisdom
L2Rec: Towards Dual-View Understanding of LLMs for Personalized Recommendation
May 26, 2026Pingjun Pan, Tingting Zhou, Peiyao Lu +3Large Language Model Personalization Recommendation Quality
Netease Cloud Music Hangzhou, China
Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories
May 26, 2026Sil Hamilton, David MimnoNarrative Generation Large Language Model Generation
Department of Information Science Cornell University
Beyond Pairwise Preferences: Listwise Reward-Aware Alignment for Diffusion Models
May 26, 2026Austin Wang, Jiaqi Han, Stefano Ermon +1Diffusion Alignment Text-To-Image Diffusion Models
Caltech · Stanford University
CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations
May 25, 2026Mike Zhang, Ali Basirat, Desmond ElliottMultilingual Language Models Cross-Lingual Consistency
Department of Computer Science (DIKU), University of Copenhagen · Centre for Language Technology (CST), University of Copenhagen · Pioneer Centre for Artificial Intelligence
Active Query Synthesis for Preference Learning
May 25, 2026Namrata Nadagouda, Nauman Ahad, Maegan Tucker +1Preference Learning Pool-Based Active Learning
Georgia Institute of Technology · Gauss Labs
Preferred, Not Safer: Pairwise Preference Is a Poor Proxy for Clinical Safety
May 25, 2026Fay Elhassan, David Sasu, Alexandra Kulinkina +2Large Language Model Evaluation Safety Claims
LiGHT Laboratory, EPFL, Lausanne, Switzerland
When In-Distribution Gains Fail: Evaluating Weak-to-Strong Reward Models under Preference Shift
May 25, 2026Khoi Le, Tri Cao, Phong Nguyen +5Preference Learning Preference Alignment Learning
National University of Singapore · VinUniversity · Nanyang Technological University
MATO: Multi-objective Personalized Alignment with Test-time Optimization for Large Language Models
May 25, 2026Linhao Luo, Thuy-Trang Vu, Van-Anh Nguyen +3Large Language Model Personalization Preference Alignment Learning
Monash University · Defence Science and Technology Group, Australia
Clarification Is Not Enough: Post-Clarification Answering Remains the Bottleneck in Multi-Turn QA
May 24, 2026Jinyan Su, Jennifer HealeyClarification Multi-Turn Interactions
1Cornell University · 2Adobe Research
From Correctness to Preference: A Framework for Personalized Agentic Reinforcement Learning
May 22, 2026Ranxu zhang, zeyang li, Jiacheng Huang +5Agentic Reinforcement Learning Reward-Only Policy Optimization Variants
University of Science and Technology of China · Alibaba Group
Human-in-the-Loop Multi-Agent Ventilator Decision Support with Contextual Bandit Preference Learning
May 22, 2026Sijia Li, Xiaoyu Tan, Qixing Wang +7Human-In-The-Loop Reinforcement Learning From Human Feedback
Shanghai University of Engineering Science, Shanghai, China · Tencent Youtu Lab, Tencent, China · Department of Critical Care Medicine, Shanghai Tenth People’s Hospital, Tongji University School of Medicine, Shanghai, China +3
Building a privacy-preserving Federated Recommender system for mobile devices
May 21, 2026Aasheesh SinghReal-World Content Recommendation Problem Privacy-Preserving Framework
Université de Montréal · Département d’informatique et de recherche opérationnelle · Faculté des arts et des sciences
PRISM: Preference-Aware Influence Function Based Data Selection Method for Efficient Fine-Tuning
May 20, 2026Qihao Lin, Guanxu Chen, Dongrui Liu +1Alignment-Aware Selection Model Fine-Tuning
Shanghai Artificial Intelligence Laboratory
APM: Evaluating Style Personalization in LLMs with Arbitrary Preference Mappings
May 20, 2026Philipp Spohn, Leander Girrbach, Zeynep AkataLarge Language Model Personalization Personalization
Technical University of Munich, Helmholtz Munich · Munich Center for Machine Learning (MCML)
When Preference Labels Fall Short: Aligning Diffusion Models from Real Data
May 19, 2026Weiyan Chen, Weijian Deng, Yao Xiao +5Diffusion Alignment Preference Alignment
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Australian National University, Canberra, Australia +2
CriterAlign: Criterion-Centric Rationale Alignment for Code Preference Judging
May 19, 2026Zhenyu Li, Aleksandar Cvejic, Zehui Chen +1Preference Alignment Code Quality
KAUST · USTC
Efficient Elicitation of Collective Disagreements
May 19, 2026Mohamed Ouaguenouni, Felipe Garrido-Lucero, Umberto Grandi +2Disagreement Majority Voting
IRIT, Université Toulouse Capitole, Toulouse, France · Center for Collective Learning, IAST, Toulouse School of Economics, Toulouse, France · Center for Collective Learning, CIAS, Corvinus University of Budapest, Budapest, Hungary +2
LambdaPO: A Lambda Style Policy Optimization for Reasoning Language Models
May 19, 2026Redacted by arXivFrictive Policy Optimization Large Language Model Alignment

Jun 19, 2026, cs.CLJ/K move · Enter open · S save

Behavioral and Representational Evidence of Binomial Ordering Preferences in Large Language Models

Zhiqing Yang, Yilun Liu, Yunpu Ma, Volker Tresp+1

Ludwig Maximilian University of Munich · Munich Center for Machine Learning

Large Language Model Bias Linguistics Preference Alignment Learning Distributional Alignment

Open paper AskPDF arXiv

Large language models (LLMs) can readily reproduce conventional expressions, yet their ability to model gradient frequency distributions remains underexplored. We investigate this using linguistic binomials, such as men and women, where both word permutations are grammatically valid but exhibit distinct, cross-linguistic variations in conventionality. We formalize binomial ordering as a distributional alignment problem, and construct a multilingual dataset of 600 binomial pairs across 8 languages. With categorical and distributional metrics, we measure and compare the corpus-derived preferences with model-induced ordering probabilities of 6 open-weight LLMs. While models often behaviorally recover the dominant corpus-preferred order, particularly for strongly conventionalized pairs, they align less well with the exact corpus preference distributions. This suggests that apparent directional order overstates how faithfully LLMs capture the statistical nuances of language use. Sparse probing verifies that the concept of preference strength is partially encoded among middle-to-late layers, and steering along probe-derived directions alters model-induced ordering distributions, demonstrating that the statistical behavioral preference of LLMs can be mechanistically measured and manipulated via internal representations.

Momentum

Related topics

Latest papers 271