Human Preference Modeling

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 44

All topics

In-Context Reward Adaptation for Robust Preference Modeling
May 28, 2026Zhenyu Sun, Zheng Xu, Ermin WeiHuman Preference Modeling In-Context Learning
Northwestern University · Meta Superintelligence Labs
Learning What Evaluators Value: A Reliable Approach to Modeling Evaluator Preferences
May 15, 2026Madeline Celi Kitch, Nihar B. ShahHuman Preference Modeling Preference Learning
Carnegie Mellon University
PRISM-X: Experiments on Personalised Fine-Tuning with Human and Simulated Users
May 13, 2026Hannah Rose Kirk, Liu Leqi, Fanzhi Zeng +4LLM Evaluation LLM Sycophancy
University of Oxford · UK AI Security Institute · University of Texas at Austin +2
Response Time Enhances Alignment with Heterogeneous Preferences
May 7, 2026Federico Echenique, Alireza Fallah, Baihe Huang +1Pairwise Preference Learning LLM Alignment
Department of Economics, University of California, Berkeley · Department of Computer Science and Ken Kennedy Institute, Rice University · Departments of Electrical Engineering and Computer Sciences, University of California, Berkeley +2
Mitigating Cognitive Bias in RLHF by Altering Rationality
May 7, 2026Tiffany Horter, Andrew Markham, Niki Trigoni +1Human Preference Modeling RL from Human Feedback
University of Oxford · Brown University
StoryAlign: Evaluating and Training Reward Models for Story Generation
May 6, 2026Haotian Xia, Hao Peng, Yunjia Qi +4Reward Modeling Language Model Generation Evaluation
Department of Computer Science and Technology, Tsinghua University
Putting HUMANS first: Efficient LAM Evaluation with Human Preference Alignment
Apr 20, 2026Woody Haosheng Gan, William Held, Diyi YangAudio-Language Model Evaluation Human Preference Modeling
University of Southern California · Stanford University · OpenAthena
Efficient Personalization of Generative User Interfaces
Apr 10, 2026Yi-Hao Peng, Jeffrey P. Bigham, Jason WuPairwise Preference Learning Human Preference Modeling
Massachusetts Institute of Technology · Carnegie Mellon University · Purdue University
Who Laughs with Whom? Disentangling Influential Factors in Humor Preferences across User Clusters and LLMs
Jan 6, 2026Soichiro Murakami, Hidetaka Kamigaito, Hiroya Takamura +1LLM Evaluation Human Preference Modeling
CyberAgent · Nara Institute of Science and Technology · Institute of Science Tokyo
HAL: Inducing Human-likeness in LLMs with Alignment
Jan 6, 2026Masum Hasan, Junjie Zhao, Ehsan HoqueReward Modeling LLM Alignment
University of Rochester
Harmonious Color Pairings: Insights from Human Preference and Natural Hue Statistics
Aug 3, 2025Ortensia Forni, Alexandre Darmon, Michael BenzaquenHuman Preference Evaluation Human Preference Modeling
Econophysics Lab, Institut Louis Bachelier, 28 Pl. de la Bourse, Palais Brongniart, 75002 Paris, France · LadHyX UMR CNRS 7646, École Polytechnique, Institut Polytechnique de Paris, 91128 Palaiseau Cedex, France · Art in Research, 33 rue Censier, 75005 Paris, France +1
DPO Unchained: Your Training Algorithm is Secretly Disentangled in Human Choice Theory (and its Loss' Convexity is Dispensable)
Jul 10, 2025Wenxuan Zhou, Shujian Zhang, Brice Magdalou +4Direct Preference Optimization Human Preference Modeling
Google DeepMind · CEE-M, Univ Montpellier, CNRS, INRAE, Institut Agro · Work done while at Google DeepMind +1
A Probabilistic Approach for Model Alignment with Human Comparisons
Mar 16, 2024Junyu Cao, Mohsen BayatiHuman Preference Modeling Statistical Learning Theory
McCombs School of Business, The University of Texas at Austin · Graduate School of Business, Stanford University
KTO: Model Alignment as Prospect Theoretic Optimization
Feb 2, 2024Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff +2LLM Alignment Human Preference Modeling
Stanford University · Contextual AI

May 28, 2026, cs.LGJ/K move · Enter open · S save

In-Context Reward Adaptation for Robust Preference Modeling

Zhenyu Sun, Zheng Xu, Ermin Wei

Northwestern University · Meta Superintelligence Labs

Human Preference Modeling In-Context Learning RL from Human Feedback

Open paper AskPDF arXiv

Reinforcement Learning from Human Feedback (RLHF) typically relies on static reward models to align Large Language Models with human preferences. However, human values are inherently diverse and heterogeneous, and a single reward model often lacks the robustness required to generalize to unseen preference domains. While existing multi-reward frameworks attempt to address this, they are often restricted to a fixed set of known domains and fail to adapt to unseen human distributions without costly retraining. In this work, we propose In-Context Reward Adaptation, a transformer-based framework designed to model diverse and unseen human preferences on the fly. By leveraging the in-context learning capabilities of transformers, our approach adaptively infers the underlying reward structure from a small set of preference demonstrations. We demonstrate that while a standard transformer architecture is insufficient for this task by characterizing an asymptotic bias to the ground-truth, incorporating human response time as an auxiliary input signal enables the model to successfully adapt to preferences from previously unseen domains. Our findings show that this approach provides a more robust foundation for preference modeling, allowing for the representation of heterogeneous rewards and preference distribution shift, and offering a scalable path toward more flexible human-AI alignment.

Momentum

Related topics

Latest papers 44