Overview

The field develops ways to learn what people value from pairwise comparisons, ratings, choices, response times, and written explanations, then uses those preferences to evaluate or guide AI systems. Common approaches include Bradley–Terry models, learned reward models, and preference optimization methods such as RLHF and DPO, applied to language, image and video generation, recommendations, and robot behavior. A central challenge is that preferences vary across people and contexts, while aggregated labels and automated judges can miss disagreement, uncertainty, or tacit criteria. Current work explores personalization and pluralistic models, but must balance capturing diverse judgments with collecting reliable feedback at scale and preserving safety constraints.

Latest papers

CardsList

70 papers6 papers in the last four weeks, down 25% on the four weeks before. 0.0% of all new papers.

  1. The shape of individual differences in judgment: shared axes, personal weights, and when their spread crosses the consensus

    Nov 17, 2026Hiroshi HigashiHuman Preference Modeling

  2. Advancing Cognitive Pairwise Comparisons for Human-Centric Artificial Intelligence and Data Science: A Human-in-the-Loop Framework

    Nov 16, 2026Kevin Kam Fung YuenPairwise ComparisonHuman Preference Modeling

  3. Personalization of Basic and Moral Values Guiding LLM-Controlled Assistant Robots

    Nov 13, 2026Mateusz Wozniak, Qi Gao, Agnieszka WykowskaPersonalized Language Model AlignmentSocial Robotics

  4. Replication of Thermal Comfort Cohort Models: No Support for Improved Thermal Preference Prediction

    Nov 9, 2026Thomas Hirsch, Katja Ceplak, Veronika Windisch +5Preference ModelingHuman Preference Modeling

  5. CropReward: A Human-Preference Benchmark and Reward Model for Aesthetic Image Cropping

    Oct 25, 2026Xiangfei Sheng, Xiaofeng Pan, Leida Li +4Reward ModelingHuman Preference Modeling

  6. Attractive Faces Skew Text-to-Image Reward Models and Leaderboards

    Oct 20, 2026Harvey Mannering, Zhiwu Huang, Adam Prügel-BennettT2I Generation EvaluationHuman Preference Evaluation

  7. Reason-guided Human Reward Modeling for Preference-based Reinforcement Learning

    Oct 16, 2026Anshul Nayak, Shahil Shaik, Jonathan Smereka +1Preference-Based RLReward Modeling

  8. Perceptually Aligned Evaluation of Style Transfer

    Oct 7, 2026Yang Deng, Eleftherios Ioannou, David Mould +3Image Style TransferAutomated Evaluation

  9. Verifiable, Articulable, and Tacit Components of Preference

    Oct 2, 2026Alexander Spangher, Sheldon S. Huang, Andreas Haupt +4Human Preference ModelingPreference Learning

  10. SpeechCritic: Learning a Diagnostic Speech Judge from Limited Human Preferences

    Sep 28, 2026Mingyue Huo, Shivam Mehta, Bhavin Jawade +2Audio-Language Model EvaluationHuman Preference Modeling

  11. LLMAdBench: A Human Preference Benchmark for Advertising in LLM Responses

    Sep 26, 2026Rui Ai, Yuqing Liu, Sitao Qiu +14Language Model Generation EvaluationHuman Preference Modeling

  12. Learning Heterogeneous Preferences

    Sep 15, 2026Shiwali Mohan, Matt Hong, Dule Shu +3Pairwise Preference LearningHuman Preference Modeling

  13. Rethinking the Implications of Human Feedback for Preference Learning in Human-Robot Collaboration

    Sep 12, 2026Qiping Zhang, Kate Candon, Debasmita Ghose +1Human Preference ModelingPreference Learning

  14. SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

    Sep 3, 2026Marco Cipriano, Leonardo Zini, Alexandra Schild +5VLM EvaluationHuman Preference Modeling

  15. Large Language Model Few-Shot Prompting with Dilemma Training Outperforms Human Surrogates in Predicting Patient Preferences

    Aug 26, 2026Natasha Ureyang, Sebastian Porsdam Mann, Yuxin Liu +9Human Preference ModelingHuman-AI Decision Making

  16. VA-Judger: Reward Modeling from Human Preference Feedback for Joint Video-Audio Generation

    Aug 19, 2026Yinming Huang, Shuyuan Tu, Xi Yan +6Reward ModelingAudio-Video Generation

  17. FMReward: Aligning and Evaluating Audio-Driven 3D Facial Animation with Human Preferences

    Aug 15, 2026Sijing Wu, Yunhao Li, Zhilin Gao +4Audio-Driven Facial AnimationReward Modeling

  18. Multi-dimensional Bias in Modeling Multi-dimensional Preferences: Evaluating the Ability of Synthetic Agents to Replace Human Participants in Conjoint Experiments

    Aug 13, 2026Ho Ting Hung, Nachiket Midha, Victor Y. Wu +1Synthetic Data EvaluationHuman Behavior Simulation

  19. PALMs: Using Multi Construct-Grounded Rationales for Modeling Population Preferences in LLMs

    Aug 2, 2026Priyanka Dey, Brihi Joshi, Preyashi Poddar +2LLM AlignmentHuman Preference Modeling

  20. Preference Elicitation for Policy Optimization and Application to Aligning Heart Transplantation with Human Values

    Jul 29, 2026Itai Zilberstein, Ioannis Anagnostides, Zachary W Sollie +2Preference ElicitationHuman Preference Modeling

  21. What do Reward Models Memorize?

    Jul 27, 2026Ivo Verhoeven, Pushkar Mishra, Ekaterina ShutovaReward ModelingPairwise Preference Evaluation

  22. StARS: Socially Appropriate Robot Actions via a Recommender System-Driven Approach

    Jul 23, 2026Erencem Ozbey, Fethiye Irmak Dogan, Jin Huang +1Human Preference ModelingHuman-Robot Interaction