Overview

Preference alignment aims to make AI outputs reflect what people value, from helpful and safe language-model responses to personalized recommendations, generated images, and robot behavior. Common approaches learn reward models from pairwise judgments and optimize with reinforcement learning from human feedback (RLHF), or use direct preference optimization (DPO); other methods steer models at inference time or adapt them to individual users and multiple objectives. Evaluations use human judgments and task-specific benchmarks, including language-model preference tests and image-generation comparisons. A central challenge is that preferences can be noisy, conflicting, culturally varied, or change over time, while optimizing imperfect reward signals can introduce bias, reduce capabilities, or reward behavior people do not actually want.

Latest papers

CardsList

112 papers7 papers in the last four weeks, down 22% on the four weeks before. 0.0% of all new papers.

  1. CIDER: A Dataset of Contextual Disclosure Boundaries for Privacy Preference Alignment

    Aug 10, 2026Bingcan Guo, Eryue Xu, Jijie Zhou +2Preference AlignmentInference-Time Language Model Alignment

  2. Compass: Continuously Aligning Social Media Feeds via In-Situ Reflections

    Aug 4, 2026Aadit Barua, Leijie Wang, Amy X. ZhangSocial Media AnalysisRecommender Systems

  3. PALMs: Using Multi Construct-Grounded Rationales for Modeling Population Preferences in LLMs

    Aug 2, 2026Priyanka Dey, Brihi Joshi, Preyashi Poddar +2LLM AlignmentHuman Preference Modeling

  4. BridgeAlign: Bridging Preference Alignment for Humanities and Social Sciences

    Jul 29, 2026Ru Peng, Haokai Xu, Xijun Gu +11Pairwise Preference LearningPreference Optimization

  5. Human Preference aligned Tabular Similarity

    Jul 27, 2026Frederik Hoppe, Astrid Franz, Marianne Michaelis +2Human Preference EvaluationTabular Representation Learning

  6. Rewarding Better Thinking for LLM Preference Alignment

    Jul 22, 2026Xubo Liu, Wenya Guo, Ruxue Yan +2Process Reward ModelsCredit Assignment in RL

  7. Align AI to Dynamic Human-AI Workflows

    Jul 15, 2026Valerie Chen, Cleotilde Gonzalez, Anita Williams Woolley +4AI AlignmentHuman-AI Interaction

  8. Deployable Human Preference Alignment in Robotics: Learning Representative Rewards from Diverse Human Preferences

    Jul 14, 2026Taehyung Kim, Gwangmo Lee, Minjun Chang +2ClusteringPreference Alignment

  9. Normative Alignment of Recommender Systems via Internal Label Shift

    Jul 12, 2026Johannes Kruse, Kasper Lindskow, Michael Riis Andersen +4Distribution MatchingRecommender Systems

  10. Multi-Objective Exploration and Preference Optimization via Mutual Information

    Jul 1, 2026Hongyan Xie, Yikun Ban, Ruiyu Fang +4Multi-Objective Language Model AlignmentMutual Information Maximization

  11. LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training

    Jun 29, 2026Shun Lei, Huaicheng Zhang, Dapeng Wu +8Text-to-Music GenerationControllable Music Generation

  12. REAR: Test-time Preference Realignment through Reward Decomposition

    Jun 29, 2026Fuxiang Zhang, Pengcheng Wang, Chenran Li +6Test-Time ScalingReward Decomposition

  13. Open Problems in Constitutional Preference Reconstruction

    Jun 29, 2026Eleanor Clifford, Michael Amir, Arduin Findeis +2LLM-as-a-JudgeLLM Decision-Making

  14. A Markov Chain Approach to Preference Alignment

    Jun 21, 2026Takuya Koriyama, Tengyuan LiangPairwise Preference LearningPairwise Comparison

  15. TuneJury: An Open Metric for Improving Music Generation Preference Alignment

    Jun 15, 2026Yonghyun Kim, Junwon Lee, Haiwen Xia +5Text-to-Music GenerationMusic Generation Evaluation

  16. Hidden Consensus:Preference-Validity Compression in Human Feedback

    Jun 9, 2026Dorcas Chia Ern Chua, Karen Myn Hui Lee, Jia Yue Tan +9Human Preference ModelingPreference Alignment

  17. DOG-DPO:Dynamic Optimization in Geometry for Safety Alignment

    Jun 4, 2026Yi Nian, Tiankai Yang, Yudi Zhang +7Direct Preference OptimizationLLM Safety Alignment

  18. Follow-Your-Preference++: Rethinking Preference Alignment for Image Inpainting

    Jun 2, 2026Junkun Yuan, Yutao Shen, Toru Aonishi +2Pairwise Preference LearningReward Modeling

  19. Fully Automated Identification of Lexical Alignment and Preference-Stage Shifts in Large Language Models

    Jun 2, 2026Thomas Stephan Juzek, Xiaoyang Ming, Jose A. HernandezLLM EvaluationLLM Alignment

  20. MENTIS: What Belief Changes Under Alignment? Measuring Multi-Scale Latent Torsion in Language Models

    May 31, 2026Partha Pratim Saha, Samarth Raina, Mayur Parvatikar +4LLM InterpretabilityPreference Alignment

  21. Position: Good Embodied Reward Models Need Bad Behavior Data

    May 31, 2026Ran Tian, Yilin Wu, Andrea BajcsyReward ModelingPreference Alignment

  22. Federated Variational Preference Alignment with Gumbel-Softmax Prior for Personalized User Preferences

    May 29, 2026Jabin Koo, Hoyoung Kim, Minwoo Jang +1LLM AlignmentPreference Alignment

  23. Differentially Private Preference Data Synthesis for Large Language Model Alignment

    May 29, 2026Fengyu Gao, Jing YangPairwise Preference LearningLLM Alignment

  24. Toward User Preference Alignment in LLM Recommendation via Explicit Context Feedback

    May 27, 2026Weizhi Zhang, Wooseong Yang, Yuxin Cui +9Recommender SystemsLarge Language Model-Based Recommendation

  25. KARMA: Karma-Aligned Reward Model Adaptation

    May 26, 2026Jared Scott, Jesse RobertsReward ModelingRL for Language Models