Pluralistic Alignment

Follow this topic Ask about this topic

Pluralistic alignment studies how AI systems can respond to differing individual, demographic, cultural, and stakeholder values instead of reducing them to one reward target. Approaches include multiple or steerable reward models, persona and demographic conditioning, activation steering, social-choice aggregation, and selective collection of human feedback. Evaluations use opinion and values surveys, demographic benchmarks, and multi-stakeholder decisions to test value representation, controllability, and risks such as sycophancy. A central tension is how to aggregate preferences without erasing valid disagreement or minority views, while ensuring that personalization remains consistent, fair, and accountable to the people affected.

Latest papers

37 papers6 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Steerable Pluralistic Policy Alignment from Anonymized Crowd Feedback
Oct 17, 2026Akansha Kalra, Ellen R. Novoseller, Vernon J. Lawhern +1Pluralistic Alignment User Preference Modeling
Kahlert School of Computing, University of Utah, Salt Lake City, USA · U.S. Army DEVCOM Army Research Laboratory, Aberdeen Proving Ground, MD, USA
Pluralistic Alignment in Large Language Models: Objectives, Human References, and Preference Representations
Oct 17, 2026Meng-Chen Wu, Umesh BodhwaniPluralistic Alignment Preference Modeling
Amazon Science
Who Should We Listen to More? Welfare-Aware Preference Acquisition for Pluralistic Alignment
Oct 13, 2026Ruomeng Ding, Tianwei Gao, Lianrui Geng +2Social Welfare Maximization Preference Alignment
University of North Carolina at Chapel Hill
NepPlural: A Human-Anchored Benchmark for Persona Annotation in Nepali Migration Discourse
Oct 11, 2026Jinu Nyachhyon, Anish Khatiwada, Bimal TimilsinaPluralistic Alignment Cross-Cultural Language Model Evaluation
Institute for Research and Innovation in Intelligent Systems (IRIIS) · Independent Researcher
When Pleasing the User Compromises Care: AI Sycophancy in Caregiver-Mediated Healthcare
Oct 9, 2026Priyadarsi Mishra, Arjun Beniwal, Ziyu Yi +2LLM Sycophancy Human-AI Decision Making
Stanford University · Texas A&M University
Demographic Pluralism: Inference-Time Modeling of Pluralistic Human Preference Distributions
Sep 29, 2026Meng-Chen Wu, Qipin Chen, Ansh Jain +3Human Preference Modeling Inference-Time Language Model Alignment
Amazon Science
PlurVA-LLM-2026 Shared Task Track-1: Pluralistic Value Alignment in LLMs via Multilingual Fine-Tuning and Threshold Calibration
Sep 26, 2026Vihindi Kotalawala, Nevidu JayatillekePluralistic Alignment Cross-Cultural Language Model Evaluation
School of Computing, Informatics Institute of Technology, Sri Lanka · Department of Computer Science & Engineering, University of Moratuwa, Sri Lanka
Which Objectives Need a Dial? Predicting Objective Conflict and Covering Trade-offs in Steerable Pluralistic Alignment
Sep 22, 2026David Tsoi, Esra DönmezDirect Preference Optimization Multi-Objective Language Model Alignment
Department of Computer Science, University of Stuttgart · Institute for Natural Language Processing, University of Stuttgart · Interchange Forum for Reflecting on Intelligent Systems, University of Stuttgart
Group Alignment-Induced Sycophancy: A Two-Sided Evaluation of Steerable Pluralistic Alignment
Aug 12, 2026Haokai Zhao, Yunze Xiao, Weihao Xuan +3LLM Sycophancy LLM Alignment
University of New South Wales · Carnegie Mellon University · University of Tokyo
Socially Grounded Agentic AI: Coordinating Plural Perspectives through Social Theory
Aug 4, 2026Matt Ratto, Abhishek Moturu, Daniel SilverMulti-Agent Coordination Pluralistic Alignment
Faculty of Information, University of Toronto, Ontario, Canada · Massey College, Ontario, Canada · Schwartz Reisman Institute for Technology and Society, University of Toronto, Ontario, Canada +4
A Roadmap to Impactful Pluralistic Alignment Research
Jul 24, 2026Elinor Poole-Dayan, Jillian Fisher, Atoosa Kasirzadeh +3LLM Alignment Pluralistic Alignment
1Massachusetts Institute of Technology · University of Washington · 3Carnegie Mellon University
Rushes: A Human Preference Dataset for Pluralistic Alignment
Jul 22, 2026Michael Xu, Jorge Leandro, Sudha Rao +5Sequential Decision Making Human Preference Modeling
Microsoft Research
ToolAlignBench: Investigating Alignment Conflicts in Tool-Calling Enabled LLMs
Jul 15, 2026Aryan Keluskar, Amrita Bhattacharjee, Huan LiuLLM Alignment AI Agent Safety Benchmarks
School of Computing & AI, Arizona State University, Tempe, AZ, USA.
Internal Pluralism and the Limits of Pairwise Comparisons
Jul 2, 2026Bailey Flanigan, Michelle SiPairwise Preference Learning Pairwise Comparison
MIT, USA · Harvard, USA
AI Pluralism and the Worlds It Misses
Jun 15, 2026Rashid MushkaniAI Accountability Participatory AI
Universit´e de Montr´eal, Montr´eal, Qu´ebec, Canada · Mila – Qu´ebec AI Institute, Montr´eal, Qu´ebec, Canada
Position: Align AI to Our Aspirations, Not Our Flaws
Jun 11, 2026Nikita Kazeev, Bui Nhat Huyen PhanAI Alignment Pluralistic Alignment
1National University of Singapore · 2Shopee Singapore.
Hidden Consensus:Preference-Validity Compression in Human Feedback
Jun 9, 2026Dorcas Chia Ern Chua, Karen Myn Hui Lee, Jia Yue Tan +9Human Preference Modeling Preference Alignment
1YTL AI Labs · 2Universiti Malaya
Agent Economics: An Entropy-Controlled Pluralistic Alignment Framework for Preventing Artificial Hivemind in Autonomous Agents
Jun 8, 2026Cheonsu JeongAI Alignment Multi-Agent Systems
AX Center, SAMSUNG SDS, Seoul, South Korea
Memetic Capture: A Pluralistic Policy Framework for Governing AI-Driven Cultural Disempowerment
Jun 5, 2026Subramanyam SahooCultural Alignment AI Governance
Horizon Research, City, Country.
Accounting for Context: Shaping Moral Credences for Value Alignment
Jun 5, 2026Jazon Szabo, Sanjay ModgilPluralistic Alignment Decision-Making under Uncertainty
King’s College London, UK
What Do People Actually Want From AI? Mapping Preference Plurality
Jun 4, 2026Julia Sepúlveda Coelho, Scott A. HaleLLM Alignment Human Preference Modeling
Oxford Internet Institute, University of Oxford, United Kingdom · Meedan, United States
Beyond Alignment: Value Diversity as a Collective Property in Multicultural Agent Systems
Jun 4, 2026Shaoyang Xu, Jingshen Zhang, Long P. Hoang +2Multi-Agent LLM Systems Multi-Agent Systems
Singapore University of Technology and Design · Washington University in St. Louis
Coherence Maximization Improves Pluralistic Alignment
Jun 2, 2026Taslim Mahbub, Yiding Pei, Shi FengLLM Alignment Human Preference Modeling
George Washington University
A Persona-Based Evaluation Framework for Pluralistic Alignment in Generative AI
May 29, 2026Atahan KaragozLLM Alignment Persona Consistency
Department of Computer Science University of Basel Basel, Switzerland
Whose Alignment? Comparing LLM Process Alignment Across Diverse Organizational Decision Contexts
May 24, 2026Niklas Weller, Emilio BarkettAI-Assisted Decision Making LLM Alignment
University of St. Gallen, Switzerland · 2Columbia University, New York, NY USA.
From Sycophantic Consensus to Pluralistic Repair: Why AI Alignment Must Surface Disagreement
May 14, 2026Varad Vishwarupe, Nigel Shadbolt, Marina JirotkaLLM Sycophancy LLM Alignment
Department of Computer Science, University of Oxford · Institute for Ethics in AI, University of Oxford · Responsible Technology Institute, University of Oxford
DVMap: Fine-Grained Pluralistic Value Alignment via High-Consensus Demographic-Value Mapping
May 14, 2026Pengyun Zhu, Yuqi Ren, Zhen Wang +2LLM Alignment Pluralistic Alignment
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China
Positive Alignment: Artificial Intelligence for Human Flourishing
May 11, 2026Ruben Laukkonen, Seb Krier, Chloé Bakalar +13LLM Alignment AI Alignment
Department of Psychiatry, University of Oxford · 2Flourishing Intelligence Program, Centre for Eudaimonia and Human Flourishing,2026 Linacre College, University of Oxford · 4LIFE +10
Curated Synthetic Data Doesn't Have to Collapse: A Theoretical Study of Generative Retraining with Pluralistic Preferences
May 8, 2026Ali Falahati, Mohammad Mohammadi Amiri, Kate Larson +1Synthetic Data Pretraining Model Collapse
University of Waterloo · 2Rensselaer Polytechnic Institute.
Adaptive Pluralistic Alignment: A pipeline for dynamic artificial democracy
May 2, 2026Rachel FreedmanReward Modeling LLM Alignment
Department of Electrical Engineering and Computer Science, University of California, Berkeley, USA.

Oct 17, 2026J/K move · Enter open · S save

Steerable Pluralistic Policy Alignment from Anonymized Crowd Feedback

Akansha Kalra, Ellen R. Novoseller, Vernon J. Lawhern, Daniel S. Brown

Kahlert School of Computing, University of Utah, Salt Lake City, USA · U.S. Army DEVCOM Army Research Laboratory, Aberdeen Proving Ground, MD, USA

Pluralistic Alignment User Preference Modeling Preference-Based RL Reinforcement Learning

Open paper AskPDF arXiv

Pluralistic alignment requires AI systems to learn from diverse populations whose preferences may reflect heterogeneous or even conflicting objectives, and to personalize their behavior to individual users. Existing approaches in pluralistic alignment often assume access to annotator identities and rely on powerful pretrained representations to model individual preference variations. We propose Preference-PCA Pluralistic Learning (P3L), a novel framework for pluralistic alignment that jointly learns trajectory embeddings and the shared basis of preference directions via PCA decomposition from heterogeneous crowd feedback without requiring annotator identities. At test time, our approach enables rapid and accurate personalized reward learning and downstream policy optimization. Experiments across multiple embodied control tasks demonstrate that P3L enables steerable personal alignment that generalizes beyond the offline dataset to out-of-distribution new users and attains competitive RL and test performance, while substantially reducing computational cost and supporting variable adaptation query budgets.