Pluralistic Alignment

Follow this topic Ask about this topic

Pluralistic alignment studies how AI systems can respond to differing individual, demographic, cultural, and stakeholder values instead of reducing them to one reward target. Approaches include multiple or steerable reward models, persona and demographic conditioning, activation steering, social-choice aggregation, and selective collection of human feedback. Evaluations use opinion and values surveys, demographic benchmarks, and multi-stakeholder decisions to test value representation, controllability, and risks such as sycophancy. A central tension is how to aggregate preferences without erasing valid disagreement or minority views, while ensuring that personalization remains consistent, fair, and accountable to the people affected.

Latest papers

37 papers6 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Relative Principals, Pluralistic Alignment, and the Structural Value Alignment Problem
Apr 22, 2026Travis LaCroixAI Alignment AI Governance
Durham University, United Kingdom
The Triadic Loop: A Framework for Negotiating Alignment in AI Co-hosted Livestreaming
Apr 20, 2026Katherine Wang, Nadia Berthouze, Aneesha SinghAI Alignment Pluralistic Alignment
University College London London, United Kingdom
Jailbreaking Large Language Models with Morality Attacks
Apr 18, 2026Ying Su, Mingen Zheng, Weili Diao +1Moral Reasoning in Language Models Language Model Safety Evaluation
South China University of Technology1 · HKUST2 · Beihang University3
APPA: Adaptive Preference Pluralistic Alignment for Fair Federated RLHF of LLMs
Apr 5, 2026Mahmoud Srewa, Tianyu Zhao, Salma ElmalakiLLM Alignment Pluralistic Alignment
Department of Electrical Engineering and Computer Science, University of California, Irvine, Irvine, CA 92697, USA
Behavioural feasible set: Value alignment constraints on AI decision support
Mar 22, 2026Taejin ParkAI Governance AI-Assisted Decision Making
Bank for International Settlements
Slurry-as-a-Service: A Modest Proposal on Scalable Pluralistic Alignment for Nutrient Optimization
Mar 2, 2026Rachel Hong, Yael Eiger, Jevan Hutson +2LLM Alignment Pluralistic Alignment
ValueMulch United States
VISPA: Pluralistic Alignment via Automatic Value Selection and Activation
Jan 19, 2026Shenyan Zheng, Jiayou Zhong, Anudeex Shetty +3LLM Alignment Multi-Objective Language Model Alignment
University of Waterloo · University of Melbourne · Macquarie University +2

Apr 22, 2026J/K move · Enter open · S save

Relative Principals, Pluralistic Alignment, and the Structural Value Alignment Problem

Travis LaCroix

Durham University, United Kingdom

AI Alignment AI Governance Pluralistic Alignment Value Alignment

Open paper AskPDF arXiv

The value alignment problem for artificial intelligence (AI) is often framed as a purely technical or normative challenge, sometimes focused on hypothetical future systems. I argue that the problem is better understood as a structural question about governance: not whether an AI system is aligned in the abstract, but whether it is aligned enough, for whom, and at what cost. Drawing on the principal-agent framework from economics, this paper reconceptualises misalignment as arising along three interacting axes: objectives, information, and principals. The three-axis framework provides a systematic way of diagnosing why misalignment arises in real-world systems and clarifies that alignment cannot be treated as a single technical property of models but an outcome shaped by how objectives are specified, how information is distributed, and whose interests count in practice. The core contribution of this paper is to show that the three-axis decomposition implies that alignment is fundamentally a problem of governance rather than engineering alone. From this perspective, alignment is inherently pluralistic and context-dependent, and resolving misalignment involves trade-offs among competing values. Because misalignment can occur along each axis -- and affect stakeholders differently -- the structural description shows that alignment cannot be "solved" through technical design alone, but must be managed through ongoing institutional processes that determine how objectives are set, how systems are evaluated, and how affected communities can contest or reshape those decisions.