cs.GTMay 12, 2026

The End Justifies the Mean: A Linear Ranking Rule for Proportional Sequential Decisions

Authors: Carmel BaharavNiclas BoehmerBailey FlaniganMaximilian T. Wittmann

Organizations: MIT, USA · Hasso Plattner Institute, University of Potsdam, Germany

Abstract

AI alignment and participatory design motivate a new democratic design problem: how to collectively choose a decision rule to use repeatedly. We study this problem for linear ranking rules, which repeatedly rank items xjx_j within batches X=(x1,,xm)(Rd)mX=(x_1,\dots,x_m)\in(\mathbb{R}^d)^m, where each item's ranking is dictated by its score θ,xj\langle θ^*,x_j\rangle according to a fixed scoring vector θθ^*. Given voters' preferred scoring vectors θ(1),,θ(n)θ^{(1)},\dots,θ^{(n)} and their population fractions α(1),,α(n)α^{(1)},\dots,α^{(n)}, we ask how to choose a collective vector θθ^* satisfying individual proportionality (IP): every voter type ii should agree with the resulting rankings to an α(i)α^{(i)}-proportional degree, either on average over time (long-run IP) or even within each batch (per-batch IP). The default rule, the arithmetic mean of the θ(i)θ^{(i)}, has been shown to be severely majoritarian; more generally, it is not clear that any fixed linear rule can balance many voters' disparate opinions. Our main result is that, surprisingly, there is a simple rule that does satisfy long-run IP: the angular mean, the spherical analog of the arithmetic mean. We then show that exact per-batch IP is impossible for fixed linear rules, but that the gap between per-batch and long-run IP shrinks quickly with batch size. Experiments on three real-world preference datasets show that all rules perform similarly when voters' preferences are homogeneous, while the angular mean substantially improves proportionality in high-disagreement regimes.

Explore similar work

May 2, 2026cs.LG

Adaptive Pluralistic Alignment: A pipeline for dynamic artificial democracy

Prevailing alignment methods target a fixed set of preferences and therefore risk forcing value lock-in as societal norms evolve over time. We introduce Adaptive Pluralistic Alignment (APA), a modular pipeline for updating pluralistically aligned AI systems to track evolving values and avoid value lock-in without repeating costly pretraining or large-scale data collection. APA has three stages: (1) learning compact personalized reward models via low-rank reward basis decomposition, (2) using these models as a jury that collectively selects among candidate outputs through social-choice-theoretic voting, and (3) efficiently adapting the jury over time by fitting new annotator weights over the fixed reward bases as values shift. The resulting system is efficient, explainable, steerable, and modular. We implement a proof-of-concept instantiation using the PRISM multi-user alignment dataset and simulated historical annotators, and provide preliminary analysis showing that jury composition and the choice of voting rule can substantially affect outcomes, particularly when jury preferences are heterogeneous. We provide full code and resulting preference datasets at https://github.com/RachelFreedman/apa.
Rachel Freedman
Jun 26, 2026cs.LG

Democratic ICAI: Debating Our Way to Steering Principles from Preferences

Preference-based alignment often struggles to capture the reasoning that underlies human judgments. Many evaluations rely on multiple interacting criteria, yet pairwise labels reveal only the final choice rather than the considerations that shape preferences. Inverse Constitutional AI (ICAI) improves interpretability in decision making by summarizing preferences into natural-language principles, but its single-pass explanations miss much of the nuance involved in complex decisions. We introduce Democratic ICAI (DICAI), a novel approach that gathers multiple competing rationales through structured persona debate, offering a broader and more expressive account of the factors influencing each comparison. From these richer signals, we derive clearer and more comprehensive steering principles and use them to guide preference decision modeling through both LLM-based and decision-tree judges, as well as downstream model training via constitution-induced preference labels. Experiments on creative preference benchmarks, MuCE-Pref and LiTBench, across multiple creative task categories show that Democratic ICAI yields a more faithful preference structure. It improves average preference prediction across tasks relative to deliberative prompting and principle-based baselines, while producing constitutions that LLM annotators prefer.
Kevin Kingslin, Anish Natekar, Ashutosh Ranjan +3
Aug 25, 2026cs.LG

MoPLEx: Estimating Plackett-Luce Mixture Models for Multi-Objective Alignment

We study learning a mixture of kk Plackett-Luce models from multi-way ranking responses from annotators that may represent heterogeneous underlying preferences. This problem has many applications in AI alignment and preference optimization. Prior work has studied mixtures of Bradley-Terry models from pairwise comparisons. However, estimating a mixture of multi-way ranking models can become theoretically unidentifiable when kk exceeds m/2m/2, where mm is the ranking length. We design an efficient algorithm to address this issue by first augmenting the rankings to a larger size (e.g., generating comparisons from a base model), followed by a gradient-based estimation to reduce inference cost (in the input embedding space). With this procedure in mind, we then fit a mixture of Plackett-Luce (PL) models via an expectation-maximization-style iteration, or MoPLEx in short. We conduct extensive experiments to verify this algorithm. First, we find that the gradient-based approximation estimates true probabilities with less than 5% error on models with up to 34 billion parameters. Second, MoPLEx improves clustering and ranking accuracy by an average of 43.7% and 15.2% over baselines using a single PL model or a mixture of Bradley-Terry models, on UltraFeedback and PERSONA datasets. These results demonstrate the effectiveness of MoPLEx for tackling multi-way rankings following heterogeneous preferences through measuring alignment via gradients.
Dongyue Li, Ziniu Zhang, Lu Wang +1