cs.AIAug 25, 2026

Algorithmic Impact Reveals the Hidden Social Choice Structure of Alignment

Authors: Zachary Wojtowicz, Michelle Si, Finale Doshi-Velez, Ariel Procaccia

Organizations: MIT Cambridge, MA 02139 · Harvard University Cambridge, MA 02138

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

When an AI algorithm makes decisions that affect more than one person, aligning it becomes a problem of social choice: how should people's divergent preferences about system behavior be reconciled and aggregated into a single coherent model? The standard approach to aligning frontier AI models\unicodex2013\unicode{x2013}reinforcement learning from human feedback\unicodex2013\unicode{x2013}largely sidesteps this question and has poor social choice guarantees. However, it remains unclear what alternative should replace it. We show that, by focusing directly on an algorithm's welfare consequences, the alignment problem can be reformulated as linear optimization over a convex impact space, which makes it amenable to the standard toolkit of welfare economics and mechanism design. This reformulation clarifies how alignment protocols translate into welfare consequences and, conversely, how a social planner's desired constraints on welfare consequences can be translated back into alignment protocols. We apply this transformation to show that voting-by-issues and random-dictatorship mechanisms are strategyproof and unanimous. Demonstrating the reverse direction, we also apply the impact representation to derive a family of alignment protocols that maximize utilitarian social welfare subject to various social desiderata, such as bounds on individual or group harm. We illustrate the welfare implications of these alignment protocols empirically using real human preferences over kidney allocation, charitable food distribution, LLM responses, and trolley problems.

Figures & tables

Figure 1: The impact-space view of social alignment. Preferences ( θn\theta_{n} , dashed) and the feasible impact set Ψˉ\bar{\Psi} live in the same Euclidean space; for finitely many deployment queries, Ψˉ\bar{\Psi} is a convex, origin-symmetric polytope. Preferences differ in magnitude and need not themselves be feasible impacts: both θA\theta_{A} and θB\theta_{B} lie outside Ψˉ\bar{\Psi} . Because welfare Un(ψ)=θn⊤ψU_{n}(\psi)=\theta_{n}^{\top}\psi is linear, each agent’s ideal impact ψn⋆=arg⁡max⁡ψ∈Ψˉθn⊤ψ\psi_{n}^{\star}=\arg\max_{\psi\in\bar{\Psi}}\theta_{n}^{\top}\psi is the vertex of Ψˉ\bar{\Psi} farthest in the direction of θn\theta_{n} , which in general is not collinear with θn\theta_{n} . The utilitarian optimum ψ⋆\psi^{\star} maximizes welfare in the weighted-average direction θˉw=∑nwnθn\bar{\theta}_{w}=\sum_{n}w_{n}\theta_{n} , where the iso-welfare line {ψ:θˉw⊤ψ=const}\{\psi:\bar{\theta}_{w}^{\top}\psi=\text{const}\} is tangent to Ψˉ\bar{\Psi} . The dashed segment conv⁡(ψA⋆,ψB⋆)\operatorname{conv}(\psi_{A}^{\star},\psi_{B}^{\star}) is the set of expected impacts attainable by strategyproof mechanisms (Section 5 ), with ψsp\psi^{\mathrm{sp}} the equal-weight random dictatorship.
Figure 2: Community Alignment dataset: distribution of individual welfare Un/U⋆U_{n}/U^{\star} across the 2387 annotators under each mechanism, at high choice precision ( β=100\beta=100 ). Top: utilitarian, RLHF, and strategyproof. Bottom: harm-bounded alignment at three welfare floors bb ; the faint gray curve repeats the utilitarian distribution for reference, and the dotted red line marks the floor. Tightening the floor (toward 00 ) binds more agents and compresses the welfare distribution, removing the harmed left tail.
Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Community Alignment dataset: per-agent welfare Un/U⋆U_{n}/U^{\star} as a function of the choice-precision parameter β\beta , for each mechanism. The shaded band is the density of all 2387 agents; solid, dashed, and dotted lines mark the best-off agent, mean welfare, and worst-off agent. As β\beta grows, choices become more deterministic, individual preferences become more powerful, and the welfare distribution spreads—widening the gap between best- and worst-off agents. Strategyproof and harm-bounded mechanisms produce lower welfare dispersion, and the bounded-harm constraints are empirically satisfied.
Figure 4: Participation externalities under the per-agent harm bound. Here each agent’s welfare floor is a fixed fraction of their own maximum achievable welfare, bn=−ϵ hΨˉ(θn)b_{n}=-\epsilon\,h_{\bar{\Psi}}(\theta_{n}) with ϵ=0.05\epsilon=0.05 (lose at most 5% of one’s own best case), rather than a single absolute floor. The left panel is the utilitarian baseline; the right is the harm-bounded model. The qualitative story matches that of the single absolute floor discussed in Section 6 : a small set of binding agents accounts for almost all of the participation externalities.
Figure 5: Community Alignment Dataset: Utilitarian vs RLHF participation alignment externalities by demographic. The figure includes 2387 annotators, with full alignment externalities averaged within each group.
Figure 6: Community Alignment Dataset: 100 person sample with granular person-on-person full externalities compared to the group-averaged full externalities. Our participation externality formalization gives us a microscope to analyze where the per-individual welfare effects are coming from.
Figure 7: Preference distribution per feature dimension for four datasets. Moral Machine and Community Alignment have high disagreement, with a high number of preferences on either side of 0, while Kidney Allocation and 412 Food Rescue have less disagreement: people disagree on the magnitude of preference, but the direction is more unanimous.
Figure 8: Expected welfare for four datasets. Bootstrap mean and std. deviations were calculated by bootstrapping ll agents 100 times and calculating the welfare of the strategyproof mechanism. Sequential addition denotes the procedure where 100 permutations of agents were generated, and welfare was calculated for each coalition created by adding a new agent.
Figure 9: Welfare variance over four datasets. The strategyproof mechanism leads to the least welfare dispersion, while RLHF follows as a far second for larger values of β\beta .
Figure 10: Welfare of the best- and worst-off agents in four datasets for three model parameters.
More in this topic

Social Welfare Maximization

See all 20 papers

Explore similar work

CardsList

  1. A Revealed Preference Framework for AI Alignment

    Mar 29, 2026Elchin SuleymanovPreference AlignmentHuman-AI Decision Making

    Department of Economics, Mitch Daniels School of Business, Purdue University, West Lafayette, Indiana, USA.

  2. Who Should We Listen to More? Welfare-Aware Preference Acquisition for Pluralistic Alignment

    Oct 13, 2026Ruomeng Ding, Tianwei Gao, Lianrui Geng +2Social Welfare MaximizationPreference Alignment

    University of North Carolina at Chapel Hill

  3. AI Alignment From Social Choice Perspectives

    Jun 19, 2026Daniel Halpern, Evi Micha, Ariel D. Procaccia +3LLM AlignmentAI Alignment

    Google Research · University of Southern California · Harvard University