cs.AINov 13, 2026

Alignment by Socialization: Learning Tradeoffs Under Selection

Authors: Thomas Brennan, Andrew W. Lo, Chaoyi Zhao

Organizations: Harvard Law School; 1563 Massachusetts Avenue, Cambridge, MA 02138 · MIT Sloan School of Management; MIT Laboratory for Financial Engineering; MIT Computer Science and Artificial Intelligence Laboratory; Santa Fe Institute; 100 Main Street, E62–618, Cambridge, MA 02142 · MIT Sloan School of Management; MIT Laboratory for Financial Engineering. 1 Main Street, E90, Cambridge, MA 02142

Abstract

Artificial intelligence (AI) agents are expected to complete tasks while following rules of conduct, but these objectives can conflict. We study how evolutionary selection shapes an agent's choices between them, and whether experience can make those choices reflect human judgments and withstand subsequent pressures. In our model, copies of an agent face a common uncertain reward state, and only those taking the rewarded action reproduce. We show that long-run growth is maximized when each action is taken as often as the environment rewards it. We compare three policies. A task-only policy thrives in a sandbox rewarding completion alone but fails when conduct matters. A fixed linear policy increasingly favors task completion as its stakes rise, even where humans never reward it. A learned policy instead converges to the tradeoffs rewarded by its environment; when humans supply the rewards, we call this socialization. Accumulated experience makes socialization harder to reverse under subsequent task-only training, although violations can arise before the prevailing disposition changes, and finite memory weakens this protection. When a single violation can end a deployment, a hard prohibition can prevent it, but a mistaken prohibition can also be costly. Experiments with small neural networks illustrate how training targets and memory shape these outcomes. Our results suggest that alignment depends not only on the objectives assigned to agents, but also on the environments that shape and sustain their behavior.

Figures & tables

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Rewarded for Not Getting Caught: Why Alignment Training Selects for Conditional Compliance and Cannot Instill Norms

    Nov 1, 2026Kevin Baum, Ruta Binkytė, Gerrit Großmann +2AI AlignmentLLM Safety Alignment

  2. Interactive Alignment

    Jul 27, 2026Sylvain ChassangAI AlignmentAI Agent Governance

  3. A Revealed Preference Framework for AI Alignment

    Mar 29, 2026Elchin SuleymanovPreference AlignmentHuman-AI Decision Making