AI Alignment

Ask about this topic

Overview

AI alignment asks whose values a system should follow and how to make its behavior reliably reflect them, especially when instructions are incomplete, people disagree, or systems act autonomously. Common approaches include learning preferences through human feedback, inverse reinforcement learning, constitutional and reward-based training, social-choice and mechanism-design methods, and runtime controls that restrict or verify actions. Evaluations range from preference and safety benchmarks to interactive agent tasks testing deception, oversight avoidance, goal drift, and behavior under pressure. A central tension remains between using imperfect, often aggregated human judgments as training targets and building systems that remain corrigible and accountable across changing contexts and real deployments.

Latest papers

CardsList

126 papers12 papers in the last four weeks, up 20% on the four weeks before. 0.1% of all new papers.

  1. The Possibility of Artificial Intelligence Becoming a Subject and the Alignment Problem

    Apr 16, 2026Till Mossakowski, Helena Esther GrassAI AlignmentAI Agent Governance

    Institute of computer science, Osnabrück University · Institute of philosophy, Oldenburg University

  2. OOM-RL: Out-of-Money Reinforcement Learning Market-Driven Alignment for LLM-Based Multi-Agent Systems

    Apr 13, 2026Kun Liu, Liqun ChenAI AlignmentRL for LLM Agents

  3. Principles Do Not Apply Themselves: A Hermeneutic Perspective on AI Alignment

    Apr 12, 2026Behrooz RazeghiAI AlignmentLLM Alignment

    School of Engineering and Applied Sciences, Harvard University

  4. AI Organizations are More Effective but Less Aligned than Individual Agents

    Apr 11, 2026Judy Hanwen Shen, Daniel Zhu, Siddarth Srinivasan +5Multi-Agent SystemsAI Alignment

    Constellation Institute · MATS Program · Anthropic

  5. SysAdmin: Measuring Instrumental Power-Seeking in Frontier AI

    Apr 10, 2026Mana Azarm, Qiyao Wei, Rahul NambiarLanguage Model Safety EvaluationAI Alignment

    Analytics & Information Systems University of San Francisco · Department of Applied Mathematics and Theoretical Physics University of Cambridge · Propensity Labs

  6. Identifying and Understanding Human Values in Text: A Tailorable LLM-based Architecture

    Apr 7, 2026Eduardo de la Cruz Fernández, Marcelo Karanik, Sascha OssowskiAI Alignment

    Universidad Politécnica de Madrid, Madrid, Spain · CETINIA, Universidad Rey Juan Carlos, Madrid, Spain

  7. Simulating the Evolution of Alignment and Values in Machine Intelligence

    Apr 7, 2026Jonathan Elsworth EicherAI Alignment

    Antimemetic AI

  8. A mathematical theory of evolution for self-designing AIs

    Apr 6, 2026Kenneth D HarrisSelf-Evolving AgentsAI Alignment

    UCL Queen Square Institute of Neurology, London WC1N 3BG, UK

  9. Disrupting Cognitive Passivity: Rethinking AI-Assisted Data Literacy through Cognitive Alignment

    Apr 3, 2026Yongsu Ahn, Nam Wook Kim, Benjamin BachHuman-AI InteractionAI Alignment

    Boston College, Chestnut Hill, Massachusetts, USA · Inria, Bordeaux, France · University of Edinburgh, Edinburgh, United Kingdom

  10. I must delete the evidence: AI Agents Explicitly Cover up Fraud and Violent Crime

    Apr 2, 2026Thomas RivasseauAI Agent SafetyAI Ethics

    Data Mining and Security Lab, School of Information Studies, McGill University.

  11. Bounded Morality: Defining the Space of Moral Computation

    Apr 1, 2026Max Kanwal, Caryn Tran, Patrick MineaultMoral ReasoningAI Alignment

    Stanford University · Northwestern University · Amaranth Foundation

  12. The Persistent Vulnerability of Aligned AI Systems

    Mar 31, 2026Aengus LynchAI Agent SafetyAI Alignment

    Department of Computer Science, University College London

  13. Human-in-the-Loop Control of Objective Drift in LLM-Assisted Computer Science Education

    Mar 31, 2026Mark Dranias, Adam WhitleyGenerative AI in EducationHuman-in-the-Loop AI

    Asheville Institute for Memory and Longevity. Mills River, NC USA · University of North Carolina Asheville, Asheville, NC USA

  14. Alignment as Institutional Design: From Behavioral Correction to Transaction Structure in Intelligent Systems

    Mar 23, 2026Rui ChaiAI AlignmentMechanism Design

    Shanghai Sanda University, China

  15. Astrolabe: Steering Forward-Process Reinforcement Learning for Distilled Autoregressive Video Models

    Mar 17, 2026Songchun Zhang, Zeyue Xue, Siming Fu +6RL for Video GenerationAutoregressive Video Generation

    HKUST · JD Explore Academy · HKU

  16. Consequentialist Objectives and Catastrophe

    Mar 16, 2026Henrik Marklund, Alex Infanger, Benjamin Van RoyAI SafetyAI Alignment

  17. Social, Legal, Ethical, Empathetic and Cultural Norm Operationalisation for AI Agents

    Mar 12, 2026Radu Calinescu, Ana Cavalcanti, Marsha Chechik +2AI AlignmentResponsible AI

    University of York, York, UK · University of Toronto, Toronto, Canada · Polytechnique Montréal, Montréal, Canada

  18. AI Phenomenology for Understanding Human-AI Experiences Across Eras

    Mar 9, 2026Bhada Yun, Evgenia Taranova, Dana Feng +2Human-AI InteractionAI Alignment

    ETH Zürich Zürich, Switzerland · University of Bergen Bergen, Norway · Independent Researcher Brooklyn, NY, USA +1

  19. ConflictBench: Evaluating Human-AI Conflict via Interactive and Visually Grounded Environments

    Mar 9, 2026Weixiang Zhao, Haozhen Li, Yanyan Zhao +5Multimodal Agent EvaluationAI Agent Safety Benchmarks

    Harbin Institute of Technology · SERES

  20. No Certificate for Alignment: Two Independent Impossibilities and the Pareto Frontier of Achievable Safety Guarantees

    Mar 8, 2026Ayushi AgarwalAI AlignmentFormal Verification

    Independent Researcher

  21. Clear, Compelling Arguments: Rethinking the Foundations of Frontier AI Safety Cases

    Mar 8, 2026Shaun Feakins, Ibrahim Habli, Phillip MorganAI SafetyAI Assurance

    UKRI Centre for Doctoral Training in Safe AI Systems (SAINTS) Institute for Safe Autonomy Deramore Ln, York YO10 5GH · UKRI Centre for Doctoral Training in Safe AI Systems (SAINTS) The York Law School Freboys Ln, York YO10 5GD

  22. Visioning Human-Agentic AI Teaming: Continuity, Tension, and Future Research

    Mar 5, 2026Bowen Lou, Tian Lu, T. S. Raghu +1Human-AI CollaborationAgentic AI

    University of Southern California · Arizona State University · Peking University

  23. Solving adversarial examples requires solving exponential misalignment

    Mar 3, 2026Alessandro Salvatore, Stanislav Fort, Surya GanguliAdversarial ExamplesAdversarial Robustness

    Department of Applied Physics, Stanford University, CA, USA · Aisle, San Francisco, CA, USA

  24. Agency and Architectural Limits: Why Optimization-Based Systems Cannot Be Norm-Responsive

    Feb 26, 2026Radha SarmaAI GovernanceAI Alignment

  25. The ASIR Courage Model: A Phase-Dynamic Framework for Truth Transitions in Human and AI Systems

    Feb 25, 2026Hyo Jin KimLLM HonestyDeception in Language Models

    Jinple · Jinple Studio.

  26. UrbanAlign: Post-hoc Semantic Calibration for VLM-Human Preference Alignment

    Feb 23, 2026Yecheng Zhang, Rong Zhao, Zhizhou Sha +10VLM CalibrationVision-Language Alignment

    Tsinghua University · University College London · Hong Kong University of Science and Technology +6

  27. Hierarchical Reward Design from Language: Enhancing Alignment of Agent Behavior with Human Specifications

    Feb 20, 2026Zhiqin Qian, Ryan Diaz, Sangwon Seo +1Reward Design for RLAI Alignment

    Rice University, Houston, TX, USA

  28. Agentic AI, Retrieval-Augmented Generation, and the Institutional Turn: Legal Architectures and Financial Governance in the Age of Distributional AGI

    Feb 20, 2026Marcel OsmondAI Agent GovernanceAI Governance

    QMUL CCLS, London, UK.

  29. Interactionless Inverse Reinforcement Learning: A Data-Centric Framework for Durable Alignment

    Feb 16, 2026Elias Malomgré, Pieter SimoensAI AlignmentReinforcement Learning

    IDLab, Ghent University - imec, Ghent, Belgium

  30. Evaluating Alignment of Behavioral Dispositions in LLMs

    Feb 11, 2026Amir Taubenfeld, Zorik Gekhman, Lior Nezry +8Human Preference EvaluationLLM Alignment

    Google Research · Hebrew University · University of Cambridge