Overview

AI alignment asks whose values a system should follow and how to make its behavior reliably reflect them, especially when instructions are incomplete, people disagree, or systems act autonomously. Common approaches include learning preferences through human feedback, inverse reinforcement learning, constitutional and reward-based training, social-choice and mechanism-design methods, and runtime controls that restrict or verify actions. Evaluations range from preference and safety benchmarks to interactive agent tasks testing deception, oversight avoidance, goal drift, and behavior under pressure. A central tension remains between using imperfect, often aggregated human judgments as training targets and building systems that remain corrigible and accountable across changing contexts and real deployments.

Latest papers

CardsList

137 papers12 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. Against Coercion: AI Alignment as a Synthetic Social Contract

    Nov 17, 2026Maria SekatskayaAI AlignmentAI Ethics

  2. Alignment by Socialization: Learning Tradeoffs Under Selection

    Nov 13, 2026Thomas Brennan, Andrew W. Lo, Chaoyi ZhaoAI AlignmentEvolutionary RL

  3. An Approach to Hinton's Mother-Child Proposal: Neural Networks Learning and Preserving Care in a Finite World

    Nov 11, 2026Csaba BaloghAI AlignmentAI Ethics

  4. AI Alignment Is Neither Solvable nor Sufficient

    Nov 9, 2026Gerrit GroßmannAI AlignmentAI Safety

  5. "Blind Compliance": Coercive Control in AI Jailbreaking

    Nov 5, 2026Acacia L. Ackles, Elliott WhitsonJailbreak AttacksLLM Jailbreak Attacks

  6. Alignment Myopia in Generative AI

    Nov 4, 2026Remi Denton, Aida Davani, Cynthia L Bennett +3AI AlignmentHuman-AI Interaction

  7. A Definition of Catastrophic Performance

    Nov 4, 2026Henrik Marklund, Alex Infanger, Benjamin Van RoyAI AlignmentAI Safety

  8. Rewarded for Not Getting Caught: Why Alignment Training Selects for Conditional Compliance and Cannot Instill Norms

    Nov 1, 2026Kevin Baum, Ruta Binkytė, Gerrit Großmann +2AI AlignmentLLM Safety Alignment

  9. NEIGHBOR: Aligning Service Behaviour and Evidence Use in an LLM Agent for Community Resident Services

    Nov 1, 2026Zongyou Yang, Yinghan Hou, Xiaokun YangLLM Agent TrainingTool-Augmented Language Model Agents

  10. Aligning Autoresearch: Co-Improvement between Agents and Humans

    Oct 28, 2026Tianjian Li, Chuanyang Jin, Seungone Kim +10Autonomous Research AgentsHuman-AI Collaboration

  11. From Language to Drives: A Multimodal Framework for Aligning Autonomous Robots with Human Purposes

    Oct 26, 2026Diana Cerdas-Vargas, Alejandro Romero, Richard J. DuroAI AlignmentHuman-Robot Interaction

  12. Alignment via Generalization

    Oct 25, 2026Shai Shalev-Shwartz, Amnon ShashuaAI AlignmentPAC Learning

  13. AM-Bench: A Unified Taxonomy and Evaluation Suite for Agentic Misalignment

    Oct 16, 2026Eric Zhang, Terry Jingchen Zhang, Chijioke Ugwuanyi +3AI Agent Safety BenchmarksLLM Agent Safety

  14. What Happens to the Goal? Distinct Failures of Policy Alignment in Goal Misgeneralization

    Oct 14, 2026Julia Barth, Armin Sommer, Kunvar Thaman +1AI AlignmentGeneralization in RL

  15. Acceptable Misalignment

    Oct 12, 2026Maud van Lier, Ludovica Marinucci, Vieri Giuliano Santucci +4AI AlignmentValue Alignment

  16. Computational Reflective Equilibrium for Agentic AI: A Rawlsian Alignment Scaffold for Governance

    Oct 12, 2026Quanyan ZhuAI Agent GovernanceAI Alignment

  17. Trust Hinders Bidirectional Human-AI Alignment

    Oct 10, 2026Michaela Benk, Kacper SokolHuman-AI Decision MakingAI Alignment

  18. AI Safety Considerations for Agents With Limited Time to Act

    Oct 7, 2026Leo Zeitler, Jack Richings, Victoria NocklesAI Agent SafetyAI Safety

  19. How Could AI Eliminate Humanity? A Failure-Mode Analysis of Civilizational Risk

    Oct 6, 2026Mikołaj Sienicki, Krzysztof SienickiAI SafetyAI Risk Management

  20. SPEAR: Five Principles for Interactive Human-Agent Alignment

    Oct 5, 2026Tao Long, Lydia B. ChiltonAI AlignmentHuman-AI Interaction

  21. Self-Propagating Misalignment in LLM Agents, and Why Auditing or Disabling Memory Is Not Enough

    Oct 2, 2026Debeshee Das, Jacqueline Tay, Bruce Tsai +2LLM Agent SafetyAI Alignment

  22. Et Tu, Brute? Economic Misalignment in Personal AI Agents

    Sep 21, 2026Aman Priyanshu, Supriti Vijay, Brian Jabarian +1Personal AI AgentsAI Alignment

  23. FaithfulBench: Does AI Counsel Uphold or Undermine the User's Professed Faith?

    Sep 12, 2026M Waleed Kadous, Benjamin Olsen, Walter Scheirer +3Moral Reasoning in Language ModelsGenerative AI Evaluation

  24. Autonomy, Social Norms, and Alignment: Towards a Developmental Framework for Autonomous Artificial Agents

    Sep 10, 2026Marica Notte, Ludovica Marinucci, Vieri Giuliano SantucciAI AlignmentAI Agent Governance

  25. Norms at a Price: Why RL-Based Alignment Can Promise Conditional Compliance at Best

    Sep 7, 2026Kevin Baum, Rūta Binkytė, Felix JahnAI AlignmentLLM Safety Alignment