AI Alignment
Overview
AI alignment asks whose values a system should follow and how to make its behavior reliably reflect them, especially when instructions are incomplete, people disagree, or systems act autonomously. Common approaches include learning preferences through human feedback, inverse reinforcement learning, constitutional and reward-based training, social-choice and mechanism-design methods, and runtime controls that restrict or verify actions. Evaluations range from preference and safety benchmarks to interactive agent tasks testing deception, oversight avoidance, goal drift, and behavior under pressure. A central tension remains between using imperfect, often aggregated human judgments as training targets and building systems that remain corrigible and accountable across changing contexts and real deployments.
Latest papers
126 papers12 papers in the last four weeks, up 20% on the four weeks before. 0.1% of all new papers.