Overview

AI safety evaluation asks whether models and AI systems behave safely—not only on isolated prompts, but across conversations, tool use, human-AI teams, and physical or high-stakes settings. Common approaches include red-teaming and adversarial testing, curated benchmarks, simulated agent tasks, human studies, runtime monitoring, and formal verification. Evaluations measure outcomes such as harmful assistance, missed hazards, refusal quality, robustness to shifts, and effects on users or interacting agents. A central challenge is that strong benchmark results provide limited assurance about rare failures or deployment behavior, so evaluations must make their assumptions and evidence limits explicit.

Latest papers

CardsList

160 papers20 papers in the last four weeks, down 5% on the four weeks before. 0.2% of all new papers.

  1. Evaluating Human-AI Safety: A Framework for Measuring Harmful Capability Uplift

    Mar 6, 2026Michelle Vaccaro, Jaeyoon Song, Abdullah Almaatouq +1AI Safety EvaluationLLM Safety Evaluation

  2. Measuring AI R&D Automation

    Mar 4, 2026Alan Chan, Ranay Padarath, Joe Kwon +2AI Risk ManagementHuman Oversight of AI

  3. SEED-SET: Scalable Evolving Experimental Design for System-level Ethical Testing

    Mar 2, 2026Anjali Parashar, Yingke Li, Eric Yang Yu +4AI Safety EvaluationBenchmark Design

  4. When can we trust untrusted monitoring? A safety case sketch across collusion strategies

    Feb 24, 2026Nelson Gardner-Challis, Jonathan Bostock, Georgiy Kozhevnikov +4AI Agent MonitoringAI Safety

  5. AI Alignment Breaks at the Edge

    Feb 23, 2026Han Bao, Yue Huang, Xiaoda Wang +5LLM AlignmentAI Safety Evaluation

  6. GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory

    Feb 12, 2026Pepijn Cobben, Xuanqiang Angelo Huang, Thao Amelia Pham +4AI Safety EvaluationAI Agent Safety Benchmarks

  7. PB-OEL: A Performance-Bounded Online Ensemble Learning Framework With Mixed Feedback for Real-Time Safety Assessment

    Mar 19, 2025Songqiao Hu, Zeyi Liu, Lufeng Hao +2Ensemble LearningAI Safety Evaluation