Overview

AI safety evaluation asks whether models and AI systems behave safely—not only on isolated prompts, but across conversations, tool use, human-AI teams, and physical or high-stakes settings. Common approaches include red-teaming and adversarial testing, curated benchmarks, simulated agent tasks, human studies, runtime monitoring, and formal verification. Evaluations measure outcomes such as harmful assistance, missed hazards, refusal quality, robustness to shifts, and effects on users or interacting agents. A central challenge is that strong benchmark results provide limited assurance about rare failures or deployment behavior, so evaluations must make their assumptions and evidence limits explicit.

Latest papers

CardsList

160 papers20 papers in the last four weeks, down 5% on the four weeks before. 0.2% of all new papers.

  1. Auditors Struggle to Catch Sabotage In End-to-End Transcripts of AI Safety Research

    Nov 16, 2026Keshav Shenoy, Anthony Zang, Eric Gan +5AI Agent AuditingAI Safety Evaluation

  2. Jev-as-a-Reward: Validating Probabilistic Safety Judges for Reinforcement Learning

    Nov 16, 2026Eunkyeong Lee, Junhyung Moon, Seunggwan Hong +1AI Safety EvaluationLLM-as-a-Judge

  3. Between Rigor and Reality: How AI Safety Benchmark Developers Understand Benchmark Use and Maintenance

    Nov 13, 2026Megan Li, Ningjing Tang, Christopher Stewart +2AI Safety EvaluationAI Safety

  4. Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing

    Nov 9, 2026Andrew Trask, Sol Messing, Vai Pahwa +27AI Safety EvaluationLLM Auditing

  5. Who Can Evaluate AI Independently? A Framework for Analysing Evaluation Infrastructure

    Nov 5, 2026Sumaya Nur Adan, Joanna Wiaterek, Nourhan Abdelrhim +4AI Safety EvaluationAI Governance

  6. A Behavioural Framework for Predicting and Understanding Loss of Control in Frontier Artificial Intelligence Systems

    Nov 5, 2026Yao Huang, Yichi Zhang, Yifan Wang +8AI Risk ManagementAI Safety Evaluation

  7. Alignment Investigation of Mythos 5's behaviour in UK AISI's July 2026 Security Incident

    Nov 5, 2026Olli Järviniemi, Alexandra Souly, Abby D'Cruz +3LLM Safety AlignmentLLM Security

  8. VeriHarm-Bench: Measuring Harmful Success in LLM Coding Agents Verifiably

    Nov 2, 2026Wei-Lin Chen, Zhepei Wei, Xinyu Zhu +3AI Agent Safety BenchmarksLLM Agent Safety

  9. Widening Harm Evaluation Coverage with a Judge that Reads the World

    Nov 2, 2026Prabhpreet Singh Setia, Bhavesh Neekhra, Debayan GuptaAI Agent SafetyAI Safety Evaluation

  10. Reasoning is topologically simple

    Nov 2, 2026Chris Kapulkin, Nathan KershawLanguage Model Reasoning EvaluationAI Safety Evaluation

  11. On the Framing Effect of AI Red-Teaming Instructions: Evidence from Prior Exercises and Controlled Simulations

    Oct 30, 2026Michael Feffer, Alice Qian, Anusha Sinha +6LLM Red TeamingAI Safety Evaluation

  12. Violet Teaming in Practice: Lessons from a Pilot Expert Workshop on Governing Complex AI Systems

    Oct 29, 2026Steph Batalis, Kait Connors, Ben C. Snyder +2AI GovernanceAI Safety Evaluation

  13. Beyond Plausible Prediction: Safety Sufficiency in Web World Models

    Oct 29, 2026Zhifei Hu, Alexandra I. Cristea, Adel Bibi +1AI Agent SafetyAI Safety Evaluation

  14. Clean Sheets and Checkpoints: Evidence Complexity for Certifying Sequential Agent Risk

    Oct 29, 2026Rohith Yanambaka Venkata, Don McBride, Tiago Sequeira +1AI Agent AuditingAI Safety Evaluation

  15. The Watchtower Imperative: Lifespan-Calibrated Evaluation for Frontier AI

    Oct 26, 2026Casey M. Williams, Jennifer Victoria ScurrellAI Safety Evaluation

  16. Trustworthy Recursive Self-Improvement in the Era of Large Models: Foundation, Risks, Evaluation and Open Challenges

    Oct 26, 2026Jiayue Pu, Ang Gao, Jinyi Bai +9Recursive Self-ImprovementLanguage Model Self-Improvement

  17. EAST: Embodied AI Safety Testing via Systematic Fault Injection in Digital Twins

    Oct 25, 2026Yuhao Tian, Tao Li, Ling Zhou +5AI Safety EvaluationRobot Safety

  18. BoardSafety: AI-Ready Data from Manufacturer Documentation for Development-Board Safety

    Oct 24, 2026Linjian Yang, Xinyan Wang, Kunpeng LiuAI Safety EvaluationLLM Safety Benchmarks

  19. Simulated Control Evals

    Oct 22, 2026Jeffrey Olmo, David Zhang, Tyler TracyAI Safety EvaluationLanguage Model Safety Evaluation

  20. Assessing the Empirical Evidence for Loss of Control from Agentic General-Purpose AI

    Oct 18, 2026Risto Uuk, Santeri Koivula, Lorenzo Pacchiardi +3AI Agent SafetyAI Safety

  21. Towards Human-AI Complementarity for AI Safety

    Oct 18, 2026Sander Lanbo Shi, Miranda Anna Christ, Eric Modesitt +20Human-AI ComplementarityHuman Oversight of AI

  22. How Intervention History Changes AI Agent Sabotage on Subsequent Tasks

    Oct 17, 2026Daniel Saeedi, Amirali AghazadehAI Agent SafetyAI Safety Evaluation

  23. What Social Media Teaches Us About Accumulative AI Risks

    Oct 15, 2026Dawei Li, Bohan Jiang, Huan LiuAI SafetyLLM Safety

  24. Limits of Statistical Assurance for Embodied and Agentic AI under Adaptive Adversaries

    Oct 13, 2026Inderjeet SinghAI AssuranceAI Safety Evaluation

  25. CompanionSafe: From Multi-turn Safety Evaluation to Guardrails for AI Companions

    Oct 13, 2026Xinyang Liao, Lingyu Li, Kaicheng Shen +4AI SafetyAI Safety Evaluation