AI Agent Safety Benchmarks

Overview

AI agent safety benchmarks test whether systems using tools, software, or physical devices follow instructions, respect permissions, avoid harmful side effects, and know when to abstain. Common approaches use executable sandboxes, simulated workplaces and embodied environments, multi-turn red teaming, and analysis of full action trajectories rather than final answers alone. Evaluations increasingly measure safety alongside task success, usability, authorization, and recovery, while runtime guards and deterministic checks are tested as possible safeguards. Results depend heavily on the model, harness, environment, and scoring method, and fragmented benchmarks make safety rankings difficult to compare; a central challenge is capturing long-horizon and real-world risks without confusing safe behavior with inability to act.

Latest papers

CardsList

240 papers29 papers in the last four weeks, down 41% on the four weeks before. 0.2% of all new papers.

  1. Safe Personal RSI: Evaluating and Repairing Personal-Agent Updates

    Nov 18, 2026Jiapeng Pei, Yuhang Yao, Sitan Liu +9LLM Safety EvaluationLLM Agent Self-Improvement

  2. MemeEconomy: Do LLM Agents Trade Ethics for Survival?

    Nov 13, 2026Syed Nazmus Sakib, Nafiul Haque, Ahnaf Tahmid Manan +2LLM Agent SafetySafety-Utility Trade-Off in Language Models

  3. LEASH: Stay Safe in Agentic Robotics

    Nov 10, 2026Qinfeng Zhu, Weiguang Zhao, Ke Sun +5Robot SafetyAgentic AI for Robotics

  4. When Coding Agents Get Arms: Safety Risks Across the Digital-Physical Boundary

    Nov 8, 2026Yiming Sun, Mi ZhangAI Agent SafetyAI Coding Agents

  5. ErosionBench: Probing Alignment Decay in Enterprise Multi-Agent Systems

    Nov 8, 2026Riya Bharathwaj, Akshara Prabhakar, Roshan Ram +4AI Agent Safety BenchmarksMulti-Agent LLM Systems

  6. ABHBench: Evaluating Moral Decision-Making of Foundation Models from an Agentic Perspective

    Nov 6, 2026Chaoran Li, Yukun Li, Zeyuan Zhao +3Moral Reasoning in Language ModelsAI Agent Safety Benchmarks

  7. A Behavioural Framework for Predicting and Understanding Loss of Control in Frontier Artificial Intelligence Systems

    Nov 5, 2026Yao Huang, Yichi Zhang, Yifan Wang +8AI Risk ManagementAI Safety Evaluation

  8. Four Bridges: Measuring Deception Propensity in a Multi-Agent Game

    Nov 2, 2026Raphael Casey, Michael Simeone, Alberto Tosato +4Deception in Language ModelsAI Agent Evaluation

  9. VeriHarm-Bench: Measuring Harmful Success in LLM Coding Agents Verifiably

    Nov 2, 2026Wei-Lin Chen, Zhepei Wei, Xinyu Zhu +3AI Agent Safety BenchmarksLLM Agent Safety

  10. Do Agents Still Follow the Rules? Benchmarking Rule Compliance under Agent Context Compaction

    Nov 2, 2026Yiwei Zha, Qingyu Meng, Herbert Bos +1AI Agent Safety BenchmarksLLM Agent Safety

  11. MemHarmBench: Evaluating Cross-Session Prompt Injection in Memory-Persistent AI Agents

    Nov 1, 2026Zhihua Wang, Yunhao Feng, Yifeng Gao +4Prompt Injection AttacksPrompt Injection Attacks on AI Agents

  12. What Standard Evaluations Hide About Detectors of Hijacked LLM Agents: Rare Attacks, Unseen Tools and Benchmark Transfer

    Oct 30, 2026Mahmoud Abbasi, Juan M. Núñez V., Amin ShahrakiLLM Agent SecurityAI Agent Monitoring

  13. BABA-Bench: Evaluating Long-Horizon Patient-Advocacy Agents

    Oct 29, 2026Shriram Vasudevan, Videet Mehta, Connor SweeneyLong-Horizon LLM AgentsLLM Agent Policy Compliance

  14. P-ContextBench: From Personal Fact Recall to Personalized Agent Behavior

    Oct 29, 2026Grant Nasir Brown, Abdul Basir Bari, Anmol Tadikonda +1LLM Agent EvaluationAI Agent Safety Benchmarks

  15. ShutdownMod: Turning Any Trajectory into a Shutdown Evaluation

    Oct 26, 2026Param Biyani, Sebastien Luyasu, Cedric Potvliege +3AI Agent SafetyAI Agent Evaluation

  16. V3^3AST-Bench: Verifiable, Valid, and Versatile Safety Evaluation for Coding Agents

    Oct 26, 2026Weitao Feng, Hengheng Xiong, Zihao Zeng +6Runtime Verification for AI AgentsAI Agent Safety

  17. SentryBench: A Cost-Sensitive Benchmark for Adversarial Robustness of Tool-Using LLM Agents in Public-Safety Deployments

    Oct 25, 2026Md Abu Yusuf, Oyshi Tabassum Aditi, Md Faysal Hosain MazedLLM Agent SafetyAI Agent Safety Benchmarks

  18. Task Success Is Not Enough: Side-effect-Aware Evaluation of Tool-Using Language Model Agents

    Oct 25, 2026Jiaju Huang, Shaobin Chen, Xinglong Liang +5LLM Agent EvaluationLLM Agent Safety

  19. BARC-Bench: Benign Asks, Risky Consequences

    Oct 23, 2026Kangkang Li, Kelvin Luu, Lily Ma +3AI Agent Safety BenchmarksLLM Agent Safety

  20. EpiSEM-Bench: Evaluating Safety and Evidence Recovery in LLM Agents under Missing Evidence

    Oct 22, 2026Qijin Zhang, Hanzhen Wang, Yuehan Zhao +4AI Agent Safety BenchmarksLLM Agent Safety

  21. CAPSafe: A Benchmark for Contextual Action and Plan Safety Reasoning in Embodied Agents

    Oct 20, 2026Youmna Farag, Svetlana Stoyanchev, Simon Keizer +2AI Agent SafetyAI Agent Safety Benchmarks

  22. RSI Control: Can AI Models Perform Covert Sabotage in the Self-Improvement Process?

    Oct 20, 2026Kaiwen Zhou, Udari Madhushani SehwagRecursive Self-ImprovementAI Control