Overview

AI agent safety asks how to keep systems that plan over multiple steps and use tools, memory, or other agents within authorized and harmless bounds. Researchers test computer-use, coding, web, and embodied agents with prompt-injection, misuse, and long-horizon trajectory benchmarks, measuring executed effects as well as final answers. Approaches include red-teaming and monitoring, safety fine-tuning and refusal training, and runtime controls such as least-privilege access, pre-execution policy gates, sandboxing, human approval, and formal verification. A central challenge is that safety can fail through ambiguous goals, changing context, tool or agent interactions, and pressure over time, while stronger restrictions can block legitimate tasks and assurances may not transfer to deployment.

Latest papers

CardsList

353 papers28 papers in the last four weeks, down 49% on the four weeks before. 0.2% of all new papers.

  1. Beyond Acuity Accuracy: A Multidimensional Framework for Evaluating Patient-Facing Emergency Triage AI Tools

    Nov 17, 2026Ruio Heerschap, Henry Foote, Mahmoud Alwakeel +19Clinical TriageClinical Language Model Evaluation

  2. Protecting the System, Stranding the User: The Limits of Multi-Agent Authorization

    Nov 14, 2026Gurpriya Bhatia, Clifford NeumanAI Agent SafetyAI Agent Governance

  3. No finite rulebook can certify every safe program: an impossibility theorem for gatekeeping AI agents

    Nov 9, 2026Tomas Perez-AcleAI Agent SafetyFormal Verification

  4. Do AI Agents need Preventive Care? An Epidemiologic Cohort Design for Agentic AI

    Nov 9, 2026Shu Jiang, Graham A. ColditzAI Agent SafetyAI Agent Reliability

  5. When Coding Agents Get Arms: Safety Risks Across the Digital-Physical Boundary

    Nov 8, 2026Yiming Sun, Mi ZhangAI Agent SafetyAI Coding Agents

  6. Aligning, Fast and Slow: A Control-Theoretic Oversight Framework for Long-Horizon Trajectories and Maintained Deployments

    Nov 5, 2026Stephen Peng, Vineet Jagadeesan Nair, Noah Broestl +3AI Agent SafetyAI Agent Governance

  7. Goal Revelation as a Bridge Between Information-Theoretic Empowerment and Capability-Based Human Power

    Nov 4, 2026Jaime Ruiz-Serra, Tomáš Gavenčiak, Jobst HeitzigAI Agent SafetyEmpowerment

  8. Trust Responsive Runtime Authorization for Agentic AI with Empirical Characterization of Policy Behavior and Parameter Sensitivity

    Nov 4, 2026Ahsan Habib Shihab, Imre Felde, Ramis Akhmedov +4Runtime Enforcement for AI AgentsAccess Control

  9. Welfare Regard in Multi-Agent Systems

    Nov 4, 2026Darion Cassel, Ariana Azarbal, Anna SoligoAI Agent SafetyMulti-Agent LLMs

  10. Airlock: Bounding the Multi-Agent Blast Radius via Selective Online Containment

    Nov 4, 2026Zhonghao Zhan, Javad Forough, Yefan Zhang +3AI Agent SafetyMulti-Agent Systems

  11. A Definition of Catastrophic Performance

    Nov 4, 2026Henrik Marklund, Alex Infanger, Benjamin Van RoyAI AlignmentAI Safety

  12. Inter-Agent Influence: Evaluating Persuasion, Deception, and Coercion in Multi-Agent Systems

    Nov 3, 2026Chandler Smith, Qi Guo, Cecilia Elena Tilli +6AI Agent SafetyMulti-Agent Systems

  13. Widening Harm Evaluation Coverage with a Judge that Reads the World

    Nov 2, 2026Prabhpreet Singh Setia, Bhavesh Neekhra, Debayan GuptaAI Agent SafetyAI Safety Evaluation

  14. Step-Zero Attack: Agent Safety Before Observability

    Nov 1, 2026Tian Qin, Yuhang Jiang, Junzhe Chen +2AI Agent SecurityAI Agent Safety

  15. Dependence-Aware Risk Assessment and Optimal Design of AI-Agent Safeguard Systems

    Oct 30, 2026Yijia Li, Maochao Xu, Peng ZhaoAI Risk ManagementAI Agent Security

  16. Beyond Plausible Prediction: Safety Sufficiency in Web World Models

    Oct 29, 2026Zhifei Hu, Alexandra I. Cristea, Adel Bibi +1AI Agent SafetyAI Safety Evaluation

  17. "Shall We Approve?" Consequence-Aware Approval for Autonomous Coding Agents

    Oct 29, 2026Weiheng Wu, Junkai Chen, Zhensu Sun +3AI Agent SafetyTool Access Control for LLM Agents

  18. Appreciation Teaches Unauthorized Execution: Self-Evolution Degrades the Robustness of Personal Agents to Indirect Prompt Injection

    Oct 29, 2026Xinyan Wang, Xiaogeng Liu, Jun Shao +1AI Agent SecurityAI Agent Safety

  19. Auditable Belief Updates as a Missing Safety Layer for LLM-Powered Multi-Agent Systems

    Oct 28, 2026Zhuoyun Li, Boxuan Wang, Jinwei Hu +2AI Agent SafetyLLM Agent Safety

  20. I-HITL: Inalienable Human-in-the-Loop

    Oct 26, 2026Samuel Breckenridge, Dani Vilardell, Wenhao Wang +2Human-in-the-Loop AIAI Agent Safety

  21. ShutdownMod: Turning Any Trajectory into a Shutdown Evaluation

    Oct 26, 2026Param Biyani, Sebastien Luyasu, Cedric Potvliege +3AI Agent SafetyAI Agent Evaluation

  22. V3^3AST-Bench: Verifiable, Valid, and Versatile Safety Evaluation for Coding Agents

    Oct 26, 2026Weitao Feng, Hengheng Xiong, Zihao Zeng +6Runtime Verification for AI AgentsAI Agent Safety

  23. A Three-Pillar Governance Framework for Safe and Trustworthy AI Agents

    Oct 21, 2026Edward C. Cheng, Jeshua Cheng, Alice SiuAI Agent GovernanceAI Agent Safety

  24. Proof-Carrying Contracts: Commitments AI Agents Cannot Break

    Oct 20, 2026Carlo Ciliberto, Alessandro Rudi, Enrico TassiRuntime Enforcement for AI AgentsAI Agent Safety