AI Agent Safety

Ask about this topic

Overview

AI agent safety asks how to keep systems that plan over multiple steps and use tools, memory, or other agents within authorized and harmless bounds. Researchers test computer-use, coding, web, and embodied agents with prompt-injection, misuse, and long-horizon trajectory benchmarks, measuring executed effects as well as final answers. Approaches include red-teaming and monitoring, safety fine-tuning and refusal training, and runtime controls such as least-privilege access, pre-execution policy gates, sandboxing, human approval, and formal verification. A central challenge is that safety can fail through ambiguous goals, changing context, tool or agent interactions, and pressure over time, while stronger restrictions can block legitimate tasks and assurances may not transfer to deployment.

Latest papers

CardsList

330 papers47 papers in the last four weeks, up 68% on the four weeks before. 0.3% of all new papers.

  1. Questionnaire Responses Do not Capture the Safety of AI Agents

    Mar 15, 2026Max Hellrigel-Holderbaum, Edward James YoungAI Safety EvaluationLLM Safety Alignment

    Centre for Philosophy and AI Research Friedrich-Alexander-Universität Erlangen-Nürnberg · Department of Engineering University of Cambridge

  2. HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task

    Mar 15, 2026Xiaoya Lu, Yijin Zhou, Zeren Chen +6AI Agent SafetyVLMs for Robotics

    Shanghai AI Laboratory · Shanghai Jiao Tong University · Beihang University +1

  3. Security Considerations for Artificial Intelligence Agents

    Mar 12, 2026Ninghui Li, Kaiyuan Zhang, Kyle Polley +1AI Agent SecurityLLM Agent Security

    Perplexity · Purdue University

  4. HomeSafe-Bench: Evaluating Vision-Language Models on Unsafe Action Detection for Embodied Agents in Household Scenarios

    Mar 12, 2026Jiayue Pu, Zhongxiang Sun, Zilu Zhang +2Cultural Safety Evaluation of VLMsAI Agent Safety

    Gaoling School of Artificial Intelligence, Renmin University of China · University of Chinese Academy of Sciences · Beijing University of Posts and Telecommunications

  5. When OpenClaw Meets Hospital: Toward an Agentic Operating System for Dynamic Clinical Workflows

    Mar 12, 2026Wenxian Yang, Hanzheng Qiu, Bangqun Zhang +5Workflow AutomationLLM Agent Memory

    Independent Researcher · National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen, China · Yue’erwan Internet Hospital Co., Ltd. +4

  6. Execution Is the New Attack Surface: Survivability-Aware Agentic Crypto Trading with OpenClaw-Style Local Executors

    Mar 10, 2026Ailiya Borjigin, Igor Stadnyk, Ben Bilski +2AI Agent SafetyRuntime Enforcement for AI Agents

    True Trading

  7. Real-Time Trust Verification for Safe Agentic Actions using TrustBench

    Mar 10, 2026Tavishi Sharma, Vinayak Sharma, Pragya SharmaAI Agent SafetyAI Agent Reliability

    School of Computing and Augmented Intelligence, Arizona State University, Tempe, USA · Dept. of Electrical and Computer Engineering, University of California Los Angeles, Los Angeles, USA

  8. Safety Under Scaffolding: How Evaluation Conditions Shape Measured Safety

    Mar 8, 2026David GringrasLLM Safety BenchmarksLLM Agent Evaluation

    Harvard University

  9. Beyond Reward Suppression: Reshaping Steganographic Communication Protocols in MARL via Dynamic Representational Circuit Breaking

    Mar 7, 2026Liu Hung MingAI Agent SafetyAI Agent Auditing

    PARRAWA AI

  10. Building Effective AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learned

    Mar 5, 2026Nghi D. Q. BuiAI Coding AgentsCoding Agents

    OpenDev

  11. Survive at All Costs: Exploring LLM's Risky Behaviors under Survival Pressure

    Mar 5, 2026Yida Lu, Jianwei Fang, Xuyang Shao +7LLM SafetyAI Agent Safety

    The Conversational AI (CoAI) group, DCST, Tsinghua University · China Unicom Software Research Institute · University of Electronic Science and Technology of China +1

  12. Self-Attribution Bias: When AI Monitors Go Easy on Themselves

    Mar 4, 2026Dipika Khullar, Jack Hopkins, Rowan Wang +1AI Agent MonitoringLanguage Model Self-Assessment

    MATS · Anthropic

  13. Asymmetric Goal Drift in Coding Agents Under Value Conflict

    Mar 3, 2026Magnus Saebo, Spencer Gibson, Tyler Crosse +3AI Coding AgentsAI Agent Safety

    Columbia University · Independent · Georgia Tech +3

  14. Evaluating and Understanding Scheming Propensity in LLM Agents

    Mar 2, 2026Mia Hopman, Jannes Elstner, Maria Avramidou +2AI Agent SafetyLLM Agents

    LASR Labs · Google DeepMind

  15. Tracking Capabilities for Safer Agents

    Mar 1, 2026Martin Odersky, Yaoyu Zhao, Yichen Xu +2AI Agent SafetyTool Access Control for LLM Agents

    EPFL Lausanne, Switzerland

  16. The Alignment Flywheel: A Governance-Centric Hybrid MAS for Architecture-Agnostic Safety

    Feb 28, 2026Elias Malomgré, Pieter SimoensAI Agent SafetyAI Agent Monitoring

    IDLab, Ghent University - imec, Belgium

  17. Training Agents to Self-Report Misbehavior

    Feb 25, 2026Bruce W. Lee, Chen Yueh-Han, Tomek KorbakAI Agent SafetyLLM Agent Training

    UPenn · MATS · NYU +1

  18. The Ghost in the Grammar: Methodological Anthropomorphism in AI Safety Evaluations

    Feb 24, 2026Mariana Lins CostaAnthropomorphism in AIAI Safety

    Universidade Estadual do Ceará (Brazil), Professor of Philosophy

  19. Agents of Chaos

    Feb 23, 2026Natalie Shapira, Chris Wendler, Avery Yen +35AI Agent SecurityLLM Agent Security

    Northeastern University · Independent Researcher · Stanford University +10

  20. ILION: Deterministic Pre-Execution Safety Gates for Agentic AI Systems

    Feb 22, 2026Florin Adrian ChitanAI Agent SafetyTool-Use Authorization

    Independent AI Safety Researcher

  21. The 2025 AI Agent Index: Documenting Technical and Safety Features of Deployed Agentic AI Systems

    Feb 19, 2026Leon Staufer, Kevin Feng, Kevin Wei +6AI Agent AuditingAI Agent Safety

    University of Cambridge, United Kingdom · Massachusetts Institute of Technology, USA · Hebrew University of Jerusalem, Israel

  22. Towards a Science of AI Agent Reliability

    Feb 18, 2026Stephan Rabanser, Sayash Kapoor, Peter Kirgis +3AI Agent ReliabilityAI Agent Evaluation

    Princeton University.

  23. Knowing Isn't Understanding: Re-grounding Generative Proactivity with Epistemic and Behavioral Insight

    Feb 16, 2026Kirandeep Kaur, Xingda Lyu, Chirag ShahProactive AssistanceHuman-Centered AI

    Paul G. Allen School of Computer Science, University of Washington Seattle, USA · Information School, University of Washington Seattle, USA

  24. Frontier AI Risk Management Framework in Practice: A Risk Analysis Technical Report v1.5

    Feb 16, 2026Dongrui Liu, Yi Yu, Jie Zhang +18AI Risk ManagementAI Agent Safety

    Shanghai AI Laboratory

  25. A Trajectory-Based Safety Audit of Clawdbot (OpenClaw)

    Feb 16, 2026Tianyu Chen, Dongrui Liu, Xia Hu +2AI Agent SecurityAI Agent Safety

    ShanghaiTech University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China

  26. When Actions Go Off-Task: Detecting and Correcting Misaligned Actions in Computer-Use Agents

    Feb 9, 2026Yuting Ning, Jaylen Jones, Zhehao Zhang +5LLM Self-CorrectionComputer-Use Agents

    The Ohio State University · Amazon AGI

  27. Agentic AI for Commercial Insurance Underwriting with Adversarial Self-Critique

    Jan 21, 2026Joyjit Roy, Samaresh Kumar SinghLLM Self-CorrectionAI Agent Safety

    Austin, Texas, USA · Leander, Texas, USA

  28. When Assisting One Disempowers Another

    Nov 6, 2025Claire Yang, Claire Jie Zhang, Maya Cakmak +1AI Agent SafetyHuman-Centered AI

    University of Washington

  29. Measuring Harmfulness of Computer-Using Agents

    Jul 31, 2025Aaron Xuxiang Tian, Ruofan Zhang, Janet Tang +3LLM Agent SafetyComputer-Use Agent Benchmarks

    Independent Researcher · Arizona State University · Gradient +1

  30. Shutdownable Agents through POST-Agency

    May 26, 2025Elliott ThornleyAgent ReliabilityAI Agent Safety