AI Agent Safety

Ask about this topic

Momentum

22 papers in the last four weeks, up 144% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 207

All topics
CardsList

  1. A Self-Evolving Agent for Longitudinal Personal Health Management

    Jul 15, 2026Haoran Li, Jiebi Deng, Tong Jin +10Agent MemoryHealthcare

    School of Data Science, Fudan University, Shanghai, China · School of Life Sciences, Beijing University of Chinese Medicine, Beijing, China · Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University, Shanghai, China +4

  2. Agent-Safety Evaluations as Load-Bearing Evidence: A Vendor-Neutral, Cross-Harness Reconstructability Metric

    Jul 14, 2026Oleg SolozobovAI Agent SafetyAI Safety Evaluation

    Independent Researcher (Global)

  3. ANCHOR: Automated Alignment Auditing for CLI Agents on Real-World Harm

    Jul 11, 2026Kefan Song, Yanjun QiAI Agent AuditingLLM Auditing

    University of Virginia, Charlottesville, VA, USA

  4. Neuro-Agentic Control: A Deep Learning-based LLM-Powered Agentic AI Framework for Controlling Security Controls

    Jul 10, 2026Saroj Gopali, Bipin Chhetri, Deepika Giri +2Language Model-Based ControlSafety-Critical Control

    Department of Computer Science1, Texas Tech University1 · Cumberland University2 · Advanced Academic Programs Science3, Johns Hopkins University3

  5. Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety

    Jul 8, 2026Yujiao ChenAlgorithmic FairnessAI Agent Safety

    Massachusetts Institute of Technology · Cambridge, MA 02139

  6. Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents

    Jul 8, 2026Harry Owiredu-AshleyLLM Red TeamingAI Agent Safety

    Independent Researcher New Jersey, USA

  7. Initiation Safety: A Missing Dimension in Generalist-Robot Safety

    Jul 8, 2026Zhijin Meng, Francisco CruzAI Agent SafetyRobot Safety

    School of Computer Science and Engineering University of New South Wales

  8. Agentic Data Environments

    Jul 8, 2026Elaine Ang, Chenxi Huang, Georgios Liargkovas +13AI Agent ReliabilityAI Agent Safety

    Columbia University

  9. Governed Individuation: Cryptographically Decoupling an Agent's Learning from Its Authority

    Jul 6, 2026Xue Qin, Simin Luan, Cong Yang +1AI Agent SecurityAI Agent Safety

    School of Software, Harbin Institute of Technology, Harbin, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China · School of Future Science and Engineering, Soochow University, Suzhou, China

  10. CAGE-1: Control, Assurance, and Governance Evaluation for Enterprise Agentic AI

    Jul 3, 2026Roopam W. SureAI Agent EvaluationAI Agent Safety

  11. Hardware-Enforced Semantic Coordination for Safety-Critical Real-Time Autonomous Systems

    Jul 2, 2026Uwe M. Borghoff, Paolo Bottoni, Remo PareschiMulti-Agent CoordinationSafety-Critical Control

    Department of Computer Science University of the Bundeswehr Munich Neubiberg, Germany · Department of Computer Science Sapienza University of Rome Rome, Italy · STAKE Lab University of Molise Campobasso, Italy

  12. Criticality-Based Guard Rail Validation for AI Agent Decisions in Autonomous Telecom Networks

    Jul 2, 2026Ravi Kant SharmaAI Agent SafetyAI Agent Governance

    Ericsson

  13. Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification

    Jul 2, 2026Yunhao Feng, Ruixiao Lin, Ming Wen +12AI Agent SafetyAI Agent Safety Benchmarks

    AntGroup · Hunan Institute of Advanced Technology · Zhejiang University +3

  14. Mnemosyne: Agentic Transaction Processing for Validating and Repairing AI-generated Workflows

    Jun 30, 2026Edward Y. Chang, Longling GengAgentic WorkflowsAI Agent Safety

    Stanford University

  15. EgoSafetyBench: A Diagnostic Egocentric Video Benchmark for Evaluating Embodied VLMs as Runtime Safety Guards

    Jun 30, 2026Siddhant Panpatil, Arth Singh, Mijin Koo +3VLM RobustnessEgocentric Video Understanding

    AIM Intelligence · Seoul National University

  16. A Contextual-Bandit Oversight Game with Two-Sided Informational Asymmetry

    Jun 30, 2026Yunjin TongReinforcement LearningImperfect-Information Games

    Stanford Graduate School of Business

  17. LabGuard: Grounding Natural-Language Laboratory Rules into Runtime Guards for Embodied Laboratory Agents

    Jun 30, 2026Jingpu Yang, Fengxian Ji, Zhengzhao Lai +8AI Agent SafetyAI Agent Monitoring

    1Wuhan University · 2MBZUAI · 3Northeastern University

  18. Certified Speculative Execution for Untrusted AI Agents

    Jun 30, 2026Chenyu Zhou, Qiliang Jiang, Shuning Wu +1AI Agent ReliabilityConstrained RL

    School of Engineering, Institute of Science Tokyo, Japan · College of Control Science and Engineering, Zhejiang University, China · Department of Electrical and Computer Engineering, National University of Singapore, Singapore

  19. The Two Genie Game: Adoption and Welfare in Audit-Grounded AI Governance

    Jun 27, 2026Darrell Lewis-SandyAlgorithmic AuditingAI Agent Safety

  20. AI Persuasive Framing in Collective Dilemmas

    Jun 26, 2026Anders Giovanni Møller, Alessia Galdeman, Arianna Pera +1AI Agent Safety

    Data Science Section, IT University of Copenhagen, Denmark · Computer Science Department, University of Milan, Italy · Copenhagen Center for Social Data Science, University of Copenhagen, Denmark

  21. Yuvion LLM: An Adversarially-Aware Large Language Model for Content And AI Safety

    Jun 26, 2026Ting Ma, Xiufeng Huang, Benlei Cui +43LLM Safety BenchmarksLLM Safety Alignment

  22. Radical AI Interpretability

    Jun 25, 2026Daniel A. Herrmann, Benjamin A. LevinsteinExplainable Artificial IntelligenceTheory of Mind

    University of North Carolina, Chapel Hill · Anthropic · University of Illinois, Urbana-Champaign

  23. Beyond Feedforward Networks: Reentry Neural Systems as the Fundamental Basis of Subjecthood and Intrinsic Safety of Next-Generation AGI

    Jun 24, 2026A. S. Ushakov, Yu. N. BerdinskRecurrent Neural NetworksAI Agent Safety

    Saint Petersburg State University, Department of High Energy Physics and Elementary Particles

  24. Governing Actions, Not Agents: Institutional Attestation as a Governance Model for Autonomous AI Systems

    Jun 24, 2026Jakob Salfeld-NebgenAI Agent SafetyAI Agent Governance

  25. Model Forensics: Investigating Whether Concerning Behavior Reflects Misalignment

    Jun 24, 2026Aditya Singh, Gerson Kroiz, Senthooran Rajamanoharan +1AI Agent Safety

    MATS · Google DeepMind

  26. The Unfireable Safety Kernel: Execution-Time AI Alignment for AI Agents and Other Escapable AI Systems

    Jun 24, 2026Seth Dobrin, Łukasz ChmielAI AlignmentAI Agent Security

    ARYA Labs PBC

  27. Critique of Agent Model

    Jun 22, 2026Eric Xing, Mingkai Deng, Jinyu HouAI Agent SafetyAgentic AI

    Institute of Foundation Models, Mohamed bin Zayed University of Artificial Intelligence · School of Computer Science, Carnegie Mellon University

  28. AutoRAS: Learning Robust Agentic Systems with Primitive Representations

    Jun 19, 2026Yang Yue, Xuancheng Zhu, Yuyang Ma +7Multi-Agent LLM SystemsAdversarial Robustness

    Beijing University of Posts and Telecommunications · China Telecom · Guangxi Transportation Science and Technology Group Co., Ltd. +1

  29. Measuring Biological Capabilities and Risks of AI Agents

    Jun 18, 2026Patricia Paskov, Jeffrey Lee, Kyle Brady +1AI for ScienceAI Agent Evaluation

    RAND Corporation, Santa Monica, Calif.

  30. Agentra: A Supervisable Multi-Agent Framework for Enterprise Intrusion Response

    Jun 16, 2026Raj Patel, Shaswata Mitra, Michele Guida +3Autonomous Cyber DefenseAI Agent Safety

    The University of Alabama, Alabama, USA · Roma Tre University, Rome, Italy

  31. Your AI Travel Agent Would Book You a Bullfight: An Agentic Benchmark for Implicit Animal Welfare in Frontier AI Models

    Jun 16, 2026Jasmine Brazilek, Joel Christoph, Maheep Chaudhary +4AI Agent EvaluationAI Agent Safety

    CaML · Harvard Kennedy School · Sentient Futures +1

  32. When Agent Automation Becomes Profitable: Quantifying and Insuring Autonomous AI Risk through Trace-Economic Underwriting

    Jun 15, 2026Binyan Xu, Xilin Dai, Fan Yang +1AI Risk ManagementAI Agent Safety

    The Chinese University of Hong Kong, Hong Kong, Hong Kong · Zhejiang University, Hangzhou, China

  33. An Evaluation of Data Leakage Risks in Tool-Using LLM Agents in Realistic Scenarios

    Jun 15, 2026Hankyul Baek, Jaewon Noh, Sang Seo +5Data LeakageLLM Agent Evaluation

    Korea AI Safety Institute · Singapore AI Safety Institute

  34. Gaming-Resistant Insurance Contracts for Autonomous AI Agents: Strategy-Proof Toll Mechanism Design

    Jun 15, 2026Hao-Hsuan ChenMechanism DesignAI Risk Management

  35. The Perils of Agency: How Developers Perceive, Prioritize, and Address Risks in Agentic AI Products

    Jun 13, 2026Hao-Ping Lee, Jessica He, David Piorkowski +3AI Risk ManagementAI Agent Safety

    Carnegie Mellon University · IBM Research · Alinia

  36. Defending against Adaptive Prompt Injection Attacks via Reasoning-enabled Task Alignment

    Jun 13, 2026Lipeng He, Yihan Wang, Jiawen Zhang +1Adversarial TrainingAI Agent Safety

    University of Waterloo Waterloo, Canada · Zhejiang University Hangzhou, China · KTH Royal Institute of Technology Stockholm, Sweden

  37. ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures

    Jun 13, 2026Kenneth Ge, Andre AssisAgent Failure AnalysisAI Coding Agents

    Anthropic Fellows Program · Constellation

  38. OSGuard: A Benchmark for Safety in Computer-Use Agents

    Jun 13, 2026Mina Mohammadmirzaei, Jeffrey FlaniganComputer-Use AgentsLLM Guardrails

    University of California, Santa Cruz

  39. Selective Agentic Recovery for UAV Autonomy with a Persistent Mission Runtime

    Jun 12, 2026Taewoo Park, Kyeonghyun Yoo, Seunghyun Yoo +1UAV NavigationRobot Failure Recovery

    Department of Electrical and Electronic Engineering Korea University

  40. Hidden in Plain Sight: Benchmarking Agent Safety Against Decomposition Attacks with DECOMPBENCH

    Jun 12, 2026Vikhyath Kothamasu, Virginia Smith, Chhavi YadavAI Agent SafetyAI Agent Safety Benchmarks

    Carnegie Mellon University · Simons Institute, UC Berkeley

  41. A Virtuous AI is an Existential Risk

    Jun 11, 2026Guillermo Del Pinal, Youngchan Lee, Min OhnLLM Safety AlignmentAI Agent Safety

    University of Massachusetts Amherst

  42. The Containment Gap: How Deployed Agentic AI Frameworks Fail Public-Facing Safety Requirements

    Jun 11, 2026Md Jafrin Hossain, Mohammad Arif Hossain, Weiqi Liu +1AI Agent SecurityAI Agent Safety

    Department of Electrical and Computer Engineering, Florida International University, Miami, FL, USA · Department of Electrical and Computer Engineering, New Jersey Institute of Technology, Newark, NJ, USA

  43. The Impossibility of Eliciting Latent Knowledge

    Jun 10, 2026Korbinian Friedl, Francis Rhys Ward, Paul Yushin Rapoport +2AI Agent SafetyLLM Honesty

    The London School of Economics and Political Science · Independent

  44. Towards Responsibly Non-Compliant Machines

    Jun 10, 2026Marija Slavkovik, Marie Farrell, Louise Dennis +3AI AccountabilityAI Agent Safety

    University of Manchester, Manchester, United Kingdom · University of Bergen Bergen, Norway · University of Manchester Manchester, United Kingdom

  45. Bootstrapped Monitoring: Leveraging Transparent Reasoning to Oversee Stronger AI Agents

    Jun 10, 2026Frank Xiao, Mary PhuongAI Agent SafetyAI Agent Monitoring

    California Institute of Technology

  46. SafeGEO: Understanding Generative Engine Optimization Risks in Recommendation Agents

    Jun 8, 2026Qianfeng Wen, Yifan Simon Liu, Xin Liu +4Generative Engine OptimizationLLM Agent Security

    Department of Computer Science, University of Toronto · Department of Mechanical & Industrial Engineering, University of Toronto · Faculty of Information, University of Toronto +3

  47. Revisiting the shutdown problem

    Jun 6, 2026David ThorstadAI Agent SafetyAI Safety

  48. Beyond Goodhart's Law: A Dynamic Benchmark for Evaluating Compliance in Multi-Agent Systems

    Jun 5, 2026Yiyang Zhao, Zhuo Zhang, Qingxuan Le +2Multi-Agent LLM SystemsAI Agent Safety

    Fudan University Shanghai, China · Shanghai Academy of AI for Science Shanghai, China · Shanghai Artificial Intelligence Laboratory Shanghai, China +1

  49. From Reward-Hack Activations to Agentic Risk States: Context-Calibrated Mechanistic Monitoring in LLM Agents

    Jun 4, 2026Patrick Wilhelm, Odej KaoReward HackingAI Agent Safety

    Distributed Operating Systems, Technische Universität Berlin, Berlin, Germany · BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany

  50. PreAct-Bench: Benchmarking Predictive Monitoring in LLMs

    Jun 3, 2026Hainiu Xu, Italo Luis da Silva, Jiangnan Ye +7LLM Safety BenchmarksMoral Reasoning in Language Models

    King’s College London · National University of Singapore · Southern University of Science and Technology +3

  51. Attack Selection in Agentic AI Control Evaluations Meaningfully Decreases Safety

    Jun 3, 2026Catherine Ge-Wang, Tyler Crosse, Benjamin Hadad +3Adversarial AttacksAI Agent Safety

    University of Oxford · Mentorship for Alignment Research Students (MARS), CAISH · Georgia Institute of Technology +3

  52. Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming

    Jun 3, 2026Nicholas SabanComputer-Use AgentsAI Agent Safety

    Patronus AI · University of California, Berkeley

  53. The Digital Apprentice: A Framework for Human-Directed Agentic AI Development

    Jun 3, 2026Travis Weber, Rohit TanejaAI Agent SafetyHuman-in-the-Loop AI

    Pheo Inc

  54. PerceptTwin: Semantic Scene Reconstruction for Iterative LLM Planning and Verification

    Jun 2, 2026Charlie Gauthier, Sacha Morin, Liam PaullLarge Language Model-Based Robot PlanningRobotics Simulation

    Department of Computer Science and Operations Research, Universit´e de Montr´eal, Montr´eal, QC, Canada. · Mila - Quebec AI Institute, Montr´eal, QC, Canada. · CIFAR AI Chair.

  55. Agentic Safety is an Epistemic Property, Not a Behavioral One

    Jun 2, 2026Charles L. Wang, Keir Dorchen, Peter JinAI AlignmentAI Agent Safety

    1Columbia University, New York, NY, USA · 2Pelagic Platforms

  56. VLESA: Vision-Language Embodied Safety Agent for Human Activity Monitoring

    Jun 2, 2026Hanjiang Hu, Yiyuan Pan, Jiaxing Li +5Egocentric Video UnderstandingAI Agent Safety

    Carnegie Mellon University · Harvard University · Mitsubishi Electric Research Laboratories

  57. LAP: An Agent-to-Instrument Protocol for Autonomous Science

    Jun 2, 2026Linwu Zhu, Liqiang Gao, Yan Chen +2AI Agents for Scientific DiscoveryAgent Communication Protocols

    Shiyanjia Lab

  58. RUBAS: Rubric-Based Reinforcement Learning for Agent Safety

    Jun 2, 2026Xian Qi Loye, Qinglin Su, Zhexin Zhang +5Reinforcement LearningAI Agent Safety

    The Conversational AI (CoAI) group, DCST, Tsinghua University · Huawei Noah’s Ark Lab

  59. Solipsistic Superintelligence is Unlikely to be Cooperative

    Jun 2, 2026Rakshit S Trivedi, Natasha Jaques, Logan Cross +2AI AlignmentMulti-Agent Collaboration

    Independent Researcher, CA, USA · University of Washington, Seattle, WA, USA · Google Deepmind, London, UK.

  60. Glass Box at Orbit: A Constitutional AI Verification Framework for Trustworthy Autonomous CubeSat Intelligence

    Jun 2, 2026Karthik Barma, Anil Sanneboyina, V C Premchand YadavAI AlignmentAI Agent Safety

    Khoury College of Computer Sciences · Northeastern University · Boston, MA 02115, USA +3