AI Agent Security

Follow this topic Ask about this topic

AI agent security studies how tool use, memory, delegation, and interaction with people or environments create risks beyond those of a standalone language model, including prompt injection, privilege misuse, data leakage, and unsafe actions. Researchers test these risks with red-teaming and execution-grounded benchmarks for coding, browsing, GUI, embodied, and multi-agent tasks, measuring whether harmful effects occur across trajectories rather than judging text alone. Defenses range from input and memory provenance checks to sandboxing, least-privilege permissions, stateful runtime authorization, action verification, and cryptographic identity or audit mechanisms. An open challenge is making controls robust to adaptive, long-horizon attacks and interactions between tools or agents while preserving useful task completion; benchmark realism and comparable measures remain uneven.

Latest papers

517 papers74 papers in the last four weeks, up 32% on the four weeks before. 0.4% of all new papers.

UNSEEN: A Cross-Stack LLM Unlearning Defense against AR-LLM Social Engineering Attacks
Apr 25, 2026Tianlong Yu, Yang Yang, Xiao Luo +6Privacy-Preserving Language Models LLM Guardrails
Hubei University, Wuhan, China · University of Southern California, Los Angeles, USA · Huazhong University of Science and Technology, Wuhan, China
Operationalizing Reconstructive Authority: Runtime Construction, Dependency Resolution, and Execution Gating in Autonomous Agent Systems
Apr 24, 2026Marcelo Fernandez - TraslaIAAI Agent Security AI Agent Safety
TraslaIA
Cross-Session Threats in AI Agents: Benchmark, Evaluation, and Algorithms
Apr 22, 2026Ari AzarafroozCybersecurity AI Agent Security
Intrinsec AI
Breaking MCP with Function Hijacking Attacks: Novel Threats for Function Calling and Agentic Models
Apr 22, 2026Yannis Belkhiter, Giulio Zizzo, Sergio Maffeis +2Adversarial Attacks on LLMs AI Agent Security
IBM Research Europe · Trinity College Dublin · Imperial College London +1
AgentSOC: A Multi-Layer Agentic AI Framework for Security Operations Automation
Apr 22, 2026Joyjit Roy, Samaresh Kumar SinghAutonomous Cyber Defense Cybersecurity
Austin, Texas, USA · Leander, Texas, USA
An AI Agent Execution Environment to Safeguard User Data
Apr 21, 2026Robert Stanley, Avi Verma, Lillian Tsai +2AI Agent Security Privacy Leakage in Language Models
University of California, Los Angeles · Google
If you're waiting for a sign... that might not be it! Mitigating Trust Boundary Confusion from Visual Injections on Vision-Language Agentic Systems
Apr 21, 2026Jiamin Chang, Minhui Xue, Ruoxi Sun +3VLM Robustness Adversarial Attacks on VLMs
University of New South Wales, Sydney, NSW 2052, Australia · CSIRO’s Data61, Marsfield, NSW 2122, Australia · Macquarie University, NSW 2109, Australia
How Adversarial Environments Mislead Agentic AI?
Apr 20, 2026Zhonghao Zhan, Huichi Zhou, Zhenhao Li +3Adversarial Robustness AI Agent Evaluation
Imperial College London
Temporal UI State Inconsistency in Desktop GUI Agents: Formalizing and Defending Against TOCTOU Attacks on Computer-Use Agents
Apr 20, 2026Wenpeng XuAdversarial Attacks Computer-Use Agents
University of California, San Diego
Owner-Harm: A Missing Threat Model for AI Agent Safety
Apr 20, 2026Dongcheng Zhang, Yiqing JiangAI Agent Security AI Agent Safety
BlueFocus Communication Group · Tongji University
From Craft to Kernel: A Governance-First Execution Architecture and Semantic ISA for Agentic Computers
Apr 20, 2026Xiangyu Wen, Yuang Zhao, Xiaoyu Xu +9AI Agent Security AI Agent Safety
CUHK · Hong Kong SAR, China · Shanghai Jiao Tong University +6
SafeAgent: A Runtime Protection Architecture for Agentic Systems
Apr 19, 2026Hailin Liu, Eugene Ilyushin, Jie Ni +1Adversarial Attacks on LLMs AI Agent Security
Lomonosov Moscow State University · Central University
Visual Inception: Compromising Long-term Planning in Agentic Recommenders via Multimodal Memory Poisoning
Apr 18, 2026Jiachen QianAI Agent Security Agent Memory Poisoning
City University of Hong Kong
Governed MCP: Kernel-Level Tool Governance for AI Agents via Logit-Based Safety Primitives
Apr 18, 2026Daeyeon SonMCP Security AI Agent Security
Independent Researcher · Republic of Korea
enclawed: A Configurable, Sector-Neutral Hardening Framework for Single-User AI Assistant Gateways
Apr 18, 2026Alfredo MetereAI Assurance AI Agent Security
Metere Consulting, LLC.
CapSeal: Capability-Sealed Secret Mediation for Secure Agent Execution
Apr 18, 2026Shutong Jin, Ruiyi Guo, Ray C. C. CheungAI Agent Security LLM Agent Security
City University of Hong Kong, Hong Kong · Beijing Foreign Studies University
Don't Make Models Guess Security and Safety: Symbolic Guardrails for Domain-Specific AI Agents
Apr 16, 2026Yining Hong, Yining She, Eunsuk Kang +2LLM Guardrails AI Agent Security
Carnegie Mellon University
Bounded Autonomy for Enterprise AI: Typed Action Contracts and Consumer-Side Execution
Apr 16, 2026Sarmad Sohail, Ghufran HaiderLLM Guardrails AI Agent Security
LinuxArena: A Control Setting for AI Agents in Live Production Software Environments
Apr 16, 2026Tyler Tracy, Ram Potham, Nick Kuhn +31Software Engineering Agents AI Agent Security
Redwood Research · EquiStamp
Challenges and Future Directions in Agentic Reverse Engineering Systems
Apr 15, 2026Salem Radey, Jack West, Kassem FawazSoftware Reverse Engineering LLM Agent Security
University of Wisconsin-Madison, Madison, USA
HadAgent: Harness-Aware Decentralized Agentic AI Serving with Proof-of-Inference Blockchain Consensus
Apr 15, 2026Landy Jimenez, Mariah Weatherspoon, Bingyu Shen +3LLM Agent Harnesses AI Agent Security
Department of Computer Science and Technology, Kean University, Union, New Jersey, U.S. · Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, Indiana, U.S. · Bellini College of Artificial Intelligence, Cybersecurity and Computing, University of South Florida, Tampa, Florida, U.S. +1
MCPThreatHive: Automated Threat Intelligence for Model Context Protocol Ecosystems
Apr 15, 2026Yi Ting Shen, Kentaroh Toyoda, Alex LeungMCP Security AI Agent Security
Vulcan Research, AIFT
SciFi: A Safe, Lightweight, User-Friendly, and Fully Autonomous Agentic AI Workflow for Scientific Applications
Apr 14, 2026Qibin Liu, Julia GonskiAgentic Workflows Scientific Workflow Automation
SLAC National Accelerator Laboratory, 2575 Sand Hill Rd, Menlo Park CA 94025 USA
Parallax: Why AI Agents That Think Must Never Act
Apr 14, 2026Joel FokouAI Agent Security Runtime Enforcement for AI Agents
Independent Researcher
ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injection
Apr 13, 2026Wei Zhao, Zhe Li, Peixin Zhang +1AI Agent Security LLM Agent Security
Singapore Management University
Context Kubernetes: Declarative Orchestration of Enterprise Knowledge for Agentic AI Systems
Apr 13, 2026Charafeddine MouzouniAI Agent Governance LLM Context Management
OPIT – Open Institute of Technology, and Cohorte AI, Paris, France.
Hardening x402: PII-Safe Agentic Payments via Pre-Execution Metadata Filtering
Apr 13, 2026Vladimir StantchevAI Agent Security Safety Filtering
SRH University Heidelberg, Germany · PRESIDIO Group, Sofia, Bulgaria
Hodoscope: Unsupervised Monitoring for AI Misbehaviors
Apr 13, 2026Ziqian Zhong, Shashwat Saxena, Aditi RaghunathanAI Agent Monitoring AI Agent Security
Carnegie Mellon University
BadSkill: Backdoor Attacks on Agent Skills via Model-in-Skill Poisoning
Apr 10, 2026Guiyao Tie, Jiawen Shi, Pan Zhou +1Backdoor Attacks AI Agent Security
Huazhong University of Science and Technology · Lehigh University
Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection
Apr 9, 2026Wenkui Yang, Chao Jin, Haisu Zhu +7GUI Agents AI Agent Security
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · 2MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences · 4ShanghaiTech University +2

Apr 25, 2026J/K move · Enter open · S save

UNSEEN: A Cross-Stack LLM Unlearning Defense against AR-LLM Social Engineering Attacks

Tianlong Yu, Yang Yang, Xiao Luo, Lihong Liu+5

Hubei University, Wuhan, China · University of Southern California, Los Angeles, USA · Huazhong University of Science and Technology, Wuhan, China

Privacy-Preserving Language Models LLM Guardrails Cybersecurity AI Agent Security

Open paper AskPDF arXiv

Emerging AR-LLM-based Social Engineering attack (e.g., SEAR) is at the edge of posing great threats to real-world social life. In such AR-LLM-SE attack, the attacker can leverage AR (Augmented Reality) glass to capture the image and vocal information of the target, using the LLM to identify the target and generate the social profile, using the LLM agents to apply social engineering strategies for conversation suggestion to win the target trust and perform phishing afterwards. Current defensive approaches, such as role-based access control or data flow tracking, are not directly applicable to the convergent AR-LLM ecosystem (considering embedded AR device and opaque LLM inference), leaving an emerging and potent social engineering threat that existing privacy paradigms are ill-equipped to address. This necessitates a shift beyond solely human-centric measures like legislation and user education toward enforceable vendor policies and platform-level restrictions. Realizing this vision, however, faces significant technical challenges: securing resource-constrained AR-embedded devices, implementing fine-grained access control within opaque LLM inferences, and governing adaptive interactive agents. To address these challenges, we present UNSEEN, a coordinated cross-stack defense that combines an AR ACL (Access Control Layer) for identity-gated sensing, F-RMU-based LLM unlearning for sensitive profile suppression, and runtime agent guardrails for adaptive interaction control. We evaluate UNSEEN in an IRB-approved user study with 60 participants and a dataset of 360 annotated conversations across realistic social scenarios.