LLM Agent Security

LLM: Large Language Model

The field examines how agents can be manipulated through indirect prompt injection, malicious tools or skills, poisoned memory, excessive permissions, and multi-agent interactions, sometimes leading to data leaks or harmful actions. Researchers evaluate these risks with red-team attacks and benchmarks that track agent trajectories, tool calls, and real effects—not just final answers. Defenses include runtime authorization and least privilege, sandboxing, provenance and information-flow controls, monitoring, and security-focused training. A central challenge is protecting agents over long, changing workflows without blocking legitimate tasks, while making evaluations realistic enough to capture adaptive attacks and deployment risks.

Latest papers

332 papers44 papers in the last four weeks, up 33% on the four weeks before. 0.3% of all new papers.

Don't Trust Stubborn Neighbors: A Security Framework for Agentic Networks
Mar 16, 2026Samira Abedini, Sina Mavali, Lea Schönherr +2Multi-Agent LLM Systems LLM Agent Security
CISPA Helmholtz Center for Information Security · University of Vienna, Faculty of Computer Science
AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystems
Mar 16, 2026Yihao Zhang, Zeming Wei, Xiaokun Luan +7LLM Agent Security Multi-Agent System Security
Peking University · Sun Yat-sen University · Wuhan University +2
ToolFlood: Beyond Selection -- Hiding Valid Tools from LLM Agents via Semantic Covering
Mar 14, 2026Hussein Jawad, Nicolas J-B BrunelLLM Agent Security Adversarial Attacks on LLMs
Capgemini Invent, Paris, France · LaMME, University Paris-Saclay, Evry, France · ENSIIE, Evry, France
GroupGuard: A Framework for Modeling and Defending Collusive Attacks in Multi-Agent Systems
Mar 14, 2026Yiling Tao, Xinran Zheng, Shuo Yang +2Graph-Based Anomaly Detection LLM Agent Security
Shenzhen International Graduate School, Tsinghua University · Department of Electrical and Electronic Engineering, The University of Hong Kong · School of Computer Science and Engineering, University of Electronic Science and Technology of China
Security Considerations for Artificial Intelligence Agents
Mar 12, 2026Ninghui Li, Kaiyuan Zhang, Kyle Polley +1AI Agent Security LLM Agent Security
Perplexity · Purdue University
You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents
Mar 12, 2026Ching-Yu Kao, Xinfeng Li, Shenyu Dai +4LLM Agent Security Prompt Injection Attacks on AI Agents
Fraunhofer AISEC · NTU · KTH +2
Governing Evolving Memory in LLM Agents: Risks, Mechanisms, and the Stability and Safety Governed Memory (SSGM) Framework
Mar 12, 2026Chingkwun Lam, Jiaxin Li, Lingfei Zhang +1LLM Agent Memory Agent Memory Management
College of Intelligent Science and Engineering, Jinan University
Taming OpenClaw: Security Analysis and Mitigation of Autonomous LLM Agent Threats
Mar 12, 2026Xinhao Deng, Yixiang Zhang, Jiaqing Wu +15AI Agent Security LLM Agent Security
Ant Group & Tsinghua University, China · Tsinghua University, China · Ant Group, China
The Attack and Defense Landscape of Agentic AI: A Comprehensive Survey
Mar 11, 2026Juhee Kim, Xiaoyuan Liu, Zhun Wang +4AI Agent Security LLM Agent Security
UC Berkeley / Seoul National University, USA · UC Berkeley, USA · University of Illinois Urbana-Champaign, USA +1
DynaTrust: Defending Multi-Agent Systems Against Sleeper Agents via Dynamic Trust Graphs
Mar 9, 2026Yu Li, Qiang Hu, Yao Zhang +3AI Agent Security LLM Agent Security
Tianjin University · Singapore Management University
From Thinker to Society: Security in Hierarchical Autonomy Evolution of AI Agents
Mar 8, 2026Xiaolei Zhang, Lu Zhou, Xiaogang Xu +5AI Agent Security LLM Agent Security
Nanjing University of Aeronautics and Astronautics, China · Collaborative Innovation Center of Novel Software Technology and Industrialization, China · The Chinese University of Hong Kong, China +6
Governance Architecture for Autonomous Agent Systems: Threats, Framework, and Engineering Practice
Mar 7, 2026Yuxu GeLLM Agent Security AI Agent Governance
University of York, Department of Computer Science, York, United Kingdom
Cross-Layer Semantic Flow Reconstruction for Attack Detection in Agentic Systems
Mar 4, 2026Qizhi Cai, Yangyang Wei, Zhipeng Chen +2AI Agent Security LLM Agent Security
Department of Software Technology, Zhejiang University, Ningbo, China · Department of Computer Science and Technology, Zhejiang University, Hangzhou, China
Contextualized Privacy Defense for LLM Agents
Mar 3, 2026Yule Wen, Yanzhe Zhang, Jianxun Lian +3Privacy Protection for LLM Agents LLM Agent Security
Tsinghua University · Georgia Tech · Microsoft +1
Quantifying Frontier LLM Capabilities for Container Sandbox Escape
Mar 1, 2026Rahul Marchand, Art O Cathain, Jerome Wynne +8LLM Safety Benchmarks LLM Agent Security
University of Oxford · UK AI Security Institute
Verifier-Bound Communication for LLM Agents: Certified Bounds on Covert Signaling
Feb 27, 2026Om TailorLLM Agent Security Privacy Leakage in Language Models
University of Maryland
Silent Egress: When Implicit Prompt Injection Makes LLM Agents Leak Without a Trace
Feb 25, 2026Qianlong Lan, Anuj Kaul, Shaun Jones +1LLM Agent Security Prompt Injection Attacks on AI Agents
eBay
SoK: Agentic Skills -- Beyond Tool Use in LLM Agents
Feb 24, 2026Yanna Jiang, Delong Li, Haiyu Deng +4Agentic AI LLM Agent Skill Learning
University of Technology Sydney · CSIRO Data61
Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
Feb 23, 2026David Schmotz, Luca Beurer-Kellner, Sahar Abdelnabi +1LLM Agent Security Prompt Injection Attacks on AI Agents
Max Planck Institute for Intelligent Systems, ELLIS Institute Tübingen, Tübingen AI Center · Snyk, Switzerland
Agents of Chaos
Feb 23, 2026Natalie Shapira, Chris Wendler, Avery Yen +35AI Agent Security LLM Agent Security
Northeastern University · Independent Researcher · Stanford University +10
Watermarking LLM Agent Trajectories
Feb 21, 2026Wenlong Meng, Chen Gong, Terry Yue Zhuo +6LLM Watermarking LLM Agent Security
Zhejiang University, China · University of Virginia, USA · Alibaba Qwen, China +1
AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks
Feb 18, 2026Tanqiu Jiang, Yuhui Wang, Jiacheng Liang +1LLM Agent Security AI Agent Security Benchmarks
Department of Computer Science, Stony Brook University, Stony Brook, USA.
Overthinking Loops in Agents: A Structural Risk via MCP Tools
Feb 16, 2026Yohan Lee, Jisoo Jang, Seoyeon Choi +2LLM Agent Security Tool-Using Agents
Yonsei University · Division of Language and AI, Hankuk University of Foreign Studies (HUFS) · Department of Data Science, Ewha Womans University
From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching
Jan 30, 2026Zhixiang Zhang, Zesen Liu, Yuchong Xie +2Adversarial Attacks LLM Agent Security
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology · Fudan University
Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs
Jan 29, 2026Xiang Zheng, Yutao Wu, Hanxun Huang +5Prompt Injection AI Coding Agents
City University of Hong Kong · Deakin University · The University of Melbourne +3
AgenTRIM: Tool Risk Mitigation for Agentic AI
Jan 18, 2026Roy Betser, Amit Giloni, Shamik Bose +4LLM Agent Security Indirect Prompt Injection
Fujitsu Research of Europe (FRE) · Fujitsu Research of India Pvt. Ltd. (FRIPL)
Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation
Dec 18, 2025Yuxuan Qiao, Dongqin Liu, Hongchang Yang +2LLM Agent Security LLM Agents
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences
MAS-Shield: A Defense Framework for Secure and Efficient LLM MAS
Nov 28, 2025Kaixiang Wang, Zhaojiacheng Zhou, Bunyod Suvonov +8Multi-Agent LLM Systems LLM Auditing
Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai 200240, China
Breaking the Code: Security Assessment of AI Code Agents Through Systematic Jailbreaking Attacks
Oct 1, 2025Shoumik Saha, Jifan Chen, Sam Mayers +3AI Coding Agents AI Agent Security
University of Maryland – College Park · AWS AI Labs · Meta Superintelligence Labs
Red-Teaming Coding Agents from a Tool-Invocation Perspective: An Empirical Security Assessment
Sep 6, 2025Yuchong Xie, Mingyu Luo, Zesen Liu +7Prompt Injection AI Coding Agents
The Hong Kong University of Science and Technology, China · Fudan University, China

Mar 16, 2026J/K move · Enter open · S save

Don't Trust Stubborn Neighbors: A Security Framework for Agentic Networks

Samira Abedini, Sina Mavali, Lea Schönherr, Martin Pawelczyk+1

CISPA Helmholtz Center for Information Security · University of Vienna, Faculty of Computer Science

Multi-Agent LLM Systems LLM Agent Security Multi-Agent System Security

Open paper AskPDF arXiv

Large Language Model (LLM)-based Multi-Agent Systems (MASs) are increasingly deployed for agentic tasks, such as web automation, itinerary planning, and collaborative problem solving. Yet, their interactive nature introduces new security risks: malicious or compromised agents can exploit communication channels to propagate misinformation and manipulate collective outcomes. In this paper, we study how such manipulation can arise and spread by borrowing the Friedkin-Johnsen opinion formation model from social sciences to propose a general theoretical framework to study LLM-MAS. Remarkably, this model closely captures LLM-MAS behavior, as we verify in extensive experiments across different network topologies and attack and defense scenarios. Theoretically and empirically, we find that a single highly stubborn and persuasive agent can take over MAS dynamics, underscoring the systems' high susceptibility to attacks by triggering a persuasion cascade that reshapes collective opinion. Our theoretical analysis reveals three mechanisms to increase system security: a) increasing the number of benign agents, b) increasing the innate stubbornness or peer-resistance of agents, or c) reducing trust in potential adversaries. Because scaling is computationally expensive and high stubbornness degrades the network's ability to reach consensus, we propose a new mechanism to mitigate threats by a trust-adaptive defense that dynamically adjusts inter-agent trust to limit adversarial influence while maintaining cooperative performance. Extensive experiments confirm that this mechanism effectively defends against manipulation.