Multi-Agent System Security

Follow this topic Ask about this topic

Multi-agent system security examines how agents can attack, mislead, or compromise one another, and how local failures can spread through communication, shared memory, delegated tools, or coordinated decisions. Work studies threats such as prompt injection, collusion, Byzantine behavior, memory poisoning, information leakage, and cascading errors, using settings from language-model workflows and debate to multi-agent reinforcement learning and robot swarms. Defenses include trust and reputation systems, provenance and authorization controls, runtime monitoring, semantic information-flow checks, and cryptographic protocols; benchmarks increasingly test different architectures, interaction paths, and long-horizon tasks. A central challenge is that safeguards effective for individual agents may not compose at system level, while stronger controls can reduce task performance or miss attacks that emerge only through multi-agent interaction.

Latest papers

181 papers29 papers in the last four weeks, up 12% on the four weeks before. 0.2% of all new papers.

Auditing Agent Harness Safety
May 14, 2026Chengzhi Liu, Yichen Guo, Yepeng Liu +8AI Agent Auditing LLM Auditing
University of California, Santa Barbara · University of California, Berkeley · Microsoft Research +2
Hierarchical Attacks for Multi-Modal Multi-Agent Reasoning
May 13, 2026Hao Zhou, Tiru Wu, Yan Jiang +3Multimodal Robustness Adversarial Attacks
Finding the Weakest Link: Adversarial Attack against Multi-Agent Communications
May 13, 2026Maxwell Standen, Junae Kim, Claudia SzaboAdversarial Attacks Adversarial Perturbations
The University of Adelaide · DST Group
Moderation by Interrogation: Uncovering Concealed Intent in Agent Social Networks
May 13, 2026Ali Al-Lawati, Nafis Tripto, Abolfazl Ansari +4Markov Chain Monte Carlo Intent Classification
The Pennsylvania State University, USA
The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems
May 12, 2026Tanzim Ahad, Ismail Hossain, Md Jahangir Alam +3AI Agent Security Benchmarks Agent Memory Poisoning
Department of Computer Science, University of Texas at El Paso, TX, USA 79902 · School of Computing, Southern Illinois University Carbondale, IL, USA 62901 · University of Illinois Urbana-Champaign, IL, USA
Control Charts for Multi-agent Systems
May 11, 2026Hayden Helm, Carey Priebe, Brandon DuderstadtMulti-Agent Systems Multi-Agent System Security
Helivan · Johns Hopkins University · Calcifer Computing
Agent Collectives Should Not Detect Their Own Imposters: A Chess Case Study
May 9, 2026Alexandre Le Mercier, Chris Develder, Thomas DemeesterAI Agent Security AI Agent Security Benchmarks
IDLab–T2K, Ghent University–imec
When Child Inherits: Modeling and Exploiting Subagent Spawn in Multi-Agent Networks
May 8, 2026Ziwen Cai, Yihe Zhang, Xiali HeiLLM Agent Security Multi-Agent System Security
University of Louisiana at Lafayette Lafayette, USA
PropGuard: Safeguarding LLM-MAS via Propagation-Aware Exploration and Remediation
May 8, 2026Bingyu Yan, Xiaoming Zhang, Jinyu Hou +4LLM Agent Security Prompt Injection Attacks on AI Agents
Beihang University · Beijing University of Posts and Telecommunications · City University of Hong Kong
Insider Attacks in Multi-Agent LLM Consensus Systems
May 8, 2026Xiaolin Sun, Zixuan Liu, Yibin Hu +1Multi-Agent LLM Systems Multi-Agent Systems
Department of Computer Science, Tulane University, New Orleans, United States of America.
MAGIQ: A Post-Quantum Multi-Agentic AI Governance System with Provable Security
May 7, 2026Sepideh Avizheh, Tushin Mallick, Alina Oprea +2AI Agent Governance Access Control
University of Calgary Calgary, Alberta, Canada · Northeastern University Boston, MA, USA
Towards Security-Auditable LLM Agents: A Unified Graph Representation
May 7, 2026Chaofan Li, Lyuye Zhang, Jintao Zhai +9LLM Agent Security Multi-Agent System Security
Huazhong University of Science and Technology, China · Nanyang Technological University, Singapore, Singapore · Chongqing University of Posts and Telecommunications, China +2
Authorization Propagation in Multi-Agent AI Systems: Identity Governance as Infrastructure
May 6, 2026Krti TallamAI Agent Governance Access Control
Kamiwaza AI
When Agents Handle Secrets: A Survey of Confidential Computing for Agentic AI
May 4, 2026Javad Forough, Marios Kogias, Hamed HaddadiCybersecurity Remote Attestation
Department of Computing Imperial College London London, United Kingdom
Catching the Infection Before It Spreads: Foresight-Guided Defense in Multi-Agent Systems
May 3, 2026Yue Ma, Ziyuan Yang, Yi ZhangMulti-Agent LLM Systems Jailbreak Defense
Sichuan University · Nanyang Technological University
When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems
May 1, 2026Lingxi Zhang, Guangtao Zheng, Hanjie ChenAdversarial Attacks Language Model Robustness
1Rice University · University of Virginia
Ambient Persuasion in a Deployed AI Agent: Unauthorized Escalation Following Routine Non-Adversarial Content Exposure
Apr 29, 2026Diego F. Cuadros, Abdoul-Aziz MaigaAgent Failure Analysis AI Agent Security
Digital Epidemiology Laboratory, Digital Futures, University of Cincinnati, Cincinnati, OH, USA · Center for Humanities and Technology (CHaT), University of Cincinnati, Cincinnati, OH, USA · Norwegian University of Science and Technology, Trondheim, Norway
Agent Name Service (ANS): A Proof-of-Concept Trust Layer for Secure AI Agent Discovery, Identity, and Governance in Kubernetes
Apr 29, 2026Akshay Mittal, Elyson De La CruzAI Agent Security AI Agent Governance
Department of Information Technology University of the Cumberlands Williamsburg, KY, USA
GAMMAF: A Common Framework for Graph-Based Anomaly Monitoring Benchmarking in LLM Multi-Agent Systems
Apr 27, 2026Pablo Mateo-Torrejón, Alfonso Sánchez-MaciánMulti-Agent LLM Systems Graph-Based Anomaly Detection
University Carlos III of Madrid · Leganés, Spain
Breaking the Secret: Economic Interventions for Combating Collusion in Embodied Multi-Agent Systems
Apr 26, 2026Qi Liu, Xiaohui Chen, Zhihui Zhao +5Mechanism Design Multi-Agent Systems
College of Computer Science and Technology, Taiyuan University of Technology, Taiyuan, China, 030024 · State Key Laboratory of Internet Architecture, Tsinghua University, Beijing, China, 100084 · China Mobile Research Institute, Beijing, China, 100053 +3
Architecture Matters for Multi-Agent Security
Apr 25, 2026Ben Hagag, William L. Anderson, Christian Schroeder de Witt +1Multi-Agent Systems Multi-Agent System Security
Carnegie Mellon University · MATS · Oxford Witt Lab +1
A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
Apr 25, 2026Kexin ChuAI Agent Security LLM Agent Security
School of Computing, University of Connecticut
Beyond Single-Agent Alignment: Preventing Context-Fragmented Violations in Multi-Agent Systems
Apr 24, 2026Jie Wu, Ming GongCybersecurity AI Agent Security Benchmarks
Atlassian
Conjunctive Prompt Attacks in Multi-Agent LLM Systems
Apr 17, 2026Nokimul Hasan Arif, Qian Lou, Mengxin ZhengMulti-Agent LLM Systems Adversarial Attacks on LLMs
University of Central Florida, USA
CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systems
Apr 14, 2026Yongxuan Wu, Xixun Lin, He Zhang +5Multi-Agent System Security Multi-Agent LLM Systems
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Griffith University, Brisbane, Australia +2
The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agents
Apr 12, 2026Xuwei Ding, Skylar Zhai, Linxin Song +6Computer-Use Agent Safety AI Agent Safety
University of Wisconsin–Madison · University of Minnesota · University of Southern California +3
ACIArena: Toward Unified Evaluation for Agent Cascading Injection
Apr 9, 2026Hengyu An, Minxi Li, Jinghuai Zhang +6Multi-Agent System Security AI Agent Security Benchmarks
Zhejiang University · State Key Laboratory of Internet Architecture, Tsinghua University · University of California, Los Angeles +2
Semantic Intent Fragmentation: A Single-Shot Compositional Attack on Multi-Agent AI Pipelines
Apr 8, 2026Tanzim Ahad, Ismail Hossain, Md Jahangir Alam +4LLM Agent Security LLM Agent Safety
Department of Computer Science, University of Texas at El Paso, TX, USA 79902 · School of Computing, Southern Illinois University Carbondale, IL, USA 62901 · Hanyang University, Seoul, South Korea +1
OpenKedge: Governing Agentic Mutation with Execution-Bound Safety and Evidence Chains
Apr 7, 2026Jun He, Deying YuAI Agent Governance Runtime Enforcement for AI Agents
HDP: A Lightweight Cryptographic Protocol for Human Delegation Provenance in Agentic AI Systems
Apr 6, 2026Asiri DalugodaAI Accountability Multi-Agent System Security
Helixar Limited, Auckland, New Zealand

May 14, 2026J/K move · Enter open · S save

Auditing Agent Harness Safety

Chengzhi Liu, Yichen Guo, Yepeng Liu, Yuzhe Yang+7

University of California, Santa Barbara · University of California, Berkeley · Microsoft Research+2

AI Agent Auditing LLM Auditing AI Agent Security Benchmarks LLM Agent Safety Multi-Agent System Security

Open paper AskPDF arXiv

LLM agents increasingly run inside execution harnesses that dispatch tools, allocate resources, and route messages between specialized components. However, a harness can return a correct, benign answer over a trajectory that accesses unauthorized resources or leaks context to the wrong agent. Output-level evaluation cannot see these failures, yet most safety benchmarks score only final outputs or terminal states, even though many violations occur mid-trajectory rather than at termination. The central question is whether the harness respects user intent, permission boundaries, and information-flow constraints throughout execution. To address this gap, we propose HarnessAudit, a framework that audits full execution trajectories across boundary compliance, execution fidelity, and system stability, with a focus on multi-agent harnesses where these risks are most pronounced. We further introduce HarnessAudit-Bench, a benchmark of 210 tasks across eight real-world domains, instantiated in both single-agent and multi-agent configurations with embedded safety constraints. Evaluating ten harness configurations across frontier models and three multi-agent frameworks, we find that: (i) task completion is misaligned with safe execution, and violations accumulate with trajectory length; (ii) safety risks vary across domains, task types, and agent roles; (iii) most violations concentrate in resource access and inter-agent information transfer; and (iv) multi-agent collaboration expands the safety risk surface, while harness design sets the upper bound of safe deployment.