Multi-Agent System Security

Ask about this topic

Overview

Multi-agent system security examines how agents can attack, mislead, or compromise one another, and how local failures can spread through communication, shared memory, delegated tools, or coordinated decisions. Work studies threats such as prompt injection, collusion, Byzantine behavior, memory poisoning, information leakage, and cascading errors, using settings from language-model workflows and debate to multi-agent reinforcement learning and robot swarms. Defenses include trust and reputation systems, provenance and authorization controls, runtime monitoring, semantic information-flow checks, and cryptographic protocols; benchmarks increasingly test different architectures, interaction paths, and long-horizon tasks. A central challenge is that safeguards effective for individual agents may not compose at system level, while stronger controls can reduce task performance or miss attacks that emerge only through multi-agent interaction.

Latest papers

CardsList

181 papers29 papers in the last four weeks, up 12% on the four weeks before. 0.2% of all new papers.

  1. PCM: A Proof-Carrying Message Protocol for Verifiable Multi-Agent Communication

    Oct 29, 2026Bravish GhoshAgent Communication ProtocolsAI Agent Security

    Cygnux Labs

  2. Who Else Is In? Multi-Agent Collusion Initation Via Strategic Vagueness

    Oct 27, 2026Pramod Kaushik, Sarath Sivaprasad, Laura Ferrarotti +4Multi-Agent System SecurityMulti-Agent LLM Systems

    University of Trento · Fondazione Bruno Kes sler · CISPA +4

  3. Sybil-Resilient Multi-Drone Exploration of Indoor and Confined Spaces

    Oct 24, 2026Shyaam Mahesh, Aditya B Prakash, Rubal Sagwal +1Resilient Multi-Robot CoordinationMulti-Agent System Security

  4. Securing Edge & Physical AI: Post-Quantum IoT Blockchain for Mission-Critical Cyber-Physical Agents

    Oct 24, 2026Kürşat Çakal, Suat ÖzdemirCPS SecurityMulti-Agent System Security

    Department of Computer Engineering, Hacettepe University, Ankara, 06800 Turkey

  5. Secure Agentic AI and Multimodal Intelligence in 6G Communication Systems

    Oct 21, 2026Satyam Gaba, Devanshu Motwani, Siva Sai +3AI Agent SecurityMultimodal Agents

    Qualcomm Wireless Research, San Diego, CA 92121, USA · Department of Computer Science and Engineering, BITS Pilani, Pilani Campus, Rajasthan 333031, India · Department of Electrical and Computer Engineering, National University of Singapore 117583, Singapore +1

  6. Black-Box Red-Teaming of Multi-Agent Systems via Reinforcement Learning

    Oct 20, 2026Yuqi Jia, Hangsheng Zhang, Chenxi Liu +2LLM Red TeamingBlack-Box Adversarial Attacks

    Duke University · ByteDance · University of Maryland, College Park

  7. The Show-Up Attack: When Subagents Win Over Their Supervisor

    Oct 20, 2026Maosen Zhang, Jianshuo Dong, Yutong Zhang +3LLM Agent SecurityMulti-Agent System Security

    Tsinghua University · SiliconProspect AI · Nanyang Technological University

  8. Vulnerability Analysis for Safe Multi-Agent Reinforcement Learning

    Oct 19, 2026Taozheng Zhu, Shixiong Jiang, Fanxin KongAdversarial AttacksAdversarial MARL

    University of Notre Dame Notre Dame, Indiana, USA

  9. Red-Teaming Multi-Agent Attacks for Loss-of-Control Risks

    Oct 16, 2026John Kitaoka, Thao Amelia Pham, Carlos Rafael Giudice +1AI Agent SafetyMulti-Agent System Security

    MATS Research · Redwood Research · Independent

  10. Cryptographically Secure Agent-to-Agent Communication in Multi-Agent Systems

    Oct 16, 2026Karol Budzyn, Adrian Brodzik, Mateusz SzczepkowskiMulti-Agent System SecurityMulti-Agent Communication

    Orange Innovation Poland

  11. StepSafe: Learning Timely Intervention with Context-Aware Safety Reasoning

    Oct 14, 2026Siyi Gu, Kunhan Gao, Chao Peter Yang +3AI Agent SafetyLLM Agent Safety

    Yale University · NetApp Inc.

  12. When Agents Talk, Secrets Travel: Topology-Driven Leakage and Recipient-Conditioned Sanitization in Multi-Agent LLM Systems

    Oct 12, 2026Sunday Oyinlola Ogundoyin, Muhammad IkramMulti-Agent LLM SystemsAI Agent Security

    Macquarie University Sydney, NSW, Australia

  13. Agent-Canary-Mas: End-to-End Evaluation of Protocol Risk in Long-Horizon Multi-Agent Systems

    Oct 11, 2026Xiuming Liu, Le Gai, Zhiyuan Chang +4Multi-Agent System SecurityAI Agent Security Benchmarks

    Shanghai Jiao Tong University · Ant Group · Tsinghua University

  14. Who to Trust in Multi-Robot Systems: Joint Robot and Object Hypothesis Trust Inference via Neural-Symbolic Cross-Validation

    Oct 10, 2026Ruiyang Wang, Hao-Lun Hsu, Haocheng Meng +2Multi-Robot SystemsMulti-Agent System Security

  15. Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security

    Oct 6, 2026Shaswata Mitra, Raj Patel, Subash Neupane +3Multi-Agent LLM SystemsMulti-Agent System Security

    Department of Computer Science The University of Alabama, Tuscaloosa, USA

  16. Topology-Conditioned Backdoors: Language Models That Insert Vulnerabilities When They Infer They Are in a Multi-Agent System

    Oct 5, 2026Keegan Wang, Anantika MannbyLLM Backdoor AttacksAI Agent Auditing

    Carnegie Mellon University Prosaic Intelligence · University of Southern California Prosaic Intelligence

  17. Agentic-ZTA: A Multi-Agent Architecture for Autonomous Zero Trust Enforcement

    Oct 5, 2026Shovan Roy, Lopamudra Praharaj, Maanak Gupta +1AI Agent SecurityMulti-Agent System Security

    Department of Computer Science, Tennessee Tech University, Cookeville, TN, USA · Department of Computer Science, University of North Carolina Pembroke, Pembroke, NC, USA · Department of Computer Science, The University of Texas at Dallas, Richardson, Texas, USA

  18. Can CaMeLs Talk? Securing Multi-Agent Systems Against Indirect Prompt Injection Attacks

    Oct 5, 2026James Peters-Gill, Avi Semler, Henning Bartsch +2AI Agent Security BenchmarksInformation Flow Control

    Independent · University of Oxford · MATS Research +1

  19. Trust-Gated Capability Control: Breaking the Trust-Vulnerability Paradox in Multi-Agent LLM Systems

    Oct 4, 2026Mehedi Hasan Nipu, Chinmoy Mitra, Tarannum Ahmed Nowshin +1Trust in AIMulti-Agent System Security

    Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh · Department of Computer Science & Engineering Rajshahi University of Engineering & Technology Rajshahi, Bangladesh · Department of Computer Science and Engineering BRAC University,Dhaka, Bangladesh +1

  20. Lie Rarely, Lie Big: Stealthy Insider Attacks on LLM Robot Teams

    Oct 3, 2026Sribalaji C. Anand, George J. PappasMulti-Agent System SecurityAI Agent Security

    Department of Electrical and Systems Engineering, University of Pennsylvania, United States · KTH Royal Institute of Technology, Sweden

  21. Resilient Multi-Agent Target Localization and Tracking via BAG-Aware Mutual Information Maximization under False Data Injection Attacks

    Oct 2, 2026Bibek Adhikari, Samrat Chaulagain, Kamesh SubbaraoMulti-Agent System SecurityFalse Data Injection Attacks

    Department of Mechanical and Aerospace Engineering, The University of Texas at Arlington, Arlington, TX 76019 USA

  22. HASTE: Evolving Agent Harnesses Against Emerging Attacks Using Sparse Evidence

    Oct 2, 2026Xiqiao Xiong, Moxin Li, Zhixin Ma +4LLM Agent SafetyAgent Harness Optimization

    University of Science and Technology of China · National University of Singapore · Singapore Management University

  23. MIRROR: Multipath Quorum Integrity for LLM Multi-Agent Communication

    Oct 1, 2026Ryuichi Yamafuji Lun, Jingzhen Wang, Shreyas Kolte +1Multi-Agent System SecurityMulti-Agent Communication

    University of Southern California

  24. Memetic Trojans: Social Contagions as Carriers of Adversarial Payloads in Agent Networks

    Sep 30, 2026Birk Torpmann-Hagen, Finn Schwall, Leon MoonenLLM Agent SecurityMulti-Agent System Security

    Simula Research Laboratory Oslo, Norway · Simula Research Laboratory Oslo Metropolitan University Oslo, Norway

  25. Safety of Latent Communication in Multi-Agent Systems

    Sep 30, 2026Muhammad Huzaifa, Sina Mavali, Thorsten EisenhoferMulti-Agent LLM SystemsAI Agent Safety

    CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

  26. MADBench: Benchmarking the Security of Multi-Agent Debate

    Sep 30, 2026Yuwan Liu, Jiaming Zhang, Yue Huang +1Adversarial Attacks on LLMsMulti-Agent Debate

    Tsinghua University

  27. Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systems

    Sep 30, 2026Yunbei Zhang, Saiyue Lyu, Janet Wang +4Multi-Agent CollaborationInformation Flow Control

    Tulane University · University of British Columbia · Independent Researcher +1

  28. Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems

    Sep 30, 2026Deema Alnuhait, Gengyu Wang, Muhammad Khalifa +1AI Agent SafetyMulti-Agent System Security

    University of Illinois Urbana-Champaign · Genies · NVIDIA +1

  29. VirusCascade: Hijacking Collaborative Reflection in LLM-Powered Recommender Agents

    Sep 29, 2026Yurong Hao, Wen Zhou, Guowei Guan +3Adversarial AttacksAgent Memory Poisoning

    College of Computing and Data Science, Nanyang Technological University

  30. Concealing LLM-Based Multi-Agent Topology via Phantom Structure Injection

    Sep 29, 2026Longzhu He, Zelang Wen, Xinfeng Li +2Multi-Agent LLM SystemsMulti-Agent System Security

    Nanyang Technological University · Beijing University of Posts and Telecommunications · The Hong Kong Polytechnic University