Multi-Agent System Security

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 200% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 112

All topics

Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security
Oct 6, 2026Shaswata Mitra, Raj Patel, Subash Neupane +3Multi-Agent LLM Systems Multi-Agent System Security
Department of Computer Science The University of Alabama, Tuscaloosa, USA
Topology-Conditioned Backdoors: Language Models That Insert Vulnerabilities When They Infer They Are in a Multi-Agent System
Oct 5, 2026Keegan Wang, Anantika MannbyLLM Backdoor Attacks AI Agent Auditing
Carnegie Mellon University Prosaic Intelligence · University of Southern California Prosaic Intelligence
Agentic-ZTA: A Multi-Agent Architecture for Autonomous Zero Trust Enforcement
Oct 5, 2026Shovan Roy, Lopamudra Praharaj, Maanak Gupta +1AI Agent Security Multi-Agent System Security
Department of Computer Science, Tennessee Tech University, Cookeville, TN, USA · Department of Computer Science, University of North Carolina Pembroke, Pembroke, NC, USA · Department of Computer Science, The University of Texas at Dallas, Richardson, Texas, USA
Can CaMeLs Talk? Securing Multi-Agent Systems Against Indirect Prompt Injection Attacks
Oct 5, 2026James Peters-Gill, Avi Semler, Henning Bartsch +2AI Agent Security Benchmarks Information Flow Control
Independent · University of Oxford · MATS Research +1
Memetic Trojans: Social Contagions as Carriers of Adversarial Payloads in Agent Networks
Sep 30, 2026Birk Torpmann-Hagen, Finn Schwall, Leon MoonenLLM Agent Security Multi-Agent System Security
Simula Research Laboratory Oslo, Norway · Simula Research Laboratory Oslo Metropolitan University Oslo, Norway
Safety of Latent Communication in Multi-Agent Systems
Sep 30, 2026Muhammad Huzaifa, Sina Mavali, Thorsten EisenhoferMulti-Agent LLM Systems AI Agent Safety
CISPA Helmholtz Center for Information Security, Saarbrücken, Germany
MADBench: Benchmarking the Security of Multi-Agent Debate
Sep 30, 2026Yuwan Liu, Jiaming Zhang, Yue Huang +1Adversarial Attacks on LLMs Multi-Agent Debate
Tsinghua University
Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systems
Sep 30, 2026Yunbei Zhang, Saiyue Lyu, Janet Wang +4Multi-Agent Collaboration Information Flow Control
Tulane University · University of British Columbia · Independent Researcher +1
VirusCascade: Hijacking Collaborative Reflection in LLM-Powered Recommender Agents
Sep 29, 2026Yurong Hao, Wen Zhou, Guowei Guan +3Adversarial Attacks Agent Memory Poisoning
College of Computing and Data Science, Nanyang Technological University
Concealing LLM-Based Multi-Agent Topology via Phantom Structure Injection
Sep 29, 2026Longzhu He, Zelang Wen, Xinfeng Li +2Multi-Agent LLM Systems Multi-Agent System Security
Nanyang Technological University · Beijing University of Posts and Telecommunications · The Hong Kong Polytechnic University
DGF-Bench: A Benchmark for Simulating and Auditing Deception Against Multi-Agent Governance Boards
Sep 28, 2026Jeremy CanaleAI Agent Auditing AI Agent Security Benchmarks
How does Adversarial Influence Scale in Multi-Agent Systems?
Sep 24, 2026Addison J. Wu, Jasin Cekinmez, Michel Liao +2Multi-Agent LLM Systems Deception in Language Models
Princeton University
Codetta: High-Capacity, Keyless, and Undetectable Multi-Agent Collusion
Sep 24, 2026Qi Pang, Virginia Smith, Wenting ZhengMulti-Agent LLM Systems Algorithmic Collusion
Carnegie Mellon University
Shutdown Sabotage Propensities in Multi-Agent Systems
Sep 23, 2026Amelie Knecht, Ulysse Schaller, Christopher Summerfield +1AI Agent Safety Multi-Agent System Security
AI Safety Research Group, University of Stuttgart, Stuttgart, Germany · University of Oxford, Oxford, United Kingdom
Agent Name Collision Attacks in Multi-Agent Systems
Sep 23, 2026Adithyan Arun KumarAgent Communication Protocols AI Agent Security
Independent Security Researcher
Emergent Collusion in Long-Horizon LLM Agent Interaction
Sep 21, 2026Xinrui Shi, Yanzhe Zhang, Diyi YangMulti-Agent Coordination LLM Agent Safety
Stanford University · Georgia Tech
A Scalable Trust Discovery Architecture for the Internet of Agents
Sep 17, 2026Song Zhang, Jiankang Yao, Hongtao Li +4Multi-Agent System Security
China Internet Network Information Center, Beijing 100070, China · National Engineering Laboratory of Internet Domain Name Management Technology, Beijing 100070, China · Building 4, No.9 West Road, Automobile Museum, Fengtai District, Beijing 100070, China +2
PAPC: Platform Mediation for Privacy-Propagation Externalities in AI-Mediated Workflows
Sep 16, 2026Tao Huang, Guosen Wu, Chen Hou +1AI Agent Governance Multi-Agent System Security
Collective Loss of Control in LLM Agent Systems: An Epidemic Account of Mutation, Contagion, and Recovery
Sep 16, 2026Xiangfan Wu, Zonghao Ying, Huiyu Wu +4Multi-Agent LLM Systems AI Agent Safety
Tencent Zhuque Lab
Agentic Societies Need a Social Harness
Sep 15, 2026Tapan Chugh, Vidushi Singh, Krish Jain +2AI Agent Reliability Multi-Agent System Security
University of Washington Seattle, Washington, USA
Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
Sep 10, 2026Divyanshu Kumar, Nitin Aravind Birur, Tanay Baswa +2AI Agent Security AI Agent Security Benchmarks
Enkrypt AI
PRIMUS: Identity, Governance, and Verification for Multi-Agent Federations
Sep 7, 2026Sasank Annapureddy, Anjaneya Prasad ThamataniLLM Agent Verification Multi-Agent Consensus
A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms
Sep 3, 2026Davide Paglieri, Logan Cross, Tim Genewein +3AI Agent Governance Multi-Agent System Security
Google DeepMind
You Can't Escape Your Own Activations : Evaluation Awareness and Multi-Agent Monitoring
Sep 2, 2026Aritra Das, Jaee Ponde, Mihir More +1AI Agent Monitoring Multi-Agent System Security
Truth Audit Labs
Public-Sharing Labels and Verbatim Field Egress in an MCP-to-A2A Agent Configuration: A Controlled Multi-Model Study
Sep 1, 2026Arpan Kumar MahapatraMCP Security LLM Agent Security
SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
Sep 1, 2026Rui Yang, Junjie Xu, Zhengyu Liu +4AI Agent Security Benchmarks Multi-Agent System Security
Johns Hopkins University · Nanyang Technological University
VERA: Authority-Preserving Edge Revocation for Federated AI-Agent Workflows
Aug 30, 2026Lifei Liu, Haoran Yu, Xiaochong JiangRuntime Enforcement for AI Agents Access Control
Independent Researcher
Mind Viruses: Self-Propagating Ideas in Multi-Agent LLM Systems
Aug 10, 2026Vassilis Papadopoulos, McNair Shah, Sam Zimmerman +1Multi-Agent LLM Systems Multi-Agent Systems
1Anthropic Fellows Program · 2EPFL · 3Anthropic
When Coordination Becomes a Threat: Communication Attacks in LLM-Controlled Multi-Robot Systems
Aug 7, 2026Zhen Huang, Zhihuang Liu, Weijia Shi +3Multi-Robot Systems Large Language Model-Based Robot Planning
College of Computer Science and Technology, National University of Defense Technology, Changsha, China
Online Security Learning in Cooperative Multi-Agent Systems under Hidden Byzantine Attacks
Aug 6, 2026Ximing Sun, Yue WangMulti-Agent System Security Robust RL
Department of Electrical and Computer Engineering University of Central Florida

Oct 6, 2026, cs.AIJ/K move · Enter open · S save

Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security

Shaswata Mitra, Raj Patel, Subash Neupane, Sudip Mittal+2

Department of Computer Science The University of Alabama, Tuscaloosa, USA

Multi-Agent LLM Systems Multi-Agent System Security

Open paper AskPDF arXiv

LLM-based multi-agent systems (MAS) engage tools, share memory, and delegate tasks, often encountering adversarial content. Current defenses for MAS are typically evaluated in isolation, focusing on one attack type at a time, which can lead to costly and hard-to-audit outcomes. This study organizes defenses into five principles, implementing them as DEFER1 (DEterministic-First Enforcement with Residual judgment), which includes a cascade of 28 checks that blocks what it can and refers the rest to a panel of four judges. In independent testing across four domains, attack success rates drop from about 30.0% to approximately 3.0%, with 78% of blocked attacks handled by deterministic checks. Only a quarter of proposals reach the judges in the security-operations domain, illustrating that the rules provide security for attacks violating clear policies, while judges manage those that only misrepresent intent. Both systems have weaknesses, such as a risk-score approval gate that inaccurately approves most attack proposals but few legitimate ones, highlighting the challenges in assessing threats accurately.

Momentum

Related topics

Latest papers 112