Multi-Agent System Security

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 200% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 112

All topics

Insider Attacks in Multi-Agent LLM Consensus Systems
May 8, 2026Xiaolin Sun, Zixuan Liu, Yibin Hu +1Multi-Agent LLM Systems Multi-Agent Systems
Department of Computer Science, Tulane University, New Orleans, United States of America.
MAGIQ: A Post-Quantum Multi-Agentic AI Governance System with Provable Security
May 7, 2026Sepideh Avizheh, Tushin Mallick, Alina Oprea +2AI Agent Governance Access Control
University of Calgary Calgary, Alberta, Canada · Northeastern University Boston, MA, USA
Towards Security-Auditable LLM Agents: A Unified Graph Representation
May 7, 2026Chaofan Li, Lyuye Zhang, Jintao Zhai +9LLM Agent Security Multi-Agent System Security
Huazhong University of Science and Technology, China · Nanyang Technological University, Singapore, Singapore · Chongqing University of Posts and Telecommunications, China +2
Authorization Propagation in Multi-Agent AI Systems: Identity Governance as Infrastructure
May 6, 2026Krti TallamAI Agent Governance Access Control
Kamiwaza AI
When Agents Handle Secrets: A Survey of Confidential Computing for Agentic AI
May 4, 2026Javad Forough, Marios Kogias, Hamed HaddadiCybersecurity Remote Attestation
Department of Computing Imperial College London London, United Kingdom
Catching the Infection Before It Spreads: Foresight-Guided Defense in Multi-Agent Systems
May 3, 2026Yue Ma, Ziyuan Yang, Yi ZhangMulti-Agent LLM Systems Jailbreak Defense
Sichuan University · Nanyang Technological University
When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems
May 1, 2026Lingxi Zhang, Guangtao Zheng, Hanjie ChenAdversarial Attacks Language Model Robustness
1Rice University · University of Virginia
Ambient Persuasion in a Deployed AI Agent: Unauthorized Escalation Following Routine Non-Adversarial Content Exposure
Apr 29, 2026Diego F. Cuadros, Abdoul-Aziz MaigaAgent Failure Analysis AI Agent Security
Digital Epidemiology Laboratory, Digital Futures, University of Cincinnati, Cincinnati, OH, USA · Center for Humanities and Technology (CHaT), University of Cincinnati, Cincinnati, OH, USA · Norwegian University of Science and Technology, Trondheim, Norway
Agent Name Service (ANS): A Proof-of-Concept Trust Layer for Secure AI Agent Discovery, Identity, and Governance in Kubernetes
Apr 29, 2026Akshay Mittal, Elyson De La CruzAI Agent Security AI Agent Governance
Department of Information Technology University of the Cumberlands Williamsburg, KY, USA
GAMMAF: A Common Framework for Graph-Based Anomaly Monitoring Benchmarking in LLM Multi-Agent Systems
Apr 27, 2026Pablo Mateo-Torrejón, Alfonso Sánchez-MaciánMulti-Agent LLM Systems AI Agent Security Benchmarks
University Carlos III of Madrid · Leganés, Spain
Breaking the Secret: Economic Interventions for Combating Collusion in Embodied Multi-Agent Systems
Apr 26, 2026Qi Liu, Xiaohui Chen, Zhihui Zhao +5Mechanism Design Multi-Agent Systems
College of Computer Science and Technology, Taiyuan University of Technology, Taiyuan, China, 030024 · State Key Laboratory of Internet Architecture, Tsinghua University, Beijing, China, 100084 · China Mobile Research Institute, Beijing, China, 100053 +3
Architecture Matters for Multi-Agent Security
Apr 25, 2026Ben Hagag, William L. Anderson, Christian Schroeder de Witt +1Multi-Agent Systems Multi-Agent System Security
Carnegie Mellon University · MATS · Oxford Witt Lab +1
A Systematic Survey of Security Threats and Defenses in LLM-Based AI Agents: A Layered Attack Surface Framework
Apr 25, 2026Kexin ChuAI Agent Security LLM Agent Security
School of Computing, University of Connecticut
Beyond Single-Agent Alignment: Preventing Context-Fragmented Violations in Multi-Agent Systems
Apr 24, 2026Jie Wu, Ming GongCybersecurity AI Agent Security Benchmarks
Atlassian
Conjunctive Prompt Attacks in Multi-Agent LLM Systems
Apr 17, 2026Nokimul Hasan Arif, Qian Lou, Mengxin ZhengMulti-Agent LLM Systems Adversarial Attacks on LLMs
University of Central Florida, USA
Detecting Multi-Agent Collusion Through Multi-Agent Interpretability
Apr 1, 2026Aaron Rose, Carissa Cullen, Sahar Abdelnabi +3AI Agent Security Benchmarks Multi-Agent System Security
University of Oxford · ELLIS Institute Tübingen MPI-IS
AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystems
Mar 16, 2026Yihao Zhang, Zeming Wei, Xiaokun Luan +7LLM Agent Security Multi-Agent System Security
Peking University · Sun Yat-sen University · Wuhan University +2
MAS-Shield: A Defense Framework for Secure and Efficient LLM MAS
Nov 28, 2025Kaixiang Wang, Zhaojiacheng Zhou, Bunyod Suvonov +8Multi-Agent LLM Systems LLM Auditing
Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai 200240, China
Seven Security Challenges in Cross-domain Multi-agent LLM Systems
May 28, 2025Ronny Ko, Jiseong Jeong, Shuyuan Zheng +4Multi-Agent LLM Systems LLM Agent Security
Osaka University · Seoul National University · Yonsei University
SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systems
Feb 6, 2024Oubo Ma, Yuwen Pu, Linkang Du +5Adversarial Attacks Partially Observable RL
Zhejiang University · Xi’an Jiaotong University · Laboratory for Big Data and Decision +2
Emergent Risks in Generative Multi-Agent Systems
Date pendingYue Huang, Yu Jiang, Wenjie Wang +12Multi-Agent LLM Systems Multi-Agent Systems
University of Notre Dame · LMU Munich · University of Washington +7
Securing People and their Machines Against Major Faults
Date pendingOhad Eitan, Idit Keidar, Ehud ShapiroMulti-Agent System Security
Technion – Israel Institute of Technology, Haifa, Israel · London School of Economics and Political Science, UK and Weizmann Institute of Science, Israel

May 8, 2026, cs.MAJ/K move · Enter open · S save

Insider Attacks in Multi-Agent LLM Consensus Systems

Xiaolin Sun, Zixuan Liu, Yibin Hu, Zizhan Zheng

Department of Computer Science, Tulane University, New Orleans, United States of America.

Multi-Agent LLM Systems Multi-Agent Systems Multi-Agent Consensus Multi-Agent System Security

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly deployed in multi-agent systems where agents communicate in natural language to solve tasks jointly. A key capability in such systems is consensus formation, where agents iteratively exchange messages and update decisions to reach a shared outcome. However, most existing multi-agent LLM frameworks assume that all participating agents are aligned with the system objective. In practice, a malicious insider may participate as a legitimate member of the group while pursuing a hidden adversarial goal. In this work, we study insider manipulation in multi-agent LLM consensus systems. We formalize the problem as a sequential decision-making task in which a malicious agent seeks to delay or prevent agreement among benign agents. To make attack optimization tractable, we propose a world-model-based framework that learns surrogate dynamics over the latent behavioral states of benign agents and then trains an attacker using reinforcement learning based on this learned model. Preliminary results show that the trained attacker reduces the benign consensus rate and prolongs disagreement more effectively than the direct malicious-prompt baseline. These results suggest that combining latent world models with reinforcement learning is a promising direction for adaptive insider attacks in language-based multi-agent systems.

Momentum

Related topics

Latest papers 112