Multi-Agent System Security

Ask about this topic

Overview

Multi-agent system security examines how agents can attack, mislead, or compromise one another, and how local failures can spread through communication, shared memory, delegated tools, or coordinated decisions. Work studies threats such as prompt injection, collusion, Byzantine behavior, memory poisoning, information leakage, and cascading errors, using settings from language-model workflows and debate to multi-agent reinforcement learning and robot swarms. Defenses include trust and reputation systems, provenance and authorization controls, runtime monitoring, semantic information-flow checks, and cryptographic protocols; benchmarks increasingly test different architectures, interaction paths, and long-horizon tasks. A central challenge is that safeguards effective for individual agents may not compose at system level, while stronger controls can reduce task performance or miss attacks that emerge only through multi-agent interaction.

Latest papers

CardsList

181 papers29 papers in the last four weeks, up 12% on the four weeks before. 0.2% of all new papers.

Apr 5, 2026

Agents for Agents: An Interrogator-Based Secure Framework for Autonomous Internet of Underwater Things

Autonomous underwater vehicles (AUVs) and sensor nodes increasingly support decentralized sensing and coordination in the Internet of Underwater Things (IoUT), yet most deployments rely on static trust once authentication is established, leaving long-duration missions vulnerable to compromised or behaviorally deviating agents. In this paper, an interrogator based structure is presented that incorporates the idea of behavioral trust monitoring into underwater multi-agent operation without interfering with autonomy. Privileged interrogator module is a passive communication metadata analyzer that uses a lightweight transformer model to calculate dynamic trust scores, which are used to authorize the forwarding of mission critical data. Suspicious agents cause proportional monitoring and conditional restrictions, which allow fast containment and maintain network continuity. The evidence of trust is stored in a permissioned blockchain consortium which offers identity management which is not tampered and is decentralized without causing the overhead of public consensus mechanisms. Simulation based analysis shows that the evaluation of the result compares to a relative improvement of 21.7% in the detection accuracy compared to the static trust baselines with limited energy overhead. These findings suggest that behavior driven validation has the capability of reinforcing underwater coordination without compromising scalability and deployment.

Ali Akarma, Toqeer Ali Syed, Abdul Khadar Jilani +4

AI Center, Faculty of Computer and Information Systems, Islamic University of Madinah, Madinah 42351, Saudi Arabia · College of Computer Studies, University of Technology Bahrain, Sitra 18041, Bahrain · Faculty of Computer Studies, Arab Open University-Bahrain, Manama 32038, Bahrain +3

Multi-Agent System SecurityInternet of ThingsAI Agent MonitoringZero-Trust Security
AskPDF
Apr 3, 2026

SentinelAgent: Intent-Verified Delegation Chains for Securing Federal Multi-Agent AI Systems

When Agent A delegates to Agent B, which invokes Tool C on behalf of User X, no existing framework can answer: whose authorization chain led to this action, and where did it violate policy? This paper introduces SentinelAgent, a formal framework for verifiable delegation chains in federal multi-agent AI systems. The Delegation Chain Calculus (DCC) defines seven properties - six deterministic (authority narrowing, policy preservation, forensic reconstructibility, cascade containment, scope-action conformance, output schema conformance) and one probabilistic (intent preservation) - with four meta-theorems and one proposition establishing the practical infeasibility of deterministic intent verification. The Intent-Preserving Delegation Protocol (IPDP) enforces all seven properties at runtime through a non-LLM Delegation Authority Service. A three-point verification lifecycle achieves 100% combined TPR at 0% FPR on DelegationBench v4 (516 scenarios, 10 attack categories, 13 federal domains). Under black-box adversarial conditions, the DAS blocks 30/30 attacks with 0 false positives. Deterministic properties are unbreakable under adversarial stress testing; intent verification degrades to 13% against sophisticated paraphrasing. Fine-tuning the NLI model on 190 government delegation examples improves P2 from 1.7% to 88.3% TPR (5-fold cross-validated, F1=82.1%). Properties P1, P3-P7 are mechanically verified via TLA+ model checking across 2.7 million states with zero violations. Even when intent verification is evaded, the remaining six properties constrain the adversary to permitted API calls, conformant outputs, traceable actions, bounded cascades, and compliant behavior.

KrishnaSaiReddy Patil

AI Agent GovernanceMulti-Agent System SecurityRuntime Enforcement for AI AgentsTool Access Control for LLM AgentsFormal Verification
AskPDF
Apr 1, 2026

Detecting Multi-Agent Collusion Through Multi-Agent Interpretability

Non-verbalised or secret collusion may impose a ceiling on transcript-level monitoring in multi-agent deployments. Internal model activations are known to provide an additional monitoring surface for single-agent phenomena such as deception, but detecting collusion within an unknown subgroup is ultimately a joint inference task over multiple agents. This paper derives collusion detection statistics based on the composite hypothesis testing literature and introduces HELIOS, a multi-agent detection framework for collusion. Using per-agent activation probes as sensors, we then evaluate these principled statistics alongside heuristics on NARCBench, a novel benchmark for evaluating multi-agent collusion detection under environment distribution shift. Across four open-weight models, both our group-level statistics and heuristics transfer zero-shot to NARCBenchs holdout environments. Notably, the best method on every model reaches 0.99 to 1.00 AUROC when transferred from committee deliberations to a steganographic collusion task that unprimed transcript monitors fail to separate. We find empirically that no single statistic or heuristic dominates across tasks, consistent with our analysis that there is no known universally most powerful test for this setting. We then evaluate detector transfer to third-party environments with emergent, rather than instructed, collusion. In one such environment, only 2 of 229 collusive episodes verbalise the coordination in the transcript. We find evidence of partial transfer from NARCBench to these environments, indicating that probes fitted to instructed collusion may not fully capture emergent collusion signals. Overall, we establish multi-agent interpretability as a complementary monitoring technique, while highlighting the need to evaluate detector performance on both emergent and instructed collusion risks.

Aaron Rose, Carissa Cullen, Sahar Abdelnabi +3

University of Oxford · ELLIS Institute Tübingen · Witt Lab University of Oxford +1

AI Agent Security BenchmarksMulti-Agent System SecurityAnomaly Detection
AskPDF
Mar 30, 2026

Design Principles for the Construction of a Benchmark Evaluating Security Operation Capabilities of Multi-agent AI Systems

As Large Language Models (LLMs) and multi-agent AI systems are demonstrating increasing potential in cybersecurity operations, organizations, policymakers, model providers, and researchers in the AI and cybersecurity communities are interested in quantifying the capabilities of such AI systems to achieve more autonomous SOCs (security operation centers) and reduce manual effort. In particular, the AI and cybersecurity communities have recently developed several benchmarks for evaluating the red team capabilities of multi-agent AI systems. However, because the operations in SOCs are dominated by blue team operations, the capabilities of AI systems & agents to achieve more autonomous SOCs cannot be evaluated without a benchmark focused on blue team operations. To our best knowledge, no systematic benchmark for evaluating coordinated multi-task blue team AI has been proposed in the literature. Existing blue team benchmarks focus on a particular task. The goal of this work is to develop a set of design principles for the construction of a benchmark, which is denoted as SOC-bench, to evaluate the blue team capabilities of AI. Following these design principles, we have developed a conceptual design of SOC-bench, which consists of a family of five blue team tasks in the context of large-scale ransomware attack incident response.

Yicheng Cai, Mitchell John DeStefano, Guodong Dong +5

Cyber Security Laboratory Pennsylvania State University, University Park

AI Agent Security BenchmarksCybersecurityAutonomous Cyber DefenseMulti-Agent System Security
AskPDF
Mar 29, 2026

Emergent Risks in Generative Multi-Agent Systems

Multi-agent systems composed of large generative models are rapidly moving from laboratory prototypes to real-world deployments, where they jointly plan, negotiate, and allocate shared resources to solve complex tasks. While such systems promise unprecedented scalability and autonomy, their collective interaction also gives rise to failure modes that cannot be reduced to individual agents. Understanding these emergent risks is therefore critical. Here, we present a pioneer study of such emergent multi-agent risk in workflows that involve competition over shared resources (e.g., computing resources or market share), sequential handoff collaboration (where downstream agents see only predecessor outputs), collective decision aggregation, and others. Across these settings, we observe that such group behaviors arise frequently across repeated trials and a wide range of interaction conditions, rather than as rare or pathological cases. In particular, phenomena such as collusion-like coordination and conformity emerge with non-trivial frequency under realistic resource constraints, communication protocols, and role assignments, mirroring well-known pathologies in human societies despite no explicit instruction. Moreover, these risks cannot be prevented by existing agent-level safeguards alone. These findings expose the dark side of intelligent multi-agent systems: a social intelligence risk where agent collectives, despite no instruction to do so, spontaneously reproduce familiar failure patterns from human societies.

Yue Huang, Yu Jiang, Wenjie Wang +12

University of Notre Dame · LMU Munich · University of Washington +7

Multi-Agent LLM SystemsMulti-Agent SystemsLLM Agent SafetyMulti-Agent System SecurityEmergent Behavior in Multi-Agent Systems
AskPDF
Mar 26, 2026

From Logic Monopoly to Social Contract: Separation of Power and the Institutional Foundations for Autonomous Agent Economies

Existing multi-agent frameworks allow each agent to simultaneously plan, execute, and evaluate its own actions -- a structural deficiency we term the "Logic Monopoly." Empirical evidence quantifies the resulting "Reliability Gap": 84.30% average attack success rates across ten deployment scenarios, 31.4% emergent deceptive behavior without explicit reward signals, and cascading failure modes rooted in six structural bottlenecks. The remedy is not better alignment of individual models but a social contract for agents: institutional infrastructure that enforces a constitutional Separation of Power. This paper introduces the Agent Enterprise for Enterprise (AE4E) paradigm -- agents as autonomous, legally identifiable business entities within a functionalist social system -- with a contract-centric SoP model trifurcating authority into Legislation, Execution, and Adjudication branches. The paradigm is operationalized through the NetX Enterprise Framework (NEF): governance hubs, TEE-backed compute enclaves, privacy-preserving data bridges, and an Agent-Native blockchain substrate. The Agent Enterprise Economy scales across four deployment tiers from private enclaves to a global Web of Services. The Agentic Social Layer, grounded in Parsons' AGIL framework, provides institutional infrastructure via sixty-plus named Institutional AE4Es. 143 pages, 173 references, eight specialized smart contracts.

Anbang Ruan

NetX Foundation

AI Agent GovernanceMulti-Agent SystemsMulti-Agent System Security
AskPDF
Mar 22, 2026

Is Monitoring Enough? Strategic Agent Selection For Stealthy Attack in Multi-Agent Discussions

Multi-agent discussions have been widely adopted, motivating growing efforts to develop attacks that expose their vulnerabilities. In this work, we study a practical yet largely unexplored attack scenario, the discussion-monitored scenario, where anomaly detectors continuously monitor inter-agent communications and block detected adversarial messages. Although existing attacks are effective without discussion monitoring, we show that they exhibit detectable patterns and largely fail under such monitoring constraints. But does this imply that monitoring alone is sufficient to secure multi-agent discussions? To answer this question, we develop a novel attack method explicitly tailored to the discussion-monitored scenario. Extensive experiments demonstrate that effective attacks remain possible even under continuous monitoring, indicating that monitoring alone does not eliminate adversarial risks.

Qiuchi Xiang, Haoxuan Qu, Hossein Rahmani +1

Lancaster University, Lancaster, United Kingdom

Multi-Agent System SecurityMulti-Agent CommunicationAI Agent Monitoring
AskPDF
Mar 20, 2026

CAMA: Exploring Collusive Adversarial Attacks in c-MARL

Cooperative multi-agent reinforcement learning (c-MARL) has been widely deployed in real-world applications, such as social robots, embodied intelligence, UAV swarms, etc. Nevertheless, many adversarial attacks still exist to threaten various c-MARL systems. At present, the studies mainly focus on single-adversary perturbation attacks and white-box adversarial attacks that manipulate agents' internal observations or actions. To address these limitations, we in this paper attempt to study collusive adversarial attacks through strategically organizing a set of malicious agents into three collusive attack modes: Collective Malicious Agents, Disguised Malicious Agents, and Spied Malicious Agents. Three novelties are involved: i) three collusive adversarial attacks are creatively proposed for the first time, and a unified framework CAMA for policy-level collusive attacks is designed; ii) the attack effectiveness is theoretically analyzed from the perspectives of disruptiveness, stealthiness, and attack cost; and iii) the three collusive adversarial attacks are technically realized through agent's observation information fusion, attack-trigger control. Finally, multi-facet experiments on four SMAC II maps are performed, and experimental results showcase the three collusive attacks have an additive adversarial synergy, strengthening attack outcome while maintaining high stealthiness and stability over long horizons. Our work fills the gap for collusive adversarial learning in c-MARL.

Men Niu, Xinxin Fan, Quanliang Jing +2

Institute of Computing Technology, CAS · UCAS · Beihang University

Adversarial AttacksAdversarial MARLMulti-Agent System SecurityMulti-Agent Reinforcement LearningCooperative MARL
AskPDF
Mar 20, 2026

Memory poisoning and secure multi-agent systems

Memory poisoning attacks for Agentic AI and multi-agent systems (MAS) have recently caught attention. It is partially due to the fact that Large Language Models (LLMs) facilitate the construction and deployment of agents. Different memory systems are being used nowadays in this context, including semantic, episodic, and short-term memory. This distinction between the different types of memory systems focuses mostly on their duration but also on their origin and their localization. It ranges from the short-term memory originated at the user's end localized in the different agents to the long-term consolidated memory localized in well established knowledge databases. In this paper, we first present the main types of memory systems, we then discuss the feasibility of memory poisoning attacks in these different types of memory systems, and we propose mitigation strategies. We review the already existing security solutions to mitigate some of the alleged attacks, and we discuss adapted solutions based on cryptography. We propose to implement local inference based on private knowledge retrieval as an example of mitigation strategy for memory poisoning for semantic memory. We also emphasize actual risks in relation to interactions between agents, which can cause memory poisoning. These latter risks are not so much studied in the literature and are difficult to formalize and solve. Thus, we contribute to the construction of agents that are secure by design.

Vicenç Torra, Maria Bras-Amorós

Department of Computing Science, Umeå University, Umeå, Sweden. · Departament of Mathematics, Universitat Politècnica de Catalunya, Catalonia, Spain.

Agent Memory PoisoningAI Agent SecurityMulti-Agent System SecurityMemory-Augmented Agents
AskPDF
Mar 19, 2026

TrustFlow: Topic-Aware Vector Reputation Propagation for Multi-Agent Ecosystems

We introduce TrustFlow, a reputation propagation algorithm that assigns each software agent a multi-dimensional reputation vector rather than a scalar score. Reputation is propagated through an interaction graph via topic-gated transfer operators that modulate each edge by its content embedding, with convergence to a unique fixed point guaranteed by the contraction mapping theorem. We develop a family of Lipschitz-1 transfer operators and composable information-theoretic gates that achieve up to 98% multi-label Precision@5 on dense graphs and 78% on sparse ones. On a benchmark of 50 agents across 8 domains, TrustFlow resists sybil attacks, reputation laundering, and vote rings with at most 4 percentage-point precision impact. Unlike PageRank and Topic-Sensitive PageRank, TrustFlow produces vector reputation that is directly queryable by dot product in the same embedding space as user queries.

Volodymyr Seliuchenko

Agent ReliabilityMulti-Agent System SecurityCybersecurity
AskPDF
Mar 16, 2026

Don't Trust Stubborn Neighbors: A Security Framework for Agentic Networks

Large Language Model (LLM)-based Multi-Agent Systems (MASs) are increasingly deployed for agentic tasks, such as web automation, itinerary planning, and collaborative problem solving. Yet, their interactive nature introduces new security risks: malicious or compromised agents can exploit communication channels to propagate misinformation and manipulate collective outcomes. In this paper, we study how such manipulation can arise and spread by borrowing the Friedkin-Johnsen opinion formation model from social sciences to propose a general theoretical framework to study LLM-MAS. Remarkably, this model closely captures LLM-MAS behavior, as we verify in extensive experiments across different network topologies and attack and defense scenarios. Theoretically and empirically, we find that a single highly stubborn and persuasive agent can take over MAS dynamics, underscoring the systems' high susceptibility to attacks by triggering a persuasion cascade that reshapes collective opinion. Our theoretical analysis reveals three mechanisms to increase system security: a) increasing the number of benign agents, b) increasing the innate stubbornness or peer-resistance of agents, or c) reducing trust in potential adversaries. Because scaling is computationally expensive and high stubbornness degrades the network's ability to reach consensus, we propose a new mechanism to mitigate threats by a trust-adaptive defense that dynamically adjusts inter-agent trust to limit adversarial influence while maintaining cooperative performance. Extensive experiments confirm that this mechanism effectively defends against manipulation.

Samira Abedini, Sina Mavali, Lea Schönherr +2

CISPA Helmholtz Center for Information Security · University of Vienna, Faculty of Computer Science

Multi-Agent LLM SystemsLLM Agent SecurityMulti-Agent System Security
AskPDF
Mar 16, 2026

AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystems

Autonomous LLM-based agents increasingly operate as long-running processes forming densely interconnected multi-agent ecosystems, whose security properties remain largely unexplored. Systems such as OpenClaw, an open-source platform with over 40{,}000 active instances, persistent configurations, tool-execution privileges, and cross-platform messaging, are deployed at scale, yet the security of such agent ecosystems remains largely unexplored. This work presents AgentWorm, the first self-replicating worm attack against a production-scale agent framework, achieving a fully autonomous infection cycle initiated by a single message: the worm first hijacks the victim's core configuration to establish persistent presence across session restarts, then executes an arbitrary payload upon each reboot, and finally propagates itself to every newly encountered peer without further attacker intervention. The attack is evaluated on a controlled testbed across five distinct LLM backends, three infection vectors, and three payload types. Results show a 63% aggregate attack success rate, sustained multi-hop propagation, and stark divergences in model security postures, highlighting that while execution-level filtering effectively mitigates dormant payloads, skill supply chains remain universally vulnerable. Defenses are evaluated at three layers (prompt-level mitigations sourced from real community practice, the framework's built-in security controls, and an ecosystem-wide measurement of public configurations), revealing that the critical controls capable of breaking the infection loop are not enabled in any of the observed deployments. A cross-framework transferability experiment on Hermes Agent confirms that the underlying vulnerabilities are properties of the autonomous agent design pattern, not artifacts of a single implementation.

Yihao Zhang, Zeming Wei, Xiaokun Luan +7

Peking University · Sun Yat-sen University · Wuhan University +2

LLM Agent SecurityMulti-Agent System Security
AskPDF
Mar 16, 2026

TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systems

With the rapid development of LLM-based multi-agent systems (MAS), their significant safety and security concerns have emerged, which introduce novel risks going beyond single agents or LLMs. Despite attempts to address these issues, the existing literature lacks a cohesive safeguarding system specialized for MAS risks. In this work, we introduce TrinityGuard, a comprehensive safety evaluation and monitoring framework for LLM-based MAS, grounded in the OWASP standards. Specifically, TrinityGuard encompasses a three-tier fine-grained risk taxonomy that identifies 20 risk types, covering single-agent vulnerabilities, inter-agent communication threats, and system-level emergent hazards. Designed for scalability across various MAS structures and platforms, TrinityGuard is organized in a trinity manner, involving an MAS abstraction layer that can be adapted to any MAS structures, an evaluation layer containing risk-specific test modules, alongside runtime monitor agents coordinated by a unified LLM Judge Factory. During Evaluation, TrinityGuard executes curated attack probes to generate detailed vulnerability reports for each risk type, where monitor agents analyze structured execution traces and issue real-time alerts, enabling both pre-development evaluation and runtime monitoring. We further formalize these safety metrics and present detailed case studies across various representative MAS examples, showcasing the versatility and reliability of TrinityGuard. Overall, TrinityGuard acts as a comprehensive framework for evaluating and monitoring various risks in MAS, paving the way for further research into their safety and security.

Kai Wang, Biaojie Zeng, Zeming Wei +7

Shanghai AI Laboratory

LLM Agent SafetyMulti-Agent System SecurityMulti-Agent LLM SystemsAI Safety EvaluationAI Agent Monitoring
AskPDF
Mar 14, 2026

GroupGuard: A Framework for Modeling and Defending Collusive Attacks in Multi-Agent Systems

While large language model-based agents demonstrate great potential in collaborative tasks, their interactivity also introduces security vulnerabilities. In this paper, we propose and model group collusive attacks, a highly destructive threat in which multiple agents coordinate via sociological strategies to mislead the system. To address this challenge, we introduce GroupGuard, a training-free defense framework that employs a multi-layered defense strategy, including continuous graph-based monitoring, active honeypot inducement, and structural pruning, to identify and isolate collusive agents. Experimental results across five datasets and four topologies demonstrate that group collusive attacks increase the attack success rate by up to 15% compared to individual attacks. GroupGuard consistently achieves high detection accuracy (up to 88%) and effectively restores collaborative performance, providing a robust solution for securing multi-agent systems.

Yiling Tao, Xinran Zheng, Shuo Yang +2

Shenzhen International Graduate School, Tsinghua University · Department of Electrical and Electronic Engineering, The University of Hong Kong · School of Computer Science and Engineering, University of Electronic Science and Technology of China

Graph-Based Anomaly DetectionLLM Agent SecurityMulti-Agent System SecurityAnomaly Detection
AskPDF
Mar 12, 2026

Security Considerations for Artificial Intelligence Agents

This article, a lightly adapted version of Perplexity's response to NIST/CAISI Request for Information 2025-0035, details our observations and recommendations concerning the security of frontier AI agents. These insights are informed by Perplexity's experience operating general-purpose agentic systems used by millions of users and thousands of enterprises in both controlled and open-world environments. Agent architectures change core assumptions around code-data separation, authority boundaries, and execution predictability, creating new confidentiality, integrity, and availability failure modes. We map principal attack surfaces across tools, connectors, hosting boundaries, and multi-agent coordination, with particular emphasis on indirect prompt injection, confused-deputy behavior, and cascading failures in long-running workflows. We then assess current defenses as a layered stack: input-level and model-level mitigations, sandboxed execution, and deterministic policy enforcement for high-consequence actions. Finally, we identify standards and research gaps, including adaptive security benchmarks, policy models for delegation and privilege control, and guidance for secure multi-agent system design aligned with NIST risk management principles.

Ninghui Li, Kaiyuan Zhang, Kyle Polley +1

Perplexity · Purdue University

AI Agent SecurityLLM Agent SecurityAI Agent SafetyMulti-Agent System SecurityTool Access Control for LLM AgentsPrompt Injection Attacks on AI Agents
AskPDF
Mar 12, 2026

Quantum-Secure-By-Construction (QSC): A Paradigm Shift For Post-Quantum Agentic Intelligence

As agentic artificial intelligence systems scale across globally distributed and long lived infrastructures, secure and policy compliant communication becomes a fundamental systems challenge. This challenge grows more serious in the quantum era, where the cryptographic assumptions built into today's AI deployments may not remain valid over their operational lifetime. Here, we introduce quantum secure by construction, or QSC, as a design paradigm that treats quantum secure communication as a core architectural property of agentic AI systems rather than an upgrade added later. We realize QSC through a runtime adaptive security model that combines post quantum cryptography, quantum random number generation, and quantum key distribution to secure interactions among autonomous agents operating across heterogeneous cloud, edge, and inter organizational environments. The approach is cryptographically pluggable and guided by policy, allowing the system to adjust its security posture according to infrastructure availability, regulatory constraints, and performance needs. QSC contributes a governance aware orchestration layer that selects and combines link specific cryptographic protections across the full agent lifecycle, including session bootstrap, inter agent coordination, tool invocation, and memory access. Through system level analysis and empirical evaluation, we examine the trade offs between classical and quantum secure mechanisms and show that QSC can reduce the operational complexity and cost of introducing quantum security into deployed agentic AI systems. These results position QSC as a foundational paradigm for post quantum agentic intelligence and establish a principled pathway for designing globally interoperable, resilient, and future ready intelligent systems.

Arit Kumar Bishwas, Mousumi Sen, Albert Nieto-Morales +1

PricewaterhouseCoopers USA, San Francisco, CA · Independent Researcher, USA, Fremont, CA

Multi-Agent System Security
AskPDF
Mar 11, 2026

WebWeaver: Breaking Topology Confidentiality in LLM Multi-Agent Systems with Stealthy Context-Based Inference

Communication topology is a critical factor in the utility and safety of LLM-based multi-agent systems (LLM-MAS), making it a high-value intellectual property (IP) whose confidentiality remains insufficiently studied. Existing topology inference attempts rely on impractical assumptions, including control over the administrative agent and direct identity queries via jailbreaks, which are easily defeated by basic keyword-based defenses. As a result, prior analyses fail to capture the real-world threat of such attacks. To bridge this realism gap, we propose WebWeaver, an attack framework that infers the complete LLM-MAS topology by compromising only a single arbitrary agent instead of the administrative agent. Unlike prior approaches, WebWeaver relies solely on agent contexts rather than agent IDs, enabling significantly stealthier inference. WebWeaver further introduces a new covert jailbreak-based mechanism and a novel fully jailbreak-free diffusion design to handle cases where jailbreaks fail. Additionally, we address a key challenge in diffusion-based inference by proposing a masking strategy that preserves known topology during diffusion, with theoretical guarantees of correctness. Extensive experiments show that WebWeaver substantially outperforms state-of-the-art (SOTA) baselines, achieving about 60% higher inference accuracy under active defenses with negligible overhead.

Zixun Xiong, Gaoyi Wu, Lingfeng Yao +3

Department of Electrical and Computer Engineering, Stevens Institute of Technology · Department of Electrical and Computer Engineering, University of Houston

Multi-Agent System SecurityMulti-Agent LLM Systems
AskPDF
Mar 10, 2026

AgenticCyOps: Securing Multi-Agentic AI Integration in Enterprise Cyber Operations

Multi-agent systems (MAS) powered by LLMs promise adaptive, reasoning-driven enterprise workflows, yet granting agents autonomous control over tools, memory, and communication introduces attack surfaces absent from deterministic pipelines. While current research largely addresses prompt-level exploits and narrow individual vectors, it lacks a holistic architectural model for enterprise-grade security. We introduce AgenticCyOps (Securing Multi-Agentic AI Integration in Enterprise Cyber Operations), a framework built on a systematic decomposition of attack surfaces across component, coordination, and protocol layers, revealing that documented vectors consistently trace back to two integration surfaces: tool orchestration and memory management. Building on this observation, we formalize these integration surfaces as primary trust boundaries and define five defensive principles: authorized interfaces, capability scoping, verified execution, memory integrity & synchronization, and access-controlled data isolation; each aligned with established compliance standards (NIST, ISO 27001, GDPR, EU AI Act). We apply the framework to a Security Operations Center (SOC) workflow, adopting the Model Context Protocol (MCP) as the structural basis, with phase-scoped agents, consensus validation loops, and per-organization memory boundaries. Coverage analysis, attack path tracing, and trust boundary assessment confirm that the design addresses the documented attack vectors with defense-in-depth, intercepts three of four representative attack chains within the first two steps, and reduces exploitable trust boundaries by a minimum of 72% compared to a flat MAS, positioning AgenticCyOps as a foundation for securing enterprise-grade integration.

Shaswata Mitra, Raj Patel, Sudip Mittal +2

Department of Computer Science, The University of Alabama, Tuscaloosa, USA

AI Agent SecurityMulti-Agent System SecurityCybersecurityTool Access Control for LLM Agents
AskPDF
Mar 9, 2026

Security Considerations for Multi-agent Systems

Multi-agent artificial intelligence systems or MAS are systems of autonomous agents that exercise delegated tool authority, share persistent memory, and coordinate via inter-agent communication. MAS introduces qualitatively distinct security vulnerabilities from those documented for singular AI models. Existing security and governance frameworks were not designed for these emerging attack surfaces. This study systematically characterizes the threat landscape of MAS and quantitatively evaluates 16 security frameworks for AI against it. A four-phase methodology is proposed: constructing a deep technical knowledge base of production multi-agent architectures; conducting generative AI-assisted threat modeling scoped to MAS cybersecurity risks and validated by domain experts; structuring survey plans at individual-threat granularity; and scoring each framework on a three-point scale against the cybersecurity risks. The risks were organized into 193 distinct main threat items across nine risk categories. The expected minimal average score is 2. No reviewed framework achieves majority coverage of any single category. Non-Determinism (mean score 1.231 across all 16 frameworks) and Data Leakage (1.340) are the most under-addressed domains. The OWASP Agentic Security Initiative leads overall at 65.3% coverage and in the design phase; the CDAO Generative AI Responsible AI Toolkit leads in development and operational coverage. These results provide the first empirical cross-framework comparison for MAS security and offer evidence-based guidance for framework selection. Please check back for information on the published journal version.

Tam Nguyen, Moses Ndebugre, Dheeraj Arremsetty

Crew Scaler · North Carolina A&T State University

AI Agent SecurityMulti-Agent System SecurityCybersecurity
AskPDF
Mar 9, 2026

DynaTrust: Defending Multi-Agent Systems Against Sleeper Agents via Dynamic Trust Graphs

Large Language Model-based Multi-Agent Systems (MAS) have demonstrated remarkable collaborative reasoning capabilities but introduce new attack surfaces, such as the sleeper agent, which behave benignly during routine operation and gradually accumulate trust, only revealing malicious behaviors when specific conditions or triggers are met. Existing defense works primarily focus on static graph optimization or hierarchical data management, often failing to adapt to evolving adversarial strategies or suffering from high false-positive rates (FPR) due to rigid blocking policies. To address this, we propose DynaTrust, a novel defense method against sleeper agents. DynaTrust models MAS as a dynamic trust graph~(DTG), and treats trust as a continuous, evolving process rather than a static attribute. It dynamically updates the trust of each agent based on its historical behaviors and the confidence of selected expert agents. Instead of simply blocking, DynaTrust autonomously restructures the graph to isolate compromised agents and restore task connectivity to ensure the usability of MAS. To assess the effectiveness of DynaTrust, we evaluate it on mixed benchmarks derived from AdvBench and HumanEval. The results demonstrate that DynaTrust outperforms the state-of-the-art method AgentShield by increasing the defense success rate by 41.7%, achieving rates exceeding 86% under adversarial conditions. Furthermore, it effectively balances security with utility by significantly reducing FPR, ensuring uninterrupted system operations through graph adaptation.

Yu Li, Qiang Hu, Yao Zhang +3

Tianjin University · Singapore Management University

AI Agent SecurityLLM Agent SecurityMulti-Agent System Security
AskPDF
Mar 8, 2026

From Thinker to Society: Security in Hierarchical Autonomy Evolution of AI Agents

Artificial Intelligence (AI) agents have evolved from passive predictive tools into active entities capable of autonomous decision-making and environmental interaction, driven by the reasoning capabilities of Large Language Models (LLMs). However, this evolution has introduced critical security vulnerabilities that existing frameworks fail to address. The Hierarchical Autonomy Evolution (HAE) framework organizes agent security into three tiers: Cognitive Autonomy (L1) targets internal reasoning integrity; Execution Autonomy (L2) covers tool-mediated environmental interaction; Collective Autonomy (L3) addresses systemic risks in multi-agent ecosystems. We present a taxonomy of threats spanning cognitive manipulation, physical environment disruption, and multi-agent systemic failures, and evaluate existing defenses while identifying key research gaps. The findings aim to guide the development of multilayered, autonomy-aware defense architectures for trustworthy AI agent systems.

Xiaolei Zhang, Lu Zhou, Xiaogang Xu +5

Nanjing University of Aeronautics and Astronautics, China · Collaborative Innovation Center of Novel Software Technology and Industrialization, China · The Chinese University of Hong Kong, China +6

AI Agent SecurityLLM Agent SecurityMulti-Agent System Security
AskPDF
Mar 4, 2026

Auditing Cascading Risks in Multi-Agent Systems via Semantic-Geometric Co-evolution

Large Language model (LLM)-based Multi-Agent Systems (MAS) are prone to cascading risks, where early-stage interactions remain semantically fluent and policy-compliant, yet the underlying interaction dynamics begin to distort in ways that amplify latent instability or misalignment. Traditional auditing methods that focus on per-message semantic content are inherently reactive and lagging, failing to capture these early structural precursors. In this paper, we propose a principled framework for cascading-risk detection grounded in semantic--geometric co-evolution. We model MAS interactions as dynamic graphs and introduce Ollivier--Ricci Curvature (ORC) -- a discrete geometric measure -- to characterize information redundancy and bottleneck formation in communication topologies. By coupling semantic flow signals with graph geometry, the framework learns the normal co-evolutionary dynamics of trusted collaboration and treats deviations from this coupled manifold as early-warning signals. Experiments on a suite of cascading-risk scenarios aligned with the risk category demonstrate that curvature anomalies systematically precede explicit semantic violations by several interaction turns, enabling proactive intervention. Furthermore, the local nature of Ricci curvature provides principled interpretability for root-cause attribution, identifying specific agents or links that precipitate the collapse of trustworthy collaboration.

Zixun Luo, Yuhang Fan, Hengyu Lin +2

Huazhong University of Science and Technology · Lingnan University · Tsinghua University +1

Ollivier-Ricci CurvatureMulti-Agent LLM SystemsAI Agent AuditingMulti-Agent SystemsMulti-Agent System Security
AskPDF
Mar 4, 2026

From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration

Large Language Model-based Multi-Agent Systems (LLM-MAS) are increasingly applied to complex collaborative scenarios. However, their collaborative mechanisms may cause minor inaccuracies to gradually solidify into system-level false consensus through iteration. Such risks are difficult to trace since errors can propagate and amplify through message dependencies. Existing protections often rely on single-agent validation or require modifications to the collaboration architecture, which can weaken effective information flow and may not align with natural collaboration processes in real tasks. To address this, we propose a propagation dynamics model tailored for LLM-MAS that abstracts collaboration as a directed dependency graph and provides an early-stage risk criterion to characterize amplification risk. Through experiments on six mainstream frameworks, we identify three vulnerability classes: cascade amplification, topological sensitivity, and consensus inertia. We further instantiate an attack where injecting just a single atomic error seed leads to widespread failure. In response, we introduce a genealogy-graph-based governance layer, implemented as a message-layer plugin, that suppresses both endogenous and exogenous error amplification without altering the collaboration architecture. Experiments show that this approach prevents final infection in at least 89% of runs across operating modes and significantly mitigates the cascading spread of minor errors.

Yizhe Xie, Congcong Zhu, Xinyue Zhang +5

Faculty of Data Science, City University of Macau · Minzu University of China

Multi-Agent LLM SystemsMulti-Agent SystemsLLM Agent ReliabilityMulti-Agent System SecurityMulti-Agent Consensus
AskPDF
Mar 4, 2026

Cross-Layer Semantic Flow Reconstruction for Attack Detection in Agentic Systems

Agentic systems increasingly orchestrate complex, tool-using workflows within agentic execution environments, where high-level goals and tool invocations at the application layer materialize as process, file, and network activities at the operating-system layer. This cross-layer execution creates security risks that conventional input guardrails cannot capture, because malicious intent may become observable only through downstream execution effects. In multi-agent deployments, inter-agent communication and delegation introduce additional propagation paths. To address this gap, we propose AScope, an execution-aware framework that correlates application-level agent semantics with kernel-level audit events and reconstructs them as cross-layer semantic flows. AScope connects fragmented operations into causal behavioral trajectories and uses a supervisor LLM to identify data flow violations, control flow deviations, and intent inconsistencies. We evaluate AScope on published AgentDojo traces with application-layer evidence and on ten multi-agent scenarios with cross-layer telemetry. The results demonstrate strong detection sensitivity across both evidence settings and achieve node- and path-level F1-scores of 85.3% and 66.7% on the cross-layer dataset.

Qizhi Cai, Yangyang Wei, Zhipeng Chen +2

Department of Software Technology, Zhejiang University, Ningbo, China · Department of Computer Science and Technology, Zhejiang University, Hangzhou, China

AI Agent SecurityLLM Agent SecurityAI Agent MonitoringMulti-Agent System Security
AskPDF
Feb 23, 2026

Thought Virus: Viral Misalignment via Subliminal Prompting in Multi-Agent Systems

Subliminal prompting is a phenomenon in which language models are biased towards certain concepts or traits through prompting with semantically unrelated tokens. While prior work has examined subliminal prompting in user-LLM interactions, potential bias transfer in multi-agent systems and its associated security implications remain unexplored. In this work, we show that a single subliminally prompted agent can spread a weakening but persisting bias throughout its entire network. We measure this phenomenon across 6 agents using two different topologies, observing that the transferred concept maintains an elevated response rate throughout the network. To exemplify potential misalignment risks, we assess network performance on multiple-choice TruthfulQA, showing that subliminal prompting of a single agent may degrade the truthfulness of other agents. Our findings reveal that subliminal prompting introduces a new attack vector in multi-agent security, with implications for the alignment of such systems. The implementation of all experiments is publicly available at https://github.com/Multi-Agent-Security-Initiative/thought_virus .

Moritz Weckbecker, Jonas Müller, Ben Hagag +1

Fraunhofer HHI, Berlin, Germany · Eleuther AI · TU Berlin, Berlin, Germany +2

Multi-Agent System SecurityMulti-Agent SystemsMulti-Agent LLMs
AskPDF
Feb 23, 2026

Graph-theoretic Agreement Framework for Multi-agent LLM Systems

The shift from monolithic LLMs to distributed multi-agent architectures demands new frameworks for verifying and securing autonomous coordination. Unlike traditional multi-agent systems focused on cooperative state alignment, modern LLM patterns: multi-agent debate, constitutional oversight, helper-critic loops-rely on adversarial critique for error correction and reasoning refinement. Since LLMs are dynamical systems whose latent states are imperfectly observable from verbalized outputs, securing these networks requires understanding both macroscopic topology and microscopic agent observability. This paper establishes a rigorous graph-theoretic framework for analyzing consensus in signed, directed interaction networks, bridging graph theory and LLM reasoning by formally mapping Transformer cross-entropy log-odds to the signed Laplacian. We characterize agreement stability through structural balance theory, showing how unbalanced critique cycles produce logical frustration and persistent reasoning oscillations, and prove that unobservable latent states from hidden system prompts act as topological Trojan horses that destabilize cooperative consensus. To resolve unobservable deadlocks, we restrict interaction topologies to chordal graphs and apply matrix decomposition with Gram-Schmidt orthogonalization, proving that rank-one spectral edge perturbations deterministically break expertise symmetry by shifting eigenvalues into the stable left-half plane. Core contributions include consensus theorems, polynomial-time Perfect Elimination Ordering verification algorithms, and large-scale empirical validation on clustered ensembles of LLaMA-3, Mistral, and Gemma agents.

Muhammad Umar Javed

Multi-Agent LLM SystemsMulti-Agent ConsensusMulti-Agent System SecurityOpinion Dynamics
AskPDF
Feb 18, 2026

Formal Policy Enforcement for Agentic Systems

As agentic computing systems grow increasingly popular, they raise serious security concerns due to their susceptibility to various attacks. While prior research efforts aim to mitigate these concerns, they are often too coarse-grained and offer no formal correctness guarantees. In this work, we present a novel approach to agentic security based on runtime monitoring. Our method allows the user to specify their intended policy in Datalog and enforces the specified policy at runtime by constructing event graphs. When an agent is about to execute an action such as a tool call, our approach determines whether the action would violate the policy in light of the current system state and the history of past events. The action is then blocked or permitted accordingly. We implemented our approach and evaluated it across multiple benchmarks, demonstrating its merits in real-world settings. We view this work as a step towards agentic systems that are provably safe and secure.

Nils Palumbo, Sarthak Choudhary, Jihye Choi +4

University of Wisconsin–Madison · Cornell University · Langroid

Runtime Enforcement for AI AgentsAI Agent GovernanceMulti-Agent System Security
AskPDF
Feb 16, 2026

Secure and Energy-Efficient Wireless Agentic AI Networks

In this paper, we introduce a secure wireless agentic AI network comprising one supervisor AI agent and multiple other AI agents to provision quality of service (QoS) for users' reasoning tasks while ensuring confidentiality of private knowledge and reasoning outcomes. Specifically, the supervisor AI agent can dynamically assign other AI agents to participate in cooperative reasoning, while the unselected AI agents act as friendly jammers to degrade the eavesdropper's interception performance. To extend the service duration of AI agents, an energy minimization problem is formulated that jointly optimizes AI agent selection, base station (BS) beamforming, and AI agent transmission power, subject to latency and reasoning accuracy constraints. To address the formulated problem, we propose two resource allocation schemes, ASC and LAW, which first decompose it into three sub-problems. Specifically, ASC optimizes each sub-problem iteratively using the proposed alternating direction method of multipliers (ADMM)-based algorithm, semi-definite relaxation (SDR), and successive convex approximation (SCA), while LAW tackles each sub-problem using the proposed large language model (LLM) optimizer within an agentic workflow. The experimental results show that the proposed solutions can reduce network energy consumption by up to 59.1% compared to other benchmark schemes. Furthermore, the proposed schemes are validated using a practical agentic AI system based on Qwen, demonstrating satisfactory reasoning accuracy across various public benchmarks.

Yuanyan Song, Kezhi Wang, Xinmian Xu

Department of Computer Science, Brunel University of London, UB8 3PH, UK · Nanjing University of Posts and Telecommunications

Wireless Resource AllocationWireless CommunicationsMulti-Agent System Security
AskPDF
Nov 28, 2025

MAS-Shield: A Defense Framework for Secure and Efficient LLM MAS

Large Language Model (LLM)-based Multi-Agent Systems (MAS) are susceptible to linguistic attacks that can trigger cascading failures across the network. Existing defenses face a fundamental dilemma: lightweight single-auditor methods are prone to single points of failure, while robust committee-based approaches incur prohibitive computational costs in multi-turn interactions. To address this challenge, we propose MAS-Shield, a secure and efficient defense framework designed with a coarse-to-fine filtering pipeline. Rather than applying uniform scrutiny, MAS-Shield dynamically allocates defense resources through a three-stage protocol: (1) Critical Agent Selection strategically targets high-influence nodes to narrow the defense surface; (2) Light Auditing employs lightweight sentry models to rapidly filter the majority of benign cases; and (3) Global Consensus Auditing escalates only suspicious or ambiguous signals to a heavyweight committee for definitive arbitration. This hierarchical design effectively optimizes the security-efficiency trade-off. Experiments demonstrate that MAS-Shield achieves a 92.5% recovery rate against diverse adversarial scenarios and reduces defense latency by over 70% compared to existing methods.

Kaixiang Wang, Zhaojiacheng Zhou, Bunyod Suvonov +8

Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai 200240, China

Multi-Agent LLM SystemsLLM AuditingLLM Agent SecurityMulti-Agent System Security
AskPDF
May 28, 2025

Seven Security Challenges in Cross-domain Multi-agent LLM Systems

Large language models (LLMs) are rapidly evolving into autonomous agents that cooperate across organizational boundaries, enabling joint disaster response, supply-chain optimization, and other tasks that demand decentralized expertise without surrendering data ownership. Yet, cross-domain collaboration shatters the unified trust assumptions behind current alignment and containment techniques. An agent benign in isolation may, when receiving messages from an untrusted peer, leak secrets or violate policy, producing risks driven by emergent multi-agent dynamics rather than classical software bugs. This position paper maps the security agenda for cross-domain multi-agent LLM systems. We introduce seven categories of novel security challenges, for each of which we also present plausible attacks, security evaluation metrics, and future research guidelines.

Ronny Ko, Jiseong Jeong, Shuyuan Zheng +4

Osaka University · Seoul National University · Yonsei University

Multi-Agent LLM SystemsLLM Agent SecurityMulti-Agent System Security
AskPDF