cs.CRMar 16, 2026

TrinityGuard: A Unified Framework for Safeguarding Multi-Agent Systems

Authors: Kai Wang, Biaojie Zeng, Zeming Wei, Chang Jin, Hefeng Zhou, Xiangtian Li, Chao Yang, Jingjing Qu, +2 more

Organizations: Shanghai AI Laboratory

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

With the rapid development of LLM-based multi-agent systems (MAS), their significant safety and security concerns have emerged, which introduce novel risks going beyond single agents or LLMs. Despite attempts to address these issues, the existing literature lacks a cohesive safeguarding system specialized for MAS risks. In this work, we introduce TrinityGuard, a comprehensive safety evaluation and monitoring framework for LLM-based MAS, grounded in the OWASP standards. Specifically, TrinityGuard encompasses a three-tier fine-grained risk taxonomy that identifies 20 risk types, covering single-agent vulnerabilities, inter-agent communication threats, and system-level emergent hazards. Designed for scalability across various MAS structures and platforms, TrinityGuard is organized in a trinity manner, involving an MAS abstraction layer that can be adapted to any MAS structures, an evaluation layer containing risk-specific test modules, alongside runtime monitor agents coordinated by a unified LLM Judge Factory. During Evaluation, TrinityGuard executes curated attack probes to generate detailed vulnerability reports for each risk type, where monitor agents analyze structured execution traces and issue real-time alerts, enabling both pre-development evaluation and runtime monitoring. We further formalize these safety metrics and present detailed case studies across various representative MAS examples, showcasing the versatility and reliability of TrinityGuard. Overall, TrinityGuard acts as a comprehensive framework for evaluating and monitoring various risks in MAS, paving the way for further research into their safety and security.

Figures & tables

Figure 1: The overall architecture design of TrinityGuard.
IDRisk NameOWASP Ref.Description
1.1Prompt InjectionLLM01Manipulating agent behavior via malicious input
1.2Jailbreak AttackLLM01Bypassing safety guidelines and ethical constraints
1.3Sensitive Info DisclosureLLM02Leaking system prompts, API keys, or private data
1.4Excessive AgencyLLM06Executing actions beyond intended scope
1.5Unauthorized Code ExecutionASI05Running malicious code or commands
1.6HallucinationLLM09Fabricating false information
Table 1: Risk Tier 1 (RT1): Single-agent atomic risks.
IDRisk NameOWASP Ref.Description
2.1Malicious PropagationASI07Malicious instructions spreading through agent network
2.2Misinformation AmplificationLLM09Errors amplified and reinforced across agents
2.3Insecure Output HandlingLLM05Unvalidated output consumed by downstream agents
2.4Goal DriftASI01Deviation from original objective over multi-turn interaction
2.5Message TamperingASI07Message content modified during transmission
2.6Identity SpoofingASI03Impersonation of a trusted agent
Table 2: Risk Tier 2 (RT2): Inter-agent communication risks.
IDRisk NameOWASP Ref.Description
3.1Cascading FailureASI08Single-point failure triggering system-wide collapse
3.2Sandbox EscapeASI05Agents accessing unauthorized resources
3.3Insufficient MonitoringASI09Lack of effective behavioral monitoring and audit
3.4Group HallucinationLLM09Collective fabrication of false information
3.5Malicious EmergenceASI01Emergence of unanticipated harmful behaviors
3.6Rogue AgentASI10Agent deviating from system objectives
Table 3: Risk Tier 3 (RT3): System-level emergent risks.
Figure 2: Summary of three-tier risks studied by TrinityGuard.
Figure 3: Fine-grained distribution of detected safety risks across the three TrinityGuard tiers for 300 synthesized MAS workflows.
MAS TypeRisk Tier 1Risk Tier 2Risk Tier 3Average Pass Rate
QA9.2%8.0%2.7%6.9%
coding5.0%12.3%0.1%5.7%
database5.3%15.0%0.0%6.6%
research9.4%13.7%0.0%7.8%
logistics4.7%17.7%3.5%8.2%
routing4.7%12.5%1.0%5.9%
Table 4: Safety evaluation Pass Rates ( ↑\uparrow ) across 300 synthesized MAS workflows.
 MASFinancialGameTravelDeep
TierRiskAnalysisDesignPlannerResearch
RT1: AtomicPrompt Injection0/40/40/40/4
Jailbreak Attack0/40/40/40/4
Sensitive Info Disclosure2/42/42/44/4
Excessive Agency0/30/30/30/3
Unauth. Code Execution0/30/31/30/3
Table 5: Safety evaluation Pass Cases ( ↑\uparrow ) across four representative MAS.

Explore similar work

CardsList
Sep 1, 2026

SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems

Safe agents can fail together. Multi-agent LLM systems (MAS) move information, state, decisions, and authority across principal boundaries, creating failures that local checks may miss. Without an execution-level view, a multi-agent setting can easily be mistaken for evidence of a genuinely multi-agent security effect. We thus systematize MAS security through an execution-centered analysis of 197 works, covering six interaction interfaces, four adversary positions, seven system-level risks, and eight recurring attack paths. We introduce an A-I-R framework that organizes attacks by adversary position, interaction interface, and resulting system-level risk, unifying otherwise fragmented attack mechanisms across MAS. We organize defenses through a five-part contract covering path target, observation, intervention, trust boundary, and recovery, and identify path closure and recovery as key challenges. We audit 44 evaluation and benchmark works and identify open challenges in isolating interaction effects, designing comparable and diagnostic metrics, supporting reuse across MAS designs, and evaluating open-system operation. Together, these findings motivate an interaction-aware view of MAS security: trace attacks end to end, test whether defenses close those paths, and evaluate system-level effects with appropriate counterfactuals.

Rui Yang, Junjie Xu, Zhengyu Liu +4

Johns Hopkins University · Nanyang Technological University

AI Agent Security BenchmarksMulti-Agent System SecurityMulti-Agent LLMs
AskPDF
May 8, 2026

PropGuard: Safeguarding LLM-MAS via Propagation-Aware Exploration and Remediation

LLM-based multi-agent systems (LLM-MAS) have become a promising paradigm for solving complex tasks through role specialization, tool use, memory, and collaborative reasoning. However, these interactions create new security risks that malicious instructions injected through messages, tools, or memories can propagate across agents and rounds, causing system-level compromise. Existing defenses largely rely on local filtering or graph-based anomaly detection, but they often fail to trace fine-grained propagation paths or remediate contaminated states without disrupting benign collaboration. We propose PropGuard, a propagation-aware framework for safeguarding LLM-MAS. PropGuard constructs a dual-view spatio-temporal graph that combines response-centric risk estimation with full-state evidence preservation. Guided by these risk priors, a GE-GRPO trained inspector sequentially explores the full-state graph to recover compact suspicious propagation subgraphs. PropGuard then verifies harmful propagation through subgraph-aware diagnosis and applies source-guided remediation to correct upstream contamination and replay affected downstream interactions. Experiments across four communication architectures and five attack settings demonstrate that PropGuard consistently lowers attack success while maintaining high task-level defense success, achieving a favorable effectiveness--efficiency trade-off.

Bingyu Yan, Xiaoming Zhang, Jinyu Hou +4

Beihang University · Beijing University of Posts and Telecommunications · City University of Hong Kong

LLM Agent SecurityPrompt Injection Attacks on AI AgentsMulti-Agent System Security
AskPDF
Nov 28, 2025

MAS-Shield: A Defense Framework for Secure and Efficient LLM MAS

Large Language Model (LLM)-based Multi-Agent Systems (MAS) are susceptible to linguistic attacks that can trigger cascading failures across the network. Existing defenses face a fundamental dilemma: lightweight single-auditor methods are prone to single points of failure, while robust committee-based approaches incur prohibitive computational costs in multi-turn interactions. To address this challenge, we propose MAS-Shield, a secure and efficient defense framework designed with a coarse-to-fine filtering pipeline. Rather than applying uniform scrutiny, MAS-Shield dynamically allocates defense resources through a three-stage protocol: (1) Critical Agent Selection strategically targets high-influence nodes to narrow the defense surface; (2) Light Auditing employs lightweight sentry models to rapidly filter the majority of benign cases; and (3) Global Consensus Auditing escalates only suspicious or ambiguous signals to a heavyweight committee for definitive arbitration. This hierarchical design effectively optimizes the security-efficiency trade-off. Experiments demonstrate that MAS-Shield achieves a 92.5% recovery rate against diverse adversarial scenarios and reduces defense latency by over 70% compared to existing methods.

Kaixiang Wang, Zhaojiacheng Zhou, Bunyod Suvonov +8

Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai 200240, China

Multi-Agent LLM SystemsLLM AuditingLLM Agent SecurityMulti-Agent System Security
AskPDF