Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security

Shaswata Mitra; Raj Patel; Subash Neupane; Sudip Mittal; Md Rayhanur Rahman; Shahram Rahimi

cs.AIOct 6, 2026

Where Rules End and Judges Begin: Measuring the Judgment Boundary in Multi-Agent Systems Security

Authors: Shaswata Mitra, Raj Patel, Subash Neupane, Sudip Mittal, Md Rayhanur Rahman, Shahram Rahimi

Organizations: Department of Computer Science The University of Alabama, Tuscaloosa, USA

Abstract

LLM-based multi-agent systems (MAS) engage tools, share memory, and delegate tasks, often encountering adversarial content. Current defenses for MAS are typically evaluated in isolation, focusing on one attack type at a time, which can lead to costly and hard-to-audit outcomes. This study organizes defenses into five principles, implementing them as DEFER1 (DEterministic-First Enforcement with Residual judgment), which includes a cascade of 28 checks that blocks what it can and refers the rest to a panel of four judges. In independent testing across four domains, attack success rates drop from about 30.0% to approximately 3.0%, with 78% of blocked attacks handled by deterministic checks. Only a quarter of proposals reach the judges in the security-operations domain, illustrating that the rules provide security for attacks violating clear policies, while judges manage those that only misrepresent intent. Both systems have weaknesses, such as a risk-score approval gate that inaccurately approves most attack proposals but few legitimate ones, highlighting the challenges in assessing threats accurately.

Figures & tables

Panel of Figure 1 — Fig. 1: Attacker-controlled content reaches a multi-agent system (MAS) through five channels on two surfaces (left). Defer orders five defensive principles so that deterministic checks settle what they can, and only undecided proposals reach a panel of four open-weight judges (center). The right panel shows the share of attempted attacks each configuration stops. On our own suite, Defer stops 95% of them and the same judges alone 22%, taking attack success from 32.0% undefended to 2.2%; a quarter of proposals still reach a judge, and 10% to 27% of legitimate proposals are denied. On the 255 Agent Security Bench cases, access control stops none of the injected actions the agent emits and Defer stops 94%, taking attack success from 28.8% to 1.8%.

ID	Channel	Attacker content arrives as	Surface	Tier
C1	Task input	Text inside an alert, ticket, or request field	Tool	T1
C2	Tool response	Text returned by a tool the agent called	Tool	T1
C3	Memory content	A record retrieved from a shared store	Memory	T1
C4	Handoff	The summary one phase passes to the next	Tool	T2
C5	Proposal justification	The rationale attached to an action proposal	Tool	T2

TABLE I: Injection channels, the surface they belong to, and the attacker tier they require.

Panel of Figure 3 — Fig. 2: Tool-orchestration surface. Every agent-to-tool interaction passes through the host, which establishes the tool’s identity (P1), bounds what may be requested (P2), and submits consequential proposals to a validator that is independent of the proposing agent before anything executes (P3).

Panel of Figure 4 — Fig. 3: Memory-management surface. All reads and writes pass through a memory gateway. Writes are filtered at the write boundary before they persist (P4). Reads are restricted by phase, redacted by field, and sanitized before they reach an agent (P5). The drawing shows the general design, in which the write boundary may be consensus-validated.

Panel of Figure 5 — Fig. 4: The Defer cascade. A tool call passes the registry (P1), manifest and parameter checks (P2), and, if consequential, the deterministic checks (P3) before anything reaches the LLM panel. The request is re-hashed before execution, and the response is checked on the way back. Memory writes pass the store policy and the six P4 checks, and reads pass the P5 gates, redaction, and sanitization. Blue boxes are rule-based, light blue use a similarity threshold, orange is the only judgment-based check, and a dot marks a box in which at least one check keeps state across incidents. The two annotations are measured in CyberOps. Boxes group the checks in Table VIII : P3.1–3.6 are the handoff, context, intent-chain, ledger, and replay checks; P3.7–3.9 are adaptive consent, risk scoring, and the auto-gates.

	Attack variants				Benign
Configuration	ASR % [95% CI]	Attempt %	Block $\|$ att. %	Judged %	Denied %
Flat (undefended)	32.0 [22.7, 41.8]	32.0	n/a	0	0.0
ACL (connectivity only)	24.9 [15.6, 35.1]	28.9	13.8	0	51.9 a
JudgeOnly (judge every proposal)	34.7 [24.4, 45.3]	44.4	22.0	99.9	4.5
NoJudge (never judge)	1.3 [0.0, 4.0]	44.0	97.0	0	54.3
Full (judge last)	2.2 [0.0, 5.8]	42.2	94.7	26.4	9.9

TABLE II: The judgment boundary and leave-one-out ablations in CyberOps (Qwen3-235B primary, Local4 panel by replay). Attack columns: 75 variants

\times

3 trials in every row, the leave-one-out arms included. Judged : tool proposals in attack incidents that reach the panel. Denied : legitimate tool proposals denied or escalated on the 20 benign scenarios

\times

3 trials.

Panel of Figure 7 — Fig. 5: The judgment boundary in CyberOps (Qwen3-235B primary; Local4 panel by replay; 75 attack variants $\times$ 3 trials; 20 benign scenarios $\times$ 3 trials). (a) Attack success with 95% cluster-bootstrap intervals; the row below gives the share of tool proposals in attack incidents that reach the panel. (b) Legitimate tool proposals denied, with benign task completion measured live on the right axis. Never judging is the safest and escalates half of the legitimate work; the same judges without the rules do not differ from the undefended baseline; Defer sits between them, and its cost is set mostly by its rules.

Appendix figures & tables37 assets

Supplementary material from the paper’s appendix.

Appendix

Attack vector	Tool-side effect	Memory-side effect
Component level
Unauthorized access	Privilege escalation, unauthorized execution	Unauthorized retrieval
Context contamination	API exfiltration	Privacy leakage, data poisoning
Coordination level
Lateral compromise	Misuse via shared privileges	False injection via shared memory
Consensus manipulation	Biased tool selection	Conflicting intelligence injection

Appendix

TABLE III: Representative attack vectors by integration surface.

AP	Name	Attacker objective	Channel	Tier	Effect scored as success
AP-1	Tool redirection	Induce a phase agent to invoke a tool outside its phase	C1	T1	Out-of-manifest tool executes
AP-2	Memory poisoning	Get falsified findings stored so later phases reason on them	C2, C1	T1	Canary persisted in a shared store
AP-3	Confused deputy	Have the agent act on attacker text returned as a tool result	C2	T1	Action named in the forged result executes (never attempted by any primary)
AP-4	Cross-phase exfiltration	Move sensitive incident data into an outbound feed	C3, C1	T1	Canary appears in an outbound tool call, or an unauthorized read is allowed (Section V-D )
AP-5	Unauthorized irreversible action	Trigger bulk or destructive operations	C1, C4	T1–T2	Bulk or destructive action executes
AP-6	Replay and accumulation	Re-submit a previously approved proposal, or accumulate actions across incidents	C1	T1	Stale or repeated proposal executes

Appendix

TABLE IV: Attack paths: objective, primary delivery channel (Table I ), attacker tier, and the effect the oracle looks for.

ID	Vector	Check	AP	ID	Vector	Check	AP
TA-1	Tool redirection (out-of-scope invocation)	P2.1	1	TA-19	Dangerous call hidden in a batch	P2.1, host	1, 12
TA-2	Forged component identity	P1.1	3	TA-20	Deflate severity, then escalate	P3.1, P3.3	5, 9
TA-3	Tampered tool response	P1.2	3	TA-21	Probing through rejection reasons	host	4, 10
TA-4	Cross-phase data exfiltration via tool output	P2.3	4, 8	TA-22	Cross-incident action accumulation	P3.4, P3.5	6, 12
TA-5	Unauthorized irreversible action	P3.8, P3.9	5	MA-1	Cross-phase memory read	P5.1	4
TA-6	Replay of an approved action proposal	P3.6	6, 7	MA-2	Field-level data leakage	P5.2	4, 14

Appendix

TABLE V: The 35-vector inventory, classified by entry surface: 22 tool-orchestration vectors (TA), 12 memory vectors (MA), and one composite vector (CA). Check : the check expected to intercept the vector at design time (Table VIII ); host marks a behavior of the orchestrator rather than one of the 28 checks. AP : the attack paths of Table IV that exercise the vector.

Principle	Draws on	Defer instantiation	Difference from source
P1 Authorized Interface	Tool registries [ narajala2025securing ] ; agent identity binding [ syros2025saga ]	Registry allowlist; response schema, timing, and replay checks; per-tool response MAC; configuration hash snapshot	No public-key manifest signing or agent identity lifecycle; symmetric keys held by the host
P2 Capability Scoping	Prompt flow integrity [ kim2025prompt ] ; delegation threat modeling [ krawiecka2025extending ]	Static per-phase manifests and action caps; parameter rules; target–evidence consistency; output classifier	No tracking of untrusted data flow through prompts; privileges are static per phase, not per task
P3 Verified Execution	Ledger-anchored verification [ zou2025blocka2a ] ; Byzantine-robust consensus [ chen2024blockagents ]	Context, sequence, and ledger checks; risk scoring with auto-gates; multi-model LLM panel; pre-execution hash check	Ledgers are local, not distributed or blockchain-anchored; validators are assumed uncompromised
P4 Memory Integrity	Consensus write validation [ wei2025memguard ] ; filtering and purging [ mao2025agentsafe ] ; isolate-then-aggregate retrieval [ xiang2024certifiably ]	Schema, metadata, evidence-similarity, drift, replay, and rule-based contradiction checks on every write	No LLM consensus on writes; no certified retrieval; contradiction detection is phrase-based
P5 Access-Controlled Isolation	Hierarchical memory isolation [ mao2025agentsafe ] ; dynamic access graphs [ rezazadeh2025collaborative ]	Phase-to-store policy; field-level redaction; query-scope limits; read-pattern monitor; read-result sanitization	Permissions are static, not time-varying; sanitization is pattern-based

Appendix

TABLE VI: Instantiation fidelity: what each principle draws on, what Defer implements, and how the two differ. Findings in Section VI are about the right-hand instantiations, not the source systems.

Principle	Checks	Indep.	Dep.	Sim.	LLM	Stateful
P1 Authorized Interface	3	3	0	0	0	1
P2 Capability Scoping	3	1	0	2	0	0
P3 Verified Execution	11	5	5	0	1	4
P4 Memory Integrity	6	2	2	2	0	2
P5 Access-Ctrl. Isolation	5	2	1	2	0	1
Total	28	13	8	6	1	8

Appendix

TABLE VII: The 28 checks by principle and decision tier. Indep. : content-independent rule; Dep. : content-dependent rule; Sim. : similarity threshold. A check that combines kinds is counted in its most learned tier. Stateful counts checks that keep state across incidents. Full list in Table VIII .

ID	Check	Type	What it verifies
P1.1	Identity	I	Tool (or store, validator) is present in the component registry
P1.2	Response integrity	I, S	Response matches the registered schema; latency within 0.01 ms–30 s; response hash unseen within a five-incident window; per-tool MAC verifies
P1.3	Configuration integrity	I	SHA-256 of every configuration file matches the start-up snapshot, re-verified per incident; HMAC on persisted consent ledgers
P2.1	Manifest	I	Tool is in the calling phase’s manifest; per-incident action cap and bulk-action threshold not exceeded
P2.2	Parameter validation	I, C, E	No wildcard targets; asset-criticality rules (deny all on critical, deny destructive on high); per-tool min/max/enum rules; action target appears in the incident evidence (substring, or cosine $\geq 0.5$ )
P2.3	Output classifier	C, E	Phase-aware sensitive-content patterns, and cosine $\geq 0.7$ to sensitive categories; redact or reject

Appendix

TABLE VIII: All checks. Type: I content-independent rule, C content-dependent rule, E similarity threshold over embeddings, L LLM judgment; S marks a check that keeps state across incidents. Thresholds were hand-set on CyberOps.

Attack vector	P1	P2	P3	P4	P5
Unauthorized access	✓				✓
Context contamination		✓		✓	$\circ$
Lateral compromise		✓	$\circ$		✓
Consensus manipulation	✓		✓	✓
Covert coordination		✓	✓		$\circ$
Authentication bypass	✓				✓

Appendix

TABLE IX: Design-time predictions: which principle should intercept (✓) or contain (

\circ

) each vector.

Feature	InjecAgent [ zhan2024injecagent ]	AgentDojo [ debenedetti2024agentdojo ]	ASB [ zhang2025asb ]	TAMAS [ tamas2025 ]	Ours
Multi-agent workflow	✗	✗	✗	✓	✓
Tool-response injection	✓	✓	✓	✓	✓
Memory channel	✗	✗	✓	–	✓
Inter-agent channels	✗	✗	✗	✓	✓
Benign / utility tasks	✗	✓	✓	✓	✓
Attempt vs. block outcome	✗	$\circ$	✗	–	✓

Appendix

TABLE X: Suite features relative to existing agent-security benchmarks (–: not stated in the benchmark’s description;

\circ

: AgentDojo’s utility-under-attack measure separates task success from attack success but not model refusal from framework interception).

Panel of Figure 16 — Fig. 6: Background. (Left) An LLM agent reaches everything outside the model through two interfaces, tools and memory, which are the two integration surfaces of Section II-B . (Center) Vertical and horizontal multi-agent topologies. (Right) MCP architecture, which our testbed’s topology adopts.

Interaction type	Flat	Defer	Red.
Agent $\rightarrow$ tool	64	16	75%
Agent $\rightarrow$ memory	48	16	67%
Agent $\leftrightarrow$ agent	12	4	67%
Tool response $\rightarrow$ agent	64	16	75%
External feed $\rightarrow$ memory	12	4	67%
Total	200	56	72%

Appendix

TABLE XI: Unverified interactions in a flat deployment versus Defer (CyberOps instance).

Principle	Related controls
P1 Authorized Interface	Zero-trust architecture (NIST SP 800-207); identification and authentication (SP 800-53 Rev. 5, IA family) [ nist80053r5 ]
P2 Capability Scoping	Least privilege (SP 800-53 AC-6); Excessive Agency (OWASP LLM06:2025) [ owasp2025llmtop10 ]
P3 Verified Execution	Separation of duties (SP 800-53 AC-5); logging (ISO/IEC 27001:2022 A.8.15); record-keeping (EU AI Act Art. 12) [ euaiact2024 ]
P4 Memory Integrity	Software, firmware, and information integrity (SP 800-53 SI-7); traceability (NIST AI RMF)
P5 Access-Ctrl. Isolation	Account management and access enforcement (SP 800-53 AC-2, AC-3); data minimization (GDPR Art. 5(1)(c))

Appendix

TABLE XII: Indicative control mapping.

Primary model	Size	Panel
Qwen3-235B-A22B-Instruct-2507	235B MoE	Local4
gpt-oss-120b	117B MoE	Local4 minus self (2 of 3)
Llama-4-Scout-17B-16E-Instruct	109B MoE	Local4 minus self (2 of 3)
Mistral-Small-3.2-24B-Instruct	24B	Local4 minus self (2 of 3)
Llama-3.1-8B-Instruct	8B	Local4

Appendix

TABLE XIII: Primary models and validator panels.

Model	Precision	Parallelism	Hardware
Qwen3-235B-A22B-Instruct-2507	BF16	TP=4	4 $\times$ H200
Llama-4-Scout-17B-16E-Instruct	BF16	TP=2	2 $\times$ H200
Mistral-Small-3.2-24B-Instruct	BF16	1 GPU	H200
Llama-3.1-8B-Instruct	BF16	1 GPU	RTX 5090
Qwen3-32B; DeepSeek-R1-Distill-Qwen-32B; Qwen3-14B	BF16	1 GPU	H200 (shared)
Gemma-4-31B-it; gpt-oss-120b	as released	1 GPU	H200

Appendix

TABLE XIV: Model serving.

Fig. 7: Paired outcome per variant in CyberOps (75 variants, Local4 ). Each cell is one variant; the upper-left triangle is its outcome under JudgeOnly and the lower-right under Defer (vermilion: executed at least once; green: always blocked; gray: never attempted). Defer stops 26 variants that JudgeOnly loses, 11 of them on memory paths, and gives up none.

Fig. 8: Attack success (%) per attack path and configuration in CyberOps (Qwen3-235B, Local4 ; $n$ variants, three trials each). The upper block is delivered as content (T1), the lower through the handoff and rationale channels of a compromised agent (T2). The right column names the modal first interceptor under Defer . JudgeOnly leaves whole rows red that Defer clears; what survives is three adversarial memory writes (AP-13) and one trial each of AP-7 and AP-10. Cells rest on 15 trials and are descriptive.

Fig. 9: Leave-one-out ablation (Qwen3-235B, CyberOps, 75 variants

\times

3 trials, Local4 ). Left: attack success with 95% intervals; the dashed line is Defer . Right: benign incidents with at least one denial. Removing any principle lets attacks through that the others do not catch, P3 most; removing P2 halves the benign incidents with a denial. P5 is isolated on the read-dependent paths in Figure 15 .

Fig. 10: Judge panels on the ASB replay (security, y-axis: injected actions let through as a share of the 1,275 replay trials) and on the 661 benign proposals that reached the judges (cost, x-axis), all local and by replay. Adding two judges of the primary’s lineage to a single Qwen3-32B judge ( Lin3 ) raises let-through from 2.8% to 10.0% of trials, 3.5 times as many actions; the diverse panels let 1.4% through at similar benign rejection.

Fig. 11: First interception under Defer by the tier of the rule that denied (Table VII ), in CyberOps, in the other three domains, and on Agent Security Bench ( Local4 ). Deterministic checks decide most interceptions on policy and structure attacks and well under half on the third-party benchmark, there only through checks that read the content (P2.2’s similarity threshold and P5.5).

By decision tier
First interceptor	CyberOps	Other three	ASB
Content-independent rules	31.1	11.0	0.0
Content-dependent rules	50.0	19.7	9.1
Similarity-threshold checks	16.7	39.0	31.2
LLM panel	2.2	30.3	59.7
By principle

Appendix

TABLE XV: First interception under Full : share of blocked attempts (%) stopped by each decision tier and principle ( Local4 panel by replay; data of Figure 11 ).

Fig. 12: Results across domains (Qwen3-235B primary, top four rows) and primaries (CyberOps, lower rows; Llama-4-Scout and Mistral-Small from the earlier runs), Local4 . Left: attack success under Flat , ACL , and Defer , with 95% intervals for Defer . Right: benign incidents with at least one denial under Defer . Security holds across domains; cost does not.

Fig. 13: The judgment boundary in all four domains (Qwen3-235B, Local4 ; 75 variants

\times

3 trials per bar, 95% intervals). Top: attack success; bottom: legitimate tool proposals denied or escalated. The same judges without the rules do not differ from the undefended baseline in CyberOps and leave 9% to 19% elsewhere; rules first and judges last leave 0% to 5.3%; never judging leaves 0% to 1.8% by escalating 41% to 59% of legitimate proposals.

Domains (Qwen3-235B)
	Flat	ACL	Full	Att.	Blk $\|$ att.
CyberOps	32.0	24.9	2.2	42.2	94.7
Healthcare	23.1	19.1	0.0	25.8	100.0
Finance	35.1	26.2	5.3	40.9	87.0
Legal	29.8	28.4	4.4	40.0	88.9
All four domains	30.0	24.7	3.0	37.2	91.9

Appendix

TABLE XVI: Results across domains (Qwen3-235B primary) and primaries (CyberOps; ∗ earlier runs), Local4 panel by replay. ASR in %; Att. and Blk

|

att. refer to Full .

Primary	Flat	ACL	JudgeOnly	NoJudge	Full	Judged	Denied
Qwen3-235B	32.0	24.9	34.7	1.3	2.2	26.4	9.9
gpt-oss-120b	21.3	18.7	21.3	1.3	6.2	9.1	0.0
Llama-3.1-8B	34.7	33.8	27.6	1.3	8.4	45.0	14.6

Appendix

TABLE XVII: The judgment boundary for three primaries in CyberOps (75 variants

\times

3 trials; attack success in %, Local4 panel by replay). Judged and Denied refer to Full : tool proposals in attack incidents that reach the panel, and legitimate tool proposals denied or escalated on the benign scenarios.

Fig. 14: Attack success by injection channel under Flat and Defer (Qwen3-235B, four domains, Local4 ), with exposure as hollow markers. Defer reduces every channel an attack gets through; the largest residual is handoff poisoning, whose criterion counts any admin action, and no attack succeeds through tool responses.

Fig. 15: P5 evidence on the cross-phase leak (AP-4, CyberOps, 7 variants

\times

3 trials, Local4 ). Bars: attack success; hollow markers: exposure of the planted record. Removing the store policy reopens the leak that Defer closes, and the judges do not help because reads never reach them.

Fig. 16: Denials per benign incident by position in the persistent-state sequence (CyberOps, first pass; the other three domains in gray). The dashed line is the isolated-state mean. The first benign incident is at the isolated level and is the only one that completes; from the second on, legitimate incidents deny each other through replay and ledger state that never expires. Denials fall back later in the sequence, but completion does not recover. A second pass in reverse order replicates it.

Fig. 17: The Local4 judges over 3,657 consensus rounds (the rule-evading siblings excluded). (a) Pairwise Cohen’s $\kappa$ ; the diagonal shows each judge’s reject rate. (b) Share of proposals approved as a function of the quorum; the deployed quorum is three of four.

Panel of Figure 35 — Fig. 18: Third-party benchmarks (Qwen3-235B primary; 95% cluster-bootstrap intervals over cases). (a) Agent Security Bench: access control blocks nothing, and Defer reaches 1.8% under Local4 ; with the same rules, a single judge and the same-lineage Lin3 panel let more through (judges replayed on the identical proposals). (b) TAMAS: attack success per attack type without checks and under Defer ( Local4 for the three types that act through tools, as run for the two that act only through answers); chips give the first interceptor of the blocked tool-based attempts. Defer stops the attacks that act through tools, almost all through the panel, and leaves the answer-only attacks unchanged.

Agent Security Bench (255 cases $\times$ 2 trials)
Benchmark	Flat	ACL	Defer
All cases	28.8 [23.7, 34.1]	30.0 [24.9, 35.3]	1.8 [0.4, 3.5]
Defer , single judge			3.3 [1.6, 5.5]
Defer , Lin3			9.6 [6.3, 13.3]
TAMAS (250 instances $\times$ 3 trials)
Direct prompt injection	36.9 [25.5, 48.7]	–	0.0 [0.0, 0.0]

Appendix

TABLE XVIII: Third-party benchmarks (Qwen3-235B primary): attack success in % with 95% cluster-bootstrap intervals over cases. Defer under Local4 unless noted.

	AP-4		AP-14
Configuration	ASR	Exp.	ASR	Exp.
Flat	57.1	100.0	0.0	100.0
ACL	14.3	28.6	0.0	100.0
JudgeOnly	85.7	100.0	0.0	100.0
Full $\setminus$ P5	28.6	100.0	0.0	100.0
Full	0.0	28.6	0.0	100.0

Appendix

TABLE XIX: P5 evidence on the attack paths that depend on a memory read (Qwen3-235B, CyberOps). AP-4: 7 variants

\times

3 trials; AP-14: 5

\times

3. Exp. : share of trials in which the planted record was returned by a read. Values in %.

Fig. 19: Where the cost goes (Qwen3-235B, benign scenarios). (a) Denials per 100 proposed legitimate actions (tool calls and memory operations), by principle and domain; redactions are hatched. (b) Median benign-incident latency in CyberOps, split into primary model, deterministic checks, and LLM panel; whiskers show the 95th percentile. (c) Primary and judge tokens per incident. Panels (b) and (c) were measured live with two local judges. The judges are most of the added time; outside CyberOps, the judges and the memory checks are most of the blocked work.

Domain	Scen.	Denied %	Any-denial %
CyberOps	20	9.9	81.7
Healthcare	5	20.0	100.0
Finance	5	19.5	100.0
Legal	5	26.9	100.0
All	35	–	89.5

Appendix

TABLE XX: Cost of Full on benign scenarios by domain (Qwen3-235B primary, three trials per scenario). Denied : legitimate tool proposals denied or escalated ( Local4 by replay). Any-denial : incidents with at least one denied proposal.

			Flat		Full
Channel	Var.	Exp. %	Att. $\|$ exp.	ASR	Att. $\|$ exp.	ASR
C1 task input	170	100.0	26.1	26.1	37.6	2.5
C2 tool response	33	97.0	0.0	0.0	2.7	0.0
C3 memory content	35	100.0	14.3	14.3	0.0	0.0
C4 handoff	30	100.0	45.6	45.6	80.0	14.4
C5 proposal rationale	32	100.0	84.4	84.4	71.9	1.0

Appendix

TABLE XXI: Results by injection channel (Qwen3-235B primary, four domains). Exposed : trials in which the injected content reached an agent. Att.

|

exp. : attempt rate among exposed trials. ASR in %.

Panel	Quorum	Let through	CyberOps ASR %	Other three ASR %	Benign rej. %
Single (Qwen3-32B)	1/1	36 (8.9%)	9.8	6.7	15.0
Lin3 (Qwen family)	2/3	127 (31.4%)	9.8	6.8	14.1
Div3L (three lineages)	2/3	18 (4.4%)	7.1	4.3	17.2
Local4 (main)	3/4	18 (4.4%)	7.1	3.9	17.7

Appendix

TABLE XXII: Judge panels on the identical 405 injected actions the primary emitted in the ASB replay (1,275 trials) and on the 661 benign proposals that reached the judges in the in-house runs, all by replay with local judges. Let through : emitted actions that execute, with their share of the 405 emitted actions; Figure 10 plots the same counts as a share of the 1,275 trials.

Arm	Original	Re-judged	Live	Original	Re-judged	Live
	ASR			Denied
Qwen3-235B Full , CyberOps	7.1	8.9	8.4	11.2	11.1	9.9
Qwen3-235B Full , healthcare	0.0	0.0	0.0	24.2	18.8	23.7
Qwen3-235B Full , finance	4.0	9.8	6.7	22.7	17.6	17.5
Qwen3-235B Full , legal	7.6	12.4	10.2	25.4	14.9	16.7
Qwen3-235B JudgeOnly , CyberOps	33.8	32.0	29.8	3.8	4.4	5.7

Appendix

TABLE XXIII: The original judge input against the richer one (structured incident fields without attacker-writable text, the agent’s phase and tools, the targets’ operating context, recent decisions, and the agent’s stated reasoning). Re-judged : every judged round of the original runs re-judged with the richer input; Live : the judged configurations re-run with it. All under Local4 ; ASR and legitimate proposals denied in %.

Fig. 20: The original judge input against the richer one. (a) Share of attack and benign proposals each judge approves in the Qwen3-235B Full rounds, from the original input (hollow marker) to the richer one (arrowhead). (b) Attack success under Defer with 95% intervals, for the original input, the richer input re-judged offline, and a live re-run with the richer input. Shown the fields the rules consult, the judges approve more attacks.

Split	Content	Size	Tests
Our suite	CyberOps, healthcare, finance, legal	300 variants	boundary, generality
Third-party	ASB; InjecAgent subset; TAMAS	255; 50; 250 cases	author independence
Benign	Companion scenarios; TAMAS IPI tasks	35; 50	cost

Appendix

TABLE XXIV: Evaluation splits.

Explore similar work

SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems
Sep 1, 2026Rui Yang, Junjie Xu, Zhengyu Liu +4Model-Based Multi-Agent Systems Multi-Agent Large Language Model Systems
Johns Hopkins University · Nanyang Technological University
MAS-Shield: A Defense Framework for Secure and Efficient LLM MAS
Nov 28, 2025Kaixiang Wang, Zhaojiacheng Zhou, Bunyod Suvonov +8LLM Defense Mechanisms Linguistics
Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai 200240, China
The Capability Paradox: How Smarter Auditors Make Multi-Agent Systems Less Secure
May 17, 2026Qiqi Liu, Thorsten Holz, Shilin Ye +1Attack-Success Rate Hijacking
University of Chinese Academy of Sciences · Max Planck Institute for Security and Privacy · Henan Yinzhu Safety Technology Co., Ltd. +1

Sep 1, 2026, cs.CRJ/K move · Enter open · S save

SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems

Rui Yang, Junjie Xu, Zhengyu Liu, Neil Fendley+3

Johns Hopkins University · Nanyang Technological University

Model-Based Multi-Agent Systems Multi-Agent Large Language Model Systems Security

Open paper AskPDF arXiv

Safe agents can fail together. Multi-agent LLM systems (MAS) move information, state, decisions, and authority across principal boundaries, creating failures that local checks may miss. Without an execution-level view, a multi-agent setting can easily be mistaken for evidence of a genuinely multi-agent security effect. We thus systematize MAS security through an execution-centered analysis of 197 works, covering six interaction interfaces, four adversary positions, seven system-level risks, and eight recurring attack paths. We introduce an A-I-R framework that organizes attacks by adversary position, interaction interface, and resulting system-level risk, unifying otherwise fragmented attack mechanisms across MAS. We organize defenses through a five-part contract covering path target, observation, intervention, trust boundary, and recovery, and identify path closure and recovery as key challenges. We audit 44 evaluation and benchmark works and identify open challenges in isolating interaction effects, designing comparable and diagnostic metrics, supporting reuse across MAS designs, and evaluating open-system operation. Together, these findings motivate an interaction-aware view of MAS security: trace attacks end to end, test whether defenses close those paths, and evaluate system-level effects with appropriate counterfactuals.

Abstract

Explore similar work

SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems

MAS-Shield: A Defense Framework for Secure and Efficient LLM MAS

The Capability Paradox: How Smarter Auditors Make Multi-Agent Systems Less Secure