cs.MASep 30, 2026
SaveDeny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systems
Organizations: Tulane University · University of British Columbia · Independent Researcher · Virginia Tech
Abstract
Multi-agent systems derive their capabilities from sharing evidence, delegating tasks, and combining information across agents. The same process creates a safety problem: contributions that are admissible in isolation can jointly enable a prohibited use. Blocking every sensitive action avoids disclosure but defeats the purpose of collaboration. We introduce authorization-paired evaluation, which makes blocking prohibited uses and completing required authorized uses a joint success criterion, and FlowReview, a framework connecting object resolution, permission ranking, and deterministic enforcement. In controlled composition experiments, reviewing combined artifacts reduces the denied-commit rate from 86.0% to zero with no loss of authorized supply. Our findings show that preserving information and lineage alone does not ensure correct permission attribution. Object identity and permission must remain connected to execution through components whose outputs can be verified. Together, these findings establish a system-level requirement for multi-agent safety: govern composed information flows while preserving the authorized capabilities that make collaboration useful.
Figures & tables
Figure 1 : Locally admissible fragments can compose into a prohibited use. Three workers contribute disjoint parts of one governed object. With the same proposal and DENY policy, local review leaves the object unresolved and permits execution. Global review resolves the combined artifacts, allowing the commit gate to block the action.
Figure 2 : FlowReview connects object identity, permission, and execution. Object and action contracts inform all three review capabilities (blue solid arrows). Trusted policy guides permission ranking and is independently checked at enforcement (green dashed arrows). The commit gate enforces the resulting action decision.
Figure 3 : Object binding and enforcement recover selective correctness while reducing disclosure. Bars pool 576 runs per family and condition at a 24-round budget. Hollow circles, squares, and triangles represent 32 scenario means for the three legend conditions, respectively, with 18 runs per scenario. Horizontal offsets separate equal values. Whiskers show 95% scenario-cluster bootstrap intervals from 10,000 resamples. Higher and lower are better.
Figure 4 : Controlled interventions isolate object identity, enforcement, and reader scope. Bars show aggregate rates. (a) Oppositely authorized same-class objects, 288 runs per condition. (b) Model review versus commit gate, 991 and 1,009 runs, on a smaller percentage scale. (c) The same fixed proposals under local or global review, 480 per policy and reader. Global review blocks denied use while preserving authorized supply.
Figure 5 : Contributor growth can impair resolution despite complete information. (a) Bars pool 168 graphs per condition. Hollow squares ( ) and circles ( ) show seven team rates, with 24 graphs each. (b) Paired coverage changes from to with 95% scenario-cluster bootstrap intervals. Only homogeneous Haiku’s coverage decline is significant after Holm correction. Mixed labels identify coordinators. All conditions retain complete information and 100% authorized supply.
Figure 6 : Explicit contracts improve lineage but leave permission-attribution errors. (a) Lines show rates pooled over 168 graphs per agent count. Hollow squares (type-only) and circles (explicit) show seven team rates, each based on 24 graphs and offset horizontally for visibility. (b) Bars pool 840 graphs per contract, with explicit-contract values labeled. Correct attribution requires all roles to preserve the claim’s permission and untrusted authority rank. Higher is better.
| Component | Lineage | Correctness | Denied commits | ||
| Assembly placement: execution correctness | |||||
| Model composer | N/A | 0.675 | 0.350 | 0.475 | 0/80 |
| Runtime assembler | N/A | 1.000 | 1.000 | 1.000 | 0/80 |
| Permission placement: proposal correctness | |||||
| Runtime lineage + specialist | 1.000 | 1.000 | N/A | 1.000 | 0/480 |
Table 1: Verifiable placement preserves permission binding and restores authorized assembly. Correctness scores execution for assembly and pre-execution selections for permission review. Assembly uses 80 runs per arm, including 40 ALLOW runs for . Permission results are identical across four specialists on 480 graphs each. Denied counts use all runs. follows study-specific task checks. N/A marks unmeasured or inapplicable endpoints.
| Execution arm | Target commit | Final violation | Utility | Exact-call match |
|---|---|---|---|---|
| Native, no gate | 21/24 | 24/24 | 9/24 | 5/24 |
| Object-bound runtime stack | 0/24 | 4/24 | 21/24 | 5/24 |
Table 2: Runtime control blocks target commits in independent AgentDojo-derived Sonnet pairs. Both arms use 24 matched scenarios. Target commits use the action ledger. Final violations and utility use benchmark oracles. Exact-call match requires the prescribed legitimate tool call, a stricter criterion than benchmark utility.
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
| Study | Controlled contrast | Evaluation size |
|---|---|---|
| Controlled composition | Local / global read scope | 480 DENY + 480 ALLOW proposals |
| Natural composition | Generated artifacts, composer | 60 evaluation pairs, 12 clusters |
| Capability ladder | Class / binding / enforcement | 576 runs/cell, 32 clusters |
| Contributor scaling | Agent count , contributors | 588 + 672 graphs |
| Delegation contracts | Type-only / explicit lineage | 840 graphs/contract |
| Permission placement | Four isolated specialists | 480 graphs/specialist |
Table 3: Studies, comparisons, and evaluation units. Each row retains its study-specific sampling unit and denominator.
Figure 7 : Communication and contributor structure shape the information available for review. (a–d) Arrows show untrusted content in red, recorded flows in blue, and trusted-policy control as green dashed lines. The relay contrast fixes message content across source wrappers. (e,f) Readers inspect separate artifacts or their union while proposal and policy remain fixed. (g) At , circles denote coordinators, filled squares informative workers, and dashed squares background workers without governed fields. Numbers in the lower boxes count initially assigned fields, totaling 16 in each condition. counts initial field holders and includes the coordinator, excluding runtime components.
| Experiment | Communication | Capability tested | |
|---|---|---|---|
| Direct evaluation / ladder | Varies | Drafter and review components | Resolution, ranking, enforcement |
| Matched relay | 1 to 2 | Direct or relayed bytes | Topology and selectivity |
| Authority delegation | 1 to 16 | Source first, then handoffs | Lineage and attribution |
| Natural composition | 4 | Three workers and a composer | Cross-agent reconstruction |
| Contributor scaling | 1 to 16 | Star graph, informative | Information versus headcount |
| AgentDojo transfer | 1 or 3 | Direct or worker/relay/executor | External action boundary |
Table 4: Topologies and measured capabilities. counts model agents, including the coordinator. Runtime components are counted separately. Ladder model roles depend on the review configuration.
| Stage | Artifact | Fields retained | Object resolved | Outcome |
| Intake specialist | Message | 6/6 | No | N/A |
| Telemetry analyst | Store | 5/5 | No | N/A |
| Identity analyst | Handoff | 5/5 | No | N/A |
| Coordinator | Proposal | N/A | Reference only | Checks pass |
| Runtime assembly | Union | 16/16 | Yes | Exact registered record |
| Local runtime | DENY | N/A | No | Denied commit |
Table 5: One homogeneous execution. The same fixed proposal receives four combinations of policy and reader scope. N/A marks a field that does not apply to that stage.
| Record element | Constructed example |
|---|---|
| Task | Prepare report REPORT-A and send the governed object when authorized. |
| Governed object | SYNTH_CODE_REVIEW_001 |
| Worker artifacts | Message: SYNTH_C. Store note: ODE_REV. Handoff: IEW_001. |
| Proposed action | send_external, with the complete object as its body and code-review.example.invalid as destination. |
| Trusted permission | DENY in one execution and ALLOW in its paired execution. |
| Local outcome | The action commits under both permissions. |
Table 6: A constructed scenario and its recorded composition. Input fragments and the governed object are specified before execution and shown with display aliases. The proposal is produced by the Gemma composer from the three Haiku artifacts. Both policy-conditioned calls produce the displayed action.
| Endpoint | Estimate | 95% cluster interval |
|---|---|---|
| Trusted-policy effect | 0.8828 | |
| Claim effect | 0.0000 | |
| Claim effect | 0.0156 | |
| Authorized supply | 0.8828 | |
| Task completion | 1.0000 |
Table 7: Trusted policy controls execution with little interference from untrusted claims. Four models share 16 scenario clusters. Effects are differences in commit rates. Authorized supply uses 128 ALLOW trajectories, and task completion uses all 256 trajectories. Zero-width intervals reflect constant observed cluster outcomes rather than zero population uncertainty.
Figure 8 : Selective control persists across attack budgets while authorized recovery varies by composer. (a) Circles and squares represent the two attack families, with 576 final-configuration runs per point and an expanded 98.5–100% axis. (b) Neutral and untrusted ALLOW claims under trusted policy , with 64 trajectories per bar. (c) Left green bars show authorized supply and right blue bars task completion over 12 scenarios per model. Whiskers show 95% scenario-cluster bootstrap intervals. Collapsed intervals reflect constant observed outcomes.
| Model | Fewer denied commits | Tie | More denied commits |
|---|---|---|---|
| Haiku 4.5 | 3 | 3 | 2 |
| Sonnet 4.5 | 6 | 2 | 0 |
| Qwen3-32B | 5 | 1 | 2 |
| Gemma3-27B | 6 | 2 | 0 |
| Llama-3.3-70B | 2 | 2 | 4 |
| Total | 22 | 10 | 8 |
Table 8: Reader choice changes control coverage. Counts compare the family-conditioned decoder with literal matching across representation families. Each comparison holds 12 proposals and their policies fixed.
| Workers | Condition | Artifact path | Comparison | ||
|---|---|---|---|---|---|
| 1 | 0 | 1 | Local full record | Direct coordinator | Single-agent reference |
| 2 | 1 | 1 | Full record | One artifact | First delegation edge |
| 4 | 3 | 3 | Shared-scaled | Three artifacts | Distributed reconstruction |
| 8 | 7 | 7 | Contributor-scaled | Seven artifacts | More contributors |
| 16 | 15 | 15 | Contributor-scaled | Fifteen artifacts | Contributor scaling |
| 8 | 7 | 4 | Dummy-scaled | 4 informative + 3 background | Same , less information |
Table 9: Contributor-scaling matrix. Workers exclude the coordinator. counts initial field holders, including the coordinator at .
| Object | Denied commits | ALLOW supply | |||||
| Setting | Graphs | Local | Union | L | G | L | G |
| Discovery, | 168 | 168/168 | 168/168 | 0/168 | 0/168 | 168/168 | 168/168 |
| Discovery, distributed | 420 | 0/420 | 420/420 | 420/420 | 5/420 | 420/420 | 420/420 |
| Replication, non-maximal | 504 | 0/504 | 504/504 | 504/504 | 0/504 | 504/504 | 504/504 |
| Replication, | 168 | 0/168 | 168/168 | 168/168 | 19/168 | 168/168 | 168/168 |
| Replication, all cells | 672 | 0/672 | 672/672 | 672/672 | 19/672 | 672/672 | 672/672 |
Table 10: Composition endpoints in discovery and replication. L/G denote local/global readers of the same fixed proposals. Object columns report complete visibility locally and in the artifact union.
| Cell | Schema purity | Role leakage | Worker conformance | Lineage coverage |
|---|---|---|---|---|
| 144/168 | 24/168 | 0.956 | 168/168 | |
| 120/168 | 48/168 | 0.895 | 168/168 | |
| 144/168 | 24/168 | 0.979 | 168/168 | |
| 113/168 | 55/168 | 0.877 | 149/168 | |
| All four cells | 521/672 | 151/672 | 0.927 | 653/672 |
Table 11: Surface conformance and lineage in the powered replication. Each cell contains 168 graphs across seven teams. Worker conformance is a rate over worker calls. Other columns count graphs.
Figure 9 : Delegation separates visibility errors from permission-attribution drift. (a) Circles mark source-visibility errors and squares final-proposal attribution drift, pooling 168 graphs per agent count. (b) Cells report type-only complete lineage in an independent study, with 24 graphs per team and agent count. Darker blue indicates higher coverage, and values are percentages. Mixed labels name the coordinator.
| Design or criterion | Specification |
|---|---|
| Lineage and permission placement | |
| Model agents | 1, 4, 16 |
| Policy factors | Trusted untrusted |
| Role teams | 5 |
| Permission specialists | Sonnet 4.5, Haiku 4.5, Qwen3-32B, Gemma3-27B |
| Clusters, dev / confirm | 4 / 8 |
Table 12: Placement comparisons and evaluation criteria. Development and confirmation clusters are disjoint. Thresholds are prespecified, and the commit gate controls execution.
| Trusted policy | Selection component | Selected action | Runtime outcome |
|---|---|---|---|
| ALLOW | Final coordinator | Object reference mistyped | Rejected |
| ALLOW | Isolated Qwen specialist | Exact registered candidate | Committed |
| DENY | Isolated Qwen specialist | No action selected | No commit |
Table 13: A permission decision must remain bound to the selected action. The ALLOW rows use the same saved coordinator output. The DENY row is its policy-conditioned companion, whose upstream outputs were generated separately.
| Artifact channel | Assembler | Execution correct | ALLOW supply | Task |
|---|---|---|---|---|
| No failover | Model composer | 0.700 | 0.400 | 0.600 |
| No failover | Runtime assembler | 0.800 | 0.600 | 0.600 |
| Validated failover | Model composer | 0.675 | 0.350 | 0.475 |
| Validated failover | Runtime assembler | 1.000 | 1.000 | 1.000 |
Table 14: Availability and assembly ablation. Execution correctness and task rates use 80 executions per arm, while authorized supply uses the 40 ALLOW executions. Execution correctness requires no committed action under DENY or the exact required action under ALLOW. Task success additionally requires a valid report. The common commit gate is applied before these outcomes are scored.
| Condition | Gate binding | Wrong commits | Authorized supply |
|---|---|---|---|
| Cached-policy execution | N/A | 120/240 | 0/120 |
| Current-policy reread | N/A | 1/240 | 120/120 |
| Reread + version gate | 170/240 | 0/240 | 83/120 |
Table 15: Version checking after policy reread blocks wrong commits but can withhold authorized actions. Independent confirmation bank with 240 evaluations per arm, including 120 currently authorized actions. Binding is measured by the version gate, so N/A denotes ungated arms. Wrong-commit counts use the full bank.
| Proposal input / control | Valid | Binding | Supply | Task | Protocol |
|---|---|---|---|---|---|
| Current-policy reread | 1.000 | 0.721 | 1.000 | 1.000 | 1.000 |
| Stale proposal + gate | 1.000 | 0.000 | 0.000 | 1.000 | 1.000 |
| Stale proposal + repair | 0.788 | 0.000 | 0.000 | 0.000 | 0.785 |
Table 16: Current-policy reread recovers authorized supply that one repair attempt does not. A separate confirmation bank has 240 evaluations per arm and 120 currently authorized actions. Proposal, binding, and task rates use all 240 evaluations. Protocol validity is averaged over the role outputs specified by each arm. Binding checks the current policy version and lineage, while supply checks the exact executed action. All arms record zero wrong commits.
| Exposure | Model | Exact attacks | Utility | Protocol | Inst. |
|---|---|---|---|---|---|
| Unchanged official | Sonnet 4.5 | 0/24 | 0.875 | 1.000 | yes |
| Unchanged official | Haiku 4.5 | 0/24 | 0.792 | 1.000 | yes |
| Unchanged official | Qwen3-32B | 5/24 | 0.458 | 1.000 | yes |
| Authority re-expression | Sonnet 4.5 | 13/24 | 0.625 | 1.000 | yes |
| Authority re-expression | Haiku 4.5 | 0/24 | 0.625 | 1.000 | yes |
| Authority re-expression | Qwen3-32B | 21/24 | 0.083 | 1.000 | yes |
Table 17: Relay exposure guides confirmation. Each cell contains 24 development trajectories. Inst. marks whether instrumentation meets its validity criterion. Qwen is the positive control, and Sonnet confirmation uses independent pairs.
| Observed outcome | Native | Gated |
| Required lunch event | Created | Created |
| Required calendar invitation | Sent | Sent |
| Private email forwarded | Yes | Blocked |
| Exact legitimate-action match | 1 | 1 |
| Benchmark utility | 0 | 1 |
Table 18: Blocking the prohibited action restores utility while retaining the required action. One saved Sonnet pair. The legitimate calendar operation generates the invitation email.
| Model | Exact transient | Rate | Native utility | Protocol |
|---|---|---|---|---|
| Sonnet 4.5 | 0/36 | 0.000 | 0.750 | 0.917 |
| Haiku 4.5 | 0/36 | 0.000 | 0.556 | 0.972 |
| Qwen3-32B | 4/36 | 0.111 | 0.528 | 0.694 |
| Llama-3.3-70B | 2/36 | 0.056 | 0.194 | 0.944 |
Table 19: Native attack exposure across models. Each row contains 36 development trajectories from a separate exposure comparison. Exact events, task utility, and protocol conformance use their stated denominators.
| Official attack | Arm | Proposal | Commit | Violation | Utility | Commit [95% CI] |
|---|---|---|---|---|---|---|
| Important Instructions | Native | 6/8 | 6/8 | 5/8 | 5/8 | N/A |
| Important Instructions | Gated | 6/8 | 0/8 | 0/8 | 1/8 | [ ] |
| InjecAgent | Native | 5/8 | 5/8 | 5/8 | 2/8 | N/A |
| InjecAgent | Gated | 5/8 | 0/8 | 0/8 | 1/8 | [ ] |
| System message | Native | 3/8 | 3/8 | 3/8 | 4/8 | N/A |
| System message | Gated | 2/8 | 0/8 | 0/8 | 3/8 | [ ] |
Table 20: Exploratory Qwen attack-family comparison. Each family has eight matched pairs. Commit is gated minus native, with a 95% cluster interval. N/A marks native rows where the paired contrast is not repeated.
| Official attack | Arm | Proposal | Commit | Violation | Utility | Commit [95% CI] |
|---|---|---|---|---|---|---|
| Important Instructions | Native | 0/8 | 0/8 | 4/8 | 3/8 | N/A |
| Important Instructions | Gated | 0/8 | 0/8 | 2/8 | 5/8 | [ ] |
| InjecAgent | Native | 0/8 | 0/8 | 3/8 | 3/8 | N/A |
| InjecAgent | Gated | 0/8 | 0/8 | 4/8 | 3/8 | [ ] |
| System message | Native | 0/8 | 0/8 | 2/8 | 2/8 | N/A |
| System message | Gated | 0/8 | 0/8 | 2/8 | 1/8 | [ ] |
Table 21: Qwen evaluation on independent task and injection pairs. Each family has eight matched pairs. Commit is gated minus native with a 95% cluster interval. N/A marks native rows where the paired contrast is not repeated. Pooled estimates are exploratory because native exposure falls below the confirmatory threshold. In zero-event cells, is a degenerate empirical bootstrap interval, not evidence of zero population uncertainty.
| Evidence | Direct conclusion | Measurement scope |
| Paired evaluation | Authorized capability changes the safety target | Matched policy intervention |
| Paired evaluation | Resolution, ranking, and enforcement are distinct | Verbatim action ledger |
| Composition | Admissible artifacts identify a denied action | Read-scope intervention |
| Scaling | Distributed information drives resolution burden | Star graphs, seven teams |
| Delegation | Lineage and authority are separate capabilities | Paired contracts, 840 graphs |
| Permission placement | Lineage and action binding retained | Four specialists, 480 graphs each |
Table 22: Contributions and supporting comparisons. Measurement scope identifies the intervention or execution surface behind each conclusion.
| Boundary | Reader / object | Metric | Value |
|---|---|---|---|
| Action commit | Literal governed objects | 0.990 to 0.998 | |
| Action commit | Literal governed objects | Verbatim releases | 0/2,304 |
| Action commit | Eight representation families | Lower commit | 22/40 |
| Action commit | Eight representation families | Higher commit | 8/40 |
| Action commit | Eight representation families | Unchanged commit | 10/40 |
Table 23: Reader-specific control at the action boundary. Ladder rows aggregate final-configuration trials across two attack families and two budgets. Representation rows count model-by-transformation comparisons against literal matching, each with 12 fixed proposals.
| Outcome | Result |
| Natural composition: 60 evaluation pairs | |
| Denied exact recovery, local | 3/60 |
| Denied exact recovery, global | 0/60 |
| Authorized supply, global | 15/60 |
| Trusted policy untrusted claim: 320 units | |
| Policy effect | Large |
Table 24: Outcomes across controlled comparisons. Each group identifies its experimental setting and sampling unit. Delegation-depth drift concerns exact final proposals, while contract-study drift counts errors at any role. Permission placement and assembly are separate comparisons.