Organizations: College of Engineering Northeastern University Boston, USA · Independent Researcher Austin, USA · Independent Researcher Santa Clara, USA · Independent Researcher San Jose, USA · Olin Business School Washington University in St. Louis St. Louis, USA
We introduce SafeRoute-MAS, a coverage-aware, risk-calibrated procedure that keeps all specialist messages or removes one using only pre-integration information. On a sealed cache of 5,328 committees, its source-compatible operating point gains +1.80 percentage points (95% CI [+1.13, +2.52]) while removing 12.76% of message tokens. We also run 4,580 controlled integrations over a stratified 458-committee sample on a public gpt-oss-120b endpoint. The sample estimate is -0.09 points; design weighting to the original workload gives +0.94 points (95% CI [-1.31, +3.50]) and 18.72% acted-input reduction. A weighted token-matched comparison likewise does not establish a selector advantage (+1.68 points, CI [-1.27, +5.10]). Only 13 of 88 cached non-neutral labels (14.8%) retain direction by repeated-call majority. Expected-corruption UCL is 4.60% after weighting, but only 2 of 11 prespecified marginals pass the support/risk rule. We therefore propose a numerical qualification with Keep-All fallback, applied here retrospectively rather than validated prospectively.
Figures & tables
Fig. 1: SafeRoute-MAS and its action-level evaluation. Visible message and committee features feed a calibrated selective gate, which chooses Keep-All or one Prune-One action; the corresponding replay measures accuracy, token cost, and risk. Ablations do not establish committee context or the diversity rule as independent sources of gain.
Method
Acc.
Δ pp
Rec./Corr.
Cov.
Tok.
UCL
Keep-All
74.89
0.00
0/0
0.0
0.0
–
SafeRoute-MAS
76.69
+1.80
164/68
43.5
12.76
3.61
Lowest self-conf.
76.63
+1.75
205/112
40.2
12.07
6.14
Completeness
76.13
+1.24
142/76
39.8
11.08
4.34
Longest
76.11
+1.22
145/80
40.1
13.59
4.50
Minority answer
75.51
+0.62
164/131
39.4
12.75
7.28
TABLE I: Previously sealed test routing results. Tok. is official specialist-message payload reduction; UCL is the 95% problem-cluster bootstrap upper limit for helpful prunes/actions.
Scope
Cov.
Risk
UCL
Corr.
LOBO Δ pp
JEEBench
50.0
0.49
1.46
0.24
+0.05
LAB-Bench
74.2
4.78
6.79
3.55
+8.76
MaScQA
36.2
2.13
4.44
0.77
-0.04
Omni-MATH-2
41.2
2.97
3.88
1.23
-0.45
SciBench
22.8
0.95
2.75
0.22
-0.09
K=2
0.0
–
–
0.00
–
TABLE II: Frozen-primary subgroup risk and coverage. Risk and UCL are conditional helpful prunes/actions; Corr. is corruptions/all. LOBO Δ uses the shell-free, no-benchmark-feature held-out experiment. Percentages throughout.
Fig. 2: Cached primary-split accuracy changes by benchmark and committee size (problem-cluster 95% CIs). Heterogeneity motivates target-specific qualification; K=2 is a frozen no-action boundary.
Estimand
Units
Δ pp (95% CI)
Tok.
Risk UCL
Frozen 458 sample
2,290
−0.09[−1.85,1.61]
12.63
5.37 a
Wtd. full workload
5,328
+0.94[−1.31,3.50]
18.72
4.60 b
Validator branch
790
−0.25[−2.84,2.14]
3.97
7.23 b
Balanced learned tail
1,500
0.00[−2.32,2.30]
16.81
4.94 b
Wtd. tail vs matched
3,599
+1.68[−1.27,5.10]
+1.63
–
TABLE III: Retrospective endpoint diagnostic. Wtd. rows use source-workload design weights; Tok. is acted integrator-input reduction. CIs/UCLs use problem clusters.
Multi-agent communication studies how autonomous agents exchange information to coordinate decisions, share knowledge, and act together under constraints.
When a multi-agent system changes its team, it also changes the messages and final answer it produces, so an end-to-end accuracy gap does not by itself identify a routing effect. We introduce method, an evaluation contract that fixes a public information boundary, downstream stack G, and finite legal team family before outcomes are generated. Complete coverage identifies exact finite-benchmark oracle regret conditional on that stack. For any finite collection of frozen policies, executing the union of their distinct selected teams is the minimal assumption-free support for every pairwise policy contrast, though not for absolute oracle regret. Two controlled tables with source-ID-disjoint splits test the instrument. On MuSiQue-12, a pre-specified privileged positive control improves regret from 0.532 to 0.402; a later public-interface control reaches 0.424 versus 0.554 but is retrospective. On HotpotQA-4, a pre-specified public direct scorer improves regret from 0.313 to 0.110. In fixed-stack Llama execution, verified route regret improves by 0.190, while the raw-answer gain is 0.010 with an interval crossing zero. A five-family ToolSandbox variant-shift validation exhaustively evaluates 16 declared teams on 14 untouched task variants (224/224 valid rows): the declared-family oracle reaches 0.768 safe-evidence completion, while the prospectively frozen router gets 0.637 (regret 0.131), failing the predeclared 0.10 criterion. A later retrospective comparator reaches 0.655, matching all-workers with 4.57 versus 5.00 workers on average. Thus COVER exposes selection headroom without manufacturing a routing win. A crossed-stack diagnostic shows absolute scores depend on G but finds no detectable router-by-finalizer interaction. COVER is an auditable measurement methodology, not a claim of stack-invariant or universal agent-routing superiority.
Multi-agent LLM systems route among model-backed advisors, yet a deployer rarely knows before shipping whether routing will help at all. Prevailing routers optimize a gate's AUC and presume that advisor complementarity suffices. We show that neither determines the deployable gain. We introduce RouteGuard, a deployment-certification framework. Routing gain decomposes as G=πΔE, and the achievable gain is governed by a conditional-regret functional Φ, not by AUC. A finite-sample certification bracket comes with a matching Le Cam lower bound, constant-sharp over the fixed-activity class, and a robustness phase transition. On two benchmarks the framework acts as a guardrail. On RouterBench (11 cross-family models) the verdict depends on the sampling unit: the protocol certifies a gain over GPT-4 under prompt-level sampling and withholds it under workload-cluster resampling, because the gain rests on 3 of 86 workload cells. On OpenRCA (three Gemini advisors) the advisors are statistically redundant: the realized oracle sits at or below the independence baseline in all pools we tested (221 RouterBench pools and three OpenRCA distributions), so the protocol correctly refuses to certify. A pre-registered semi-synthetic control confirms calibration: the protocol certifies a genuine gain once m≥m⋆ and does not certify a true null. Code and frozen artifacts will be released with the published version.
Anchen Sun, Kaiqi Yang
Department of Electrical and Computer Engineering, University of Miami, Coral Gables, FL 33146 USA · Department of Mathematics, University of Miami, Coral Gables, FL 33146 USA
When a tool-using LLM agent team violates policy, aggregate outcomes do not reveal whether an upstream message changed the executor's action. We introduce SwarmSafe-Delta, a paired executable protocol that freezes upstream computation, varies real, hidden, and format-matched placebo delivery, and scores a single committed action against deterministic benign and risky policy states under preassigned budgets. Across four models and constructed optional tasks, real delivery at N=4 produces 21.4% risky-world violations, versus 5.0% with placebo and 1.0% with hidden delivery (paired differences +16.4 and +20.3 percentage points). Real delivery also improves benign task success, illustrating that suppressing communication sacrifices utility. In a second regime constructed to require an upstream lookup route, real delivery improves safe completion by 74.2 points relative to hidden delivery. These effects quantify message-delivery contrasts under specified information dependencies, not naturally occurring failure rates. Exploratory action-level analyses show that the effect depends strongly on whether the upstream message recommends a prerequisite check or direct execution; model differences also largely track these message compositions. The contribution is a controlled, execution-grounded measurement protocol rather than a general claim that agent communication is harmful or protective.