Safe to Omit, Worth Omitting? Keeping the Answer and Cutting Latency When an LLM Agent Skips Another's Message
Organizations: University of Houston · Argonne National Laboratory · Oregon State University
Abstract
In multi-agent systems, a receiver agent often waits for a helper agent's message (text or its latent KV cache), which can improve performance but adds latency. We study when a receiver agent can skip the message safely (the final answer rarely changes) and beneficially (the time saved exceeds the cost of deciding). Our mechanism reads the receiver agent's confidence without the message and uses a finite-sample test to certify a threshold at which, with high probability, at most 5% of skipped answers change; if none passes, the system keeps communicating. We test it on OpenBookQA, ARC-Challenge, and MMLU-Pro with three Qwen model pairs, on cross-family agents, and on SQuAD and GSM8K. Four insights emerge. First, oracle headroom is weak evidence that messages help: control inputs without answer-specific content, matched to change answers as often as real messages, reproduce a median 76% of the oracle's gain. Second, whether a message can be skipped depends on how often and how predictably it alters the answer, not on its accuracy; every certified KV-cache path is less accurate than the receiver agent alone. Third, where text messages help, skipping keeps most of their gain: on unseen questions in the four such settings, the policies skip 59 to 80% and keep an estimated 76 to 95% of the gain. In two prospective tests with all rules fixed in advance, our mechanism skips a three-agent system's verifier on 39.9% of sealed questions, keeping 12.55 of its 12.80-point gain, and a 32B helper on 68.4% of held-out questions, keeping its whole gain. Fourth, safety does not determine efficiency: at batch size one, skipping text saves hundreds of milliseconds per query and skipping KV cache far less. Certification extends to SQuAD but not to GSM8K. In eleven tests on unseen questions, every certified policy we could test kept its change rate's 95% upper bound below 5% under our answer parsers.
Figures & tables
| Real paths | Null / real, all paths | Null / real gain, per path | ||||||||||
| Correct answers | Oracle | Headroom | Gain | Unstratified | Stratified | |||||||
| Pair | Benchmark ( ) | Text | C2C | headroom (pp) | Unstratified | Stratified | over | Text | C2C | Text | C2C | |
| small | OBQA (742) | 285 | 346 | 366 | 17.79 | 1.05 | 0.79 | 0.66 | 0.72 ∗ | 0.69 ∗ | 0.80 ∗ | 0.80 ∗ |
| small | ARC (299) | 110 | 126 | 155 | 16.39 | 1.15 | 0.73 | 0.74 | 0.85 | 0.75 ∗ | 0.84 | 0.86 |
| medium | OBQA (742) | 490 | 523 | 486 | 9.43 | 1.30 | 1.00 † | 0.90 | 0.80 ∗ | 1.13 | 0.97 | 1.20 |
| medium | ARC (299) | 217 | 221 | 202 | 9.03 | 0.95 † | 0.94 † | 0.85 † | 0.74 | 1.71 ∗ | 0.79 | 1.48 |
| Skipped | Accuracy (%) | Text | Policy | Kept | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Policy | Unseen questions | (%) | Changed | Text | Policy | [95% CI] | [95% CI] | (%) | ||
| Llama-8B/OBQA/Text | held out (744) | 59.1 | 3/440 | 80.38 | 83.06 | 82.66 | 2.69 [0.40, 4.97] | 2.28 [ , 4.44] | 85 | 2.96 |
| Llama-8B/ARC/Text | test split (1,172) | 67.7 | 15/794 | 81.31 | 84.73 | 84.56 | 3.41 [1.71, 5.12] | 3.24 [1.62, 4.86] | 95 | 3.39 |
| large/OBQA/Text ‡ | held out (744) | 80.0 | 12/595 | 83.06 | 86.96 | 86.02 | 3.90 [1.88, 5.78] | 2.96 [1.21, 4.57] | 76 | 4.00 |
| large/OBQA/Text+fact ‡ | held out (744) | 75.0 | 16/558 | 83.06 | 91.13 | 89.65 | 8.06 [5.91, 10.22] | 6.59 [4.57, 8.47] | 82 | 3.75 |
| Skipped | Changed / skipped | Lost corrections | Accuracy | Gain kept (points) | Saving (ms) | ||
|---|---|---|---|---|---|---|---|
| System | Policy | (%) | (bound, %) | (bound, %) | (%) | [95% CI] | [95% CI] |
| three agents | joint, | 39.9 | 5/798 (1.46) | 5 (0.58) | 76.35 | 12.55 [11.10, 14.15] | 483 [365, 604] |
| three agents | joint, | 49.6 | 15/991 (2.48) | 14 (1.17) | 75.90 | 12.10 [10.65, 13.65] | 618 [501, 732] |
| three agents | label-free | 54.7 | 29/1,094 (3.78) | 26 (1.90) | 75.35 | 11.55 [10.15, 13.05] | 697 [581, 818] |
| 32B pair | joint, | 68.4 | 2/509 (1.41) | 1 (0.75) | 92.88 | 2.55 [0.94, 4.17] | 1,377 [1,219, 1,533] |
| 32B pair | joint, | 79.8 | 6/594 (2.19) | 4 (1.37) | 92.61 | 2.28 [0.67, 3.90] | 1,525 [1,377, 1,667] |
| Changed | acc (pp) | Saving (ms) | ||||||||
| Pair | Task | Path | Skipped (%) | Dev. | Unseen | Level | [95% CI] | Gain (pp) | Gain Kept (%) | [95% CI] |
| medium | OBQA | C2C ∗ | 52.6 | 11/390 | held out 8/388 | 0.025 | [ , ] | – | 79.7 [48.1, 129.1] | |
| medium | ARC | C2C | 63.2 | 4/189 | held out 15/733 ∘ | 0.001 | [ , ] | – | 100.7 [74.5, 133.1] | |
| large | OBQA | Text ‡ | 77.1 | 19/572 | held out 12/595 | 0.001 | [ , ] | 3.8 | 75 | 517.1 [447.8, 583.6] |
| large | OBQA | C2C ‡ | 77.1 | 13/572 | held out 13/595 | 0.001 | [ , ] | – | 36.6 [18.0, 56.8] | |
| large | ARC | Text | 95.0 | 9/284 | sealed 20/1,100 | 0.001 | [ , ] | 2.0 | 33 | 717.6 [670.6, 763.2] |
Appendix figures & tables45 assets
Supplementary material from the paper’s appendix.
Appendix
| Evidence | Settings or policies | Protocol fixed | Appendix |
| Certification of settings | |||
| Original settings | small and large pairs, OBQA and ARC (6) | before calibration statistics | A , C |
| Reused certification ‡ | large/OBQA/Text | earlier 100-test family; exposed | B |
| Coupled labels ‡ | large/OBQA/C2C | before calibration statistics | B |
| Medium pair | 4 settings | before outputs | B , C |
| MMLU-Pro | large (2), then small and medium pairs (4) | before outputs | C |
| Disagreements | Smallest | Deployed | Coverage | Changed | Correct: policy | ||||
| Pair | Task | Reference | / questions | AUROC | 0.999 bound | (%) | / skipped | / reference | |
| small | OBQA | Text | 339/742 | 0.641 | 0.232 | fallback | 0.0 | – | 346/346 |
| small | OBQA | C2C | 350/742 | 0.700 | 0.161 | fallback | 0.0 | – | 366/366 |
| small | ARC | Text | 145/299 | 0.634 | 0.394 | fallback | 0.0 | – | 126/126 |
| small | ARC | C2C | 160/299 | 0.641 | 0.296 | fallback | 0.0 | – | 155/155 |
| medium | OBQA | Text | 174/742 | 0.787 | 0.096 | fallback | 0.0 | – | 523/523 |
| Full calibration split | Reviewed out | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Pair | Task | Reference | Deployed | Tested | Re-split | Needed | |||||
| small | OBQA | Text | fallback | 0.05 | 65 | 5 | 0.894 | – | – | 0.00 | never |
| small | OBQA | C2C | fallback | 0.05 | 65 | 2 | 0.363 | – | – | 0.00 | 21,793 |
| small | ARC | Text | fallback | 0.10 | 33 | 5 | 0.995 | – | – | 0.00 | never |
| small | ARC | C2C | fallback | 0.10 | 33 | 2 | 0.773 | – | – | 0.00 | never |
| medium | OBQA | Text | fallback | 0.05 | 601 | 36 | 0.884 | – | – | 0.00 | never |
| Post-hoc variants | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Pair | Task | Reference | Primary | Fixed sequence | Entropy | Margin | Learned | ||
| small | both | both | – | – | – | – | – | – | – |
| medium | OBQA | Text | – | – | 0.45 | – | – | – | – |
| medium | OBQA | C2C | 0.55 | – | 0.65 | 0.55 | 0.55 | 0.55 | – |
| medium | ARC | Text | – | – | – | – | – | – | – |
| medium | ARC | C2C | 0.60 | – | 0.75 | 0.60 | 0.60 | 0.60 | – |
| Best candidate | Correct | invalid | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Pair | Reference | Disagreement (%) | AUROC | Deployed | Reference | Reference | Re-split | ||||
| small | Text | 46.76 | 0.644 | 1 | 102/280 | fallback | 403 | 516 | 151 | 229 | 0.000 |
| small | C2C | 60.24 | 0.626 | 1 | 106/280 | fallback | 403 | 462 | 151 | 420 | 0.000 |
| medium | Text | 42.71 | 0.717 | 1 | 64/690 | fallback | 813 | 913 | 160 | 18 | 0.000 |
| medium | C2C | 53.39 | 0.765 | 0.193 | 29/690 | fallback | 813 | 530 | 160 | 1124 | 0.000 |
| Changed / skipped | |||||
|---|---|---|---|---|---|
| Category | Skipped | Coverage (%) | Text | C2C | |
| biology | 144 | 100 | 69.4 | 1/100 | 0/100 |
| business | 177 | 47 | 26.6 | 1/47 | 2/47 |
| chemistry | 257 | 60 | 23.3 | 4/60 | 6/60 |
| computer science | 93 | 42 | 45.2 | 4/42 | 1/42 |
| economics | 190 | 116 | 61.1 | 2/116 | 0/116 |
| (a) Overall | ||||
|---|---|---|---|---|
| Reference | Strategy | Skipped | Coverage (%) | Changed |
| Text | pooled certificate | 1,069 | 40.5 | 40/1,069 |
| Text | per discipline, | 292 | 11.1 | 6/292 |
| Text | per discipline, | 0 | 0.0 | – |
| C2C | pooled certificate | 1,069 | 40.5 | 37/1,069 |
| C2C | per discipline, | 293 | 11.1 | 3/293 |
| Deployed | Coverage (%) | Saving (ms) | |||||
| Setting | certified | predicted | development (certified) | fit (predicted) | measured | predicted, with predicted coverage | predicted, with panel coverage |
| medium/OBQA/C2C | 0.55 | 0.55 | 52.6 | 55.1 | 79.7 | 64.7 | 58.4 |
| medium/ARC/C2C | 0.60 | 0.60 | 63.2 | 61.0 | 100.7 | 98.9 | 97.0 |
| large/OBQA/Text | 0.80 | 0.80 | 77.1 | 80.0 | 517.1 | 536.6 | 506.1 |
| large/OBQA/C2C | 0.80 | 0.85 | 77.1 | 85.0 | 36.6 | 26.6 | 19.0 |
| large/ARC/Text | 0.95 | 0.90 | 95.0 | 90.0 | 717.6 | 670.6 | 700.4 |
| large/OBQA/Text | large/ARC/Text | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Coverage (%) | Changed | Retained | Coverage (%) | Changed | Retained | ||||||
| 0.01 | fallback | 0 | – | – | – | fallback | 0 | – | – | – | |
| 0.02 | 0.65 | 62.0 | 6/460 | 1.24 | 1.000 | 0.80 | 77.9 | 0/233 | 1.56 | 1.000 | |
| 0.03 | 0.75 | 72.2 | 15/536 | 2.17 | 0.893 | 0.85 | 84.9 | 1/254 | 2.55 | 0.833 | |
| 0.04 | 0.80 | 77.1 | 19/572 | 3.08 | 0.750 | 0.95 | 95.0 | 9/284 | 3.80 | 0.333 | |
| 0.05 | 0.80 | 77.1 | 19/572 | 3.85 | 0.750 | 0.95 | 95.0 | 9/284 | 4.75 | 0.333 | |
| Candidates accepted (of 20) | Largest accepted | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Setting | lost- correction | change- rate | both | lost-correction ( ) | both ( ) | change-rate (deployed) | without reviewed | ||
| large/OBQA/Text ‡ | 0.01 | 1,366 | 13 | 16 | 13 | 0.65 | 0.65 | 0.80 | 0.60 |
| large/OBQA/Text ‡ | 0.02 | 1,366 | 15 | 16 | 15 | 0.75 | 0.75 | 0.80 | 0.75 |
| large/ARC/Text | 0.01 | 448 | 0 | 19 | 0 | infeasible | infeasible | 0.95 | infeasible |
| large/ARC/Text | 0.02 | 448 | 16 | 19 | 16 | 0.80 | 0.80 | 0.95 | infeasible |
| large/MMLU-Pro/Text | 0.01 | 6,000 | 9 | 8 | 8 | 0.45 | 0.40 | 0.40 | 0.40 |
| Coverage | Lost | Gained | acc (points) | Kept | Saving (ms) | ||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Setting | (%) | corrections | answers | 95% CI | (points) | (of ) | 95% CI | ||||
| large/OBQA/Text ‡ | 0.01 | 0.65 | 62.0 | 3 | 3 | [ , ] | 3.77 | 1.000 | 393.4 | [310.3, 468.8] | |
| large/OBQA/Text ‡ | 0.02 | 0.75 | 72.2 | 9 | 6 | [ , ] | 3.77 | 0.893 | 490.0 | [411.9, 560.5] | |
| large/ARC/Text | 0.01 | infeasible | – | – | – | – | – | 2.01 | – | – | – |
| large/ARC/Text | 0.02 | 0.80 | 77.9 | 0 | 0 | [0.00, 0.00] | 2.01 | 1.000 | 588.6 | [522.4, 656.1] | |
| large/MMLU-Pro/Text | 0.01 | 0.40 | 40.5 | 12 | 14 | [ , ] | 1.40 | 1.054 | 346.3 | [249.8, 443.0] | |
| System | OBQA | ARC |
|---|---|---|
| Qwen3-1.7B alone (medium receiver) | 490 (66.0) | 217 (72.6) |
| medium pair, Text | 523 (70.5) | 221 (73.9) |
| medium pair, C2C | 486 (65.5) | 202 (67.6) |
| Qwen3-8B alone (large receiver) | 617 (83.2) | 268 (89.6) |
| Qwen2.5-7B-Instruct alone (large helper) a | 640 (86.3) | 262 (87.6) |
| large pair, C2C | 598 (80.6) | 266 (89.0) |
| Oracle gain (questions) | Null/real ratio [95% CI] | ||||||
|---|---|---|---|---|---|---|---|
| Pair | Task | Real | Null | Independent | Text | C2C | Joint |
| small | OBQA | 213 | 203 | 0.686 [0.595, 0.782] | 0.719 [0.606, 0.845] | 0.686 [0.584, 0.804] | 0.664 [0.573, 0.759] |
| small | ARC | 94 | 90 | 0.738 [0.608, 0.889] | 0.851 [0.671, 1.080] | 0.747 [0.598, 0.953] | 0.737 [0.605, 0.893] |
| medium | OBQA | 103 | 106 | 0.805 [0.692, 0.937] | 0.800 [0.680, 0.945] | 1.125 [0.880, 1.464] | 0.903 [0.769, 1.048] |
| medium | ARC † | 31 | 28 | 0.772 [0.535, 1.060] | 0.742 [0.505, 1.057] | 1.705 [1.019, 3.449] | 0.845 [0.567, 1.144] |
| large | OBQA | 53 | 62 | 0.701 [0.578, 0.856] | 0.547 [0.443, 0.665] | 1.313 [0.882, 2.175] | 0.761 [0.613, 0.935] |
| Changed by reference | Eligible for null | Oracle gain over | Null / real gain [95% CI] | |||||||
| Pair | Task | Reference | right ( ) | wrong ( ) | right | wrong | Real | Null | Stratified | Unstratified |
| small | OBQA | Text | 75 | 264 | 179 | 411 | 136 | 109.19 | 0.803 [0.687, 0.940] | 0.719 [0.606, 0.845] |
| small | ARC | Text | 39 | 106 | 60 | 171 | 55 | 46.18 | 0.840 [0.680, 1.074] | 0.851 [0.671, 1.080] |
| medium | OBQA | Text | 49 | 125 | 103 | 150 | 82 | 79.46 | 0.969 [0.859, 1.115] | 0.800 [0.680, 0.945] |
| medium | ARC | Text | 21 | 31 | 32 | 41 | 25 | 19.67 | 0.787 [0.601, 1.004] | 0.742 [0.505, 1.057] |
| large | OBQA | Text | 18 | 54 | 82 | 84 | 46 | 37.24 | 0.809 [0.702, 0.924] | 0.547 [0.443, 0.665] |
| Text | C2C | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Null share | wrong | right | wrong | right | ||||||
| Pair | Task | right ( ) | right | wrong | share right | wrong | right | wrong | share right | wrong |
| small | OBQA | 0.414 | 136 | 128 | 0.515 | 75 | 147 | 137 | 0.518 | 66 |
| small | ARC | 0.436 | 55 | 51 | 0.519 | 39 | 69 | 67 | 0.507 | 24 |
| medium | OBQA | 0.637 | 82 | 43 | 0.656 | 49 | 44 | 39 | 0.530 | 48 |
| medium | ARC | 0.634 | 25 | 6 | 0.806 | 21 | 10 | 13 | 0.435 | 25 |
| AUROC | P(certify) by calibration size | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Dataset | Receiver | Disagreement (%) [95% CI] | First token | Whole answer | Per-token mean | Changes in lowest 25% | Eligible | 1,000 | 2,000 | 3,000 | Contain (%) |
| SQuAD | Qwen3-8B | 24.5 [20.4, 29.0] | 0.751 | 0.805 | 0.791 | 7 of 100 | no | 0.022 | 0.075 | 0.192 | 68 |
| SQuAD | Llama-8B | 14.3 [11.0, 18.1] | 0.773 | 0.836 | 0.738 | 1 of 100 | yes | 0.598 | 0.975 | 1.000 | 60 |
| NQ (passage) | Qwen3-8B | 28.3 [23.9, 32.9] | 0.719 | 0.745 | 0.743 | 12 of 100 | no | 0.000 | 0.000 | 0.002 | 51 |
| NQ (passage) | Llama-8B | 22.3 [18.3, 26.6] | 0.742 | 0.808 | 0.754 | 3 of 100 | yes | 0.039 | 0.179 | 0.393 | 44 |
| TriviaQA | Qwen3-8B | 40.5 [35.7, 45.5] | 0.746 | 0.792 | 0.784 | 9 of 100 | no | 0.000 | 0.000 | 0.002 | 16 |
| CP | CP | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0.05 | 0.000188 | 111 | 2 | 0.080 | 0.097 | 0.55 | 0.350 | 1,166 | 58 | 0.519 | 0.073 |
| 0.10 | 0.000655 | 212 | 3 | 0.0057 | 0.060 | 0.60 | 0.398 | 1,261 | 72 | 0.887 | 0.080 |
| 0.15 | 0.00161 | 307 | 3 | 0.042 | 0.65 | 0.433 | 1,322 | 76 | 0.903 | 0.080 | |
| 0.20 | 0.00492 | 409 | 4 | 0.036 | 0.70 | 0.469 | 1,390 | 85 | 0.973 | 0.084 | |
| 0.25 | 0.0161 | 519 | 7 | 0.037 | 0.75 | 0.511 | 1,501 | 107 | 0.9999 | 0.094 | |
| 0.30 | 0.0421 | 592 | 11 | 0.043 | 0.80 | 0.559 | 1,607 | 138 | 1.000 | 0.110 |
| Mean latency | Components | ||||||
|---|---|---|---|---|---|---|---|
| Policy | Skipped on panel (%) | Policy | Reference | Mean saving [95% CI] | Relative saving (%) [95% CI] | Policy | Reference |
| joint, | 39.8 | 1,573 | 2,056 | 483 [365, 604] | 23.5 [17.6, 29.5] | 538 / 39 / 797 / 199 | 534 / – / 1,283 / 239 |
| joint, | 50.8 | 1,400 | 2,017 | 618 [501, 732] | 30.6 [24.7, 36.6] | 526 / 39 / 644 / 190 | 524 / – / 1,255 / 238 |
| label-free | 56.2 | 1,355 | 2,051 | 697 [581, 818] | 34.0 [28.1, 40.0] | 536 / 40 / 588 / 190 | 532 / – / 1,278 / 241 |
| Path | Accuracy (%) [95% CI] | Mean generation time (ms) |
|---|---|---|
| C alone | 61.7 [58.7, 64.7] | 197 |
| A+C (cheap path) | 63.1 [60.2, 66.2] | 715 |
| A + B without facts + C | 63.6 [60.7, 66.6] | 2,188 |
| A+B+C (reference) | 77.5 [74.9, 80.1] | 2,092 |
| B+C (no A) | 85.2 [83.0, 87.3] | 1,205 |
| C reading the two facts | 93.2 [91.7, 94.8] | 195 |
| # | Deviation | When |
|---|---|---|
| 1 | Leak shares quoted in the protocol were measured on 96 prompt-tuning questions; full-split values are reported (superseded by 9). | before all outputs |
| 2 | Generation ran on single GPUs (A, B, and C on one A100); the timing layout is unchanged. | before all outputs |
| 3 | Scheduling fix (GPU lock, chained submission) after two jobs collided before writing any record. | before all outputs |
| 4 | Fit-split controls (C alone, C reading the facts) added; descriptive. | before all outputs |
| 5 | The freeze record names its parent commit; the freeze commit is identified by a tag. | before all outputs |
| 6 | Remaining shards ran on a second cluster with the same GPU model and environment, behind an exact-match gate on four fit questions, with a per-shard lock; scheduling-only extensions followed. | before all outputs; extended after Cal and after Dev |
| Mean latency (ms) | Correct answers | |||||||
| Population | Reference | Skipped | Policy | Reference | Saving (ms) [95% CI] | Policy | Reference | acc (points) [95% CI] |
| (a) Development end-to-end replay on 128-question panels, OBQA and ARC | ||||||||
| medium/OBQA | C2C | 66/128 | 335.6 | 415.2 | 79.7 [48.1, 129.1] | 94 | 93 | [0.00, ] |
| medium/ARC | C2C | 77/128 | 352.4 | 453.1 | 100.7 [74.5, 133.1] | 85 | 85 | [0.00, 0.00] |
| large/OBQA | Text | 97/128 | 505.1 | 1022.2 | 517.1 [447.8, 583.6] | 112 | 113 | [ , ] |
| large/OBQA | C2C | 97/128 | 325.9 | 362.4 | 36.6 [18.0, 56.8] | 102 | 101 | [0.00, ] |
| Saving (ms) [95% CI] | ||||||
|---|---|---|---|---|---|---|
| Pair | Task | Reference | Busy GPU time | Latency | Busy minus latency | Busy / latency |
| large | OBQA | Text | 513.5 [448.7, 574.2] | 517.1 [447.8, 583.6] | [ , ] | 0.993 |
| large | OBQA | C2C | 32.3 [14.4, 51.8] | 36.6 [18.0, 56.8] | [ , ] | 0.882 |
| large | ARC | Text | 705.2 [659.1, 750.0] | 717.6 [670.6, 763.2] | [ , ] | 0.983 |
| large | ARC | C2C | 106.6 [77.8, 138.2] | 110.9 [82.0, 142.8] | [ , ] | 0.961 |
| First configuration | Second configuration | ||||||
| Pair | Task | Reference | Original run | Repeat 1 | Repeat 2 | Repeat 3 | Saving class |
| (a) All requests | |||||||
| large | OBQA | Text | 517.1 [447.8, 583.6] | 395.5 [325.3, 458.3] | 427.6 [376.4, 474.4] | 401.8 [335.6, 460.6] | ✓ above 0 |
| large | OBQA | C2C | 36.6 [18.0, 56.8] | 29.4 [14.9, 45.5] | 30.3 [15.7, 46.5] | 30.4 [15.5, 46.6] | ✓ above 0 |
| large | ARC | Text | 717.6 [670.6, 763.2] | 573.5 [536.0, 608.7] | 585.4 [546.7, 622.9] | 574.2 [536.3, 611.2] | ✓ above 0 |
| large | ARC | C2C | 110.9 [82.0, 142.8] | 91.4 [68.9, 116.2] | 90.5 [67.3, 115.4] | 90.2 [66.9, 115.8] | ✓ above 0 |
| Paired mean saving (ms) [95% CI] | |||||
|---|---|---|---|---|---|
| Pair | Task | Reference | Original | Reuse | Argmax |
| large | OBQA | Text | 436.3 [378.6, 503.2] | 437.8 [378.1, 506.0] | 589.7 [518.0, 670.3] |
| large | OBQA | C2C | 31.2 [16.7, 47.3] | 31.0 [16.5, 47.0] | 182.9 [158.2, 208.1] |
| large | ARC | Text | 565.6 [528.6, 601.4] | 564.4 [527.5, 599.6] | 762.7 [717.4, 804.3] |
| large | ARC | C2C | 91.9 [69.5, 116.6] | 90.5 [68.3, 115.0] | 280.8 [252.7, 310.4] |
| medium | OBQA | C2C | 50.1 [36.0, 65.6] | 49.9 [35.7, 65.6] | 125.2 [98.3, 154.9] |
| Coverage | Largest SMD | Eq. ( 4 ) saving (ms) with | ||||
|---|---|---|---|---|---|---|
| Policy | panel | population | value | feature | panel coverage | population coverage |
| large/OBQA/Text | 0.758 | 0.771 | 0.136 | helper message | 517.2 | 527.1 |
| large/OBQA/C2C | 0.758 | 0.771 | 0.115 | reference output | 36.6 | 38.0 |
| large/ARC/Text | 0.938 | 0.950 | 0.053 | question | 717.8 | 727.8 |
| large/ARC/C2C | 0.898 | 0.903 | 0.057 | reference output | 110.9 | 111.7 |
| large/MMLU-Pro/Text | 0.375 | 0.405 | 0.212 | receiver output | 317.0 | 346.0 |
| Mean latency (ms) | Saving (ms) [95% CI] | Throughput | |||||
|---|---|---|---|---|---|---|---|
| Policy | Refer- ence | Whole batch | Early return | Early return | Whole batch | Reference / whole batch / early return | |
| large/OBQA/Text | 1 | 797 | 380 | 376 | 421 [402, 441] | 416 [395, 437] | 1.26 / 2.63 / 2.66 |
| 8 | 1,219 | 1,044 | 613 | 606 [565, 647] | 176 [116, 239] | 6.55 / 7.64 / 6.75 | |
| 32 | 2,293 | 2,134 | 1,232 | 1,061 [1,000, 1,120] | 159 [92, 224] | 13.73 / 14.72 / 14.75 | |
| large/ARC/Text | 1 | 866 | 285 | 285 | 581 [558, 604] | 581 [558, 604] | 1.15 / 3.51 / 3.51 |
| 8 | 1,420 | 720 | 520 | 900 [852, 948] | 700 [568, 823] | 5.54 / 10.88 / 10.18 | |
| Versus , at / 32 | |||||
|---|---|---|---|---|---|
| Policy | Skipped at / 8 / 32 | Changed among skipped (%) at / 8 / 32 | Routes | Policy answers | Reference answers |
| large/OBQA/Text | 571 / 573 / 572 | 19 (3.3) / 18 (3.1) / 19 (3.3) | 6 / 3 | 5 / 5 | 4 / 7 |
| large/ARC/Text | 284 / 283 / 283 | 9 (3.2) / 9 (3.2) / 8 (2.8) | 1 / 3 | 0 / 2 | 2 / 1 |
| Llama-8B/OBQA/Text | 438 / 442 / 440 | 7 (1.6) / 7 (1.6) / 7 (1.6) | 6 / 6 | 9 / 9 | 7 / 4 |
| Llama-8B/ARC/Text | 206 / 206 / 206 | 4 (1.9) / 5 (2.4) / 5 (2.4) | 2 / 4 | 5 / 3 | 3 / 2 |
| Policy | Changed / skipped | Change rate (%) [95% CI] | Level | Always skip (%) | |
|---|---|---|---|---|---|
| medium/OBQA/C2C, | 8/388 | 2.06 [0.89, 4.02] | 0.025 | 17.47 | |
| medium/OBQA/C2C, | 6/346 | 1.73 [0.64, 3.74] | 0.025 | 17.47 | |
| large/OBQA/Text, | 12/595 | 2.02 [1.05, 3.50] | 0.001 | 8.20 | |
| large/OBQA/C2C, | 13/595 | 2.18 [1.17, 3.71] | 0.001 | 8.33 | |
| large/OBQA/Text+fact, | 16/558 | 2.87 [1.65, 4.61] | 0.025 | 11.02 |
| Coverage (%) | |||||||
|---|---|---|---|---|---|---|---|
| Policy | Test questions | Test | Develop- ment | Changed / skipped | Change rate (%) [95% CI] | Always skip (%) | |
| Llama-8B/OBQA/Text, | held-out OBQA (744) | 59.1 | 59.0 | 3/440 | 0.68 [0.14, 1.98] | 11.96 | |
| Llama-8B/ARC/Text, | ARC test (1,172) | 67.7 | 68.9 | 15/794 | 1.89 [1.06, 3.10] | 10.75 | |
| medium/ARC/C2C, | ARC test (1,172) | 62.5 | 63.2 | 15/733 | 2.05 [1.15, 3.35] | 14.85 | |
| Coverage (%) | Changed at | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Policy | (%) | Skipped | rotated | original | Changed | Change rate (%) [95% CI] | Original order: changed / skipped | |||
| large/ARC/Text | 72.1 | 1,100 | 93.9 | 93.9 | 27 | 2.45 [1.62, 3.55] | 0 | 27 | 20/1,100 | |
| large/ARC/C2C | 72.1 | 1,049 | 89.5 | 89.3 | 18 | 1.72 [1.02, 2.70] | 3 | 15 | 11/1,047 | |
| large/OBQA/Text | 46.1 | 582 | 78.2 | 80.0 | 13 | 2.23 [1.19, 3.79] | 1 | 12 | 12/595 | |
| large/OBQA/C2C | 46.1 | 582 | 78.2 | 80.0 | 12 | 2.06 [1.07, 3.57] | 1 | 11 | 13/595 | |
| large/OBQA/Text+fact | 46.1 | 545 | 73.3 | 75.0 | 14 | 2.57 [1.41, 4.27] | 2 | 12 | 16/558 | |
| Skipped questions with | Skipped questions with | |||||||
|---|---|---|---|---|---|---|---|---|
| Setting | share (%) | change rate (%) [95% CI] | change rate (%) [95% CI] | calibration | ||||
| large/OBQA/Text | 0.80 | 572 | 61.9 | 2/354 | 0.56 [0.07, 2.03] | 17/218 | 7.80 [4.61, 12.19] | 0.214 |
| large/OBQA/C2C | 0.80 | 572 | 61.9 | 1/354 | 0.28 [0.01, 1.56] | 12/218 | 5.50 [2.88, 9.42] | 0.46 |
| large/ARC/Text | 0.95 | 284 | 76.4 | 0/217 | 0.00 [0.00, 1.69] | 9/67 | 13.43 [6.33, 23.97] | 0.427 |
| large/ARC/C2C | 0.90 | 270 | 80.4 | 1/217 | 0.46 [0.01, 2.54] | 2/53 | 3.77 [0.46, 12.98] | 0.765 |
| medium/OBQA/C2C | 0.55 | 390 | 82.3 | 7/321 | 2.18 [0.88, 4.44] | 4/69 | 5.80 [1.60, 14.18] | 0.979 |
| Development AUROC | Within | |||||||
|---|---|---|---|---|---|---|---|---|
| Policy | all | [95% CI] | disagreements / questions | tie-broken | AUROC | disagreements / questions | (%) | Disagreements at (%) |
| medium/OBQA/C2C | 0.840 | 0.739 [0.690, 0.789] | 124/421 | 0.848 | 0.777 | 7/321 | 43.3 | 94.7 |
| medium/ARC/C2C | 0.910 | 0.748 [0.658, 0.837] | 47/125 | 0.913 | 0.682 | 1/174 | 58.2 | 97.9 |
| large/OBQA/Text | 0.873 | 0.769 [0.707, 0.826] | 70/388 | 0.876 | 0.753 | 2/354 | 47.7 | 97.2 |
| large/OBQA/C2C | 0.883 | 0.779 [0.724, 0.831] | 59/388 | 0.884 | 0.697 | 1/354 | 47.7 | 98.3 |
| large/ARC/Text | 0.952 | 0.796 [0.686, 0.896] | 15/82 | 0.952 | – | 0/217 | 72.6 | 100 |
| Calibration, | Development, | Development, deployed | ||||
|---|---|---|---|---|---|---|
| Setting | Coverage (%) | Change rate (%) [95% CI] | Coverage (%) | Change rate (%) [95% CI] | ||
| large/OBQA/Text | 1/642 | 47.7 | 0.56 [0.07, 2.03] | 77.1 | 3.32 [2.01, 5.14] | |
| large/OBQA/C2C | 2/642 | 47.7 | 0.28 [0.01, 1.56] | 77.1 | 2.27 [1.22, 3.86] | |
| large/ARC/Text | 0/321 | 72.6 | 0.00 [0.00, 1.69] | 95.0 | 3.17 [1.46, 5.93] | |
| large/ARC/C2C | 2/321 | 72.6 | 0.46 [0.01, 2.54] | 90.3 | 1.11 [0.23, 3.21] | |
| large/MMLU-Pro/Text | 14/1442 | 24.4 | 1.24 [0.54, 2.43] | 40.5 | 3.74 [2.69, 5.06] | |
| Skipped, | Skipped, | Routed | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Policy | ms | ms | ms | Saving (ms) [95% CI] | Share from | Recomposed (ms) [95% CI] | |||
| large/OBQA/Text | 332.9 | 60 | 194.5 | 37 | 31 | 517.1 [447.8, 583.6] | 0.369 | 294.9 [211.5, 372.4] | |
| large/OBQA/C2C | 29.1 | 60 | 17.7 | 37 | 31 | 36.6 [18.0, 56.8] | 0.378 | 6.7 [ , ] | |
| large/ARC/Text | 557.3 | 93 | 163.1 | 27 | 8 | 717.6 [670.6, 763.2] | 0.226 | 545.6 [476.1, 614.0] | |
| large/ARC/C2C | 94.7 | 93 | 20.5 | 22 | 13 | 110.9 [82.0, 142.8] | 0.178 | 83.2 [55.2, 111.0] | |
| large/MMLU-Pro/Text | 204.5 | 30 | 144.0 | 18 | 80 | 346.3 [249.8, 443.0] | 0.413 | 196.0 [113.6, 287.0] | |
| Development questions with | Development questions with | |||||||
|---|---|---|---|---|---|---|---|---|
| Receiver | Task | Changed | Change rate (%) [95% CI] | Changed | Change rate (%) [95% CI] | Both answers parse: changed / questions | ||
| small | OBQA | 0 | – | – | 742 | 558 | 75.2 [71.9, 78.3] | 521/696 |
| small | ARC | 0 | – | – | 299 | 209 | 69.9 [64.4, 75.0] | 180/258 |
| medium | OBQA | 321 | 19 | 5.9 [3.6, 9.1] | 421 | 164 | 39.0 [34.3, 43.8] | 176/732 |
| medium | ARC | 174 | 11 | 6.3 [3.2, 11.0] | 125 | 42 | 33.6 [25.4, 42.6] | 51/297 |
| large | OBQA | 354 | 9 | 2.5 [1.2, 4.8] | 388 | 128 | 33.0 [28.3, 37.9] | 134/739 |
| Method family | Controlled quantity | Finite-sample guarantee | Cheap path is a prefix |
|---|---|---|---|
| Risk-controlled early exit | agreement with the full model | yes | yes |
| Routing and cascades | correctness or quality | some | no |
| Online cascade learning | agreement with the large model | no | no |
| Selective LLM judging | agreement with human labels | yes | no |
| Skipping a message (ours) | agreement with a deployed path | yes | no |
Multi-Agent LLM Systems
Multi-agent large language model systems coordinate multiple AI agents to solve tasks through communication, specialization, shared tools, and memory.
See all 1,119 papers