The Coordination Tax and the Heterogeneity Dividend: Break-Even Conditions for Multi-Agent LLM Pipelines
Authors: Andreas Zettl, Diego Gosmar
Organizations: Member, VoiceInteroperability.ai Linux Foundation AI & Data Berlin, Germany · Member, VoiceInteroperability.ai Linux Foundation AI & Data Turin, Italy
Multi-agent LLM pipelines built on interoperability protocols such as the Open Floor Protocol (OFP) or A2A are usually justified on capability grounds, with their runtime cost treated as an overhead of unknown size. Comparing such a pipeline with one model call is underspecified, because a pipeline can give each role a different model. We give a closed-form model that sets the coordination tax (decomposition overhead plus per-handoff protocol cost) against the heterogeneity dividend from routing work to a cheaper model, with a break-even threshold and a feasibility bound, α<1/r, and measure every term, with OFP as the protocol under test, on 135 HumanEval and ClassEval tasks in 2430 runs of six architectures with Claude Opus 5.5 and Haiku 5.5. The protocol adds 7.2 ms per handoff, 0.11% of pipeline wall time. Decomposition does not pay on these tasks: a planner-coder-reviewer pipeline costs 3.22x as much as a single strong call, takes 3.24x as long, and solves no more tasks. Moving planner and reviewer to the cheap model cuts the pipeline's cost to 0.39x but changes pass@1 by -6.7 points, leaving it dearer, slower and less accurate than the single call, and the break-even forecast misses by -19% because the cheap roles inflate token volume. The saving lies in choosing a model per task instead: an oracle choice between the two single calls solves 94.3% of runs at half the strong call's cost per solved task.
Figures & tables
r=0.50
r=0.33
r=0.20
r=0.025
α=1.5
0.67
0.50
0.42
0.34
α=2.0
1.00
0.75
0.63
0.51
α=2.5
—
0.90
0.75
0.62
α=3.0
—
0.99
0.83
0.68
α=4.0
—
—
0.94
0.77
feasibility ceiling 1/r
2.00
3.03
5.00
40.0
Table 1: Minimum routable fraction f⋆ required for a decomposed pipeline to undercut a single strong call, by decomposition inflation α and price ratio r . Dashes mark infeasible regions where α≥1/r and no routing policy suffices. The last column is the price ratio of the two models in this study.
Figure 1: The break-even threshold f⋆=(α−1)/[α(1−r)] as a function of decomposition inflation, for four price ratios. Each curve terminates where it meets f⋆=1 , at α=1/r : to the right of that point the decomposed architecture cannot be made cheaper than a single strong call by any routing policy, however well the protocol is implemented. The curves are steepest near α=1 , meaning that the first increments of decomposition overhead are the expensive ones.
Figure 2: The six conditions. Top row: a single call on the strong model (A), and the three-role pipeline as plain calls (B), over OFP with the strong model in every role (C), and over OFP with the cheap model as planner and reviewer (D). Bottom row: the two baselines, a single call on the cheap model (A ′ ) and a router (R). Only B and C differ in whether handoffs traverse the protocol; only C and D differ in model assignment.
Condition
pass@1 (%)
$/solved
Cost ($)
Wall (s)
Tokens in / out (reasoning)
Function tier (HumanEval, 50 tasks)
A: single call, strong
100.0
0.0063
0.0056
2.8
274 / 261 (16)
A ′ : single call, cheap
98.7
0.00019
0.00014
1.7
274 / 323 (125)
R: router
98.7
0.00045
0.00022
2.7
609 / 396 (183)
B: multi-call, plain
100.0
0.0279
0.0272
12.8
1521 / 1091 (212)
C: multi-call, OFP uniform
100.0
0.0279
0.0270
13.0
1522 / 1090 (215)
Table 2: Results by tier: 3 runs per task and condition. pass@1 is the share of runs solved; cost per solved task is total spend divided by solved runs; cost and wall time are medians per run; token counts are means per run, and output includes the reasoning tokens shown in brackets.
Figure 3: Accuracy–cost frontier by tier. Each point is one condition: pass@1 against cost per solved task (log scale). Open circles are single calls (A, A ′ , R); filled squares are three-role pipelines (B, C, D). Conditions toward the upper left dominate those below and to their right.
Comparison
Cost ratio
p
Wall-time ratio
p
Δ pass@1 (pp)
p
B vs A
3.20 [3.07, 3.35]
<10−4
3.20 [3.06, 3.35]
<10−4
-0.5 [-3.2, +2.5]
0.31
C vs A
3.22 [3.10, 3.37]
<10−4
3.24 [3.10, 3.39]
<10−4
-1.5 [-4.0, +1.0]
0.28
C vs B
1.01 [0.99, 1.02]
0.36
1.01 [1.00, 1.03]
0.15
-1.0 [-3.5, +1.5]
0.38
D vs C
0.39 [0.38, 0.41]
<10−4
0.80 [0.77, 0.84]
<10−4
-6.7 [-11.1, -3.0]
0.002
D vs A
1.27 [1.21, 1.33]
<10−4
2.60 [2.46, 2.74]
<10−4
-8.1 [-12.3, -4.2]
<10−4
A ′ vs A
0.04 [0.04, 0.04]
<10−4
0.79 [0.73, 0.87]
<10−4
-24.2 [-31.4, -17.5]
<10−4
Table 3: Paired comparisons over all 135 tasks. Cost and wall-time ratios are ratios of mean per-task values; Δ pass@1 is the mean per-task difference in percentage points. Brackets give 95% bootstrap intervals over tasks; p -values are from the Wilcoxon signed-rank test on per-task differences.
Condition
Did not run
of which timeouts
Failed tests
Truncated
Refusals
A
1
0
24
0
0
A ′
2
0
121
0
0
R
1
0
115
0
0
B
0
0
27
0
1
C
1
0
30
0
0
D
0
0
58
0
0
Table 4: Unsolved runs by cause, over all tasks. “Did not run” includes timeouts, which are also counted separately. Truncated runs hit the 16,000-token output limit in at least one call; refusals count calls declined by a model’s safety classifiers.