Concurrent multi-agent coding promises division of labor across modules, robustness through redundancy, and parallel exploration at the natural granularity of multi-file projects. Realtime collaborative editing protocols solve this coordination problem for human teams via Conflict-free Replicated Data Types (CRDTs), but the LLMs underneath generate one token at a time and existing multi-agent coding systems inherit this serial limit: they either sequence agents through phase handoffs or pool independent samples without coordination, and a single agent abandons up to half of hard tasks with a one-file stub-and-exit. AgentRoom is a realtime collaborative editing protocol for concurrent coding agents. Its runtime layer exposes file-level claim, status, and broadcast as MCP tools on a CRDT-merged shared filesystem. Five frontier coding-CLI models ran four backend coding tasks, with cross-language checks in Python DevBench and Rust+axum. For CLI-stable models, AgentRoom with 2 agents abandons fewer tasks than Solo and has less run-to-run variation. At matched-compute, one positive mean LLM-judge contrast puts AgentRoom over parallel-merge. The other contrast, a bundle probe, puts full AgentRoom above each partial case: an ordering rather than a percentage split. Coordination, not parallelism or CRDT-merge, bears the load.
Figures & tables
Figure 1: T4 six-condition coordination ablation: AgentRoom is the top condition, while concurrency without coordination (parallel-merge) and sequential hand-off (ChatDev-style) fall below the single-agent baseline.
Figure 2: AgentRoom architecture: N coding agents share a workspace mediated by a CRDT; the MCP server exposes claim , release , broadcast , state , read .
Figure 3: Real-time CRDT merge during a single T4 Sonnet AgentRoom ×2 run. The MCP-mediated chat (left) drives concurrent character-level edits in the shared file (right); both α ’s and β ’s insertions co-exist on the same line via the Yrs CRDT, with no byte-level conflict raised in this run. Per-message Bezier leaders link each chat broadcast to the code line it talks about.
Figure 4: Solo (slate) vs AgentRoom ×2 (rust) score distributions for the three CLI-stable models on T4 (LLM-judge composite, budget-fair pool). σ contracts by ∼30 – 45 % under AgentRoom in every panel (Sonnet 0.23→0.14 , Haiku 0.25→0.17 , Codex 0.27→0.15 ); mean shift varies by model.
Condition
Mean
n
σ
Note
ChatDev-style (sequential 3-phase)
0.333
6
0.197
sequential paradigm baseline
Parallel merge (2 indep + union)
0.456
12
0.178
concurrent, no coordination
Solo (1 agent)
0.544
32
0.222
single-agent baseline
Shared only (2 agents, no MCP)
0.575
11
0.155
shared CRDT, no signalling
Shared + collab prompt, noMCP
0.588
7
0.188
+ prompt, no MCP tools
AgentRoom (2 agents, full)
0.669
14
0.140
CRDT + room MCP + collab prompt
Table 1: T4 Sonnet 4.6 six-condition ablation (LLM-judge composite, budget-fair 30 – 700 s pool). Parallel-merge vs AgentRoom: +0.213 , Welch t=3.35 , p=0.003 . The shared+collab-noMCP probe ran under a 1200 s wall budget; its runs are filtered to the same 30 – 700 s envelope as every other cell (Appendix B.13 ). Cross-model replication in Appendix B , Table 8 .
Figure 5: Agent-count scaling on T4 Sonnet 4.6 ( N=1 – 4 , budget-fair pool). Left: mean LLM-judge composite with σ band peaks at N=2 then declines. Middle: mean tests passing peaks at N=3 . Right: AgentRoom message count rises super-linearly past N=3 , consistent with a single-broadcast-channel coordination overhead.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Cochran-Mantel-Haenszel forest plot of Solo-vs-AgentRoom 1 -file abandonment across all 12 model × task strata (Sonnet/Haiku/Codex × T1/T2/T4/T5).
Figure 7: Bootstrap effect-size distributions ( 10000 resamples) for the Tier III cross-model cells. Only Haiku T4 has its 95% CI strictly above 1 ; the other five cells span 0.79 – 1.96× point-estimate with CIs that include 1 .
Model
SWE-bench V.
Solo (n)
× 2 (n)
ES (AST / regex)
95% CI (AST)
Haiku 4.5
73.3%
0.520 (13)
0.929 (8)
1.79 × / 1.58 ×
[1.24, 3.35] ∗
Codex GPT-5.4
∼ 80%
0.248 (9)
0.487 (13)
1.96 × / 1.07 ×
[0.85, 8.85]
Sonnet 4.6
79.6%
0.463 (11)
0.630 (15)
1.36 × / 1.14 ×
[0.84, 2.29]
Appendix
Table 2: Cross-model T4 ES with bootstrap 95% CI (10 000 resamples), AST-scored subsample of the full pool ( ≥30 s elapsed). n is smaller than the LLM-scored pools because the AST scorer covers a subsample. Headline scorer is AST (TypeScript compiler API); regex-scorer values after the slash for cross-check. ∗ CI excludes 1. Two CLI-unstable models (Gemini, GPT-5.4-mini) excluded (see § 4.4 ). Reported as exploratory conjecture: only Haiku T4 clears the CI under the AST scorer.
Task
Solo (n)
×2 (n)
ES
T1
0.919 (4)
0.910 (7)
0.99 ×
T2
0.840 (3)
0.904 (4)
1.08 ×
T3
0.887 (3)
0.943 (4)
1.06 ×
T4
0.796 (10)
0.958 (10)
1.20 ×
T5
0.697 (7)
0.810 (7)
1.16 ×
Appendix
Table 3: Sonnet 4.6 ES across the difficulty gradient (regex/AST scorer, outlier-filtered ≥0.3 ).
Pair
Score (n)
Tests
Sonnet × 2 (homo)
0.669 (14)
32.8
Sonnet+Codex
0.721 (3)
16.0
Sonnet+Gemini
0.542 (3)
22.0
Appendix
Table 4: Heterogeneous vs. homogeneous ×2 on T4 (LLM-judge, budget-fair). Tests is the mean over all budget-fair runs in the cell ( n=17 / 4 / 3 ), which is not the LLM-scored n shown in the Score column.
Figure 8: Cost–quality Pareto on T4. Shaded region is Pareto-dominated; dotted lines are iso-efficiency (quality/cost) contours. 2× Haiku AgentRoom clears 1× Sonnet at matched plotted cost.
Condition
N
Wall (s)
Compute (s)
Quality
Solo
1
550
550
0.544
Parallel merge
2
558
1077
0.456
Shared only
2
601
1201
0.575
AgentRoom × 2
2
561
1072
0.669
AgentRoom × 3
3
566
1572
0.553
Appendix
Table 5: Wall vs compute on T4 Sonnet (LLM-judge composite, budget-fair pool).
Model
Solo (n)
× 2 (n)
× 3 (n)
ES( × 2)
Tests ( × 2)
Haiku 4.5
0.416 (8)
0.707 (7)
0.822 (1)
1.70 ×
38
Sonnet 4.6
0.618 (7)
0.674 (7)
0.730 (2)
1.09 ×
57
GPT-5.4
0.667 (9)
0.531 (7)
0.761 (1)
0.79 ×
8
Gemini Flash (unstable)
0.851 (3)
0.506 (9)
0.246 (3)
0.59 ×
12
Appendix
Table 6: T5 (trading platform, 900 s budget), AST scorer, per-model cells. All CLI-stable cells are point-estimate-positive on ×2 except Codex; CIs do not exclude 1 at this n . The Gemini row is CLI-unstable and informational only (§ 4.4 ); sparse heterogeneous cells are in the text above.
Behavior
Runs exhibiting (of 6)
Module claiming via broadcast
6/6
Plan adjustment after reading teammate
5/6
Export list shared for teammate
4/6
Proactive gap filling
3/6
Cross-agent bug fix
2/6
Apology for file boundary violation
2/6
Appendix
Table 7: Frequency of unprompted collaborative behaviors across 6 valid ×2 AgentRoom runs on T4 Sonnet.
Baselines and partial bundles
Ours
Model
Solo
Parallel merge
Shared only
noMCP
AgentRoom ×2
Sonnet 4.6
0.544 ( n=32 )
0.456 ( n=12 )
0.575 ( n=11 )
0.588 ( n=7 )
0.669 ( n=14 )
Haiku 4.5
0.530 ( n=35 )
0.383 ( n=12 )
0.657 ( n=7 )
0.579 ( n=6 )
0.662 ( n=17 )
Codex GPT-5.4
0.384 ( n=12 )
–
–
–
0.529 ( n=17 )
Appendix
Table 8: Cross-model T4 ablation (LLM-judge composite, budget-fair pool). For Sonnet and Haiku, AgentRoom is the top condition and parallel-merge falls below solo. Codex non-Solo/AgentRoom cells were not collected (vendor-CLI hang).
Thresholds
Sonnet 4.6
Haiku 4.5
Codex GPT-5.4
CMH pooled
d thr
f thr
Solo
Room
Solo
Room
Solo
Room
OR
95% CI
p
<150
≤1
6/33
0/17
12/35
1/17
9/17
0/21
24.2
[3.5, 169]
0.00001
<150
≤2
6/33
0/17
12/35
1/17
9/17
2/21
11.9
[3.0, 47]
0.00005
<150
≤3
6/33
0/17
12/35
1/17
10/17
6/21
6.3
[2.1, 19]
0.00069
<200
≤1
6/33
0/17
12/35
1/17
9/17
0/21
24.2
[3.5, 169]
0.00001
<200
≤2 (paper)
6/33
0/17
12/35
1/17
9/17
2/21
11.9
[3.0, 47]
0.00005
Appendix
Table 9: Threshold sensitivity for the Tier I CMH. Each row is a threshold combination; cells report Solo/AgentRoom abandonment counts per model and the resulting CMH common OR with 95% CI and p -value.
Pool
Solo
Parallel merge
Shared only
AgentRoom ×2
30 – 700 s (paper, budget-fair)
0.544 ( n=32 )
0.456 ( n=12 )
0.575 ( n=11 )
0.669 ( n=14 )
≥30 s (full pool)
0.464 ( n=42 )
0.436 ( n=13 )
0.527 ( n=14 )
0.639 ( n=15 )
Appendix
Table 10: Pool sensitivity for the T4 Sonnet ablation (LLM-judge composite). The substrate-vs-AgentRoom ordering is preserved under both pools; the budget-fair pool is primary.
Model
Solo (bare, budget-fair)
Solo + collab prompt ( n=3 )
AgentRoom ×2 (budget-fair)
Sonnet 4.6
0.544 ( n=32 )
0.448 (range 0.20–0.62)
0.669 ( n=14 )
Haiku 4.5
0.530 ( n=35 )
0.604 (range 0.55–0.66)
0.662 ( n=17 )
Appendix
Table 11: Solo + collaboration-style prompt without teammates, T4, n=3 each (LLM-judge composite). The structured prompt alone is model-dependent: it hurts Sonnet (channel is the load-bearing variable) and helps Haiku (the structured prompt also acts as workflow scaffolding); the channel and prompt-as-workflow are not separately identified for Haiku at this n .
Task
Solo (PASS, dur)
× 2 AgentRoom (PASS, dur)
arxiv-digest
PASS, 70 s
PASS, 120 s
readtime
PASS, 110 s
PASS, 100 s
chakin
PASS, 130 s
PASS, 130 s
geotext
FAIL, 603 s timeout
PASS, 390 s
hone
PASS, 90 s
PASS, 160 s
hybrid-images
PASS, 100 s
PASS, 170 s
Appendix
Table 12: DevBench cross-domain validation (Haiku 4.5, 600 s budget). Pass = hidden pytest suite passes; durations in seconds. geotext : ×2 completes within budget while solo times out; stocktrends, textcnn : both fail (task too hard for budget).
Figure 9: Single representative T4 Sonnet AgentRoom ×2 run. Top swim-lane: agent α broadcasts (timestamps in-card); middle: per-file ownership Gantt (rust = α , slate = β ); bottom: agent β . Bezier S-curve leaders link each chat broadcast to its time on the Gantt. The whole protocol (claim, conflict-detect, hand-off, completion broadcast) emerges from the advisory six-step prompt (§ 2.3 ) without any orchestrator.
Figure 10: The two scorers measure the same thing loosely, not interchangeably. Each point is one T4 run carrying both a regex composite and an LLM-judge composite ( n=246 , the runs scored by both); Pearson r=0.67 . The spread off the diagonal is why the body uses one scorer throughout and reports the other as a cross-check rather than averaging them.
Removed dimension
Solo mean
× 2 mean
ES
(none, full scorer)
0.786
0.928
1.18
Type Safety
0.721
0.904
1.25
Defensive Coding
0.815
0.944
1.16
Test Quality
0.854
0.956
1.12
Design
0.793
0.921
1.16
Test Pass Rate
0.749
0.916
1.22
Appendix
Table 13: Scorer sensitivity: leave-one-out on T4 (Sonnet 4.6, solo vs × 2 AgentRoom). The solo mean reported here (0.786) is computed over a smaller subset (4 runs) than the canonical solo subset used in Table 1 ; the smaller subset is the intersection of solo runs for which we retained per-dimension intermediate scorer outputs at the time of writing, kept here only as a robustness check. The AgentRoom > solo ordering is preserved under every single-dimension ablation and the ratio stays within 1.12–1.25 on this subset.
Understanding large codebases is a long-horizon task for Large Language Model (LLM) agents: answering a single question can require building and running the software, tracing execution across files, and synthesizing evidence over tens of minutes. On SWE-Atlas QnA, a benchmark of long-horizon questions over production repositories, a single Claude Code agent (Opus 4.6) resolves only 32.3% of tasks. Dividing the work among agents with clean contexts mitigates this limitation. However, the subtasks of code comprehension are interdependent. One agent's findings can rewrite another's task, so agents must coordinate during execution, not only at phase boundaries. Existing multi-agent systems support such exchange only between phases, through staged handoffs or synchronized rounds. Communication and work remain mutually exclusive. A discovery made mid-execution cannot be shared until the next boundary. We present AgentRadio, an asynchronous message-passing layer that equips coding-agent harnesses with three primitives: threads, messages, and waiting for mentions. The last runs as a background task, surfacing teammates' messages without interrupting foreground work, so each agent remains passively aware of its peers and folds new findings into its ongoing task. Under a five-phase protocol of division of labor and negotiation, four agents organized by AgentRadio resolve 62.1% of tasks, 29.8 points above a single agent and above Claude Code with the newer Opus 4.8 (57.2%). Rubric-level analysis shows the gain growing with task difficulty, consistent with mid-course correction as the underlying mechanism. Our code is available at https://github.com/Coral-Protocol/AgentRadio.
Xinxing Ren, Qianbo Zang, Ziyan Wang +4
1Coral AI Labs · 2SnT, Université du Luxembourg · 3King’s College London +1
Multi-agent coding systems are designed to tackle complex software engineering tasks through collaboration. However, existing evaluations typically assume configured organizations are followed faithfully, whereas reality differs. This behavioral gap, combined with differences in underlying system components, prevents clear attribution of observed gains. To this end, we introduce OpenCollab, a multi-agent coding framework that provides a unified infrastructure for programmable collaboration and controllable runtime. Specifically, OpenCollab unifies organization design, enforces experimental control on a shared runtime, and tracks execution through fine-grained event streams. On this basis, we define Adherence to quantify whether the declared organization is actually realized. Our experiments reveal that agents collaborate very differently across configurations: changing any single dimension shifts Adherence, from 47.2% to as high as 97.2%. Furthermore, extensive agentic coding benchmarks show that OpenCollab (Duo) establishes new SOTA performance compared to the mainstream harnesses such as Mini-SWE-agent, OpenHands, Codex CLI, and Claude Code, showing that a well-designed organization can outperform strong existing harnesses, while OpenCollab's single-agent configuration uses the fewest tokens across all evaluated suites. OpenCollab establishes a unified multi-agent infrastructure for easy programmable collaboration and controlled causal evaluation.
Chun-Wah Hsu, Kai Gong, Yu Wu +12
Shanghai Jiao Tong University · University of Cambridge · Nanyang Technological University +4
Multi-agent Large Language Model (LLM) systems offer a way to decompose complex tasks, such as coding, through parallelization and context isolation. However, adding agents in practice introduces inter-agent communication overhead, which incurs extra cost and can sometimes offset the efficiency gains. We formalize multi-agent orchestration as a graph partitioning problem that captures the communication-to-computation trade-off: task decomposition can shorten critical-path computation, but cross-agent dependencies require costly context transfer. We instantiate this view in repository-level software engineering and present Cohesion-aware Coder (Co-Coder), which builds dependency graphs from static analysis, isolates structural hub files, partitions the graph via community detection, and executes the partition with a dependency-aware scheduler. Across 28 real-world tasks on DevEval and CodeProjectEval, Co-Coder advances the Pareto-frontier over sequential and file-based parallel baselines as well as Claude Code with Agent Teams, lifting pass rate by up to 14.0%, achieving up to a 2.10x wall-clock speedup, and reducing API cost by up to 35%, with the largest gains on the most dependency-dense projects. Co-coder demonstrates how cohesion-aware orchestration can make parallel coding agents both theoretically grounded and practically efficient, suggesting a broader design principle for multi-agent systems.
Xu Yang, Lunyiu Nie, Ethan Chandra +3
1The University of Texas at Austin · University of Oxford