Learning How Much to Collaborate: Difficulty-Aware Topology Selection for Multi-Agent Code Generation

Yunsong Hong

cs.MASep 12, 2026

Learning How Much to Collaborate: Difficulty-Aware Topology Selection for Multi-Agent Code Generation

Authors: Yunsong Hong

Organizations: School of Computer Science, The University of Sydney, Sydney, NSW 2006, Australia

Abstract

Multi-agent systems for code generation are deployed with a single communication topology, chosen once for every problem. This is the wrong granularity. Evaluating five topologies on 614 problems from APPS, HumanEval+ and LiveCodeBench, we find that the advantage of hierarchical collaboration over a single agent grows from 2.4 points of pass@1 on the easiest third of problems to 21.1 points on the hardest third, while its token cost stays about ten times higher. We propose the Difficulty-Aware Topology Selector (DATS), which predicts each topology's probability of solving a problem and selects the one maximising predicted success minus cost. Its predictor is a graph network that treats the five topologies as nodes of a connectivity order rather than independent labels, worth 1.7 points over a flat multi-label head. Because the cost penalty is a single scalar recalibrable without retraining, routers compare at equal spend: under this budget-matched protocol six cost-aware methods span 21.6 percentage points, and two baselines leading DATS fall behind once calibrated to it. Fixed at 40% of the always-hierarchical cost, DATS reaches 77.7% pass@1 against 73.6% (always-hierarchical) and 74.3% (strongest learned competitor), all eleven pairwise McNemar comparisons surviving Holm-Bonferroni correction. The 4.1-point gain holds across four backbones spanning fourteen points of capability, and replacing the 39 interpretable features with a graph network or a pretrained encoder shifts accuracy by at most 1.3 points, never significantly. A cross-domain study on 400 mathematical reasoning problems reproduces the effect, the gap widening from 2.5 to 20.9 points.

Figures & tables

Figure 1: The five collaboration topologies. Single issues one call. Chain passes a draft from proposer (P) to critic (C) to reviser (R). Star has a hub (H) decompose the problem and delegate to workers (W), whose returns are merged deterministically. FullMesh lets three peers (A) read each other’s drafts over two rounds. HRM adds a manager (M) above two leads (L), each supervising two workers (w), with review flowing upward. Filled nodes indicate agents that see the full problem statement.

Figure 2: Dats . A problem statement is mapped to 39 features and a shared trunk; a graph network then passes messages over the five candidate topologies, arranged by connectivity, and reads out one success probability per topology. The selector subtracts a cost term whose coefficient is calibrated to the target budget on held-out data. Only the selected topology is executed.

Figure 3: The topology-graph predictor. (a) A problem

x

is mapped by

\phi

to 39 difficulty features and a shared trunk to a context

z\in\mathbb{R}^{d}

. (b) The five candidate topologies are the nodes of the topology graph

G_{\mathcal{T}}

, joined along the connectivity order Single–Chain–Star–FullMesh with the hierarchical mesh composing Star and FullMesh; each node is initialised from

z

(dashed) and its own structural descriptor

e_{t}

(agent/edge counts shown), then exchanges messages with its structural neighbours (orange) for

L=2

rounds. (c) A per-node sigmoid readout produces one success probability per topology, which the cost-aware selector ranks. The graph the network reasons over is the output space of topologies, not the absent program graph of the input.

Suite	$n$	Difficulty	Words	Tests
APPS-Introductory	100	0.25	178	12.4
APPS-Interview	100	0.50	296	18.7
APPS-Competition	100	0.75	412	21.3
HumanEval+	164	0.20	67	24.6
LiveCodeBench	150	0.59	341	16.2
Total	614	0.44	232	19.0

Table 1: Composition of the evaluation set. Difficulty is the normalised statement rating; words and tests are per-problem means.

Topology	Calls	Tokens	Ratio	USD	Lat. (s)
Single	1	2,632	1.00	0.095	8.4
Chain	3	13,425	5.10	0.483	26.1
Star	4	14,083	5.35	0.507	21.7
FullMesh	6	23,004	8.74	0.828	44.3
HRM	5–8	26,189	9.95	0.943	38.6
Dats	2.6	10,476	3.98	0.377	17.1

Table 2: Measured cost profile per topology on Claude Opus 4. Tokens are per-problem means over the full grid; ratio is relative to Single.

Suite	Single	Chain	Star	FullMesh	HRM
APPS-Introductory	88.0	91.0	89.0	90.0	92.0
APPS-Interview	57.0	66.0	63.0	67.0	71.0
APPS-Competition	19.0	32.0	27.0	35.0	41.0
HumanEval+	90.9	93.3	92.1	94.5	95.1
LiveCodeBench	46.0	56.0	52.7	57.3	61.3
Overall	62.21	69.38	66.61	70.52	73.62

Table 3: pass@1 (%) by topology and suite on Claude Opus 4. The last row gives the number of solved problems out of 614.

Figure 4: pass@1 by difficulty tertile. Arrows mark the HRM-minus-Single gap, which widens from 2.4 to 21.1 points. Dats exceeds every fixed topology on the hard tertile while spending 40% of the HRM budget overall.

	Method	Solved	pass@1 (%)	95% CI	USD/problem	% of HRM cost
fixed	Always-Single	382	62.21	[58.3, 66.0]	0.095	10.1
	Always-Chain	426	69.38	[65.6, 72.9]	0.483	51.3
	Always-Star	409	66.61	[62.8, 70.2]	0.507	53.8
	Always-FullMesh	433	70.52	[66.8, 74.0]	0.828	87.8
	Always-HRM	452	73.62	[70.0, 77.0]	0.943	100.0
budget-matched	Random	421	68.57	[64.8, 72.1]	0.571	60.6

Table 4: Budget-matched comparison on Claude Opus 4, 614 problems, five-fold cross-validation. Learned routers are calibrated to

B=40\%

of the always-hierarchical cost. Fixed topologies are shown at their natural cost and are not budget-matched. Best budget-matched result in bold.

Figure 5: Cost–accuracy plane. The curve is Dats calibrated to budgets between 15% and 100% of the always-hierarchical cost. Filled markers are the fixed topologies; open markers are the cost-aware baselines at the matched budget of $0.377.

	shared $\lambda=0.2$				budget-matched at 40%
Method	pass@1 (%)	USD	% of HRM	rank	pass@1 (%)	USD	% of HRM	rank
RandomForest-Router	76.55	0.489	51.9	1	74.27	0.374	39.6	2
RouterR1-lite	75.90	0.442	46.9	2	73.45	0.376	39.9	3
Dats	75.41	0.324	34.4	3	77.69	0.377	40.0	1
MasRouter-lite	71.99	0.286	30.3	4	72.64	0.375	39.8	4
Dats -E2E	70.85	0.312	33.1	5	71.66	0.361	38.3	5

Table 5: The same six cost-aware routers under two conventions. Left: everyone receives the same coefficient

\lambda=0.2

. Right: everyone receives their own coefficient, calibrated to

B=40\%

of the always-hierarchical cost. Rank is by pass@1 within each half.

Alternative	$\Delta$	$b$	$c$	$\chi^{2}$	$p$	$p_{\text{Holm}}$
Always-Single	15.47	107	12	74.25	6.9e–18	7.6e–17
Always-Star	11.07	106	38	31.17	2.4e–8	2.4e–7
Always-Chain	8.31	79	28	23.36	1.3e–6	1.2e–5
Random	9.12	108	52	18.91	1.4e–5	1.1e–4
Difficulty-Threshold	7.33	79	34	17.13	3.5e–5	2.4e–4
Dats -E2E	6.03	64	27	14.24	1.6e–4	9.6e–4

Table 6: McNemar tests of Dats against each alternative, Holm–Bonferroni corrected over the family of eleven. Here

b

is the number of problems Dats solves and the alternative does not, and

c

the reverse.

Suite	Single	Chain	Star	FullMesh	HRM	Dats (%)	Always-HRM (%)	Oracle (%)
APPS-Introductory	71	21	3	2	3	92.0	92.0	98.0
APPS-Interview	39	33	6	8	14	74.0	71.0	88.0
APPS-Competition	6	22	7	18	47	50.0	41.0	66.0
HumanEval+	131	23	3	3	4	96.3	95.1	98.2
LiveCodeBench	54	64	9	11	12	68.7	61.3	88.7
Total	301	163	28	42	80	77.69	73.62	88.93

Table 7: Routing distribution and accuracy by suite. Counts are problems assigned to each topology by Dats ; the last three columns compare Dats against the always-hierarchical policy and the oracle on the same problems.

Figure 6: Share of problems routed to each topology, by suite. Segments below 9% are unlabelled. APPS-I, APPS-Int and APPS-C are the three APPS levels; HE+ is HumanEval+ and LCB is LiveCodeBench.

Figure 7: Share of problems routed to each topology, conditioned on algorithmic category. Categories are ordered by the share sent to a single call.

Backbone	Single	Chain	Star	FullMesh	HRM	Dats	Oracle	$\Delta$	$p$
Claude Opus 4	62.21	69.38	66.61	70.52	73.62	77.69	88.93	+4.07	0.0138
DeepSeek-V3	57.33	64.50	61.73	65.47	68.40	72.48	85.02	+4.08	0.027
GPT-4o-mini	51.47	59.12	56.35	60.42	63.36	67.59	80.94	+4.23	0.017
Qwen2.5-Coder-32B	48.37	55.21	53.09	57.17	60.10	64.17	78.01	+4.07	0.025

Table 8: Generalisation across backbones. Each row is an independent train–evaluate cycle at the same 40% budget.

\Delta

is Dats minus Always-HRM;

p

is the uncorrected McNemar value for that comparison.

Method	GSM8K	MATH	All	USD	% HRM
Single	93.5	62.0	77.75	0.041	10.3
Chain	95.0	68.5	81.75	0.209	52.5
Star	94.0	66.0	80.00	0.221	55.5
FullMesh	95.5	70.0	82.75	0.357	89.7
HRM	96.0	73.0	84.50	0.398	100.0
Dats	96.5	79.5	88.00	0.160	40.3

Table 9: Cross-domain check on 400 mathematical reasoning problems, Claude Opus 4, exact-match accuracy (%). Dats is calibrated to 40% of the always-hierarchical cost measured on this grid.

Figure 8: The collaboration gap widens with difficulty on mathematical reasoning as it does on code. Arrows mark the distance between a single agent and the hierarchical topology; Dats stays above both while spending 40% of the hierarchical budget.

Figure 9: Reliability of the predicted success probabilities, pooled over the five topology heads and all folds. Grey bars show the relative population of each bin.

Figure 10: Accuracy lost when each feature group is removed and the router is retrained at the same budget. Coloured bars are significant under a paired McNemar test at

p<0.05

Variant	pass@1 (%)	$\Delta$	USD
feature groups removed
Difficulty	73.45	$-4.24$	0.377
Algorithmic	74.59	$-3.10$	0.376
Complexity	75.08	$-2.61$	0.377
Semantic	75.24	$-2.45$	0.375
Structural	76.38	$-1.31$	0.377

Table 10: Ablations at the 40% budget. Upper panel removes one feature group; lower panel varies the architecture.

\Delta

is relative to the full model at 77.69%.

Encoder	Trainable	Fit time	pass@1 (%)	$\Delta$	$b$ / $c$	$p$
Dats : 39 interpretable features	13.7K	28 s CPU	77.69	—	—	—
Text-GNN: 2-layer GCN over the statement graph	0.21M	4 min GPU	76.55	$-1.14$	31/38	0.47
CodeBERT, frozen, plus the same head	0.11M	3 min GPU	76.87	$-0.82$	29/34	0.61
GraphCodeBERT, frozen, plus the same head	0.11M	3 min GPU	77.36	$-0.33$	27/29	0.89
GraphCodeBERT, fine-tuned end to end	125M	36 min GPU	78.18	$+0.49$	33/30	0.80
Dats -Hybrid: 39 features and frozen GraphCodeBERT	0.13M	6 min GPU	78.99	$+1.30$	36/28	0.38

Table 11: Representation ladder. Every row shares the topology-graph head, the selection rule, the folds and the 40% budget; only the encoder changes. Here

b

counts problems solved by that row and not by Dats ,

c

the reverse, and

p

is the uncorrected McNemar value against Dats . Fit time is the cost of training the router itself, not of running any topology.

Figure 11: Accuracy and realised spend as the cost coefficient varies. The shaded band marks the range of values selected by budget calibration across the five outer folds.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Pair	$b$	$c$	$\chi^{2}$	$p_{\text{Holm}}$
Single–HRM	29	99	37.20	1.1e–8	$\ast$
Single–FullMesh	68	119	13.37	2.3e–3	$\ast$
Single–Chain	61	105	11.14	6.8e–3	$\ast$
Star–HRM	60	103	10.82	7.0e–3	$\ast$
Single–Star	69	96	4.10	0.215
Chain–HRM	58	84	4.40	0.215

Appendix

Table 12: Pairwise McNemar tests over the five topologies, Holm–Bonferroni corrected over ten comparisons. For pair

A

–

B

b

counts problems solved by

A

only and

c

problems solved by

B

only.

Multi-Agent Coding

Multi-agent coding studies how AI agents can collaborate to generate, test, debug, and maintain software.

See all 82 papers

Feb 19, 2026

AgentConductor: Topology Evolution for Multi-Agent Competition-Level Code Generation

Large language model(LLM)-driven multi-agent systems(MAS) coordinate specialized agents through predefined interaction topologies and have shown promise for complex tasks such as competition-level code generation. Recent studies demonstrate that carefully designed multi-agent workflows and communication graphs can significantly improve code generation performance by leveraging collaborative reasoning. However, existing methods neither adapt topology density to task difficulty nor iteratively refine the topology within an instance using execution feedback, which leads to redundant communication and performance bottlenecks. To address these issues, we propose AgentConductor: a reinforcement learning-optimized MAS with an LLM-based orchestrator agent as its core, which enables end-to-end feedback-driven dynamic generation of interaction topologies. For each query, AgentConductor infers agent roles and task difficulty, then constructs a task-adapted, density-aware layered directed acyclic graph (DAG) topology, underpinned by two key innovations. First, we design a novel topological density function that captures communication-aware mathematical characterizations of multi-agent interactions. Second, we adopt difficulty interval partitioning to avoid excessive pruning for precise topological density upper bound measurement per difficulty level and finer-grained control. Empirically, across three competition-level and two foundational code datasets, AgentConductor achieves state-of-the-art accuracy, outperforming the strongest baseline by up to 14.6% in pass@1 accuracy, 13% in density reduction, and 68% in token cost reduction.

Siyu Wang, Ruotian Lu, Zhihao Yang +7

Shanghai Jiao Tong University · Meituan

LLM Agent Workflow Optimization Multi-Agent Coding Multi-Agent System Optimization RL for LLM Agents

Ask PDF

May 7, 2026

Retrieval-Conditioned Topology Selection with Provable Budget Conservation for Multi-Agent Code Generation

Multi-agent LLM systems for code generation face a fundamental routing problem: the optimal orchestration topology depends on the structural complexity of the code under modification, yet existing systems select topologies without consulting the codebase. We present Retrieval-Guided Adaptive Orchestration (RGAO), an architecture that closes this loop by extracting a structural complexity vector from a hierarchical code index before selecting the orchestration topology. RGAO operates within Code-Agent, a multi-agent framework whose sub-agents are governed by formal contracts with six-dimensional budget vectors. Our headline contribution is the composition of two previously separate lines of work -- complexity-conditioned LLM routing and formal resource algebras -- yielding a property neither admits alone: provable budget conservation under retrieval-conditioned dynamic topology selection. Concretely we contribute: (1) a complexity-conditioned topology router that reduces proxy-measured misrouting from 30.1% to 8.2%; (2) a budget algebra with a structural-induction conservation theorem; and (3) a hierarchical code retrieval engine. Empirical evaluation demonstrates sub-millisecond DAG construction and linear tree-index scalability.

Abhijit Talluri, Pujith Anne, Bhagavan Choudary Pendiyala +1

Independent Researcher

Multi-Agent Orchestration Multi-Agent Coding LLM Agent Routing

Ask PDF

Mar 20, 2026

GoAgent: Group-of-Agents Communication Topology Generation for LLM-based Multi-Agent Systems

Large language model (LLM)-based multi-agent systems (MAS) have demonstrated exceptional capabilities in solving complex tasks, yet their effectiveness depends heavily on the underlying communication topology that coordinates agent interactions. Within these systems, successful problem-solving often necessitates task-specific group structures to divide and conquer subtasks. However, most existing approaches generate communication topologies in a node-centric manner, leaving group structures to emerge implicitly from local connectivity decisions rather than modeling them explicitly, often leading to suboptimal coordination and unnecessary communication overhead. To address this limitation, we propose GoAgent (Group-of-Agents), a communication topology generation method that explicitly treats collaborative groups as the atomic units of MAS construction. Specifically, GoAgent first enumerates task-relevant candidate groups through an LLM and then autoregressively selects and connects these groups as atomic units to construct the final communication graph, jointly capturing intra-group cohesion and inter-group coordination. To mitigate communication redundancy and noise propagation inherent in expanding topologies, we further introduce a conditional information bottleneck (CIB) objective that compresses inter-group communication, preserving task-relevant signals while filtering out redundant historical noise. Extensive experiments on six benchmarks demonstrate the state-of-the-art performance of GoAgent with 93.84% average accuracy while reducing token consumption by about 17%.

Hongjiang Chen, Xin Zheng, Yixin Liu +7

School of Cyberspace, Hangzhou Dianzi University, Hangzhou, China · School of Computing Technologies, RMIT University, Melbourne, Australia · School of Information and Communication Technology, Griffith University, Goldcoast, Australia

Multi-Agent Communication Topology Multi-Agent Communication Multi-Agent LLMs

Ask PDF

Abstract

Multi-Agent Coding

Explore similar work

AgentConductor: Topology Evolution for Multi-Agent Competition-Level Code Generation

Retrieval-Conditioned Topology Selection with Provable Budget Conservation for Multi-Agent Code Generation

GoAgent: Group-of-Agents Communication Topology Generation for LLM-based Multi-Agent Systems