As large language models from diverse providers converge toward comparable benchmark performance, the traditional paradigm of selecting a single best model per task yields diminishing returns. We argue that orchestration topology -- the structural composition of how multiple agents are coordinated, parallelized, and synthesized -- now dominates system-level performance over individual model capability. We present AdaptOrch, a formal framework for task-adaptive multi-agent orchestration that dynamically selects among four canonical topologies (parallel, sequential, hierarchical, and hybrid) based on task dependency graphs and empirically derived domain characteristics. Our framework introduces three key contributions: (1) a Performance Convergence Scaling Law, formalizing conditions under which orchestration selection outweighs model selection; (2) a Topology Routing Algorithm that maps task decomposition DAGs to optimal orchestration patterns in O(|V| + |E|) time; and (3) an Adaptive Synthesis Protocol with provable termination guarantees and heuristic consistency scoring for parallel agent outputs. We validate AdaptOrch across coding (SWE-bench), reasoning (GPQA), and retrieval-augmented generation tasks, demonstrating that topology-aware orchestration achieves 12-23% improvement over static single-topology baselines, even when using identical underlying models. Our results establish orchestration design as a first-class optimization target independent of model scaling.
Figures & tables
Figure 1: Paradigm shift from model selection (left) to orchestration design (right). When model capabilities converge, the dominant optimization variable becomes the structural topology of agent coordination.
Figure 2: AdaptOrch pipeline. The Topology Router (Algorithm 1 ) selects the optimal execution topology based on DAG structural properties ( ω , δ , γ ). Failed syntheses trigger re-routing with adjusted coupling estimates.
Model
MMLU
HumanEval
ARC-C
MATH
GPT-4o-mini
82.0
87.2
93.1
70.2
Claude 3.5 Haiku
83.1
88.7
92.4
69.5
Gemini 2.0 Flash
81.4
86.9
91.8
71.8
Llama 3.3 70B
82.6
85.3
93.7
68.1
Qwen 2.5 72B
83.8
87.8
94.2
72.4
ϵ (max gap)
0.024
0.034
0.024
0.043
Table 1: ϵ -Convergence evidence. All models fall within ϵ of the best model on each benchmark, validating Definition 1 .
Figure 3: ϵ -Convergence evidence across four benchmarks. All five models score within ϵ of the best, validating the convergence assumption (Definition 1 ). Dashed line: best model score; shaded band: ϵ range.
Method
SWE-bench Verified
GPQA Diamond
HotpotQA
Acc
Lat.
Tok(K)
Acc
Lat.
Tok(K)
F1
Lat.
Tok(K)
Single Best
42.8
1.0 ×
12.3
46.2
1.0 ×
4.1
68.3
1.0 ×
6.8
MoA-3L
48.1
3.2 ×
84.6
49.8
2.8 ×
31.2
71.6
2.5 ×
47.3
Static-Parallel
47.3
1.4 ×
52.1
44.1
1.3 ×
18.7
72.8
1.2 ×
28.4
Static-Sequential
45.6
2.8 ×
48.9
50.3
2.4 ×
16.4
69.1
2.1 ×
26.1
LLM-Blender
44.9
1.8 ×
61.7
47.7
1.6 ×
22.3
70.4
1.5 ×
34.8
Table 2: Main results across three benchmarks. AdaptOrch selects topology per-task. Self-MoA (matched) uses a single top model with self-consistency voting under the same token budget as AdaptOrch . Best results in bold , second-best underlined . Δ shows improvement over Single Best baseline. Tok(K) = average tokens consumed per instance in thousands.
Figure 4: Main results comparison across three benchmarks. AdaptOrch achieves the highest accuracy on all tasks while maintaining competitive latency. Error bars show ±1 standard deviation over 3 runs.
Figure 5: Pareto front: accuracy vs. latency. AdaptOrch achieves the best accuracy-latency tradeoff across benchmarks, dominating other methods in the Pareto sense.
Figure 6: Token efficiency analysis. (Left) Total token consumption per instance. (Center) Accuracy per 1M tokens. (Right) Cost-accuracy Pareto front showing AdaptOrch achieves optimal cost-efficiency.
Domain
τP (Parallel)
τS (Sequential)
τH (Hierarchical)
τX (Hybrid)
SWE-bench
24
0
14
62
GPQA
8
41
35
16
HotpotQA
18
3
8
71
Average
16.7
14.7
19.0
49.7
Table 3: Topology routing distribution (%) by benchmark domain. The router adapts topology selection to domain characteristics.
Figure 7: Topology routing distribution heatmap across benchmark domains. Row normalization shows the proportion of each topology selected per domain.
Configuration
Accuracy
Δ
AdaptOrch (full)
52.6
—
− Adaptive routing (fixed τX )
49.8
−2.8
− Synthesis protocol (naive concat)
47.1
−5.5
− DAG coupling (uniform c=0.5 )
50.3
−2.3
− Re-routing on failure
51.0
−1.6
− Task decomposition (1 subtask)
42.8
−9.8
Table 4: Ablation study on SWE-bench Verified (500 instances).
Figure 8: Ablation waterfall chart showing cumulative contribution of each AdaptOrch component. Bars show accuracy drop when each component is removed independently.
Figure 9: Sensitivity of task accuracy to coupling threshold θγ across SWE-bench and GPQA. Shaded regions show 95% bootstrap confidence intervals ( n=30 trials per setting). Optimal range: [0.55,0.65] .
Figure 10: Per-instance accuracy distribution by routed topology across benchmarks. Violin plots show density; white dots indicate median. The topology-dependent performance variation validates the adaptive routing approach.
Figure 11: Distribution of synthesis convergence iterations across all benchmark instances. 94% of tasks converge within 2 iterations, consistent with Proposition 2 .
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12: PCA projection of DAG feature space (width ω , depth, density, coupling ratio) colored by KMeans clusters ( k=4 ). The four clusters correspond to dominant topology patterns: Chain (sequential), Wide-Shallow (parallel), Deep-Narrow (hierarchical), and Diamond (fan-out/fan-in). Cluster centroids are marked with × .
DAG Cluster
Dominant τ
n
Single Best
Δ AdaptOrch
Chain (sequential)
τS
187
54.2%
+3.8 pp
Wide-Shallow (parallel)
τP
294
49.1%
+12.6 pp
Deep-Narrow (hierarchical)
τH
112
47.8%
+9.2 pp
Diamond (fan-out/fan-in)
τX
105
51.3%
+11.4 pp
Appendix
Table 5: Per-cluster accuracy gain ( Δ ) of AdaptOrch over Single Best, averaged across all three benchmarks. n : number of instances assigned to each cluster.
Table 6: Router confusion matrix: predicted topology vs. oracle-optimal topology. Values are instance counts. Overall router accuracy: 81.2% (567/698).
Agentic orchestration delegates tasks to sub-agents with independent contexts, providing a key path to extending the effective execution horizon of large language model agents. However, existing orchestration methods face two limitations: in execution, current methods fix sub-agents' goals and harnesses at creation time, limiting their adaptability to dynamic real-world environments; in optimization, sparse and delayed feedback constrains effective learning of the orchestration policy. To address these limitations, we introduce POrchestra, an on-the-fly proactive agentic orchestration framework. It allows sub-agents to proactively communicate with the orchestrator at any logical point to request harness adjustments while maintaining continuity of local execution. This mechanism not only enables sub-agents to adapt seamlessly to dynamic environments but also naturally inspires On-Policy Orchestration Distillation (OPOD) based on hindsight feedback. OPOD attributes sub-agents' proactive revision requests back to preceding orchestration decisions to derive targeted improvement signals, and leverages these signals as process supervision alongside the final reward to optimize the orchestration policy. We evaluate POrchestra on three long-horizon agent benchmarks spanning deep research, dynamic digital assistance, and software engineering. POrchestra improves task success by an average of 9.1% relative to the strongest baseline; combined with OPOD, it improves small-orchestrator success by an average of 10.0% relative to existing training baselines. POrchestra also achieves a more favorable cost-performance trade-off. Our code is publicly available at https://github.com/XueyangFeng/POrchestra.
Xueyang Feng, Chaozhuo Li, Xiaohe Bo +4
Gaoling School of Artificial Intelligence, Renmin University of China · Beijing Academy of Artificial Intelligence · The Chinese University of Hong Kong Shenzhen
Large language model (LLM) multi-agent systems typically rely on rigid orchestration, committing either to flat per-query routing or to hand-engineered task decomposition, so decomposition depth, worker choice, and inference budget are not jointly optimized under one objective. We introduce Uno-Orchestra, a unified orchestration policy that selectively decomposes a task and dispatches each subtask to an admissible (model, primitive) pair, with both decisions learned together from curated RL trajectories grounded in real worker interactions. Against 22 baselines on a 13-benchmark suite spanning math, code, knowledge, long-context, and agentic tool-use, Uno-Orchestra reaches 77.0% macro pass@1, roughly 16% above the strongest workflow baseline, at roughly an order of magnitude lower per-query cost, advancing the accuracy-efficiency frontier of selective delegation.
Multi-Agent Systems (MAS) built on Large Language Models (LLMs) require effective orchestration to coordinate specialized agents, yet training such orchestrators is hindered by limited supervision and high computational cost. We propose Orchestration Reward Modeling (OrchRM), a self-supervised framework for evaluating orchestration quality without human annotations. OrchRM leverages intermediate artifacts from multi-agent executions to construct win-lose pairs for Bradley-Terry reward model training. Unlike existing MAS test-time scaling and orchestrator training frameworks that rely on costly sub-agent rollouts, OrchRM operates directly at the orchestration level, enabling efficient and high-performing reward-guided orchestrator training and MAS test-time scaling. OrchRM improves training efficiency by up to 10x in token usage while improving MAS test-time scaling performance by up to 8% in accuracy. These gains consistently transfer across multiple domains, including mathematical reasoning, web-based question answering, and multi-hop reasoning, demonstrating orchestration-level reward modeling as a scalable direction for robust multi-agent orchestration. Code will be available at https://github.com/Wang-ML-Lab/OrchRM.
King Yeung Tsang, Zihao Zhao, Vishal Venkataramani +5