Organizations: Gaoling School of Artificial Intelligence, Renmin University of China · Beijing Academy of Artificial Intelligence · The Chinese University of Hong Kong Shenzhen
Agentic orchestration delegates tasks to sub-agents with independent contexts, providing a key path to extending the effective execution horizon of large language model agents. However, existing orchestration methods face two limitations: in execution, current methods fix sub-agents' goals and harnesses at creation time, limiting their adaptability to dynamic real-world environments; in optimization, sparse and delayed feedback constrains effective learning of the orchestration policy. To address these limitations, we introduce POrchestra, an on-the-fly proactive agentic orchestration framework. It allows sub-agents to proactively communicate with the orchestrator at any logical point to request harness adjustments while maintaining continuity of local execution. This mechanism not only enables sub-agents to adapt seamlessly to dynamic environments but also naturally inspires On-Policy Orchestration Distillation (OPOD) based on hindsight feedback. OPOD attributes sub-agents' proactive revision requests back to preceding orchestration decisions to derive targeted improvement signals, and leverages these signals as process supervision alongside the final reward to optimize the orchestration policy. We evaluate POrchestra on three long-horizon agent benchmarks spanning deep research, dynamic digital assistance, and software engineering. POrchestra improves task success by an average of 9.1% relative to the strongest baseline; combined with OPOD, it improves small-orchestrator success by an average of 10.0% relative to existing training baselines. POrchestra also achieves a more favorable cost-performance trade-off. Our code is publicly available at https://github.com/XueyangFeng/POrchestra.
Figures & tables
Figure 1: Comparison of orchestration paradigms. (a) Offline orchestration fixes harnesses and workflows. (b) Re-orchestration adapts harnesses upon completion or predefined triggers. (c) POrchestra integrates execution and harness adaptation via proactive sub-agent communication.
Figure 2: Overview of POrchestra and OPOD. (a) Sub-agent execution and harness adaptation form a coherent workflow. (b) Structured proactive communication guides harness revisions. (c) OPOD combines feedback-based self-distillation with terminal rewards to optimize the orchestrator.
Figure 3: Analysis of POrchestra . Task success by (a,b) task length and (c) dynamicity. (d) Intervention mechanisms under controlled intervention and end-to-end evaluation. Recovery is in percent, delay in execution steps (including idle), and cost in CA per case.
Figure 4: Cost–performance trade-offs on GAIA and GAIA2.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Hyperparameter analysis of λ on GAIA2.
Figure 6: Ablations on GAIA2: (a) harness revisions and (b) communication protocol components. Protocol ablation scores average two runs.
Benchmark
Method
Orchestration
Communication
Sub-agent
Total
AOrchestra
0.236
—
0.390
0.625
Orchestra-o1
0.201
—
0.690
0.891
[4pt/2pt] GAIA
POrchestra
0.151
0.193
0.213
0.557
AOrchestra
0.208
—
1.001
1.209
Orchestra-o1
0.244
—
0.875
1.120
[4pt/2pt] GAIA2
POrchestra
0.097
0.065
0.559
0.721
Appendix
Table 3: Cost breakdown on GAIA and GAIA2 (CA per task).
Objective
Execution
Search
Adaptability
Time
Overall
Forward KL
50.0
71.9
28.1
21.9
43.0
JSD
40.6
75.0
37.5
28.1
45.3
Reverse KL
43.8
84.4
37.5
31.3
49.2
Appendix
Table 4: Distillation objective comparison on GAIA2. All values are success rates (%). Reverse KL uses the full OPOD results from Table .
open , scroll_up , scroll_down , goto , edit , str_replace , insert , create , delete_lines , search_file , search_dir , find_file , state , and bash .
14
Official patch evaluation.
Appendix
Table 5: Benchmark subsets, available tools, and evaluation procedures.
Benchmark
Framework
Task pool
Passes
Train decisions
GAIA
Orchestra-o1
2,339
2
11,746
AOrchestra
2,339
2
9,977
POrchestra
2,339
2
9,999
GAIA2
Orchestra-o1
512
4
5,276
AOrchestra
512
4
4,087
POrchestra
512
4
4,539
Appendix
Table 6: SFT data by benchmark and orchestration framework. Task pools and collection passes precede filtering; decision counts refer to the retained examples.
Parameter
SFT
OPOD
Parameter update
Full fine-tuning
LoRA, r=32 , α=64
LoRA dropout
—
0.05
Learning rate
10−5
10−5
Schedule / warmup ratio
Cosine / 0.05
Cosine / 0.05
Effective batch size
16
16
Per-device batch / accumulation
1 / 2
1 / 2
Appendix
Table 7: Main SFT and joint OPOD training settings.
As large language models from diverse providers converge toward comparable benchmark performance, the traditional paradigm of selecting a single best model per task yields diminishing returns. We argue that orchestration topology -- the structural composition of how multiple agents are coordinated, parallelized, and synthesized -- now dominates system-level performance over individual model capability. We present AdaptOrch, a formal framework for task-adaptive multi-agent orchestration that dynamically selects among four canonical topologies (parallel, sequential, hierarchical, and hybrid) based on task dependency graphs and empirically derived domain characteristics. Our framework introduces three key contributions: (1) a Performance Convergence Scaling Law, formalizing conditions under which orchestration selection outweighs model selection; (2) a Topology Routing Algorithm that maps task decomposition DAGs to optimal orchestration patterns in O(|V| + |E|) time; and (3) an Adaptive Synthesis Protocol with provable termination guarantees and heuristic consistency scoring for parallel agent outputs. We validate AdaptOrch across coding (SWE-bench), reasoning (GPQA), and retrieval-augmented generation tasks, demonstrating that topology-aware orchestration achieves 12-23% improvement over static single-topology baselines, even when using identical underlying models. Our results establish orchestration design as a first-class optimization target independent of model scaling.
Geunbin Yu
Department of Artificial Intelligence, Korea National Open University
Large language model (LLM) multi-agent systems typically rely on rigid orchestration, committing either to flat per-query routing or to hand-engineered task decomposition, so decomposition depth, worker choice, and inference budget are not jointly optimized under one objective. We introduce Uno-Orchestra, a unified orchestration policy that selectively decomposes a task and dispatches each subtask to an admissible (model, primitive) pair, with both decisions learned together from curated RL trajectories grounded in real worker interactions. Against 22 baselines on a 13-benchmark suite spanning math, code, knowledge, long-context, and agentic tool-use, Uno-Orchestra reaches 77.0% macro pass@1, roughly 16% above the strongest workflow baseline, at roughly an order of magnitude lower per-query cost, advancing the accuracy-efficiency frontier of selective delegation.
Large language models (LLMs) provide a flexible foundation for multi-agent systems, but their effectiveness and computational cost depend critically on orchestration design. Across different tasks, role design, capacity assignment, and dependency construction jointly affect both solution quality and execution efficiency. Existing approaches automate parts of this design process, yet they often optimize these decisions partially or sequentially, and rely on execution-level feedback that provides limited credit assignment for local orchestration decisions. We propose LEMON (Learning Executable Multi-Agent Orchestration via Counterfactual Reinforcement Learning), an LLM-based orchestrator that learns to design efficient multi-agent orchestration. Given a task, LEMON designs a unified orchestration specification that composes customized agent duties, capacity levels, and dependency relations. To train the orchestrator, we augment the orchestration-level Group Relative Policy Optimization (GRPO) objective with a localized counterfactual credit signal that edits role, capacity, or dependency fields and applies the resulting reward contrast only to the edited spans. Experiments on six reasoning and coding benchmarks, including MMLU, GSM8K, AQuA, MultiArith, SVAMP, and HumanEval, show that LEMON achieves the best average performance among evaluated multi-agent orchestration methods while improving the accuracy-token trade-off.