AgentOrch: How Close Can Global Orchestration Come to Step-by-Step Interaction?

Zijie Liu; Qi Yang Huang; Jinhao Duan; Yueh Che Lee; Thomas A. Joseph; Zayne Turner; Adam Seligman; Tianlong Chen

cs.AIOct 24, 2026

AgentOrch: How Close Can Global Orchestration Come to Step-by-Step Interaction?

Authors: Zijie Liu, Qi Yang Huang, Jinhao Duan, Yueh Che Lee, Thomas A. Joseph, Zayne Turner, Adam Seligman, Tianlong Chen,

Organizations: University of North Carolina at Chapel Hill · Workato

Abstract

Enterprise workflow agents typically rely on step-by-step interaction (Step-by-Step): the agent executes an action, observes its result, and invokes a large language model (LLM) again to decide the next step. Global orchestration (Global) instead generates a complete executable workflow in advance and executes it without further LLM calls, substantially reducing inference overhead, but it remains less accurate than Step-by-Step. We study how close Global can come to Step-by-Step and what narrows the remaining gap. We introduce AgentOrch, a benchmark that directly compares global orchestration with step-by-step interaction on 650 enterprise SOPs and 3,220 runtime variants under matched tools, environments, and evaluation. Step-by-Step remains more accurate for every evaluated model, but the gap has shrunk by about 79% from early models to the strongest recent model. We identify three routes to further narrowing this gap. Incremental workflow construction improves Global accuracy by an average of 9.98 percentage points (pp) without runtime execution feedback. Revising complete workflows with execution feedback improves all evaluated models, including 6.13 pp on held-out runtime variants. Targeted training on complete workflows improves accuracy by an average of 12.18 pp across all five trained models. Although a gap remains, global orchestration offers a promising path toward enterprise agents that deliver higher accuracy with fewer LLM calls.

Figures & tables

Figure 1: Global orchestration reduces inference overhead but remains less accurate than step-by-step interaction. Left: Step-by-Step uses on average 23.2 $\times$ more LLM calls per SOP and 3.64 $\times$ more completion tokens per runtime variant than Global . Right: Step-by-Step achieves higher Orchestration Accuracy across all evaluated models, while the gap narrows to approximately 8 percentage points for Claude Opus 5T.

Figure 2: Overview of the AgentOrch benchmark construction and model evaluation pipeline. Left: Reference recipes define the benchmark SOPs. We extract their tool operations, map them to a fixed tool bank, construct task specifications and runtime variants, and validate the resulting workflow logic and fields. Reference executions provide the behavior used for evaluation. Right: Under Global , the model generates one complete workflow before execution and reuses it across runtime variants. Under Step-by-Step , the model selects tool calls sequentially using intermediate execution results. Both protocols are executed in the same simulated environment and evaluated against the corresponding reference behavior.

Benchmark	Domain	Interface	Execution		Global Orchestration		Outcome Evaluation
			E2E Exec.	Step-by-Step	Global	Reference Workflow	State Grading	Side-Effect Check
WebArena ( Zhou et al., 2024 )	Web / GUI	Browser	✓	✓	✗	✗	✓	$\triangle$
WorkArena++ ( Boisvert et al., 2024 )	Enterprise / GUI	Browser / GUI	✓	✓	✗	✗	✓	$\triangle$
OSWorld ( Xie et al., 2024 )	OS / GUI	Desktop GUI	✓	✓	✗	✗	$\triangle$	$\triangle$
AppWorld ( Trivedi et al., 2024 )	API / Tool	APIs	✓	✓	✗	✗	✓	✓
$\tau$ -bench ( Yao et al., 2024 )	API / Tool	Tools + User	✓	✓	✗	✗	✓	✓

Table 1: Comparison of evaluation settings across agent and workflow benchmarks. E2E Exec. indicates whether generated actions or workflows are executed end-to-end in an environment. Step-by-Step indicates whether the model can make new orchestration decisions using intermediate execution observations. Global indicates whether the model can generate a complete workflow before execution. Reference Workflow indicates whether executable reference workflow structures are available for evaluation. State Grading evaluates the resulting environment state. Side-Effect Check evaluates external effects produced during execution. ✓ = yes,

\triangle

= partial, and ✗ = no.

Figure 3: Scaling does not consistently narrow the GLOBAL–STEP-BY-STEP gap. Across Qwen2.5 and Gemma-3, the differences in Orchestration Accuracy, Final Accuracy, and Error Rate do not decrease monotonically with model size.

Model	Global	Input-Cond. Global	Step-by-Step	$\bm{\Delta}$
Qwen3-4B	13.75	13.04	41.84	-0.71
Gemma-4-12B	14.26	12.73	49.83	-1.53
Gemma-4-26B	4.05	3.66	52.10	-0.39
Claude Opus 5	46.44	48.79	63.52	+2.35
DeepSeek-V4	27.01	25.62	58.99	-1.39
Qwen3.8-27B	28.57	29.19	53.31	+0.62

Table 2: Conditioning global orchestration on concrete runtime inputs does not narrow the gap. Orchestration Accuracy (%);

\Delta

is Input-Conditioned Global minus Global in percentage points.

		Qwen			Gemma			Llama			GPT			GLM
Metric	Setting	Qwen2.5 3B	Qwen3 4B	$\Delta$	Gemma-3 12B	Gemma-4 12B	$\Delta$	Llama-3.1 70B	Llama-3.3 70B	$\Delta$	GPT-3.5	GPT-4o	$\Delta$	GLM-4.5 FP8	GLM-5.2 FP8	$\Delta$
Orch. $\uparrow$	Global	2.19	13.75	+11.56	8.46	14.26	+5.80	10.81	10.45	-0.36	10.11	28.09	+17.98	12.93	19.23	+6.30
	Step-by-Step	34.16	41.84	+7.68	37.85	49.83	+11.98	38.92	42.98	+4.06	50.78	50.12	-0.66	54.47	62.16	+7.69
	Gap	31.97	28.09	-3.88	29.39	35.57	+6.18	28.11	32.53	+4.42	40.67	22.03	-18.64	41.54	42.93	+1.39
Final $\uparrow$	Global	0.59	3.91	+3.32	1.83	4.07	+2.24	3.35	4.29	+0.94	4.32	10.34	+6.02	5.19	8.57	+3.38
	Step-by-Step	9.50	17.30	+7.80	13.07	27.76	+14.69	14.41	20.31	+5.90	30.06	29.13	-0.93	38.29	48.20	+9.91

Table 3: Newer model generations generally improve global orchestration, but do not consistently narrow the gap to step-by-step interaction. Scores are reported in percent; Gap and

\Delta

are reported in percentage points (pp). For accuracy metrics, Gap is Step-by-Step minus Global ; for Error Rate, Gap is Global minus Step-by-Step .

\Delta

denotes newer generation minus earlier generation.

	Qwen3-4B		Qwen3-30B		Qwen3.8-27B		DeepSeek-V4		GLM-4.5-Air		GLM-4.5-FP8		GLM-5.2-FP8		Claude-Opus-5
Metric	Base	Think	Base	Think	Base	Think	Base	Think	Base	Think	Base	Think	Base	Think	Base	Think
Orch.	13.75%	10.25%	8.35%	14.75%	28.57%	33.02%	27.01%	17.14%	12.28%	10.72%	12.93%	20.09%	19.23%	17.75%	46.44%	59.88%
Final	3.91%	2.70%	2.95%	5.19%	12.67%	17.05%	11.43%	9.66%	4.78%	4.25%	5.19%	8.48%	8.57%	9.29%	23.54%	32.33%
Side	16.40%	13.04%	9.44%	16.83%	31.80%	35.71%	29.94%	17.52%	14.22%	11.61%	13.88%	21.46%	21.15%	18.66%	50.16%	63.98%
Trace	11.11%	7.45%	7.26%	12.68%	25.33%	30.33%	24.07%	16.76%	10.34%	9.83%	11.99%	18.72%	17.31%	16.84%	42.72%	55.78%
Term.	16.30%	8.42%	13.73%	20.12%	28.98%	35.16%	28.98%	20.06%	11.71%	13.45%	10.81%	19.88%	18.94%	14.32%	42.24%	48.29%

Table 4: Reasoning does not consistently improve global orchestration. We report global-mode performance for base and thinking variants across models. For each base–thinking pair, the better-performing score is highlighted and bolded (higher is better for accuracy metrics; lower is better for error rate).

Model	Global	Local	$\bm{\Delta}$
Llama-3.3-70B	10.4	34.9	+24.5
Qwen2.5-3B	2.2	22.8	+20.6
Mistral-Small-3.1-24B	14.1	27.3	+13.1
Qwen3.8-27B-Thinking	33.0	41.5	+8.5
Qwen3-30B-A3B	8.4	8.5	+0.1
Qwen3-4B	13.8	6.9	-6.9

Table 5: Incremental plan construction can improve global orchestration without execution feedback. Orchestration Accuracy (%);

\Delta

is Local minus Global .

	Replan			Validation-only
Model	Repair	Valid	Share	$\bm{\Delta}$	Recovery	R–S
Mistral-3.1-24B	+7.09	+0.52	93.2%	+6.10	80.2%	+1.51
Qwen3-30B-A3B	+2.75	+0.66	80.6%	+1.70	49.9%	+1.71
Qwen3-4B	+4.88	0.00	100.0%	+4.20	86.1%	+0.68
Across models	+4.91	+0.39	92.6%	+4.00	75.5%	+1.30

Table 6: Decomposition of Replan gains. Repair and Valid: gains from initially invalid and initially valid workflows. Validation-only: revision using pre-execution validation errors without execution. Recovery

=\Delta/(\mathrm{Repair}+\mathrm{Valid})

Figure 4: Replan consistently improves global orchestration across models. Performance over Rounds 0–3 for eight models when execution feedback is collected from all runtime variants of each SOP. Accuracy metrics generally increase across revision rounds, while Error Rate decreases.

Model	Orch. $\uparrow$		Final $\uparrow$		Side $\uparrow$		Trace $\uparrow$		Term. $\uparrow$		Mean Acc. $\uparrow$			Error $\downarrow$
	Base	Final	Base	Final	Base	Final	Base	Final	Base	Final	Base	Final	$\bm{\Delta}$	Base	Final	$\bm{\Delta}$
Qwen3.8-27B	29.11	40.07	14.37	22.22	33.02	44.34	25.19	35.79	28.73	38.81	26.08	36.25	+10.16	45.22	31.58	-13.64
DeepSeek-V4-Flash	27.21	35.46	12.35	16.69	30.90	40.62	23.51	30.29	28.48	33.18	24.49	31.25	+6.76	44.86	31.27	-13.59
Mistral-Small-3.1-24B	14.52	21.89	6.93	9.82	18.29	26.72	10.74	17.06	16.38	24.55	13.37	20.01	+6.64	67.60	51.21	-16.39
Gemma-4-26B-A4B	3.82	9.62	1.91	4.29	4.03	10.65	3.61	8.59	4.96	9.92	3.67	8.61	+4.95	90.65	79.59	-11.06
Gemma-4-12B	14.47	20.24	4.34	6.67	16.90	23.98	12.04	16.51	12.87	17.57	12.12	16.99	+4.87	66.93	54.68	-12.25

Table 7: Replan improvements generalize to unseen runtime variants. We report performance before Replan ( Base , Round 0) and after three Replan rounds ( Final , Round 3). At each round, the model receives execution feedback from only the two random runtime variants of each SOP, while evaluation is performed on the remaining held-out variants that are never used for feedback. For Mean Accuracy and Error Rate, we additionally report the percentage-point change (

\Delta

). Mean Accuracy averages Orchestration, Final, Side-effect, Execution-trace, and Terminal-output Accuracy. Higher accuracy and lower Error Rate are better. No model weights are updated.

	Orch. Acc. (%) $\uparrow$				Final Acc. (%) $\uparrow$				Trace Acc. (%) $\uparrow$				Error Rate (%) $\downarrow$
Model	Base	SFT	RL	$\bm{\Delta}$	Base	SFT	RL	$\bm{\Delta}$	Base	SFT	RL	$\bm{\Delta}$	Base	SFT	RL	$\bm{\Delta}$
Qwen3.8-27B	28.57	41.96	41.14	+13.39	12.67	18.48	18.26	+5.81	25.33	35.87	35.24	+10.54	43.88	17.08	19.38	$-26.80$
Qwen3-4B	13.75	25.46	27.16	+13.41	3.91	10.78	10.71	+6.87	11.11	19.80	21.43	+10.32	61.96	40.99	36.12	$-25.84$
Gemma-4-26B-A4B	4.05	19.89	23.93	+19.88	1.80	8.45	9.35	+7.55	3.99	15.40	18.66	+14.67	89.22	56.61	48.23	$-40.99$
Gemma-4-31B	12.71	19.55	19.10	+6.84	4.78	9.78	9.22	+5.00	12.07	16.03	15.93	+3.96	74.94	57.17	56.55	$-18.39$
Gemma-4-12B	14.26	17.92	21.66	+7.40	4.07	8.51	8.54	+4.47	12.44	14.29	17.33	+4.89	65.78	63.94	52.76	$-13.02$

Table 8: Targeted training improves global orchestration. Results under Global ;

\Delta

compares the better of SFT and RL with Base (pp).

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	SFT	Qwen RL	Gemma RL
Method	LoRA ( $r$ =16, $\alpha$ =32)	GRPO	GRPO + LoRA ( $r$ =16, $\alpha$ =32)
Learning rate	$1\times 10^{-4}$	$3\times 10^{-6}$ (27B)	$1\times 10^{-5}$
Batch / rollouts	16 seq.	16 / 8 rollouts	8 prompts $\times$ 8 gen.
Max. length	81,920	65,536 / 6,144	4,096 completion
KL coefficient	–	0.001	0.001

Appendix

Table 9: Training hyperparameters.

		Orchestration Accuracy			Final Accuracy			Error Rate
Family	Scale	Global	Step-by-Step	$\Delta$	Global	Step-by-Step	$\Delta$	Global	Step-by-Step	$\Delta$
Qwen2.5	0.5B	0.13%	36.46%	+36.33	0.00%	12.45%	+12.45	99.78%	0.22%	$-99.56$
	1.5B	1.66%	31.41%	+29.75	0.22%	8.91%	+8.69	96.43%	0.25%	$-96.18$
	7B	2.63%	30.50%	+27.87	0.47%	7.42%	+6.95	93.45%	1.09%	$-92.36$
	14B	5.33%	41.54%	+36.21	1.61%	21.46%	+19.85	85.40%	0.03%	$-85.37$
	32B	16.18%	45.64%	+29.46	6.40%	24.41%	+18.01	58.14%	0.34%	$-57.80$

Appendix

Table 10: Scaling does not consistently narrow the Global – Step-by-Step gap. We report performance under the Global and Step-by-Step (react) settings.

\Delta

denotes Step-by-Step minus Global in percentage points (pp).

	Qwen3-4B		Qwen3-30B		Qwen3.8-27B		DeepSeek-V4		GLM-4.5-Air		GLM-4.5-FP8		GLM-5.2-FP8		Claude-Opus-5
Metric	Base	Think	Base	Think	Base	Think	Base	Think	Base	Think	Base	Think	Base	Think	Base	Think
Orch.	13.75%	10.25%	8.35%	14.75%	28.57%	33.02%	27.01%	17.14%	12.28%	10.72%	12.93%	20.09%	19.23%	17.75%	46.44%	59.88%
Final	3.91%	2.70%	2.95%	5.19%	12.67%	17.05%	11.43%	9.66%	4.78%	4.25%	5.19%	8.48%	8.57%	9.29%	23.54%	32.33%
Side	16.40%	13.04%	9.44%	16.83%	31.80%	35.71%	29.94%	17.52%	14.22%	11.61%	13.88%	21.46%	21.15%	18.66%	50.16%	63.98%
Trace	11.11%	7.45%	7.26%	12.68%	25.33%	30.33%	24.07%	16.76%	10.34%	9.83%	11.99%	18.72%	17.31%	16.84%	42.72%	55.78%
Term.	16.30%	8.42%	13.73%	20.12%	28.98%	35.16%	28.98%	20.06%	11.71%	13.45%	10.81%	19.88%	18.94%	14.32%	42.24%	48.29%

Appendix

Table 11: Reasoning does not consistently improve orchestration performance. We compare base and thinking variants under both Global and Step-by-Step (react) execution. Within each base–thinking pair and execution setting, the better-performing score is highlighted and bolded (higher is better for accuracy metrics; lower is better for error rate). Mean accuracy is the average of orchestration, final, side-effect, execution-trace, and terminal-output accuracy.

Model	Two-Var. $\Delta$ Mean Acc.	Gap vs. All-Var.	Two-Var. Error Red.	Gap vs. All-Var.
Qwen3.8-27B	+10.42	+0.21	14.00	-0.53
DeepSeek-V4-Flash	+6.88	-0.02	13.32	-0.07
Mistral-Small-3.1-24B	+7.02	+0.16	17.08	+0.28
Gemma-4-26B-A4B	+5.12	+0.27	11.55	-0.62
Gemma-4-12B	+4.70	+0.50	12.52	+1.31
Qwen3-4B	+4.13	-0.02	13.14	-0.34

Appendix

Table 12: Effect of limiting Replan feedback to two runtime variants per SOP. All columns evaluate on all runtime variants. Gaps compare two-variant feedback with all-variant feedback. Error reduction is

\text{Error}_{R0}-\text{Error}_{R3}

. All values are in pp.

Model	Seen $\Delta$ Mean Acc.	Held-out–Seen Gap	Seen Error Red.	Held-out–Seen Gap
Qwen3.8-27B	+10.81	-0.65	14.55	-0.91
DeepSeek-V4-Flash	+7.07	-0.31	12.92	+0.67
Mistral-Small-3.1-24B	+7.59	-0.96	18.13	-1.74
Gemma-4-26B-A4B	+5.38	-0.43	12.29	-1.23
Gemma-4-12B	+4.45	+0.42	12.92	-0.67
Qwen3-4B	+3.63	+0.83	13.16	-0.03

Appendix

Table 13: Transfer from feedback variants to held-out variants. Replan feedback uses two variants of each SOP. The transfer gap is the held-out value minus the seen value; values near zero indicate similar improvement on both subsets.

Model	Mean Accuracy $\uparrow$			Error Rate $\downarrow$
	Base	Final	$\bm{\Delta}$	Base	Final	$\bm{\Delta}$
Qwen3.8-27B	26.27	36.13	+9.86	42.96	28.43	-14.53
DeepSeek-V4-Flash	24.72	31.05	+6.32	42.87	29.69	-13.18
Mistral-Small-3.1-24B	13.31	20.40	+7.09	67.35	48.79	-18.56
Gemma-4-26B-A4B	4.12	9.20	+5.09	88.88	77.31	-11.57
Gemma-4-12B	12.17	16.89	+4.72	65.65	53.09	-12.56

Appendix

Table 14: Replan performance on a subset of two held-out variants per SOP. Feedback uses two variants of each SOP; evaluation uses two other variants never used for feedback. Base is Round 0 and Final is Round 3.

Model	Orch. Up	Orch. Down	Final Up	Final Down
Qwen3.8-27B	428	31	161	9
DeepSeek-V4-Flash	335	51	110	26
Mistral-Small-3.1-24B	357	31	68	12
Gemma-4-26B-A4B	240	12	56	10
Gemma-4-12B	264	6	49	4
Qwen3-4B	249	12	41	6

Appendix

Table 15: Variant-level improvements and regressions on held-out variants. Counts of held-out variants whose score increases (Up) or decreases (Down) from Round 0 to Round 3 under two-variant feedback. Unchanged variants are omitted.

Model	Repaired	Parse / Shape	Validator	Validator (%)
Qwen3.8-27B	97	16	81	83.5
DeepSeek-V4-Flash	84	12	72	85.7
Mistral-Small-3.1-24B	117	13	104	88.9
Gemma-4-12B	79	9	70	88.6
Qwen3-4B	85	13	72	84.7
Qwen3-30B-A3B	40	5	35	87.5

Appendix

Table 16: Failure types of invalid workflows repaired by Replan . Repaired workflows are invalid at Round 0 and valid by Round 3. Parse/Shape: failures in response parsing or plan-structure checks. Validator: workflows that parse successfully but fail schema, type, or reference validation. Together, these checks detect all 641 repaired workflows before execution.

Metric	Reference-Action Replay
Tool Selection	97.20%
Tool Order	97.20%
Tool Execution	97.20%
Required Arguments	99.97%
Control Path	97.20%
Execution Trace Accuracy	97.75%

Appendix

Table 17: Simulator fidelity under reference-action replay. Recorded tool calls from the real-backend reference executions are replayed one action at a time in the simulated backend and evaluated using the same benchmark evaluator as model-generated runs.

Subset	# SOP	# Var.	GLOBAL	STEP-BY-STEP	Gap	95% CI	GLOBAL Err.	STEP Err.
All workflows
All	650	3220	13.8	46.4	+32.6	[31.2, 34.2]	70.7	0.5
Workflow size
Short (1–2 calls)	234	1035	18.8	45.2	+26.3	[24.5, 28.4]	60.9	0.5
Medium (3–5 calls)	251	1149	14.2	47.3	+33.1	[30.9, 35.1]	69.3	0.5
Long ( $\geq$ 6 calls)	165	1036	8.3	46.6	+38.3	[35.1, 41.3]	82.0	0.7

Appendix

Table 18: Performance across workflow characteristics. Orchestration Accuracy and Error Rate are reported in percent. The gap is Step-by-Step minus Global Orchestration Accuracy in percentage points. Confidence intervals are 95% bootstrap intervals clustered by SOP. Workflow characteristics are derived from the reference workflow, except fault injection, which is defined per runtime variant.

	Total LLM Calls			Calls / Variant		Completion Tokens (M)			Tokens / Variant		Tokens / Call
Model	Global	Step	$\times$	Global	Step	Global	Step	$\times$	Global	Step	Global	Step
Open-weight models
Gemma-3-12B	649	12,098	18.6 $\times$	0.20	3.76	0.54	1.92	3.55 $\times$	167.8	595.7	833	159
Gemma-3-27B	649	14,253	22.0 $\times$	0.20	4.43	0.95	2.49	2.62 $\times$	295.8	774.3	1,468	175
Gemma-4-12B	649	13,888	21.4 $\times$	0.20	4.31	1.18	3.29	2.79 $\times$	365.1	1,020.3	1,812	237
Gemma-4-26B-A4B	649	17,198	26.5 $\times$	0.20	5.34	0.95	4.10	4.30 $\times$	296.2	1,272.9	1,470	238

Appendix

Table 19: Inference usage under global and step-by-step orchestration. The

\times

columns report Step-by-Step / Global . For closed-source models, completion-token counts are estimated from retained output text using median characters-per-token ratios measured from the corresponding non-thinking runs; all other quantities are directly measured.

Model	GLOBAL (%)	STEP-BY-STEP / ReAct (%)
GPT-3.5 Turbo	10.11	50.78
GPT-4o	28.09	50.12
Llama 3.1 8B	4.45	34.82
Llama 3.1 70B	10.81	38.92
Mistral Nemo	2.26	34.27
Command R	6.64	35.97

Appendix

Table 20: Orchestration accuracy under GLOBAL orchestration and STEP-BY-STEP (ReAct) interaction. Results are reported as percentages.

LLM Agent Orchestration

Large language model agent orchestration coordinates models, tools, and workflows to complete complex tasks reliably and efficiently.

See all 463 papers

May 21, 2026

Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost

Agent orchestration frameworks have proliferated, collectively exceeding 290,000 GitHub stars across LangGraph, CrewAI, Google ADK, OpenAI Agents SDK, Semantic Kernel, Strands, and LlamaIndex. All follow the same pattern: an external orchestrator above the LLM, injecting instructions and routing decisions every turn. Recent work has shown this architecture is dominated for procedural tasks by simply providing the procedure in a frontier model's system prompt [Dennis et al., 2026a], at the cost of consuming the context window, requiring a frontier model for every conversation, and exposing proprietary procedures to third-party providers. Compiling the procedure into the weights of a small fine-tuned model -- creating a subterranean agent -- should resolve all of these concerns, and prior work (SimpleTOD, FireAct, SynTOD, WorkflowLLM, Agent Lumos) has shown the technique works. Yet developer adoption has overwhelmingly favored orchestration. We identify three perceived barriers and address each empirically across travel booking (14 nodes), Zoom support (14 nodes, product-specific knowledge), and insurance claims (55 nodes, 6 decision hubs).

Simon Dennis, Rivaan Patil, Kevin Shabahang +1

i14, University of Melbourne · i14

LLM Agent Orchestration Agentic Workflows LLM Agent Workflow Optimization LLM Agent Training

Ask PDF

Apr 30, 2026

In-Context Prompting Obsoletes Agent Orchestration for Procedural Tasks

Agent orchestration frameworks -- LangGraph, CrewAI, Google ADK, OpenAI Agents SDK, and others -- place an external orchestrator above the LLM, tracking state and injecting routing instructions at every turn. We present a controlled comparison showing that for procedural tasks, this architecture is dominated by a simpler alternative: putting the entire procedure in the system prompt and letting the model self-orchestrate. Across three domains -- travel booking (14 nodes), Zoom technical support (14 nodes), and insurance claims processing (55 nodes) -- we evaluate 200 conversations per condition using LLM-as-judge scoring on five quality criteria. The in-context approach scores 4.53--5.00 on a 5-point scale while a LangGraph orchestrator using the same model scores 4.17--4.84. The orchestrated system fails on 24% of travel, 9% of Zoom, and 17% of insurance conversations, compared to 11.5%, 0.5%, and 5% for the in-context baseline. While external orchestration may have been necessary for earlier models, advances in frontier model capabilities have made it unnecessary for multi-turn conversations following a defined procedure.

Simon Dennis, Michael Diamond, Rivaan Patil +2

i14, University of Melbourne · i14

LLM Prompting LLM Agent Orchestration Conversational Agents

Ask PDF

Mar 20, 2026

Utility-Guided Agent Orchestration for Efficient LLM Tool Use

Tool-using large language model (LLM) agents often face a fundamental tension between answer quality and execution cost. Fixed workflows are stable but inflexible, while free-form multi-step reasoning methods such as ReAct may improve task performance at the expense of excessive tool calls, longer trajectories, higher token consumption, and increased latency. In this paper, we study agent orchestration as an explicit decision problem rather than leaving it entirely to prompt-level behavior. We propose a utility-guided orchestration policy that selects among actions such as respond, retrieve, tool call, verify, and stop by balancing estimated gain, step cost, uncertainty, and redundancy. Our goal is not to claim universally best task performance, but to provide a controllable and analyzable policy framework for studying quality-cost trade-offs in tool-using LLM agents. Experiments across direct answering, threshold control, fixed workflows, ReAct, and several policy variants show that explicit orchestration signals substantially affect agent behavior. Additional analyses on cost definitions, workflow fairness, and redundancy control further demonstrate that lightweight utility design can provide a defensible and practical mechanism for agent control.

Boyan Liu, Gongming Zhao, Hongli Xu

University of Science and Technology of China, Hefei, Anhui, China · Suzhou Institute for Advanced Research, University of Science and Technology of China

LLM Agent Orchestration LLM Tool Use LLM Agent Workflow Optimization LLM Decision-Making

Ask PDF

Abstract

LLM Agent Orchestration

Explore similar work

Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost

In-Context Prompting Obsoletes Agent Orchestration for Procedural Tasks

Utility-Guided Agent Orchestration for Efficient LLM Tool Use