CB-Orchestrator: Adaptive Workflow Optimization for LLM Agents via Contextual Bandits

Jiahang Sun; Zhiwei Shang; Zhipiao Liu; Hongwei Yang; Guoqing Xie; Shuang Qiu; Zhongxiang Dai

cs.LGOct 9, 2026

CB-Orchestrator: Adaptive Workflow Optimization for LLM Agents via Contextual Bandits

Authors: Jiahang Sun, Zhiwei Shang, Zhipiao Liu, Hongwei Yang, Guoqing Xie, Shuang Qiu, Zhongxiang Dai

Organizations: The Chinese University of Hong Kong, Shenzhen · Huawei Technologies · City University of Hong Kong

Abstract

Large Language Model (LLM) agents have demonstrated remarkable capabilities in tackling complex tasks through agentic workflows. However, manually designing these workflows is labor-intensive and lacks flexibility to adapt to diverse queries. While automated workflow optimization is a promising direction, existing methods often incur expensive API costs during training-phase feedback collection. Furthermore, they fail to adaptively reuse high-quality workflows, which creates a bottleneck for both performance and cost-effectiveness. To address these limitations, we propose CB-Orchestrator, an adaptive framework that decouples workflow generation from selection. In the first stage, we construct a diverse pool of candidate workflows via evolutionary search. In the second stage, we formulate workflow selection as a Contextual Bandit problem, which enables sample-efficient learning by balancing exploration and exploitation, thereby significantly reducing the feedback required for training. During inference, the model adaptively selects the optimal workflow tailored to each query. Evaluations across five diverse benchmarks demonstrate that CB-Orchestrator consistently outperforms all baselines. Notably, on the MATH dataset, CB-Orchestrator achieves better performance than strong recent baselines while reducing training-phase API token overhead by no less than 40.55% and total end-to-end training (including workflow pool generation) token overhead by no less than 29.73%.

Figures & tables

Figure 1: CB-Orchestrator vs. prior workflow-optimization paradigms.

Method	MATH	GSM8K	MBPP	HumanEval	Avg .
Basic Prompting
Vanilla	46.29	87.45	71.83	87.08	73.16
CoT ( Wei et al., 2022 )	46.40	87.10	71.83	88.13	73.37
CoT SC (5-shot) ( Wang et al., 2023 )	47.91	87.57	73.60	88.60	74.42
Fixed Multi-Agent Systems
LLM-Debate ( Du et al., 2024 )	48.54	89.47	70.29	88.68	74.25

Table 1: The performance of CB-Orchestrator compared to baseline methods on math reasoning and code generation benchmarks. Backbone is GPT-4o-mini for all experiments. CB-Orchestrator consistently achieves the best performance across all benchmarks.

Figure 2: Cost-Efficiency comparison on the MATH benchmark.

Figure 3: Ablation results on MATH and MBPP. Black outlines indicate our proposed approaches.

Baseline Methods
Method	M $\rightarrow$ G	G $\rightarrow$ M
GPTSwarm	89.96	45.18
AFlow	91.95	49.39
MaAS	92.80	51.02
CB-Orchestrator (Ours)
Source Pool & Source NN	89.72	53.62

Table 2: Cross-domain transferability analysis. M and G denote MATH and GSM8K benchmarks, respectively. We evaluate the performance of CB-Orchestrator compared to baseline methods.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Domain	Train	Test	Workflows	Operators
MATH	Math reasoning	119	486	68	Custom, ScEnsemble, Programmer
GSM8K	Math reasoning	264	1055	72	Custom, ScEnsemble, Programmer
HumanEval	Code generation	33	131	31	Custom, CustomCodeGenerate, ScEnsemble, Test
MBPP	Code generation	86	341	81	Custom, CustomCodeGenerate, ScEnsemble, Test
GAIA	Multi-tool QA	93	301	36	Custom, Reasoning, Decompose, Search, FileAnalysis, Programmer, ScEnsemble, FinalAnswer

Appendix

Table 3: Dataset statistics

Stage 1: Workflow Pool Construction
Parameter	Sym.	Val.	Parameter	Sym.	Val.
Evolution rounds	$N_{\text{evolve}}$	2	Transfer rounds	$N_{\text{transfer}}$	2
Gen. temperature	-	1.0	Exec. temperature	-	0
Similar queries	$k$	5
Stage 2: Bandit Training
Embedding dim	$d$	2560	Learning rate	$\eta$	$10^{-5}$

Appendix

Table 4: Hyperparameters for CB-Orchestrator.

Dataset	Train Queries	Covered	Coverage	Pool Size
MATH	119	86	72.3%	68
GSM8K	100	98	98.0%	72
HumanEval	33	31	93.9%	31
MBPP	86	82	95.3%	81

Appendix

Table 5: Workflow pool coverage across datasets.

Method	Pool Gen.	Training	Inference	Total	Solve %
LLM-Debate	–	–	13.73	13.73	48.54
DyLAN	–	38.30	9.39	47.68	48.63
GPTSwarm	–	27.70	3.89	31.59	47.88
AFlow	–	62.88	4.66	67.54	51.28
MaAS	–	5.43	2.16	7.60	51.82
CB-Orch ( $T$ =200)	0.59	0.78	2.24	3.61	51.19

Appendix

Table 6: Detailed cost breakdown on MATH benchmark (all values in millions of tokens).

Method	Train Input	Train Output	Infer Input	Infer Output
LLM-Debate	–	–	3.28	10.46
DyLAN	22.15	16.15	6.08	3.30
GPTSwarm	21.33	6.37	3.11	0.79
AFlow	33.83	29.05	2.51	2.15
MaAS	3.05	2.38	1.31	0.85
CB-Orch ( $T$ =1200)	3.24	1.76	1.60	0.70

Appendix

Table 7: Input/output token breakdown on MATH (millions of tokens).

Figure 4: Cumulative average reward during bandit training on MATH. The curve shows rapid convergence at around Round 650, indicating that the bandit policy has learned effective workflow selection.

Operator	Description & Interface
Custom	General-purpose LLM call with customizable prompts. Flexible for planning, reflection, decomposition, formatting, etc.
	(input: str, instruction: str) -> {response: str}
ScEnsemble	Self-consistency voting across multiple solutions. Requires 3+ solutions for effective voting.
	(solutions: List[str], problem: str) -> {response: str}
Programmer	Generates and executes Python code in a sandbox. Auto-retries on errors. Returns computed answer.
	(problem: str, analysis: str) -> {code: str, output: str}

Appendix

Table 8: Operators for math reasoning benchmarks (MATH, GSM8K).

Operator	Description & Interface
Custom	General-purpose LLM call for reasoning about code structure and logic.
	(input: str, instruction: str) -> {response: str}
CustomCodeGenerate	Specialized code generation with entry point awareness.
	(problem: str, entry_point: str, instruction: str) -> {response: str}
ScEnsemble	Self-consistency voting for code solutions.
	(solutions: List[str], problem: str) -> {response: str}

Appendix

Table 9: Operators for code generation benchmarks (HumanEval, MBPP).

Operator	Description & Interface
Custom	General-purpose LLM call for flexible reasoning and formatting.
	(input: str, instruction: str) -> {response: str}
Reasoning	Multi-step structured reasoning. Can receive context from other operators.
	(question: str, context: str) -> {thought: str, answer: str}
Decompose	Breaks complex questions into sub-tasks with dependency tracking.
	(question: str, context: str) -> {analysis: str, subtasks: List, final_aggregation: str}

Appendix

Table 10: Operators for multi-tool QA benchmark (GAIA).

Operation	Time	Frequency
Matrix update: $A\leftarrow A+xx^{\top}$	$O(d^{2})$	Per sample
Cholesky decomposition: $A=LL^{\top}$	$O(d^{3})$	Per batch
Uncertainty: $u(x)=\sqrt{x^{\top}A^{-1}x}$	$O(d^{2})$	Per (query, workflow)
Neural network forward pass	$O(d\cdot h)$	Per (query, workflow)

Appendix

Table 11: Computational complexity of key operations.

Method	MATH	HumanEval	Avg.
Vanilla	31.93	80.06	56.00
MaAS	42.97	85.26	64.12
CB-Orchestrator	44.36	84.12	64.24

Appendix

Table 12: Scalability analysis using Llama-3.1-70B as the backbone LLM. We compare CB-Orchestrator against Vanilla and MaAS baselines.

Framework Variants
Method	MATH	MBPP	Avg.
PureRandom	47.99	81.64	64.81
WithoutEvo	51.89	84.17	68.03
Exploration Strategy Variants
RandomNN	52.59	82.34	67.47
GreedyNN	52.63	83.93	68.28

Appendix

Table 13: Ablation results on MATH and MBPP benchmarks. We compare different bandit selection strategies and evaluate the impact of the evolution process. The (w/ dup) variants allow duplicate (query, workflow) pair sampling during training.

Fixed Multi-Agent Systems
Method	L1	L2	L3	Avg.
Vanilla	7.5	4.4	0	4.7
TapeAgent	23.7	14.5	10.2	16.6
Sibyl	21.5	15.7	4.1	15.6
Autonomous Multi-agent Systems
AutoAgents	16.1	0	0	5.2

Appendix

Table 14: The performance of CB-Orchestrator compared to baseline methods on the GAIA benchmark. Backbone is GPT-4o-mini for all experiments.

Rank	Dist.	Shared Structure
1	0.32	Both ask “Bekah has [N] brass digits… how many distinct numbers” (identical template, different N)
2	0.44	Both ask circular seating of Senate committee with party constraints
3	0.45	Both ask “put 4 balls in 3 boxes” with distinguishability variants
4	0.46	Both ask area of isosceles right triangle given altitude to hypotenuse
5	0.47	Both show identical triangle diagrams (Asymptote code), ask for $x$

Appendix

Table 15: All top-5 closest query pairs. Every pair shares problem structure or reasoning pattern.

Rank	Dist.	Query A Domain	Query B Domain
1	1.42	Parametric line equations	Circular permutations
2	1.42	Triangle angle trisectors	Counting odd 5-digit numbers
3	1.41	Simplify $(\sqrt{6}+\sqrt{24})^{2}$	Probability of consecutive seats
4	1.41	3-digit palindrome multiples of 3	Simplify radical expression
5	1.40	Polygon perimeter geometry	Product divisibility probability

Appendix

Table 16: All top-5 farthest query pairs. Every pair involves completely different mathematical domains.

Type	Dist.	Workflow A	Workflow B
Close	0.13	Custom $\to$ Programmer(1 $\times$ ) $\to$ ScEnsemble $\to$ Format	Custom $\to$ Programmer(3 $\times$ ) $\to$ ScEnsemble $\to$ Format
Close	0.13	Custom $\to$ Programmer(3 $\times$ loop) $\to$ ScEnsemble $\to$ Format	Custom $\to$ Programmer(3 $\times$ ) $\to$ ScEnsemble $\to$ Format
Close	0.14	Custom $\to$ Programmer(3 $\times$ ) $\to$ ScEnsemble $\to$ Format	Custom $\to$ Custom(3 $\times$ ) $\to$ ScEnsemble $\to$ Format
Far	0.93	Custom $\to$ ScEnsemble $\to$ Programmer	Programmer only (hardcoded)
Far	0.93	Custom $\to$ Programmer(3 $\times$ ) $\to$ ScEnsemble $\to$ Programmer $\to$ Format	Custom $\to$ ScEnsemble $\to$ Programmer
Far	0.93	Custom $\to$ Programmer(3 $\times$ ) $\to$ ScEnsemble $\to$ Format	Custom $\to$ Custom $\to$ Custom $\to$ Programmer (no ensemble)

Appendix

Table 17: Workflow pairs at distance extremes. Close pairs share operator sequences; far pairs differ fundamentally.

Metric	Value	Interpretation
KNN Prediction Accuracy	84.75%	Correct prediction of success/failure
Random Baseline (majority)	51.08%	Baseline without smoothness
Pearson Correlation ( $r$ )	0.73	Strong positive correlation
$p$ -value	$<10^{-6}$	Statistically significant

Appendix

Table 18: KNN reward prediction results (

k=3

). High accuracy and correlation confirm reward smoothness.

Distance Type	Mean Distance	Sample Size
Success–Success (intra)	0.40	2,138 pairs
Success–Failure (cross)	0.50	1,066 pairs

Appendix

Table 19: Success workflow clustering analysis. Successful workflows cluster together.

LLM Agent Workflow Optimization

LLM agent workflow optimization studies how to design and run multi-step agents for better task success, reliability, latency, and cost.

See all 256 papers

WorkflowGen:an adaptive workflow generation mechanism driven by trajectory experience
Mar 22, 2026Ruocan Wei, Shufeng Wang, Ziwei ShiAgentic Workflows LLM Agent Workflow Optimization
China Telecom Cloud, Beijing, China
FlowBank: Query-Adaptive Agentic Workflows Optimization through Precompute-and-Reuse
Jun 9, 2026Lingzhi Yuan, Chenghao Deng, Fangxu Yu +3Multi-Agent LLM Systems Multi-Agent System Optimization
University of Maryland, College Park · Amazon
FlowBot: Inducing LLM Workflows with Bilevel Optimization and Textual Gradients
Apr 29, 2026Hongyeon Yu, Young-Bum Kim, Yoon KimAgentic Workflows LLM Agent Workflow Optimization
Naver Search US · Massachusetts Institute of Technology

Mar 22, 2026J/K move · Enter open · S save

WorkflowGen:an adaptive workflow generation mechanism driven by trajectory experience

Ruocan Wei, Shufeng Wang, Ziwei Shi

China Telecom Cloud, Beijing, China

Agentic Workflows LLM Agent Workflow Optimization Memory-Augmented Language Model Agents

Open paper AskPDF arXiv

Large language model (LLM) agents often suffer from high reasoning overhead, excessive token consumption, unstable execution, and inability to reuse past experiences in complex tasks like business queries, tool use, and workflow orchestration. Traditional methods generate workflows from scratch for every query, leading to high cost, slow response, and poor robustness. We propose WorkflowGen, an adaptive, trajectory experience-driven framework for automatic workflow generation that reduces token usage and improves efficiency and success rate. Early in execution, WorkflowGen captures full trajectories and extracts reusable knowledge at both node and workflow levels, including error fingerprints, optimal tool mappings, parameter schemas, execution paths, and exception-avoidance strategies. It then employs a closed-loop mechanism that performs lightweight generation only on variable nodes via trajectory rewriting, experience updating, and template induction. A three-tier adaptive routing strategy dynamically selects among direct reuse, rewriting-based generation, and full initialization based on semantic similarity to historical queries. Without large annotated datasets, we qualitatively compare WorkflowGen against real-time planning, static single trajectory, and basic in-context learning baselines. Our method reduces token consumption by over 40 percent compared to real-time planning, improves success rate by 20 percent on medium-similarity queries through proactive error avoidance and adaptive fallback, and enhances deployability via modular, traceable experiences and cross-scenario adaptability. WorkflowGen achieves a practical balance of efficiency, robustness, and interpretability, addressing key limitations of existing approaches.

Abstract

LLM Agent Workflow Optimization

Explore similar work

WorkflowGen:an adaptive workflow generation mechanism driven by trajectory experience

FlowBank: Query-Adaptive Agentic Workflows Optimization through Precompute-and-Reuse

FlowBot: Inducing LLM Workflows with Bilevel Optimization and Textual Gradients