Divide, Diagnose, and Compose: Optimizing Agentic Pipelines and Harnesses

Divij Handa; Christopher Tensmeyer; Jihyung Kil; Vlad I. Morariu

cs.MANov 5, 2026

Divide, Diagnose, and Compose: Optimizing Agentic Pipelines and Harnesses

Authors: Divij Handa, Christopher Tensmeyer, Jihyung Kil, Vlad I. Morariu

Organizations: Arizona State University · Adobe

Abstract

Building agentic pipelines that achieve high task performance while keeping cost and latency low requires jointly optimizing heterogeneous and coupled design choices, including prompts, topology, model selection, and tool policies. Manually searching this combinatorial design space is expensive and time-consuming, while existing automated optimizers typically operate within a predefined subset of this space, modify the pipeline monolithically, or provide little structured evidence for pipeline changes. To mitigate these challenges, we introduce the Decomposed Credit-Assignment Framework (DeCAF), which adapts the design space to the given benchmark and pipeline, optimizes each dimension separately, and generates reasoning traces for every decision. Given a task specification, benchmark examples, and a seed pipeline, DeCAF identifies relevant optimization dimensions and generates diverse experimental hypotheses for each. Independent coding-agent sessions then optimize each dimension under a restricted edit scope, jointly evaluating performance, inference cost, and latency while recording all hypotheses, edits, and outcomes of each experiment. A final synthesis session combines compatible findings across all dimensions and uses a component-level trajectory evaluator, which derives role-specific rubrics from each agent's instructions, to localize underperforming components and guide cross-dimension revisions. Across a diverse suite of agentic benchmarks, DeCAF improves average task performance by 2.58 percentage points, while reducing inference cost by 4.30% and end-to-end latency by 42.45% compared to the best overall baseline.

Figures & tables

Figure 1: Improvements of DeCAF over Meta-Harness (best baseline) across all benchmarks.

Figure 2: Given task details, development examples, seed agentic pipeline, and objectives, DeCAF first identifies task-relevant optimization dimensions and generates diverse experimental hypotheses for each ( Divide ). Independent coding-agent sessions then optimize copies of the seed pipeline under dimension-specific edit constraints, producing Pareto candidates, execution trajectories, and structured findings ( Diagnose ). Finally, a synthesis optimizer combines compatible dimension-level findings and performs targeted cross-dimension revisions to produce a unified pipeline on the performance–cost–latency frontier with the help of a component or node-level diagnosis tool ( Compose ).

		Deployment / instance		Optimization
Method	Task score $\uparrow$	Cost ( $)$ \downarrow$	Latency (s) $\downarrow$	Cost ( $)$ \downarrow$	Time $\downarrow$
AppWorld (TGC)
Seed pipeline (Sonnet-4.6-Medium)	78.17 ±1.21	0.2242	55.1	–	–
Stronger backbone (Opus-4.8-High)	87.20 ±0.89	0.4273	114.3	–	–
GEPA	74.40 ±1.19	0.2901	58.8	279.04	14:57
MASPO	78.17 ±1.21	0.2242	55.1	103.57	02:06

Table 1: Task scores (%

\pm

standard-error), cost-per-instance, latency-per-instance, one-time cost of optimization, and wall-clock time (hh:mm) of all optimizers. Each row is an average of two runs.

Figure 3: AppWorld optimization trajectories as a function of cumulative optimizer cost. Each point corresponds to an evaluated validation candidate; the panels report TGC, per-instance inference cost, and latency, respectively, while stars denote the pipeline ultimately selected by each optimizer. Trajectories need not be monotonic as they show individual evaluated candidates not the best so far.

	Task performance		Per instance		Optimizer overhead
Method	Score $\uparrow$	$\Delta$ (pp)	Cost ( $)$ \downarrow$	Latency (s) $\downarrow$	Cost ( $)$ \downarrow$	Wall-clock (h:mm:ss) $\downarrow$
AppWorld	TGC (%)
DeCAF (full)	84.23 ±0.89	0.00	0.1840	32.81	191.86	04:27:57
No Dimensions	79.17 ±1.19	–5.06	0.2273	56.63	153.59	02:59:56
Single Shared Session	80.67 ±0.84	–3.56	0.2349	55.92	174.80	07:01:55
No Experiment Proposals	74.40 ±2.98	–9.83	0.3838	78.47	134.23	06:12:08

Table 2: Ablation study on the AppWorld and LiveCodeBench benchmarks.

\Delta

denotes the absolute change in the benchmark-specific task performance metric relative to the full DeCAF method. Per-instance columns report test-time deployment metrics, while optimizer columns report one-time optimization overhead.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

		Test difficulty $\uparrow$			Per instance $\downarrow$		Optimizer $\downarrow$
Method	TGC $\uparrow$	1	2	3	Cost ($)	Latency (s)	Cost ($)	Latency
Base MAS
Opus-4.8-High	87.20 ±0.89	94.74 ±0.00	93.75 ±0.00	75.40 ±2.38	0.4273	114.3	–	–
Sonnet-4.6-Medium	78.17 ±1.21	86.55 ±0.59	84.03 ±2.50	66.14 ±1.91	0.2242	55.1	–	–
Haiku-4.5-Low	35.71 ±0.59	64.91 ±5.27	31.25 ±0.00	12.70 ±6.35	0.1094	86.5	–	–
GEPA	74.40 ±1.19	84.21 ±5.26	79.17 ±0.00	63.49 ±3.17	0.2901	58.8	279.04	14:56:44

Appendix

Table 3: Comparison of optimization methods on the AppWorld test set. Task goal completion (TGC) and per-difficulty performance are reported in % (mean

\pm

standard error). Per-instance columns report test-time cost (USD) and latency per task; optimizer columns report the total cost (USD) and wall-clock latency (h:mm:ss) of the optimizer.

\uparrow

\downarrow

indicate whether higher or lower is better.

	pass@1 $\uparrow$			Per instance $\downarrow$		Optimizer $\downarrow$
Method	Total	Medium	Hard	Cost ($)	Latency (s)	Cost ($)	Latency
Base MAS
GPT-5.4-nano	47.33 ±1.13	59.88 ±2.55	39.20 ±0.33	0.0051	29.3	–	–
GPT-5.4-mini	44.50 ±1.78	65.84 ±2.29	30.67 ±1.49	0.0085	10.5	–	–
GPT-5.4	55.34 ±0.56	82.31 ±1.09	37.87 ±0.27	0.0236	12.5	–	–
GEPA	48.30 ±1.70	62.96 ±3.70	38.80 ±0.40	0.0055	37.2	24.60	9:30:41

Appendix

Table 4: Comparison of optimization methods on the LiveCodeBench test set. Total and per-difficulty pass@1 are reported in % (mean

\pm

standard error). Per-instance columns report test-time cost (USD) and latency per task; optimizer columns report the total cost (USD) and wall-clock latency (h:mm:ss) of the optimizer.

\uparrow

\downarrow

indicate whether higher or lower is better.

	Performance (%) ( $\uparrow$ )				Per instance		Optimizer
Method	Under attack	Benign	Attacked	ASR ( $\downarrow$ )	Cost ($)	Latency (s)	Cost ($)	Latency
Base MAS
Qwen-3.5-397b-a17b-FP8	38.6 ±1.6	96.1 ±1.3	65.3 ±0.3	55.5 ±1.0	0.0163	16.5	–	–
Gemma-4-31B-it	69.5 ±0.7	90.3 ±0.7	82.8 ±0.3	18.8 ±0.7	0.0060	49.3	–	–
GEPA	82.1 ±1.0	86.4 ±0.7	82.8 ±1.0	0.7 ±0.0	0.0137	24.2	32.92	4:17:37
MASPO	85.1 ±1.3	92.2 ±0.0	86.0 ±1.0	1.6 ±0.3	0.0147	36.7	21.05	1:14:38

Appendix

Table 5: Results on AgentDojo. Utility and attack success rate (ASR) are reported in %, with the run-level standard error as subscript.

\downarrow

: lower is better.

	OOD test $\uparrow$					Per instance $\downarrow$		Optimizer $\downarrow$
Method	HumanEval	AIME 2026	BFCL	Average	Cheat. $\downarrow$	Cost	Latency	Cost	Latency
Base model
Qwen3-4B	31.71 ±0.00	1.67 ±0.96	38.16 ±0.01	23.85 ±0.32	–	–	–	–	–
Gemma-3-4B	0.61 ±0.00	0.00 ±0.00	19.73 ±0.00	6.78 ±0.00	–	–	–	–	–
Instruct model
Qwen3-4B	76.22 ±0.00	52.50 ±2.10	62.73 ±0.07	63.82 ±0.70	–	–	–	–	–

Appendix

Table 6: Per-model breakdown of the OOD test results of PostTrainBench (mean

\pm

standard error). Each method is evaluated on two held-out base models and three benchmarks. Their average and the cheating rate are reported in %. Optimizer columns report total cost (USD) and wall-clock latency (h:mm:ss); per-instance columns report the averaged cost and latency over the test instances.

\uparrow

\downarrow

indicate whether higher or lower is better.

Failure type	Injection	Example
Incorrect contribution	Modify a task-relevant conclusion or intermediate result while preserving the surrounding output.	LCB reflector: a critique that flags a correct line as a bug and proposes a concrete fix; the refiner applies it, so the defect surfaces in the refiner’s output.
Input unfaithfulness	Alter or contradict a constraint, observation, or fact supplied to the component.	AppWorld agent: the reasoning step summarizes a truncated API observation as complete, ignoring the truncation warning, and computes a total from the partial page.
Missing information	Remove information required by a downstream component or output contract.	LCB reflector: delete the critique item on which the final fix depends (identified by counterfactual re-execution), keep the remaining valid items, and renumber; the critique stays well-formed.
Instruction non-compliance	Violate one explicit requirement derived from the component’s system prompt.	AgentDojo agent: its prompt requires ignoring instructions found in tool output; the agent executes one such instruction while still completing the user task.

Appendix

Table 7: Controlled failures used to evaluate node_eval . Corruptions preserve the format and style of the original output, contain no marker of the injected location, and in several cases produce a visible defect only in a downstream component.

Injected failure	$N$	Top-1 localization (%)
Incorrect contribution	22	90.91
Input unfaithfulness	29	79.31
Missing information	22	86.36
Instruction non-compliance	27	55.56
Overall	100	77.00
Random Component	100	31.81

Appendix

Table 8: Top-1 failure-localization accuracy of node_eval under controlled component corruptions. Overall shows the accuracy of node_eval , while Random Component shows the accuracy if a random component is returned every turn.

LLM Agent Workflow Optimization

LLM agent workflow optimization studies how to design and run multi-step agents for better task success, reliability, latency, and cost.

See all 276 papers

AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent
Apr 7, 2026Wenyue Hua, Sripad Karne, Qian Xie +4LLM Agent Workflow Optimization LLM Agent Efficiency
Microsoft Research, AI Frontiers · Columbia University · Cornell University
TabAgent: Distilling LLM Agent Components into Cheap Classifiers
Feb 18, 2026Ido Levy, Segev Shlomov, Avi Yaeli +5LLM Agent Workflow Optimization AI Agent Architecture
IBM Research, Haifa, Israel
AgentRouter: Heterogeneous Model Routing for Cost-Optimal Multi-Step Agentic Workflows
Sep 19, 2026Rudrendu Kumar Paul, Sourav NandyLLM Agent Routing Cost-Aware Inference
Boston University, Boston, MA, USA · University of Texas at Austin, Austin, TX, USA

Apr 7, 2026J/K move · Enter open · S save

AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent

Wenyue Hua, Sripad Karne, Qian Xie, Armaan Agrawal+3

Microsoft Research, AI Frontiers · Columbia University · Cornell University

LLM Agent Workflow Optimization LLM Agent Efficiency LLM Agents

Open paper AskPDF arXiv

AI agents are increasingly deployed in real-world applications, including systems such as Manus, OpenClaw, and coding agents. Existing research has primarily focused on server-side efficiency, proposing methods such as caching, speculative execution, traffic scheduling, and load balancing to reduce the cost of serving agentic workloads. However, as users increasingly construct agents by composing local tools, remote APIs, and diverse models, an equally important optimization problem arises on the client side. Client-side optimization asks how developers should allocate the resources available to them, including model choice, local tools, and API budget across pipeline stages, subject to application-specific quality, cost, and latency constraints. Because these objectives depend on the task and deployment setting, they cannot be determined by server-side systems alone. We introduce AgentOpt, the first framework-agnostic Python package for client-side agent optimization. We first study model selection, a high-impact optimization lever in multi-step agent pipelines. Given a pipeline and a small evaluation set, the goal is to find the most cost-effective assignment of models to pipeline roles. This problem is consequential in practice: at matched accuracy, the cost gap between the best and worst model combinations can reach 13-32x in our experiments. To efficiently explore the exponentially growing combination space, AgentOpt implements ten search algorithms, including UCB-E, UCB-E with Low-Rank Factorization, Arm Elimination, Epsilon-LUCB, Threshold Successive Elimination, and Bayesian Optimization. Across four benchmarks, UCB-E recovers near-optimal accuracy while reducing evaluation budget by 62-76% relative to brute-force search. Code and benchmark results available at https://agentoptimizer.github.io/agentopt/.

Abstract

LLM Agent Workflow Optimization

Explore similar work

AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent

TabAgent: Distilling LLM Agent Components into Cheap Classifiers

AgentRouter: Heterogeneous Model Routing for Cost-Optimal Multi-Step Agentic Workflows