Planner-as-Router: Joint Plan-Time Model Routing for Cost-Efficient Multi-Agent Workflows

Vivek Kumar Singh; Preeti Priyam; Gautam Bhowmick

cs.AISep 26, 2026

Planner-as-Router: Joint Plan-Time Model Routing for Cost-Efficient Multi-Agent Workflows

Authors: Vivek Kumar Singh, Preeti Priyam, Gautam Bhowmick

Organizations: Independent Researcher McKinney, TX, USA · Independent Researcher Chicago, IL, USA

Abstract

Running large language model (LLM) agents in production gets expensive fast. A frontier model (the largest, most capable tier) is accurate but can cost 25 times what a small model costs per token, and the gap compounds once a workflow chains several calls together. Planner-as-Router (PaR) attacks this from a different angle. Instead of leaving model-tier selection to some component downstream, it folds the choice into planning itself. As the planner breaks a query into subtasks, it also assigns each one a model size tier (small, mid, or frontier, ordered by capability and price), so the dependencies between subtasks are visible before any specialist runs. Unlike per-call routers such as cascade routing, which look at one node at a time, PaR sees the whole workflow up front and needs no separate router model or training data. We evaluate PaR with EntBench, a benchmark of 54 enterprise agentic tasks across seven classes, graded by actually running the generated Structured Query Language (SQL) and MongoDB queries against live databases. Over 1,157 evaluations spanning eight routers and three seeds, PaR stays on the observed cost-accuracy frontier. It matches a sink-frontier heuristic (frontier model on terminal nodes only) in accuracy at comparable cost and a faithful FrugalGPT cascade at lower cost, and cuts cost 44% against all-frontier routing while giving up 2.9 points of accuracy. Several accuracy gaps fall inside the plus-or-minus six-point confidence interval of a 54-task study, so we frame PaR's advantage as frontier position rather than a clean accuracy win. We also report a preliminary observation, not a validated result: a small pilot hints that cheap routing may carry a hidden compounding penalty on compositional workflows, which we frame as a hypothesis for future measurement. PaR, EntBench, and all evaluation code are open source.

Figures & tables

Fig. 1: Planner-as-Router: tier assignment happens at plan time. The planner assigns a model tier to every subtask before any specialist runs; the dispatcher executes subtasks in dependency order, escalating one tier once on failure.

Class	Tasks	Type	Evaluator
SQL-Gen	8	Capability	Execution bag equiv.
Mongo-Gen	8	Capability	Execution bag equiv.
Extract	7	Capability	Field-level F1
SQL-Compose	8	Compositional	SQL + consumer check
Cross-Recon	8	Compositional	Multi-source, 3-stage
MultiTool-Plan	8	Compositional	Tool set + ordering

TABLE I: EntBench Task Classes (54 Tasks, Seven Classes)

Fig. 2: EntBench execution-based evaluation pipeline. A task flows through six stages, executing generated SQL/MongoDB against live backends so a task is correct only if its execution reproduces the gold result—capturing failure modes (bad joins, empty aggregates, malformed plans) that text comparison misses. The planner, specialist prompts, retry policy, and 3 seeds are held fixed across all eight routers; only per-subtask tier assignment varies.

Router	Description
PaR	Proposed: planner assigns a tier per subtask at plan time
PaR-Lite	Cascaded: Haiku classifier skips planner on simple tasks
AllFrontier	Every node on Opus 4.7 (accuracy upper bound)
AllSmall	Every node on Haiku 4.5 (cost lower bound)
SinkFrontier	Frontier on terminal nodes only, small elsewhere
SourceFrontier	Frontier on root nodes only, small elsewhere

TABLE II: The Eight Routing Strategies Evaluated on EntBench

Router	Acc.	Avg Cost (USD)	vs. PaR
AllFrontier	29.0%	0.0182	$+2.9$ pp, $+80\%$
SourceFrontier	28.5%	0.0146	$+2.4$ pp, $+45\%$
PaR	26.1%	0.0101	baseline
SinkFrontier	25.4%	0.0127	$-0.7$ pp, $+26\%$
AllSmall	24.6%	0.0039	$-1.5$ pp, $-61\%$
PaR-Lite	24.6%	0.0105	$-1.5$ pp, $+4\%$

TABLE III: Per-Router Results Excluding SQL-Compose (

n=138

per Router, Three Seeds)

Fig. 3: Cost–accuracy frontier on EntBench (excluding SQL-Compose). Bubble color encodes the compounding factor $\rho$ (a pilot value for PaR, measured for AllFrontier, unmeasured otherwise). PaR sits on the Pareto frontier at comparable cost to the sink-frontier heuristic.

Class	PaR	AllF	Src	Sink	AllS	Frug
SQL-Gen	50.0	50.0	58.3	50.0	50.0	50.0
Mongo-Gen	25.0	37.5	39.1	37.5	25.0	25.0
Extract	28.6	28.6	28.6	28.6	23.8	19.0
Policy-Action	28.6	28.6	28.6	23.8	28.6	28.6
Cross-Recon	12.5	12.5	12.5	8.3	12.5	12.5
MultiTool-Plan	12.5	16.7	4.2	4.2	4.2	12.5

TABLE IV: Per-Class Accuracy (%) by Router, Averaged Over Three Seeds

Class	Avg subtasks	Small %	Mid %	Frontier %
SQL-Gen	1.0	75	25	0
Mongo-Gen	1.0	88	12	0
Extract	1.0	95	5	0
Policy-Action	1.2	40	60	0
SQL-Compose	2.1	12	78	10
Cross-Recon	2.8	25	55	19

TABLE V: Planner Tier Assignment by Task Class (PaR, 162 Runs). Tier Columns Give the Percentage of Subtasks Routed to Each Tier.

Router	$\rho$	Actual	Naive Pred.	Status
PaR	3.33	25.0%*	77.5%*	Pilot, high
AllFrontier	1.27	27.8%	43.1%	Measured, low

TABLE VI: Compounding Error (

\rho

). (*)

\rho

Values From a 21-Task Pilot Sweep

Fig. 4: Compounding-error mechanism (illustrative). Cheap-tier assignments (left) can suffer cascade failures when a small-tier node fails and corrupts downstream input; all-frontier routing (right) keeps per-node failure rare, so compounding stays mild. PaR’s $\rho$ is a 21-task pilot, not a validated measurement.

Explore similar work

Sep 19, 2026cs.AI

AgentRouter: Heterogeneous Model Routing for Cost-Optimal Multi-Step Agentic Workflows

Enterprise agentic systems that route every trajectory step to a frontier model waste 60-80% of their inference budget on subtasks that smaller models handle equally well. Existing routing solutions optimize single-turn query assignment but ignore a property unique to agentic workflows: subtask complexity varies widely within a single trajectory. A planning step may require frontier-class reasoning while a subsequent formatting step needs only a 7B model. We formalize step-level model routing as a sequential assignment problem over agent trajectories and propose AgentRouter, a lightweight classifier (12M parameters, <5ms overhead per step on an A100 GPU) that maps each trajectory step to one of four model tiers using five features extractable at routing time. Trained on 50,000 annotated agent trajectory steps spanning planning, coding, research, and data analysis tasks, AgentRouter achieves 72% cost reduction relative to frontier-only baselines, retaining 97.3% of frontier-only quality (less than 3% degradation in end-to-end task completion); per-step routing accuracy reaches 91% on minimal-complexity steps and 85% on efficient-tier steps, with 76-82% on the harder mid-range and frontier tiers. On the same benchmarks, RouteLLM and FrugalGPT (applied per-step) achieve only 31% and 44% cost reduction respectively, because their single-turn training signal misses trajectory-level quality dependencies.

Rudrendu Kumar Paul, Sourav Nandy

Boston University, Boston, MA, USA · University of Texas at Austin, Austin, TX, USA

LLM Agent Routing Cost-Aware Inference Agentic Workflows

Ask PDF

Sep 28, 2026cs.AI

RSI-Router: Evolving Subtask-Level LLM Routing and Skills for Cost-Efficient Agents

Practical deployment of large language model (LLM) agents requires strong task performance at affordable inference cost. For long-horizon agentic tasks, this performance-cost trade-off can be improved through within-task large-small model collaboration, as smaller models can handle some stages even when they cannot solve the full task. In this paper, we introduce RSI-router, a routing framework that constructs subtask-level model assignments and model-specific skills through recursive self-improvement over accumulated experience. Each iteration consists of four stages: Subtask Mining derives subtask definitions and identification rules from training trajectories; Routing Strategy Evolution proposes and evaluates diverse model assignments; Model-Specific Skill Evolution compares routed and large-model-only trajectories to diagnose failures and develop reusable execution skills; and Pareto-Optimal Router Selection updates the Pareto population using historical and newly generated routers while retaining dominated routers as experience for subsequent evolution. Routing between DeepSeek-V4.1-Flash and Qwen3.5-9B, RSI-router consistently surpasses the DeepSeek-only baseline at roughly half the inference cost (48.3%) across five agentic benchmarks. In particular, on ALFWorld, ScienceWorld, and WebShop, it cuts inference cost by 74.7-82.2% while simultaneously improving performance; on Terminal-Bench 2.0, it achieves a 16.7% relative performance gain at 18.0% lower cost. Moreover, RSI-router establishes a stronger performance--cost Pareto frontier than 9 routing methods.

Hao Li, Hangfan Zhang, Zhiyao Cui +5

Shanghai Artificial Intelligence Laboratory · Northwestern Polytechnical University

Cost-Aware Inference LLM Agent Workflow Optimization LLM Agent Skill Learning LLM Routing Long-Horizon LLM Agents Recursive Self-Improvement+1

Ask PDF

Apr 26, 2026cs.CL

GraphPlanner: Graph Memory-Augmented Agentic Routing for Multi-Agent LLMs

LLM routing has achieved promising results in integrating the strengths of diverse models while balancing efficiency and performance. However, to support more realistic and challenging applications, routing must extend into agentic LLM settings, where task planning, multi-round cooperation among heterogeneous agents, and memory utilization are indispensable. To address this gap, we propose GraphPlanner, a heterogeneous graph memory-augmented agentic router for multi-agent LLMs that generates routing workflows for each query and supports both inductive and transductive inference. GraphPlanner formulates workflow generation as a Markov Decision Process (MDP), where at each step it selects both the LLM backbone and the agent role, including Planner, Executor, and Summarizer. By leveraging a heterogeneous graph, denoted as GARNet, to capture interaction memories among queries, agents, and responses, GraphPlanner integrates historical memory and workflow memory into richer state representations. The entire pipeline is optimized with reinforcement learning, jointly improving task-specific performance and computational efficiency. We evaluate GraphPlanner across 14 diverse LLM tasks and demonstrate that: (1) GraphPlanner outperforms strong single-round and multi-round routers, improving accuracy by up to 9.3% while reducing GPU cost from 186.26 GiB to 1.04 GiB; (2) GraphPlanner generalizes robustly to unseen tasks and LLMs, exhibiting strong zero-shot capabilities; and (3) GraphPlanner effectively leverages historical memories, supporting both inductive and transductive inference for more adaptive routing. Our code for GraphPlanner is released at https://github.com/ulab-uiuc/GraphPlanner.

Tao Feng, Haozhen Zhang, Zijie Lei +2

Department of Computer Science University of Illinois Urbana-Champaign Urbana, IL, USA

Multi-Agent LLM Systems LLM Agent Orchestration LLM Agent Workflow Optimization Graph-Based Agent Memory LLM Routing LLM Agent Routing+1

Ask PDF