Explainable Model Routing for Agentic Workflows

Mika Okamoto; Ansel Kaplan Erol; Mark Riedl

cs.AIApr 4, 2026

Explainable Model Routing for Agentic Workflows

Authors: Mika Okamoto, Ansel Kaplan Erol, Mark Riedl

Organizations: Georgia Institute of Technology, USA

Abstract

Modern agentic workflows decompose complex tasks into specialized subtasks and route them to diverse models to minimize cost without sacrificing quality. However, current routing architectures focus exclusively on performance optimization, leaving underlying trade-offs between model capability and cost unrecorded. Without clear rationale, developers cannot distinguish between intelligent efficiency -- using specialized models for appropriate tasks -- and latent failures caused by budget-driven model selection. We present Topaz, a framework that introduces formal auditability to agentic routing. Topaz replaces silent model assignments with an inherently interpretable router that incorporates three components: (i) skill-based profiling that synthesizes performance across diverse benchmarks into granular capability profiles (ii) fully traceable routing algorithms that utilize budget-based and multi-objective optimization to produce clear traces of how skill-match scores were weighed against costs, and (iii) developer-facing explanations that translate these traces into natural language, allowing users to audit system logic and iteratively tune the cost-quality tradeoff. By making routing decisions interpretable, Topaz enables users to understand, trust, and meaningfully steer routed agentic systems.

Figures & tables

Figure 1. Topaz Architecture. Public benchmarks are synthesized to form model capability profiles. Then, for a new agentic workflow, each subtask is analyzed for complexity and skill requirements. The Topaz routing engine balances skill match and cost, yielding model assignments for each subtask while providing explainable traces for developers. Architecture diagram showing two profiling pipelines feeding into a central routing engine. The bottom pipeline synthesizes public benchmarks into model capability profiles. The top pipeline analyzes an agentic workflow's subtasks for complexity, token-length, and skill requirements. The pipelines share a unified skill taxonomy. The routing engine balances skill matching against cost, producing model assignments and a routing explanation with skill-driven rationale and cost-quality tradeoff justification, with a feedback loop for user-tuned quality sensitivities.

Model	Math	Logic	Code	Tool	Fact.	Write	Instr.	Summ.	$p^{in}$ ($)	$p^{out}$ ($)
Claude-Opus-4.5	.967	.966	.974	.988	.955	.969	.979	.963	5.00	25.00
Gemini-3-Pro	.999	.988	.981	.953	.999	.999	.984	.996	2.00	12.00
GPT-5.2	.991	.974	.992	.849	.981	.971	.903	.995	1.75	14.00
Llama-4-Maverick	.660	.626	.433	.504	.826	.851	.719	.817	0.15	0.60
Mistral-Small-3.1	.506	.578	.593	.544	.704	.872	.763	.817	0.10	0.30

Table 1. Model skill profiles from Topaz and costs per million tokens (USD).

Figure 2. Customer Support Agentic Workflow with colors representing quality sensitivity. The AI Agent parses and classifies tickets, queries relevant documents, and reasons to form a technical diagnosis, lastly escalating or directly responding to the customer. Flowchart of a six-stage customer support pipeline. Ticket Classification (low sensitivity, green) feeds into Knowledge Base Search (moderate sensitivity, yellow), which feeds into Technical Diagnosis (high sensitivity, red). Technical Diagnosis branches into two paths: resolved leads to Refund Calculation (high sensitivity, red) then Response Drafting (high sensitivity, red); escalate leads to Escalation Summary (low sensitivity, green). Each stage lists its required skills, such as logic and tool use for Technical Diagnosis, and math and logic for Refund Calculation.

$c_{\text{global}}$	Assignments	Trace-driven Explanation of Routing Decisions
0.00	TC → Gemini-3-Pro KB → Claude-Opus-4.5 TD → Claude-Opus-4.5 RC → Gemini-3-Pro RD → Gemini-3-Pro ES → Gemini-3-Pro	With cost sensitivity set to zero, the engine prioritized raw performance over cost, selecting Claude Opus for the most complex tasks—Technical Diagnosis and Knowledge Base Search—due to its superior logical reasoning and tool-use capabilities. Gemini 3 Pro was routed to tasks like Refund Calculation and Response Drafting where its high-tier mathematical reasoning and writing quality were the primary success factors. For simpler objectives like Ticket Classification and Escalation Summary, the system selected Gemini because it fully satisfied the required skill sets [ $\ldots$ ]
0.50	TC → Mistral-Small-3.1 KB → Gemini-3-Pro TD → Gemini-3-Pro RC → Gemini-3-Pro RD → Gemini-3-Pro ES → Mistral-Small-3.1	For high-complexity tasks like Technical Diagnosis and Refund Calculation, Gemini was selected because it offered the strongest alignment with critical skills like logical reasoning and mathematical precision while maintaining a significantly better cost-to-performance ratio than Claude. In contrast, for simpler or high-volume tasks such as Ticket Classification and Escalation Summary, the engine prioritized Mistral to maximize cost savings, as its summarization and extraction capabilities were sufficient for the lower quality-sensitivity requirements. Ultimately, the system reserves premium compute for tasks requiring specialized skills like tool use [ $\ldots$ ]
1.00	TC → Mistral-Small-3.1 KB → Mistral-Small-3.1 TD → Gemini-3-Pro RC → Mistral-Small-3.1 RD → Mistral-Small-3.1 ES → Mistral-Small-3.1	For most tasks, including Ticket Classification and Response Drafting, the engine prioritized Mistral because your high cost sensitivity outweighed the marginal quality gains of larger models, as Mistral provided sufficient capabilities for summarization and writing. However, for the high-complexity Technical Diagnosis task, the system selected Gemini to ensure the superior logical reasoning and tool-use performance necessary for that specific objective. Even for difficult tasks like Refund Calculation, the engine opted for more economical models like Mistral, effectively trading off specialized mathematical reasoning to adhere to your strict cost constraints.
TC=Ticket Classification, KB=Knowledge Base Search, TD=Technical Diagnosis, RC=Refund Calculation, RD=Response Drafting, ES=Escalation Summary. See Appendix B.5 for full score breakdowns and results at $c_{\text{global}}=0.05,0.95$ .

Table 2. Routing assignments and system-generated explanations across cost sensitivity settings.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	Skill Weights ( $w$ )
Benchmark	Math	Logic	Code	Tool	Fact.	Write	Instr.	Summ.	Max Score
TextArena	—	.15	—	—	.15	.35	.35	—	1481
SearchArena	—	—	—	.30	.20	.20	—	.30	1224
BFCL v4	—	.15	—	.70	—	—	.15	—	77.47
SWE-bench Verified	—	.40	.30	.30	—	—	—	—	75.4
LiveCodeBench	.10	.30	.50	—	—	—	.10	—	86.41

Appendix

Table 5. Benchmark skill weight decompositions used by Topaz to compute model capability profiles. Each row defines how a benchmark’s score is attributed across skills, based on analysis of the benchmark’s design, question types, and evaluation criteria.

	Skill Requirements ( $R_{t,s}$ )								Routing Params		Token Est.
Subtask	Math	Logic	Code	Tool	Fact.	Write	Instr.	Summ.	$q$	$k$	$\sigma_{\text{in}}$	$\sigma_{\text{out}}$
Ticket Classification	—	.10	—	—	—	—	.40	.50	0.65	0.25	400	80
Knowledge Base Search	—	—	—	.40	—	—	.30	.30	0.55	0.50	500	1000
Technical Diagnosis	—	.40	—	.30	—	—	.10	.20	1.00	0.95	2000	500
Refund Calculation	.40	.40	—	—	—	—	.20	—	0.95	0.80	1200	200
Response Drafting	—	—	—	—	—	.60	.40	—	0.90	0.60	1500	400

Appendix

Table 6. Customer support pipeline subtask profiles. Skill requirements (

R_{t,s}

) define the required capability distribution; complexity (

k

) and quality sensitivity (

q

) parameterize the routing objective. Token count estimates

\sigma_{\text{in}},\sigma_{\text{out}}

determine per-call cost.

		Selected Model				Runner-up
$c_{\text{global}}$	Task	Model	$M$	$C$	$S$	Model	$M$	$C$	$S$	$\Delta$	Decisive Factor
$c_{\text{global}}=0.00$ : quality-only (cost weight $0$ for all tasks)
0.00	TC	Gemini-3	1.00	.143	.650	all tied	1.00	0.143	.650	.000	Tiebreak best non-capped $M$
	KB	Claude-O	.995	1.00	.547	Gemini-3	.981	.154	.540	.007	Highest $M$ via tool_use
	TD	Claude-O	.711	1.00	.711	Gemini-3	.709	.144	.709	.002	Highest $M$ in logic+tool_use
	RC	Gemini-3	.697	.142	.662	GPT-5.2	.691	.145	.657	.005	Best math+logic $M$

Appendix

Table 7. Routing decisions across cost sensitivity settings showing the selected model, runner-up, and the dominant factor in each decision.

M

= skill match (dot product),

C

= normalized cost penalty,

S

= final objective score,

\Delta

= score margin over runner-up.

Budget	Assignments	Trace-driven Explanation of Routing Decisions
$5.00	TC $\to$ Mistral-Small-3.1 KB $\to$ Mistral-Small-3.1 TD $\to$ Llama-4-Maverick RC $\to$ Llama-4-Maverick RD $\to$ Mistral-Small-3.1 ES $\to$ Mistral-Small-3.1	To maximize quality within the budget, the engine routed lower-complexity tasks like Ticket Classification and Escalation Summary to Mistral, as its summarization and instruction-following capabilities provided the best value for standard requirements. For the highly sensitive Technical Diagnosis and Refund Calculation tasks, the system upgraded to Llama-4-Maverick to leverage its stronger logical and mathematical reasoning. Because top-tier frontier models like GPT-5.2 or Gemini-3-Pro were cost-prohibitive for the entire six-task sequence, these selections represent the most effective performance-to-cost tradeoff available under your constraints.
$50.00	TC $\to$ Mistral-Small-3.1 KB $\to$ Gemini-3-Pro TD $\to$ Gemini-3-Pro RC $\to$ Gemini-3-Pro RD $\to$ Gemini-3-Pro ES $\to$ Mistral-Small-3.1	We prioritized Gemini for high-stakes tasks like Technical Diagnosis, Refund Calculation, and Response Drafting because these required the advanced logical reasoning, mathematical accuracy, and writing quality necessary to meet your high quality-sensitivity targets. To stay within budget, we routed simpler tasks like Ticket Classification and Escalation Summary to the more economical Mistral, leveraging its efficiency for basic summarization and instruction following where precision was less critical. While elite models like Claude Opus were evaluated, they were excluded to prevent a budget overrun, leaving Gemini as the optimal balance for providing necessary tool-use and complex reasoning capabilities. This strategy ensures that your most complex requirements receive top-tier performance while maintaining overall cost-effectiveness for routine operations.
$100.00	TC $\to$ Mistral-Small-3.1 KB $\to$ Gemini-3-Pro TD $\to$ Claude-Opus-4.5 RC $\to$ Gemini-3-Pro RD $\to$ Gemini-3-Pro ES $\to$ Mistral-Small-3.1	We prioritized the most resource-intensive task, Technical Diagnosis, by selecting Claude-Opus to leverage its superior logical reasoning and tool-use capabilities where quality sensitivity was at its maximum. To offset this premium cost, we routed low-complexity tasks like Ticket Classification and Escalation Summary to Mistral, which provides sufficient summarization and extraction skills at a fraction of the price. The remaining high-sensitivity tasks, such as Refund Calculation and Response Drafting, were assigned to Gemini to ensure high-tier mathematical and writing precision without exceeding the total budget. This strategy successfully balances peak performance for critical reasoning steps with aggressive cost-saving on tasks where the user indicated lower quality requirements.
TC=Ticket Classification, KB=Knowledge Base Search, TD=Technical Diagnosis, RC=Refund Calculation, RD=Response Drafting, ES=Escalation Summary.

Appendix

Table 8. Routing assignments and system-generated explanations across budget constraints (per 1,000 pipeline runs).

LLM Agent Routing

LLM agent routing selects models, tools, skills, or collaboration workflows for each task or step to balance capability, cost, latency, and reliability.

See all 99 papers

AgentRouter: Heterogeneous Model Routing for Cost-Optimal Multi-Step Agentic Workflows
Sep 19, 2026Rudrendu Kumar Paul, Sourav NandyLLM Agent Routing Cost-Aware Inference
Boston University, Boston, MA, USA · University of Texas at Austin, Austin, TX, USA
Switchcraft: AI Model Router for Agentic Tool Calling
May 8, 2026Sharad Agarwal, Pooria Namyar, Alec Wolman +3Tool-Augmented Language Model Agents Cost-Aware Inference
Microsoft Research · Microsoft · Stanford
Planner-as-Router: Joint Plan-Time Model Routing for Cost-Efficient Multi-Agent Workflows
Sep 26, 2026Vivek Kumar Singh, Preeti Priyam, Gautam BhowmickLLM Agent Workflow Optimization LLM Routing
Independent Researcher McKinney, TX, USA · Independent Researcher Chicago, IL, USA

Sep 19, 2026J/K move · Enter open · S save

AgentRouter: Heterogeneous Model Routing for Cost-Optimal Multi-Step Agentic Workflows

Rudrendu Kumar Paul, Sourav Nandy

Boston University, Boston, MA, USA · University of Texas at Austin, Austin, TX, USA

LLM Agent Routing Cost-Aware Inference Agentic Workflows

Open paper AskPDF arXiv

Enterprise agentic systems that route every trajectory step to a frontier model waste 60-80% of their inference budget on subtasks that smaller models handle equally well. Existing routing solutions optimize single-turn query assignment but ignore a property unique to agentic workflows: subtask complexity varies widely within a single trajectory. A planning step may require frontier-class reasoning while a subsequent formatting step needs only a 7B model. We formalize step-level model routing as a sequential assignment problem over agent trajectories and propose AgentRouter, a lightweight classifier (12M parameters, <5ms overhead per step on an A100 GPU) that maps each trajectory step to one of four model tiers using five features extractable at routing time. Trained on 50,000 annotated agent trajectory steps spanning planning, coding, research, and data analysis tasks, AgentRouter achieves 72% cost reduction relative to frontier-only baselines, retaining 97.3% of frontier-only quality (less than 3% degradation in end-to-end task completion); per-step routing accuracy reaches 91% on minimal-complexity steps and 85% on efficient-tier steps, with 76-82% on the harder mid-range and frontier tiers. On the same benchmarks, RouteLLM and FrugalGPT (applied per-step) achieve only 31% and 44% cost reduction respectively, because their single-turn training signal misses trajectory-level quality dependencies.

Abstract

LLM Agent Routing

Explore similar work

AgentRouter: Heterogeneous Model Routing for Cost-Optimal Multi-Step Agentic Workflows

Switchcraft: AI Model Router for Agentic Tool Calling

Planner-as-Router: Joint Plan-Time Model Routing for Cost-Efficient Multi-Agent Workflows