RouteWeaver: Weaving Mode Selection and Execution into Unified LLM Routing

Xiaohan Wang; Haozhen Zhang; Qingyuan Liu; Tao Feng; Wenya Wang

cs.CLOct 14, 2026

RouteWeaver: Weaving Mode Selection and Execution into Unified LLM Routing

Authors: Xiaohan Wang, Haozhen Zhang, Qingyuan Liu, Tao Feng, Wenya Wang

Organizations: Nanyang Technological University · Huazhong University of Science and Technology · University of Illinois Urbana-Champaign

Abstract

LLM routing dynamically selects and coordinates heterogeneous language models across diverse tasks. However, existing approaches are largely built around a predefined execution paradigm, such as single-round, multi-round, or agentic routing, which fixes how model calls are organized and limits adaptation to different query needs. We propose RouteWeaver, a unified routing framework that integrates these routing modes within a single LLM policy, jointly learning which mode to use and how to select models within that mode. To jointly optimize mode selection and within-mode execution, we develop COMET-GRPO (COordinated Mode-Execution Training), which decouples the trajectory-level advantage into separate signals for mode selection and within-mode execution, enabling more targeted credit assignment to each decision level. We further introduce progressive exploration, which gradually shifts training from externally assigned modes to autonomous selection, so that the router first learns how to execute each mode before deciding when to use it. We evaluate RouteWeaver across mathematics, coding, knowledge, logical reasoning, and factual recall. RouteWeaver achieves an average accuracy of 61.4%, an 11.03% relative improvement over the strongest baseline, and outperforms all compared baselines in each of the five domains. Ablations show that both COMET-GRPO and progressive exploration are important for adaptive routing; without progressive exploration, the routing policy collapses entirely to single-round execution.

Figures & tables

Figure 1: Three execution paradigms. Top left, single-round: the router selects one Worker for a single call. Top right, multi-round: the router selects Workers sequentially using accumulated observations. Bottom, agentic: the router selects Workers, assigns roles, and specifies dependencies between calls; the depicted workflow is illustrative, and its roles and dependencies vary across queries.

Figure 2: Overview of RouteWeaver . (a) One routing policy $\pi_{\theta}$ selects an execution mode, then calls frozen Workers;Multi-round calls automatically use the latest observation, while agentic calls include role assignments and router-selected references to earlier outputs.Returned observations are appended to its context until it emits the final answer. (b) COMET-GRPO computes mode-level and within-mode advantages per reward component, mixes them (Eq. (7)), and assigns $A_{i}^{\mathrm{mode}}$ to mode-selection tokens and $A_{i}^{\mathrm{inner}}$ to subsequent router tokens. (c) Progressive training moves mode selection from assigned modes to the router in three stages.

Single-round Router
Method	Math	Code	Knowledge	Reason	Recall	Avg
Router-KNN	61.7	34.8	38.3	68.3	40.4	48.7
Router-MLP	58.7	33.2	40.0	59.2	34.2	45.1
RouterDC	61.3	29.4	34.2	57.5	35.0	43.5
GraphRouter	64.6	36.1	34.2	69.2	39.6	48.7
Multi-round Router

Table 1: Accuracy (%) on the evaluation benchmarks; RouteWeaver is trained with

\alpha=0

. Domain accuracy is the unweighted mean of its constituent dataset accuracies; Avg is the unweighted mean of the five domain accuracies. Best results are in bold ; numbers in parentheses give the relative improvement of RouteWeaver over the strongest baseline in each column.

Figure 3: Fixed-reference Worker inference cost per query against average accuracy. The curve shows RouteWeaver trained with

\alpha\in\{0,0.1,0.3,0.5\}

; markers show the two strongest baselines of each execution paradigm in Table 1 . The

\alpha=0

cost is reconstructed from token usage.

	Accuracy (%)						Mode usage
Variant	Math	Code	Knowledge	Reason	Recall	Avg	S / M / A
RouteWeaver	74.6	58.2	55.0	76.7	42.5	61.4	.35 / .41 / .24
w/o COMET-GRPO	72.3	57.2	49.2	74.1	41.2	58.8	.13 / .82 / .05
w/o progressive exploration	70.2	48.6	50.3	68.3	40.4	55.6	1.00 / .00 / .00

Table 2: Ablation of the two learning components, all trained with

\alpha=0

under the same budget. Accuracy (%) is reported by domain. Mode usage is reported as the aggregate proportion of evaluation queries assigned to single-round (S), multi-round (M), and agentic (A) execution.

Figure 4: Share of selected execution modes across evaluation domains for RouteWeaver (

\alpha=0

). Each bar shows the proportion of test queries routed to single-round, multi-round, and agentic execution.

Method	AIME	GPQA-D	BBH	Avg
GraphRouter (single-round)	23.5	49.5	24.7	32.6
Router-R1 (multi-round)	45.6	62.1	28.6	45.4
MasRouter (agentic)	28.1	43.8	28.2	33.4
RouteWeaver	57.0	69.2	35.0	53.7

Table 3: Accuracy (%) on benchmarks held out from router training across three domains: AIME (Math), GPQA-Diamond (Knowledge), and BBH (Reason). We compare one representative router from each execution paradigm.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Element	Illustrative form	Produced by
Mode selection	<mode>agentic</mode>	Router
Worker call	<route model="worker_2" role="verifier" refs="n1"> Check the proposed solution.</route>	Router
Returned output	<observation id="n2">…</observation>	Environment
Final response	<answer>…</answer>	Router

Appendix

Table 4: Schematic of the shared trajectory representation.

Panel of Figure 5 — Figure 5: Abbreviated trajectories of the reported router, one per mode (Omni-MATH evaluation questions). Observation bodies and parts of long sub-queries are elided (…), the routing-state messages that the environment inserts between agentic layers are omitted, and line breaks inside routes are added for display.

Domain	Dataset	#Queries	Per batch
Math	Omni-MATH	700	7
Code	APPS, LiveCodeBench	400, 300	4, 3
Knowledge	SuperGPQA	600	6
Reason	SATBench	700	7
Recall	TriviaQA, PopQA ( Mallen et al., 2023 )	200, 300	5
Total		3,200	32

Appendix

Table 5: Training data by domain.

ID	Model	Capability description in the router prompt
worker_1	Qwen3-8B	Dense language model with 8.2B parameters. Supports multilingual instruction following, dialogue, mathematical and logical reasoning, and code generation.
worker_2	Llama-3.1-8B-Instruct	Dense language model with 8B parameters. Instruction-tuned for multilingual dialogue, with capabilities in general reasoning, text generation, and coding.
worker_3	Qwen3-30B-A3B-Instruct-2507	Mixture-of-experts language model with 30.5B total and 3.3B active parameters. Supports instruction following, text comprehension, logical reasoning, mathematics, science, and coding.
worker_4	Gemma-3-27B-IT	Dense language model in the 27B parameter class. Supports multilingual question answering, summarization, reasoning, mathematical problem solving, and code generation.
worker_5	Llama-3.3-70B-Instruct	Dense language model with 70B parameters. Instruction-tuned for multilingual dialogue, with capabilities in reasoning, instruction following, and text and code generation.
worker_6	Gemini-2.5-Flash-Lite	Sparse mixture-of-experts model with undisclosed parameter counts. Supports translation, classification, information extraction, coding, mathematics, and reasoning.

Appendix

Table 6: Worker pool and the capability descriptions shown to the router.

ID	Model	Input	Output
worker_1	Qwen3-8B	0.020	0.050
worker_2	Llama-3.1-8B-Instruct	0.020	0.050
worker_3	Qwen3-30B-A3B-Instruct-2507	0.100	0.300
worker_4	Gemma-3-27B-IT	0.119	0.200
worker_5	Llama-3.3-70B-Instruct	0.100	0.320
worker_6	Gemini-2.5-Flash-Lite	0.100	0.400

Appendix

Table 7: Fixed reference prices (USD per million tokens).

LLM Routing

Large language model routing selects models, tools, experts, or inference strategies to balance answer quality, cost, latency, and deployment constraints.

See all 345 papers

Sep 29, 2026

FlexRouter: Learning Complementary Model Sets for Flexible LLM Routing

Existing Large Language Model (LLM) routing methods score LLMs independently to select top-

k

models. However, this ignores model correlations and enforces a rigid computational budget. Consequently, routers often select redundant models that share failure modes, limiting the overall probability of success. To address this, we propose FlexRouter, a routing framework that explicitly models model complementarity. FlexRouter optimizes for answer coverage, maximizing the probability that at least one selected model yields a correct response. This objective aligns with practical inference pipelines where multiple candidate outputs are generated and a downstream verifier or user selects the final one. We formulate routing as a coverage-oriented subset selection problem and model the routing policy using Determinantal Point Processes (DPPs), which naturally capture both model competence and redundancy. To directly optimize coverage without requiring a ground-truth target subset, we introduce a training objective based on marginalizing over failure sets. During inference, we employ a greedy strategy based on marginal log-determinant gains, enabling the router to adaptively determine subset sizes without a predefined budget. Extensive experiments on the large-scale RouterEval benchmark demonstrate that our proposed FlexRouter achieves higher coverage with lower redundancy across both in-domain and out-of-domain tasks than strong baselines while maintaining flexible inference cost.

Wang Wei, Harry Yang, Tiankai Yang +6

Virginia Tech · Independent Researcher · University of Southern California +2

Adaptive Model Routing LLM Routing

Ask PDF

Jul 26, 2026

WISERouter: LLM Routing with Workload Budget Constraint

Large language models (LLMs) achieve impressive performance across multiple domains, but using the most capable model for every query is prohibitive at scale. LLM routing exploits diversity in model capability and cost by assigning each query to a suitable model to balance utility and budget. Current methods have two limitations: (i) they either use heuristics that do not always enforce the budget constraint or impose a fixed per-query budget that cannot adapt across the workload and leads to suboptimal performance; (ii) they require supervised learning on a dense dataset with statistics for every query-model pair, which is expensive to collect. To address these challenges, we formulate LLM routing as a constrained contextual multi-armed bandit problem and introduce WISERouter (WR for short), a framework that supports offline learning from historical interactions as well as online learning with exploration. We further prove that WR-Online achieves a sublinear regret bound of

O(\sqrt{T})

over a time horizon

T

. Empirical results on RouterBench and SWE-Bench demonstrate that (i) WR-Offline surpasses existing baselines in performance under a fixed budget and adheres more closely to budget constraints, and (ii) WR-Online achieves comparable performance to the baselines, while using substantially less exploration data.

Yifei Li, Zihui Gao, Laks V. S. Lakshmanan

The University of British Columbia

LLM Routing Contextual Bandits

Ask PDF

Aug 7, 2026

LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers

No single large language model (LLM) is optimal across all queries and budget constraints, making model routing essential for cost-effective deployment. Existing routers adopt diverse formulations and implementations, making fair comparison and extension difficult. We present a unified formulation of LLM routing as a sequential decision process characterized by five components: context encoders, model encoders, scoring functions, decision rules, and learning signals, covering single-turn, multi-turn, and personalized routing. Based on this formulation, we develop an automated pipeline for constructing routing supervision and evaluating routers jointly on response quality and inference cost. The resulting benchmark, xRouteBench, spans generic LLM, memory-augmented, vision, time-series, and personalized routing tasks. We further introduce LLMRouter, an open-source modular infrastructure with more than 16 representative routers. Our empirical study shows that learned routers outperform the strongest fixed-model baseline by 14.6% relatively, lightweight routers become more competitive under tight cost constraints, and user-conditioned routing consistently improves personalization.

Tao Feng, Fangxu Yu, Haozhen Zhang +9

University of Illinois Urbana-Champaign · University of Maryland, College Park · 3Nanyang Technological University +2

LLM Evaluation LLM Routing

Ask PDF

Abstract

LLM Routing

Explore similar work

FlexRouter: Learning Complementary Model Sets for Flexible LLM Routing

WISERouter: LLM Routing with Workload Budget Constraint

LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers