EvoRouter: Cost-Efficient Multi-Agent Routing via Insight-Guided On-Policy Distillation

Yitao Xiao; Xu Zhang; Ye Cheng; Shaoyong Guo; Yan Sun; Ziyue Zhu; Sitao Xie; Guo Tong

cs.AIOct 26, 2026

EvoRouter: Cost-Efficient Multi-Agent Routing via Insight-Guided On-Policy Distillation

Authors: Yitao Xiao, Xu Zhang, Ye Cheng, Shaoyong Guo, Yan Sun, Ziyue Zhu, Sitao Xie, Guo Tong,

Organizations: Beijing University of Posts and Telecommunications, Beijing, China · Fudan University, Shanghai Innovation Institute, Shanghai, China · Nanyang Technological University, Singapore

Abstract

Serving web queries at scale with multi-agent systems requires jointly configuring large language models, collaboration modes, agent counts, interaction rounds, and skills. However, the resulting configuration space grows combinatorially, while sparse execution rewards provide limited fine-grained supervision for interdependent planning decisions. We propose EvoRouter, a self-evolving multi-agent routing framework driven by insight-guided on-policy distillation. A dedicated expert periodically synthesizes performance-cost feedback from configuration trajectories across tasks into reusable conditional routing insights about capability requirements, returns from resource scaling, and interactions among configuration choices. A privileged teacher combines these evolving insights with current execution feedback to provide dense token-level supervision for the router's own planning trajectories through on-policy distillation. Cost-aware reinforcement learning jointly optimizes task performance and computational cost. Together, these training signals internalize execution insights into the routing policy. The updated policy generates new trajectories that support subsequent insight extraction, forming an execution-driven self-improvement loop. We evaluate EvoRouter on 19 in-domain datasets spanning five domains and two out-of-domain benchmarks. EvoRouter achieves an average in-domain accuracy of 68.69%, outperforming the strongest routing baseline by 7.23 percentage points while reducing inference compute cost by 50.95%. Our code is open-sourced at https://github.com/CenYangtze/EvoRouter.

Figures & tables

Figure 2. Motivation and performance of EvoRouter. Left: Two challenges and our solutions. Right: EvoRouter’s performance.

Method	Routing scope	Learning signal	Training mechanism
RouteLLM ( Ong et al., 2025 )	Single-model choice	Pairwise preferences	Supervised learning
Router-R1 ( Zhang et al., 2025 )	Multi-round calls	Format, outcome, cost	Outcome-based RL
MasRouter ( Yue et al., 2025b )	Modes, roles, models	Utility–cost reward	Policy gradients
DAAO ( Su et al., 2026 )	Depth, operators, models	Success, utility, cost	Difficulty calibration
MAS-Orchestra ( Ke et al., 2026 )	Holistic MAS design	Task-level rewards	Function-calling GRPO
Skill-MAS ( Lin et al., 2026 )	Meta skill	Rollout contrasts	Nonparametric reflection

Table 1. Representative learning methods for MAS routing.

Figure 3. Empirical insights into MAS configuration. (a) Cross-domain sensitivity to different resource scaling; (b) Non-monotonic returns across interaction rounds (left) and agent counts (right). (c) Combinatorial synergy between model compositions and collaboration modes.

Figure 4. EvoRouter’s initialization and evolution loop.

	In-Domain						Out-of-Domain
Method	Math	Commonsense	Multimodal	QA	Knowledge	Average	Code	Reasoning	Average
Vanilla LLMs
Qwen3.5-35B-A3B	16.15 $\pm 0.77$	85.00 $\pm 1.02$	29.74 $\pm 0.51$	48.46 $\pm 0.89$	46.67 $\pm 2.86$	46.23 $\pm 0.15$	42.29 $\pm 1.14$	61.50 $\pm 2.78$	52.54 $\pm 0.96$
Qwen3.5-122B-A10B	25.38 $\pm 1.02$	84.62 $\pm 1.33$	27.69 $\pm 1.54$	56.15 $\pm 0.26$	48.21 $\pm 0.51$	50.23 $\pm 0.71$	49.14 $\pm 1.14$	68.00 $\pm 2.18$	59.20 $\pm 1.22$
DeepSeek-V4-Flash	45.77 $\pm 2.31$	70.77 $\pm 0.77$	6.67 $\pm 1.36$	47.95 $\pm 0.77$	31.79 $\pm 2.24$	43.46 $\pm 0.43$	40.57 $\pm 0.99$	57.50 $\pm 3.46$	49.60 $\pm 2.31$
Qwen3.5-397B-A17B	36.15 $\pm 1.02$	88.85 $\pm 0.67$	29.74 $\pm 0.51$	57.95 $\pm 0.92$	55.90 $\pm 3.55$	55.23 $\pm 0.77$	50.86 $\pm 0.57$	75.00 $\pm 1.32$	63.73 $\pm 0.80$

Table 2. Overall accuracy (%) on in-domain and out-of-domain benchmarks. The two Average columns report sample-weighted accuracy. Best and second-best scores in each column are shown in bold and underline , respectively.

Figure 5. Left: The Pareto frontier of the evaluated baselines; Right: Mean total LLM inference cost, including router calls.

Training Stage	In-Domain		Out-of-Domain
Training Stage	Accuracy	Cost	Accuracy	Cost
EvoRouter-4B	68.69 $\pm 0.95$	15280.79 $\pm 770.31$	74.67 $\pm 0.96$	25952.47 $\pm 2673.56$
w/o Insights	64.54 $\pm 1.52$	18245.46 $\pm 977.52$	70.93 $\pm 2.44$	25453.12 $\pm 2075.82$
w/o OPD	63.23 $\pm 1.31$	16467.21 $\pm 852.09$	69.87 $\pm 1.41$	24419.74 $\pm 2017.43$
w/o RL+OPD	59.77 $\pm 0.38$	16789.73 $\pm 691.28$	63.20 $\pm 1.16$	30158.45 $\pm 2555.80$
w/o SFT+RL+OPD	53.23 $\pm 0.23$	25472.25 $\pm 1278.80$	52.80 $\pm 2.40$	19795.09 $\pm 1654.15$

Table 3. Component ablations of EvoRouter-4B.

Figure 6. Training dynamics of EvoRouter-LLaMa-8B.

Figure 7. Sensitivity of EvoRouter-4B to coefficient

\lambda_{C}

Figure 8. Per-question MAS cost distributions of EvoRouter-4B under different cost penalties.

Figure 9. Generalization to unseen routing resources.

Figure 10. A practical case of EvoRouter.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 11 — Figure 11. Example insights distilled during EvoRouter’s evolution.

Figure 12. Training losses.

Model	Act. Params (B)	Model	Act. Params (B)
Qwen3.5-35B-A3B	3	DeepSeek-V4-Flash	13
Qwen3-4B ∗	4	Qwen3.5-397B-A17B	17
Qwen3.8-Flash ‡	6	Kimi-K2.8-Preview ‡	32
Qwen3-8B ∗	8	GLM-5	40
Llama-3.1-8B ∗	8	Qwen3.8-Max ‡	95
Qwen3.5-122B-A10B	10

Appendix

Table 4. Active parameter counts of the models. For dense models ( ∗ ), the total parameter count is reported. ‡ denotes models used only in Section 5.4 .

LLM Agent Routing

LLM agent routing selects models, tools, skills, or collaboration workflows for each task or step to balance capability, cost, latency, and reliability.

See all 99 papers

EvolveRouter: Co-Evolving Routing and Prompt for Multi-Agent Question Answering
Apr 6, 2026Jiatan Huang, Zheyuan Zhang, Kaiwen Shi +2LLM Agent Routing Multi-Agent Collaboration
University of Connecticut, USA · University of Notre Dame, USA
RSI-Router: Evolving Subtask-Level LLM Routing and Skills for Cost-Efficient Agents
Sep 28, 2026Hao Li, Hangfan Zhang, Zhiyao Cui +5Cost-Aware Inference LLM Agent Workflow Optimization
Shanghai Artificial Intelligence Laboratory · Northwestern Polytechnical University
EntroRouter: Learning Efficient Model Routing via Entropy Regulation
Jun 28, 2026Kaiyi Zhang, Xueliang Zhao, Zhuocheng Gong +2Adaptive Model Routing Cost-Aware Inference
Gaoling School of Artificial Intelligence, Renmin University of China · Ant International · The University of Hong Kong +1

Apr 6, 2026J/K move · Enter open · S save

EvolveRouter: Co-Evolving Routing and Prompt for Multi-Agent Question Answering

Jiatan Huang, Zheyuan Zhang, Kaiwen Shi, Yanfang Ye+1

University of Connecticut, USA · University of Notre Dame, USA

LLM Agent Routing Multi-Agent Collaboration Multi-Agent Reasoning Prompt Optimization

Open paper AskPDF arXiv

Large language model agents often exhibit complementary strengths, making routing a promising approach for multi-agent question answering. However, existing routing methods remain limited in two important ways: they typically optimize over a fixed pool of agents without improving the agents themselves, and they often rely on rigid collaboration schemes that cannot adapt the number of participating agents to the query. We propose EvolveRouter, a trainable framework that addresses both limitations by jointly improving agent quality and collaboration structure. First, EvolveRouter couples graph-based query routing with targeted instruction refinement in a closed-loop co-evolution process, allowing router diagnostics to guide agent improvement while refined agents provide cleaner supervision for routing. Second, it introduces an adaptive inference strategy that dynamically determines the effective collaboration size for each query through router-weighted answer agreement. Together, these designs enable more capable and more efficient multi-agent reasoning. Experiments on five question answering benchmarks show that EvolveRouter consistently outperforms SOTA routing baselines in both F1 and exact match, while further analysis confirms the benefits of closed-loop refinement and adaptive collaboration.

Abstract

LLM Agent Routing

Explore similar work

EvolveRouter: Co-Evolving Routing and Prompt for Multi-Agent Question Answering

RSI-Router: Evolving Subtask-Level LLM Routing and Skills for Cost-Efficient Agents

EntroRouter: Learning Efficient Model Routing via Entropy Regulation