cs.CLNov 7, 2026

MultiTurnRoute: A Platform for Building and Evaluating Multi-Turn LLM Routers

Authors: Jiaqi Xue, Mengxin Zheng, Heng Huang, Qian Lou

Organizations: LoopLeap Lab, University of Central Florida · University of Maryland, College Park

Abstract

In a multi-turn conversation, the choice of language model at one turn can affect which model should answer the next. The selected model's reply shapes the context for the next query, and its cached prompt prefix can reduce the cost of using that model again. We introduce MultiTurnRoute, a platform for building and evaluating multi-turn LLM routers. A shared interface gives built-in and user-defined routers the current query, dialogue history, and an estimate of each model's reusable prefix length. Our routing algorithm, Chain-of-Route (CoR), uses a learned gate over recent and recurrent dialogue history to predict response quality and estimates cost from each model's reusable prefix. MultiTurnRoute Arena ranks routers on recorded conversations and compares session quality and cost in interactive runs where a simulated user responds to each reply. On a benchmark drawn from WildChat and LMSYS-Chat-1M with per-turn quality annotations for five LLMs, CoR reduces the normalized cost required to match the strongest model's quality by up to 20.3% relative to the strongest routing baseline, while matching or improving peak response quality.

Figures & tables

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. MTRouter: Cost-Aware Multi-Turn LLM Routing with History-Model Joint Embeddings

    Apr 26, 2026Yiqun Zhang, Hao Li, Zihan Wang +6Cost-Aware InferenceLLM Routing

  2. SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations

    Sep 10, 2026Yu Wang, Yuchen Li, Rui Kong +11Multi-Turn Dialogue EvaluationLLM Routing

  3. Response-Informed Value Estimation for Cost-Aware Multi-Turn LLM Routing

    Oct 16, 2026Haotian Yan, Jiaqi Liu, Bingyi Jing +1LLM RoutingCost-Aware Inference