LLM Routing

LLM: Large Language Model

Overview

Large language model routing studies how to assign a query, workflow step, or token to an appropriate model, expert, tool, retrieval path, or amount of computation. Common approaches use query features, confidence or uncertainty, hidden-state signals, and response disagreement to choose a capable low-cost option or escalate when needed, using classifiers, contextual bandits, reinforcement learning, and calibrated cascades. Systems work also schedules heterogeneous models and GPUs around budgets, latency, energy, cache reuse, privacy, and service guarantees, with evaluations spanning reasoning, coding, question answering, retrieval-augmented generation, agents, and production workloads. A central tension is that oracle model choices can show substantial gains, while deployable routers depend on reliable quality estimates and supervision and can falter under distribution shifts, weak model specialization, noisy evaluation, or routing overhead.

Latest papers

CardsList

383 papers30 papers in the last four weeks, down 52% on the four weeks before. 0.2% of all new papers.

  1. ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference

    Feb 27, 2026Siyuan Ma, Bo Gao, Xiaojun Jia +6Efficient Language Model ReasoningLLM Routing

  2. Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference

    Feb 25, 2026Bo-Wei Chen, Chung-Chi Chen, An-Zi YenCost-Aware InferenceLLM Routing

  3. Sustainable LLM Inference using Context-Aware Model Switching

    Feb 25, 2026Yuvarani, Akashdeep Singh, Zahra Fathanah +3LLM Inference EfficiencyLLM Inference

  4. Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey

    Feb 23, 2026Yasmin Moslem, John D. KelleherLLM RoutingEfficient Language Model Inference

  5. RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models

    Feb 20, 2026Sai Hao, Hao Zeng, Hongxin Wei +1Set-Valued PredictionLLM Routing

  6. Multi-Agentic AI for Fairness-Aware and Accelerated Multi-modal Large Model Inference in Real-world Mobile Edge Networks

    Feb 6, 2026Haiyuan Li, Hari Madhukumar, Shuangyi Yan +2Efficient Multimodal InferenceLLM Inference Scheduling

  7. Disentangling Intrinsic Importance from Emergent Structure in Multi-Expert Orchestration

    Feb 4, 2026Sudipto Ghosh, Sujoy Nath, Sunny Manchanda +1Multi-Agent OrchestrationLLM Routing

  8. When LLM Routers Overpay: The Cost of Strong-Model Preference

    Feb 3, 2026Guannan Lai, Long Chen, Han-Jia YeLLM RoutingAdaptive Model Routing

  9. Learning to Route and Schedule LLMs from User Retrials via Contextual Queueing Bandits

    Feb 2, 2026Seoungbin Bae, Junyoung Son, Dabeen LeeLLM ServingLLM Routing

  10. Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations

    Jan 29, 2026Baris Askin, Shivam Patel, Anupam Nayak +4LLM RoutingFederated Learning

  11. HAPS: Hierarchical LLM Routing with Joint Architecture and Parameter Search

    Jan 9, 2026Zihang Tian, Rui Li, Jingsen Zhang +3LLM RoutingNeural Architecture Search

  12. Poodle: Seamlessly Scaling Down Large Language Models with Just-in-Time Model Replacement

    Dec 5, 2025Nils Strassenburg, Boris Glavic, Tilmann RablCost-Aware InferenceSmall Language Models

  13. GMTRouter: Personalized LLM Router over Multi-turn User Interactions

    Oct 29, 2025Yihang Sun, Encheng Xie, Tao Feng +1LLM PersonalizationLLM Routing

  14. VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing

    Sep 5, 2025Jiahuan Yu, Aryan Taneja, Junfeng Lin +1LLM ServingEnergy-Efficient ML

  15. PrismServe: Efficient LLM Serving on Heterogeneous Accelerator Clusters

    Aug 9, 2025Jinhao Zhang, Zicheng Yan, Boyang Zhang +5LLM ServingLLM Inference Acceleration

  16. FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Data

    Jul 14, 2025Tao Feng, Haozhen Zhang, Zijie Lei +5LLM EvaluationLLM Routing

  17. MiCRo: Mixture Modeling and Context-aware Routing for Personalized Preference Learning

    May 30, 2025Jingyan Shen, Jiarui Yao, Rui Yang +5Pairwise Preference LearningReward Modeling

  18. A Unified Approach to Routing and Cascading for LLMs

    Oct 14, 2024Jasper Dekoninck, Maximilian Baader, Martin VechevLLM RoutingLanguage Model Cascading