cs.AIFeb 3, 2026

When LLM Routers Overpay: The Cost of Strong-Model Preference

Authors: Guannan Lai, Long Chen, Han-Jia Ye

Organizations: School of Artificial Intelligence, Nanjing University, China · National Key Laboratory for Novel Software Technology, Nanjing University, China.

Abstract

LLM routing aims to reduce inference costs by assigning easy queries to less expensive models and reserving stronger models for cases in which they are needed. However, we find that existing routers often violate this principle under loose cost budgets: as the budget increases, they increasingly select the strongest and most expensive model even when less expensive feasible candidates provide comparable or identical utility. We refer to this behavior as strong-model over-selection under loose budgets. Our analysis shows that ambiguity alone does not explain this phenomenon. Substantial overpayment can arise when differently priced models offer equivalent or near-equivalent utility and the router retains a persistent model-level preference for costly models; budget relaxation exposes this preference by making premium candidates feasible. Motivated by this diagnosis, we propose EquiRouter, a decision-aligned router that learns query-dependent model rankings, preserves meaningful utility differences, and explicitly prefers lower-cost models among equivalent candidates through a Cost-Aware Ranking Objective and a lightweight Query--Model Interaction Representation. Experiments across diverse text, multimodal, continuous-utility, and large-model-pool settings show that EquiRouter consistently improves cost efficiency while maintaining competitive peak and whole-curve performance. Code is available at https://github.com/LAMDA-Model-Reuse/EquiRouter.

Figures & tables

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Dynamic LLM Routers are Often Misguided

    Oct 2, 2026Sam Wang, Julia White, Sahibzada Allahyar +3LLM RoutingCost-Aware Inference

  2. LLMRouter: Unified Infrastructure for Developing, Evaluating, and Deploying LLM Routers

    Aug 7, 2026Tao Feng, Fangxu Yu, Haozhen Zhang +9LLM EvaluationLLM Routing

  3. WISERouter: LLM Routing with Workload Budget Constraint

    Jul 26, 2026Yifei Li, Zihui Gao, Laks V. S. LakshmananLLM RoutingContextual Bandits