LLM Routing

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 88% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 224

All topics

ModelLens: Finding the Best for Your Task from Myriads of Models
May 8, 2026Rui Cai, Weijie Jacky Mo, Xiaofei Wen +5Model Selection LLM Routing
University of California, Davis · Arizona State University · Morgan Stanley
Towards Distributed Inference of LLMs on a P2P Network
May 7, 2026Shabari S Nair, Krishanu SainiLLM Inference KV Caching
Department of Computer Science, The University of Texas at Austin.
Is Escalation Worth It? On the Depth of LLM Cascades
May 7, 2026Dylan BouchardCost-Aware Inference LLM Routing
A Regime Theory of Controller Class Selection for LLM Action Decisions
May 7, 2026Zhaoyang Jiang, Zhizhong Fu, Yunsoo Kim +4Selective Prediction Language Model-Based Control
University of Glasgow · UESTC, Chengdu, China · University College London
Policy-Guided Stepwise Model Routing for Cost-Effective Reasoning
May 7, 2026Wenwen Si, Insup Lee, Osbert BastaniCost-Aware Inference Efficient Language Model Reasoning
Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 19104, USA
Adaptive Consensus in LLM Ensembles via Sequential Evidence Accumulation: Automatic Budget Identification and Calibrated Commit Signals
May 5, 2026Roberto E. MedinaLLM Inference Efficiency Multi-Agent Consensus
Independent Researcher
Complexity Horizons of Compressed Models in Analog Circuit Analysis
May 4, 2026Pacome Simon MbonimpaLLM Evaluation LLM Compression
Carnegie Mellon University-Africa
Zero-Shot Confidence Estimation for Small LLMs: When Supervised Baselines Aren't Worth Training
May 4, 2026Luong N. NguyenConfidence Estimation in Language Models Language Model Self-Assessment
Model Routing as a Trust Problem: Route Receipts for Adaptive AI Systems
May 3, 2026Vincent SchmalbachAI Accountability Adaptive Model Routing
Independent Researcher
Rethinking LLM Ensembling from the Perspective of Mixture Models
May 1, 2026Jiale Fu, Yuchu Jiang, Peijun Wu +3LLM Inference Acceleration LLM Routing
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education · Southeast University · Qiyuan Tech +2
RouteProfile: Graph-Based Profiling for Cold-Start LLM Routing
Apr 30, 2026Jingjun Xu, Hongji Pu, Tao Feng +3Adaptive Model Routing LLM Routing
University of Illinois Urbana-Champaign · Nanyang Technological University
Optimized Deferral for Imbalanced Settings
Apr 30, 2026Corinna Cortes, Anqi Mao, Mehryar Mohri +1Cost-Sensitive Learning Learning to Defer
Google Research, New York, NY · Courant Institute of Mathematical Sciences, New York, NY
When to Vote, When to Rewrite: Disagreement-Guided Strategy Routing for Test-Time Scaling
Apr 29, 2026Zhimin Lin, Yixin Ji, Jinpeng Li +5Test-Time Scaling LLM Routing
School of Computer Science and Technology, Soochow University · Department of Foundation Model, 2012 Labs, Huawei · 3Harbin Institute of Technology, Shenzhen (HITSZ)
Learning to Route Queries to Heads for Attention-based Re-ranking with Large Language Models
Apr 27, 2026Yuxing Tian, Fengran Mo, Zhiqi Huang +2Attention Head Analysis Learning to Rank
Université de Montréal · Capital One · McGill University & MILA
GraphPlanner: Graph Memory-Augmented Agentic Routing for Multi-Agent LLMs
Apr 26, 2026Tao Feng, Haozhen Zhang, Zijie Lei +2Multi-Agent LLM Systems LLM Agent Orchestration
Department of Computer Science University of Illinois Urbana-Champaign Urbana, IL, USA
RouteNLP: Closed-Loop LLM Routing with Conformal Cascading and Distillation Co-Optimization
Apr 26, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLLM Serving Cost-Aware Inference
1The University of Hong Kong · 2Stellaris AI Limited
MTRouter: Cost-Aware Multi-Turn LLM Routing with History-Model Joint Embeddings
Apr 26, 2026Yiqun Zhang, Hao Li, Zihan Wang +6Cost-Aware Inference LLM Routing
School of Computer Science and Engineering, Northeastern University Shenyang 110819, China · Shanghai Artificial Intelligence Laboratory
Scaling Multi-Node Mixture-of-Experts Inference Using Expert Activation Patterns
Apr 25, 2026Abhimanyu Bambhaniya, Geonhwa Jeong, Jason Park +6Expert Load Balancing Mixture-of-Experts Inference
1Georgia Institute of Technology, USA · 2Meta Platforms, Inc.
Mixture of Heterogeneous Grouped Experts for Language Modeling
Apr 25, 2026Zhicheng Ma, Xiang Liu, Zhaoxiang Liu +5Expert Load Balancing Mixture-of-Experts Language Models
1Data Science & Artificial Intelligence Research Institute, China Unicom · 2Unicom Data Intelligence, China Unicom
RouteLMT: Learned Sample Routing for Hybrid LLM Translation Deployment
Apr 24, 2026Yingfeng Luo, Hongyu Liu, Dingyang Lin +6LLM Routing Machine Translation
School of Computer Science and Engineering, Northeastern University, Shenyang, China · NiuTrans Research, Shenyang, China
EverydayGPT: Confidence-Gated Routing for Efficient and Safe Hybrid GPT-RAG Conversational QA
Apr 24, 2026Jaspreet Singh NahalLLM Inference Efficiency Retrieval-Augmented Generation
B.Tech Computer Science Engineering (Artificial Intelligence) Dr. A.P.J. Abdul Kalam Technical University, Delhi, India
State Transfer Reveals Reuse in Controlled Routing
Apr 20, 2026Yanzhen Lu, Zhicheng Qian, Muchen Jiang +1Prompt Learning Transfer Learning
Shanghai Qizhi Institute · Southeast University · Nanjing Whale Cloud +1
Polysemantic Experts, Monosemantic Paths: Routing as Control in MoEs
Apr 20, 2026Charles Ye, Bo Yuan, Lee SharkeyLLM Interpretability Neural Network Interpretability
Independent · Georgia Institute of Technology · Goodfire
ABLE: Representing and Mapping LLMs via Attribution-Based Large-model Embedding
Apr 20, 2026Zirui Wang, Yusen Hou, Shaofeng Liang +4LLM Auditing Language Model Fingerprinting
The Hong Kong University of Science and Technology (Guangzhou) · Deep Interdisciplinary Intelligence Lab (DI2Lab)
RankGuide: Tensor-Rank-Guided Routing and Steering for Efficient Reasoning
Apr 17, 2026Jiayi Tian, Yupeng Su, Ryan Solgi +2LLM Routing Efficient Language Model Inference
University of California, Santa Barbara · Intel Labs
POLAR: Online Learning for LoRA Adapter Caching and Routing in Edge LLM Serving
Apr 17, 2026Shaoang Li, Jian LiLLM Serving Low-Rank Adaptation
Stony Brook University
Privacy-Preserving LLMs Routing
Apr 17, 2026Xidong Wu, Yukuan Zhang, Yuqiong Ji +3Secure Multi-Party Computation Privacy-Preserving ML
Department of Electrical and Computer Engineering, University of Pittsburgh · Department of Computer Science, University of Central Florida · Department of Computer and Information Science, University of Pennsylvania +2
Route to Rome Attack: Directing LLM Routers to Expensive Models via Adversarial Suffix Optimization
Apr 16, 2026Haochun Tang, Yuliang Yan, Jiahua Lu +2Adversarial Prompt Generation Cost-Aware Inference
1Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University · 2The Hong Kong University of Science and Technology (Guangzhou)
Pangu-ACE: Adaptive Cascaded Experts for Educational Response Generation on EduBench
Apr 16, 2026Dinghao Li, Wenlong Zhou, Zhimin Chen +5Generative AI in Education LLM Routing
TRACER: Trace-Based Adaptive Cost-Efficient Routing for LLM Classification
Apr 16, 2026Adam RidaNeural Surrogate Modeling LLM Inference Efficiency
Independent Researcher.

May 8, 2026, cs.LGJ/K move · Enter open · S save

ModelLens: Finding the Best for Your Task from Myriads of Models

Rui Cai, Weijie Jacky Mo, Xiaofei Wen, Qiyao Ma+4

University of California, Davis · Arizona State University · Morgan Stanley

Model Selection LLM Routing

Open paper AskPDF arXiv

The open-source model ecosystem now contains hundreds of thousands of pretrained models, yet picking the best model for a new dataset is increasingly infeasible: new models and unbenchmarked datasets emerge continuously, leaving practitioners with no prior records on either side. Existing approaches handle only fragments of this in-the-wild setting: AutoML and transferability estimation select models from small predefined pools or require expensive per-model forward passes on the target dataset, while model routing presupposes a given candidate pool. We introduce ModelLens, a unified framework for model recommendation in the wild. Our key insight is that public leaderboard interactions, though scattered and noisy, collectively trace out an implicit atlas of model capabilities across heterogeneous evaluation settings, a signal rich enough to learn from directly. By learning a performance-aware latent space over model--dataset--metric tuples, ModelLens ranks unseen models on unseen datasets without running candidates on the target dataset. On a new benchmark of 1.62M evaluation records spanning 47K models and 9.6K datasets, ModelLens surpasses baselines that either rely on metadata alone or require running each candidate on the target dataset. Its recommended Top-K pools further improve multiple representative routing methods by up to 81% across diverse QA benchmarks. Case studies on recently released benchmarks further confirm generalization to both text and vision-language tasks.

Momentum

Related topics

Latest papers 224