LLM Routing

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 88% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 224

All topics

VIA-SD: Verification via Intra-Model Routing for Speculative Decoding
Jun 10, 2026Yuchen Xian, Yang He, Yunqiu Xu +1Speculative Decoding LLM Inference Acceleration
ReLER, The State Key Lab of Brain Machine Intelligence, Zhejiang University · College of Artificial Intelligence, Zhejiang University · CFAR, Agency for Science, Technology and Research, Singapore +2
ReCal: Reward Calibration for RL-based LLM Routing
Jun 10, 2026Qihang Yu, Hanwen Tong, Zhengqi Zhang +5Multi-Objective Reinforcement Learning RL for Language Models
Zhejiang University · Ant Group · Shanghai AI Laboratory
Continual LLM Upcycling: A Predictor-Gated Bank-Wise Sparsity Training Recipe for Dense-to-Sparse LLMs
Jun 9, 2026Ruixuan Huang, Jinyuan Shi, Hantao Huang +5Continual Learning for LLMs LLM Compression
1HKUST · 2Moffett AI · 3Independent Contributor
Entropy-Aware Domain-Routed Mixture-of-Experts Speech-LLM Framework: A Case Study of Multi-Domain Child-Adult ASR
Jun 9, 2026Mohan Shi, Kaiyuan Zhang, Zilai Wang +3Mixture-of-Experts Language Models Automatic Speech Recognition
University of California, Los Angeles, USA
BUDDY: BUdget-Driven DYnamic Depth Routing for Adaptive Large Language Model Inference
Jun 8, 2026Yuhua Zhou, Shaoqi Yu, Shichao Weng +4LLM Pruning Cost-Aware Inference
College of Computer Science and Technology, Zhejiang University · Zhejiang Lab · Ant Group +1
Online Pandora's Box for Contextual LLM Cascading
Jun 5, 2026Alexandre Belloni, Yan Chen, Yehua WeiLLM Decision-Making LLM Routing
The Fuqua School of Business, Duke University
WhiFlash: Accelerating Speculative Decoding with Token-Level Cross-Paradigm Routing
Jun 5, 2026Young D. Kwon, Miles Williams, Rui Li +2Speculative Decoding LLM Inference Acceleration
Samsung AI Center, Cambridge, UK
From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routing
Jun 5, 2026Guannan Lai, Haoran Hu, Long Chen +2LLM Evaluation LLM Routing
School of Artificial Intelligence, Nanjing University · National Key Laboratory for Novel Software Technology, Nanjing University · Hong Kong University of Science and Technology +1
When to Think Deeply: Inhibitory Deliberation for LLM Reasoning
Jun 4, 2026Zhixuan He, Yue FengEfficient Language Model Reasoning Inference-Time Compute Allocation
University of Birmingham, United Kingdom
Learning to Route LLMs from Implicit Cost-Performance Preferences via Meta-Learning
Jun 4, 2026Jiahao Zeng, Ming Tang, Ningning DingMeta-Learning Cost-Aware Inference
Hong Kong University of Science and Technology (Guangzhou) · Southern University of Science and Technology
IR3DE: A Linear Router for Large Language Models
Jun 4, 2026Eros Fanì, Oğuzhan ErsoyLLM Inference LLM Routing
Gensyn
Failed Reasoning Traces Tell You What Is Fixable (But Not by Reading Them)
Jun 3, 2026Nizar Islah, Istabrak Abbes, Irina Rish +2Test-Time Scaling LLM Routing
1Mila - Quebec AI Institute · 2Université de Montréal · 3Polytechnique Montréal +1
RIZZ: Routing Interactions to Near Zero-Interference Zones for Continual Adaptation of Black-Box Agents
Jun 2, 2026Sonali Goel, Pranav Vaidhyanathan, Lucas Schorling +2Continual Learning for LLM Agents LLM Routing
Department of Engineering Science University of Oxford
From Prompt to Service: An SLM-Based Agent Orchestration Gateway for AI-Driven Virtual Worlds
Jun 2, 2026Louis Nisiotis, Aimilios HadjiliasiLLM Agent Orchestration Small Language Models
Department of Computing, Engineering and Mathematics UCLan Cyprus
When Model Merging Breaks Routing: Training-Free Calibration for MoE
Jun 2, 2026Canbin Huang, Tianyuan Shi, Xiaojun Quan +3Mixture of Experts LLM Routing
School of Computer Science and Engineering, Sun Yat-sen University, China · Shenzhen Loop Area Institute, China · Meituan, Inc., China +1
RASER: Recoverability-Aware Selective Escalation Router for Multi-Hop Question Answering
Jun 1, 2026Yuyang Li, Zihe Yan, Tobias KäferMulti-Hop QA LLM Inference Efficiency
Institute AIFB, Karlsruhe Institute of Technology, Karlsruhe, Germany · Shanghai Jiao Tong University, Shanghai, China
LayerRoute: Input-Conditioned Adaptive Layer Skipping via LoRA Fine-Tuning for Agentic Language Models
Jun 1, 2026Prateek Kumar SikdarLLM Inference Acceleration LLM Agents
Accenture Bengaluru, India
Lodestar: An Online-Learning LLM Inference Router
May 31, 2026Gangmuk Lim, Wanyu Zhao, Brighten Godfrey +3Inference-Time Optimization LLM Inference Scheduling
UIUC · Bytedance · University of Edinburgh
PR2: Predictive Routing Replay for MoE-Based LLM Reinforcement Learning
May 29, 2026Daize Dong, Junlin Chen, Haolong Jia +9RL for Language Model Reasoning Expert Routing
♣Rutgers University · ♦AMD · ⋆MBZUAI
Eigenvectors of Experts are Training-free Non-collapsing Routers
May 29, 2026Giang Do, Hung Le, Truyen TranParameter-Free Mixture-of-Experts Routing Mixture of Experts
Applied Artificial Intelligence Intiative (A2I2), Deakin University, Victoria, Australia.
UniScale: Adaptive Unified Inference Scaling via Online Joint Optimization of Model Routing and Test-Time Scaling
May 29, 2026Kaiyu Huang, Xingyu Wang, Mingze Kong +6Cost-Aware Inference Inference-Time Scaling
School of Computer Science and Technology, Tongji University · Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong, Shenzhen · School of Data Science, The Chinese University of Hong Kong, Shenzhen +2
OrcaRouter: A Production-Oriented LLM Router with Hybrid Offline-Online Learning
May 29, 2026Zhenghua Bao, Fengya Tian, Chris Zhang +3Model Selection LLM Routing
Continuum AI
Triaging Threats to Specialized Guardrails
May 29, 2026Wenjie Jacky Mo, Xiaofei Wen, Rui Cai +6LLM Safety Benchmarks LLM Guardrails
University of California, Davis
A Dual-Path Architecture for Scaling Compute and Capacity in LLMs
May 28, 2026Markus Frey, Behzad Shomali, Joachim Koehler +1Recurrent Transformers Language Model Scaling
Lamarr Institute · Fraunhofer IAIS · University of Bonn
Leveraging Routing Dynamics in Mixture-of-Experts Models for Efficient Language Adaptation
May 28, 2026Aditi Khandelwal, Marius Mosbach, Verna Dankers +2Multilingual Language Models Continual Learning for LLMs
Mila – Quebec AI Institute & McGill University
DynaGraph: Lightweight Multi-Model Interaction Framework via Dynamic Topological Reconfiguration
May 28, 2026Yanxing Guo, Zihao Zheng, Fangzhou Wu +4Multi-Agent LLM Systems LLM Routing
Peking University · Nanjing University · Beijing Advanced Innovation Center for Integrated Circuits +2
Rethinking Stepwise Model Routing: A Cost-Efficient Table Reasoning Perspective
May 28, 2026Shenghao Ye, Yuxiang Wang, Yu Guo +3Cost-Aware Inference LLM Uncertainty Estimation
University of Science and Technology of China · The University of Melbourne · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Rubric-Guided Process Reward for Stepwise Model Routing
May 28, 2026Shenghao Ye, Yu Guo, Zhengheng Li +2Process Reward Models RL for Language Model Reasoning
University of Science and Technology of China · Southeast University
The Routing Plateau: Understanding the Accuracy Limits of LLM Routers
May 27, 2026Yifan Lu, Qiyue Zhang, Shenrun Zhang +4LLM Evaluation Adaptive Model Routing
Rice University · Amazon
HRBench: Benchmarking and Understanding Thinking-Mode Switch Strategies in Hybrid-Reasoning LLMs
May 27, 2026Yansong Ning, Mianpeng Liu, Jingwen Ye +2LLM Evaluation Adaptive Inference
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou) · AIPD, Tencent

Jun 10, 2026, cs.CLJ/K move · Enter open · S save

VIA-SD: Verification via Intra-Model Routing for Speculative Decoding

Yuchen Xian, Yang He, Yunqiu Xu, Yi Yang

ReLER, The State Key Lab of Brain Machine Intelligence, Zhejiang University · College of Artificial Intelligence, Zhejiang University · CFAR, Agency for Science, Technology and Research, Singapore+2

Speculative Decoding LLM Inference Acceleration LLM Routing

Open paper AskPDF arXiv

Speculative decoding (SD) addresses the high inference costs of LLMs by having lightweight drafters generate candidates for large verifiers to validate in parallel. Existing draft-verify methods use binary decisions: accept or fully recompute. Yet we find that many rejected tokens can be verified correctly by a slim submodel derived from the full verifier via intra-model routing, instead of the full verifier. This motivates our slim-verifier to handle tokens requiring moderate verification resources, reducing expensive large-model calls. We propose Verification via Intra-Model Routing for Speculative Decoding (VIA-SD), a multi-tier framework using a routed slim-verifier. Draft tokens are processed hierarchically: direct acceptance for high-confidence cases, slim-verifier regeneration for medium-confidence cases, and full-model verification for uncertain cases. Across four representative tasks and multiple model families, VIA-SD reduces rejection rates by 0.10-0.22 and delivers 10-20% speedups over strong SD baselines, while achieving 2.5-3x acceleration over non-drafting decoding. Moreover, VIA-SD is compatible with existing SD frameworks without modifying their training procedures. Our results suggest multi-tier SD as a general paradigm for scalable and efficient LLM inference. Project page: https://zju-xyc.github.io/VIA-SD-Project-Page/

Momentum

Related topics

Latest papers 224