LLM Request Scheduling

LLM: Large Language Model

Ask about this topic

Momentum

1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 16

All topics
CardsList

  1. Denoising Surface: Modeling and Predicting Inference Cost for Diffusion LLM Serving

    Sep 30, 2026Haoyu Zheng, Fangcheng Fu, Binhang Yuan +6Cost-Aware InferenceDiffusion Model Serving

    Wuhan University · Shanghai Jiao Tong University · The Hong Kong University of Science and Technology +2

  2. LLM Inference Under Bursty Workload Distribution: Modifying the WAIT Algorithm

    Aug 6, 2026Anjali Gangadhar Katageria, Shobha Rani, Raghu Nandan SenguptaLLM Inference SchedulingOnline Resource Allocation

    Department of Mathematics, Indian Institute of Technology Dharwad, Dharwad, Karnataka, India · Department of Management Sciences, Indian Institute of Technology Kanpur, Kanpur, Uttar Pradesh, India

  3. BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving

    Jul 9, 2026Yuanjie Zhu, Liangwei Yang, Ke Xu +4LLM Inference SchedulingDiffusion Model Serving

    University of Illinois Chicago, USA

  4. When Words Predict Workload

    Jul 6, 2026Anubhab BanerjeeOn-Device Language Model InferenceLLM Inference

    Nokia Germany

  5. Sangam: Efficiently Serving Diffusion LLMs with the AR Stack

    Jul 5, 2026Nitin Kedia, Saurabh Agarwal, Myungjin Lee +1LLM Inference SchedulingLLM Inference Serving

    The University of Texas at Austin Austin, United States · Cisco Research Bellevue, United States

  6. Towards Load-Aware Prefill Deflection for Disaggregated LLM Serving

    Jul 2, 2026Shrikara Arun, Anjaly Parayil, Srikant Bharadwaj +2LLM ServingDisaggregated LLM Serving

    Microsoft

  7. Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice

    Jun 21, 2026Li Kong, Qi Qi, Yinyu Ye +1LLM ServingLLM Inference Scheduling

    Gaoling School of Artificial Intelligence Renmin University of China Beijing, China · Department of Management Science and Engineering Stanford University Stanford, CA 94305-4026 · Department of Industrial Engineering and Decision Analytics HKUST Hongkong, China

  8. Beyond Prediction: Tail-Aware Scheduling for LLM Inference

    Jun 16, 2026Yueying Li, Yuanfan Chen, Jiayang Chen +6LLM ServingInference-Time Optimization

    Cornell University, Computer Science Department, NY, USA · Microsoft Azure System Research, WA, USA · Cornell University, Electrical and Computer Engineering Department, NY, USA +2

  9. OrcaRouter: A Production-Oriented LLM Router with Hybrid Offline-Online Learning

    May 29, 2026Zhenghua Bao, Fengya Tian, Chris Zhang +3Model SelectionLLM Routing

    Continuum AI

  10. Harmonizing Real-Time Constraints and Long-Horizon Reasoning: An Asynchronous Agentic Framework for Dynamic Scheduling

    May 28, 2026Shijie Cao, Yuan Yuan, Jing LiuLanguage Model-Based ControlReal-Time Scheduling

    School of Computer Science and Engineering, Beihang University, Beijing 100191, China · Shenzhen Loop Area Institute, Shenzhen, China · Qingdao Research Institute, Beihang University +3

  11. PRISM: Fast Online LLM Serving via Scheduling-Memory Co-design

    May 9, 2026Xingyu Qu, Tianhao Lin, Yiqi Li +2LLM ServingKV Caching

    School of Computer Science, Wuhan University · Amazon

  12. Joint Optimization of Trajectory Control, Resource Allocation, and Task Offloading for Multi-UAV-Assisted IoV

    May 6, 2026Maoxin Ji, Qiong Wu, Pingyi Fan +4Trajectory OptimizationIntelligent Transportation Systems

    School of Internet of Things Engineering, Jiangnan University, Wuxi 214122, China · School of Information Engineering, Jiangxi Provincial Key Laboratory of Advanced Signal Processing and Intelligent Communications, Nanchang University, Nanchang 330031, China · Department of Electronic Engineering, State Key laboratory of Space Network and Communications, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China +4

  13. HiveMind: OS-Inspired Scheduling for Concurrent LLM Agent Workloads

    Apr 18, 2026Justice Owusu Agyemang, Jerry John Kponyo, Obed Kwasi Somuah +3LLM Inference EfficiencyAI Coding Agents

    Sperix Labs · VIA Cybersecurity Lab, KNUST · Quantum and Assistive Technologies Lab, KNUST

  14. Learning to Route and Schedule LLMs from User Retrials via Contextual Queueing Bandits

    Feb 2, 2026Seoungbin Bae, Junyoung Son, Dabeen LeeLLM ServingLLM Routing

    Department of Industrial & Systems Engineering, KAIST · Graduate School of Data Science, KAIST · Department of Mathematical Sciences, Seoul National University

  15. Ranking Before Serving: Low-Latency LLM Serving via Pairwise Learning-to-Rank

    Sep 25, 2025Yiheng Tao, Yihe Zhang, Matthew Dearing +4LLM ServingLearning to Rank

    University of Illinois Chicago, Chicago, IL, USA · Argonne National Laboratory, Lemont, IL, USA

  16. LLM Serving Optimization with Variable Prefill and Decode Lengths

    Aug 8, 2025Meixuan Wang, Yinyu Ye, Zijie ZhouLLM ServingLLM Inference Scheduling

    Department of Computer Science and Technology, Tsinghua University · Department of Management Science and Engineering, Stanford University & Department of Industrial Engineering and Decision Analytics, HKUST · Department of Industrial Engineering and Decision Analytics, HKUST