LLM Serving

LLM: Large Language Model

Ask about this topic

Momentum

17 papers in the last four weeks, up 143% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 142

All topics
CardsList

  1. Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving

    Apr 28, 2026Shan Yu, Junyi Shu, Yuanjiang Ni +14LLM Inference EfficiencyMulti-Agent LLM Systems

    UCLA · Alibaba Cloud Computing · Alibaba Group +6

  2. GLM-5 Serving Parameter Tuning for OpenClaw: Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads

    Apr 27, 2026Minjie Hua, Ning Wang, Peijun Yang +2LLM Inference EfficiencyLLM Serving

    China Unicom

  3. Latency and Cost of Multi-Agent Intelligent Tutoring at Scale

    Apr 27, 2026Iizalaarab Elhaimeur, Nikos ChrisochoidesLLM Inference EfficiencyMulti-Agent LLM Systems

    Center for Real-Time Computing · Computer Science Department · Old Dominion University +2

  4. RouteNLP: Closed-Loop LLM Routing with Conformal Cascading and Distillation Co-Optimization

    Apr 26, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLLM ServingCost-Aware Inference

    1The University of Hong Kong · 2Stellaris AI Limited

  5. Continuous Semantic Caching for Low-Cost LLM Serving

    Apr 21, 2026Baran Atalar, Xutong Liu, Jinhang Zuo +3LLM ServingCost-Aware Inference

    Carnegie Mellon University Pittsburgh, Pennsylvania, USA · University of Washington Tacoma Tacoma, Washington, USA · City University of Hong Kong China +1

  6. SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving

    Apr 21, 2026Jinda Jia, Jisen Li, Zhongzhu Zhou +8LLM ServingKV Caching

    Together AI

  7. SLO-Guard: Crash-Aware, Budget-Consistent Autotuning for SLO-Constrained LLM Serving

    Apr 19, 2026Christian LysenstøenLLM Inference EfficiencyLLM Serving

    Inland Norway University of Applied Sciences · Visiting Student, University of California, Berkeley

  8. Bit-Flip Vulnerability of Shared KV-Cache Blocks in LLM Serving Systems

    Apr 19, 2026Yuji Yamamoto, Satoshi MatsuuraLLM ServingAdversarial Attacks on LLMs

    Institute of Science Tokyo, Tokyo, Japan

  9. POLAR: Online Learning for LoRA Adapter Caching and Routing in Edge LLM Serving

    Apr 17, 2026Shaoang Li, Jian LiLLM ServingLow-Rank Adaptation

    Stony Brook University

  10. Sparse Prefix Caching for Hybrid and Recurrent LLM Serving

    Apr 17, 2026Mikhail Shirokikh, Sergey NikolenkoLLM ServingKV-Cache Management

  11. ELMoE-3D: Leveraging Intrinsic Elasticity of MoE for Hybrid-Bonding-Enabled Self-Speculative Decoding in On-Premises Serving

    Apr 16, 2026Yuseon Choi, Jingu Lee, Jungjun Oh +5LLM ServingMixture-of-Experts Inference

    KAIST · Samsung Electronics · GIST

  12. PipeLive: Efficient Live In-place Pipeline Parallelism Reconfiguration for Dynamic LLM Serving

    Apr 14, 2026Xu Bai, Muhammed Tawfiqul Islam, Chen Wang +1LLM Inference EfficiencyLLM Serving

    DisNet Lab University of Melbourne Melbourne, VIC, Australia · IBM Research Yorktown Heights, NY, USA

  13. FlexServe: A Fast and Secure LLM Serving System for Mobile Devices with Flexible Resource Isolation

    Mar 10, 2026Yinpeng Wu, Yitong Chen, Lixiang Wang +3LLM ServingOn-Device Language Model Inference

    Institute of Parallel and Distributed Systems, Shanghai Jiao Tong University

  14. MoEless: Efficient MoE LLM Serving with Serverless Experts

    Mar 6, 2026Hanfei Yu, Bei Ouyang, Shwai He +2LLM Inference EfficiencyExpert Load Balancing

    University of Maryland College Park

  15. Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving

    Feb 27, 2026Ferran Agullo, Joan Oliveras, Chen Wang +5LLM Inference EfficiencyLLM Serving

    Barcelona Supercomputing Center (BSC), Barcelona, Spain · Universitat Politècnica de Catalunya - BarcelonaTech (UPC), Barcelona, Spain · IBM Research, New York, USA

  16. Learning to Route and Schedule LLMs from User Retrials via Contextual Queueing Bandits

    Feb 2, 2026Seoungbin Bae, Junyoung Son, Dabeen LeeLLM ServingLLM Routing

    Department of Industrial & Systems Engineering, KAIST · Graduate School of Data Science, KAIST · Department of Mathematical Sciences, Seoul National University

  17. xGR: Efficient Generative Recommendation Serving at Scale

    Dec 12, 2025Qingxiao Sun, Tongxuan Liu, Shen Zhang +13LLM ServingLLM Inference Acceleration

    Beihang University · JD Company · University of Science and Technology Beijing +1

  18. Systematic Exploration of Multi-core Architectures for Efficient LLM Serving using WaferAI-SIM

    Oct 7, 2025Tianhao Zhu, Dahu Feng, Erhu Feng +1AI Accelerator InferenceLLM Serving

    Institute of Parallel and Distributed Systems, Shanghai Jiao Tong University · Department of Precision Instrument, Tsinghua University

  19. Ranking Before Serving: Low-Latency LLM Serving via Pairwise Learning-to-Rank

    Sep 25, 2025Yiheng Tao, Yihe Zhang, Matthew Dearing +4LLM ServingLearning to Rank

    University of Illinois Chicago, Chicago, IL, USA · Argonne National Laboratory, Lemont, IL, USA

  20. VoltanaLLM: Energy-Efficient and SLO-Aware Disaggregated LLM Serving via Adaptive Frequency Control and State-Space Routing

    Sep 5, 2025Jiahuan Yu, Aryan Taneja, Junfeng Lin +1LLM ServingEnergy-Efficient ML

    Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Champaign, USA · Department of Precision Instrument, Tsinghua University, Beijing, China

  21. LLM Serving Optimization with Variable Prefill and Decode Lengths

    Aug 8, 2025Meixuan Wang, Yinyu Ye, Zijie ZhouLLM ServingLLM Inference Scheduling

    Department of Computer Science and Technology, Tsinghua University · Department of Management Science and Engineering, Stanford University & Department of Industrial Engineering and Decision Analytics, HKUST · Department of Industrial Engineering and Decision Analytics, HKUST

  22. OUTLETS: Output-Length Prediction from Speculative Decoding Backbones

    Date pendingWeihuang Wen, Yingying Liu, Yichuan Liu +5LLM ServingLLM Inference Scheduling

    The Chinese University of Hong Kong, Shenzhen · The University of Hong Kong · Huawei Technologies Co., Ltd.