LLM Inference Efficiency

LLM: Large Language Model

Ask about this topic

Momentum

47 papers in the last four weeks, up 262% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 234

All topics
CardsList

  1. Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation

    May 8, 2026Joon Ha Kim, Geon-Woo Kim, Anoop Rachakonda +1LLM Inference EfficiencyLLM Inference

    University of Texas at Austin

  2. Learning Agent Routing From Early Experience

    May 8, 2026Yimin Wang, Jiahao Qiu, Xuan Qi +6LLM Inference EfficiencyLLM Agent Routing

    University of Michigan · Shanghai Jiao Tong University · AI Lab, Princeton University +3

  3. Efficient Serving for Dynamic Agent Workflows with Prediction-based KV-Cache Management

    May 7, 2026Haoyu Zheng, Fangcheng Fu, Jia Wu +6LLM Inference EfficiencyKV Caching

    Wuhan University · Shanghai Jiao Tong University · Macquarie University +2

  4. Towards Generation-Efficient Uncertainty Estimation in Large Language Models

    May 7, 2026Mingcheng Zhu, Yu Liu, Tingting ZhuLLM Inference EfficiencyUncertainty Quantification

    University of Oxford

  5. One Pool, Two Caches: Adaptive HBM Partitioning for Accelerating Generative Recommender Serving

    May 6, 2026Wenjun Yu, Shuguang Han, Amelie Chi ZhouLLM Inference EfficiencyGPU Acceleration

    Hong Kong Baptist University · Hong Kong, China · Alibaba Inc +1

  6. Adaptive Consensus in LLM Ensembles via Sequential Evidence Accumulation: Automatic Budget Identification and Calibrated Commit Signals

    May 5, 2026Roberto E. MedinaLLM Inference EfficiencyMulti-Agent Consensus

    Independent Researcher

  7. Tile-Level Activation Overlap for Efficient LLM Inference

    May 5, 2026Abhinav Jangda, Tyler Sorensen, Sebastian Burckhardt +3LLM Inference EfficiencyGPU Kernel Optimization

    Microsoft Research · Microsoft

  8. Human-Less LLM Serving: Quantifying the Human Tax on Throughput

    May 3, 2026Jianhui Lian, Li Chen, Dan Li +1LLM Inference EfficiencyLLM Serving

    Tsinghua SIGS · Beijing Zhongguancun Laboratory · Tsinghua University

  9. AgentStop: Terminating Local AI Agents Early to Save Energy in Consumer Devices

    May 1, 2026Dzung Pham, Kleomenis Katevas, Ali Shahin Shamsabadi +1LLM Inference EfficiencyOn-Device Language Model Inference

    University of Massachusetts Amherst Amherst, Massachusetts, USA · Brave Software London, UK · Brave Software, Imperial College London London, UK

  10. SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters

    May 1, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuLLM Inference EfficiencyLLM Agent Orchestration

  11. Silicon Showdown: Performance, Efficiency, and Ecosystem Barriers in Consumer-Grade LLM Inference

    May 1, 2026Abdurrahman Javat, Allan KazakovLLM Inference EfficiencyLLM Quantization

    Bahcesehir University, Istanbul, Türkiye

  12. Agent Capsules: Quality-Gated Granularity Control for Multi-Agent LLM Pipelines

    May 1, 2026Aninda RayLLM Inference EfficiencyMulti-Agent LLM Systems

  13. Context Recycling for Long-Horizon LLM Inference

    May 1, 2026Derek ThomasLLM Inference EfficiencyLong-Context Language Model Inference

    Independent Researcher · contextforge

  14. LLM-Guided Runtime Parameter Optimization for Energy-Efficient Model Inference

    Apr 29, 2026Katelyn Crumpacker, Dimitrios NikolopoulosLLM Inference EfficiencyEnergy-Efficient ML

    Virginia Polytechnic and State University

  15. Evergreen: Efficient Claim Verification for Semantic Aggregates

    Apr 28, 2026Alexander W. Lee, Benjamin Han, Shayak Sen +3LLM Inference EfficiencyClaim Verification

    Brown University · Snowflake Inc.

  16. Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving

    Apr 28, 2026Shan Yu, Junyi Shu, Yuanjiang Ni +14LLM Inference EfficiencyMulti-Agent LLM Systems

    UCLA · Alibaba Cloud Computing · Alibaba Group +6

  17. PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference

    Apr 27, 2026Ishan Patel, Ishan JoshiLLM Inference EfficiencyMulti-Agent LLM Systems

    Independent Researcher

  18. GLM-5 Serving Parameter Tuning for OpenClaw: Single-Deployment MaaS Inference Optimization for Long-Context Agent Workloads

    Apr 27, 2026Minjie Hua, Ning Wang, Peijun Yang +2LLM Inference EfficiencyLLM Serving

    China Unicom

  19. Latency and Cost of Multi-Agent Intelligent Tutoring at Scale

    Apr 27, 2026Iizalaarab Elhaimeur, Nikos ChrisochoidesLLM Inference EfficiencyMulti-Agent LLM Systems

    Center for Real-Time Computing · Computer Science Department · Old Dominion University +2

  20. Scalable LLM-based Coding of Dialogue in Healthcare Simulation: Balancing Coding Performance, Processing Time, and Environmental Impact

    Apr 25, 2026Kiyoshige Garces, Gloria Milena Fernandez-Nieto, Linxuan Zhao +4LLM Inference EfficiencyLLM Prompting

    RMIT University · Monash University · Adelaide University +2

  21. PExA: Parallel Exploration Agent for Complex Text-to-SQL

    Apr 24, 2026Tanmay Parekh, Ella Hofmann-Coyle, Shuyi Wang +3LLM Inference EfficiencyText-to-SQL

    University of California Los Angeles · Bloomberg

  22. EverydayGPT: Confidence-Gated Routing for Efficient and Safe Hybrid GPT-RAG Conversational QA

    Apr 24, 2026Jaspreet Singh NahalLLM Inference EfficiencyRetrieval-Augmented Generation

    B.Tech Computer Science Engineering (Artificial Intelligence) Dr. A.P.J. Abdul Kalam Technical University, Delhi, India

  23. Sub-Token Routing for KV Cache Compression

    Apr 23, 2026Wei Jiang, Wei WangLLM Inference EfficiencyKV Caching

    Futurewei Technologies San Jose, CA 95131, USA

  24. TRACES: Tagging Reasoning Steps for Adaptive Cost-Efficient Early-Stopping

    Apr 22, 2026Yannis Belkhiter, Seshu Tirupathi, Giulio Zizzo +1LLM Inference EfficiencyEarly Stopping

    IBM Research Europe · Trinity College Dublin · Dublin, Ireland +1

  25. RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory

    Apr 22, 2026Fei Zuo, Zikang Zhou, Hao Cong +2LLM Inference EfficiencyMixed-Precision Quantization

    1BA TechWorks (BMW Group) · 2National University of Singapore · 3Tsinghua University

  26. Are Large Language Models Economically Viable for Industry Deployment?

    Apr 21, 2026Abdullah Mohammad, Sushant Kumar Ray, Pushkar Arora +5LLM Inference EfficiencyLLM Evaluation

    DSEU-Okhla, New Delhi, India · University of Delhi, New Delhi, India · 6, 8Macquarie University, Sydney, Australia +1

  27. How Much Thinking is Enough? Quantifying and Understanding Redundancy in LLM Reasoning

    Apr 21, 2026Zhiyuan Zhai, Xinkai You, Wenjing Yan +1LLM Inference EfficiencyRL for Language Model Reasoning

    Fudan University · The Chinese University of Hong Kong

  28. Learning to Seek Help: Dynamic Collaboration Between Small and Large Language Models

    Apr 20, 2026Hang Zeng, Xiangyu Liu, Yong Hu +5LLM Inference EfficiencySmall Language Models

    Shanghai Jiao Tong University, Shanghai, China · WeChat Tencent, Beijing, China · State University of New York at Buffalo, New York, United States.

  29. SLO-Guard: Crash-Aware, Budget-Consistent Autotuning for SLO-Constrained LLM Serving

    Apr 19, 2026Christian LysenstøenLLM Inference EfficiencyLLM Serving

    Inland Norway University of Applied Sciences · Visiting Student, University of California, Berkeley

  30. ONTO: A Token-Efficient Columnar Notation for LLM Input Optimization

    Apr 19, 2026Harshavardhanan DeekeswarLLM Inference EfficiencyToken Efficiency

    Independent Researcher · Chennai, India

  31. Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning

    Apr 19, 2026Raman Saparkhan, Majd Hawasly, Md Rizwan Parvez +1LLM Inference EfficiencySelf-Consistency Decoding

    Carnegie Mellon University Qatar · Qatar Computing Research Institute

  32. Hive: A Multi-Agent Infrastructure for Algorithm- and Task-Level Scaling

    Apr 19, 2026Zizhang Luo, Yuhao Luo, Youwei Xiao +3LLM Inference EfficiencyMulti-Agent LLM Systems

    Peking University Beijing, China

  33. HiveMind: OS-Inspired Scheduling for Concurrent LLM Agent Workloads

    Apr 18, 2026Justice Owusu Agyemang, Jerry John Kponyo, Obed Kwasi Somuah +3LLM Inference EfficiencyAI Coding Agents

    Sperix Labs · VIA Cybersecurity Lab, KNUST · Quantum and Assistive Technologies Lab, KNUST

  34. When Agents Go Quiet: Output Generation Capacity and Format-Cost Separation for LLM Document Synthesis

    Apr 17, 2026Justice Owusu Agyemang, Michael Agyare, Miriam Kobbinah +2LLM Inference EfficiencyLong-Form Document Generation

    Sperix Labs · GCTU, Ghana · NCA, Ghana +1

  35. Atropos: Improving Cost-Benefit Trade-off of LLM-based Agents under Self-Consistency with Early Termination and Model Hotswap

    Apr 16, 2026Naryeong Kim, Shin YooLLM Inference Efficiency

    KAIST · Daejeon, Republic of Korea

  36. TRACER: Trace-Based Adaptive Cost-Efficient Routing for LLM Classification

    Apr 16, 2026Adam RidaNeural Surrogate ModelingLLM Inference Efficiency

    Independent Researcher.

  37. PAC-CF: Calibrating Irreversible Frontier Pruning in LLM-Guided Search

    Apr 15, 2026Tianhao Qian, Jiayu Chen, Zhenyu Sun +1LLM Inference EfficiencyInference-Time Search

  38. PipeLive: Efficient Live In-place Pipeline Parallelism Reconfiguration for Dynamic LLM Serving

    Apr 14, 2026Xu Bai, Muhammed Tawfiqul Islam, Chen Wang +1LLM Inference EfficiencyLLM Serving

    DisNet Lab University of Melbourne Melbourne, VIC, Australia · IBM Research Yorktown Heights, NY, USA

  39. Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning

    Apr 2, 2026Rafael Pardinas, Ehsan Kamalloo, David Vazquez +1LLM Inference EfficiencyRL for Language Model Reasoning

    ServiceNow AI Research

  40. When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models

    Mar 27, 2026Juan Gabriel Kostelec, Qinghai GuoLLM Inference EfficiencyLLM Evaluation

    Huawei Switzerland · ACS Lab, Huawei Technologies

  41. Clinical Note Bloat Reduction for Efficient LLM Use

    Mar 21, 2026Jordan L. Cahoon, Chloe Stanwyck, Asad Aali +6LLM Inference EfficiencyElectronic Health Records

    Department of Biomedical Data Science, Stanford University, Stanford, CA · Department of Pathology, Stanford University, Stanford, CA · Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University, Stanford, CA +5

  42. MoEless: Efficient MoE LLM Serving with Serverless Experts

    Mar 6, 2026Hanfei Yu, Bei Ouyang, Shwai He +2LLM Inference EfficiencyExpert Load Balancing

    University of Maryland College Park

  43. Same Pieces, Different Servers: A Tetris Benchmark for AI Agents as Served

    Mar 2, 2026Haochuan WangLLM Inference EfficiencyLLM Agent Evaluation

    Massachusetts Institute of Technology

  44. Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving

    Feb 27, 2026Ferran Agullo, Joan Oliveras, Chen Wang +5LLM Inference EfficiencyLLM Serving

    Barcelona Supercomputing Center (BSC), Barcelona, Spain · Universitat Politècnica de Catalunya - BarcelonaTech (UPC), Barcelona, Spain · IBM Research, New York, USA

  45. Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens

    Feb 13, 2026Wei-Lin Chen, Liqian Peng, Tian Tan +5LLM Inference EfficiencyInference-Time Scaling

    University of Virginia · Google

  46. More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)

    Jan 29, 2026Sagi Meir, Tommer D. Keidar, Noam Levi +2LLM Inference EfficiencyLLM Evaluation

    School of Chemistry, Tel Aviv University, Tel Aviv 6997801, Israel · The Center for Physics and Chemistry of Living Systems, Tel Aviv University, Tel Aviv 6997801, Israel · School of Physics and Astronomy, Tel Aviv University, Tel Aviv 6997801, Israel +1

  47. Nalar: Workflow-Aware Management of Agentic Applications

    Jan 8, 2026Saurabh Agarwal, Marco Laju, Donghyun Son +4LLM Inference EfficiencyLLM Agent Orchestration

    UT-Austin · Cisco-Research

  48. Step-Tagging: Toward controlling the generation of Language Reasoning Models through step monitoring

    Dec 16, 2025Yannis Belkhiter, Seshu Tirupathi, Giulio Zizzo +1LLM Inference EfficiencyEfficient Language Model Reasoning

    1IBM Research Europe, Dublin, Ireland · 2Trinity College Dublin, Ireland · 3ADAPT Research Centre, Dublin, Ireland

  49. Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

    Nov 11, 2025Jon Saad-Falcon, Avanika Narayan, Hakki Orhun Akengin +12LLM Inference EfficiencyAI Accelerator Inference

    Stanford University · Together AI

  50. Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm

    Sep 28, 2025Kaisen Yang, Tinghe Zhang, Rushi Shah +4LLM Inference EfficiencyLLM Planning

    1Qizhi Institute · 2Dept. of Computer Science, Tsinghua University · College of AI, Tsinghua University +4

  51. EquivPruner: Boosting Efficiency and Quality in LLM-Based Search via Action Pruning

    May 22, 2025Jiawei Liu, Qisi Chen, Jianshu Zhang +2LLM Inference EfficiencySemantic Textual Similarity

    School of Computer Science and Technology, University of Science and Technology of China · Department of Data Science, City University of Hong Kong · iFLYTEK Research

  52. Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models

    Mar 31, 2025Rui Wang, Hongru Wang, Boyang Xue +9LLM Inference EfficiencyCost-Aware Inference

    University of Edinburgh, Edinburgh, UK · University of Macau, Macau SAR, China · University of Illinois Urbana-Champaign, USA

  53. Saving GPU Hours in LLM Inference System Development and Online Workloads with Simulation and DBMS-Inspired Cache Replacement Policies

    Nov 12, 2024Kyoungmin Kim, Jiacheng Li, Kijae Hong +3LLM Inference EfficiencyLLM Inference

    EPFL, Switzerland · NUS, Singapore · POSTECH, South Korea

  54. UltraQuant: 4-bit KV Caching for Context-Heavy Agents

    Date pendingInesh Chakrabarti, David Limpus, Aditi Ghai Rana +4LLM Inference EfficiencyGPU Kernel Optimization

    Advanced Micro Devices · University of California, Los Angeles · Purdue University