LLM Inference Efficiency

LLM: Large Language Model

Ask about this topic

Momentum

47 papers in the last four weeks, up 262% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 234

All topics
CardsList

  1. Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization

    Jul 9, 2026Jiantong Jiang, Peiyu Yang, Rui Zhang +1LLM Inference EfficiencyLLM Serving

    School of Computing and Information Systems, The University of Melbourne · School of Computer Science and Technology, Huazhong University of Science and Technology (www.ruizhang.info)

  2. Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems

    Jul 9, 2026Kalle Kujanpää, Ning Liu, Shahnawaz Alam +4LLM Inference EfficiencyLLM Agent Self-Improvement

    Amazon, Fulfillment Technologies & Robotics

  3. The Harness Effect: How Orchestration Design Sets the Token Economics of Enterprise Agentic AI

    Jul 8, 2026Muayad Sayed Ali, Aliaksandra Novik, Anji Boddupally +28LLM Inference EfficiencyLLM Agent Orchestration

    Writer, Inc.

  4. Doomed from the Start: Early Abort of LLM Agent Episodes via a Recall-Controlled Probe Cascade

    Jul 7, 2026Kai Ruan, Zihe Huang, Ziqi Zhou +4LLM Inference EfficiencyEarly Stopping

    Gaoling School of Artificial Intelligence, Renmin University of China · Institute of Computing Technology, Chinese Academy of Sciences · Duke University +3

  5. Think Before You Grid-Search: Floor-First Triage for LLM Serving

    Jul 7, 2026Yihua LiuLLM Inference EfficiencyLLM Serving

    Taikang Insurance Group

  6. Inject or Navigate? Token-Efficient Retrieval for LLM Analysis of Transactional Legal Documents

    Jul 7, 2026Mahmoud Hany, Mourad ElSheraey, Mahmoud Said +1LLM Inference EfficiencyLegal IR

    Syntheia Pty Ltd

  7. Is Your NPU Ready for LLMs? Dissecting the Hidden Efficiency Bottlenecks in Mobile LLM Inference

    Jul 6, 2026Guanyu Cai, Ruiming Tian, Lang Yang +4LLM Inference EfficiencyAI Accelerator Inference

    Tsinghua University · Beijing Jiaotong University

  8. Risk-Constrained Freshness-Aware Semantic Caching for Open-Web Retrieval-Augmented LLMs

    Jul 5, 2026Muhammad Mansoor, Tahir Ahmad, Yeo-Chan YoonLLM Inference EfficiencyRetrieval-Augmented Generation

    Jeju National University, Jeju, Republic of Korea

  9. Knowing When to Stop: Predicting Execution-Consistency Convergence in Text-to-SQL

    Jul 4, 2026Yaron Anavi, Mor Aisenberg, Nadav Nesher +2LLM Inference EfficiencyLLM Inference

    GIGASPACES

  10. SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference

    Jul 3, 2026Huajun Bai, Weiwei Lv, Huichuan Zheng +2LLM Inference EfficiencyLLM Agent Orchestration

    Tsinghua University · Meituan

  11. WattGPU: Predicting Inference Power and Latency on Unseen GPUs and LLMs

    Jul 2, 2026Mauricio Fadel Argerich, Jonathan Fürst, Marta Patiño-MartínezLLM Inference EfficiencyEnergy-Efficient ML

    Universidad Politécnica de Madrid, C. de los Ciruelos, 28660 Boadilla del Monte, Madrid, Spain · Zurich University of Applied Sciences, Gertrudstrasse 15, 8400 Winterthur, Switzerland

  12. Scaling with Confidence: Calibrating Confidence of LLMs for Adaptive Test Time Scaling

    Jul 2, 2026Xuqing Yang, Yi Yuan, Shanzhe Lei +1LLM Inference EfficiencyRL for Language Models

    Shanghai Jiao Tong University · Southeast University · Shanghai AI Laboratory

  13. Precomputing the Future-Offset Average in TriAttention

    Jul 1, 2026Amarnath MukherjeeLLM Inference EfficiencyKV Caching

    Hozhoke, Inc.

  14. From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving

    Jun 30, 2026Jie Li, Tongyang Wang, Yong ChenLLM Inference EfficiencyLLM Serving

    Texas Tech University, Department of Computer Science, Lubbock, Texas, USA

  15. Fork-Think with Confidence

    Jun 30, 2026Zena Al-Khalili, Rafi Hakim, Dietrich Klakow +1LLM Inference EfficiencyEfficient Language Model Reasoning

    Saarland Informatics Campus, Saarland University, Germany

  16. When Reranking Hurts: Uncertainty-Based Gating for Few-Shot Reranking

    Jun 30, 2026Orian Dabod, Amir DN Cohen, Gabriel StanovskyLLM Inference EfficiencyFew-Shot Learning

    The Hebrew University of Jerusalem · OriginAI / Ramat Gan, Israel

  17. Quantization Inflates Reasoning: Token Inflation as a Hidden Cost of Low-Bit Reasoning Models

    Jun 24, 2026Xinyu Lian, Walid Krichene, Beichen Huang +4LLM Inference EfficiencyLLM Quantization

    University of Illinois Urbana-Champaign · Microsoft · Anyscale +1

  18. ASAP: Agent-System Co-Design for Wall-Clock-Centered Auto HPO Research for ML Experiments

    Jun 23, 2026Taicheng Guo, Haomin Zhuang, Kehan Guo +4LLM Inference EfficiencyLarge Language Model-Guided Optimization

    University of Notre Dame United States

  19. CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference

    Jun 23, 2026Xiaolin Lin, Jingcun Wang, Olga Kondrateva +3LLM Inference EfficiencyKV Caching

    Technical University of Darmstadt, Darmstadt, Germany · University of Notre Dame, Notre Dame, IN, USA · Technical University of Ilmenau, Ilmenau, Germany

  20. The Language-Energy Divide: Measuring Energy Costs of Multilingual LLM Inference

    Jun 20, 2026Naihao Deng, Alissa Shen, Yiming Feng +5LLM Inference EfficiencyMultilingual Language Models

    University of Michigan · University of Cambridge · University of Aberdeen

  21. Does Mixture-of-Experts Actually Help Inference on Consumer and Edge Hardware? An Empirical Study

    Jun 19, 2026Alfarizy Alfarizy, Hung Truong Thanh Nguyen, René Richard +2LLM Inference EfficiencyOn-Device Language Model Inference

    Analytics Everywhere Lab, University of New Brunswick, Fredericton, NB, Canada · TSAI Lab, University of New Brunswick, Fredericton, NB, Canada

  22. Latent Personal Memory: Represent personal memory as dynamic soft prompts

    Jun 18, 2026Debrup Das, Avinash Amballa, Yashas Malur Saidutta +3LLM Inference EfficiencyMemory-Efficient Fine-Tuning

    University of Massachusetts Amherst · Samsung Research America

  23. Think Again or Think Longer? Selective Verification for Budget-Aware Reasoning

    Jun 18, 2026Sajib Acharjee Dip, Dawei Zhou, Liqing ZhangLLM Inference EfficiencyCost-Aware Inference

    Department of Computer Science, Virginia Tech · Fralin Biomedical Research Institute, Virginia Tech · FBRI Cancer Research Center, Washington, DC

  24. CacheWeaver: Cache-Aware Evidence Ordering for Efficient Grounded RAG Inference

    Jun 18, 2026Kaizhen Tan, Rong Gu, Mingyuan LiLLM Inference EfficiencyRetrieval-Augmented Generation

    Heinz College of Information Systems and Public Policy, Carnegie Mellon University, USA

  25. SMEPilot: Characterizing and Optimizing LLM Inference with Scalable Matrix Extensions

    Jun 15, 2026Feiyang Chen, Haibo ChenLLM Inference EfficiencyLLM Inference Acceleration

    IPADS, Shanghai Jiao Tong University

  26. DynAMO:Dynamic Asset Management Orchestration via Topological Multi-Agent Scheduling

    Jun 14, 2026Kanishk Kushwaha, Vikrant Vinod Bansode, Harsh Vardhan +1LLM Inference EfficiencyMulti-Agent Orchestration

    Gati Shakti Vishwavidyalaya · IBM Research

  27. Service-Induced Congestion in Memory-Constrained LLM Serving

    Jun 14, 2026Ruicheng Ao, Jing Dong, Gan Luo +1LLM Inference EfficiencyLLM Serving

    Institute for Data, Systems, and Society, Massachusetts Institute of Technology, Cambridge, MA 02139 · Columbia Business School, Columbia University, New York, NY 10027 · School of Mathematical Sciences, Peking University, Beijing, China

  28. The Price of Anarchy in Disaggregated Inference

    Jun 11, 2026Athos GeorgiouLLM Inference EfficiencyGame Theory

    NCA

  29. Can I Buy Your KV Cache?

    Jun 11, 2026Luoyuan ZhangLLM Inference EfficiencyKV Caching

    Harbin Institute of Technology, Shenzhen (HITSZ)

  30. MiniPIC: Flexible Position-Independent Caching in <100LOC

    Jun 11, 2026Nathan Ordonez, Thomas ParnellLLM Inference EfficiencyKV Caching

    IBM Research · IBM Research Zurich, Switzerland