LLM Inference Efficiency

LLM: Large Language Model

Ask about this topic

Momentum

47 papers in the last four weeks, up 262% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 234

All topics
CardsList

  1. MARS: Margin-Adversarial Risk-controlled Stopping for Parallel LLM Test-time Scaling

    Jun 11, 2026Wenbo Chen, Puheng Li, Mengyang Liu +2LLM Inference EfficiencySelf-Consistency Decoding

    Amazon · Stanford University · University of Pennsylvania

  2. Arbor: Tree Search as a Cognition Layer for Autonomous Agents

    Jun 10, 2026Neha Prakriya, Chaojun Hou, Zheng Gong +5LLM Inference EfficiencyMulti-Agent System Optimization

    Training and Inference Optimization Team AMD

  3. Closing the Social-Semantic Gap: SPSD for Edge-Based Prompt Compression in Cloud LLM Inference

    Jun 10, 2026Abhinit Sen, Ajeet Kumar, Manaranjan PradhanLLM Inference EfficiencyOn-Device Language Model Inference

    Indian School of Business, Hyderabad, India

  4. Less Context, Better Agents: Efficient Context Engineering for Long-Horizon Tool-Using LLM Agents

    Jun 8, 2026Abhilasha Lodha, Mahsa Pahlavikhah Varnosfaderani, Abir Chakraborty +1LLM Inference EfficiencyTool-Augmented Language Model Agents

    Microsoft

  5. RKSC: Reasoning-Aware KV Cache Sharing and Confident Early Exit for Multi-Step LLM Inference

    Jun 7, 2026Anirudh SekarLLM Inference EfficiencyKV Caching

    Accepted to the ICML 2026 Workshop on Statistical Frameworks for Uncertainty in Agentic Systems, Seoul, South Korea, 2026.

  6. Value-Aware Stochastic KV Cache Eviction for Reasoning Models

    Jun 2, 2026Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu +3LLM Inference EfficiencyKV Caching

    University of Southern California · University of Chicago

  7. MOSAIC: Efficient Mixture-of-Agent Scheduling via Adaptive Aggregation and Inference Concurrency

    Jun 2, 2026Saptarshi Mitra, Yifan Zhang, Rachid Karami +5LLM Inference EfficiencyMulti-Agent System Optimization

    University of California, Irvine, USA · University of Southern California, Los Angeles, USA · Intel, USA

  8. RASER: Recoverability-Aware Selective Escalation Router for Multi-Hop Question Answering

    Jun 1, 2026Yuyang Li, Zihe Yan, Tobias KäferMulti-Hop QALLM Inference Efficiency

    Institute AIFB, Karlsruhe Institute of Technology, Karlsruhe, Germany · Shanghai Jiao Tong University, Shanghai, China

  9. DOT-MoE: Differentiable Optimal Transport for MoEfication

    Jun 1, 2026Udbhav Bamba, Arnav Chavan, Aryamaan Thakur +2LLM Inference EfficiencyMixture-of-Experts Models

    Amazon

  10. Early Diagnosis of Wasted Computation in Multi-Agent LLM Systems via Failure-Aware Observability

    May 31, 2026Xianyou Li, Weiran Yan, Yichao Wu +4LLM Inference EfficiencyMulti-Agent LLM Systems

    New York University · New York, NY 10012, USA · Independent Researcher +7

  11. Thinking Economically: A Hierarchical Framework for Adaptive-Complexity Reasoning in LLMs

    May 31, 2026Yubo Gao, Haotian Wu, Hong Chen +8LLM Inference EfficiencyCoT Reasoning

    The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology · Nanyang Technological University +1

  12. Leyline: KV Cache Directives for Agentic Inference

    May 31, 2026Bole Ma, Jan Eitzinger, Harald KoestlerLLM Inference EfficiencyKV Caching

    Erlangen National High Performance Computing Center, Erlangen, Germany

  13. SAGE: A Novelty Gate for Efficient Memory Evolution in Agentic LLMs

    May 29, 2026Sijia Wang, Dhanajit Brahma, Ricardo HenaoLLM Inference EfficiencyAgent Memory

    Duke University

  14. Efficient Test-Time Finetuning of LLMs via Convex Reconstruction and Gradient Caching

    May 28, 2026Alaa Khamis, Alaa MaaloufLLM Inference EfficiencyFine-Tuning

    Department of Computer Science University of Haifa

  15. MarginGate: Sparse Margin-Triggered Verification for Batch-Invariant LLM Inference

    May 28, 2026Kexin Chu, Yang Zhou, Wei ZhangLLM Inference EfficiencyLanguage Model Decoding

    University of Connecticut · UC Davis

  16. Anchorless Diversification for Parallel LLM Ideation

    May 28, 2026Fares Nabil Ibrahim, Nafis Saami Azad, Raiyan Abdul BatenLLM Inference EfficiencyLLM Inference

    Bellini College of Artificial Intelligence, Cybersecurity, and Computing, University of South Florida, USA

  17. ParaTool: Shifting Tool Representations from Context to Parameters

    May 28, 2026Zekai Yu, Qi Meng, Qizhi Chu +3LLM Inference EfficiencyLLM Tool Use

    1Beijing University of Posts and Telecommunications, Beijing, China

  18. Analyzing Quality-Latency-Resource Trade-offs in a Technical Documentation RAG Assistant Using LoRA Adaptation

    May 27, 2026Evgenii Palnikov, Elizaveta GavrilovaLLM Inference EfficiencyRetrieval-Augmented Generation

    1HSE University

  19. A Policy-Driven Runtime Layer for Agentic LLM Serving

    May 26, 2026Rui Zhang, Chaeeun Kim, Liting HuLLM Inference EfficiencyMulti-Agent LLM Systems

    University of California, Santa Cruz

  20. The Coverage Illusion: From Pre-retrieval Routing Failure to Post-retrieval Cascades in a Production RAG System

    May 26, 2026Zafar Hussain, Kristoffer NielboLLM Inference EfficiencyRetrieval-Augmented Generation

    Aarhus University, Denmark

  21. ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inference

    May 26, 2026Xiongwei Zhu, Xiaojian Liao, Tianyang Jiang +3LLM Inference EfficiencyExpert Offloading

    School of Computer Science and Engineering, Beihang University, Beijing 100191, China · Huawei Technologies Ltd.

  22. Stateful Inference for Low-Latency Multi-Agent Tool Calling

    May 25, 2026Victor NorgrenLLM Inference EfficiencyMulti-Agent LLM Systems

    LayerScale, Inc.

  23. A general tensor-structured compression scheme for efficient large language models

    May 25, 2026Ying Lu, Peng-Fei Zhou, Qi-Xuan Fang +3LLM Inference EfficiencyLLM Compression

    School of Physical Sciences, University of Chinese Academy of Sciences, Beijing, 100049, China. · Kavli Institute for Theoretical Sciences, University of Chinese Academy of Sciences, Beijing, 100190, China. · Center for Quantum Physics and Intelligent Sciences, Department of Physics, Capital Normal University, Beijing, 100048, China. +1

  24. MVR-cache: Optimizing Semantic Caching via Multi-Vector Retrieval and Learned Prompt Segmentation

    May 24, 2026Ali Noshad, Zishan Zheng, Yinjun WuLLM Inference EfficiencyRL for Combinatorial Optimization

    School of Computer Science, Peking University, Beijing, China · School of Information, Renmin University of China, Beijing, China

  25. Inference Time Context Sparsity: Illusion or Opportunity?

    May 22, 2026Sahil Joshi, Prithvi Dixit, Agniva Chowdhury +5LLM Inference EfficiencyStructured Sparsity

    α Rice University · β UC Berkeley · γ IIT Bombay

  26. Parallel Context Compaction for Long-Horizon LLM Agent Serving

    May 22, 2026Musa Cim, Burak Topcu, Chita Das +1LLM Inference EfficiencyLLM Agent Workflow Optimization

    The Pennsylvania State University

  27. ModeSwitch-LLM: A Lightweight Phase-Aware Controller for Cross-Mode LLM Inference on a Single GPU

    May 21, 2026Aman Sunesh, Ali Alshehhi, Hivansh DhakneLLM Inference EfficiencyLLM Serving

    Department of Computer Engineering NYU Abu Dhabi Abu Dhabi, United Arab Emirates · Courant Institute of Mathematical Sciences New York University New York, United States · Department of Computer Engineering New York University New York, United States

  28. E3E^3-Agent: An Executable and Evolving Agent for Resource Management of Edge Generative Inference

    May 21, 2026Rui Bao, Yaping Sun, Zhiyong Chen +4LLM Inference EfficiencyEdge Inference

    Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai 200240, China · Department of Broadband Communication, Pengcheng Laboratory, Shenzhen 518000, China · China Mobile Research Institute, Beijing 100053, China

  29. IdleSpec: Exploiting Idle Time via Speculative Planning for LLM Agents

    May 21, 2026Daewon Choi, Kyunghyun Park, Woomin Song +4LLM Inference EfficiencyLong-Horizon LLM Agents

    1KAIST · 3Together AI · 2Amazon AGI

  30. Planning in the LLM Era: Building for Reliability and Efficiency

    May 21, 2026Michael Katz, Harsha Kokel, Kavitha Srinivas +1LLM Inference EfficiencyLLM Planning

    IBM Research