LLM Inference Efficiency

LLM: Large Language Model

Ask about this topic

Momentum

47 papers in the last four weeks, up 262% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 234

All topics
CardsList

  1. Self-Consistency from Only Two Samples: CoT-PoT Ensembling for Efficient LLM Reasoning

    Apr 19, 2026Raman Saparkhan, Majd Hawasly, Md Rizwan Parvez +1LLM Inference EfficiencySelf-Consistency Decoding

    Carnegie Mellon University Qatar · Qatar Computing Research Institute

  2. Hive: A Multi-Agent Infrastructure for Algorithm- and Task-Level Scaling

    Apr 19, 2026Zizhang Luo, Yuhao Luo, Youwei Xiao +3LLM Inference EfficiencyMulti-Agent LLM Systems

    Peking University Beijing, China

  3. HiveMind: OS-Inspired Scheduling for Concurrent LLM Agent Workloads

    Apr 18, 2026Justice Owusu Agyemang, Jerry John Kponyo, Obed Kwasi Somuah +3LLM Inference EfficiencyAI Coding Agents

    Sperix Labs · VIA Cybersecurity Lab, KNUST · Quantum and Assistive Technologies Lab, KNUST

  4. When Agents Go Quiet: Output Generation Capacity and Format-Cost Separation for LLM Document Synthesis

    Apr 17, 2026Justice Owusu Agyemang, Michael Agyare, Miriam Kobbinah +2LLM Inference EfficiencyLong-Form Document Generation

    Sperix Labs · GCTU, Ghana · NCA, Ghana +1

  5. Atropos: Improving Cost-Benefit Trade-off of LLM-based Agents under Self-Consistency with Early Termination and Model Hotswap

    Apr 16, 2026Naryeong Kim, Shin YooLLM Inference Efficiency

    KAIST · Daejeon, Republic of Korea

  6. TRACER: Trace-Based Adaptive Cost-Efficient Routing for LLM Classification

    Apr 16, 2026Adam RidaNeural Surrogate ModelingLLM Inference Efficiency

    Independent Researcher.

  7. PAC-CF: Calibrating Irreversible Frontier Pruning in LLM-Guided Search

    Apr 15, 2026Tianhao Qian, Jiayu Chen, Zhenyu Sun +1LLM Inference EfficiencyInference-Time Search

  8. PipeLive: Efficient Live In-place Pipeline Parallelism Reconfiguration for Dynamic LLM Serving

    Apr 14, 2026Xu Bai, Muhammed Tawfiqul Islam, Chen Wang +1LLM Inference EfficiencyLLM Serving

    DisNet Lab University of Melbourne Melbourne, VIC, Australia · IBM Research Yorktown Heights, NY, USA

  9. Apriel-Reasoner: RL Post-Training for General-Purpose and Efficient Reasoning

    Apr 2, 2026Rafael Pardinas, Ehsan Kamalloo, David Vazquez +1LLM Inference EfficiencyRL for Language Model Reasoning

    ServiceNow AI Research

  10. When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models

    Mar 27, 2026Juan Gabriel Kostelec, Qinghai GuoLLM Inference EfficiencyLLM Evaluation

    Huawei Switzerland · ACS Lab, Huawei Technologies

  11. Clinical Note Bloat Reduction for Efficient LLM Use

    Mar 21, 2026Jordan L. Cahoon, Chloe Stanwyck, Asad Aali +6LLM Inference EfficiencyElectronic Health Records

    Department of Biomedical Data Science, Stanford University, Stanford, CA · Department of Pathology, Stanford University, Stanford, CA · Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University, Stanford, CA +5

  12. MoEless: Efficient MoE LLM Serving with Serverless Experts

    Mar 6, 2026Hanfei Yu, Bei Ouyang, Shwai He +2LLM Inference EfficiencyExpert Load Balancing

    University of Maryland College Park

  13. Same Pieces, Different Servers: A Tetris Benchmark for AI Agents as Served

    Mar 2, 2026Haochuan WangLLM Inference EfficiencyLLM Agent Evaluation

    Massachusetts Institute of Technology

  14. Data Driven Optimization of GPU efficiency for Distributed LLM-Adapter Serving

    Feb 27, 2026Ferran Agullo, Joan Oliveras, Chen Wang +5LLM Inference EfficiencyLLM Serving

    Barcelona Supercomputing Center (BSC), Barcelona, Spain · Universitat Politècnica de Catalunya - BarcelonaTech (UPC), Barcelona, Spain · IBM Research, New York, USA

  15. Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokens

    Feb 13, 2026Wei-Lin Chen, Liqian Peng, Tian Tan +5LLM Inference EfficiencyInference-Time Scaling

    University of Virginia · Google

  16. More Bang for the Buck: Improving the Inference of Large Language Models at a Fixed Budget using Reset and Discard (ReD)

    Jan 29, 2026Sagi Meir, Tommer D. Keidar, Noam Levi +2LLM Inference EfficiencyLLM Evaluation

    School of Chemistry, Tel Aviv University, Tel Aviv 6997801, Israel · The Center for Physics and Chemistry of Living Systems, Tel Aviv University, Tel Aviv 6997801, Israel · School of Physics and Astronomy, Tel Aviv University, Tel Aviv 6997801, Israel +1

  17. Nalar: Workflow-Aware Management of Agentic Applications

    Jan 8, 2026Saurabh Agarwal, Marco Laju, Donghyun Son +4LLM Inference EfficiencyLLM Agent Orchestration

    UT-Austin · Cisco-Research

  18. Step-Tagging: Toward controlling the generation of Language Reasoning Models through step monitoring

    Dec 16, 2025Yannis Belkhiter, Seshu Tirupathi, Giulio Zizzo +1LLM Inference EfficiencyEfficient Language Model Reasoning

    1IBM Research Europe, Dublin, Ireland · 2Trinity College Dublin, Ireland · 3ADAPT Research Centre, Dublin, Ireland

  19. Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

    Nov 11, 2025Jon Saad-Falcon, Avanika Narayan, Hakki Orhun Akengin +12LLM Inference EfficiencyAI Accelerator Inference

    Stanford University · Together AI

  20. Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm

    Sep 28, 2025Kaisen Yang, Tinghe Zhang, Rushi Shah +4LLM Inference EfficiencyLLM Planning

    1Qizhi Institute · 2Dept. of Computer Science, Tsinghua University · College of AI, Tsinghua University +4

  21. EquivPruner: Boosting Efficiency and Quality in LLM-Based Search via Action Pruning

    May 22, 2025Jiawei Liu, Qisi Chen, Jianshu Zhang +2LLM Inference EfficiencySemantic Textual Similarity

    School of Computer Science and Technology, University of Science and Technology of China · Department of Data Science, City University of Hong Kong · iFLYTEK Research

  22. Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models

    Mar 31, 2025Rui Wang, Hongru Wang, Boyang Xue +9LLM Inference EfficiencyCost-Aware Inference

    University of Edinburgh, Edinburgh, UK · University of Macau, Macau SAR, China · University of Illinois Urbana-Champaign, USA

  23. Saving GPU Hours in LLM Inference System Development and Online Workloads with Simulation and DBMS-Inspired Cache Replacement Policies

    Nov 12, 2024Kyoungmin Kim, Jiacheng Li, Kijae Hong +3LLM Inference EfficiencyLLM Inference

    EPFL, Switzerland · NUS, Singapore · POSTECH, South Korea

  24. UltraQuant: 4-bit KV Caching for Context-Heavy Agents

    Date pendingInesh Chakrabarti, David Limpus, Aditi Ghai Rana +4LLM Inference EfficiencyGPU Kernel Optimization

    Advanced Micro Devices · University of California, Los Angeles · Purdue University