LLM Inference Efficiency

LLM: Large Language Model

Overview

The field studies how to serve large language models faster and more cheaply while preserving answer quality, especially for long contexts, reasoning, and multi-step agents. Systems approaches improve batching, scheduling, speculative decoding, and key-value cache storage through methods such as quantization, eviction, and reuse. Model-side approaches adapt reasoning budgets, route queries between models, and compress prompts or context to avoid unnecessary computation. A central challenge is that gains depend on the model, workload, hardware, and latency or energy goals, so evaluations increasingly compare quality, cost, latency, and energy together on tasks such as LongBench, GSM8K, and MATH.

Latest papers

CardsList

368 papers17 papers in the last four weeks, down 67% on the four weeks before. 0.1% of all new papers.

  1. Nalar: Workflow-Aware Management of Agentic Applications

    Jan 8, 2026Saurabh Agarwal, Marco Laju, Donghyun Son +4LLM Inference EfficiencyLLM Agent Orchestration

  2. Step-Tagging: Toward controlling the generation of Language Reasoning Models through step monitoring

    Dec 16, 2025Yannis Belkhiter, Seshu Tirupathi, Giulio Zizzo +1Reasoning Trace AnalysisEfficient Language Model Reasoning

  3. Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

    Nov 11, 2025Jon Saad-Falcon, Avanika Narayan, Hakki Orhun Akengin +12LLM Inference EfficiencyAI Accelerator Inference

  4. Explore-Execute Chain: Towards an Efficient Structured Reasoning Paradigm

    Sep 28, 2025Kaisen Yang, Tinghe Zhang, Rushi Shah +4LLM ReasoningStructured Reasoning

  5. JigSLO: Solving the Resource Puzzle for Efficient Multi-LLM Serving

    Sep 8, 2025Kyungmin Bin, Seungbeom Choi, Jimyoung Son +6LLM Inference EfficiencyLLM Inference Scheduling

  6. EquivPruner: Boosting Efficiency and Quality in LLM-Based Search via Action Pruning

    May 22, 2025Jiawei Liu, Qisi Chen, Jianshu Zhang +2LLM Inference EfficiencySemantic Textual Similarity

  7. Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models

    Mar 31, 2025Rui Wang, Hongru Wang, Boyang Xue +9LLM Inference EfficiencyCost-Aware Inference

  8. Saving GPU Hours in LLM Inference System Development and Online Workloads with Simulation and DBMS-Inspired Cache Replacement Policies

    Nov 12, 2024Kyoungmin Kim, Jiacheng Li, Kijae Hong +3LLM Inference EfficiencyLLM Inference