LLM Inference Efficiency

LLM: Large Language Model

Ask about this topic

Momentum

47 papers in the last four weeks, up 262% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 234

All topics
CardsList

  1. Attention via Black-Box Vector Search

    Oct 7, 2026Stepan Zharkov, Krish Singal, Ashwin Padaki +1Efficient AttentionSparse Attention

    University of Pennsylvania

  2. ReFold: Training-Free Reversible Inter-Turn Context Folding for Long-Horizon Agents

    Oct 6, 2026Yupeng Su, Jiayi Tian, Zheng Zhang +1LLM Inference EfficiencyLong-Context Language Model Inference

    University of California, Santa Barbara · Intel

  3. Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell

    Oct 6, 2026Hochan Son, Kyungdoe Han, Jaehan Koh +3LLM Inference EfficiencyMulti-Agent LLM Systems

    University of California, Los Angeles · University of Wisconsin · HCLTech America

  4. Monte Carlo Estimation for KV Cache Eviction

    Oct 6, 2026Ahsan Bilal, Muhammad Ahmed Mohsin, Muhammad Umer +5LLM Inference EfficiencyKV-Cache Eviction

    University of Oklahoma · Stanford University · University of Houston +3

  5. Evaluating Inference Compute for Generative AI: A Framework for Enterprise Workloads

    Oct 5, 2026Abbas Raza Ali, Muhammad Ajmal Siddiqui, Moona ZahidLLM Inference EfficiencyDisaggregated LLM Serving

    Citigroup Inc., London, United Kingdom · Ernst & Young LLP, London, United Kingdom · NVIDIA Corporation, London, United Kingdom

  6. Expanding LLM Reasoning

    Oct 4, 2026Rian Atri, Evan LuoLLM Inference EfficiencyInference-Time Scaling

    Keiji AI · University of California, Berkeley

  7. Beyond State-of-the-Art: Standardising Environmental Impact Metrics for AI Research

    Oct 1, 2026Lachlan McGinness, Dan Pagendam, Robert OffnerLLM Inference EfficiencyEnergy-Efficient ML

    School of Computing, Australian National University · Commonwealth Scientific and Industrial Research Organisation

  8. From Discovery to Decision: Finite-Budget Recoverability in LLM Voting

    Oct 1, 2026Shaoang Li, Jian LiLLM Inference EfficiencyLLM Inference

    Stony Brook University

  9. Beyond Leaderboards: Tokenomics of Agentic Small Language Model Ensembles

    Oct 1, 2026Alexei N. Skurikhin, Emily M. Taylor, Nathan A. DeBardelebenLLM Inference EfficiencyLanguage Model Generation Evaluation

    Los Alamos National Laboratory Los Alamos, New Mexico, USA

  10. Characterizing High Bandwidth Flash for LLM Serving

    Sep 30, 2026Zack Yu, Chloe Wong, Coleman Hooper +7LLM Inference EfficiencyLLM Serving

    University of California, Berkeley Berkeley, California, USA · FuriosaAI Seoul, South Korea · University of California, Berkeley; ICSI; LBNL Berkeley, California, USA

  11. Adaptive Self-Consistency: From Black-Box Sampling to Distribution-Valued Feedback

    Sep 30, 2026Jingkai Huang, Yunfan Zhang, Will Ma +2LLM Inference EfficiencyAdaptive Inference

    Stern School of Business New York University · School of Management Zhejiang University

  12. Purlin: Separating Orchestration from the Datapath of Collectives

    Sep 29, 2026Osayamen Jonathan Aimuyo, Swapnil Gandhi, Christos KozyrakisLLM Inference EfficiencyHigh-Performance Computing

    Stanford University · NVIDIA & Stanford University

  13. Beyond Energy: When Sustainability Dimensions Reshape LLM Serving Decisions

    Sep 28, 2026Tianyao Shi, Xipeng Shen, Yi DingLLM Inference EfficiencyLLM Serving

    Elmore Family School of Electrical and Computer Engineering, Purdue University, USA

  14. WavePP: High-Throughput Pipeline Parallel LLM Prefill under Prefix Reuse

    Sep 28, 2026Aaryam SharmaLLM Inference EfficiencyLLM Serving

    Baseten

  15. SCBO: Semantically Coherent Batching and Ordering for LLM-Based Social Surveys

    Sep 28, 2026Yuanzi Li, Lingjie Wang, Zihang Tian +2LLM Inference EfficiencyLLM Prompting

    Renmin University of China

  16. Beneath the Tokens: A Performance Engineering Study of Multi-Token Prediction in GPU-Accelerated LLM Inference

    Sep 28, 2026Suwesh Prasad SahLLM Inference EfficiencyMulti-Token Prediction

  17. AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs

    Sep 28, 2026Cheuk Hang Lau, Zeyu Cao, Kevin Wong Cheuk Yin +6LLM Inference EfficiencyLLM Inference

    Imperial College London · University of Cambridge · University of Oxford

  18. Spexis: Speculative Lookahead Scheduling for LLM Inference

    Sep 28, 2026Hyungyu Jung, Jaehyeok Yu, Hoonseo Choi +3LLM Inference EfficiencyLLM Inference Scheduling

    Seoul National University · Hanyang University

  19. Greenpixie's AI Token Methodology: Assessing the Energy, Water and CO2-eq Impact of AI Tokens for Open and Closed Weight Models

    Sep 27, 2026Joshua Horswill, Ross Hunter, Matt Clifford +1LLM Inference EfficiencyLLM Inference

    Greenpixie Ltd., London, UK.

  20. You Only Edit Once: Incentivizing In-Context Capability of LLMs via Local Demonstration Refinement

    Sep 27, 2026Jiarong Wen, Qi Wang, Yun Qu +8LLM Inference EfficiencyIn-Context Example Selection

    Department of Automation, Tsinghua University

  21. JEV vs. LLMs as Rubric Judges: Cheaper, Faster, and Wrong in the Same Places

    Sep 24, 2026Delip Rao, Chris Callison-BurchLLM Inference EfficiencyLLM Evaluation

    University of Pennsylvania

  22. When Fancy Eviction Fails: Rethinking Cache Replacement For LLM Prefix Reuse

    Sep 24, 2026Yiyu Liu, Minlan Yu, Juncheng YangLLM Inference EfficiencyKV Caching

    Harvard University

  23. Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models

    Sep 22, 2026Moritz Laber, Zohair Shafi, Germans Savcisens +6LLM Inference EfficiencyLanguage Model Scaling Laws

    Northeastern University, Boston, Massachusetts, USA · Complexity Science Hub Vienna, Vienna, Austria · Santa Fe Institute, Santa Fe, New Mexico, USA

  24. COMED: The Missing Middle Between Routing and Collaboration in Multi-LLM Inference

    Sep 22, 2026Norah Alballa, Wenxuan Zhang, Salma Kharrat +4LLM Inference EfficiencyLLM Routing

    KAUST · LUMS

  25. REFLEX with Jev for Efficient Selective Control in LLM Agents

    Sep 22, 2026Tiantong Wu, Wei Yang Bryan LimLLM Inference EfficiencyLLM Agent Evaluation

    Nanyang Technological University Singapore

  26. Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference

    Sep 22, 2026Md Mostafizer Rahman, Md Faizul Ibne Amin, Md Shahajada Mia +2LLM Inference EfficiencyLong-Context Language Model Inference

    Lucy Family Institute for Data & Society, University of Notre Dame, IN, USA · The University of Aizu, Aizuwakamatsu, Japan · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame, IN, USA

  27. Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directions

    Sep 21, 2026Niloofar Gholipour, Marcos Assuncao, Gursimran Singh +8LLM Inference EfficiencyReinforcement Learning

    École de technologie supérieure, Univ. of Québec, Canada · Huawei Technologies, Canada · The Univ. of Melbourne, Australia +1

  28. Open-Jev Judgments on CallScreenBench: Calibrated One-Pass Scam Screening with a Small Language Model

    Sep 21, 2026Simiao Ren, Kidus Zewde, Xingyu Shen +6LLM Inference EfficiencyLLM Evaluation

    Scam.ai (Reality Inc.)

  29. ValueDiff: Value-Geometric KV Cache Eviction for Sink-Suppressed LLMs

    Sep 20, 2026Junyoung Park, Jungwook Choi, Mingu LeeLLM Inference EfficiencyKV-Cache Eviction

    Qualcomm AI Research · Hanyang University

  30. Not All AI Agents Are Equal: Characterizing Resource and Performance Dynamics

    Sep 17, 2026Wonmi Choi, Minuk Park, Zhixiong Niu +3LLM Inference EfficiencyLLM Agent Workflow Optimization

    Department of Computer Science and Engineering, Korea University · Microsoft Research Asia