LLM Inference Acceleration

LLM: Large Language Model

Ask about this topic

Momentum

108 papers in the last four weeks, up 120% on the four weeks before. 1.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 677

All topics
CardsList

  1. IrekoGPT: Turning Structured Pruning into Post-Hoc Slimmable LLMs

    Sep 30, 2026Pietro Moriello, Pietro Buzzega, Angelo Porrello +1LLM CompressionLLM Inference Acceleration

    AImageLab, University of Modena and Reggio Emilia, Italy

  2. CommunityKV: Efficient Long-Context Decoding via Graph Partitioning

    Sep 30, 2026Joe McKenna, Anastasios Alexandridis, Nathan Susanj +1Long-Context Language Model InferenceKV-Cache Management

    Amazon AGI

  3. Beyond Accuracy: Prefix-Invariant Realizations of Low-Precision Fast Matrix Multiplication

    Sep 30, 2026Shuxiao Xie, Shuyang Xie, Yuan Cao +3LLM Inference AccelerationLow-Bit Quantization

  4. EchoPress: Query-Agnostic KV Cache Pruning via Virtual Context Reconstruction

    Sep 30, 2026Jiawei Lin, Saibo Geng, Thomas BourgeatKV-Cache ManagementLLM Inference Acceleration

    EPFL

  5. LampAttention: Look-Ahead Mixed-Precision FlashAttention for Dedicated Accelerators

    Sep 30, 2026Stanislav Budzinskiy, Marian Gloser, Tolunay Yilmaz +5AI Accelerator InferenceLLM Inference Acceleration

    Faculty of Mathematics University of Vienna, Austria · Huawei Heisenberg Research Center, Munich, Germany · Huawei Technologies Co. Ltd

  6. Working Around the Compute Ceiling: Byte-Exact Memory in Galahad Makes LLM Reading a One-Time Cost LLM Reading a One-Time Cost

    Sep 30, 2026Sietse SchelpeLLM ServingLong-Context Language Model Inference

    Corbenic AI

  7. PatchKV: Weight-Space Compensation of KV Cache

    Sep 30, 2026Chanryeol Lee, Chanhyuk Lee, Yeonwoo Choi +2Long-Context Language Model InferenceLLM Inference Acceleration

    KAIST

  8. SparseEngine: Sparse-First Inference Engine

    Sep 30, 2026Jitai Hao, Quansheng Gu, Qiang Huang +1LLM InferenceLong-Context Language Model Inference

  9. Audio Token Attention Is Predictable Before the Language Model Runs

    Sep 30, 2026Kyoungjun Park, Yunzhe Li, Lili QiuAudio Token CompressionAudio Understanding

    The University of Texas at Austin

  10. STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization

    Sep 29, 2026Bingchen Yao, Haobo Xu, Haokun Lin +6LLM QuantizationLLM Inference Acceleration

    Zhejiang University · Tsinghua University · NLPR & MAIS, Institute of Automation, CAS +3

  11. LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization

    Sep 29, 2026Yi Pan, Haocheng Xi, Kan Zhu +10LLM QuantizationLLM Inference Acceleration

    UC Berkeley · University of Washington · MIT +2

  12. KV-Kaizen: Learning Context-Adaptive Cache Compression Choices

    Sep 29, 2026Joao Monteiro, Louis Béthune, Anastasiia Filippova +3LLM Inference AccelerationLow-Rank Compression

    Apple

  13. Can a Cacheable Decision Model Follow Rules?

    Sep 29, 2026Dushyant Rajput, Nirdesh Chauhan, Siddharth KosarajuLLM Inference AccelerationLLM Decision-Making

    AltSlate Labs LLP

  14. DScale: Scaling Block-Diffusion Speculative Decoding with Adaptive Verification

    Sep 29, 2026Rongjian Chen, Minxian Xu, Zhengxin Fang +2Speculative DecodingLLM Inference Acceleration

    Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518052, China · University of Chinese Academy of Sciences, Beijing 100049, China · Victoria University of Wellington, Wellington, New Zealand +1

  15. LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter

    Sep 29, 2026Hao-Yuan He, Peng-Fei Liu, Si Shen +1Long-Context Language Model InferenceBlockwise Diffusion Language Models

  16. LatCom: Cross-Agent Latent Compression for Efficient Multi-Agent Collaboration

    Sep 29, 2026Shinan Zhang, Tao Zhang, Qihui Zhu +7Multi-Agent LLM SystemsMulti-Agent Collaboration

    University of Science and Technology of China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center

  17. ARC-KV: Amortizing Anchor Search for Reconstruction-Based KV Cache Compaction

    Sep 29, 2026Zheyu Shen, Guanhua Wang, Dezhan Tu +5Long-Context Language ModelingKV-Cache Management

    University of Maryland, College Park · Meta

  18. QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching

    Sep 29, 2026Zunhai Su, Yuxuan Sun, Jianchao Tan +5LLM Inference AccelerationAttention Quantization

    The University of Hong Kong · Meituan LongCat Team · South China University of Technology +1

  19. MoRE: Scaling mixture of experts with hardware-aware low-rank routing

    Sep 28, 2026Honam Wong, Surbhi Goel, Enric Boix-AdseràParameter-Free Mixture-of-Experts RoutingLow-Rank Matrix Decomposition

    University of Pennsylvania · The Wharton School, University of Pennsylvania

  20. Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding

    Sep 28, 2026Haohui Zhang, Keyu Chen, Haocheng Sun +4Speculative DecodingLLM Inference Acceleration

    Shanghai Jiao Tong University · Tencent YouTu Lab · Xiamen University

  21. Improving Test-Time Scaling with Adaptive Looped Transformers

    Sep 28, 2026Yichen You, Tianyu Fu, Aosong Feng +4Test-Time ScalingRecurrent Transformers

    Tsinghua University · Yale University

  22. SANTA++: Sampling Attention through Representative Keys

    Sep 28, 2026Kyle Lee, Christian Z. Pratt, Ruoyu Fang +4Memory-Efficient InferenceLLM Inference Acceleration

    Department of Electrical and Computer Engineering University of California, Santa Barbara, CA, USA · Flucta, San Francisco, CA, USA

  23. Tetra: Serving Leech-Lattice Quantized LLMs at 2.7 Bits per Parameter

    Sep 28, 2026Pier-Jean MalandrinoLLM QuantizationLLM Inference Acceleration

    Scub, Bordeaux, France

  24. Beneath the Tokens: A Performance Engineering Study of Multi-Token Prediction in GPU-Accelerated LLM Inference

    Sep 28, 2026Suwesh Prasad SahLLM Inference EfficiencyMulti-Token Prediction

  25. CacheRepair: Learning to Repair Cross-Chunk Context in RAG for KV Cache Fusion

    Sep 28, 2026Genglin Wang, Wangsong Yin, Yeerzhati Abudunuer +3Retrieval-Augmented GenerationKV-Cache Management

    The Chinese University of Hong Kong · Peking University · The Hong Kong University of Science and Technology

  26. TempoKV: Timely Staging of LLM KV Caches for Memory-Semantic Flash

    Sep 28, 2026Jay H. Park, Hyungjun Kim, Dong KimKV-Cache OffloadingKV-Cache Management

    Samsung Semiconductor

  27. Dynamic Flow, Static Graph: KV Cache Reuse for Efficient LLM Serving on Mobile NPUs

    Sep 28, 2026Zhengxiang Huang, Shengheng Chen, Chaoyue Niu +6AI Accelerator InferenceOn-Device Language Model Inference

    Shanghai Jiao Tong University Shanghai, China · Alibaba Group Hangzhou, China · Alibaba Group Hangzhou, Zhejiang, China

  28. The Model Knows When to Stop: Training-Free Early Stopping for Long-Context Reading

    Sep 28, 2026Muath Alyobi, Mohamed Eltahir, Almoayyad Abuljdail +3Early StoppingLong-Context Language Model Evaluation

    King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Department of Computer Science, Edge Hill University, Ormskirk, England

  29. PulseInfer: I/O-Centric Sparse KV Cache Offloading for Efficient Long-Context LLM Decoding

    Sep 28, 2026Qiuyang Zhang, Kai Zhou, Kai Lu +6KV-Cache OffloadingLong-Context Language Model Inference

    Huazhong University of Science and Technology Wuhan, China · UCloud Technology Co., Ltd. Shanghai, China

  30. Shallow Queries, Mature Values: Depth-Asynchronous Self-Speculation for Looped Transformers

    Sep 28, 2026Guanghao Li, Zihan Su, Hao Yu +6Speculative DecodingRecurrent Transformers

    Tsinghua University · Southern University of Science and Technology · The University of Hong Kong +2