LLM Inference Acceleration

LLM: Large Language Model

Ask about this topic

Momentum

108 papers in the last four weeks, up 120% on the four weeks before. 1.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 677

All topics
CardsList

  1. Density Field State Space Models: 1-Bit Distillation, Efficient Inference, and Knowledge Organization in Mamba-2

    Apr 28, 2026Chirag ShindeModel CompressionLLM Quantization

    Independent Researcher

  2. AHASD: Asynchronous Heterogeneous Architecture for LLM Adaptive Drafting Speculative Decoding on Mobile Devices

    Apr 28, 2026Ma Zirui, Fan Zhihua, Li Wenxing +4Compute-in-MemoryOn-Device Language Model Inference

    State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences

  3. Rethinking Layer Redundancy: Calibration Matters More Than Search in LLM Depth Pruning

    Apr 27, 2026Minkyu Kim, Vincent-Daniel Yun, Youngrae Kim +3LLM PruningLLM Inference Acceleration

    Neural Superintelligence Lab, MODULABS, Republic of Korea · University of Southern California, United States · Boston University, United States +2

  4. Less is More: Lightweight Prompt Compression for Question Answering Applications on Edge Devices

    Apr 27, 2026Zihuai Xu, Ruofei Hou, Yang Xu +3Retrieval-Augmented GenerationOn-Device Language Model Inference

    Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China, 215123

  5. Salca: A Sparsity-Aware Hardware Accelerator for Efficient Long-Context Attention Decoding

    Apr 27, 2026Wang Fan, Wei Cao, Xi Zha +5Self-AttentionLong-Context Language Model Inference

    Fudan University

  6. AgenticCache: Cache-Driven Asynchronous Planning for Embodied AI Agents

    Apr 27, 2026Hojoon Kim, Yuheng Wu, Thierry TambeLLM PlanningLLM Inference Acceleration

    Seoul National University, Seoul, South Korea · Stanford University, Stanford, California, USA

  7. Pruning via Causal Attribution Preserves Reasoning Performance in Large Language Models

    Apr 27, 2026Amogh Sheth, Biruk Assefa, Yi Wen Huang +2LLM PruningSelf-Attention

    Edison Academy Magnet School · State University of New York College at Plattsburgh · The University of Texas at Austin +2

  8. DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

    Apr 26, 2026DeepSeek-AI, Anyi Xu, Bangcai Lin +316Mixture-of-Experts Language ModelsLong-Context Language Model Inference

  9. Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning

    Apr 26, 2026Zichuan Fu, Xian Wu, Guojing Li +7Efficient Language Model ReasoningLLM Inference Acceleration

    City University of Hong Kong · Tencent Jarvis Lab · Renmin University of China +1

  10. Hybrid JIT-CUDA Graph Optimization for Low-Latency Large Language Model Inference

    Apr 25, 2026Divakar Kumar Yadav, Tian ZhaoLLM InferenceLLM Inference Acceleration

    Department of Computer Science University of Wisconsin-Milwaukee Milwaukee, WI, USA

  11. Large Language Models Decide Early and Explain Later

    Apr 24, 2026Ayan Datta, Zhixue Zhao, Bhuvanesh Verma +3Efficient Language Model ReasoningLLM Inference Acceleration

    IIIT Hyderabad · University of Sheffield, UK · Goethe University, Frankfurt am Main, Germany

  12. ResRank: Unifying Retrieval and Listwise Reranking via End-to-End Joint Training with Residual Passage Compression

    Apr 24, 2026Xiaojie Ke, Shuai Zhang, Liansheng Sun +6Learning to RankLLM Reranking

    Qwen Applications Business Group of Alibaba

  13. LayerBoost: Layer-Aware Attention Reduction for Efficient LLMs

    Apr 23, 2026Mohamed Ali Souibgui, Jan Fostier, Rodrigo Abadía-Heredia +3Transformer AttentionLLM Inference Acceleration

    Openchip & Software Technologies, S.L.

  14. SparKV: Overhead-Aware KV Cache Loading for Efficient On-Device LLM Inference

    Apr 23, 2026Hongyao Liu, Liuqun Zhai, Junyi Wang +3On-Device Language Model InferenceKV-Cache Offloading

    Department of Computer Science, City University of Hong Kong, Hong Kong SAR.

  15. FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compression

    Apr 22, 2026Ye Qiao, Yian Wang, Zhiheng Chen +2LLM CompressionGPU Acceleration

    University of California, Irvine Irvine, California, USA

  16. MCAP: Deployment-Time Layer Profiling for Memory-Constrained LLM Inference

    Apr 22, 2026Anurita DasMemory-Efficient OptimizationMemory-Efficient Inference

    Genovation Technological Solutions Pvt Ltd

  17. Simplified Sparse Attention via Gist Tokens

    Apr 22, 2026Yuzhen Mao, Michael Y. Li, Emily B. FoxSelf-AttentionLong-Context Language Model Inference

    Stanford University

  18. FairyFuse: Multiplication-Free LLM Inference on CPUs via Fused Ternary Kernels

    Apr 22, 2026Fei Zuo, Xiaoyan Xi, Quanyi Zeng +2Efficient Neural Network InferenceTernary Quantization

    BA TechWorks · Peking University

  19. Super Apriel: One Checkpoint, Many Speeds

    Apr 21, 2026SLAM Labs, :, Oleksiy Ostapenko +13LLM Inference AccelerationAttention Mechanisms

    SLAM Labs · ServiceNow Research · University of Toronto +4

  20. Micro Language Models Enable Instant Responses

    Apr 21, 2026Wen Cheng, Tuochao Chen, Karim Helwani +3On-Device Language Model InferenceSmall Language Models

    Paul G. Allen School of Computer Science & Engineering, University of Washington · Meta AI

  21. SimDiff: Depth Pruning via Similarity and Difference

    Apr 21, 2026Yuli Chen, Shuhao Zhang, Fanshen Meng +4LLM PruningLLM Inference Acceleration

  22. DASH-KV: Accelerating Long-Context LLM Inference via Asymmetric KV Cache Hashing

    Apr 21, 2026Jinyu Guo, Zhihan Zhang, Jiehui Xie +7Long-Context Language Model InferenceLLM Inference Acceleration

    School of Information and Software Engineering, University of Electronic Science and Technology of China · School of Computer Science and Engineering, University of Electronic Science and Technology of China · Bangladesh University of Engineering and Technology +2

  23. SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving

    Apr 21, 2026Jinda Jia, Jisen Li, Zhongzhu Zhou +8LLM ServingKV Caching

    Together AI

  24. Efficient Mixture-of-Experts LLM Inference with Apple Silicon NPUs

    Apr 20, 2026Afsara Benazir, Felix Xiaozhu LinAI Accelerator InferenceEnergy-Efficient ML

    University of Virginia Charlottesville, Virginia, USA

  25. River-LLM: Large Language Model Seamless Exit Based on KV Share

    Apr 20, 2026Yingtao Shen, An ZouDecoder-Only Language ModelsKV-Cache Management

    Shanghai Jiao Tong University Shanghai, China

  26. Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing

    Apr 20, 2026Ziyang LiuLLM Inference AccelerationParallel Decoding

    Independent Researcher

  27. AQPIM: Breaking the PIM Capacity Wall for LLMs with In-Memory Activation Quantization

    Apr 20, 2026Kosuke Matsushima, Yasuyuki Okoshi, Masato Motomura +1Compute-in-MemoryLong-Context Language Model Inference

    Institute of Science Tokyo†

  28. Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefilling

    Apr 20, 2026Yujie Chen, Tailai Chen, Yifeng Gao +4Self-AttentionLong-Context Language Model Inference

    Shanghai Jiao Tong University · University of California, San Diego · Carnegie Mellon University

  29. Unlocking the Edge deployment and ondevice acceleration of multi-LoRA enabled one-for-all foundational LLM

    Apr 20, 2026Sravanth Kodavanti, Sowmya Vajrala, Srinivas Miriyala +13On-Device Language Model InferenceLLM Inference

    Samsung Research Institute Bangalore, India · Samsung Electronics, Suwon, South Korea

  30. M100: An Orchestrated Dataflow Architecture Powering General AI Computing

    Apr 20, 2026Yan Xie, Changkui Mao, Changsong Wu +34AI Accelerator InferenceLLM Inference Acceleration

    Li Auto