Memory-Efficient Inference

Ask about this topic

Momentum

18 papers in the last four weeks, up 157% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 101

All topics
CardsList

  1. Gram-Space: Structure-Preserving Codebook Compression for Memory-Efficient Neuro-Symbolic AI

    Aug 2, 2026Weilun Wang, Wantong LiMemory-Efficient InferenceVector Symbolic Architectures

    University of California, Riverside

  2. SemPIC: Learning Semantic Position-Independent KV Caches

    Jul 30, 2026Hui Xie, Peng Xiao, Yutong Deng +3KV CachingMemory-Efficient Inference

    Institute of Artificial Intelligence, Beihang University · State Key Laboratory of Complex & Critical Software Environment · Shen Yuan Honors College, Beihang University +1

  3. WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization

    Jul 30, 2026Fanzhe Wei, Li Liu, Ziyang Wang +1KV CachingMemory-Efficient Inference

    Metask Lab

  4. Memory Efficient Tabular Foundation Models

    Jul 30, 2026Shuting Luo, Monika Mikhail Kanaan, Cameron Gordon +2Model CompressionTabular Foundation Models

    Commonwealth Bank of Australia, Australia · Australian Institute for Machine Learning, University of Adelaide, Adelaide, Australia

  5. LLMET: Enabling Cross-Layer Evaluation of Emerging M3D Memories for Energy-Efficient LLM Serving

    Jul 29, 2026Ming-Yen Lee, Hanchen Yang, Faaiq Waqar +4LLM Inference EfficiencyLLM Serving

    Georgia Institute of Technology, Atlanta, GA, USA · Samsung Semiconductor Inc., San Jose, CA, USA

  6. Eviction as Estimation: A Fixed-Lag Smoothing View of Test-Time Memory, and When Measuring Beats Accumulating

    Jul 27, 2026Maruthi Vemula, Neeraj Praneeth GajulaKV CachingKV-Cache Eviction

    University of North Carolina at Chapel Hill · New York University

  7. RIS-Kernel: A Model-Agnostic Architecture for Long-Context LLM Inference via Sparse Attention

    Jul 24, 2026Anderson R. SantosLong-Context Language Model InferenceMemory-Efficient Inference

    Faculty of Computing, Federal University of Uberlândia (UFU), Brazil.

  8. Lean-SAM2: Target-Anchored Memory and Encoder Acceleration for SAM2

    Jul 22, 2026Xudong Ouyang, Wenlun Zhang, Yimin Xu +2Memory-Augmented Neural NetworksVideo Object Segmentation

    School of Biomedical Engineering, Hainan University, Hainan 570228, China · Department of Electronics and Electrical Engineering, Keio University, Kanagawa 223-8522, Japan · College of Computer Science, Shenyang Aerospace University, Shenyang 110135, China +1

  9. SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs

    Jul 20, 2026Huzaifa Shaaban Kabakibo, Eric Schniedermeyer, Artem Burchanow +1On-Device Language Model InferenceMemory-Efficient Inference

    Department of Computer Networks, Paderborn University, Paderborn, Germany

  10. Taurus: Accelerating Out-of-Core Graph Neural Network Inference on Billion-Scale Graphs

    Jul 19, 2026Pranjal Naman, Yogesh SimmhanEfficient Neural Network InferenceGraph Neural Networks

    Department of Computational and Data Sciences (CDS), Indian Institute of Science (IISc), Bangalore 560012 India

  11. SlimPer: Make Personalization Model Slim and Smart

    Jul 14, 2026Siqi Wang, Xianjie Chen, Shaofeng Deng +42Learning to RankMemory-Efficient Inference

  12. On Exploring Input Resolution Scaling For Anytime LiDAR Object Detection

    Jul 9, 2026Ahmet Soyyigit, Shuochao Yao, Heechul YunAdaptive InferenceLidar-Based 3D Object Detection

    The National Defense University, Istanbul, Turkiye · George Mason University, Fairfax, VA, USA · The University of Kansas, Lawrence, KS, USA

  13. Akashic: A Low-Overhead LLM Inference Service with MemAttention

    Jul 7, 2026Yang Liu, Zhaokai Luo, Huayi Jin +7LLM ServingMemory-Augmented Language Models

    Xiaohongshu Inc., China · ShangHai JiaoTong University · Peking University

  14. MiLSD: A Micro Line-Segment Detector for Resource-Constrained Devices

    Jul 7, 2026Parsa Hassani Shariat Panahi, Amir Hossein Jalilvand, M. Hassan NajafiTiny MLMemory-Efficient Inference

  15. AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference

    Jul 4, 2026Sadra SaremiOn-Device Language Model InferenceMemory-Efficient Inference

    Department of Physics, Iran University of Science and Technology, Tehran 16846-13114, Iran.

  16. The risk of KV cache compression

    Jul 1, 2026Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno +2Transformer InferenceKV Caching

    RWTH Aachen University · Stanford University

  17. Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM

    Jun 28, 2026Shuvendu Roy, Mengyao Zhai, Hossein Hajimirsadeghi +1Long-Context Language Model InferenceKV-Cache Eviction

    RBC Borealis

  18. Affix Cache for Diffusion Large Language Models

    Jun 26, 2026Kaihua Liang, An Zhong, Xin Tan +4KV CachingMemory-Efficient Inference

    KAUST · CUHK · LUMS

  19. Geometry-Aware Online Scheduling for LLM Serving: From Theoretical Bound to System Practice

    Jun 21, 2026Li Kong, Qi Qi, Yinyu Ye +1LLM ServingLLM Inference Scheduling

    Gaoling School of Artificial Intelligence Renmin University of China Beijing, China · Department of Management Science and Engineering Stanford University Stanford, CA 94305-4026 · Department of Industrial Engineering and Decision Analytics HKUST Hongkong, China

  20. WiSP: A Working-Set View of Mixture-of-Experts Serving on Extremely Low-Resource Hardware

    Jun 20, 2026Jiamu Zhang, Liang Wu, Mayank Darbari +1Expert OffloadingMixture-of-Experts Inference

    Nokia, Sunnyvale, CA 94085, USA

  21. Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers

    Jun 20, 2026Xin GaoTransformer InferenceKV Caching

    Department of Mathematics and Statistics York University Toronto, M3J 1P3

  22. Memory Is No Longer a Bottleneck: Memory-Efficient Graph Filtering for Scalable Collaborative Filtering

    Jun 19, 2026Jin-Duk Park, Won-Yong ShinMemory-Efficient InferenceKrylov Subspace Methods

    School of Mathematics and Computing (Computational Science and Engineering), Yonsei University, Seoul 03722, Republic of Korea

  23. MawForge: Memory-Bounded Expert Materialization for Local Mixture-of-Experts Inference

    Jun 17, 2026Craig OpieExpert OffloadingOn-Device Language Model Inference

    Holocron Security, Inc.

  24. Sticky Routing: Training MoE Models for Memory-Efficient Inference

    Jun 12, 2026Ali KayyamMixture-of-Experts InferenceMemory-Efficient Inference

    BrainChip Inc., Laguna Hills, CA

  25. VQ4SNN: Vector Quantization for Memory-Efficient FPGA Spiking Neural Networks

    Jun 12, 2026Dimitrios Sekertzis, Giorgos DimitrakopoulosMemory-Efficient InferenceVector Quantization

    Integrated Circuits Lab, Electrical and Computer Engineering, Democritus University of Thrace (DUTH), Greece

  26. Recursive Binding on a Budget: Subspace Carving in Order-p Tensor Memories

    Jun 9, 2026Travis Pence, Daisuke Yamada, Vikas SinghMemory-Efficient InferenceVector Symbolic Architectures

    Department of Computer Science, University of Wisconsin-Madison, United States · Department of Biostatistics and Med. Info., University of Wisconsin-Madison, United States

  27. FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention

    Jun 8, 2026Yan Wang, Qifan Zhang, Jiachen Yu +12KV CachingLong-Context Language Model Inference

    Independent Researchers · Tencent · The Hong Kong University of Science and Technology (Guangzhou) +1

  28. Operator Fusion for LLM Inference on the Tensix Architecture

    Jun 3, 2026Qingbo Wu, Ke Li, Wenzhu Wang +3Efficient Transformer InferenceAI Accelerator Inference

    KylinSoft Co., Ltd, Tianjin, China · openKylin Community, Tianjin, China · College of Computer Science and Technology, National University of Defense Technology, Changsha, China

  29. PrimeSVT: An Automated Memory-aware Pruning Framework with Prioritized Compression Policy for Spiking Vision Transformers

    Jun 2, 2026Rachmad Vidya Wicaksana Putra, Achyuta Muthuvelan, Alberto Marchisio +1Memory-Efficient InferenceSpiking Neural Networks

    eBRAIN Lab, Division of Engineering, New York University (NYU) Abu Dhabi, United Arab Emirates (UAE)

  30. AURA: Action-Gated Memory for Robot Policies at Constant VRAM

    Jun 1, 2026Josef ChenVisuomotor Policy LearningMemory-Efficient Inference

    KAIKAKU