Memory-Efficient Optimization

Follow this topic Ask about this topic

Momentum

1 paper in the last four weeks, down 83% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 64

All topics

Value-Aware Stochastic KV Cache Eviction for Reasoning Models
Jun 2, 2026Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu +3LLM Inference Efficiency KV Caching
University of Southern California · University of Chicago
GRZO: Group-Relative Zeroth-Order Optimization for Large Language Model Fine-Tuning
Jun 1, 2026Liyan Tan, Yequan Zhao, Yifan Yang +3Fine-Tuning Memory-Efficient Fine-Tuning
University of California, Santa Barbara
LoRe: Adaptive Interaction-Evaluation Routing with Per-Step Interaction Budgets for Iterative Graph Solvers
May 27, 2026Jintao Li, Yong-Yi Wang, Zheng-An Wang +1Efficient Inference Inference-Time Optimization
Beijing Key Laboratory of Fault-Tolerant Quantum Computing, Beijing Academy of Quantum Information Sciences, Beijing, China · Beijing National Laboratory for Condensed Matter Physics, Institute of Physics, CAS, Beijing, China · Beijing Key Laboratory of Advanced Quantum Technology, Beijing, China +1
GONDOR to the Rescue: Satisficing Planning with Low Memory
May 27, 2026Yonatan Vernik, Alexander Tuisov, Alexander ShleyfmanTree Search Memory-Efficient Optimization
Computer Science Department, Bar-Ilan University · Independent Researcher
ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inference
May 26, 2026Xiongwei Zhu, Xiaojian Liao, Tianyang Jiang +3LLM Inference Efficiency Expert Offloading
School of Computer Science and Engineering, Beihang University, Beijing 100191, China · Huawei Technologies Ltd.
Reparametrizing Shampoo and SOAP for Subspace Basis Updates and BFloat16 Storage
May 25, 2026Alan Milligan, Zikun Xu, Simon Lacoste-Julien +2Deep Learning Optimization Memory-Efficient Optimization
Mila & Université de Montréal · Microsoft · Concordia University & Mila +1
A Simple Plug-in for Improving Eviction-Based KV Cache Compression
May 22, 2026Yuping Lin, Jiayuan Ding, Yue Xing +3KV Caching Memory-Efficient Optimization
1Michigan State University · 2Hippocratic AI
Asymmetric Virtual Memory Paging for Hybrid Mamba-Transformer Inference
May 21, 2026An Xuan NguyenKV Caching Memory-Efficient Optimization
Codepawl Ho Chi Minh City, Vietnam
ArborKV: Structure-Aware KV Cache Management for Scaling Tree-based LLM Reasoning
May 21, 2026Yeqiu Chen, Ziyan Liu, Zhenxin Huang +3Inference-Time Search Tree of Thoughts
University of Science and Technology of China
OCTOPUS: Optimized KV Cache for Transformers via Octahedral Parametrization Under optimal Squared error quantization
May 20, 2026Mark Boss, Vikram Voleti, Simon Donné +1KV Caching Memory-Efficient Optimization
Stability AI
ChunkFT: Byte-Streamed Optimization for Memory-Efficient Full Fine-Tuning
May 20, 2026Yongkang Liu, Zijing Wang, Mengjie Zhao +7Fine-Tuning Memory-Efficient Fine-Tuning
Northeastern University, China · Shanghai Jiao Tong University, China · CIS, LMU Munich, Germany +2
VeriCache: Turning Lossy KV Cache into Lossless LLM Inference
May 17, 2026Jiayi Yao, Samuel Shen, Kuntai Du +7KV-Cache Offloading KV Caching
University of Chicago · Tensormesh Inc. · Samsung Semiconductor +1
Self-Pruned Key-Value Attention: Learning When to Write by Predicting Future Utility
May 13, 2026Gergely Szilvasy, Manuel Faysse, Maria Lomeli +5Self-Attention KV Caching
Meta FAIR · MICS, CentraleSupélec
Compander-Aligned Query Geometry for Quantized Zeroth-Order Optimization
May 11, 2026Yao Shu, Zilin ZhuLLM Quantization Zeroth-Order Optimization
HKUST-GZ
PowerStep: Memory-Efficient Adaptive Optimization via $\ell_p$ -Norm Steepest Descent
May 11, 2026Yao Lu, Dengdong Fan, Shixun Zhang +1Gradient Descent Memory-Efficient Optimization
Pengcheng Laboratory · Peking University
Not All Thoughts Need HBM: Semantics-Aware Memory Hierarchy for LLM Reasoning
May 10, 2026Aojie Yuan, Tianqi Shen, Dajun ZhangKV-Cache Offloading KV Caching
University of Southern California · †University of Wisconsin–Madison
Navigating LLM Valley: From AdamW to Memory-Efficient and Matrix-Based Optimizers
May 9, 2026Aditya RanganathDeep Learning Optimization Memory-Efficient Optimization
Center for Applied Scientific Computing Lawrence Livermore National Laboratory Livermore, CA, USA
Budget-aware Auto Optimizer Configurator
May 6, 2026Kang Liu, Wei Peng, Jianchen HuDeep Learning Optimization Memory-Efficient Optimization
School of Future Technology Xi’an Jiaotong University Xi’an, China · School of Automation Science and Engineering Xi’an Jiaotong University Xi’an, China
On Adaptivity in Zeroth-Order Optimization
May 5, 2026Hassan Dbouk, Nidham Gazagnadou, Matthias Reisser +1Zeroth-Order Optimization LLM Fine-Tuning
Qualcomm AI Research
StreamIndex: Memory-Bounded Compressed Sparse Attention via Streaming Top-k
May 4, 2026Jaber Jaber, Osama JaberGPU Kernel Optimization Self-Attention
RightNow AI
AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments
May 1, 2026Zhijie Cai, Haolong Chen, Guangxu ZhuMemory-Efficient Fine-Tuning Zeroth-Order Optimization
Shenzhen Research Institute of Big Data · The Chinese University of Hong Kong-Shenzhen · Shenzhen Loop Area Institute
MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
Apr 24, 2026Xin Wang, Chi Ma, Shaobin Chen +14KV-Cache Offloading KV Caching
School of Computer Science, Wuhan University · Meituan · Nvidia
MCAP: Deployment-Time Layer Profiling for Memory-Constrained LLM Inference
Apr 22, 2026Anurita DasMemory-Efficient Optimization Memory-Efficient Inference
Genovation Technological Solutions Pvt Ltd
LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction
Apr 22, 2026Enshuai Zhou, Yifan Hao, Chao Wang +7KV Caching Long-Context Language Model Inference
University of Science and Technology of China, Hefei, China · State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
RateQuant: Optimal Mixed-Precision KV Cache Quantization via Rate-Distortion Theory
Apr 22, 2026Fei Zuo, Zikang Zhou, Hao Cong +2LLM Inference Efficiency Mixed-Precision Quantization
1BA TechWorks (BMW Group) · 2National University of Singapore · 3Tsinghua University
SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving
Apr 21, 2026Jinda Jia, Jisen Li, Zhongzhu Zhou +8LLM Serving KV Caching
Together AI
Neural Garbage Collection: Learning to Forget while Learning to Reason
Apr 20, 2026Michael Y. Li, Jubayer Ibn Hamid, Emily B. Fox +1LLM Compression Memory-Efficient Optimization
Stanford University
MoE-nD: Per-Layer Mixture-of-Experts Routing for Multi-Axis KV Cache Compression
Apr 20, 2026Libo Sun, Peixiong He, Po-Wei Harn +1LLM Inference KV Caching
Department of Computer Science and Software Engineering, Auburn University, Auburn, AL, USA · Department of Information Management, National Central University, Taoyuan, Taiwan
MemoSight: Unifying Context Compression and Multi Token Prediction for Reasoning Acceleration
Apr 16, 2026Xinyu Liu, Xin Liu, Bo Jin +8Multi-Token Prediction KV Caching
School of Computer Science and Engineering, Northeastern University, China · Meituan Inc. · NiuTrans Research, Shenyang, China
BudgetMem: Training-Free Selective Memory for Cost-Efficient Long-Context Processing in Language Models
Nov 7, 2025Chandra Vamsi Krishna Alla, Harish Naidu Gaddam, Manohar Kommi +1LLM Compression Long-Context Language Model Inference
University of Texas at Arlington

Jun 2, 2026, cs.LGJ/K move · Enter open · S save

Value-Aware Stochastic KV Cache Eviction for Reasoning Models

Ting-Yun Chang, Harvey Yiyun Fu, Deqing Fu, Chenghao Yang+2

University of Southern California · University of Chicago

LLM Inference Efficiency KV Caching Memory-Efficient Optimization KV-Cache Eviction KV-Cache Compression Large Reasoning Models

Open paper AskPDF arXiv

Reasoning models improve accuracy through extended chains of thought, but their long outputs create a memory and compute bottleneck. KV cache eviction methods reduce this cost by evicting unimportant key-value pairs from the cache, yet they often yield worse accuracy than selection-based sparse attention alternatives, which keep the full KV cache. We identify key factors crucial to KV cache eviction accuracy. First, a small fraction of value states have abnormally large magnitudes, and evicting them causes catastrophic failure where models enter repetitive reasoning loops. Second, introducing stochasticity during eviction improves accuracy by increasing cache diversity. Based on these findings, we propose Value-aware Stochastic KV Cache Eviction (VaSE), a training-free recipe that protects large-magnitude value states and promotes diverse eviction decisions. Across six reasoning tasks, Qwen3 models using VaSE with 4x KV cache compression yield higher average accuracies than SOTA selection method at the same sparsity, while outperforming the strongest eviction method by more than 4%. Overall, VaSE bridges the gap between efficiency and accuracy, supporting FlashAttention2 and enabling a static memory footprint for reasoning models.

Momentum

Related topics

Latest papers 64