Graphics Processing Unit Memory

Follow this topic Ask about this topic

Momentum

8 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 40

All topics

KV-streams for Efficient Compaction in Agentic Reinforcement Learning
Sep 28, 2026Emiliano Penaloza, Dane Malenfant, Dheeraj Vattikonda +15Compaction Agentic Reinforcement Learning
Mila · Microsoft · Université de Montréal +9
Tetra: Serving Leech-Lattice Quantized LLMs at 2.7 Bits per Parameter
Sep 28, 2026Pier-Jean MalandrinoLarge Language Model Quantization Model Weights
Scub, Bordeaux, France
DPS: Dual-Mode Precision LLM Serving with Semi-Unified Memory
Sep 28, 2026Xuan Truong Nguyen, Tien Son Pham, Tuan Duc Chu +2Graphics Processing Unit Memory Kv-Cache Management
Department of Next Generation Semiconductor Convergence and Open Sharing System (COSS), Seoul National University, Seoul 08826, South Korea · Van-Lang Institute of Semiconductor Technology (VIST), Hanoi, Vietnam · Efficient Computation Research Group, Moreh Vietnam +1
Does Execution Require Target KV Fidelity? A Mixed-Fidelity KV Runtime for LLM Serving
Sep 27, 2026Jiantong Jiang, Yue Yang, Peiyu Yang +1Large Language Model Serving Keys And Value
University of Melbourne · Maincode
OLED-MoE: Accelerating MoE-Based dLLM Inference via Inter-Iteration Locality-Aware Expert Offloading
Sep 27, 2026Jingyuan Xiao, Jiayue Wang, Yitao Hu +7Mixture-Of-Expert Inference Offloading
Tianjin University, Tianjin, China · Tianjin University Tianjin, China
Ask the Tool, Don't Guess: Agent Tool Calls Hold Their Progress, and the Serving System Should Read It
Sep 16, 2026Yipeng Liu, Yingqiang Zhang, Feifei Li +1Call Key-Value Cache
Tsinghua University, China · Alibaba Cloud Computing, China · Zhejiang University, China
Layer-wise Curriculum Learning for Efficient LLM Compression
Sep 16, 2026Donggeon Lee, Dooyeon Na, Seungmin Oh +1Large Language Model Compression Adaptive Curriculum
Ajou University, South Korea
Rethinking Heterogeneous System Disaggregation for Subquadratic Attention
Sep 14, 2026Arya Tschand, Yaosheng Fu, Vikram Sharma Mailthody +6Linear Attention Graphics Processing Unit Memory
Harvard University Cambridge, Massachusetts, USA · NVIDIA Santa Clara, California, USA
LazyTrain: Limited-resource Allocation toward Zero-waste Yield Optimization in Large Language Model Training
Aug 12, 2026Xiaojun Wu, Cehao Yang, Honghao Liu +5Large Language Model Training Graphics Processing Unit Memory
1IDEA Research · 2The Hong Kong University of Science and Technology (Guangzhou) · 3DataArcTech Ltd.
SPECTRA: Pushing the KV Cache Beyond the 2-Bit Cliff via Spectral Transform Coding
Aug 8, 2026Jiamu Zhang, Liang Wu, Kelly Wan +2Key-Value Cache Cache
Nokia · Rice University
Pin Once, Swap Light: Subspace-Aligned Centroid-Residual Training for Efficient Ultra-LoRA Serving
Aug 4, 2026Xiang Li, Pengcheng Wang, Huazheng Wang +1Low-Rank Adaptation Adapters Hip-Lora
1Purdue University · 2Oregon State University
Gram-Space: Structure-Preserving Codebook Compression for Memory-Efficient Neuro-Symbolic AI
Aug 2, 2026Weilun Wang, Wantong LiNeuro-Symbolic Framework Codebooks
University of California, Riverside
PagedWeight: Efficient MoE LLM Serving with Dynamic Quality-Aware Weight Quantization
Jul 17, 2026Yuchen Yang, Yifan Zhao, Anisha Dasgupta +1Graphics Processing Unit Memory Mixture-Of-Experts Large Language Models
University of Illinois Urbana-Champaign
EMAGN: Efficient Multi-Attention Graph Network via Learned Clustering for Scalable Traffic Forecasting
Jul 14, 2026Mingxing Xu, Rakesh Chowdary Machineni, Ke Liu +7Interleaved Graph Attention Accurate Traffic Forecasting
Shanghai Jiao Tong University · University of Michigan, Ann Arbor · University of California, Berkeley +5
MosaicKV: Serving Long-Context LLM with Dynamic Two-D KV Cache Compression
Jul 1, 2026Sheng Qiang, Ruiwei Chen, Yinpeng Wu +5Key-Value Cache Compression Large Language Model Compression
Institute of Parallel and Distributed Systems, Shanghai Jiao Tong University, Shanghai, China
SeKV: Resolution-Adaptive KV Cache with Hierarchical Semantic Memory for Long-Context LLM Inference
Jun 30, 2026Amirhossein Abaskohi, Giuseppe Carenini, Peter West +1Key-Value Cache Compression Key-Value Cache
University of British Columbia · Microsoft Research
CrossPool: Efficient Multi-LLM Serving for Cold MoE Models through KV-Cache and Weight Disaggregation
Jun 23, 2026Zhuoren Ye, Tianyu Wo, Dinghao Xue +4Kv-Cache Management Graphics Processing Unit Memory
School of Software, Beihang University
GRINQH: Graded Input-based Quantization Hierarchy for Efficient LLM Generation
Jun 22, 2026Jette Oberländer, Jan Finkbeiner, Catherine M. Schöfmann +1Large Language Model Quantization Gradient Quantization
Fakultät für Informatik, RWTH Aachen, Aachen, 52074, Germany · Peter Grünberg Institut, Forschungszentrum Jülich GmbH, Jülich, 52425, Germany · Fakultät für Elektrotechnik und Informationstechnik, RWTH Aachen, Aachen, 52062, Germany
EvoGS: Constructing Continuous-Layered Gaussian Splatting with Evolution Tree for Scalable 3D Streaming
Jun 5, 2026Yuang Shi, Simone Gasparini, Géraldine Morin +1Gaussian Splatting Volumetric Video
National University of Singapore, Singapore · IRIT - University of Toulouse, France · IPAL, IRL2955, Singapore
RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention
Jun 4, 2026Yang Liu, ZhaoKai Luo, HuaYi Jin +5Kv-Cache Management Key-Value Cache
†Xiaohongshu Inc., China · ¶Huawei Cloud · §Peking University
STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language Models
Jun 1, 2026Yuhang Han, Wenzheng Yang, Yujie Chen +4Key-Value Cache Compression Graphics Processing Unit Memory
1EPIC Lab, SJTU · 2HKUST (GZ) · 3The University of Sydney +3
BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding
May 29, 2026Liang He, Jingbo Wen, Qishi Zhan +4Speculative Decoding Graphics Processing Unit Memory
Shanghai Institute of Optics and Fine Mechanics · The University of Sydney · Marquette University +4
Reducing the GPU Memory Bottleneck with Lossless Compression for ML -- Extended
May 29, 2026Aditya K Kamath, Arvind Krishnamurthy, Marco Canini +1Graphics Processing Unit Memory LLM Inference Optimization
University of Washington Seattle, WA, USA · KAUST Thuwal, Saudi Arabia
Memory-Bound but Not Bandwidth-Limited: The Physical AI Inference Gap in Batch-1 LLM Decode
May 28, 2026Josef ChenLLM Inference Optimization Graphics Processing Unit Memory
KAIKAKU
CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM
May 24, 2026Yubo Li, Yidi MiaoKv-Cache Management Key-Value Cache Eviction
Carnegie Mellon University
Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption
May 19, 2026Mert Yildiz, Pietro Spadaccino, Alexey Rolich +2Preemption Offloading
Sapienza University of Rome, Rome, Italy
KVDrive: A Holistic Multi-Tier KV Cache Management System for Long-Context LLM Inference
May 18, 2026Jian Lin, Jiazhi Mi, Zicong Hong +5Kv-Cache Management Key-Value Cache
Hong Kong University of Science and Technology, China · Xi’an Jiaotong University, China
VeriCache: Turning Lossy KV Cache into Lossless LLM Inference
May 17, 2026Jiayi Yao, Samuel Shen, Kuntai Du +7Key-Value Cache LLM Inference Optimization
University of Chicago · Tensormesh Inc. · Samsung Semiconductor +1
A Few GPUs, A Whole Lotta Scale: Faithful LLM Training Emulation with PrismLLM
May 15, 2026Shaoke Xi, ChonLam Lao, Boyi Jia +11Large Language Model Training Large Language Models(Llms
Alibaba Group · Harvard University · Shanghai Jiao Tong University +1
OOM-Free Alpamayo via CPU-GPU Memory Swapping for Vision-Language-Action Models
May 12, 2026Seungwoo Roh, Huiyeong Kim, Jong-Chan KimDiffusion-Based Vision-Language-Actions Graphics Processing Unit Memory
Graduate School of Automobile and Mobility, Kookmin University, Korea
ChunkFlow: Communication-Aware Chunked Prefetching for Layerwise Offloading in Distributed Diffusion Transformer Inference
May 11, 2026Han Meng, Danny Willow Liu, Dong LiOffloading Preemption
University of California, Merced · University of Chicago · University of California, Merced and Yotta Labs +1
Budget-aware Auto Optimizer Configurator
May 6, 2026Kang Liu, Wei Peng, Jianchen HuOptimizer Design Graphics Processing Unit Memory
School of Future Technology Xi’an Jiaotong University Xi’an, China · School of Automation Science and Engineering Xi’an Jiaotong University Xi’an, China
A Queueing-Theoretic Framework for Stability Analysis of LLM Inference with KV Cache Memory Constraints
May 6, 2026Chengyi Nie, Nian Si, Zijie ZhouLLM Inference Optimization Graphics Processing Unit Memory
Department of Electrical and Computer Engineering, Stony Brook University · Department of Industrial Engineering and Decision Analytics, HKUST
AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments
May 1, 2026Zhijie Cai, Haolong Chen, Guangxu ZhuAdam Zeroth-Order Optimization
Shenzhen Research Institute of Big Data · The Chinese University of Hong Kong-Shenzhen · Shenzhen Loop Area Institute
AGoQ: Activation and Gradient Quantization for Memory-Efficient Distributed Training of LLMs
May 1, 2026Wenxiang Lin, Juntao Huang, Luhan Zhang +5Large Language Model Quantization Gradient Quantization
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen · Huawei Technologies Ltd.
Lightweight Distillation of SAM 3 and DINOv3 for Edge-Deployable Individual-Level Livestock Monitoring and Longitudinal Visual Analytics
Apr 29, 2026Haiyu Yang, Miel HostensDinov3 Vision Encoders
College of Agriculture and Life Sciences, Cornell University, Ithaca, NY 14853 USA
AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
Apr 28, 2026Zhongkai Yu, Haotian Ye, Chenyang Zhou +9Graphics Processing Unit Memory Linear Attention
University of California, San Diego · Columbia University · Yonsei University +2
MTServe: Efficient Serving for Generative Recommendation Models with Hierarchical Caches
Apr 24, 2026Xin Wang, Chi Ma, Shaobin Chen +14Sequential Recommendation Graphics Processing Unit Memory
School of Computer Science, Wuhan University · Meituan · Nvidia
Stream-CQSA: Exact Out-of-Memory Recovery for Attention
Apr 22, 2026Yiming Bian, Joshua M. AkeyGraphics Processing Unit Memory
Lewis-Sigler Institute of Integrative Genomics, Princeton University, Princeton, NJ 08540
Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines
Apr 17, 2026Junwan Kim, Hyunkyung BaeMultimodal Large Language Models LLM Inference Optimization
New York University

Sep 28, 2026, cs.LGJ/K move · Enter open · S save

KV-streams for Efficient Compaction in Agentic Reinforcement Learning

Emiliano Penaloza, Dane Malenfant, Dheeraj Vattikonda, Roger Creus Castanyer+14

Mila · Microsoft · Université de Montréal+9

Compaction Agentic Reinforcement Learning Graphics Processing Unit Memory Streaming Context Length

Open paper AskPDF arXiv

Scaling the horizon of agentic LLMs is bottlenecked by the need to fit ever longer context traces in GPU memory. Context compaction has been the most popular mechanism to alleviate this issue, keeping GPU memory constant for a given trace. Unfortunately, most compaction strategies rely on prefilling the LLM context many times over, hindering training throughput. To alleviate this bottleneck and enable efficient trainable compaction, we propose KV-streams, a plug-and-play strategy compatible with any compaction strategy that substantially increases throughput while showing no evidence of hindering performance. KV-streams enable scalable compaction by streaming the KV cache forward rather than flushing it after each compaction. We show that KV-streams enable three different compaction strategies, achieving a 2.6 to 5x wall-clock speedup in training. Beyond efficiency, we find that the streamed KV cache can act as a recurrent state, carrying forward information that has long since disappeared from the context. Specifically, in a controlled setting we show that, contrary to prior work, RL alone is all that is needed for this behavior to emerge. Overall, we show KV-streams to be an efficient and lightweight plug-and-play addition to any post-training pipeline.

Momentum

Related topics

Latest papers 40