Memory-Efficient Fine-Tuning

Ask about this topic

Momentum

5 papers in the last four weeks, up 67% on the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 33

All topics
CardsList

  1. MemFLoRA: Memory-Floor LoRA for CNN Adaptation at the Edge

    Oct 6, 2026Mehmet Emre Akbulut, Johannes Geier, Ulf SchlichtmannOn-Device TrainingMemory-Efficient Fine-Tuning

    Technical University of Munich Munich, Germany

  2. LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches

    Oct 5, 2026Shaokun Zhang, Yifan Zhang, Jian Hu +5Memory-Efficient Fine-TuningRL for Language Model Reasoning

    NVIDIA

  3. RoSA: Rotational Sparse Adaptation for Memory-Efficient Fine-Tuning

    Oct 5, 2026Muhammad Azeem Lodhi, Chao Zhou, Rebekka BurkholzMemory-Efficient Fine-TuningLLM Fine-Tuning

    Saarland University, Saarbrücken, Germany · CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

  4. TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning

    Oct 1, 2026Jichao Jiang, Cristian McGee, El Houcine Bergou +2Memory-Efficient Fine-TuningLLM Fine-Tuning

    University of Central Florida · Mohammed VI Polytechnic University

  5. FedLore: Communication and Memory Efficient Federated Learning via Shared Gradient Low-Rank Projection

    Oct 1, 2026Junkang LiuFederated Learning AggregationMemory-Efficient Fine-Tuning

    Tianjin University

  6. AIM-ZO: Activation-Informed Subspace Maintenance for Zeroth-Order LLM Fine-Tuning

    Sep 28, 2026Yue Xie, Zhi Zheng, Yunpeng Ba +5Memory-Efficient Fine-TuningZeroth-Order Optimization

    Southern University of Science and Technology · National University of Singapore · Huawei Technologies Ltd. +1

  7. TerMeZO: Ternary Sparse Zeroth-Order Optimization for Fine-tuning BitNet Models at the Edge

    Sep 27, 2026Houssem Sifaou, Prabodh Katti, Bipin Rajendran +1Memory-Efficient Fine-TuningZeroth-Order Optimization

    Institute for Intelligent Networked Systems, Northeastern University London, UK

  8. MpSub: A Momentum pp-Dimensional Subspace Trust-Region Method for Derivative-Free Fine-Tuning of Large Language Models

    Sep 7, 2026Yuyang Wang, Haoyu Yao, Pengcheng XieMemory-Efficient Fine-TuningZeroth-Order Optimization

    Xi’an Jiaotong University, Xi’an, China · Lawrence Berkeley National Laboratory, Berkeley, CA, USA

  9. Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation

    Sep 1, 2026Wentao Ye, Zhanming Shen, Zhiqing Xiao +3Memory-Efficient Fine-TuningLLM Fine-Tuning

    Zhejiang University · Hunyuan, Tencent

  10. SubZero+: Memory-Efficient Adaptive Zeroth-Order LLM Fine-Tuning in Random Subspaces

    Aug 16, 2026Ziming Yu, Shuyao Xiao, Xingyu Zhao +6Memory-Efficient Fine-TuningZeroth-Order Optimization

    Beijing Normal University · Singapore Management University · Xiaomi Inc. +2

  11. Curvature-Aware Zeroth-Order Optimization for Memory-Efficient Test-Time Adaptation

    Aug 12, 2026Junming Zhang, Shuyu Yin, Peilin Liu +2On-Device TrainingTest-Time Adaptation

    Shanghai Jiao Tong University, China

  12. ZeroLock: Concurrent Memory-Efficient LLM Training via Modular Update Decoupling

    Aug 8, 2026Wentao Dai, Xuanran Li, Yuxiang Zhang +2On-Device TrainingMemory-Efficient Fine-Tuning

    Department of Computer Science and Engineering, Southern University of Science and Technology, China · Department of Mathematics, Southern University of Science and Technology, China · School of Computing, Montclair State University, New Jersey, USA

  13. Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUs

    Aug 5, 2026Kuanysh Akhmetzhanov, Jurn-Gyu ParkFine-TuningOn-Device Language Model Inference

    Department of Computer Science, Nazarbayev University, Astana 010000, Kazakhstan

  14. Pin Once, Swap Light: Subspace-Aligned Centroid-Residual Training for Efficient Ultra-LoRA Serving

    Aug 4, 2026Xiang Li, Pengcheng Wang, Huazheng Wang +1LLM Inference EfficiencyLLM Serving

    1Purdue University · 2Oregon State University

  15. Noise-Aware Shrinkage for Differentially Private Zeroth-Order Fine-Tuning of Large Language Models

    Aug 4, 2026Lele Zheng, Weifeng Kong, Xinyi Zhang +3Memory-Efficient Fine-TuningPrivacy-Preserving Language Models

    School of Computer Science and Technology, Xidian University

  16. How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model

    Jul 28, 2026Mahendra Singh Rathor, Anagheem AzzamMemory-Efficient Fine-TuningSmall Language Models

    Independent Researcher · Jaipur, India · Hamburg, Germany

  17. \k{appa}-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating

    Jul 24, 2026Jianghui Wang, Silong Yong, Francesco Orabona +3Fine-TuningMemory-Efficient Fine-Tuning

    King Abdullah University of Science and Technology · Carnegie Mellon University

  18. Long-Context Fine-Tuning with Limited VRAM

    Jul 16, 2026Vladimir Fedosov, Aleksandr Sazhin, Artemiy Grinenko +1Fine-TuningKV-Cache Offloading

    BMW Group

  19. CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA

    Jul 11, 2026Gengyu Zhang, Haiyin Ran, Zhengbao He +4Fine-TuningMemory-Efficient Fine-Tuning

    Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai 200240, China

  20. Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning

    Jul 10, 2026Ivan Ilin, Philip Zmushko, Peter RichtárikMemory-Efficient Fine-TuningLow-Rank Adaptation

    KAUST

  21. Efficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUs

    Jul 2, 2026Altay Toktassyn, Jurn-Gyu ParkFine-TuningMemory-Efficient Fine-Tuning

    Department of Computer Science, Nazarbayev University, Astana 010000, Kazakhstan

  22. InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories

    Jul 2, 2026Qianyu Chen, Ziteng Feng, Canran Xiao +1Continual Learning for LLMsMemory-Efficient Fine-Tuning

    1Nanyang Technological University · University of Science and Technology of China · 3Shenzhen Campus of Sun Yat-sen University

  23. Latent Personal Memory: Represent personal memory as dynamic soft prompts

    Jun 18, 2026Debrup Das, Avinash Amballa, Yashas Malur Saidutta +3LLM Inference EfficiencyMemory-Efficient Fine-Tuning

    University of Massachusetts Amherst · Samsung Research America

  24. Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices

    Jun 17, 2026Hassan Dbouk, Matthias Reisser, Prathamesh Mandke +2Memory-Efficient Fine-TuningLLM Fine-Tuning

    Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc. · work done while at Qualcomm

  25. How Small Can You Go? LoRA Fine-Tuning 270M-8B Models for Merchant Information Extraction in Financial Transactions

    Jun 6, 2026Donghao Huang, Tomas Drietomsky, Benjamin Barrett +1Financial ServicesMemory-Efficient Fine-Tuning

    Research and Development, Mastercard, Arlington, VA, USA · School of Computing and Information Systems, Singapore Management University, Singapore · Economic Intelligence, Mastercard, Prague, Czech Republic

  26. Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs

    Jun 3, 2026Wanhao Yu, Ziyan Wang, Zheng Wang +7Fine-TuningMemory-Efficient Fine-Tuning

    University of North Carolina at Charlotte · University of Houston · State University of New York at Binghamton +2

  27. GRZO: Group-Relative Zeroth-Order Optimization for Large Language Model Fine-Tuning

    Jun 1, 2026Liyan Tan, Yequan Zhao, Yifan Yang +3Fine-TuningMemory-Efficient Fine-Tuning

    University of California, Santa Barbara

  28. Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning

    May 22, 2026Yoshihiko Fujisawa, Yuma Ichikawa, Yudai Fujimoto +2LLM QuantizationMemory-Efficient Fine-Tuning

    Fujitsu Limited · Institute of Science Tokyo · RIKEN Center for AIP +1

  29. ChunkFT: Byte-Streamed Optimization for Memory-Efficient Full Fine-Tuning

    May 20, 2026Yongkang Liu, Zijing Wang, Mengjie Zhao +7Fine-TuningMemory-Efficient Fine-Tuning

    Northeastern University, China · Shanghai Jiao Tong University, China · CIS, LMU Munich, Germany +2

  30. AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning

    May 19, 2026Ziye Chen, Hongbin Lin, Chenyu Zhang +3Memory-Efficient Fine-TuningZeroth-Order Optimization