Parameter-Efficient Fine-Tuning

Also known as PEFT

Ask about this topic

Overview

Parameter-efficient fine-tuning studies how to adapt pretrained models while training only a small subset of weights or adding lightweight modules, reducing resource costs and sometimes helping preserve existing capabilities. Common approaches include low-rank updates such as LoRA and quantized QLoRA, adapters and soft prompts, selective layer or rank tuning, and sparse or routed experts. Methods are evaluated on language benchmarks such as GLUE and mathematical reasoning, as well as vision, speech, multimodal, and continual-learning tasks, often with limited data or hardware. A central tension is that fewer trainable parameters do not necessarily mean lower memory use or faster training, and accuracy, robustness, and retention can vary with the model, task, and placement of updates.

Latest papers

CardsList

610 papers75 papers in the last four weeks, up 12% on the four weeks before. 0.4% of all new papers.

  1. CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA

    Jul 11, 2026Gengyu Zhang, Haiyin Ran, Zhengbao He +4Fine-TuningMemory-Efficient Fine-Tuning

    Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai 200240, China

  2. Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning

    Jul 10, 2026Ivan Ilin, Philip Zmushko, Peter RichtárikMemory-Efficient Fine-TuningLow-Rank Adaptation

    KAUST

  3. Super Weights in LLMs and the Failure of Selective Training

    Jul 9, 2026Shreyas Subramanian, Adewale Akinfaderin, Akarsha SehwagLLM PruningFine-Tuning

    Amazon Web Services

  4. LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models

    Jul 8, 2026Sojung An, Junha Lee, Sujeong You +2Vision Foundation Model AdaptationLow-Rank Adaptation

    Korea University, Seoul, Republic of Korea · Korea Institute of Industrial Technology, Ansan, Republic of Korea · Seoul National University, Seoul, Republic of Korea

  5. Prompt-Adapter Context Routing for Parameter-Efficient Multi-Shot Long Video Extrapolation

    Jul 7, 2026Anna Córdoba, Adam Puente Tercero, Nerea Angulo Hijo +4Video Diffusion ModelsVideo Prediction

    Instituto de Investigación en Visión Artificial

  6. FourTune: Towards Fully 4-Bit Efficient Post-Training for Diffusion Models

    Jul 7, 2026Bowen Xue, Zihan Min, Xingyang Li +8Diffusion Model QuantizationQuantization-Aware Training

    Nunchux AI · MIT · Stanford +2

  7. Learning Only What Valid Adapters Can Express: Subspace-Constrained Adaptation Against Fine-Tuning Poisoning

    Jul 6, 2026Fabien PollyLLM Backdoor AttacksLow-Rank Adaptation

    Independent researcher

  8. Localized LoRA-MoE: Block-wise Low-Rank Experts With Adaptive Routing

    Jul 6, 2026Babak Barazandeh, Subhabrata Majumdar, Vinay Prithyani +1Adaptive Model RoutingLow-Rank Adaptation

    Fortinet · Indian Institute of Management Bangalore · Citadel Securities +1

  9. Listen, Think, Transcribe: Continuous Latent Test-Time Scaling for ASR

    Jul 6, 2026Ho Lam Chung, Yiming Chen, Dau-Cheng Lyu +2Test-Time ScalingAutomatic Speech Recognition

    National Taiwan University, Taiwan · ASUS, Taiwan · National University of Singapore, Singapore

  10. RSLoRA: Training-free Rank Allocation for LoRA via Representational Sensitivity Probing

    Jul 5, 2026Jiaqi Liu, Haidong Kang, Qihui Zhao +1Low-Rank AdaptationAdaptive Rank Allocation

    School of Computer Science and Technology Dalian University of Technology · Hebei Key Laboratory of Marine Perception Network and Data Processing Northeastern University · School of Computer and Communication Engineering Northeastern University +1

  11. ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning

    Jul 4, 2026Wentao LuContinual Learning for LLMsLow-Rank Adaptation

  12. Punching Above Their Weight: Classification-Head Fine-Tuning of Tiny Language Models (TLMs) for Verifiable Multiple-Choice Tasks

    Jul 4, 2026Bhavesh Sood, Jaromir SavelkaFine-TuningMultiple-Choice Question Answering

    Carnegie Mellon University

  13. CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection

    Jul 3, 2026Wen Dong, Zhao Wang, Shuangqing Zhang +5Continual Anomaly DetectionMixture of Experts

    Nanjing University, Nanjing, China · China Mobile Zijin Innovation Institute, Nanjing, China · Nanjing University of Science and Technology, Nanjing, China

  14. Efficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUs

    Jul 2, 2026Altay Toktassyn, Jurn-Gyu ParkFine-TuningMemory-Efficient Fine-Tuning

    Department of Computer Science, Nazarbayev University, Astana 010000, Kazakhstan

  15. EPnG: Adaptive Expert Prune-and-Grow for Parameter-Efficient MoE Fine-tuning

    Jul 2, 2026Ahin Lee, Sehyun Yun, Taesik GongMixture-of-Experts PruningMixture-of-Experts Language Models

    Ulsan National Institute of Science & Technology (UNIST)

  16. ZO-Act: Efficient Zeroth-Order Fine-Tuning via One-Shot Activation-Informed Low-Rank Subspaces

    Jul 1, 2026Xun Dong, Yibo Xu, Naigang Wang +3Zeroth-Order OptimizationLLM Fine-Tuning

    University at Albany, SUNY · IBM T. J. Watson Research Center

  17. Meta-Transfer Learning for mmWave Beam Alignment

    Jul 1, 2026Ahmet Nuri Cevik, Sinem ColeriMeta-LearningWireless Communications

  18. ECoSim: Data Efficient Fine-Tuning for Controllable Traffic Simulation

    Jul 1, 2026Yu-Hsiang Chen, Wei-Jer Chang, Yi-Ting Chen +1Synthetic Data Generation for RoboticsAutonomous Driving Simulation

    National Yang Ming Chiao Tung University, Taiwan · University of California, Berkeley, USA

  19. DroneFINE: Domain-Aware Parameter-Efficient Fine-Tuning of Vision-Language Detectors for Drone Images

    Jul 1, 2026Ke Wu, Yanan Zhang, Yingjie Gao +5Open-Vocabulary Object DetectionVLM Adaptation

    National College for Excellent Engineers, Beihang University, Beijing, China · School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China · School of Computer Science and Engineering, Beihang University, Beijing, China +2

  20. Wake up for Touch! Mask-isolated Tactile Alignment Learning in MLLMs

    Jul 1, 2026Yoonhyung Park, Minji Kim, Sungwon Moon +1Visuo-Tactile Representation LearningMultimodal Large Language Models

    Division of Artificial Intelligence & Software, Ewha Womans University

  21. FRAME: Learning the Adaptation Domain with a Mixture of Fractional-Fourier Experts

    Jun 30, 2026Tom Saliencro, Maya Lindqvist, Rohan Desai +2Low-Rank AdaptationMixture-of-Experts Models

    University of California, Irvine · University of Washington

  22. CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation

    Jun 30, 2026Pengcheng Wang, Ziran Liu, Wei Wang +1LLM Fine-TuningLow-Rank Matrix Decomposition

    Purdue University · Shanghai Institute for Mathematics and Interdisciplinary Sciences / Fudan University · Futurewei Technologies, Inc.

  23. Nonlinearity-Aware LoRA: Structured Gate Adaptation under Low-Rank Constraints

    Jun 30, 2026Shuai Yuan, Sudong Cai, Bingzhi Chen +4LLM Fine-TuningLow-Rank Adaptation

    1Beijing Institute of Technology, Zhuhai · 2The Hong Kong Polytechnic University · 3The University of Osaka +1

  24. Clinically Structured Rank-Gated LoRA for Cross-Benchmark Medical Question Answering

    Jun 30, 2026Hao Gong, Ruilin Gong, Yining HuangNonlinear LoRALow-Rank Adaptation

    University of Chinese Academy of Sciences · Henan University of Technology · Meta Emergence Laboratory

  25. Mixture-of-Control: State-Aware Fine-Tuning for Transformer-based Models

    Jun 30, 2026Duc Anh Nguyen, Tien Ngoc Luu, Tung Pham +1Fine-TuningTransformer

    Qualcomm AI Research. Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc.

  26. Dual-Adaptive SAM3: Hierarchical Routing over Low-Rank Expert Layers for Parameter-Efficient Medical Image Segmentation

    Jun 30, 2026Ying Chen, Jinyue Li, Kun Wang +2VLM AdaptationText-Guided Image Segmentation

    Shenzhen Research Institute, The Chinese University of Hong Kong · University of Science and Technology of China · Nanyang Technological University +1

  27. Seeing Through Multiple Views: Parameter-Efficient Fine-Tuning via Selective Neurons for Consistent Radiology Report Generation

    Jun 30, 2026Yucheng Chen, Jinjing Zhu, Yang Yu +7Radiology Report GenerationMulti-View Consistency

    MedVisAI Lab · Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore · Centre of AI in Medicine, Singapore +3

  28. Semantic-Driven Scale and Spatial Selection for Efficient Cross-Modal Alignment in Referring Remote Sensing Image Segmentation

    Jun 29, 2026Kun Li, Shengxi Gui, Francesco Nex +1Remote Sensing Image SegmentationCross-Modal Alignment

    University of Liverpool, UK · University of Michigan, USA · University of Twente, NL +1

  29. ScAle: Attention Head Scaling as a Minimal Adapter for Spatial Reasoning in Vision Language Models

    Jun 28, 2026Rahul Chowdhury, Timothy A Rupprecht, Xuan Shen +2Visual Spatial ReasoningVLM Adaptation

    Northeastern University, Boston, USA · EmbodyX Inc., USA · Zhejiang University, China

  30. BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning

    Jun 28, 2026Zhibin Duan, Yuhong Wang, Jiahong Fu +3Bayesian Neural NetworksLow-Rank Adaptation

    National Key Lab of Radar Signal Processing, Xidian University, Xi’an, Shaanxi 710071, China · School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi, 710049, China