Parameter-Efficient Fine-Tuning

Also known as PEFT

Ask about this topic

Overview

Parameter-efficient fine-tuning studies how to adapt pretrained models while training only a small subset of weights or adding lightweight modules, reducing resource costs and sometimes helping preserve existing capabilities. Common approaches include low-rank updates such as LoRA and quantized QLoRA, adapters and soft prompts, selective layer or rank tuning, and sparse or routed experts. Methods are evaluated on language benchmarks such as GLUE and mathematical reasoning, as well as vision, speech, multimodal, and continual-learning tasks, often with limited data or hardware. A central tension is that fewer trainable parameters do not necessarily mean lower memory use or faster training, and accuracy, robustness, and retention can vary with the model, task, and placement of updates.

Latest papers

CardsList

610 papers75 papers in the last four weeks, up 12% on the four weeks before. 0.4% of all new papers.

  1. Exploring parameter-efficient fine-tuning (PEFT) of billion-parameter vision models with QLoRA and DoRA: insights into generalization for limited-data image classification under a 98:1 test-to-train regime

    Mar 18, 2026Haiyu Yang, Sumit Sharma, Enhong Liu +1Precision Livestock FarmingParameter-Efficient Fine-Tuning

    Cornell University, College of Agriculture and Life Sciences, Ithaca, NY 14853

  2. Parameter-Efficient Modality-Balanced Symmetric Fusion for Multimodal Remote Sensing Semantic Segmentation

    Mar 18, 2026Haocheng Li, Juepeng Zheng, Shuangxi Miao +4Remote Sensing Image SegmentationMultimodal Fusion

    the College of Land Science and Technology, China Agricultural University, and also with the Key Laboratory of Remote Sensing for Agri-Hazards, Ministry of Agriculture and Rural Affairs, Beijing 100083, China · the School of Artificial Intelligence, Sun Yat-Sen University, Zhuhai, China · Henan Polytechnic University, and also with the Key Laboratory of Spatio-Temporal Information and Ecological Restoration of Mines, Ministry of Natural Resources of the People’s Republic of China, Jiaozuo 454003, China +4

  3. Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition

    Mar 18, 2026Yuxiang Mei, Delai Qiu, Shengping Liu +2Multilingual ASRSpeech Language Models

    Shanghai Engineering Research Center of Intelligent Education and Bigdata, Shanghai Normal University, Shanghai, 200234, China. · Unisound AI Technology Co., Ltd., Beijing, China. · SHNU-Unisound Natural Human-Computer Interaction Lab, Shanghai Normal University.

  4. Motion-Adaptive Temporal Attention for Lightweight Video Generation with Stable Diffusion

    Mar 18, 2026Rui Hong, Shuxue QuanVideo Diffusion ModelsTemporal Attention

    George Mason University, Fairfax, VA, USA · Independent Researcher

  5. ACE-LoRA: Graph-Attentive Context Enhancement for Parameter-Efficient Adaptation of Medical Vision-Language Models

    Mar 17, 2026M. Arda Aydın, Melih B. Yilmaz, Aykut Koç +1VLM AdaptationMedical VLMs

    Bilkent University · National Magnetic Resonance Research Center (UMRAM)

  6. Exploring different approaches to customize language models for domain-specific text-to-code generation

    Mar 17, 2026Luís Freire, Fernanda A. Andaló, Nicki Skafte DetlefsenCode GenerationLLM Code Generation

    Technical University of Denmark · The LEGO Group

  7. Spectral Rectification for Parameter-Efficient Adaptation of Foundation Models in Colonoscopy Depth Estimation

    Mar 16, 2026Xiaoxian Zhang, Minghai Shi, Lei LiMonocular Depth EstimationVision Foundation Model Adaptation

    Department of Biomedical Engineering, National University of Singapore, Singapore · Department of Radiotherapy, The First Affiliated Hospital of Xi’an Jiaotong University, Xi’an, China

  8. Lightweight User-Personalization Method for Closed Split Computing

    Mar 16, 2026Yuya Okada, Takayuki NishioEdge ComputingParameter-Efficient Fine-Tuning

    School of Engineering, Institute of Science Tokyo, Tokyo 152-8550, Japan

  9. AdapterTune: Zero-Initialized Low-Rank Adapters for Frozen Vision Transformers

    Mar 16, 2026Salim KhazemAdapter TuningLow-Rank Adaptation

    Talan Research Center, Paris, France

  10. Parameter-Efficient Quality Estimation via Frozen Recursive Models

    Mar 15, 2026Umar Abubacar, Roman Bauer, Diptesh KanojiaLow-Resource Language ProcessingMT Evaluation

    NICE Research Group, University of Surrey, UK · Surrey Institute for People-Centred AI (PAI), University of Surrey, UK

  11. Uni-MDTrack: Learning Decoupled Memory and Dynamic States for Parameter-Efficient Visual Tracking in All Modality

    Mar 15, 2026Wenrui Cai, Zhenyi Lu, Yuzhe Li +4Multimodal Visual Object TrackingVisual Object Tracking

    School of Computer Science and Engineering, Beihang University, Beijing 100191, China

  12. All-day Multi-scenes Lifelong Vision-and-Language Navigation with Tucker Adaptation

    Mar 15, 2026Xudong Wang, Gan Li, Zhiyu Liu +3Tucker DecompositionVision-Language Navigation

    State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · North University of China +1

  13. Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation

    Mar 15, 2026Xi Xiao, Chenrui Ma, Yunbei Zhang +7Low-Rank AdaptationParameter-Efficient Fine-Tuning

    University of Alabama at Birmingham · University of Virginia · Tulane University +5

  14. Selective Fine-Tuning of GPT Architectures for Parameter-Efficient Clinical Text Classification

    Mar 15, 2026Fariba Afrin Irany, Sampson AkwafuoSelective Fine-TuningText Classification

    Clarksville, TN, USA · California State University Fullerton, California, USA

  15. IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoring

    Mar 14, 2026Xuan Cui, Huiyue Li, Run Zeng +5LLM Fine-TuningLow-Rank Adaptation

    Chongqing Technology and Business University · Shanghai Jiao Tong University

  16. Manifold-Orthogonal Dual-spectrum Extrapolation for Parameterized Physics-Informed Neural Networks

    Mar 14, 2026Zhangyong Liang, Huanhuan GaoPDE Operator LearningPhysics-Informed ML

  17. Sparse-Dense Mixture of Experts Adapter for Multi-Modal Tracking

    Mar 14, 2026Yabin Zhu, Jianqi Li, Chenglong Li +3Multimodal Visual Object TrackingParameter-Efficient Fine-Tuning

    School of Public Security and Emergency Management, Anhui University of Science and Technology, Hefei 231131, China. · Information Materials and Intelligent Sensing Laboratory of Anhui Province, Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Artificial Intelligence, Anhui University, Hefei 230601, China. · School of Artificial Intelligence, Anhui University of Science and Technology, Hefei 231131, China. +1

  18. Cross-Lingual Transfer and Parameter-Efficient Adaptation in the Turkic Language Family: A Theoretical Framework for Low-Resource Language Models

    Mar 13, 2026O. Ibrahimzade, K. TabasaranskyCross-Lingual TransferMultilingual Language Models

  19. HIFICL: High-Fidelity In-Context Learning for Multimodal Tasks

    Mar 13, 2026Xiaoyu Li, Yuhang Liu, Xuanshuo Kang +4Multimodal ICLMultimodal Large Language Models

    University of Electronic Science and Technology of China · Institute of Electronics and Information Industry Technology of Kashgar

  20. Expert Pyramid Tuning: Efficient Parameter Fine-Tuning for Expertise-Driven Task Allocation

    Mar 13, 2026Jia-Chen Zhang, Zhen-Wei Yan, Yu-Jie Xiong +1Parameter-Efficient Fine-TuningLow-Rank Adaptation

    School of Electronic and Electrical Engineering, Shanghai University of Engineering Science 333 Longteng Road, Shanghai, China

  21. Annotation Entropy Predicts Per-Example Learning Dynamics in LoRA Fine-Tuning

    Mar 12, 2026Brady SteeleParameter-Efficient Fine-TuningAnnotator Disagreement

    Georgia Institute of Technology

  22. NeuroLoRA: Context-Aware Neuromodulation for Parameter-Efficient Multi-Task Adaptation

    Mar 12, 2026Yuxin Yang, Haoran Zhang, Mingxuan Li +6Parameter-Efficient Fine-TuningLow-Rank Adaptation

    Shanghai University · Fudan University

  23. Representation Finetuning for Continual Learning

    Mar 11, 2026Haihua Luo, Xuming Ran, Tommi Kärkkäinen +4Continual LearningFine-Tuning

    Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland · National University of Singapore, Singapore, Singapore · School of Computer Science and Technology, Dalian University of Technology, Dalian, China

  24. Parameter-Efficient Token Embedding Editing for Clinical Class-Level Unlearning

    Mar 11, 2026Iyad Ait Hou, Shrenik Borad, Harsh Sharma +3Machine UnlearningLanguage Model Editing

    Department of Computer Science, The George Washington University, Washington, DC · Center for Speech and Language Processing, Johns Hopkins University, Baltimore, MD

  25. GaLoRA: Parameter-Efficient Graph-Aware LLMs for Node Classification

    Mar 11, 2026Mayur Choudhary, Saptarshi Sengupta, Katerina PotikaLLMs for Graph LearningText-Attributed Graph Learning

    Department of Computer Science, San Jose State University

  26. ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning

    Mar 10, 2026Ruizhong Qiu, Hanqing Zeng, Yinglong Xia +15Parameter-Efficient Fine-TuningLow-Rank Adaptation

    University of Illinois Urbana-Champaign, IL, USA · Meta AI Menlo Park, CA, USA · Washington University St. Louis, WA, USA +1

  27. GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection

    Mar 10, 2026Kai Yao, Zhenghan Song, Kaixin Wu +5Sparse Fine-TuningSelective Fine-Tuning

    1Ant Group · 2Cornell University · University of Liverpool +2

  28. TemporalDoRA: Temporal PEFT for Robust Surgical Video Question Answering

    Mar 10, 2026Luca Carlini, Chiara Lena, Cesare Hassan +4Video QAParameter-Efficient Fine-Tuning

    Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB), Politecnico di Milano, Italy · IRCCS Humanitas Research Hospital, Italy · UCL Hawkes Institute and Department of Computer Science, University College London, UK +1

  29. On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning

    Mar 10, 2026Muhammad Ahmad, Jingjing Zheng, Yankai CaoParameter-Efficient Fine-TuningCatastrophic Forgetting

    Department of Electrical and Computer Engineering, The University of British Columbia · Department of Mathematics, The University of British Columbia · Department of Chemical and Biological Engineering, The University of British Columbia

  30. Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation

    Mar 10, 2026Luxi Lin, Zhihang Lin, Zhanpeng Zeng +5Speculative DecodingLLM Inference Acceleration

    Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Institute of Artificial Intelligence (TeleAI), China Telecom · Shanghai Innovation Institute +1