Parameter-Efficient Fine-Tuning

Also known as PEFT

Ask about this topic

Overview

Parameter-efficient fine-tuning studies how to adapt pretrained models while training only a small subset of weights or adding lightweight modules, reducing resource costs and sometimes helping preserve existing capabilities. Common approaches include low-rank updates such as LoRA and quantized QLoRA, adapters and soft prompts, selective layer or rank tuning, and sparse or routed experts. Methods are evaluated on language benchmarks such as GLUE and mathematical reasoning, as well as vision, speech, multimodal, and continual-learning tasks, often with limited data or hardware. A central tension is that fewer trainable parameters do not necessarily mean lower memory use or faster training, and accuracy, robustness, and retention can vary with the model, task, and placement of updates.

Latest papers

CardsList

610 papers75 papers in the last four weeks, up 12% on the four weeks before. 0.4% of all new papers.

  1. Frame2Freq: Spectral Adapters for Fine-Grained Video Understanding

    Feb 21, 2026Thinesh Thiyakesan Ponbagavathi, Constantin Seibold, Alina RoitbergVideo Action RecognitionVision Foundation Model Adaptation

    Institute for Artificial Intelligence, University of Stuttgart, Germany · Intelligent Assistive Systems Lab, University of Hildesheim · University Hospital Heidelberg, Diagnostic and Interventional Radiology

  2. Parameter-Efficient Domain Adaptation of Physics-Informed Self-Attention based GNNs for AC Power Flow Prediction

    Feb 20, 2026Redwanul Karim, Changhun Kim, Timon Conrad +7Physics-Informed GNNsPower Systems

    Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Institute of Electrical Energy Systems, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany

  3. Continual-NExT: A Unified Comprehension And Generation Continual Learning Framework

    Feb 20, 2026Jingyang Qiao, Zhizhong Zhang, Xin Tan +3Cross-Modal Representation TransferMultimodal Large Language Models

    East China Normal University · Shanghai Innovation Institute · Shanghai Artificial Intelligence Laboratory +1

  4. Calibrated Adaptation: Bayesian Stiefel Manifold Priors for Reliable Parameter-Efficient Fine-Tuning

    Feb 19, 2026Ibne Farabi Shihab, Sanjeda Akter, Anuj SharmaBayesian LoRAParameter-Efficient Fine-Tuning

    Department of Computer Science, Iowa State University · Department of Civil, Construction and Environmental Engineering, Iowa State University

  5. Revisiting Weight Regularization for Low-Rank Continual Learning

    Feb 19, 2026Yaoyue Zheng, Yin Zhang, Joost van de Weijer +4Incremental LearningElastic Weight Consolidation

    State Key Laboratory of Human-Machine Hybrid Augmented Intelligence Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Shaanxi, China · Computer Vision Center, Barcelona, Spain · Universitat Aut`onoma de Barcelona, Barcelona, Spain +3

  6. LORA-CRAFT: Cross-layer Rank Adaptation via Frozen Tucker Decomposition of Pre-trained Attention Weights

    Feb 19, 2026Kasun Dewage, Marianna Pensky, Shankadeep Mondal +1Tensor DecompositionLow-Rank Adaptation

    University of Central Florida Orlando, FL 32816, USA

  7. Training Large Reasoning Models Efficiently via Progressive Thought Encoding

    Feb 18, 2026Zeliang Zhang, Xiaodong Liu, Hao Cheng +3Efficient Language Model ReasoningLLM Reasoning

    University of Rochester · Microsoft Research

  8. A Contrastive Learning Framework Empowered by Attention-based Feature Adaptation for Street-View Image Classification

    Feb 18, 2026Qi You, Yitai Cheng, Zichao Zeng +1VLM AdaptationImage Classification

    3DIMPact & SpaceTimeLab, University College London

  9. Optimizing Soft Prompt Tuning via Structural Evolution

    Feb 18, 2026Zhenzhen Huang, Chaoning Zhang, Haoyu Bian +8Prompt TuningParameter-Efficient Fine-Tuning

    School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China · School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China · College of Professional and Continuing Education, The Hong Kong Polytechnic University, Hong Kong, China +3

  10. D2-LoRA: A Synergistic Approach to Differential and Directional Low-Rank Adaptation

    Feb 16, 2026Nozomu Fujisawa, Masaaki KondoLLM Fine-TuningLow-Rank Adaptation

    Keio University

  11. Efficient Text-Guided Convolutional Adapter for the Diffusion Model

    Feb 16, 2026Aryan Das, Koushik Biswas, Swalpa Kumar Roy +2Diffusion Model AdaptationConditional Image Generation

    VIT Bhopal · IIIT Delhi · Tezpur University Assam +1

  12. Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts

    Feb 16, 2026Buze Zhang, Jinkai Tao, Zilang Zeng +4Parameter-Efficient Fine-TuningRepresentation Geometry in Language Models

    School of Software Engineering, Xi’an Jiaotong University, China · School of Information, Central University of Finance and Economics, Beijing, China · Department of Artificial Intelligence, Hong Kong University of Science and Technology (Guangzhou), Guangzhou, Guangdong, China +2

  13. The Appeal and Reality of Recycling LoRAs with Adaptive Merging

    Feb 12, 2026Haokun Liu, Gyung Hyun Je, Marco Ciccone +3LLM Fine-TuningLanguage Model Merging

    University of Toronto, Canada · Vector Institute, Toronto, Canada · Mistral AI (work done before joining) +2

  14. Parameter-Efficient Temporal Domain Generalization

    Feb 12, 2026Zekun Cai, Xinyuan Song, Yiheng Yao +3Domain GeneralizationParameter-Efficient Fine-Tuning

    The University of Tokyo, Tokyo, Japan · LocationMind, Tokyo, Japan · Emory University, Atlanta, GA, USA +1

  15. SHINE: A Scalable In-Context Hypernetwork for Mapping Context to LoRA in a Single Pass

    Feb 6, 2026Yewei Liu, Xiyuan Wang, Yansheng Mao +3Knowledge Augmentation for Language ModelsLow-Rank Adaptation

    Institute for Artificial Intelligence, Peking University · School of Electronics Engineering and Computer Science, Peking University · University of Oxford +1

  16. Nonlinearity as Rank: Generative Low-Rank Adapter with Radial Basis Functions

    Feb 5, 2026Yihao Ouyang, Shiwei Li, Haozhao Wang +5Nonlinear LoRALow-Rank Adaptation

    Huazhong University of Science and Technology, Wuhan, China

  17. Least but not Last: Fine-tuning Intermediate Principal Components for Better Performance-Forgetting Trade-Offs

    Feb 3, 2026Alessio Quercia, Arya Bangun, Ira Assent +1Low-Rank AdaptationParameter-Efficient Fine-Tuning

    IAS-8, Forschungszentrum Juelich, Juelich, Germany · Department of Computer Science, RWTH Aachen University, Aachen, Germany · Aarhus University, Aarhus, Denmark

  18. Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Models

    Jan 28, 2026Moule Lin, Shuhao Guan, Andrea Patane +2Bayesian Neural NetworksConfidence Estimation in Language Models

    School of Computer Science and Statistics, Trinity College Dublin, Dublin, Ireland and Lero the Research Ireland Centre for Software, Ireland · School of Computer Science, University College Dublin, Dublin, Ireland

  19. DR-LoRA: Dynamic Rank LoRA for Fine-Tuning Mixture-of-Experts Models

    Jan 8, 2026Guanzhi Deng, Bo Li, Ronghao Chen +5Mixture-of-Experts Language ModelsFine-Tuning

    City University of Hong Kong · Tsinghua University · Peking University +2

  20. How (Mis)calibrated is your Federated CLIP and what to do about it?

    Dec 3, 2025Mainak Singha, Masih Aminbeidokhti, Paolo Casari +3Vision-Language ModelsVLM Adaptation

    University of Trento, Italy · ´Ecole de technologie sup´erieure, QC, Canada · AMIAD, Pˆole Recherche, Palaiseau, France +2

  21. Energy-Efficient Federated Learning via Adaptive Encoder Freezing for MRI-to-CT Conversion: A Green AI-Guided Research

    Nov 25, 2025Ciro Benito Raggio, Lucia Migliorelli, Nils Skupien +6Energy-Efficient MLFederated Learning

    Institute of Biomedical Engineering Karlsruhe Institute of Technology Fritz-Haber-Weg 1, Karlsruhe 76131 Baden-Württemberg, Germany · Department of Political Science Università Degli Studi Di Teramo Via Renato Balizarini 1, Teramo 64100 Italy · Department of Radiation Oncology University Medical Center Schleswig-Holstein Feldstrasse 21, Kiel 24105 Schleswig-Holstein, Germany +1

  22. FeRA: Frequency-Energy Constrained Routing for Effective Diffusion Adaptation Fine-Tuning

    Nov 22, 2025Bo Yin, Xiaobin Hu, Xingyu Zhou +7Fine-TuningAdaptive Model Routing

    National University of Singapore · University of Electronic Science and Technology of China · Nanyang Technological University +4

  23. Parameter Importance-Driven Continual Learning for Foundation Models

    Nov 19, 2025Lingxiang Wang, Hainan Zhang, Zhiming ZhengContinual Learning for LLMsParameter-Efficient Fine-Tuning

    Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University

  24. Mixtures of SubExperts for Large Language Continual Learning

    Nov 9, 2025Haeyong Kang, Hee Suk Yoon, Dahua Feng +1Continual Learning for LLMsMixture-of-Experts Language Models

    Korea Advanced Institute of Science and Technology (KAIST)

  25. VisCoP: Visual Probing for Video Domain Adaptation of Vision Language Models

    Oct 15, 2025Dominick Reilly, Manish Kumar Govind, Le Xue +1Domain AdaptationVLM Adaptation

    University of North Carolina at Charlotte · Elorian AI

  26. Hierarchical Alignment: Block-Level LoRA-DPO for Preference Tuning in Large Language Models

    Oct 14, 2025Yishen Chen, Qi Dong, Yukun ZhangLLM AlignmentPreference Alignment

    Fudan University Shanghai, China · The Chinese University of Hong Kong Hong Kong, China

  27. Robust and Efficient Finetuning of Vision Foundation Models via Implicit Ensembling

    Oct 8, 2025Masih Aminbeidokhti, Heitor Rapela Medeiros, Eric Granger +1Vision Foundation Model AdaptationOOD Generalization

  28. On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs

    Sep 22, 2025Rongguang Ye, Ming Tang, Edith C. H. NgaiMixed-Precision QuantizationLLM Quantization

    Southern University of Science and Technology · The University of Hong Kong

  29. When is Fine-Tuning Scalable? Certified Recovery from Noisy Data and Linear-Cost Fine-Tuning on Structured Task Shifts

    Aug 31, 2025Anastasis Kratsios, Tin Sum Cheng, Bum Jun Kim +1Fine-TuningOOD Generalization

    Department of Mathematics, McMaster University, Canada · Department of Mathematics and Computer Science, University of Basel, Switzerland · Department of Statistical Sciences, University of Toronto, Canada +1

  30. From Isolation to Alignment: Unified LoRA for Efficient Multi-Task Learning

    Aug 7, 2025Jinda Liu, Yi Chang, Yuan WuMulti-Task LearningLow-Rank Adaptation

    School of Artificial Intelligence, Jilin University · 2Southern University of Science and Technology · 3International Center of Future Science, Jilin University +1