Parameter-Efficient Fine-Tuning

Also known as PEFT

Parameter-efficient fine-tuning studies how to adapt pretrained models while training only a small subset of weights or adding lightweight modules, reducing resource costs and sometimes helping preserve existing capabilities. Common approaches include low-rank updates such as LoRA and quantized QLoRA, adapters and soft prompts, selective layer or rank tuning, and sparse or routed experts. Methods are evaluated on language benchmarks such as GLUE and mathematical reasoning, as well as vision, speech, multimodal, and continual-learning tasks, often with limited data or hardware. A central tension is that fewer trainable parameters do not necessarily mean lower memory use or faster training, and accuracy, robustness, and retention can vary with the model, task, and placement of updates.

Latest papers

610 papers75 papers in the last four weeks, up 12% on the four weeks before. 0.4% of all new papers.

LoRSA: Toward Generalizable Parameter-Efficient Fine-Tuning for Biomedical Downstream Tasks
Aug 7, 2026Saed Moradi, Benyamin Ghojogh, M. Hadi Sepanj +2Domain Generalization Vision Foundation Model Adaptation
Department of Oncology, McMaster University, Hamilton, Canada · Department of Systems Design Engineering, University of Waterloo, Waterloo, Canada · Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Canada. +1
YOLO-PEFT: Parameter-Efficient Fine-Tuning on YOLO Family
Aug 7, 2026Xu Lin, WenJie Nie, Jinlong Peng +4Object Detection Real-Time Object Detection
Tencent · Xiamen University
Efficient Adaptation of LLMs for Hate Speech Detection in Low-Resource Languages: A Comparative Study on Roman Urdu
Aug 6, 2026Toneema Zubair, Muhammad Junaid Asif, Faisal Kamiran +2LLM Evaluation Hate Speech Detection
Department of Computer Sciences Information Technology University Lahore (54000), Pakistan · Artificial Intelligence Technology Centre National Centre for Physics Islamabad (44000), Pakistan
KV-Skill: Forging Expertise in the Model's Native Language
Aug 5, 2026Zhaowei Han, Xiang Zhang, Bing Han +3Knowledge Augmentation for Language Models Parameter-Efficient Fine-Tuning
University of Michigan, Ann Arbor, MI, USA
Adapting Vision Foundation Models with Cascaded Semantics
Aug 5, 2026Xi Xiao, Xingjian Li, Cheng Han +8Vision Foundation Model Adaptation Visual Prompt Learning
University of Alabama at Birmingham · Carnegie Mellon University · University of Missouri–Kansas City +4
Beyond Rotations: AuroOFT for Expressive Quantized Orthogonal Fine-Tuning
Aug 5, 2026Yue Han, Dianlin WangLLM Quantization Fine-Tuning
College of Systems Engineering, National University of Defense Technology · College of Computer Science and Technology, National University of Defense Technology
Beyond Full-Model Rollback: AuroSFT for Adapter-State Multi-Task Fine-Tuning
Aug 5, 2026Yue Han, Ziniu LiuSupervised Fine-Tuning Multi-Task Learning
College of Systems Engineering, National University of Defense Technology · College of Computer Science and Technology, National University of Defense Technology
Personalized Federated Sparse Adaptation of Time-Series Foundation Models
Aug 5, 2026Priyanka Nihalchandani, Naman Srivastava, Varun Ojha +1Mixture of Experts Time Series Forecasting
Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science, Bengaluru, India · School of Computing, Newcastle University, Newcastle upon Tyne, United Kingdom
Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUs
Aug 5, 2026Kuanysh Akhmetzhanov, Jurn-Gyu ParkFine-Tuning On-Device Language Model Inference
Department of Computer Science, Nazarbayev University, Astana 010000, Kazakhstan
HyPASE: Hyperbolic Geometry for Parameter-Efficient Speech Emotion Fine-Tuning Framework for Large Audio-Language Models
Aug 5, 2026Tian Jin, Ruikang Zhang, Zefeng Zhao +2Audio-Language Models Multimodal Emotion Recognition
Tongji University Shanghai, China · Peking University Beijing, China · The Chinese University of Hong Kong, Shenzhen Shenzhen, China
Geometry-Informed Parameter-Efficient Fine-Tuning of Pre-trained Molecular GNNs for Blood-Brain Barrier Permeability Prediction
Aug 4, 2026Marco Vieto Vega, Long D. Nguyen, Binh P. NguyenGraph Neural Networks Molecular Property Prediction
School of Mathematics and Statistics Victoria University of Wellington Wellington, New Zealand
Omega-S: A Functional Resilience Index for LLM Fine-Tuning
Aug 4, 2026Alberto AcedoRegularization-Based Continual Learning Continual Learning for LLMs
Biome Makers Inc.
Pin Once, Swap Light: Subspace-Aligned Centroid-Residual Training for Efficient Ultra-LoRA Serving
Aug 4, 2026Xiang Li, Pengcheng Wang, Huazheng Wang +1LLM Inference Efficiency LLM Serving
1Purdue University · 2Oregon State University
MoEGen: Mixture-of-Experts for Instance-Adaptive LoRA Generation
Aug 4, 2026Yiming Zeng, Lei Lu, Zexin Li +10LLM Fine-Tuning Low-Rank Adaptation
University of Connecticut · Northeastern University · Nanyang Technological University +5
LoCA: Forward-Only LLM Tuning after One-Shot Calibration with Local Credit Assignment
Aug 4, 2026Linhan Xia, Rui Liu, Zhaofeng Zhang +3Fine-Tuning LLM Fine-Tuning
University of Oklahoma · 4Tencent · 2Imperial College London +5
Z-PEFT: Zero-shot Backdoor Detection in Parameter-Efficient Fine-Tuning via Canonical Spectral Signatures
Aug 3, 2026Nicola Pitzalis, Donald Shenaj, Giacomo Cignoni +3Backdoor Detection Parameter-Efficient Fine-Tuning
University of Pisa
Analyzing Speech Condition Effects in Dysarthric ASR: A Layer-wise Probing Study
Aug 3, 2026Darwin Jelestin Muthu, Navya Gupta, Wei Lin Tay +3Speech Processing Low-Resource Speech Recognition
Singapore Institute of Technology
SPECTRA: Band-Routed Embedding and Stage-Wise LoRA for Cross-Sensor Fine-Tuning of Geospatial Foundation Models
Aug 3, 2026Xingyan Li, Jordan A. Caraballo-Vega, Jie Gong +2Remote Sensing Image Segmentation Geospatial Foundation Models
University of Maryland, Baltimore County Baltimore, Maryland, USA · NASA Goddard Space Flight Center Greenbelt, Maryland, USA
Not the Dimension, the Norm: What Matters in Gradient-Free Weight Perturbation of Language Models
Aug 3, 2026Taeyeong Kim, Ahhyun Kim, TaeHyeon Kim +1Zeroth-Order Optimization LLM Fine-Tuning
Chosun University · Korea University Sejong Campus
Mechanistic Interpretability-Guided Selective Fine-Tuning of Vision-Language Models for Centimeter-Level Flood Depth Estimation
Aug 2, 2026Nafis Fuad, Xiaodong Qian, Dongxiao ZhuVLM Adaptation Depth Estimation
Department of Civil & Environmental Engineering Wayne State University Detroit, MI, USA · Department of Civil & Environmental Engineering Institute for AI and Data Science (AIDaS) Wayne State University Detroit, MI, USA · Department of Computer Science, Institute for AI and Data Science (AIDaS) Wayne State University Detroit, MI, USA
MineGrad: Gradient Inversion Attacks on LoRA Fine-Tuning
Aug 2, 2026Hasin Us Sami, Swapneel Sen, Basak GulerGradient Inversion Attacks Privacy Leakage in Language Models
University of California, Riverside, CA
FedChronos: Federated Fine-Tuning of Time-Series Foundation Models for Privacy-Preserving Commodity Price Forecasting
Aug 2, 2026Amit Sharma, Nitin Auluck, Akramul AzimTime Series Forecasting Federated Fine-Tuning
Indian Institute of Technology Ropar, Punjab, India · Ontario Tech University, Oshawa, Canada
EulerLoRA: Rank-Driven Jump Dynamics for Calibrated Parameter-Efficient Fine-Tuning
Aug 2, 2026Srinivas Anumasa, Dianbo LiuUncertainty Quantification Low-Rank Adaptation
National University of Singapore
The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs
Jul 31, 2026Jiajia Tang, Sizhe Yuen, Francisco Gomez Medina +2Continual Learning for LLMs Low-Rank Adaptation
The Alan Turing Institute, London, UK · King’s College London, London, UK · University of Southampton, Southampton, UK
MoPET: Parameter-Efficient Mixture-of-Experts for Unified Medical Image Classification
Jul 31, 2026Sebastian Doerrich, Daniel Würtinger, Francesco Di Salvo +2Medical Imaging Foundation Models Medical Image Classification
xAILab Bamberg, University of Bamberg, Bamberg, Germany
Federated Foundation Models Fine-Tuning with Heterogeneous Compressed Clients
Jul 31, 2026Shengkun Zhu, Jinshan Zeng, Zhihua Allen-Zhao +5Federated Learning Aggregation Federated Fine-Tuning
The Hong Kong Polytechnic University · OceanBase, Ant Group · School of Management, Xi’an Jiaotong University +3
Parameter-Efficient Fine-Tuning for Spiking Point Cloud Models
Jul 31, 2026Zihao Guo, Jihua Zhu, Yiding Sun +2Point Cloud Learning Spiking Neural Networks
Xi’an Jiaotong University · Nanyang Technological University
SAM+D: Parameter-Efficient Dimensional Lifting of SAM-Family Models via Depth-Routed LoRA and Depth Shifting
Jul 31, 2026Yu Song, Hao Sun, Shiyu Teng +2Image Segmentation Vision Foundation Model Adaptation
College of Information Science and Engineering, Ritsumeikan University 2-150 Iwakura-cho, Ibaraki, Osaka 567-8570, Japan
FedWeave: Rethinking the Unit of Specialization in Heterogeneous Federated MoE-LoRA
Jul 29, 2026Donghang Duan, Xu Zheng, Lizong Zhang +2Federated Learning Aggregation LLM Routing
University of Electronic Science and Technology of China · 2Zhejiang University
How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model
Jul 28, 2026Mahendra Singh Rathor, Anagheem AzzamMemory-Efficient Fine-Tuning Small Language Models
Independent Researcher · Jaipur, India · Hamburg, Germany

Aug 7, 2026J/K move · Enter open · S save

LoRSA: Toward Generalizable Parameter-Efficient Fine-Tuning for Biomedical Downstream Tasks

Saed Moradi, Benyamin Ghojogh, M. Hadi Sepanj, Yimin Yang+1

Department of Oncology, McMaster University, Hamilton, Canada · Department of Systems Design Engineering, University of Waterloo, Waterloo, Canada · Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Canada.+1

Domain Generalization Vision Foundation Model Adaptation Low-Rank Adaptation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Parameter-efficient fine-tuning enables the adaptation of vision foundation models to biomedical tasks under limited computational resources, but a single low-rank update can constrain all task-specific changes to one narrow parameter subspace. This restriction may prevent the model from simultaneously representing globally shared task structure and localized residual directions required for generalization to unseen imaging domains. We introduce LoRSA, a global--residual adaptation framework that jointly learns a dense low-rank component and a dynamically structured-sparse low-rank component. The dense component captures globally coordinated task adaptation, while the structured component provides complementary residual corrections whose support evolves during training. We characterize the representational capacity, approximation properties, rank structure, and singular-subspace complementarity of this decomposition. We evaluate LoRSA for four-class breast-density classification using DINOv3-Base, with VinDr-Mammo as the source domain and MammosighTR and RSNA as unseen external domains. LoRSA remains competitive on the internal validation set and achieves the best external macro-F1 on both target datasets, improving upon the strongest competing method by 2.15 percentage points on MammosighTR and 3.09 percentage points on RSNA. Weight-matrix analysis further shows that approximately

92\%

of the energy of each adaptation component lies outside the bilateral singular subspace of the other, indicating that the two components learn largely complementary update directions. These results suggest that organizing adaptation capacity into distinct global and residual paths can improve the external-domain generalization of parameter-efficiently adapted biomedical vision models.