Parameter-Efficient Fine-Tuning

Also known as PEFT

Parameter-efficient fine-tuning studies how to adapt pretrained models while training only a small subset of weights or adding lightweight modules, reducing resource costs and sometimes helping preserve existing capabilities. Common approaches include low-rank updates such as LoRA and quantized QLoRA, adapters and soft prompts, selective layer or rank tuning, and sparse or routed experts. Methods are evaluated on language benchmarks such as GLUE and mathematical reasoning, as well as vision, speech, multimodal, and continual-learning tasks, often with limited data or hardware. A central tension is that fewer trainable parameters do not necessarily mean lower memory use or faster training, and accuracy, robustness, and retention can vary with the model, task, and placement of updates.

Latest papers

610 papers75 papers in the last four weeks, up 12% on the four weeks before. 0.4% of all new papers.

FedHisto-PAST: Parameter-Efficient Stain-Aware Federated Learning for Cross-Site Lung Histopathology Classification
Sep 25, 2026Muhammad Muhtasim Shahriar, M. M. Golam Hafiz, Saad Aloteibi +1Histopathology Image Classification Federated Learning
Department of Computer Science, International Islamic University Chittagong (IIUC), Chittagong, Bangladesh · Department of Computer Science, American International University-Bangladesh (AIUB), Dhaka, Bangladesh · Department of Computer Science and Engineering, College of Applied Studies, King Saud University, Riyadh, 11437, Saudi Arabia +2
FAVoR: Measuring and Mitigating Author-Style Homogenization in Federated Personalized Generation
Sep 25, 2026Lu Han, Jingyao Zhang, Katy Ilonka Gero +1Personalized Language Models Personalized Text Generation
School of Computer Science, The University of Sydney Sydney, NSW 2006, Australia
EndoFSA: Endoscopic Few-Shot Image Generation via Rank-Constrained Parameter Adaptation
Sep 24, 2026Panagiota Gatoula, Grigoris Karypidis, Dimitris K. IakovidisImage Generation Medical Image Synthesis
Dept. of Computer Science and Biomedical Informatics University of Thessaly Lamia, Greece
Adaptive Fisher-Whitened Cross-Covariance for Low-Resource Speech Recognition
Sep 24, 2026Asmee Mishra, Mengjie Qian, Brechtje Post +1Speech Foundation Models Low-Resource Speech Recognition
Department of Engineering, University of Cambridge, United Kingdom · Phonetics Laboratory, University of Cambridge, United Kingdom
Automatic Rank Allocation for Low-Rank Adaptation in Large Language Models via lp Regularization
Sep 24, 2026Zebang Xie, Chuanyang Zheng, Yik-Chung Wu +1Low-Rank Adaptation Adaptive Rank Allocation
National University of Singapore, Singapore · Huawei, Hong Kong · The University of Hong Kong, Hong Kong +1
Can One Adapted Model Do It All? Fine-Tuning Strategy Selection for Customer Support LLMs
Sep 23, 2026Md Tahmid Rahman Laskar, Xue-Yong Fu, Shashi Bhushan TNFine-Tuning Customer Support Automation
Dialpad Inc.
From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMs
Sep 22, 2026Zhentao Tan, Chang Liu, Yao Liu +2Mixture-of-Experts Language Models Sparse Fine-Tuning
Alibaba Group · Southeast University
LoRA-generating hypernetworks for efficient on-device LLM generative personalization
Sep 21, 2026Sean Augenstein, Li Ding, Jihwan Lee +2On-Device Language Model Inference Low-Rank Adaptation
Google · Formerly at Google, work done while at Google
HyperCLIP++: Fine-tuning CLIP forOpen-vocabulary Semantic Segmentation in Hyperbolic Space
Sep 21, 2026Zelin Peng, Zhengqin Xu, Changsong Wen +4VLM Adaptation Open-Vocabulary Semantic Segmentation
MoE Key Lab of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China · State Key Laboratory of Infrared Physics, Shanghai Institute of Technical Physics, Chinese Academy of Science, Shanghai, 200083, China · Meituan, Shanghai, 200051, China
0.5%>100%: Bidirectional Reciprocal Learning for Referring Image Segmentation
Sep 21, 2026Xiaoqiang Lu, Licheng Jiao, Lingling Li +5VLM Adaptation Vision Foundation Model Adaptation
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education, International Research Center for Intelligent Perception and Computation, School of Artificial Intelligence, Xidian University, Xi’an, Shaanxi Province, 710071, China
Comparative Performance and Parameter-Efficient Adaptation of DINOv2 for Active Trachoma Classification
Sep 20, 2026Kibrom Gebremedhin, Hadush Hailu, Bruk Gebregziabher +1Channel Attention Vision Foundation Model Adaptation
Department of Computer Science Mekelle University Mekelle, Ethiopia · Department of Computer Science Maharishi International University Fairfield, IA, USA · Signal Technologies Germany +1
FreqDINO++: A Frequency-Guided Multi-Task Routing Vision Foundation Model for Universal Ultrasound Analysis
Sep 17, 2026Qing Xu, Yixuan Zhang, Yue Li +7Ultrasound Image Segmentation Medical Imaging Foundation Models
School of Computer Science, University of Nottingham Ningbo China, Ningbo, Zhejiang, China · School of Mathematical Sciences, University of Nottingham Ningbo China, Ningbo, Zhejiang, China · School of Computer Science, University of Nottingham, Nottingham NG72RD, UK +3
CoReLoop: Parameter-Efficient Controlled Recurrent Refinement for Audio Deepfake Detection
Sep 17, 2026Kunyu Feng, Yuxiang Wang, Li Wang +2Audio Deepfake Detection Domain Generalization
The Chinese University of Hong Kong, Shenzhen · Amphion Technology Co., Ltd.
GAPrompt++: Multi-Granular Geometry-Aware Point Cloud Prompt for 3D Vision Model
Sep 17, 2026Zixiang Ai, Zhenyu Cui, Yufei Guo +4Point Cloud Learning Vision Foundation Model Adaptation
Wangxuan Institute of Computer Technology, Peking University, Beijing 100871, China · Intelligent Science and Technology Academy of CASIC, Beijing 100041, China · Institute of Software, Chinese Academy of Sciences, Beijing 100190, China +1
Not All Layers Need Tuning: Diagnosing and Directing Adaptation in Vision-Language-Action Models
Sep 16, 2026Shahram Najam Syed, Arthur Jakobsson, Prayuj Sachdev +1VLA Model Adaptation Low-Rank Adaptation
Robotics Institute, Carnegie Mellon University, Pittsburgh, USA · Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, USA
Position Anchor Tuning: Towards Efficient Adaptation of Pre-Trained Point Cloud Transformers
Sep 16, 2026Zheng Liu, Xin Gao, Jinchao Zhu +1Parameter-Efficient Fine-Tuning Efficient Transformer Inference
School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China, and also with the Beijing Engineering Research Center of Industrial Spectrum Imaging, Beijing 100083, China · College of Software, Nankai University, Tianjin 300350, China · Department of Automation, BNRist, Tsinghua University, Beijing 100084, China
Encoder Awakening via Adapters: Effective Domain-Adaptive Fine-tuning of Speech-LLMs
Sep 16, 2026Mohan Shi, Zilai Wang, Natarajan Balaji Shankar +3Domain Adaptation for ASR Adapter Tuning
Department of Electrical and Computer Engineering University of California Los Angeles Los Angeles, USA
The Unbearable Weight: Scaling Models and Methods for UAV Audio Classification
Sep 15, 2026Andrew P. Berg, Qian Zhang, Mia Y. WangLightweight CNNs Audio Classification
Department of Computer Science, College of Charleston, Charleston SC, USA. · Department of Engineering, College of Charleston, Charleston SC, USA.
Lumen: Parameter-Efficient Alignment of Pretrained Vision and Language Encoders for Zero-Shot Computational Pathology
Sep 15, 2026Kiarash Tajbakhsh, Abdelrahman Faqieh, Michael Jopiti +9Medical VLMs Vision-Language Alignment
Institute of Tissue Medicine and Pathology, University of Bern, Switzerland.
CLARE: Scalable Class-Incremental Continual Learning via a Sparsity-Based Framework
Sep 15, 2026Yunxiang Fu, Meng Lou, Zicheng Liao +1Class-Incremental Learning Continual Learning
School of Computing and Data Science, The University of Hong Kong · Hong Kong Generative AI Research and Development Center
Safe Error Correction for Language Models: Frozen-Base Adjustment with Capability Preservation
Sep 14, 2026Gautam KishoreLogit Adjustment Parameter-Efficient Fine-Tuning
Eulogik
Sequential Adapter Stacking for Cross-Lingual Low-Resource ASR
Sep 14, 2026Thai Thi Thanh Thao Dang, Mengjie Qian, Kate KnillLow-Resource Speech Recognition Parameter-Efficient Fine-Tuning
Department of Engineering, University of Cambridge, UK
Parameter-Efficient Adaptation of Pretrained Language Models for Time-Series Forecasting
Sep 14, 2026Tamanna Kumavat, Georg Brunner, Kyriakos FlourisTime Series Forecasting LLM-Based Time Series Forecasting
University of Zurich, Switzerland · Computer Vision Laboratory, ETH Zurich, Switzerland · MRC Biostatistics Unit, University of Cambridge, United Kingdom
Pre-PEFT Probing: Weight Statistics and Perturbation Robustness for Layer Selection in VLM Vision Encoders
Sep 14, 2026Qingtao Xia, Jiahua Bao, Siyao Cheng +1VLM Adaptation Parameter-Efficient Fine-Tuning
Faculty of Computing Harbin Institute of Technology Harbin, China
Parameter-Efficient Fine-Tuning of Foundation Models for Liver Tumor Segmentation in CT
Sep 12, 2026Ramtin Mojtahedi, Mohammad Hamghalam, Jacob J. Peoples +2Medical Image Segmentation Parameter-Efficient Fine-Tuning
School of Computing, Queen’s University, Kingston, ON, Canada · Toronto General Research Institute, University Health Network, Toronto, ON, Canada · Department of Electrical Engineering, Qa.C., Islamic Azad University, Qazvin, Iran +3
LoRA Fine-Tuned Models for Control Systems Course Q&A: A Multidimensional Evaluation of Model Scale and Rank Effects
Sep 12, 2026Shaowen Lu, Chengxu Liu, Ping Zhou +1Educational QA LLM Fine-Tuning
State Key Laboratory of Synthetical Automation for Process Industries Northeastern University Shenyang, China
LayerRoute: Adaptive Layer-Skipping with LoRA-Preserved Quality for Efficient LLM Inference
Sep 12, 2026Prateek Kumar SikdarEfficient Transformer Inference LLM Inference Acceleration
Accenture
Multimodal Foundation Models Adaptation based on Domain-Aware Relaxed Orthogonal Subspace for Remote Sensing
Sep 12, 2026Han Luo, Ruoyu Yang, Yinhe Liu +1Vision Foundation Model Adaptation Parameter-Efficient Fine-Tuning
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430072, China
Partition-Invariant Tuning for 3D Scene Understanding
Sep 11, 2026Hongqiang Lin, Tianle Wang, Shuiwang Li +4Point Cloud Learning Parameter-Efficient Fine-Tuning
ZJU · GUT · THU +3
Rank-Efficient LoRA via Joint Tangent-Space Optimization under Isotropic Curvature
Sep 10, 2026Zihan Zhu, Zhehang Du, Xuyang Chen +5Low-Rank Adaptation Parameter-Efficient Fine-Tuning
University of Pennsylvania · DRW Associates LLC · University of Chicago

Sep 25, 2026J/K move · Enter open · S save

FedHisto-PAST: Parameter-Efficient Stain-Aware Federated Learning for Cross-Site Lung Histopathology Classification

Muhammad Muhtasim Shahriar, M. M. Golam Hafiz, Saad Aloteibi, Mohammad Ali Moni

Department of Computer Science, International Islamic University Chittagong (IIUC), Chittagong, Bangladesh · Department of Computer Science, American International University-Bangladesh (AIUB), Dhaka, Bangladesh · Department of Computer Science and Engineering, College of Applied Studies, King Saud University, Riyadh, 11437, Saudi Arabia+2

Histopathology Image Classification Federated Learning Vision Foundation Model Adaptation Non-IID Federated Learning Federated Learning Aggregation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Cross-site lung histopathology classification must account for stain variation, non-IID client data, missing classes, and the cost of adapting large pathology encoders. This study evaluates FedHisto-PAST v2 for three-way classification of adenocarcinoma (ACA), Normal, and squamous cell carcinoma (SCC). FedHisto-PAST v2 combines a frozen HIBOU-B foundation model with parameter-efficient adaptation, stain-conditioned paired-view prediction and feature consistency, reliability-aware prototype learning, and adaptive federated aggregation. Experiments used a five-client, non-IID, raw-data-local simulation with fixed internal evaluation, client-level analysis, component ablations, communication accounting, and a development-influenced exploratory LungHist700 cohort. All principal methods achieved near- ceiling internal performance, which limited discrimination on the fixed split. On LungHist700, FedHisto- PAST v2 achieved a Macro-F1 of 0.728560 and a balanced accuracy of 0.730454. Higher recognition of Normal and SCC was accompanied by lower ACA recall, and calibration remained imperfect. Prediction-level consistency was the only component with a clearly supported independent contribution in the external ablation analysis. Feature consistency and prototype regularization showed no conclusive independent overall gains in Macro-F1. The framework updated 1.253841% of the model parameters. The results provide exploratory cross-dataset evidence for stain-aware, parameter-efficient federation; they do not establish formal privacy, patient-level independence, prospective deployment, or clinical validation.