Parameter-Efficient Fine-Tuning

Also known as PEFT

Parameter-efficient fine-tuning studies how to adapt pretrained models while training only a small subset of weights or adding lightweight modules, reducing resource costs and sometimes helping preserve existing capabilities. Common approaches include low-rank updates such as LoRA and quantized QLoRA, adapters and soft prompts, selective layer or rank tuning, and sparse or routed experts. Methods are evaluated on language benchmarks such as GLUE and mathematical reasoning, as well as vision, speech, multimodal, and continual-learning tasks, often with limited data or hardware. A central tension is that fewer trainable parameters do not necessarily mean lower memory use or faster training, and accuracy, robustness, and retention can vary with the model, task, and placement of updates.

Latest papers

610 papers75 papers in the last four weeks, up 12% on the four weeks before. 0.4% of all new papers.

Beyond Feature Fusion: Contextual Bayesian PEFT for Multimodal Uncertainty Estimation
Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinLow-Rank Adaptation Bayesian LoRA
Dalhousie University, Halifax NS, Canada
JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models
Apr 17, 2026Alexandra Dragomir, Ioana Pintilie, Antonio Barbalau +6Continual Learning for LLMs Low-Rank Adaptation
Bitdefender · ETH Zurich · University of Bucharest
PEFT of SLM for Telecommunications Customer Support: A Comparative Study of LoRA Configurations with Energy Consumption Analysis
Apr 17, 2026Lucas Tamic, Ilan Jaffeux-Cheniout, Xavier MarjouCustomer Support Automation Small Language Models
Orange, France
LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learning
Apr 16, 2026Bowen Ping, Zijun Chen, Tingfeng Hui +4Reinforcement Learning RL for Language Model Reasoning
1Peking University · 2Shanghai Jiao Tong University · 3Beijing University of Posts and Telecommunications
SLQ: Bridging Modalities via Shared Latent Queries for Retrieval with Frozen MLLMs
Apr 15, 2026Haoran Lou, Ziyan Liu, Chunxiao Fan +6Multimodal IR Cross-Modal Retrieval
School of Electronic Engineering, Beijing University of Posts and Telecommunications, China · Xiaohongshu Inc., China
Efficient Multi-View 3D Object Detection by Dynamic Token Selection and Fine-Tuning
Apr 15, 2026Danish Nazir, Antoine Hanna-Asaad, Lucas Görnhardt +3Multi-View 3D Object Detection 3D ViTs
Group Innovation, Volkswagen AG, 38440, Wolfsburg, Germany · Institute for Communications Technology, Technische Universität Braunschweig, 38106 Braunschweig, Germany
TLoRA+: A Low-Rank Parameter-Efficient Fine-Tuning Method for Large Language Models
Apr 15, 2026Yarui Cao, Kai LiuLLM Fine-Tuning Parameter-Efficient Fine-Tuning
Department of Computer Science, Clemson University, Clemson, SC 29634, USA
SEATrack: Simple, Efficient, and Adaptive Multimodal Tracker
Apr 14, 2026Junbin Su, Ziteng Xue, Shihui Zhang +3Multimodal Visual Object Tracking Efficient Multimodal Inference
School of Artificial Intelligence (School of Software), Yanshan University · School of Software, Beihang University · Hebei Key Laboratory of Computer Virtual Technology and System Integration +3
Unlocking the Potential of Grounding DINO in Videos: Parameter-Efficient Adaptation for Limited-Data Spatial-Temporal Localization
Apr 14, 2026Zanyi Wang, Fan Li, Dengyang Jiang +4Temporal Video Grounding VLM Adaptation
SGIT AI Lab, State Grid Corporation of China · University of HongKong · Zhejiang University of Technology
Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions
Apr 12, 2026Wenhao Zhang, Lin Mu, Li Ni +2Nonlinear LoRA LLM Fine-Tuning
Anhui University · University of Science and Technology of China
SpectralLoRA: Is Low-Frequency Structure Sufficient for LoRA Adaptation? A Spectral Analysis of Weight Updates
Apr 12, 2026Rajveer SinghLow-Rank Adaptation Parameter-Efficient Fine-Tuning
Indian Institute of Technology Roorkee
A Layer-wise Analysis of Supervised Fine-Tuning
Apr 12, 2026Qinghua Zhao, Xueling Gong, Xinyu Chen +2Supervised Fine-Tuning Fine-Tuning
Hefei University · Lanzhou University
Parameter Efficient Fine-tuning for Domain-specific Gastrointestinal Disease Recognition
Apr 12, 2026Sanjaya Poudel, Nikita Kunwor, Raj Simkhada +3Vision Foundation Model Adaptation Medical Diagnosis
Auburn University · Tribhuvan University · The University of Texas at Austin +2
Mining Attribute Subspaces for Efficient Fine-tuning of 3D Foundation Models
Apr 11, 2026Yu Jiang, Hanwen Jiang, Ahmed Abdelkader +4Parameter-Efficient Fine-Tuning Low-Rank Adaptation
The University of Texas at Austin · Shanghai Jiao Tong University · Adobe Research +1
What and Where to Adapt: Structure-Semantics Co-Tuning for Machine Vision Compression via Synergistic Adapters
Apr 11, 2026Shaobo Liu, Haobo Xiong, Kai Liu +1Entropy Modeling Learned Image Compression
Xidian University
New Hybrid Fine-Tuning Paradigm for LLMs: Algorithm Design and Convergence Analysis Framework
Apr 10, 2026Shaocong Ma, Peiran Yu, Heng HuangFine-Tuning LLM Fine-Tuning
Department of Computer Science University of Maryland College Park, MD 20742, USA · Department of Computer Science and Engineering University of Texas Arlington Arlington, TX 76019, USA
MP-ISMoE: Mixed-Precision Interactive Side Mixture-of-Experts for Efficient Transfer Learning
Apr 10, 2026Yutong Zhang, Zimeng Wu, Shangcai Liao +2Transfer Learning Memory-Efficient Fine-Tuning
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, China · School of Computer Science and Engineering, Beihang University, Beijing, China · College of Information Technology, United Arab Emirates University, Al Ain, Abu Dhabi, United Arab Emirates
The nextAI Solution to the NeurIPS 2023 LLM Efficiency Challenge
Apr 10, 2026Gyuwon Park, DongIl Shin, SolGil Oh +2LLM Fine-Tuning Efficient Language Model Training
Department of Computer Science and Engineering, UNIST · CJ Corporation
A Little Rank Goes a Long Way: Random Scaffolds with LoRA Adapters Are All You Need
Apr 9, 2026Hananel Hazan, Yanbo Zhang, Benedikt Hartl +1Random Feature Methods Parameter-Efficient Fine-Tuning
Allen Discovery Center at Tufts University, Medford, Massachusetts, USA · Institute for Theoretical Physics, TU Wien, Wien, Austria · Wyss Institute for Biologically Inspired Engineering, Harvard University, Boston, Massachusetts, USA
SOLAR: Communication-Efficient Model Adaptation via Subspace-Oriented Latent Adapter Reparametrization
Apr 9, 2026Seyed Mahmoud Sajjadi Mohammadabadi, Xiaolong Ma, Lei Yang +2Model Compression Neural Network Compression
University of Nevada, Reno, Reno, NV, USA · Argonne National Laboratory, Lemont, IL, USA · University of Houston, Houston, TX, USA +1
An empirical study of LoRA-based fine-tuning of large language models for automated test case generation
Apr 8, 2026Milad Moradi, Ke Yan, David Colwell +1Automated Test Generation LLM Fine-Tuning
AI Research Lab, Tricentis, Vienna, Austria · AI Research Lab, Tricentis, Sydney, Australia
A Parameter-Efficient Transfer Learning Approach through Multitask Prompt Distillation and Decomposition for Clinical NLP
Apr 8, 2026Cheng Peng, Mengxian Lyu, Ziyi Chen +1Parameter-Efficient Fine-Tuning Transfer Learning
Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, Florida, USA · Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, Florida, USA
Fine-tuning Whisper for Pashto ASR: strategies and scale
Apr 7, 2026Hanif RahmanAutomatic Speech Recognition Low-Resource Speech Recognition
Independent Researcher.
Visual prompting reimagined: The power of the Activation Prompts
Apr 7, 2026Yihua Zhang, Hongkang Li, Yuguang Yao +5Visual Prompt Learning Vision Foundation Model Adaptation
Michigan State University · University of Pennsylvania · New Jersey Institute of Technology +2
TalkLoRA: Communication-Aware Mixture of Low-Rank Adaptation for Large Language Models
Apr 7, 2026Lin Mu, Haiyang Wang, Li Ni +4Mixture-of-Experts Language Models Low-Rank Adaptation
Anhui University · Hefei University · University of Science and Technology of China
Constraint-Driven Warm-Freeze for Efficient Transfer Learning in Photovoltaic Systems
Apr 7, 2026Yasmeen Saeed, Ahmed Sharshar, Mohsen GuizaniFine-Tuning Transfer Learning
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates
Semantic-Topological Graph Reasoning for Language-Guided Pulmonary Screening
Apr 7, 2026Chenyu Xue, Yiran Liu, Mian Zhou +2Text-Guided Image Segmentation Medical Image Segmentation
Xi’an Jiaotong-Liverpool University, Jiangsu, China · University College London, London, United Kingdom · University of Liverpool, Liverpool, United Kingdom
Physics-Aligned Spectral Mamba: Decoupling Semantics and Dynamics for Few-Shot Hyperspectral Target Detection
Apr 7, 2026Luqi Gong, Qixin Xie, Yue Chen +4Hyperspectral Imaging Parameter-Efficient Fine-Tuning
Research Center for Space Computing System, Zhejiang Lab, Hangzhou 311121, China · State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
Improving Sparse Memory Finetuning
Apr 6, 2026Satyam Goyal, Anirudh Kanchi, Garv Shah +1Continual Learning for LLMs Sparse Fine-Tuning
University of Michigan, Ann Arbor
FLeX: Fourier-based Low-rank EXpansion for multilingual transfer
Apr 6, 2026Gaurav NarasimhanLLM Fine-Tuning Multilingual Code Generation
Department of Computer Science, Stanford University

Apr 17, 2026J/K move · Enter open · S save

Beyond Feature Fusion: Contextual Bayesian PEFT for Multimodal Uncertainty Estimation

Habibeh Naderi, Behrouz Haji Soleimani, Stan Matwin

Dalhousie University, Halifax NS, Canada

Low-Rank Adaptation Bayesian LoRA Multimodal Learning Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

We introduce CoCo-LoRA, a multimodal, uncertainty-aware parameter-efficient fine-tuning method for text prediction tasks accompanied by audio context. Existing PEFT approaches such as LoRA are efficient but typically deterministic, while recent Bayesian low-rank adapters model uncertainty in a lightweight way yet remain largely unimodal and condition uncertainty primarily on internal text features. This leaves them poorly equipped to reflect uncertainty driven by external acoustic factors such as background noise, channel variability, or speaking style, which can materially affect reliability in speech-centered applications. CoCo-LoRA addresses this gap by conditioning a contextual variational posterior in the low-rank space on both local text-derived adapter features and an audio-derived context signal. A pooled audio embedding is projected once into a shared context space and then adapted through lightweight layer-wise heads, enabling global-to-local, depth-specific modulation of the adapter uncertainty and update without high-dimensional multimodal fusion. Stochasticity is confined to a compact latent component in the rank space, preserving PEFT scalability while producing audio-sensitive, heteroscedastic uncertainty. Based on our evaluations across diverse tasks and backbone combinations, CoCo-LoRA consistently matches or outperforms text-only PEFT and conventional feature-fusion transfer baselines, particularly on high-coverage labels where reliable adaptation is critical. The results indicate that using audio as a contextual uncertainty signal, rather than as a fused feature stream, provides a robust and parameter-efficient alternative for multimodal low-resource prediction.