Parameter-Efficient Fine-Tuning

Also known as PEFT

Parameter-efficient fine-tuning studies how to adapt pretrained models while training only a small subset of weights or adding lightweight modules, reducing resource costs and sometimes helping preserve existing capabilities. Common approaches include low-rank updates such as LoRA and quantized QLoRA, adapters and soft prompts, selective layer or rank tuning, and sparse or routed experts. Methods are evaluated on language benchmarks such as GLUE and mathematical reasoning, as well as vision, speech, multimodal, and continual-learning tasks, often with limited data or hardware. A central tension is that fewer trainable parameters do not necessarily mean lower memory use or faster training, and accuracy, robustness, and retention can vary with the model, task, and placement of updates.

Latest papers

610 papers75 papers in the last four weeks, up 12% on the four weeks before. 0.4% of all new papers.

FUSEye: Training-Light Fisheye Detection with Overlapping Views and Zero-Initialized Adapters
Oct 2, 2026Wenya Su, Kai Luo, Di Wen +5Object Detection Efficient Object Detection
School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China · Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Germany
Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation
Oct 1, 2026Damiano Marsili, Raphi Kang, Aditya Mehta +2Mixture of Experts Mixture-of-Experts Models
California Institute of Technology
Task-Oriented Rank Adaptation for Continual Learning in Text Classification
Oct 1, 2026Rey Sanchez Lopez, Eduardo Morales Manzanares, Hugo Jair EscalanteContinual Learning Low-Rank Adaptation
INAOE, Puebla, Mexico · The University of Texas at El Paso, El Paso, TX, USA
Beyond Domain-Level Adaptation: Margin-Oriented Semantic-Appearance Interaction Correction for Personalized Federated Vision-Language Models
Oct 1, 2026Wentao Yue, Qingyu Mao, Tianyou Lai +2VLM Adaptation Federated Fine-Tuning
Parameter-Efficient Distributionally Robust Adaptation of Tabular Foundation Models under Subpopulation Shift
Oct 1, 2026Seonghwi Kim, Sung Ho Jo, Minwoo ChaeDistribution Shift Robustness Tabular Foundation Models
Pohang University of Science and Technology
Efficient Task Adaptation in Large Language Models: A Survey of Weight-Based, Prompt-Based, and Embedding-Based Adaptations
Oct 1, 2026Jungwon Park, Changin Choi, Jimyeong Kim +2LLM Fine-Tuning In-Context Learning
RICS · Daegu Gyeongbuk Institute of Science and Technology · IPAI +3
Analysis of Quantized and Efficiently Adapted Protein Language Models
Sep 30, 2026Ilan Yaniv Zeisler, Sebastian Clancy, Pouriya Bayat +8Protein Language Models Quantization-Aware LoRA
Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada. · Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada. · Department of Computer Science, University of Toronto, Toronto, Ontario, Canada. +2
Score the Update, Not the Token: Descent-Aligned Routing for Combinatorial LoRA Experts
Sep 30, 2026Priya Nair, Lukas Brenner, Maya Lindqvist +5Mixture-of-Experts Language Models Parameter-Free Mixture-of-Experts Routing
University of California, Irvine · University of Washington
ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs
Sep 30, 2026Hang Yin, Haozhe Wang, Yuhua Luo +3Continual Learning for LLMs Low-Rank Adaptation
Shanghai Jiao Tong University · Shanghai Innovation Institute · Beijing Zhongguancun Academy +3
Dynamic LoRA-Experts and Prototype-Ensemble Matching for Class-Incremental Learning
Sep 30, 2026Hongwei Zhao, Rui Liu, Yansong LiuClass-Incremental Learning Prototype-Based Classification
School of Computer Science and Engineering, Beihang University
T-Router: Learning Thalamic Routing for Reasoning with Parameter-Efficient Reinforcement Learning
Sep 30, 2026Liuxian Ma, Jiale Dai, Jiaqi Li +1Reinforcement Learning RL for Language Model Reasoning
College of Artificial Intelligence, Tsinghua University · State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Beijing Institute for General Artificial Intelligence
A foundation model for energy and radiation systems built on heterogeneous scientific interfaces
Sep 29, 2026Samrendra Roy, Tapas Tripura, Yoon Pyo Lee +2Physical Foundation Models PDE Surrogate Modeling
Department of Nuclear, Plasma, and Radiological Engineering, University of Illinois Urbana-Champaign, Urbana, IL 61801, USA · Department of Applied Mechanics, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India · Yardi School of Artificial Intelligence (ScAI), Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India +1
FedLAFP: Low-Rank Aggregation Meets Full-Rank Personalization in Federated Fine-Tuning
Sep 29, 2026Mengjun Yi, Huaian Gu, Yinghao Ai +2Low-Rank Adaptation Federated Fine-Tuning
State Key Laboratory for Novel Software Technology · School of Artificial Intelligence · School of Computer Science +1
Beyond Low-Rank Parameterization: Narrowing the Gap Between LoRA and Full Fine-Tuning via Gradient Decomposition
Sep 29, 2026Yihao Ouyang, Shiwei Li, Haozhao Wang +5Low-Rank Adaptation Parameter-Efficient Fine-Tuning
Huazhong University of Science and Technology, Wuhan, China · Hebei University of Technology, Tianjin, China
CF-LoRA: Decoupled Factor Aggregation and Adaptation-Aware Client Clustering for Federated LoRA Fine-Tuning
Sep 29, 2026Mengjun Yi, Langxing Yang, Suhan Guo +2Federated Learning Aggregation Federated Fine-Tuning
State Key Laboratory for Novel Software Technology · School of Artificial Intelligence School of Electronic Science and Engineering Nanjing University
Reducing the Adaptation Gap Through Reachable Fisher Geometry
Sep 28, 2026Wasif Jalal, Sachin Deb, Asif SalekinParameter-Efficient Fine-Tuning Group Fairness
Arizona State University Tempe, Arizona, USA
RoboFL: Federated Expert Assembly for World Action Models
Sep 28, 2026Rongyu Zhang, Ruizhi Fan, Yunfan Lou +8Mixture of Experts World Action Models
Nanjing University · Hong Kong University of Science and Technology · State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University
Distribution-Conditioned Task Routing for Class-Incremental Learning
Sep 28, 2026Longhuan Xu, Zhipeng Zhou, Wei Ji +3Class-Incremental Learning Classification
Nanjing University · Nanyang Technological University · Shanghai Jiao Tong University
Learn Here, Move Less Elsewhere: Input-Conditioned Plasticity from Retained-Domain Activation Atlases
Sep 28, 2026Jiangtao Lin, Bangyang Wei, Yihang Ding +2Continual Learning for LLMs Domain Adaptation for LLMs
Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Vehicle and Mobility, Tsinghua University · SZ DJI Technology Co., Ltd. +1
Loop Dropout: Regularizing Shared Updates in Looped Language Models
Sep 28, 2026Zirui Zhu, Hailun Xu, Xuanlei Zhao +5Low-Rank Adaptation Parameter-Efficient Fine-Tuning
Optimizing the Phi-2 Small Language Model for Real-time Chatbot Applications Using Parameter-Efficient Fine-Tuning (PEFT) with QLoRA Quantization
Sep 27, 2026PhanTan Khanh Nguyen, Ashfaq Ali Shafin, Khandaker Mamun AhmedOn-Device Language Model Inference Small Language Models
Beacom College of Computer and Cyber Sciences, Dakota State University · Mathematics and Computer Science, Augustana College
Resource-Aware Parameter-Efficient Model Adaptation for Onboard High-Dimensional Data
Sep 27, 2026Qiyang Zhang, Xinhao Li, Lei Shi +4Fine-Tuning Low-Rank Adaptation
Beijing University of Posts and Telecommunications · Wuhan University · Communication University of China +1
PoE-Fuse: Precision-Weighted Expert Fusion for Bi-Temporal Change Understanding
Sep 27, 2026Haruki Watase, Shunya Nagashima, Takayuki NishimuraRemote Sensing Change Detection Remote Sensing
Neurogica Inc., Tokyo, Japan
QuPID: Quantum Parameter-Efficient Input-Dependent Retrieval Adaptation for Medical RAG
Sep 27, 2026Hyojun Ahn, Emily Jimin Roh, Soohyun Park +3Quantum Feature Maps Medical Image Retrieval
Korea University · Sookmyung Women’s University · Virginia Tech +2
Understanding and Exploiting Anisotropy in Post-Training
Sep 26, 2026Samyak Jha, Harshvardhan Saini, Yizhen Liao +2LLM Post-Training Representation Geometry in Language Models
Indian Institute of Technology Dhanbad · National University of Singapore
Reuse or Relearn? A Spectral View of Earth Observation Foundation Models
Sep 26, 2026Mehmet Ozgur Turkoglu, Valerio Marsocci, Dominik J. Mühlematter +3Fine-Tuning Geospatial Foundation Models
Agroscope, EO of Agroecosystems · ESA, Φ-lab · ETH Zurich
Stabilizing the Dynamic Low-Rank Training
Sep 26, 2026Zhonghan Xu, Ling Wang, Junhao Chen +2Low-Rank Approximation Neural Network Training Dynamics
School of Communications and Information Engineering University of Electronic Science and Technology of China
Spectral Reversal: Counteracting Singular Value Bias for Graph Prompting
Sep 26, 2026Hanxu Yang, Yuhuan Zhao, Xiaodong He +1Graph Neural Networks Spectral Bias
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, Sichuan, China
Freeze the Decoder, Heal the Encoder: Parameter-Efficient Adaptation for SVD-Based KV-Cache Compression
Sep 25, 2026Yufeng WangKV-Cache Compression Parameter-Efficient Fine-Tuning
Awaken, Then Scale: Tiny Adaptation for MRI Reconstruction
Sep 25, 2026Mohammed Wattad, Tamir Shor, Alexander M. BronsteinDomain Adaptation Parameter-Efficient Fine-Tuning
Faculty of Computer Science, Technion-Israel Institute of Technology, Haifa, Israel

Oct 2, 2026J/K move · Enter open · S save

FUSEye: Training-Light Fisheye Detection with Overlapping Views and Zero-Initialized Adapters

Wenya Su, Kai Luo, Di Wen, Ruiping Liu+4

School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China · Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Germany

Object Detection Efficient Object Detection Parameter-Efficient Fine-Tuning Adapter Tuning

Open paper AskPDF arXiv

Fisheye cameras give mobile robots a single-sensor, low-cost view of their surroundings, yet the COCO-pretrained detectors that practitioners routinely reuse fail on them: strong radial distortion warps local image structure, while boundary compression shrinks objects to near-invisible sizes. Full fine-tuning closes much of the gap but requires abundant fisheye labels and compute. We present FUSEye, a training-light framework that turns a frozen-backbone COCO-pretrained extra-large YOLO26 detector (YOLO26-x) into a fisheye detector. FUSEye adds roughly 227k new parameters while updating the inserted modules and the pretrained detection head. It addresses the transfer gap at three causally linked levels. At the input level, overlapping grid view generation and box remapping (GridViews) enlarge compressed boundary regions. At the feature level, zero-initialized residual adapters (Z-Adapters) correct distortion-induced feature misalignment. At the decision level, learned cross-projection agreement fusion (AgreeFusion) promotes low-confidence detections only when they are supported by consistent evidence across multiple views. On the WoodScape surround-view fisheye benchmark, FUSEye raises YOLO26-x from 0.148 to 0.266 mAP50 and retains 84.3% fully fine-tuned accuracy. Moreover, randomly using only 25% of the labeled training images, FUSEye achieves 0.2597 mAP50, retaining 97.6% of its full-label performance. FUSEye also consistently improves YOLOv8-11 detectors, showing that the recipe is architecture-agnostic. Source code will be available at https://github.com/Su-wenya/FUSEye.