Parameter-Efficient Fine-Tuning

Also known as PEFT

Parameter-efficient fine-tuning studies how to adapt pretrained models while training only a small subset of weights or adding lightweight modules, reducing resource costs and sometimes helping preserve existing capabilities. Common approaches include low-rank updates such as LoRA and quantized QLoRA, adapters and soft prompts, selective layer or rank tuning, and sparse or routed experts. Methods are evaluated on language benchmarks such as GLUE and mathematical reasoning, as well as vision, speech, multimodal, and continual-learning tasks, often with limited data or hardware. A central tension is that fewer trainable parameters do not necessarily mean lower memory use or faster training, and accuracy, robustness, and retention can vary with the model, task, and placement of updates.

Latest papers

610 papers75 papers in the last four weeks, up 12% on the four weeks before. 0.4% of all new papers.

PointTPA: Dynamic Network Parameter Adaptation for 3D Scene Understanding
Apr 6, 2026Siyuan Liu, Chaoqun Zheng, Xin Zhou +33D Scene Understanding Test-Time Adaptation
Huazhong University of Science and Technology
TAPE: A two-stage parameter-efficient adaptation framework for foundation models in OCT-OCTA analysis
Apr 6, 2026Xiaofei Su, Zengshuo Wang, Minghe Sun +2Vision Foundation Model Adaptation Medical Image Segmentation
Nankai University Optometry & Vision Science Institute, National Key Laboratory of Intelligent Tracking and Forecasting for Infectious Diseases, Tianjin Key Laboratory of Intelligent Robotic, Institute of Robotics and Automatic Information System, Nankai University, Tianjin, 300350, China · Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen, 518083, China
Parameter-Efficient Semantic Augmentation for Enhancing Open-Vocabulary Object Detection
Apr 6, 2026Weihao Cao, Runqi Wang, Xiaoyue Duan +3Open-Vocabulary Object Detection Vision Foundation Model Adaptation
State Key Laboratory of Advanced Rail Autonomous Operation, Beijing Jiaotong University · Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University · School of Computer Science & Technology, Beijing Jiaotong University +1
Aletheia: Gradient-Guided Layer Selection for Efficient LoRA Fine-Tuning Across Architectures
Apr 4, 2026Abdulmalek SaketLLM Fine-Tuning Efficient Language Model Training
Royal Fenice Kft / ALETHEIA PROTOCOL research, Budapest, Hungary
Scalable Variational Bayesian Fine-Tuning of LLMs via Orthogonalized Low-Rank Adapters
Apr 3, 2026Haotian Xiang, Bingcong Li, Qin LuLLM Uncertainty Estimation Bayesian LoRA
School of Electrical and Computer Engineering, University of Georgia, Athens, GA, USA · Department of Computer Science, ETH Zurich, Zurich, Switzerland
Activation-Efficient Fine-Tuning via Sequence-Compressed Low-Rank Subspaces
Apr 3, 2026Irene Tenison, Stella Ahn, Miriam Kim +2Memory-Efficient Fine-Tuning LLM Fine-Tuning
MIT CSAIL · Harvard SEAS
A Paradigm Shift: Fully End-to-End Training for Temporal Sentence Grounding in Videos
Apr 3, 2026Allen He, Qi Liu, Kun Liu +2Temporal Video Grounding Parameter-Efficient Fine-Tuning
BASIS International School Park Lane Harbour · UCAS · JD Explore Academy +1
FourierMoE: Fourier Mixture-of-Experts Adaptation of Large Language Models
Apr 2, 2026Juyong Jiang, Fan Wang, Hong Qi +2LLM Fine-Tuning Parameter-Efficient Fine-Tuning
The Hong Kong University of Science and Technology (Guangzhou) · The Hong Kong University of Science and Technology
MiCA Learns More Knowledge Than LoRA and Full Fine-Tuning
Apr 2, 2026Sten Rüdiger, Sebastian RaschkaLLM Fine-Tuning Parameter-Efficient Fine-Tuning
Independent Researcher · RAIR Lab
S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models
Apr 1, 2026Jack YoungParameter-Efficient Fine-Tuning Recurrent Transformers
Indiana University
From Baselines to Preferences: A Comparative Study of LoRA/QLoRA and Preference Optimization for Mental Health Text Classification
Apr 1, 2026Mihael ArcanPreference Optimization Supervised Fine-Tuning
Home Lab, Galway, Ireland
TALENT: Target-aware Efficient Tuning for Referring Image Segmentation
Apr 1, 2026Shuo Jin, Siyue Yu, Bingfeng Zhang +3Referring Image Segmentation Parameter-Efficient Fine-Tuning
XJTLU · University of Liverpool · China University of Petroleum (East China) +2
CoLA: Cross-Modal Low-rank Adaptation for Multimodal Downstream Tasks
Apr 1, 2026Wish Suharitdamrong, Tony Alex, Muhammad Awais +1Cross-Modal Learning VLM Adaptation
Surrey Institute for People-Centred AI, University of Surrey, Guildford, GU2 7XH, UK · Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK
Meteorology-Driven GPT4AP: A Multi-Task Forecasting LLM for Atmospheric Air Pollution in Data-Scarce Settings
Mar 31, 2026Prasanjit Dey, Soumyabrata Dev, Bianca Schoen-PhelanAir Quality Prediction LLM-Based Time Series Forecasting
ADAPT SFI Research Centre, School of Computer Science, Technological University Dublin, Ireland · ADAPT SFI Research Centre, School of Computer Science, University College Dublin · School of Computer Science, Technological University Dublin, Ireland
Curvature-Guided LoRA: Matching Full Fine-Tuning in Function Space
Mar 31, 2026Frédéric Zheng, Alexandre ProutièreLow-Rank Adaptation Parameter-Efficient Fine-Tuning
KTH, Stockholm, Sweden
One-for-All: A Lightweight Stabilized and Parameter-Efficient Pre-trained LLM for Time Series Forecasting
Mar 31, 2026Prasanjit Dey, Soumyabrata Dev, Bianca Schoen-PhelanLLM-Based Time Series Forecasting Multivariate Time Series Forecasting
ADAPT SFI Research Centre, School of Computer Science, Technological University Dublin, Ireland · ADAPT SFI Research Centre, School of Computer Science, University College Dublin
Adapting SAM to Nuclei Instance Segmentation and Classification via Cooperative Fine-Grained Refinement
Mar 30, 2026Jingze Su, Tianle Zhu, Jiaxin Cai +6Instance Segmentation Histopathology Image Segmentation
College of Computer and Data Science, Fuzhou University, Fuzhou, China · College of Physics and Information Engineering, Fuzhou University, Fuzhou, China · School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou, China
On the Role of Encoder Depth: Pruning Whisper and LoRA Fine-Tuning in SLAM-ASR
Mar 30, 2026Ganesh Pavan Kartikeya Bharadwaj Kolluri, Michael Kampouridis, Ravi ShekharNeural Network Pruning Speech Language Models
School of Computer Science and Electronic Engineering, University of Essex
OPRO: Orthogonal Panel-Relative Operators for Panel-Aware In-Context Image Generation
Mar 29, 2026Sanghyeon Lee, Minwoo Lee, Euijin Shin +3Image Generation Parameter-Efficient Fine-Tuning
Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Korea
Decompose, Mix, Adapt: A Unified Framework for Parameter-Efficient Neural Network Recombination and Compression
Mar 28, 2026Nazia Tasnim, Shrimai Prabhumoye, Bryan A. PlummerNeural Network Compression Parameter-Efficient Fine-Tuning
Boston University · NVIDIA, Boston University
From Pen to Pixel: Translating Hand-Drawn Plots into Graphical APIs via a Novel Benchmark and Efficient Adapter
Mar 27, 2026Zhenghao Xu, Mengning YangData Visualization Chart Understanding
School of Big Data and Software Engineering, Chongqing University, Chongqing, China · School of Big Data & Software Engineering, Chongqing University, Chongqing, China.
GazeQwen: Lightweight Gaze-Conditioned LLM Modulation for Streaming Video Understanding
Mar 26, 2026Trong Thang Pham, Hien Nguyen, Ngan LeMultimodal Large Language Models Streaming Video Understanding
University of Arkansas · University of Houston
Correcting Suppressed Log-Probabilities in Language Models with Post-Transformer Adapters
Mar 25, 2026Bryan SanchezLLM Alignment Political Bias in Language Models
Parameter-Efficient Fine-Tuning for Medical Text Summarization: A Comparative Study of Lora, Prompt Tuning, and Full Fine-Tuning
Mar 23, 2026Ulugbek Shernazarov, Rostislav Svitsov, Bin ShiParameter-Efficient Fine-Tuning Text Summarization
Telecom SudParis, Institut Polytechnique de Paris, France
Parameter-efficient Prompt Tuning and Hierarchical Textual Guidance for Few-shot Whole Slide Image Classification
Mar 23, 2026Jayanie Bogahawatte, Sachith Seneviratne, Saman HalgamugeWhole-Slide Image Analysis Parameter-Efficient Fine-Tuning
AI, Optimization and Pattern Recognition Research Group, Dept. of Mechanical Eng., University of Melbourne, Australia
Efficient Fine-Tuning Methods for Portuguese Question Answering: A Comparative Study of PEFT on BERTimbau and Exploratory Evaluation of Generative LLMs
Mar 22, 2026Mariela M. Nina, Caio Veloso Costa, Lilian Berton +1Parameter-Efficient Fine-Tuning Question Answering
Institute of Science and Technology, Federal University of São Paulo (UNIFESP), São José dos Campos, SP, Brazil
Frequency Switching Mechanism for Parameter-E!cient Multi-Task Learning
Mar 22, 2026Shih-Wen Liu, Yen-Chang Chen, Wei-Ta Chu +2Multi-Task Learning Parameter-Efficient Fine-Tuning
National Cheng Kung University, Tainan, Taiwan · NVIDIA Research, Taiwan
Learning Progressive Adaptation for Multi-Modal Tracking
Mar 22, 2026He Wang, Tianyang Xu, Zhangyong Tang +2Multimodal Visual Object Tracking Multimodal Fusion
School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China · Centre for Vision, Speech and Signal Processing, University of Surrey, GU2 7XH Guildford, U.K.
FAAR: Efficient Frequency-Aware Multi-Task Fine-Tuning via Automatic Rank Selection
Mar 20, 2026Maxime Fontana, Michael Spratling, Miaojing ShiNonlinear LoRA Parameter-Efficient Fine-Tuning
King’s College London London, UK · University of Luxembourg Esch-sur-Alzette, Luxembourg · Tongji University Shanghai, China
An Empirical Study of SFT-DPO Interaction and Parameterization in Small Language Models
Mar 20, 2026Yuming Feng, Christy YangDirect Preference Optimization Supervised Fine-Tuning
Department of Computer Science, Stanford University

Apr 6, 2026J/K move · Enter open · S save

PointTPA: Dynamic Network Parameter Adaptation for 3D Scene Understanding

Siyuan Liu, Chaoqun Zheng, Xin Zhou, Tianrui Feng+2

Huazhong University of Science and Technology

3D Scene Understanding Test-Time Adaptation 3D Semantic Segmentation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Scene-level point cloud understanding remains challenging due to diverse geometries, imbalanced category distributions, and highly varied spatial layouts. Existing methods improve object-level performance but rely on static network parameters during inference, limiting their adaptability to dynamic scene data. We propose PointTPA, a Test-time Parameter Adaptation framework that generates input-aware network parameters for scene-level point clouds. PointTPA adopts a Serialization-based Neighborhood Grouping (SNG) to form locally coherent patches and a Dynamic Parameter Projector (DPP) to produce patch-wise adaptive weights, enabling the backbone to adjust its behavior according to scene-specific variations while maintaining a low parameter overhead. Integrated into the PTv3 structure, PointTPA demonstrates strong parameter efficiency by introducing two lightweight modules of less than 2% of the backbone's parameters. Despite this minimal parameter overhead, PointTPA achieves 78.4% mIoU on ScanNet validation, surpassing existing parameter-efficient fine-tuning (PEFT) methods across multiple benchmarks, highlighting the efficacy of our test-time dynamic network parameter adaptation mechanism in enhancing 3D scene understanding. The code is available at https://github.com/H-EmbodVis/PointTPA.