Parameter-Efficient Fine-Tuning

Also known as PEFT

Ask about this topic

Overview

Parameter-efficient fine-tuning studies how to adapt pretrained models while training only a small subset of weights or adding lightweight modules, reducing resource costs and sometimes helping preserve existing capabilities. Common approaches include low-rank updates such as LoRA and quantized QLoRA, adapters and soft prompts, selective layer or rank tuning, and sparse or routed experts. Methods are evaluated on language benchmarks such as GLUE and mathematical reasoning, as well as vision, speech, multimodal, and continual-learning tasks, often with limited data or hardware. A central tension is that fewer trainable parameters do not necessarily mean lower memory use or faster training, and accuracy, robustness, and retention can vary with the model, task, and placement of updates.

Latest papers

CardsList

610 papers75 papers in the last four weeks, up 12% on the four weeks before. 0.4% of all new papers.

  1. EPM-JEPA: Operator-Side Experience Modulation in JEPA-Family World Models

    Jun 11, 2026Vedant PandyaTest-Time AdaptationWorld Model Learning

    School of Artificial Intelligence and Data Engineering (SAIDE) Indian Institute of Technology Jodhpur

  2. LoRA-Muon: Spectral Steepest Descent on the Low-Rank Manifold

    Jun 11, 2026Franz Louis Cesista, Katherine Crowson, Cédric Simal +1Deep Learning OptimizationRiemannian Optimization

    Ateneo de Manila University · EleutherAI · NaXys +1

  3. Beyond Prompt and Parser Engineering: Soft-Prompt Tuning for Automated LLM Evaluation

    Jun 10, 2026Selen Erkan, Bastian Boll, Kristian Kersting +2LLM EvaluationInstruction Following

    Aleph Alpha Research Lab1141 · TU Darmstadt Hessian.AI Lab1141

  4. Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training

    Jun 10, 2026Michal Chudoba, Sergey Alyaev, Petra Galuscakova +1LLM Fine-TuningMultimodal Large Language Models

    University of Stavanger, Stavanger, Norway · NORCE Research, Bergen, Norway

  5. Substrate Asymmetry in User-Side Memory: A Diagnostic Framework

    Jun 10, 2026Youwang DengRetrieval-Augmented GenerationLLM Personalization

    EpistemicaLab — Independent Research

  6. Parameter-Efficient Adapter Tuning for Tabular-Image Multimodal Learning

    Jun 10, 2026Jiaqi LuoFine-TuningAdapter Tuning

    School of Mathematical Sciences, Soochow University, No.1 Shizi Street, Suzhou, 215006, Jiangsu Province, China

  7. GRAFT: Gain-Recalibrated Adapters for Transformer-Based Neural Population Activity Modeling

    Jun 9, 2026Xiangsheng Ge, Yang XieBrain-Computer InterfacesTransformer

    Shanghai Jiao Tong University

  8. Small Data, Big Noise: Adversarial Training for Robust Parameter-Efficient Fine-Tuning

    Jun 9, 2026Eitan Cohen, Idan Simai, Uri ShahamAdversarial TrainingNeural Network Robustness

    Bar-Ilan University · Ramat Gan, Israel

  9. 5% > 100%: Flatness Preference is All You Need for Multimodal Parameter-Efficient Fine-Tuning

    Jun 9, 2026Yifan Zhu, Can Lin, Hangjie Yuan +4Deep Learning OptimizationFlat Minima

    School of Computer Sciences, Beijing University of Posts and Telecommunications, Beijing 100876, China · College of Computer Science and Technology, Zhejiang University, Zhejiang 310027, China · ETH Zürich, Switzerland

  10. Advancing the State-of-the-Art in Empirical Privacy Auditing

    Jun 9, 2026Nicole Mitchell, Galen Andrew, Arun Ganesh +2Data LeakagePrivacy Auditing

    Google Research San Francisco, CA · Google Research Seattle, WA

  11. A Unified Adaptive Feature Composition Framework for Multi-Task Generalization in Wireless Foundation Models

    Jun 9, 2026Yuxuan Shi, Tingting Yang, Kangning Ma +4Wireless Foundation ModelsAdaptive Model Routing

    Department of Broadband Communication, Pengcheng Laboratory, Shenzhen, 518000, China

  12. Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning

    Jun 8, 2026Ghodsiyeh Rostami, Po-Han Chen, Mahdi S. HosseiniFine-TuningVision Foundation Model Adaptation

    Concordia University · Concordia University Mila - Québec AI Institute

  13. LiteVSR: Lightweight Adaptation of Frozen Diffusion Transformers for Video Super-Resolution

    Jun 8, 2026Yu Cao, Ziquan Liu, Zhensong Zhang +3Video Diffusion ModelsDiffusion Transformer

    Queen Mary University of London · Huawei Darwin Research Center · Imperial College London

  14. How Small Can You Go? LoRA Fine-Tuning 270M-8B Models for Merchant Information Extraction in Financial Transactions

    Jun 6, 2026Donghao Huang, Tomas Drietomsky, Benjamin Barrett +1Financial ServicesMemory-Efficient Fine-Tuning

    Research and Development, Mastercard, Arlington, VA, USA · School of Computing and Information Systems, Singapore Management University, Singapore · Economic Intelligence, Mastercard, Prague, Czech Republic

  15. The Fine-Tuning Trap: Evaluating Negative Transfer and the Role of PEFT in Sub-1B Mathematical Reasoning

    Jun 5, 2026Rahul Nair, Chun TaoFine-TuningSmall Language Models

  16. TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning

    Jun 4, 2026Marius Dragoi, Ioana Pintilie, Alexandra Dragomir +2Continual LearningLow-Rank Adaptation

    Bitdefender, Romania

  17. Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution

    Jun 4, 2026Liliana Hotsko, Yinxi Li, Yuntian Deng +1Software EngineeringLLM Fine-Tuning

    University of Waterloo

  18. Rethinking LoRA Memory Through the Lens of KV Cache Compression

    Jun 4, 2026Chunsheng Zuo, Liaoyaqi Wang, William Jurayj +2Retrieval-Augmented GenerationMemory-Augmented Language Models

    Johns Hopkins University

  19. KV-Control: Parameter-Efficient K/V Injection for Trajectory-Controlled Text-to-Motion

    Jun 4, 2026Tengjiao Sun, Pengcheng Fang, Xiaoyu Zhan +4Text-to-Motion GenerationHuman Motion Generation

    University of Southampton · 2Mogo AI Ltd. · 3Nanjing University

  20. Dominant-Layer ZO: A Single Layer Dominates Zeroth-Order Fine-Tuning of LLMs

    Jun 3, 2026Wanhao Yu, Ziyan Wang, Zheng Wang +7Fine-TuningMemory-Efficient Fine-Tuning

    University of North Carolina at Charlotte · University of Houston · State University of New York at Binghamton +2

  21. TaDA: Calibrated Probe Gating for Task-Domain LoRA Merging

    Jun 3, 2026Huy Quoc To, Fuyi Li, Guangyan Huang +1Adapter TuningLow-Rank Adaptation

    Deakin University, Melbourne, VIC, Australia · Adelaide University, Adelaide, SA, Australia

  22. Parameter-Efficient Fine-Tuning with Learnable Rank

    Jun 3, 2026Arpit Garg, Simon Lucey, Hemanth SaratchandranNonlinear LoRALow-Rank Adaptation

    Australian Institute for Machine Learning Adelaide University

  23. Where Do We (Not) Need Temporal Context in Low-Resource Video Task Adaptation?

    Jun 2, 2026Luc P. J. Sträter, Hazel DoughtyLinear ProbingVision Foundation Model Adaptation

    Leiden University

  24. Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter

    Jun 2, 2026Zhengbao He, Ruiqi Ding, Zhehao Huang +3Adapter TuningLow-Rank Adaptation

    Institute of Image Processing and Pattern Recognition, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China · Shanghai Key Laboratory of Flexible Medical Robotics, Tongren Hospital, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China

  25. Message Tuning Outshines Graph Prompt Tuning: A Prismatic Space Perspective

    Jun 2, 2026Yancheng Chen, Dun Ma, Shuai Zhang +6Prompt TuningGraph Foundation Models

    Academy of Mathematics and Systems Science, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Institute of Information Engineering, Chinese Academy of Sciences +4

  26. EURO-5K: When Does Domain Pretraining Matter? Benchmarking Transformers for EU Reporting Obligation Extraction

    Jun 2, 2026Marios Koniaris, Vasileios Kotronis, Eugenia Giannini +1Language Model PretrainingLegal NLP

    Division of Computer Science, School of Electrical and Computer Engineering, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780, Athens, Greece. · Department of Humanities Social Sciences and Law, School of Applied Mathematical and Physical Sciences, National Technical University of Athens, Iroon Polytechniou 9, Zographou Campus, 15780, Athens, Greece.

  27. CRAM: Centroid-Routing and Adaptive MoE for Multimodal Continual Instruction Tuning

    Jun 1, 2026Jun-Tao Tang, Zhen-Hao Xie, Yu-Cheng Shi +1Mixture-of-Experts ModelsMultimodal Instruction Tuning

    School of Artificial Intelligence, Nanjing University, China · State Key Laboratory of Novel Software Technology, Nanjing University, China

  28. On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters

    Jun 1, 2026Mind Lab, :, Vin Bo +64LLM Fine-TuningAdapter Tuning

  29. FACT: A Simple and Efficient Framework for Active Finetuning

    Jun 1, 2026Wenshuai Xu, You Song, Yuzhuo Cui +3Active LearningImage Classification

  30. Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks

    Jun 1, 2026Nermeen Abou Baker, David Rohrschneider, Uwe HandmannFine-TuningInstance Segmentation

    Ruhr West University of applied sciences · Lützowstraße 5, 46236 Bottrop, Germany: Ruhr West University of Applied Sciences