Parameter-Efficient Fine-Tuning

Also known as PEFT

Overview

Parameter-efficient fine-tuning studies how to adapt pretrained models while training only a small subset of weights or adding lightweight modules, reducing resource costs and sometimes helping preserve existing capabilities. Common approaches include low-rank updates such as LoRA and quantized QLoRA, adapters and soft prompts, selective layer or rank tuning, and sparse or routed experts. Methods are evaluated on language benchmarks such as GLUE and mathematical reasoning, as well as vision, speech, multimodal, and continual-learning tasks, often with limited data or hardware. A central tension is that fewer trainable parameters do not necessarily mean lower memory use or faster training, and accuracy, robustness, and retention can vary with the model, task, and placement of updates.

Latest papers

CardsList

646 papers42 papers in the last four weeks, down 51% on the four weeks before. 0.3% of all new papers.

  1. Parameter-Efficient Adaptation of Self-supervised Vision Backbones Survives Distribution Shift in Radar Micro-Doppler HAR

    Nov 19, 2026Keryan Chelouche, Petr Dobias, Nistor Grozavu +1Human Activity RecognitionParameter-Efficient Fine-Tuning

  2. Continual Tool-Space Extension Learning for Multimodal Agents

    Nov 18, 2026Jinbae Seo, Jungin Park, Hyunsung Jang +3Multimodal Tool UseMultimodal Agents

  3. State-Message Interaction Adapter for Cross-Domain Few-Shot Segmentation

    Nov 16, 2026Hang Xing, Yan Xia, Yongjiang Liu +2Few-Shot Semantic SegmentationSemantic Segmentation

  4. Tight PAC-Bayes Generalisation Guarantees for Large Language Model Judges

    Nov 16, 2026Tom A. Lamb, Philip H. S. Torr, Tim G. J. RudnerLLM-as-a-JudgeLanguage Model Safety Evaluation

  5. MarineSparkLM-4B: A Lightweight Language Model for Marine and Offshore Engineering Built by Parameter-Efficient Adaptation

    Nov 14, 2026Cheng Siong ChinLLM Fine-TuningDomain Adaptation for LLMs

  6. Adaptable Watermarking: Transforming Image Watermarking Models via Lightweight Adapters

    Nov 13, 2026Mehrdad Saberi, Vishal Asnani, Shruti Agarwal +2Robust WatermarkingParameter-Efficient Fine-Tuning

  7. Direction or Magnitude? Learning Scalars over Task-Calibrated Directions

    Nov 13, 2026Ziqing Wen, Xiaohui Song, Jiahuan Wang +5LLM Fine-TuningParameter-Efficient Fine-Tuning

  8. Sustainable Federated Fine-Tuning with LoRA: Environmental Impact and Performance Footprint

    Nov 12, 2026Pedro Augusto Londe Ribeiro, Rodrigo Moreira, Larissa Ferreira Rodrigues Moreira +3Energy-Efficient MLFederated Fine-Tuning

  9. Quantum-inspired orthogonal adapters for certifiably robust medical image classification

    Nov 12, 2026Olga Tsurkan, Laia Domingo, Victoria Orlova +3Certified RobustnessMedical Image Classification

  10. Paired Geometric Supervision for Generalizable Deepfake Detection

    Nov 11, 2026Luke Collins, Chang-Tsun Li, Duc Thanh Nguyen +1Deepfake DetectionVision Foundation Model Adaptation

  11. 3R-Adapter: Retrieval, Rewiring, and Refinement for Efficient Adaptation of 3D Reconstruction Model

    Nov 9, 2026Zixiang Ai, Zhenyu Cui, Xu Zou +13D ReconstructionFeed-Forward 3D Reconstruction

  12. VeilTune: Secure Fine-Tuning of Private LoRA Adapters on Public Pretrained Language Models

    Nov 9, 2026Huiyu Xie, Tanping Zhou, Tao Hu +2Fine-Tuning Language ModelsPrivacy-Preserving Language Models

  13. TID System for Speech-COSER Challenge 2026: ASR Track

    Nov 8, 2026Fernando López, Jordi LuqueAutomatic Speech RecognitionSpeech Recognition Benchmarks

  14. Full Fine-Tuning is not the Problem: Why Adam Fails and SGD Succeeds in Few-Shot CLIP Adaptation

    Nov 8, 2026Nairouz Mrabah, Ismail Ben Ayed, Eric GrangerFew-Shot VLM AdaptationAdaptive Gradient Methods

  15. Not All Layers Are Equal in Image-to-Video Transfer

    Nov 8, 2026Thinesh Thiyakesan Ponbagavathi, Constantin Seibold, Alina RoitbergFine-Grained Action UnderstandingVision Foundation Model Adaptation

  16. The Surprising Effectiveness of Deleting Weights in LLM Post-Training

    Nov 7, 2026Jack Lu, Zhenbang Yang, Mike Lasby +3LLM Post-TrainingParameter-Efficient Fine-Tuning

  17. FedLoVA: Value-Only Aggregation for Federated LoRA Fine-Tuning of Large Language Models

    Nov 6, 2026Ensieh Khazaei, Baturalp Buyukates, Dimitrios HatzinakosLLM Fine-TuningFederated Fine-Tuning

  18. From What Is Used to What Is Missing: Rethinking Dynamic Rank in LoRA

    Nov 5, 2026Peng Xue, Guodong Fan, Guang-Yong Chen +1Low-Rank AdaptationParameter-Efficient Fine-Tuning

  19. Backbone-Informed Adaptation after EEG Pretraining: A Capacity-Controlled Study

    Nov 4, 2026Mingyang Jiang, Yamin Li, Fan Ma +3ElectroencephalographyEEG Foundation Models

  20. MATA: Motion-aligned Temporal Adapter

    Nov 4, 2026Ashish Kumar Singh, Prakash Chandra ChhipaVideo Action RecognitionFew-Shot Action Recognition

  21. Training Specialist LLMs on Your Phone: Forward-Only Optimization via Block-Spectral Coordinates

    Nov 3, 2026Liyan Tan, Yequan Zhao, Ziyue Liu +3On-Device TrainingLLM Training

  22. SaMA: Morpho Adaptation via Asymmetric Expansion of Kronecker Product

    Nov 3, 2026An Nguyen, Anh TongLow-Rank AdaptationParameter-Efficient Fine-Tuning

  23. Mamba3R: Fixed-Capacity Selective State Memory for Long-Sequence 3D Reconstruction

    Nov 3, 2026Zhenxiang Xiong, Tianchen Deng, Nailin Wang +3Streaming 3D Reconstruction3D Reconstruction

  24. EEG-HealthLLM: Hierarchical Adaptation of Large Language Models for Unified Heterogeneous EEG Health Assessment

    Nov 1, 2026Ruidong Ma, Jisheng Dang, Ronghao Lin +7ElectroencephalographyEEG Decoding

  25. Scute: Exact Batched Serving for PEFT Adapters Beyond LoRA

    Oct 31, 2026Thomas Walker, Richard BaraniukLLM InferenceLLM Inference Acceleration

  26. ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning

    Oct 30, 2026Chenyu Zhou, Yuxuan Zeng, Qi Deng +2LLM Fine-TuningLow-Rank Adaptation

  27. Breaking Information Islands in Sparse Tuning via Small-World Connectivity

    Oct 30, 2026Xianchao Guan, Zijun Xiong, Yifeng Wang +5Sparse Fine-TuningFine-Tuning Language Models