Parameter-Efficient Fine-Tuning Methods

Ask about this topic

Momentum

10 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 101

All topics
CardsList

  1. RoSA: Rotational Sparse Adaptation for Memory-Efficient Fine-Tuning

    Oct 5, 2026Muhammad Azeem Lodhi, Chao Zhou, Rebekka BurkholzParameter-Efficient Fine-Tuning Methods

    Saarland University, Saarbrücken, Germany · CISPA Helmholtz Center for Information Security, Saarbrücken, Germany

  2. AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Models

    Oct 1, 2026Arash Lagzian, Paniz Halvachi, Junming Zhang +2MuonAdam

    National University of Singapore, Singapore · Independent Researcher · LUMIA Lab, School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China

  3. ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs

    Sep 30, 2026Hang Yin, Haozhe Wang, Yuhua Luo +3Replay-Based Continual LearningParameter-Efficient Fine-Tuning Methods

    Shanghai Jiao Tong University · Shanghai Innovation Institute · Beijing Zhongguancun Academy +3

  4. Fenchel Tilting: Weighted Correction for Efficient Finetuning of Generative Models

    Sep 30, 2026Maksim Bobrin, Maksim Zhdanov, Dmitry DylovGenerative Flow NetworksGenerative Models

    AXXX · Applied AI Institute · MBZUAI

  5. MaskCoFT: Masked Co-Adaptive Fine-Tuning for Memory-Efficient MoE Inference

    Sep 28, 2026Junfeng Wu, Zehao Fan, Hadjer Benmeziane +3Mixture-Of-ExpertsParameter-Efficient Fine-Tuning Methods

    Department of Industrial and Systems Engineering, Rensselaer Polytechnic Institute · Department of Electrical, Computer, and Systems Engineering, Rensselaer Polytechnic Institute · IBM Research

  6. Optimizing the Phi-2 Small Language Model for Real-time Chatbot Applications Using Parameter-Efficient Fine-Tuning (PEFT) with QLoRA Quantization

    Sep 27, 2026PhanTan Khanh Nguyen, Ashfaq Ali Shafin, Khandaker Mamun AhmedLarge Language Model QuantizationSmall Large Language Models

    Beacom College of Computer and Cyber Sciences, Dakota State University · Mathematics and Computer Science, Augustana College

  7. From Experts to Sub-experts: Fine-grained Parameter-Efficient Fine-Tuning for MoE LLMs

    Sep 22, 2026Zhentao Tan, Chang Liu, Yao Liu +2Parameter-Efficient Fine-Tuning MethodsMixture-Of-Experts Large Language Models

    Alibaba Group · Southeast University

  8. GAPrompt++: Multi-Granular Geometry-Aware Point Cloud Prompt for 3D Vision Model

    Sep 17, 2026Zixiang Ai, Zhenyu Cui, Yufei Guo +4Point Clouds3D Geometry

    Wangxuan Institute of Computer Technology, Peking University, Beijing 100871, China · Intelligent Science and Technology Academy of CASIC, Beijing 100041, China · Institute of Software, Chinese Academy of Sciences, Beijing 100190, China +1

  9. Pre-PEFT Probing: Weight Statistics and Perturbation Robustness for Layer Selection in VLM Vision Encoders

    Sep 14, 2026Qingtao Xia, Jiahua Bao, Siyao Cheng +1Parameter-Efficient Fine-Tuning MethodsVision-Language Model Adaptation

    Faculty of Computing Harbin Institute of Technology Harbin, China

  10. Hyperparameter Scaling Laws Across MoE Sparsity

    Sep 8, 2026Changxin Tian, Kunlong Chen, Jia Liu +3HyperparameterMixture-Of-Experts Architectures

    Ling Team, Ant Group

  11. MI-PEFT: Mixture-of-Experts Integrated Parameter-Efficient Fine-Tuning Protein Language Models Improves Acidophilic Proteins Classification

    Sep 7, 2026Honghan ShenLanguage ModelingParameter-Efficient Fine-Tuning Methods

    University of Wisconsin–Madison, Madison, WI 53706, USA

  12. RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks

    Aug 31, 2026Xingran Chen, Rohit Bhagat, Ghadir Ayache +3Hip-LoraParameter-Efficient Fine-Tuning Methods

    Engineering Systems and Design Pillar, Singapore University of Technology and Design, Singapore 487372 · Department of Electrical and Computer Engineering, Rutgers University, Piscataway Township, NJ 08854, USA · LinkedIn, New York, NY 10118, USA +2

  13. From Manuals to Maintenance: Fine-Tuning MedGemma for Multi-Modal Imaging System Support in Low-Resource Settings

    Aug 9, 2026Bernes Lorier Atabonfack, Zion Kongbi Nfo, Ahmed Tahiru Issah +9Multimodal Clinical DataModel Fine-Tuning

    Carnegie Mellon University Africa, Kigali, Rwanda · IngenziAI, Kigali, Rwanda · Medical Artificial Intelligence Laboratory, Lagos, Nigeria +5

  14. Circuit Fine-Tuning for Compute-Efficient Transformer Adaptation

    Aug 8, 2026Uri Z. Kialy, Gil Ben-ArtziParameter-Efficient Fine-Tuning MethodsSelf-Supervised Vision Transformers

    School of Computer Science, Ariel University, Israel

  15. TEMPER: Tensorized Efficient Manifold-constrained Parameterization for Expressive Residual Routing

    Aug 8, 2026Yuxuan Gu, Wuyang Zhou, Huijun Xing +1Residual ConnectionsReparameterization

    Department of Electrical and Electronic Engineering, Imperial College London

  16. YOLO-PEFT: Parameter-Efficient Fine-Tuning on YOLO Family

    Aug 7, 2026Xu Lin, WenJie Nie, Jinlong Peng +4Yolo26Parameter-Efficient Fine-Tuning Methods

    Tencent · Xiamen University

  17. Beyond Rotations: AuroOFT for Expressive Quantized Orthogonal Fine-Tuning

    Aug 5, 2026Yue Han, Dianlin WangLarge Language Model QuantizationParameter-Efficient Fine-Tuning Methods

    College of Systems Engineering, National University of Defense Technology · College of Computer Science and Technology, National University of Defense Technology

  18. Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUs

    Aug 5, 2026Kuanysh Akhmetzhanov, Jurn-Gyu ParkParameter-Efficient Fine-Tuning MethodsSmall Large Language Models

    Department of Computer Science, Nazarbayev University, Astana 010000, Kazakhstan

  19. HyPASE: Hyperbolic Geometry for Parameter-Efficient Speech Emotion Fine-Tuning Framework for Large Audio-Language Models

    Aug 5, 2026Tian Jin, Ruikang Zhang, Zefeng Zhao +2Large Audio Language ModelsEmotion Recognition

    Tongji University Shanghai, China · Peking University Beijing, China · The Chinese University of Hong Kong, Shenzhen Shenzhen, China

  20. Z-PEFT: Zero-shot Backdoor Detection in Parameter-Efficient Fine-Tuning via Canonical Spectral Signatures

    Aug 3, 2026Nicola Pitzalis, Donald Shenaj, Giacomo Cignoni +3Clean Label Backdoor AttackParameter-Efficient Fine-Tuning Methods

    University of Pisa

  21. EulerLoRA: Rank-Driven Jump Dynamics for Calibrated Parameter-Efficient Fine-Tuning

    Aug 2, 2026Srinivas Anumasa, Dianbo LiuLow-Rank Adaptation AdaptersParameter-Efficient Fine-Tuning Methods

    National University of Singapore

  22. The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs

    Jul 31, 2026Jiajia Tang, Sizhe Yuen, Francisco Gomez Medina +2Parameter-Efficient Fine-Tuning MethodsAutomated

    The Alan Turing Institute, London, UK · King’s College London, London, UK · University of Southampton, Southampton, UK

  23. Federated Foundation Models Fine-Tuning with Heterogeneous Compressed Clients

    Jul 31, 2026Shengkun Zhu, Jinshan Zeng, Zhihua Allen-Zhao +5FedavgParameter-Efficient Fine-Tuning Methods

    The Hong Kong Polytechnic University · OceanBase, Ant Group · School of Management, Xi’an Jiaotong University +3

  24. How Small Can You Go? A Controlled Study of LoRA Rank, Target Modules, and Quantization Trade-offs for Text-to-SQL on a 60M-Parameter Model

    Jul 28, 2026Mahendra Singh Rathor, Anagheem AzzamLarge Language Model QuantizationParameter-Efficient Fine-Tuning Methods

    Independent Researcher · Jaipur, India · Hamburg, Germany

  25. Three-Pronged Spectral Control for Federated Parameter Efficient Fine Tuning

    Jul 23, 2026Shiva Raj Pokhrel, Dipsan Bhattarai, Anwar WalidParameter-Efficient Fine-Tuning MethodsTuning

    School of IT, Deakin University, Australia · Columbia University, USA

  26. SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

    Jul 22, 2026Dongfang Li, Xiaodong Luo, Ruoyu Sun +62Optimization ModelingLarge Language Model Training

  27. Point Ladder Tuning: Parameter-Efficient Hierarchical Adaptation for 3D Point Cloud Understanding

    Jul 21, 2026Junlin Chang, Longhao Zou, Rui LiPoint CloudsParameter-Efficient Fine-Tuning Methods

    Beihang University, Beijing · Pengcheng Laboratory, Shenzhen

  28. Manifold-Constrained Hyper-Connections for Parameter-Efficient Finetuning

    Jul 20, 2026Valentijn Oldenburg, Floris de Kam, Bente Zuijdam +4Parameter-Efficient Fine-Tuning MethodsModel Fine-Tuning

    University of Amsterdam, Netherlands.

  29. AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning

    Jul 20, 2026Bas Meuwissen, Vasileios Tsouvalas, Nirvana MeratniaParameter-Efficient Fine-Tuning MethodsPretraining

    Eindhoven University of Technology, Eindhoven, The Netherlands

  30. Federated Lightweight Fine-Tuning

    Jul 20, 2026Radhakrishna Achanta, Will ReedParameter-Efficient Fine-Tuning MethodsModel Fine-Tuning

    Cisco Systems Inc., USA

  31. Self-Improving is Often Sudden: Enlightenment-style Finetuning for Large-Scale Models

    Jul 15, 2026Jing-Xiao Liao, Tianwei Zhang, Yu-Hao Jiang +3Parameter-Efficient Fine-Tuning MethodsModel Size

    Frontier of Artificial Network Lab, Department of Data Science, City University of Hong Kong, Hong Kong SAR, China · The Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China, The Chinese University of Hong Kong-Shenzhen, Shenzhen, China · School of Computer Science and Artificial Intelligence, Guangdong University of Education, Guangzhou 510303, China +1

  32. Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning

    Jul 10, 2026Ivan Ilin, Philip Zmushko, Peter RichtárikParameter-Efficient Fine-Tuning MethodsModel Fine-Tuning

    KAUST

  33. LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models

    Jul 8, 2026Sojung An, Junha Lee, Sujeong You +2Parameter-Efficient Fine-Tuning MethodsVision Foundation Models

    Korea University, Seoul, Republic of Korea · Korea Institute of Industrial Technology, Ansan, Republic of Korea · Seoul National University, Seoul, Republic of Korea

  34. Efficient PEFT Methods with Adaptive Checkpointing for Vision Models and VLMs on Resource Constrained Consumer-GPUs

    Jul 2, 2026Altay Toktassyn, Jurn-Gyu ParkParameter-Efficient Fine-Tuning MethodsVision Foundation Models

    Department of Computer Science, Nazarbayev University, Astana 010000, Kazakhstan

  35. EPnG: Adaptive Expert Prune-and-Grow for Parameter-Efficient MoE Fine-tuning

    Jul 2, 2026Ahin Lee, Sehyun Yun, Taesik GongParameter-Efficient Fine-Tuning MethodsMixture-Of-Experts

    Ulsan National Institute of Science & Technology (UNIST)

  36. Model Merging as Probabilistic Inference in Fine-Tuning Parameter Space

    Jul 2, 2026Long Minh Bui, Tuan Anh Le Van, Tung Phi Duc +3Continual Model MergingProbabilistic Inference

    Washington State University · Hanoi University of Science and Technology

  37. Token Geometry

    Jul 1, 2026Kathan ShahParameter-Efficient Fine-Tuning MethodsMatrix

    Preprint.

  38. SpiralFovea: Input-Adaptive Foveated Tokenization as a Third Lever of Resource-Adaptive Inference

    Jul 1, 2026Kyan Mahajan, Mohammad SaqlainParameter-Efficient Fine-Tuning MethodsWireless Foundation Models

    Indian Institute of Information Technology, Allahabad, Uttar Pradesh, India.

  39. Mixture-of-Control: State-Aware Fine-Tuning for Transformer-based Models

    Jun 30, 2026Duc Anh Nguyen, Tien Ngoc Luu, Tung Pham +1Parameter-Efficient Fine-Tuning MethodsTransformer Architectures

    Qualcomm AI Research

  40. Structured Hyperedge Adaptation for Parameter-Efficient Fine-Tuning of Vision Transformers

    Jun 21, 2026Edwin Kwadwo Tenagyei, Lei Wang, Ugochukwu Ejike Akpudo +2Parameter-Efficient Fine-Tuning MethodsVision Transformer

    Griffith University, Nathan QLD 4111, Australia · Data61/CSIRO, Canberra ACT 2601, Australia

  41. MixedPEFT: Combining Multiple PEFT Methods with Mixed Objectives for Unsupervised Domain Adaptation

    Jun 20, 2026Mohammed Rawhani, Dervis Karaboga, Ozkan Ufuk Nalbantoglu +2Domain AdaptationParameter-Efficient Fine-Tuning Methods

    Computer Engineering Department Erciyes University Kayseri, Türkiye

  42. Predicting Mergeability of Parameter-Efficient Fine-Tuning Updates

    Jun 17, 2026Lin Tang, Wei Zhang, Jing Li +3Large Language Model AdaptationParameter-Efficient Fine-Tuning Methods

    1Sichuan University, Chengdu, China · University of Electronic Science and Technology of China, Chengdu, China

  43. ARIADNE: Agnostic Routing for Inference-time Adapter DyNamic sElection

    Jun 17, 2026Enrico Cassano, Michał Brzozowski, Zuzanna Dubanowska +2Parameter-Efficient Fine-Tuning MethodsLarge Language Model Routing

    University of Turin · Samsung AI Center, Warsaw, Poland

  44. LiFT: Local Search via Linear Programming for Overfitting-Controlled Transformers

    Jun 15, 2026Abhishek Shukla, Anikeit Khanna, Ankur Sinha +1Parameter-Efficient Fine-Tuning MethodsModel Fine-Tuning

    Department of Management Sciences, Indian Institute of Technology Kanpur, Kanpur-208016, Uttar Pradesh, India · Brij Disa Centre for Data Science and AI, Indian Institute of Management Ahmedabad, Ahmedabad-380015, Gujarat, India · Department of Civil Engineering, Indian Institute of Technology Kanpur, Kanpur-208016, Uttar Pradesh, India +1

  45. Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning

    Jun 12, 2026Dmitriy Bystrov, Daniil Medyakov, Dmitry Bylinkin +1Parameter-Efficient Fine-Tuning MethodsZeroth-Order Optimization

    1Moscow Independent Research Institute of Artificial Intelligence · 2Basic Research of Artificial Intelligence Laboratory (BRAIn Lab) · 3Innopolis University

  46. From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model Merging

    Jun 10, 2026Zhenqian Zhu, Yamin Hu, Yiya Diao +3Continual Model MergingClean Label Backdoor Attack

    Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, 518055, China. · Shenzhen Key Laboratory of Media Security, Shenzhen University, Shenzhen, 518060, China.

  47. Small Data, Big Noise: Adversarial Training for Robust Parameter-Efficient Fine-Tuning

    Jun 9, 2026Eitan Cohen, Idan Simai, Uri ShahamParameter-Efficient Fine-Tuning MethodsAdversarial Training

    Bar-Ilan University · Ramat Gan, Israel

  48. 5% > 100%: Flatness Preference is All You Need for Multimodal Parameter-Efficient Fine-Tuning

    Jun 9, 2026Yifan Zhu, Can Lin, Hangjie Yuan +4Parameter-Efficient Fine-Tuning MethodsModel Fine-Tuning

    School of Computer Sciences, Beijing University of Posts and Telecommunications, Beijing 100876, China · College of Computer Science and Technology, Zhejiang University, Zhejiang 310027, China · ETH Zürich, Switzerland

  49. Fisher-Guided Progressive Parameter Selection for Adaptive Fine-Tuning

    Jun 8, 2026Ghodsiyeh Rostami, Po-Han Chen, Mahdi S. HosseiniParameter-Efficient Fine-Tuning MethodsParameter-Efficient Adaptation

    Concordia University · Concordia University Mila - Québec AI Institute

  50. Parameter-Efficient Fine-Tuning with Learnable Rank

    Jun 3, 2026Arpit Garg, Simon Lucey, Hemanth SaratchandranParameter-Efficient Fine-Tuning MethodsLow-Rank Adaptation Framework

    Australian Institute for Machine Learning Adelaide University

  51. Where Do We (Not) Need Temporal Context in Low-Resource Video Task Adaptation?

    Jun 2, 2026Luc P. J. Sträter, Hazel DoughtyVideo Foundation ModelsParameter-Efficient Fine-Tuning Methods

    Leiden University

  52. DECA: Decentralizing Block-Wise Adam for Efficient LLM Full-Parameter Fine-Tuning on Non-IID Data

    Jun 2, 2026Yunsheng Yuan, Shaowei Li, Kai Wang +5Parameter-Efficient Fine-Tuning MethodsLarge Language Model Fine-Tuning

    School of Computer Science and Technology, Shandong University, Qingdao China · School of Mathematical Science, Peking University, China · IEIT SYSTEM, China +2

  53. On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters

    Jun 1, 2026Mind Lab, :, Vin Bo +64Parameter-Efficient Fine-Tuning MethodsFoundation Model

  54. FACT: A Simple and Efficient Framework for Active Finetuning

    Jun 1, 2026Wenshuai Xu, You Song, Yuzhuo Cui +3Parameter-Efficient Fine-Tuning MethodsModel Fine-Tuning

  55. Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks

    Jun 1, 2026Nermeen Abou Baker, David Rohrschneider, Uwe HandmannParameter-Efficient Fine-Tuning MethodsPretraining

    Ruhr West University of applied sciences · Lützowstraße 5, 46236 Bottrop, Germany: Ruhr West University of Applied Sciences

  56. TRACE: Discovering Task-Specific Parameter via Adaptation-Aware Probing for Continual Fine-Tuning

    May 29, 2026Xiaosong Han, Ke Chen, Xindi Dai +7Parameter-Efficient Fine-Tuning MethodsEfficient Task Adaptation

    Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education · College of Computer Science and Technology, Jilin University Changchun, China · College of Software, Jilin University Changchun, China +3

  57. On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference

    May 28, 2026Daniel Dold, Emanuel Sommer, Julius Kobialka +2Parameter-Efficient Fine-Tuning MethodsAnisotropic Loss Landscapes

    HTWG Konstanz · LMU Munich · Munich Center for Machine Learning (MCML)

  58. FoRA: Fisher-orthogonal Rank Adaptation for Parameter-Efficient Fine-Tuning

    May 28, 2026Juneyoung Park, Seongbae Lee, Han-Sang Lee +6Parameter-Efficient Fine-Tuning MethodsModel Fine-Tuning

    OptAI Inc · LG Uplus

  59. PEFT-Arena: Understanding Parameter-Efficient Finetuning from a Stability-Plasticity Perspective

    May 27, 2026Yangyi Huang, Ruotian Peng, Zeju Qiu +4Parameter-Efficient Fine-Tuning MethodsModel Fine-Tuning

    The Chinese University of Hong Kong · Westlake University · MPI for Intelligent Systems

  60. SIGMA: Bridging Structural and Distributional Gaps for Vision Foundation Model Adaptation

    May 27, 2026Lingyu Xiong, Jinjin Shi, Xuran Xu +3Recent Vision Foundation ModelsParameter-Efficient Fine-Tuning Methods

    Xiaomi Corporation