Low-Rank Adaptation

Also known as LoRA

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

538 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

Stable-LoRA: Stabilizing Feature Learning of Low-Rank Adaptation
Mar 5, 2026Yize Wu, Ke Gao, Ling Li +1Low-Rank Adaptation LLM Fine-Tuning
Intelligent Software Research Center, Institute of Software, CAS, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
Crab $^{+}$ : A Scalable and Unified Audio-Visual Scene Understanding Model with Explicit Cooperation
Mar 4, 2026Dongnuan Cai, Henghui Du, Chang Zhou +5Audio-Visual Understanding Unified Multimodal Models
Gaoling School of Artificial Intelligence, Renmin University of China · Institute of Artificial Intelligence of China Telecom (TeleAI) · AI Technology Center, Online Video Business Unit, Tencent PCG +2
Spectral Surgery: Training-Free Refinement of LoRA via Gradient-Guided Singular Value Reweighting
Mar 4, 2026Zailong Tian, Yanzhe Chen, Zhuoheng Han +1Low-Rank Adaptation Adapter Tuning
School of Computing and Information Systems, Singapore Management University · School of Computing, National University of Singapore · State Key Laboratory for Multimedia Information Processing, Peking University
BLOCK: An Open-Source Bi-Stage MLLM Character-to-Skin Pipeline for Minecraft
Mar 4, 2026Hengquan GuoConditional Image Generation Texture Synthesis
ShanghaiTech University
HiLoRA: Hierarchical Low-Rank Adaptation for Personalized Federated Learning
Mar 3, 2026Zihao Peng, Nan Zou, Jiandian Zeng +4Federated Learning Federated Fine-Tuning
Beijing Normal University · Zhengzhou University
GLoRIA: Gated Low-Rank Interpretable Adaptation for Dialectal ASR
Mar 2, 2026Pouya Mehralian, Melissa Farasyn, Anne Breitbarth +2Automatic Speech Recognition Domain Adaptation for ASR
ESAT/PSI, KU Leuven, Belgium · ∆iaLing, Ghent University, Belgium · GLiMS & MULTPIPLES, Ghent University, Belgium
ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs
Mar 2, 2026Xunlei Chen, Jinyu Guo, Yuang Li +5Machine Unlearning LLM Unlearning
School of Information and Software Engineering, University of Electronic Science and Technology of China · School of Computer Science and Engineering, University of Electronic Science and Technology of China
A Diffusion-Driven Fine-Grained Nodule Synthesis Framework for Enhanced Lung Nodule Detection from Chest Radiographs
Mar 2, 2026Aryan Goyal, Shreshtha Singh, Ashish Mittal +3Lung Nodule Detection Synthetic Data Augmentation
Qure.ai, India · Indian Institute of Technology, Bombay
DeLo: Dual Decomposed Low-Rank Experts Collaboration for Continual Missing Modality Learning
Mar 2, 2026Xiwei Liu, Yulong Li, Feilong Tang +1Missing-Modality Learning Multimodal Large Language Models
Mohamed bin Zayed University of Artificial Intelligence · Monash University
Scalable Multi-Task Low-Rank Model Adaptation
Mar 2, 2026Zichen Tian, Antoine Ledent, Qianru SunLow-Rank Adaptation Multi-Task Learning
Singapore Management University
Quantum-Inspired Fine-Tuning for Few-Shot AIGC Detection via Phase-Structured Reparameterization
Mar 2, 2026Kaiyang Xing, Han Fang, Zhaoyun Chen +4Parameter-Efficient Fine-Tuning Quantum-Inspired ML
Anhui University, Hefei, China · National University of Singapore, Singapore, Singapore · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China +1
HypeLoRA: Hyper-Network-Generated LoRA Adapters for Calibrated Language Model Fine-Tuning
Mar 1, 2026Bartosz Trojan, Filip GębalaLow-Rank Adaptation Language Model Calibration
Upper-Secondary Schools of Communications in Cracow
Understanding LoRA as Knowledge Memory: An Empirical Analysis
Mar 1, 2026Seungju Back, Dongwoo Lee, Naun Kang +4Continual Learning for LLMs Memory-Augmented Language Models
KAIST · Samsung SDS · New York University
MARS: Harmonizing Multimodal Convergence via Adaptive Rank Search
Feb 28, 2026Minkyoung Cho, Insu Jang, Shuowei Jin +5Multimodal Large Language Models Multimodal Optimization
University of Michigan · NVIDIA
Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis
Feb 28, 2026Youngjin Yoo, Han Liu, Bogdan Georgescu +14Medical Imaging Foundation Models Low-Rank Adaptation
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ USA · Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany · Department of Radiology, University of Maryland Medical Center, Baltimore, MD USA +4
CoMoL: Efficient Mixture of LoRA Experts via Dynamic Core Space Merging
Feb 28, 2026Jie Cao, Zhenxuan Fan, Zhuonan Wang +8Mixture-of-Experts Models Parameter-Efficient Fine-Tuning
Zhejiang University · Wechat, Tencent
FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
Feb 27, 2026Haoran Zhang, Dongjun Kim, Seohyeon Cha +1Federated Fine-Tuning Federated Learning
Chandra Department of Electrical and Computer Engineering, The University of Texas at Austin, TX, USA.
Task-Driven Subspace Decomposition for Knowledge Sharing and Isolation in LoRA-based Continual Learning
Feb 27, 2026Lingfeng He, De Cheng, Huaijie Wang +3Continual Learning Low-Rank Adaptation
State Key Laboratory of Integrated Services Networks, School of Telecommunications Engineering, Xidian University, Xi’an, China · School of Electronic Engineering, Xidian University, Xi’an, China · AUTHORERR: Missing \icmlaffiliation.
CeRA: Breaking the Linear Ceiling of Low-Rank Adaptation with Non-linearity Retained at Inference
Feb 26, 2026Hung-Hsuan ChenNonlinear LoRA Adapter Tuning
Computer Science and Information Engineering National Central University Taoyuan, Taiwan
When LoRA Betrays: Backdooring Text-to-Image Models by Masquerading as Benign Adapters
Feb 25, 2026Liangwei Lyu, Jiaqi Xu, Jianwei Ding +1T2I Diffusion Models Low-Rank Adaptation
People’s Public Security University of China
Mechanistically Guided LoRA Improves Paraphrase Consistency in Medical Vision-Language Models
Feb 24, 2026Binesh Sadanandan, Vahid BehzadanMedical VLMs Medical VQA
SAIL Lab, University of New Haven, CT, USA
OmniOCR: Generalist OCR for Ethnic Minority Languages
Feb 24, 2026Bonan Liu, Zeyu Zhang, Bingbing Meng +5Optical Character Recognition Low-Resource Language Processing
Southwest Minzu University · AI Geeks
ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decomposition
Feb 24, 2026Xindian Ma, Rundong Kong, Peng Zhang +2LLM Fine-Tuning Low-Rank Adaptation
Tianjin University, Tianjin, China
Rethinking LoRA for Privacy-Preserving Federated Learning in Large Models
Feb 23, 2026Jin Liu, Yinbin Miao, Ning Xi +1Federated Fine-Tuning Low-Rank Adaptation
School of Cyber Engineering, Xidian University · College of Intelligence and Computing, Tianjin University
ChimeraLoRA: Multi-Head LoRA-Guided Synthetic Datasets
Feb 23, 2026Hoyoung Kim, Minwoo Jang, Jabin Koo +2Synthetic Data Augmentation Diffusion Model Fine-Tuning
Graduate School of AI, POSTECH · Dept. of CSE, POSTECH · NAVER AI Lab
Astra: Activation-Space Tail-Eigenvector Low-Rank Adaptation of Large Language Models
Feb 22, 2026Kainan Liu, Yong Zhang, Ning Cheng +4LLM Fine-Tuning Low-Rank Adaptation
Ping An Technology (Shenzhen) Co., Ltd., China
CRAFT-LoRA: Content-Style Personalization via Rank-Constrained Adaptation and Training-Free Fusion
Feb 21, 2026Yu Li, Yujun Cai, Chi ZhangImage Generation T2I Diffusion Models
AGI Lab, Westlake University · George Washington University · The University of Queensland
Deep LoRA-Unfolding Networks for Image Restoration
Feb 21, 2026Xiangming Wang, Haijin Zeng, Benteng Sun +4Deep Unfolding Image Restoration
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen 518055, Guangdong, China · Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai, China · School of Intelligence Science and Technology, Nanjing University, Suzhou, China +1
Parameter-Efficient Domain Adaptation of Physics-Informed Self-Attention based GNNs for AC Power Flow Prediction
Feb 20, 2026Redwanul Karim, Changhun Kim, Timon Conrad +7Physics-Informed GNNs Power Systems
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany · Institute of Electrical Energy Systems, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany
Revisiting Weight Regularization for Low-Rank Continual Learning
Feb 19, 2026Yaoyue Zheng, Yin Zhang, Joost van de Weijer +4Incremental Learning Elastic Weight Consolidation
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Shaanxi, China · Computer Vision Center, Barcelona, Spain · Universitat Aut`onoma de Barcelona, Barcelona, Spain +3

Mar 5, 2026J/K move · Enter open · S save

Stable-LoRA: Stabilizing Feature Learning of Low-Rank Adaptation

Yize Wu, Ke Gao, Ling Li, Yanjun Wu

Intelligent Software Research Center, Institute of Software, CAS, Beijing, China · University of Chinese Academy of Sciences, Beijing, China

Low-Rank Adaptation LLM Fine-Tuning Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Low-Rank Adaptation (LoRA) is a widely adopted parameter-efficient method for fine-tuning Large Langauge Models. It updates the weight matrix as

W=W_0+sBA

, where

W_0

is the original frozen weight,

s

is a scaling factor and

A

B

are trainable low-rank matrices. Despite its robust empirical effectiveness, the theoretical foundations of LoRA remain insufficiently understood, particularly with respect to feature learning stability. In this paper, we first establish that, LoRA can, in principle, naturally achieve and sustain stable feature learning (i.e., be self-stabilized) under appropriate hyper-parameters and initializations of

A

and

B

. However, we also uncover a fundamental limitation that the necessary non-zero initialization of

A

compromises self-stability, leading to suboptimal performances. To address this challenge, we propose Stable-LoRA, a weight-shrinkage optimization strategy that dynamically enhances stability of LoRA feature learning. By progressively shrinking

A

during the earliest training steps, Stable-LoRA is both theoretically and empirically validated to effectively eliminate instability of LoRA feature learning while preserving the benefits of the non-zero start. Experiments show that Stable-LoRA consistently outperforms other baselines across diverse models and tasks, with no additional memory usage and only negligible computation overheads. The code is available at https://github.com/Yize-Wu/Stable-LoRA.