Low-Rank Adaptation

Also known as LoRA

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

542 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

Towards Lightweight Adaptation of Speech Enhancement Models in Real-World Environments
Mar 8, 2026Longbiao Cheng, Shih-Chii LiuSpeech Enhancement Memory-Efficient Fine-Tuning
Institute of Neuroinformatics, University of Zurich and ETH Zurich
Adaptive Capacity Allocation for Vision Language Action Fine-tuning
Mar 8, 2026Donghoon Kim, Minji Bae, Unghui Nam +4VLA Model Training Nonlinear LoRA
Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, Republic of Korea · Department of Computer Science and Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea
Domain-Specific Quality Estimation for Machine Translation in Low-Resource Scenarios
Mar 7, 2026Namrata Patil Gurav, Akashdeep Ranu, Archchana Sindhujan +1MT Evaluation Machine Translation
Institute for People-Centred AI, University of Surrey, United Kingdom
Lifelong Embodied Navigation Learning
Mar 6, 2026Xudong Wang, Jiahua Dong, Baichen Liu +3Embodied Navigation Continual Learning for LLM Agents
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Mohamed bin Zayed University of Artificial Intelligence
Stable-LoRA: Stabilizing Feature Learning of Low-Rank Adaptation
Mar 5, 2026Yize Wu, Ke Gao, Ling Li +1Low-Rank Adaptation LLM Fine-Tuning
Intelligent Software Research Center, Institute of Software, CAS, Beijing, China · University of Chinese Academy of Sciences, Beijing, China
Crab $^{+}$ : A Scalable and Unified Audio-Visual Scene Understanding Model with Explicit Cooperation
Mar 4, 2026Dongnuan Cai, Henghui Du, Chang Zhou +5Audio-Visual Understanding Unified Multimodal Models
Gaoling School of Artificial Intelligence, Renmin University of China · Institute of Artificial Intelligence of China Telecom (TeleAI) · AI Technology Center, Online Video Business Unit, Tencent PCG +2
Spectral Surgery: Training-Free Refinement of LoRA via Gradient-Guided Singular Value Reweighting
Mar 4, 2026Zailong Tian, Yanzhe Chen, Zhuoheng Han +1Low-Rank Adaptation Adapter Tuning
School of Computing and Information Systems, Singapore Management University · School of Computing, National University of Singapore · State Key Laboratory for Multimedia Information Processing, Peking University
BLOCK: An Open-Source Bi-Stage MLLM Character-to-Skin Pipeline for Minecraft
Mar 4, 2026Hengquan GuoConditional Image Generation Texture Synthesis
ShanghaiTech University
HiLoRA: Hierarchical Low-Rank Adaptation for Personalized Federated Learning
Mar 3, 2026Zihao Peng, Nan Zou, Jiandian Zeng +4Federated Learning Federated Fine-Tuning
Beijing Normal University · Zhengzhou University
GLoRIA: Gated Low-Rank Interpretable Adaptation for Dialectal ASR
Mar 2, 2026Pouya Mehralian, Melissa Farasyn, Anne Breitbarth +2Automatic Speech Recognition Domain Adaptation for ASR
ESAT/PSI, KU Leuven, Belgium · ∆iaLing, Ghent University, Belgium · GLiMS & MULTPIPLES, Ghent University, Belgium
ALTER: Asymmetric LoRA for Token-Entropy-Guided Unlearning of LLMs
Mar 2, 2026Xunlei Chen, Jinyu Guo, Yuang Li +5Machine Unlearning LLM Unlearning
School of Information and Software Engineering, University of Electronic Science and Technology of China · School of Computer Science and Engineering, University of Electronic Science and Technology of China
A Diffusion-Driven Fine-Grained Nodule Synthesis Framework for Enhanced Lung Nodule Detection from Chest Radiographs
Mar 2, 2026Aryan Goyal, Shreshtha Singh, Ashish Mittal +3Lung Nodule Detection Synthetic Data Augmentation
Qure.ai, India · Indian Institute of Technology, Bombay
DeLo: Dual Decomposed Low-Rank Experts Collaboration for Continual Missing Modality Learning
Mar 2, 2026Xiwei Liu, Yulong Li, Feilong Tang +1Missing-Modality Learning Multimodal Large Language Models
Mohamed bin Zayed University of Artificial Intelligence · Monash University
Scalable Multi-Task Low-Rank Model Adaptation
Mar 2, 2026Zichen Tian, Antoine Ledent, Qianru SunLow-Rank Adaptation Multi-Task Learning
Singapore Management University
Quantum-Inspired Fine-Tuning for Few-Shot AIGC Detection via Phase-Structured Reparameterization
Mar 2, 2026Kaiyang Xing, Han Fang, Zhaoyun Chen +4Parameter-Efficient Fine-Tuning Quantum-Inspired ML
Anhui University, Hefei, China · National University of Singapore, Singapore, Singapore · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China +1
HypeLoRA: Hyper-Network-Generated LoRA Adapters for Calibrated Language Model Fine-Tuning
Mar 1, 2026Bartosz Trojan, Filip GębalaLow-Rank Adaptation Language Model Calibration
Upper-Secondary Schools of Communications in Cracow
Understanding LoRA as Knowledge Memory: An Empirical Analysis
Mar 1, 2026Seungju Back, Dongwoo Lee, Naun Kang +4Continual Learning for LLMs Memory-Augmented Language Models
KAIST · Samsung SDS · New York University
MARS: Harmonizing Multimodal Convergence via Adaptive Rank Search
Feb 28, 2026Minkyoung Cho, Insu Jang, Shuowei Jin +5Multimodal Large Language Models Multimodal Optimization
University of Michigan · NVIDIA
Specializing Foundation Models via Mixture of Low-Rank Experts for Comprehensive Head CT Analysis
Feb 28, 2026Youngjin Yoo, Han Liu, Bogdan Georgescu +14Medical Imaging Foundation Models Low-Rank Adaptation
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ USA · Department of Computed Tomography, Siemens Healthineers, Forchheim, Germany · Department of Radiology, University of Maryland Medical Center, Baltimore, MD USA +4
CoMoL: Efficient Mixture of LoRA Experts via Dynamic Core Space Merging
Feb 28, 2026Jie Cao, Zhenxuan Fan, Zhuonan Wang +8Mixture-of-Experts Models Parameter-Efficient Fine-Tuning
Zhejiang University · Wechat, Tencent
FedRot-LoRA: Mitigating Rotational Misalignment in Federated LoRA
Feb 27, 2026Haoran Zhang, Dongjun Kim, Seohyeon Cha +1Federated Fine-Tuning Federated Learning
Chandra Department of Electrical and Computer Engineering, The University of Texas at Austin, TX, USA.
Task-Driven Subspace Decomposition for Knowledge Sharing and Isolation in LoRA-based Continual Learning
Feb 27, 2026Lingfeng He, De Cheng, Huaijie Wang +3Continual Learning Low-Rank Adaptation
State Key Laboratory of Integrated Services Networks, School of Telecommunications Engineering, Xidian University, Xi’an, China · School of Electronic Engineering, Xidian University, Xi’an, China · AUTHORERR: Missing \icmlaffiliation.
CeRA: Breaking the Linear Ceiling of Low-Rank Adaptation with Non-linearity Retained at Inference
Feb 26, 2026Hung-Hsuan ChenNonlinear LoRA Adapter Tuning
Computer Science and Information Engineering National Central University Taoyuan, Taiwan
When LoRA Betrays: Backdooring Text-to-Image Models by Masquerading as Benign Adapters
Feb 25, 2026Liangwei Lyu, Jiaqi Xu, Jianwei Ding +1T2I Diffusion Models Low-Rank Adaptation
People’s Public Security University of China
Mechanistically Guided LoRA Improves Paraphrase Consistency in Medical Vision-Language Models
Feb 24, 2026Binesh Sadanandan, Vahid BehzadanMedical VLMs Medical VQA
SAIL Lab, University of New Haven, CT, USA
OmniOCR: Generalist OCR for Ethnic Minority Languages
Feb 24, 2026Bonan Liu, Zeyu Zhang, Bingbing Meng +5Optical Character Recognition Low-Resource Language Processing
Southwest Minzu University · AI Geeks
ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decomposition
Feb 24, 2026Xindian Ma, Rundong Kong, Peng Zhang +2LLM Fine-Tuning Low-Rank Adaptation
Tianjin University, Tianjin, China
Rethinking LoRA for Privacy-Preserving Federated Learning in Large Models
Feb 23, 2026Jin Liu, Yinbin Miao, Ning Xi +1Federated Fine-Tuning Low-Rank Adaptation
School of Cyber Engineering, Xidian University · College of Intelligence and Computing, Tianjin University
ChimeraLoRA: Multi-Head LoRA-Guided Synthetic Datasets
Feb 23, 2026Hoyoung Kim, Minwoo Jang, Jabin Koo +2Synthetic Data Augmentation Diffusion Model Fine-Tuning
Graduate School of AI, POSTECH · Dept. of CSE, POSTECH · NAVER AI Lab
Astra: Activation-Space Tail-Eigenvector Low-Rank Adaptation of Large Language Models
Feb 22, 2026Kainan Liu, Yong Zhang, Ning Cheng +4LLM Fine-Tuning Low-Rank Adaptation
Ping An Technology (Shenzhen) Co., Ltd., China

Mar 8, 2026J/K move · Enter open · S save

Towards Lightweight Adaptation of Speech Enhancement Models in Real-World Environments

Longbiao Cheng, Shih-Chii Liu

Institute of Neuroinformatics, University of Zurich and ETH Zurich

Speech Enhancement Memory-Efficient Fine-Tuning Low-Rank Adaptation

Open paper AskPDF arXiv

Recent studies have shown that post-deployment adaptation can improve the robustness of speech enhancement models in unseen noise conditions. However, existing methods often incur prohibitive computational and memory costs, limiting their suitability for on-device deployment. In this work, we investigate model adaptation in realistic settings with dynamic acoustic scene changes and propose a lightweight framework that augments a frozen backbone with low-rank adapters updated via self-supervised training. Experiments on sequential scene evaluations spanning 111 environments across 37 noise types and three signal-to-noise ratio ranges, including the challenging [-8, 0] dB range, show that our method updates fewer than 1% of the base model's parameters while achieving an average 1.51 dB SI-SDR improvement within only 20 updates per scene. Compared to state-of-the-art approaches, our framework achieves competitive or superior perceptual quality with smoother and more stable convergence, demonstrating its practicality for lightweight on-device adaptation of speech enhancement models under real-world acoustic conditions.