Low-Rank Adaptation

Also known as LoRA

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

542 papers91 papers in the last four weeks, up 82% on the four weeks before. 0.5% of all new papers.

Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization
Jun 25, 2026Xuancheng Xu, Gengyun Jia, Bing-Kun BaoDisentangled Representation Learning Video Diffusion Models
Nanjing University of Posts and Telecommunications · Hefei University of Technology · Peng Cheng Laboratory
Memory-Efficient Policy Libraries with Low-Rank Adaptation in Reinforcement Learning
Jun 24, 2026Samuel Valland Lyngset, Tor Viljen Raanaas, Gard Sveipe +4RL for Robotics Low-Rank Adaptation
Department of Informatics, University of Oslo
Priority-Aware Learning-Unlearning Correction for Dynamic Decentralized LoRA Fine-Tuning
Jun 22, 2026Nuocheng Yang, Yechen He, Sihua Wang +3Federated Unlearning Communication-Efficient Distributed Training
Beijing Laboratory of Advanced Information Network, and the Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China · Information Systems Technology and Design Pillar, Singapore University of Technology and Design, 487372, Singapore
Subspace-Constrained Federated Learning with Low-Rank Adaptation
Jun 21, 2026Neranjan Senarath, Rohit Muralitharan, Sadia AsifFederated Learning Aggregation Non-IID Federated Learning
Department of ECSE Rensselaer Polytechnic Institute Troy, NY 12180, USA · Department of Computer Science Rensselaer Polytechnic Institute Troy, NY 12180, USA
Enhancing LLMs for Graph Tasks via Graph-aware LoRA Generation
Jun 21, 2026Junshu Sun, Wanxing Chang, Qingming Huang +1Graph Representation Learning Low-Rank Adaptation
State Key Lab. of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · DAMO Academy, Alibaba Group, Hangzhou, China
Bayesian Adaptation Gym: A Benchmark for the Bayesian Low-Rank Adaptation of Multi-Modal Language Models
Jun 20, 2026Colin Samplawski, Ramneet Kaur, Manoj Acharya +2Distribution Shift Robustness Multimodal Large Language Models
Neuro-Symbolic Computing and Intelligence Research Group, Computer Science Laboratory, SRI International
Predicting Mergeability of Parameter-Efficient Fine-Tuning Updates
Jun 17, 2026Lin Tang, Wei Zhang, Jing Li +3Low-Rank Adaptation Language Model Merging
1Sichuan University, Chengdu, China · University of Electronic Science and Technology of China, Chengdu, China
Techniques for Peak Memory Reduction for LoRA Fine-tuning of LLMs on Edge Devices
Jun 17, 2026Hassan Dbouk, Matthias Reisser, Prathamesh Mandke +2Memory-Efficient Fine-Tuning LLM Fine-Tuning
Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc. · work done while at Qualcomm
Skill-to-LoRA: From Using Skills to Learning Behaviors for Token-Efficient LLM Agents
Jun 15, 2026Tianyi Zhang, Zhonghao QiAI Coding Agents LLM Agent Skill Learning
1The Chinese University of Hong Kong
SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation
Jun 15, 2026Junghun Oh, Sungyong Baik, Kyoung Mu LeeLow-Rank Adaptation Parameter-Efficient Fine-Tuning
Dept. of ECE, ASRI, Seoul National University · Dept. of Artificial Intelligence, Dept. of Data Science, Hanyang University · Dept. of ECE, ASRI, IPAI, Seoul National University
Variable-Rate Deep Image Compression based on Low-Rank Adaptation by Progressive Learning
Jun 15, 2026Xing-Yu Xu, Chen-Hsiu Huang, Ja-Ling WuLearned Image Compression Low-Rank Adaptation
Communications and Multimedia Lab National Taiwan University Taipei, Taiwan
PreLort: Prefix-Nested LoRA for Federated Fine-Tuning under Rank Heterogeneity
Jun 14, 2026Muhammad Waseem, Nurbek Tastan, Andrej Jovanovic +4Federated Learning Aggregation LLM Fine-Tuning
1MBZUAI, UAE · University of Cambridge, UK · 3Flower Labs, UK +1
Prompt2Effect: Training-Free Image-to-Video Model Specialization via LoRA Generation
Jun 11, 2026Xiaomeng Yang, Yanyu Li, Gordon Guocheng Qian +7Image-to-Video Generation Video Diffusion Models
1Northeastern University, 2Snap Inc.
Compressing Image Style Training into a Single Model Forward
Jun 11, 2026Zhongjie Duan, Yingda ChenImage Style Transfer Low-Rank Adaptation
ModelScope Team, Alibaba Group
Beyond LoRA: Is Sparsity-Induced Adaptation Better?
Jun 11, 2026Elijah Cadenhead, Cristian McGee, Xin Li +2Low-Rank Adaptation Parameter-Efficient Fine-Tuning
School of Data, Mathematical and Statistical Sciences, University of Central Florida, United States · Department of Computer Science, University of Central Florida, United States · College of Computing, Mohammed VI Polytechnic University (UM6P), Morocco
LoRA-Muon: Spectral Steepest Descent on the Low-Rank Manifold
Jun 11, 2026Franz Louis Cesista, Katherine Crowson, Cédric Simal +1Deep Learning Optimization Riemannian Optimization
Ateneo de Manila University · EleutherAI · NaXys +1
The Hidden Power of Scaling Factor in LoRA Optimization
Jun 11, 2026Zicheng Zhang, Haoran Li, Jiaxing Wang +10Low-Rank Adaptation Hyperparameter Optimization
JD · School of Mathematical Sciences, UCAS · School of Mathematical Sciences, NKU +1
Doc-to-Atom: Learning to Compile and Compose Memory Atoms
Jun 10, 2026Xingjian Diao, Wenbo Li, Yashas Malur Saidutta +3Memory-Augmented Language Models Long-Context Language Model Inference
AI Center-Mountain View, Samsung Electronics · Dartmouth College
Null-Space Constrained Low-Rank Adaptation for Response-Specified Large Language Model Unlearning
Jun 9, 2026Bocheng Ju, Jianhua Wang, Chengliang Liu +1Low-Rank Adaptation LLM Unlearning
Beijing Key Laboratory of Security and Privacy in Intelligent Transportation, Beijing Jiaotong University, P.R.China · College of Computer Science and Technology, Taiyuan University of Technology, Taiyuan, China, 030024 · Institute of Computing Technologies, China Academy of Railway Sciences Corporation Limited, Beijing 100081, China
Recoverable but Not Stationary:Local Linear Structures in Weights and Activations
Jun 9, 2026Irina Piontkovskaia, Sergey NikolenkoLow-Rank Adaptation Activation Steering
St. Petersburg Department of the Steklov Institute of Mathematics, St. Petersburg, Russia · St. Petersburg State University
SSR-Merge: Subspace Signal Routing for Training-Free LoRA Merging in Diffusion Models
Jun 9, 2026Zhengxuan Wei, Yi Dong, Zonghui Li +6Low-Rank Adaptation Diffusion Models
School of Intelligence Science and Technology, Nanjing University, China · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd., China · ShanghaiTech University +2
Which LoRA? An Empirical Study on the Effectiveness of LoRA Techniques During Multilingual Instruction Tuning
Jun 9, 2026Thamali Wijewardhana, Napoleon H. Reyes, Surangika RanathungaLow-Rank Adaptation Cross-Lingual Transfer
School of Mathematical and Computational Sciences Massey University, Auckland, New Zealand
PermDoRA -- Understanding Adapter Interference in Language Models: Limits of Parameter-Space Geometry
Jun 9, 2026Gowtham Sivaramakrishnan, Sarvesha Kumar Kombaiah Seetha, Kishan Gupta Balaji +1Low-Rank Adaptation LLM Security
Independent Researcher
Shift-Dependent Asymmetry: Orthogonal Inverse Low-Rank Adaptation for Federated Medical Segmentation
Jun 7, 2026Xingyue Zhao, Wenke Huang, Linghao Zhuang +7Vision Foundation Model Adaptation Non-IID Federated Learning
Department of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences · College of Computing and Data Science, Nanyang Technological University, Singapore +4
TALAN: Task-Aligned Latent Adaptation Networks for Targeted Post-Training of Large Language Models
Jun 5, 2026Chengkai Zhang, Ziteng Liu, Junpu Wang +4Supervised Fine-Tuning Adapter Tuning
Meta AI
TailLoR: Protecting Principal Components in Parameter-Efficient Continual Learning
Jun 4, 2026Marius Dragoi, Ioana Pintilie, Alexandra Dragomir +2Continual Learning Low-Rank Adaptation
Bitdefender, Romania
Amortizing Federated Adaptation: Hypernetwork Driven LoRA for Personalized Foundation Models
Jun 4, 2026Sunny Gupta, Shambhavi Shanker, Amit SethiFederated Learning Aggregation Non-IID Federated Learning
Indian Institute of Technology, Bombay
High-Dimensional Theory of LoRA Fine-Tuning in a Solvable Attention Model
Jun 4, 2026O. Duranthon, F. Boncoraglio, L. ZdeborováFine-Tuning Low-Rank Adaptation
Statistical Physics of Computation Laboratory, École Polytechnique Fédérale de Lausanne (EPFL) CH-1015 Lausanne
TaDA: Calibrated Probe Gating for Task-Domain LoRA Merging
Jun 3, 2026Huy Quoc To, Fuyi Li, Guangyan Huang +1Adapter Tuning Low-Rank Adaptation
Deakin University, Melbourne, VIC, Australia · Adelaide University, Adelaide, SA, Australia
Crafting Your Evolving Dreams: Concept-Incremental Versatile Customization
Jun 3, 2026Jiahua Dong, Wenqi Liang, Hongliu Li +7T2I Generation Low-Rank Adaptation
Shanghai Jiao Tong University, Shanghai, China

Jun 25, 2026J/K move · Enter open · S save

Disco-LoRA: Disentangled Composition of Content, Style, and Motion for Multi-concept Video Customization

Xuancheng Xu, Gengyun Jia, Bing-Kun Bao

Nanjing University of Posts and Telecommunications · Hefei University of Technology · Peng Cheng Laboratory

Disentangled Representation Learning Video Diffusion Models Low-Rank Adaptation Controllable Video Generation

Open paper AskPDF arXiv

Video customization based on Text-to-Video (T2V) models aims to learn specific features from reference data to generate controllable videos. While significant strides have been made in image stylization and video motion customization, simultaneously controlling multiple concepts, such as content, style, and motion, remains a major challenge. In this work, we systematically define the task of multi-concept video customization, which requires the joint control of content, style, and motion. To facilitate research in this area, we construct a comprehensive benchmark and propose Disco-LoRA, a unified framework designed to tackle this problem by disentangling and flexibly recombining different concepts in two stages: (1) We decompose the objective into two sub-tasks: Content-Style and Content-Motion. Each sub-task is addressed using our Iterative Dual-LoRA Disentanglement Framework, which effectively disentangles distinct concepts within the data. (2) We identify layer-wise weight trends as crucial for LoRA identity, while weight magnitudes dictate composability. To harmonize these scales, we propose a Z-score-based statistical regularization that aligns weight distributions, preserving layer-wise trends while minimizing interference between different LoRAs. Extensive experiments show that Disco-LoRA excels in multi-concept video customization, effectively preserving appearance, style, and motion for controllable text-to-video generation.