Low-Rank Adaptation

Also known as LoRA

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

579 papers65 papers in the last four weeks, down 19% on the four weeks before. 0.4% of all new papers.

  1. Maximizing Capacity Utilization of LoRA Adapters for Mitigating the Learning-Forgetting Trade-off

    Nov 12, 2026Tomohiro Sawada, Kartik GoyalLow-Rank AdaptationCatastrophic Forgetting

  2. SALT: State-Aligned Low-Rank Transfer for Single-Temporal Cross-Dataset Structural Localization in Remote Sensing

    Nov 12, 2026Aggelos Psiris, Georgios Angelidis, Giorgos Sfikas +4Remote SensingCross-Domain Generalization

  3. TokenStream: Multi-Subject Personalized Video Generation with Token-wise LoRA and Guided Attention

    Nov 12, 2026Hung Nguyen, Quang Qui-Vinh Nguyen, Khoi NguyenIdentity-Preserving Video GenerationSubject-Driven Video Generation

  4. Sustainable Federated Fine-Tuning with LoRA: Environmental Impact and Performance Footprint

    Nov 12, 2026Pedro Augusto Londe Ribeiro, Rodrigo Moreira, Larissa Ferreira Rodrigues Moreira +3Energy-Efficient MLFederated Fine-Tuning

  5. WaveletLoRA: Frequency-Aware Content-Style Decomposition for Personalized Image Generation

    Nov 11, 2026Peiyao Wang, Jiahui Sun, Weining Wang +1Image GenerationSubject-Driven Image Generation

  6. A Margin Perspective on LoRA: Robustness to Catastrophic Forgetting and Adapter Merging (MaLoRA)

    Nov 10, 2026Ziqing Xu, Hancheng Min, Lachlan Ewen MacDonald +4Low-Rank AdaptationModel Merging

  7. Train High, Ship Low: Efficient LoRA Synchronization for Distributed On-Policy Post-Training

    Nov 9, 2026Shenghui Li, Li Ju, Thiemo VoigtCommunication-Efficient Distributed TrainingGradient Compression

  8. PROLA: Principal-Orthogonal Low-rank Adaptation for Predictive Spatiotemporal Weather Downscaling

    Nov 8, 2026Minseo Yoon, Minseong Bae, Sojin Lee +1Low-Rank AdaptationWeather Downscaling

  9. Zeroth-Order Riemannian Optimization on Fixed-Rank Manifolds for LLM Fine-Tuning

    Nov 7, 2026Xinjue Wang, Xiuheng Wang, Yejun Zhang +2Zeroth-Order OptimizationRiemannian Optimization

  10. FedLoVA: Value-Only Aggregation for Federated LoRA Fine-Tuning of Large Language Models

    Nov 6, 2026Ensieh Khazaei, Baturalp Buyukates, Dimitrios HatzinakosLLM Fine-TuningFederated Fine-Tuning

  11. From LoRA to Language: Recovering Skill Documents from Adapters

    Nov 6, 2026Saranya Dintakurthy Swadha, Pavan C Shekar, Arpit Jain +1LLM Agent Skill LearningLow-Rank Adaptation

  12. Spectral and Subspace Dynamics of LoRA in Supervised Fine-Tuning and Reinforcement Learning

    Nov 5, 2026Chenruo Liu, Zihan Wang, Yijun Dong +1Low-Rank AdaptationRL Fine-Tuning

  13. Axis-LoRA: Probing and Adapting Embedding Models for Personal-Memory Retrieval

    Nov 5, 2026Dohoon Kim, Junho Park, Hyunwoong Bae +2Text Embedding ModelsKnowledge Retrieval in Language Models

  14. Adapters as Data: The Latent Space of LoRAs

    Nov 5, 2026Moritz Grävinghoff, Konstantin Wehmeyer, Aron Asefaw +2Low-Rank AdaptationAutoencoders

  15. Align Before Aggregation: Basis-Consistent Federated LoRA under Heterogeneous Ranks

    Nov 5, 2026Pengpeng Qiao, Yang Cao, Lingling Zhang +5Federated Fine-TuningFederated Learning Aggregation

  16. From What Is Used to What Is Missing: Rethinking Dynamic Rank in LoRA

    Nov 5, 2026Peng Xue, Guodong Fan, Guang-Yong Chen +1Low-Rank AdaptationParameter-Efficient Fine-Tuning

  17. The Behavioral Rate of Fine-Tuning: Compression as a Measure of Useful Information

    Nov 4, 2026Joey David, Paul Caillon, Alexandre AllauzenFine-TuningModel Compression

  18. Model Merging Does Not Require Uniform LoRA Ranks

    Nov 4, 2026Zirong Xie, Zainab Afolabi, Nikita Kozodoi +1Language Model MergingModel Merging

  19. SaMA: Morpho Adaptation via Asymmetric Expansion of Kronecker Product

    Nov 3, 2026An Nguyen, Anh TongLow-Rank AdaptationParameter-Efficient Fine-Tuning

  20. LIRA: A More Interpretable Dynamic Loss-Informed Rank Allocation for Low-Rank Adaptation

    Nov 3, 2026Sihao Wang, Bozheng Li, Xuemiao Xu +3Low-Rank Adaptation

  21. EpiLoRA: Fusing Many LoRA Adapters into Low-Precision GEMMs for Serving and RL

    Nov 2, 2026Hieu Tran, Hong Yumulti-lora inferenceLLM Serving

  22. BISECT: Bilevel Spectral Decomposition for Low-Rank Continual Learning

    Nov 2, 2026Tom Saliencro, Wen-Hsuan Liu, Rohan Desai +2Exemplar-Free Class-Incremental LearningContinual LoRA

  23. ASCoR: Asymmetric Semantic-Spatial Learning for Remote Sensing Domain Generalization

    Nov 2, 2026Zitong Wang, Shuchang Lyu, Meng Li +3Domain Generalization for Semantic SegmentationRemote Sensing Image Segmentation

  24. Unlocking the Potential of Vision Foundation Models for Multi-Task Dense Prediction via Mixture of Task Adapters

    Nov 2, 2026Yunzhi Zhuge, Xinzhuo Yu, Sitong Gong +3Multi-Task Dense PredictionVision Foundation Model Adaptation

  25. LoRA-Patch: Defending Against Backdoored LoRA Adapters via Directional Steering

    Nov 2, 2026Shengyan Huo, Peihai Jiang, Xixiang Lyu +1Backdoor Defense in LLMsLLM Backdoor Attacks