Low-Rank Adaptation

Also known as LoRA

Ask about this topic

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

538 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

  1. Frequency Switching Mechanism for Parameter-E!cient Multi-Task Learning

    Mar 22, 2026Shih-Wen Liu, Yen-Chang Chen, Wei-Ta Chu +2Multi-Task LearningParameter-Efficient Fine-Tuning

    National Cheng Kung University, Tainan, Taiwan · NVIDIA Research, Taiwan

  2. FAAR: Efficient Frequency-Aware Multi-Task Fine-Tuning via Automatic Rank Selection

    Mar 20, 2026Maxime Fontana, Michael Spratling, Miaojing ShiNonlinear LoRAParameter-Efficient Fine-Tuning

    King’s College London London, UK · University of Luxembourg Esch-sur-Alzette, Luxembourg · Tongji University Shanghai, China

  3. Exploring parameter-efficient fine-tuning (PEFT) of billion-parameter vision models with QLoRA and DoRA: insights into generalization for limited-data image classification under a 98:1 test-to-train regime

    Mar 18, 2026Haiyu Yang, Sumit Sharma, Enhong Liu +1Precision Livestock FarmingParameter-Efficient Fine-Tuning

    Cornell University, College of Agriculture and Life Sciences, Ithaca, NY 14853

  4. Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition

    Mar 18, 2026Yuxiang Mei, Delai Qiu, Shengping Liu +2Multilingual ASRSpeech Language Models

    Shanghai Engineering Research Center of Intelligent Education and Bigdata, Shanghai Normal University, Shanghai, 200234, China. · Unisound AI Technology Co., Ltd., Beijing, China. · SHNU-Unisound Natural Human-Computer Interaction Lab, Shanghai Normal University.

  5. Early Intervention for VFM-based Multimodal Medical Image Classification

    Mar 18, 2026Qijie Wei, Hailan Lin, Xirong LiCross-Modal Representation LearningVision Foundation Model Adaptation

    Renmin University of China, Beijing 100872, China

  6. EvoTale: Continual Character Customization for Expanding Story Worlds

    Mar 17, 2026Jinlu Zhang, Qiyun Wang, Baoxiang Du +6Image GenerationSubject-Driven Image Generation

    Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R., China. · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 510000, P.R., China. · Fuxi AI Lab, NetEase Inc., Hangzhou, 310000, P.R., China.

  7. W2T: LoRA Weights Already Know What They Can Do

    Mar 16, 2026Xiaolong Han, Ferrante Neri, Zijian Jiang +4Weight-Space LearningLow-Rank Adaptation

    University of Surrey · Stanford University · University of Notre Dame

  8. MoLoRA: Composable Specialization via Per-Token Adapter Routing

    Mar 16, 2026Shrey Shah, Justin WagleLLM RoutingLow-Rank Adaptation

  9. AdapterTune: Zero-Initialized Low-Rank Adapters for Frozen Vision Transformers

    Mar 16, 2026Salim KhazemAdapter TuningLow-Rank Adaptation

    Talan Research Center, Paris, France

  10. Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation

    Mar 15, 2026Xi Xiao, Chenrui Ma, Yunbei Zhang +7Low-Rank AdaptationParameter-Efficient Fine-Tuning

    University of Alabama at Birmingham · University of Virginia · Tulane University +5

  11. IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoring

    Mar 14, 2026Xuan Cui, Huiyue Li, Run Zeng +5LLM Fine-TuningLow-Rank Adaptation

    Chongqing Technology and Business University · Shanghai Jiao Tong University

  12. Expert Pyramid Tuning: Efficient Parameter Fine-Tuning for Expertise-Driven Task Allocation

    Mar 13, 2026Jia-Chen Zhang, Zhen-Wei Yan, Yu-Jie Xiong +1Parameter-Efficient Fine-TuningLow-Rank Adaptation

    School of Electronic and Electrical Engineering, Shanghai University of Engineering Science 333 Longteng Road, Shanghai, China

  13. NeuroLoRA: Context-Aware Neuromodulation for Parameter-Efficient Multi-Task Adaptation

    Mar 12, 2026Yuxin Yang, Haoran Zhang, Mingxuan Li +6Parameter-Efficient Fine-TuningLow-Rank Adaptation

    Shanghai University · Fudan University

  14. AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization

    Mar 12, 2026Qiyang Li, Rui Kong, Yuchen Li +5LLM Inference AccelerationLow-Rank Adaptation

    Baidu Inc. · Shanghai Jiao Tong University

  15. Reversible Lifelong Model Editing via Semantic Routing-Based LoRA

    Mar 11, 2026Haihua Luo, Xuming Ran, Tommi Kärkkäinen +4Model EditingLanguage Model Editing

    Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland · National University of Singapore, Singapore · School of Computer Science and Technology, Dalian University of Technology, Dalian, China

  16. Med-DualLoRA: Local Adaptation of Foundation Models for 3D Cardiac MRI

    Mar 11, 2026Joan Perramon-Llussà, Amelia Jiménez-Sánchez, Grzegorz Skorupko +4Federated Fine-TuningMedical Imaging Foundation Models

    Artificial Intelligence in Medicine Lab (BCN-AIM), Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Spain · Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona, Spain

  17. Bilevel Layer-Positioning LoRA for Real Image Dehazing

    Mar 11, 2026Yan Zhang, Long Ma, Yuxin Feng +3Image DehazingLow-Rank Adaptation

    School of Computer Science and Engineering, Sun Yat-sen University · Dalian University of Technology · Xidian University +1

  18. ID-LoRA: Identity-Driven Audio-Video Personalization with In-Context LoRA

    Mar 10, 2026Aviad Dahan, Moran Yanuka, Noa Kraicer +2Audio-Video GenerationIdentity-Preserving Video Generation

    Tel Aviv University

  19. ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning

    Mar 10, 2026Ruizhong Qiu, Hanqing Zeng, Yinglong Xia +15Parameter-Efficient Fine-TuningLow-Rank Adaptation

    University of Illinois Urbana-Champaign, IL, USA · Meta AI Menlo Park, CA, USA · Washington University St. Louis, WA, USA +1

  20. TemporalDoRA: Temporal PEFT for Robust Surgical Video Question Answering

    Mar 10, 2026Luca Carlini, Chiara Lena, Cesare Hassan +4Video QAParameter-Efficient Fine-Tuning

    Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB), Politecnico di Milano, Italy · IRCCS Humanitas Research Hospital, Italy · UCL Hawkes Institute and Department of Computer Science, University College London, UK +1

  21. On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning

    Mar 10, 2026Muhammad Ahmad, Jingjing Zheng, Yankai CaoParameter-Efficient Fine-TuningCatastrophic Forgetting

    Department of Electrical and Computer Engineering, The University of British Columbia · Department of Mathematics, The University of British Columbia · Department of Chemical and Biological Engineering, The University of British Columbia

  22. ForgeDreamer: Industrial Text-to-3D Generation with Multi-Expert LoRA and Cross-View Hypergraph

    Mar 10, 2026Junhao Cai, Deyu Zeng, Junhao Pang +3Text-to-3D GenerationLow-Rank Adaptation

    Shenzhen University, Shenzhen, Guangdong 518060, China · Guangzhou Maritime University, Guangzhou, Guangdong 510725, China

  23. Alfa: Attentive Low-Rank Filter Adaptation for Structure-Aware Cross-Domain Personalized Gaze Estimation

    Mar 9, 2026He-Yen Hsieh, Wei-Te Mark Ting, H. T. KungGaze EstimationLow-Rank Adaptation

    Harvard University

  24. Video2LoRA: Unified Semantic-Controlled Video Generation via Per-Reference-Video LoRA

    Mar 9, 2026Zexi Wu, Baolu Li, Jing Dai +5Controllable Video GenerationVideo-Reference Video Generation

    Dalian University of Technology · HKUST

  25. Stabilized Fine-Tuning with LoRA in Federated Learning: Mitigating the Side Effect of Client Size and Rank via the Scaling Factor

    Mar 9, 2026Jiayu Huang, Xiaohu Wu, Tiantian He +1Federated Fine-TuningLow-Rank Adaptation

    Beijing University of Posts and Telecommunications, Beijing, China · Agency for Science, Technology and Research, Singapore, Singapore

  26. FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning

    Mar 9, 2026Peishen Yan, Yang Hua, Hao Wang +4Federated Fine-TuningFederated Learning Aggregation

    Shanghai Jiao Tong University · Queen’s University Belfast · Stevens Institute of Technology +2

  27. Towards Lightweight Adaptation of Speech Enhancement Models in Real-World Environments

    Mar 8, 2026Longbiao Cheng, Shih-Chii LiuSpeech EnhancementMemory-Efficient Fine-Tuning

    Institute of Neuroinformatics, University of Zurich and ETH Zurich

  28. Adaptive Capacity Allocation for Vision Language Action Fine-tuning

    Mar 8, 2026Donghoon Kim, Minji Bae, Unghui Nam +4VLA Model TrainingNonlinear LoRA

    Department of Electrical and Computer Engineering, Seoul National University, Seoul 08826, Republic of Korea · Department of Computer Science and Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea

  29. Domain-Specific Quality Estimation for Machine Translation in Low-Resource Scenarios

    Mar 7, 2026Namrata Patil Gurav, Akashdeep Ranu, Archchana Sindhujan +1MT EvaluationMachine Translation

    Institute for People-Centred AI, University of Surrey, United Kingdom

  30. Lifelong Embodied Navigation Learning

    Mar 6, 2026Xudong Wang, Jiahua Dong, Baichen Liu +3Embodied NavigationContinual Learning for LLM Agents

    State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Mohamed bin Zayed University of Artificial Intelligence