Low-Rank Adaptation

Also known as LoRA

Ask about this topic

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

542 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

  1. Parameter-Efficient Fine-Tuning for Medical Text Summarization: A Comparative Study of Lora, Prompt Tuning, and Full Fine-Tuning

    Mar 23, 2026Ulugbek Shernazarov, Rostislav Svitsov, Bin ShiParameter-Efficient Fine-TuningText Summarization

    Telecom SudParis, Institut Polytechnique de Paris, France

  2. The Golden Subspace: Where Efficiency Meets Generalization in Continual Test-Time Adaptation

    Mar 23, 2026Guannan Lai, Da-Wei Zhou, Zhenguo Li +1Continual Test-Time AdaptationLow-Rank Adaptation

    School of Artificial Intelligence, Nanjing University · National Key Laboratory for Novel Software Technology, Nanjing University · Hong Kong University of Science and Technology +1

  3. Instruction-Tuned, but Not More Verifiable Instruction-Following: A Cross-Task Diagnosis for LoRA Adapters

    Mar 23, 2026Junyi ZouInstruction TuningInstruction Following

    Zjydiary Group

  4. Not All Layers Are Created Equal: Adaptive LoRA Ranks for Personalized Image Generation

    Mar 23, 2026Donald Shenaj, Federico Errica, Antonio CartaLow-Rank AdaptationAdaptive Rank Allocation

    University of Pisa · NEC Laboratories Europe

  5. Frequency Switching Mechanism for Parameter-E!cient Multi-Task Learning

    Mar 22, 2026Shih-Wen Liu, Yen-Chang Chen, Wei-Ta Chu +2Multi-Task LearningParameter-Efficient Fine-Tuning

    National Cheng Kung University, Tainan, Taiwan · NVIDIA Research, Taiwan

  6. FAAR: Efficient Frequency-Aware Multi-Task Fine-Tuning via Automatic Rank Selection

    Mar 20, 2026Maxime Fontana, Michael Spratling, Miaojing ShiNonlinear LoRAParameter-Efficient Fine-Tuning

    King’s College London London, UK · University of Luxembourg Esch-sur-Alzette, Luxembourg · Tongji University Shanghai, China

  7. Exploring parameter-efficient fine-tuning (PEFT) of billion-parameter vision models with QLoRA and DoRA: insights into generalization for limited-data image classification under a 98:1 test-to-train regime

    Mar 18, 2026Haiyu Yang, Sumit Sharma, Enhong Liu +1Precision Livestock FarmingParameter-Efficient Fine-Tuning

    Cornell University, College of Agriculture and Life Sciences, Ithaca, NY 14853

  8. Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognition

    Mar 18, 2026Yuxiang Mei, Delai Qiu, Shengping Liu +2Multilingual ASRSpeech Language Models

    Shanghai Engineering Research Center of Intelligent Education and Bigdata, Shanghai Normal University, Shanghai, 200234, China. · Unisound AI Technology Co., Ltd., Beijing, China. · SHNU-Unisound Natural Human-Computer Interaction Lab, Shanghai Normal University.

  9. Early Intervention for VFM-based Multimodal Medical Image Classification

    Mar 18, 2026Qijie Wei, Hailan Lin, Xirong LiCross-Modal Representation LearningVision Foundation Model Adaptation

    Renmin University of China, Beijing 100872, China

  10. EvoTale: Continual Character Customization for Expanding Story Worlds

    Mar 17, 2026Jinlu Zhang, Qiyun Wang, Baoxiang Du +6Image GenerationSubject-Driven Image Generation

    Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R., China. · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, 510000, P.R., China. · Fuxi AI Lab, NetEase Inc., Hangzhou, 310000, P.R., China.

  11. W2T: LoRA Weights Already Know What They Can Do

    Mar 16, 2026Xiaolong Han, Ferrante Neri, Zijian Jiang +4Weight-Space LearningLow-Rank Adaptation

    University of Surrey · Stanford University · University of Notre Dame

  12. MoLoRA: Composable Specialization via Per-Token Adapter Routing

    Mar 16, 2026Shrey Shah, Justin WagleLLM RoutingLow-Rank Adaptation

  13. AdapterTune: Zero-Initialized Low-Rank Adapters for Frozen Vision Transformers

    Mar 16, 2026Salim KhazemAdapter TuningLow-Rank Adaptation

    Talan Research Center, Paris, France

  14. Not All Directions Matter: Towards Structured and Task-Aware Low-Rank Model Adaptation

    Mar 15, 2026Xi Xiao, Chenrui Ma, Yunbei Zhang +7Low-Rank AdaptationParameter-Efficient Fine-Tuning

    University of Alabama at Birmingham · University of Virginia · Tulane University +5

  15. IGU-LoRA: Adaptive Rank Allocation via Integrated Gradients and Uncertainty-Aware Scoring

    Mar 14, 2026Xuan Cui, Huiyue Li, Run Zeng +5LLM Fine-TuningLow-Rank Adaptation

    Chongqing Technology and Business University · Shanghai Jiao Tong University

  16. Expert Pyramid Tuning: Efficient Parameter Fine-Tuning for Expertise-Driven Task Allocation

    Mar 13, 2026Jia-Chen Zhang, Zhen-Wei Yan, Yu-Jie Xiong +1Parameter-Efficient Fine-TuningLow-Rank Adaptation

    School of Electronic and Electrical Engineering, Shanghai University of Engineering Science 333 Longteng Road, Shanghai, China

  17. NeuroLoRA: Context-Aware Neuromodulation for Parameter-Efficient Multi-Task Adaptation

    Mar 12, 2026Yuxin Yang, Haoran Zhang, Mingxuan Li +6Parameter-Efficient Fine-TuningLow-Rank Adaptation

    Shanghai University · Fudan University

  18. AdaFuse: Accelerating Dynamic Adapter Inference via Token-Level Pre-Gating and Fused Kernel Optimization

    Mar 12, 2026Qiyang Li, Rui Kong, Yuchen Li +5LLM Inference AccelerationLow-Rank Adaptation

    Baidu Inc. · Shanghai Jiao Tong University

  19. Reversible Lifelong Model Editing via Semantic Routing-Based LoRA

    Mar 11, 2026Haihua Luo, Xuming Ran, Tommi Kärkkäinen +4Model EditingLanguage Model Editing

    Faculty of Information Technology, University of Jyväskylä, Jyväskylä, Finland · National University of Singapore, Singapore · School of Computer Science and Technology, Dalian University of Technology, Dalian, China

  20. Med-DualLoRA: Local Adaptation of Foundation Models for 3D Cardiac MRI

    Mar 11, 2026Joan Perramon-Llussà, Amelia Jiménez-Sánchez, Grzegorz Skorupko +4Federated Fine-TuningMedical Imaging Foundation Models

    Artificial Intelligence in Medicine Lab (BCN-AIM), Departament de Matemàtiques i Informàtica, Universitat de Barcelona, Spain · Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona, Spain

  21. Bilevel Layer-Positioning LoRA for Real Image Dehazing

    Mar 11, 2026Yan Zhang, Long Ma, Yuxin Feng +3Image DehazingLow-Rank Adaptation

    School of Computer Science and Engineering, Sun Yat-sen University · Dalian University of Technology · Xidian University +1

  22. ID-LoRA: Identity-Driven Audio-Video Personalization with In-Context LoRA

    Mar 10, 2026Aviad Dahan, Moran Yanuka, Noa Kraicer +2Audio-Video GenerationIdentity-Preserving Video Generation

    Tel Aviv University

  23. ReMix: Reinforcement routing for mixtures of LoRAs in LLM finetuning

    Mar 10, 2026Ruizhong Qiu, Hanqing Zeng, Yinglong Xia +15Parameter-Efficient Fine-TuningLow-Rank Adaptation

    University of Illinois Urbana-Champaign, IL, USA · Meta AI Menlo Park, CA, USA · Washington University St. Louis, WA, USA +1

  24. TemporalDoRA: Temporal PEFT for Robust Surgical Video Question Answering

    Mar 10, 2026Luca Carlini, Chiara Lena, Cesare Hassan +4Video QAParameter-Efficient Fine-Tuning

    Dipartimento di Elettronica, Informazione e Bioingegneria (DEIB), Politecnico di Milano, Italy · IRCCS Humanitas Research Hospital, Italy · UCL Hawkes Institute and Department of Computer Science, University College London, UK +1

  25. On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning

    Mar 10, 2026Muhammad Ahmad, Jingjing Zheng, Yankai CaoParameter-Efficient Fine-TuningCatastrophic Forgetting

    Department of Electrical and Computer Engineering, The University of British Columbia · Department of Mathematics, The University of British Columbia · Department of Chemical and Biological Engineering, The University of British Columbia

  26. ForgeDreamer: Industrial Text-to-3D Generation with Multi-Expert LoRA and Cross-View Hypergraph

    Mar 10, 2026Junhao Cai, Deyu Zeng, Junhao Pang +3Text-to-3D GenerationLow-Rank Adaptation

    Shenzhen University, Shenzhen, Guangdong 518060, China · Guangzhou Maritime University, Guangzhou, Guangdong 510725, China

  27. Alfa: Attentive Low-Rank Filter Adaptation for Structure-Aware Cross-Domain Personalized Gaze Estimation

    Mar 9, 2026He-Yen Hsieh, Wei-Te Mark Ting, H. T. KungGaze EstimationLow-Rank Adaptation

    Harvard University

  28. Video2LoRA: Unified Semantic-Controlled Video Generation via Per-Reference-Video LoRA

    Mar 9, 2026Zexi Wu, Baolu Li, Jing Dai +5Controllable Video GenerationVideo-Reference Video Generation

    Dalian University of Technology · HKUST

  29. Stabilized Fine-Tuning with LoRA in Federated Learning: Mitigating the Side Effect of Client Size and Rank via the Scaling Factor

    Mar 9, 2026Jiayu Huang, Xiaohu Wu, Tiantian He +1Federated Fine-TuningLow-Rank Adaptation

    Beijing University of Posts and Telecommunications, Beijing, China · Agency for Science, Technology and Research, Singapore, Singapore

  30. FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuning

    Mar 9, 2026Peishen Yan, Yang Hua, Hao Wang +4Federated Fine-TuningFederated Learning Aggregation

    Shanghai Jiao Tong University · Queen’s University Belfast · Stevens Institute of Technology +2