Low-Rank Adaptation

Also known as LoRA

Ask about this topic

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

542 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

  1. Learning in the Fisher Subspace: A Guided Initialization for LoRA Fine-Tuning

    May 1, 2026Zhi-Quan Feng, Ying-Jia Lin, Hung-Yu KaoLLM Fine-TuningLow-Rank Adaptation

    Department of Computer Science and Information Engineering, National Cheng Kung University · Department of Artificial Intelligence, Chang Gung University · Department of Computer Science, National Tsing Hua University

  2. Post-Optimization Adaptive Rank Allocation for LoRA

    Apr 30, 2026Vishnuprasadh Kumaravelu, Sunil Gupta, P. K. SrijithLow-Rank AdaptationLow-Rank Compression

    1Indian Institute of Technology Hyderabad · 2Deakin University

  3. BoostLoRA: Growing Effective Rank by Boosting Adapters

    Apr 30, 2026Raviteja Anantha, Nick Levato, Layne C. PriceLLM Fine-TuningAdapter Tuning

    Amazon

  4. Generalizing the Geometry of Model Merging Through Frechet Averages

    Apr 29, 2026Marvin F. da Silva, Mohammed Adnan, Felix Dangel +1Low-Rank AdaptationModel Merging

    Faculty of Computer Science, Dalhousie University · 2Vector Insitute for Artificial Intelligence · Schulich School of Engineering, University of Calgary +2

  5. Decoupling Knowledge and Task Subspaces for Composable Parametric Retrieval Augmented Generation

    Apr 29, 2026Weihang Su, Hanwen Zhang, Qingyao Ai +1Retrieval-Augmented GenerationLow-Rank Adaptation

    Department of Computer Science and Technology, Tsinghua University

  6. Towards Unified Multi-task EEG Analysis with Low-Rank Adaptation

    Apr 28, 2026Sicheng Dai, Kai Chen, Hongwang Xiao +2ElectroencephalographyMulti-Task Learning

    Institute of Automation, Chinese Academy of Sciences · School of Artifcial Intelligence, University of Chinese Academy of Sciences · State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology +2

  7. BaLoRA: Bayesian Low-Rank Adaptation of Large Scale Models

    Apr 27, 2026Dario Coscia, Sindy Löwe, Max WellingBayesian Neural NetworksUncertainty Quantification

    mathLab, SISSA · CuspAI

  8. Fed-DLoRA: Efficient Wireless Federated Learning with Dynamic Low-Rank Adaptation

    Apr 27, 2026Huaicheng Li, Junhui Zhao, Haoyu Quan +1Communication-Efficient Distributed TrainingLow-Rank Adaptation

    School of Electronic and Information Engineering, Beijing Jiaotong University, Beijing 100044, China · School of Communication and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

  9. The Override Gap: A Magnitude Account of Knowledge Conflict Failure in Hypernetwork-Based Instant LLM Adaptation

    Apr 26, 2026Shuaizhi Cheng, Xiang Shi, Zhiwei Zhang +1Knowledge Conflicts in Language ModelsLLM Fine-Tuning

    Harbin Institute of Technology · KigLand Machine Learning Lab · Imperial College London

  10. Where Should LoRA Go? Component-Type Placement in Hybrid Language Models

    Apr 24, 2026Hector Borobia, Elies Seguí-Mas, Guillermina Tormo-CarbóLow-Rank AdaptationRecurrent Transformers

    VRAIN – Valencian Research Institute for Artificial Intelligence, Universitat Politècnica de València, Valencia, Spain · Department of Economics and Social Sciences, Universitat Politècnica de València, Valencia, Spain · Department of Business Organisation, Universitat Politècnica de València, Valencia, Spain

  11. Foundation models for discovering robust biomarkers of neurological disorders from dynamic functional connectivity

    Apr 23, 2026Deepank Girish, Yi Hao Chan, Sukrit Gupta +2Medical Imaging Foundation ModelsFunctional Connectivity

    College of Computing and Data Science, Nanyang Technological University, Singapore

  12. Low-Rank Adaptation Redux for Large Models

    Apr 23, 2026Bingcong Li, Yilang Zhang, Georgios B. GiannakisAdapter TuningLow-Rank Adaptation

    Dept. of CS, ETH Zürich, Switzerland · Dept. of ECE, U. of Minnesota, USA · Morgan Stanley, USA

  13. GiVA: Gradient-Informed Bases for Vector-Based Adaptation

    Apr 23, 2026Neeraj Gangwar, Rishabh Deshmukh, Michael Shavlovsky +4Low-Rank AdaptationParameter-Efficient Fine-Tuning

    University of Illinois Urbana-Champaign · Amazon · Stanford University

  14. Formalising the Logit Shift Induced by LoRA: A Technical Note

    Apr 22, 2026Xiang Shi, Shuaizhi Cheng, Mingwei LiLow-Rank AdaptationParameter-Efficient Fine-Tuning

    Imperial College London · Harbin Institute of Technology · KigLand Learning Lab

  15. RDP LoRA: Geometry-Driven Identification for Parameter-Efficient Adaptation in Large Language Models

    Apr 21, 2026Yusuf Çelebi, Yağız Asker, Özay Ezerceli +4LLM Fine-TuningLow-Rank Adaptation

  16. ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning

    Apr 21, 2026Xianming Li, Zongxi Li, Tsz-fung Andrew Lee +3Fine-TuningAdapter Tuning

    PolyU · School of Data Science, Lingnan University, Hong Kong SAR

  17. SAMoRA: Semantic-Aware Mixture of LoRA Experts for Task-Adaptive Learning

    Apr 21, 2026Boyan Shi, Wei Chen, Shuyuan Zhao +4Multi-Task LearningLow-Rank Adaptation

    School of Computer Science and Technology, Beijing Jiaotong University, China · 2Guangxi Key Lab of Trusted Software, Guilin University of Electronic Technology, China · 3Beijing Key Lab of Traffic Data Mining and Embodied Intelligence, China +2

  18. Low-Rank Adaptation for Critic Learning in Off-Policy Reinforcement Learning

    Apr 21, 2026Yuan Zhuang, Yuexin Bian, Sihong He +7Low-Rank AdaptationOff-Policy RL

    University of Connecticut · University of California San Diego · University of Texas at Arlington +1

  19. TLoRA: Task-aware Low Rank Adaptation of Large Language Models

    Apr 20, 2026Weicheng Lin, Yi Zhang, Jiawei Dang +1LLM Fine-TuningLow-Rank Adaptation

    College of Computer Science and Software Engineering, Shenzhen University, China

  20. GraphLoRA: Structure-Aware Low-Rank Adaptation for Large Language Model Recommendation

    Apr 20, 2026Lin Mu, Guoji Wang, Li Ni +4Low-Rank AdaptationLarge Language Model-Based Recommendation

    Anhui University · Hefei University · University of Science and Technology of China

  21. Evolutionary Negative Module Pruning for Better LoRA Merging

    Apr 20, 2026Anda Cao, Zhuo Gou, Yi Wang +5Low-Rank AdaptationLanguage Model Merging

    College of Computer Science and Technology, Zhejiang University · School of Software Technology, Zhejiang University · 3State Key Laboratory of Blockchain and Security, Zhejiang University +1

  22. HiP-LoRA: Budgeted Spectral Plasticity for Robust Low-Rank Adaptation

    Apr 20, 2026Lixian Chen, Jianhong TanContinual Learning for LLMsLow-Rank Adaptation

    Guangdong University of Technology

  23. R-FLoRA: Residual-Statistic-Gated Low-Rank Adaptation for Single-Image Face Morphing Attack Detection

    Apr 19, 2026Raghavendra RamachandraImage Forgery DetectionVision Foundation Model Adaptation

    SAFE Center, Department of Information Security and Communication Technology, Norwegian University of Science and Technology (NTNU), Norway

  24. Crowded in B-Space: Calibrating Shared Directions for LoRA Merging

    Apr 18, 2026Yixuan Tang, Yi YangAdapter TuningLow-Rank Adaptation

    The Hong Kong University of Science and Technology

  25. Cross-Modal Bayesian Low-Rank Adaptation for Uncertainty-Aware Multimodal Learning

    Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinLow-Rank AdaptationUncertainty-Aware Multimodal Fusion

    Dalhousie University, Halifax NS, Canada

  26. Beyond Feature Fusion: Contextual Bayesian PEFT for Multimodal Uncertainty Estimation

    Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinLow-Rank AdaptationBayesian LoRA

    Dalhousie University, Halifax NS, Canada

  27. JumpLoRA: Sparse Adapters for Continual Learning in Large Language Models

    Apr 17, 2026Alexandra Dragomir, Ioana Pintilie, Antonio Barbalau +6Continual Learning for LLMsLow-Rank Adaptation

    Bitdefender · ETH Zurich · University of Bucharest

  28. POLAR: Online Learning for LoRA Adapter Caching and Routing in Edge LLM Serving

    Apr 17, 2026Shaoang Li, Jian LiLLM ServingLow-Rank Adaptation

    Stony Brook University

  29. PEFT of SLM for Telecommunications Customer Support: A Comparative Study of LoRA Configurations with Energy Consumption Analysis

    Apr 17, 2026Lucas Tamic, Ilan Jaffeux-Cheniout, Xavier MarjouCustomer Support AutomationSmall Language Models

    Orange, France

  30. TLoRA+: A Low-Rank Parameter-Efficient Fine-Tuning Method for Large Language Models

    Apr 15, 2026Yarui Cao, Kai LiuLLM Fine-TuningParameter-Efficient Fine-Tuning

    Department of Computer Science, Clemson University, Clemson, SC 29634, USA