Low-Rank Adaptation

Also known as LoRA

Ask about this topic

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

542 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

  1. LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models

    Jul 8, 2026Sojung An, Junha Lee, Sujeong You +2Vision Foundation Model AdaptationLow-Rank Adaptation

    Korea University, Seoul, Republic of Korea · Korea Institute of Industrial Technology, Ansan, Republic of Korea · Seoul National University, Seoul, Republic of Korea

  2. Cross-Contextual Vision-Language Adaptation with LoRA for Personalized Severe Adverse Event Detection in Clinical Wound Monitoring

    Jul 6, 2026Aditi Naiknaware, Jian Sun, Aminreza Khandan +4VLM AdaptationLow-Rank Adaptation

    1San Diego State University · University of California, Los Angeles · University of California, Davis

  3. Learning Only What Valid Adapters Can Express: Subspace-Constrained Adaptation Against Fine-Tuning Poisoning

    Jul 6, 2026Fabien PollyLLM Backdoor AttacksLow-Rank Adaptation

    Independent researcher

  4. Localized LoRA-MoE: Block-wise Low-Rank Experts With Adaptive Routing

    Jul 6, 2026Babak Barazandeh, Subhabrata Majumdar, Vinay Prithyani +1Adaptive Model RoutingLow-Rank Adaptation

    Fortinet · Indian Institute of Management Bangalore · Citadel Securities +1

  5. LILAC: Layer-Wise Independent LoRAs and Cascaded Conditioning for Multi-Concept Customization of Diffusion Models

    Jul 6, 2026Marian Lupascu, Sebastian Ripa, Mihai Trascau +2Subject-Driven Image GenerationCompositional Image Generation

    Adobe Research, Romania · International Computer High School of Bucharest, Romania

  6. RSLoRA: Training-free Rank Allocation for LoRA via Representational Sensitivity Probing

    Jul 5, 2026Jiaqi Liu, Haidong Kang, Qihui Zhao +1Low-Rank AdaptationAdaptive Rank Allocation

    School of Computer Science and Technology Dalian University of Technology · Hebei Key Laboratory of Marine Perception Network and Data Processing Northeastern University · School of Computer and Communication Engineering Northeastern University +1

  7. SAD-LoRA: Spectral Alignment for Low-Rank Knowledge Distillation

    Jul 5, 2026Omer Tariq, Syed Muhammad Raza, Jeongbae SonLow-Rank AdaptationLow-Rank Compression

    Neubility Inc, Seoul, South Korea

  8. ReCoLoRA: Spectrum-Aware Recursive Consolidation for Continual LLM Fine-Tuning

    Jul 4, 2026Wentao LuContinual Learning for LLMsLow-Rank Adaptation

  9. Punching Above Their Weight: Classification-Head Fine-Tuning of Tiny Language Models (TLMs) for Verifiable Multiple-Choice Tasks

    Jul 4, 2026Bhavesh Sood, Jaromir SavelkaFine-TuningMultiple-Choice Question Answering

    Carnegie Mellon University

  10. Co-Adaptive Multi-Task LoRA: Transfer-Aware, Label-Free Control of Domain Participation

    Jul 3, 2026Wei Zhang, Lin Tang, Ming Zhao +1Multi-Task LearningData Selection

    1Sichuan University, Chengdu, China · University of Electronic Science and Technology of China, Chengdu, China

  11. Stacked LoRA for Subject-Adaptive EEG Foundation Models in Motor Imagery Decoding

    Jul 3, 2026Aymen Sarhane, Fouad Lbakali, Mouad Souissi +2Cross-Subject EEG DecodingBrain-Computer Interfaces

    IMT Atlantique, Lab-STICC, Brest, France

  12. ReLo-IRR: Reflection-Guided LoRA Framework for Image Reflection Removal

    Jul 3, 2026Chaoqun Wang, Yuehuan Wei, Haoxiang Cao +1Diffusion Models for Image RestorationSingle-Image Reflection Removal

    School of Artificial Intelligence, South China Normal University, Guangzhou, China · University of Science and Technology of China, Hefei, China

  13. When Does Resolution Help a Frozen Backbone? Global Attention at Resolution Predicts Scalable Adaptation for Camouflaged and Marine Animal Segmentation

    Jul 2, 2026Tyler Rust, Chandra KambhamettuImage SegmentationVision Foundation Model Adaptation

    University of Delaware · University of South Florida

  14. Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation

    Jul 2, 2026Jijie Zhang, Zhe Ren, Quan Zhang +1Low-Rank AdaptationLanguage Model Calibration

    School of Artificial Intelligence, Jilin University · Michigan State University

  15. EPnG: Adaptive Expert Prune-and-Grow for Parameter-Efficient MoE Fine-tuning

    Jul 2, 2026Ahin Lee, Sehyun Yun, Taesik GongMixture-of-Experts PruningMixture-of-Experts Language Models

    Ulsan National Institute of Science & Technology (UNIST)

  16. FRAME: Learning the Adaptation Domain with a Mixture of Fractional-Fourier Experts

    Jun 30, 2026Tom Saliencro, Maya Lindqvist, Rohan Desai +2Low-Rank AdaptationMixture-of-Experts Models

    University of California, Irvine · University of Washington

  17. CORA: Per-Slice Coherent Orthogonal Rotation for SVD-based Low-Rank Adaptation

    Jun 30, 2026Pengcheng Wang, Ziran Liu, Wei Wang +1LLM Fine-TuningLow-Rank Matrix Decomposition

    Purdue University · Shanghai Institute for Mathematics and Interdisciplinary Sciences / Fudan University · Futurewei Technologies, Inc.

  18. Geometry-Preserving Orthonormal Initialization for Low-Rank Adaptation in RLVR

    Jun 30, 2026Ruijia Zhang, Jiacheng Zhu, Hanqing Zhu +1Low-Rank AdaptationReinforcement Learning with Verifiable Rewards

    Department of Applied Mathematics and Statistics, Johns Hopkins University, MD, USA. · Meta Superintelligence Labs. · Department of Electrical and Computer Engineering, The University of Texas at Austin, TX, USA. +1

  19. Nonlinearity-Aware LoRA: Structured Gate Adaptation under Low-Rank Constraints

    Jun 30, 2026Shuai Yuan, Sudong Cai, Bingzhi Chen +4LLM Fine-TuningLow-Rank Adaptation

    1Beijing Institute of Technology, Zhuhai · 2The Hong Kong Polytechnic University · 3The University of Osaka +1

  20. Clinically Structured Rank-Gated LoRA for Cross-Benchmark Medical Question Answering

    Jun 30, 2026Hao Gong, Ruilin Gong, Yining HuangNonlinear LoRALow-Rank Adaptation

    University of Chinese Academy of Sciences · Henan University of Technology · Meta Emergence Laboratory

  21. Learning to Select, Not Relearn: Hard-Routed Mixtures of Reasoning LoRAs

    Jun 30, 2026Seyed Alireza Molavi, Zhan Su, Yan Hu +3Mixture-of-Experts Language ModelsLow-Rank Adaptation

    1Halmstad University, Halmstad, Sweden · 2The Chinese University of Hong Kong, Shenzhen, China

  22. Dual-Adaptive SAM3: Hierarchical Routing over Low-Rank Expert Layers for Parameter-Efficient Medical Image Segmentation

    Jun 30, 2026Ying Chen, Jinyue Li, Kun Wang +2VLM AdaptationText-Guided Image Segmentation

    Shenzhen Research Institute, The Chinese University of Hong Kong · University of Science and Technology of China · Nanyang Technological University +1

  23. Towards Continual Motion-Language Agents: LoRA Variants for Incremental Motion Understanding and Generation

    Jun 29, 2026Bertram Taetz, Hugo Albuquerque Cosme da Silva, Gabriele Bleser-TaetzText-to-Motion GenerationLow-Rank Adaptation

    IT & Engineering International University of Applied Sciences Erfurt, Germany

  24. EVAF: A Test-Retest Protocol for Selective Parametric Consolidation

    Jun 29, 2026Haoliang HanAgent MemoryContinual Learning for LLM Agents

    Institute of Biomedical Strategy, China Pharmaceutical University

  25. Mechanistically Eliciting Latent Behaviors in Language Models

    Jun 28, 2026Andrew Mack, Nina Panickssery, Alexander Matt TurnerLLM AlignmentLanguage Model Safety Evaluation

    †Principles of Intelligence · ‡Anthropic · §Independent

  26. TriageRA-CCF: Source-Side Clinical Confidence and Coverage Signals for Adaptive Rank Budgeting in Medical LLMs

    Jun 28, 2026Shucan Ji, Yining Huang, Hongliang GuoLow-Rank AdaptationMedical QA

    College of Computer Science, Sichuan University, Chengdu, China · Meta Emergence Laboratory

  27. BaRA: Bayesian Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning

    Jun 28, 2026Zhibin Duan, Yuhong Wang, Jiahong Fu +3Bayesian Neural NetworksLow-Rank Adaptation

    National Key Lab of Radar Signal Processing, Xidian University, Xi’an, Shaanxi 710071, China · School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi, 710049, China

  28. SOS-LoRA: Static Orthogonal-Subspace Low-Rank Adaptation with Fixed Multi-Scale Scaling

    Jun 27, 2026Yupeng Chang, Yuan Wu, Yi ChangLow-Rank AdaptationLLM Fine-Tuning

    School of Artificial Intelligence, Jilin University · Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China · International Center of Future Science, Jilin University

  29. MixTTA: Low-Rank Cross-Channel Mixing for Reliable Test-Time Adaptation

    Jun 26, 2026Mansoo Jung, Youngwook Kim, Jungwoo LeeDistribution Shift RobustnessTest-Time Adaptation

    Seoul National University, Seoul, Republic of Korea · Kookmin University, Seoul, Republic of Korea · HodooAI Labs, Seoul, Republic of Korea

  30. When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning

    Jun 26, 2026Tanguy Dieudonné, Giulia Lanzillotta, Enis Simsar +2Continual LearningLow-Rank Adaptation

    Department of Computer Science, ETH Zurich, Zurich, Switzerland.