Low-Rank Adaptation

Also known as LoRA

Ask about this topic

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

542 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

  1. Frames2LoRA: Parametric Video Internalization for Vision-Language Models

    Jun 3, 2026Manan Suri, Sarvesh Baskar, Dinesh ManochaVLM AdaptationEfficient VLM Inference

    University of Maryland, College Park

  2. OLIVE: Online Low-Rank Incremental Learning for Efficient Adaptive Exoskeletons

    Jun 3, 2026Dong Liu, Yanxuan Yu, Ben Lengerich +2Robotic ControlLow-Rank Adaptation

    University of California, Los Angeles Los Angeles, California, USA · Columbia University New York, New York, USA · University of Wisconsin-Madison Madison, Wisconsin, USA +1

  3. Parameter-Efficient Fine-Tuning with Learnable Rank

    Jun 3, 2026Arpit Garg, Simon Lucey, Hemanth SaratchandranNonlinear LoRALow-Rank Adaptation

    Australian Institute for Machine Learning Adelaide University

  4. Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data

    Jun 2, 2026Devleena Das, Rajeev Patwari, Elliott Delaye +1Mixed-Precision QuantizationLLM Quantization

    Advanced Micro Devices, Inc. (AMD)

  5. Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter

    Jun 2, 2026Zhengbao He, Ruiqi Ding, Zhehao Huang +3Adapter TuningLow-Rank Adaptation

    Institute of Image Processing and Pattern Recognition, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China · Shanghai Key Laboratory of Flexible Medical Robotics, Tongren Hospital, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China

  6. Normality-Preserving Continual Industrial Anomaly Detection via Orthogonal LoRA Banks

    Jun 1, 2026Weibai Fang, Haijun Che, Feiyang Ren +1Industrial Anomaly DetectionAnomaly Localization

  7. Low-Rank Adaptation of Frozen Vision-Language Models for Blind Image Quality Assessment

    Jun 1, 2026Bishr Omer Adam, Xu LiVLM AdaptationLow-Rank Adaptation

    School of Electronics and Information, Northwestern Polytechnical University, Xi’an, China

  8. Parameter-Efficient Fine-Tuning of Large Pretrained Models for Instance Segmentation Tasks

    Jun 1, 2026Nermeen Abou Baker, David Rohrschneider, Uwe HandmannFine-TuningInstance Segmentation

    Ruhr West University of applied sciences · Lützowstraße 5, 46236 Bottrop, Germany: Ruhr West University of Applied Sciences

  9. Beyond Low-Rank: Low-Rank Sparse Prompting via Spiking Neural Network and Prompt Factorization

    Jun 1, 2026Yumiao Zhao, Bo Jiang, Beibei Wang +3Vision Foundation Model AdaptationLow-Rank Adaptation

    Information Materials and Intelligent Sensing Laboratory of Anhui Province, Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University

  10. G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs

    Jun 1, 2026Yuhan Wang, Yibo Ding, Yutong Ye +4Continual LearningLow-Rank Adaptation

    School of Computer Science and Engineering, Beihang University Beijing, China · Department of Statistics, Columbia University New York, NY, USA · College of Computer Science, Beijing University of Technology Beijing, China

  11. GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptation

    May 31, 2026Shihao Zhang, Rayan SaabLLM QuantizationLow-Rank Adaptation

    Department of Mathematics, University of California San Diego

  12. PRISM: Gauge-Invariant Tangent-Space Differentially Private LoRA

    May 31, 2026Shihao Wang, Xueru ZhangDifferentially Private Stochastic Gradient DescentLow-Rank Adaptation

    Department of Computer Science and Engineering, The Ohio State University, Columbus, OH, USA.

  13. Subliminal Learning is a LoRA Artifact

    May 30, 2026Todd Nief, Harvey Yiyun Fu, Mark Muchane +1LLM Fine-TuningLow-Rank Adaptation

    Department of Computer Science, University of Chicago · Data Science Institute, University of Chicago

  14. Finer Parameter Steps for Low-Rank PEFT: A Controlled Study with CP Tensor Adapters

    May 29, 2026Xinjue Wang, Xiuheng Wang, Yejun Zhang +3Tensor DecompositionAdapter Tuning

    Department of Information and Communications Engineering, Aalto University, 02150, Espoo, Finland · Université de Lorraine, CNRS, CRAN, France · Department of Computer Science, Aalto University, 02150, Espoo, Finland +1

  15. From "Weak" Signals to Strong Models: Preference Delta Aggregation with LoRA Merging

    May 29, 2026Qi Sun, Siyue Zhang, Yulin Chen +3Pairwise Preference LearningLow-Rank Adaptation

    NYU Shanghai · NTU · NYU +1

  16. ARCA: Adapter-Residual Credit Assignment When Token Signals Degenerate

    May 29, 2026Rodney Lafuente-MercadoToken-Level Credit AssignmentRL for Language Model Reasoning

    Scale AI

  17. Balanced LoRA: Removing Parameter Invariance to Accelerate Convergence

    May 29, 2026Valérie Castin, Kimia Nadjahi, Pierre Ablin +1LLM Fine-TuningLow-Rank Adaptation

    1 ´Ecole Normale Sup´erieure PSL, Paris, France · 2CNRS · 3Apple, Paris, France.

  18. A Pilot Study on Curator-Guided Multilingual Art Description for Blind and Low-Vision Audiences with Small Vision-Language Models

    May 29, 2026Iosif Tsangko, Andreas Triantafyllopoulos, George Margetis +2Image CaptioningVLM Adaptation

    Technical University of Munich Munich, Germany · Foundation for Research and Technology – Hellas Heraklion, Greece · National University of Science and Technology Politehnica Bucharest Bucharest, Romania

  19. CSULoRA: Closest Safe Update Low-Rank Adaptation

    May 28, 2026Oleksandr Marchenko Breneur, Adelaide Danilov, Aria Nourbakhsh +1Safety-Preserving Fine-Tuning of Language ModelsAdapter Tuning

    Department of Computer Science, University of Luxembourg Esch-sur-Alzette, Luxembourg

  20. Foundation-Preserving Optimization in Generalized Eigenspace

    May 28, 2026Dongjun Kim, Adrian de Wynter, Huancheng Chen +2Low-Rank AdaptationParameter-Efficient Fine-Tuning

    The University of Texas at Austin · Microsoft · Microsoft AI +1

  21. Token-Level Generalization in LoRA Adapter Backdoors: Attack Characterization and Behavioral Detection

    May 28, 2026Travis LelleLLM Backdoor AttacksBackdoor Detection

  22. iLoRA: Bayesian Low-Rank Adaptation with Latent Interaction Graphs for Microbiome Diagnosis

    May 28, 2026Yang Song, Yixuan Zhang, Lingfa Meng +5Graph Structure LearningMedical Diagnosis

    Section of Health Data Science & AI, Department of Public Health, University of Copenhagen, Copenhagen, Denmark · University of Copenhagen, Copenhagen, Denmark · Rutgers University, New Brunswick, NJ, USA +1

  23. SLAD : Shared LoRA Adapters for Task Specific Distillation

    May 28, 2026Reda Bensaid, Yassir Bendou, Vincent Gripon +1Vision Foundation Model AdaptationAdapter Tuning

    IMT Atlantique, Brest, France · Polytechnique Montr´eal, Montr´eal, Canada

  24. NaRA: Noise-Aware LoRA for Parameter-Efficient Fine-Tuning of Diffusion LLMs

    May 28, 2026Shuaidi Wang, Zhan Zhuang, Ruping Huang +1Low-Rank AdaptationDiffusion Language Models

    Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China · Department of Computer Science, City University of Hong Kong, Hong Kong, China · Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong, China

  25. On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference

    May 28, 2026Daniel Dold, Emanuel Sommer, Julius Kobialka +2Mode ConnectivityBayesian Inference

    HTWG Konstanz · LMU Munich · Munich Center for Machine Learning (MCML)

  26. Mask the Target: A Plug-and-Play Regularizer Against LoRA Forgetting

    May 28, 2026Runze Xu, Arpit Garg, Hemanth Saratchandran +1Continual Learning for LLMsLLM Fine-Tuning

    Australian Institute for Machine Learning Adelaide University Adelaide SA 5000

  27. FedSmoothLoRA: Toward Smoother and Faster Convergence in Federated Low-Rank Adaptation

    May 28, 2026Zehao Wang, Guanglei Yang, Yihan Zeng +4Low-Rank AdaptationFederated Fine-Tuning

    1Harbin Institute of Technology · 2Huawei Noah’s Ark Lab · University College Dublin

  28. FoRA: Fisher-orthogonal Rank Adaptation for Parameter-Efficient Fine-Tuning

    May 28, 2026Juneyoung Park, Seongbae Lee, Han-Sang Lee +6LLM Fine-TuningAdapter Tuning

    OptAI Inc · LG Uplus

  29. Janus-LoRA: A Balanced Low-Rank Adaptation for Continual Learning

    May 27, 2026Cheng Chen, Pengpeng Zeng, Yuyu Guo +3Stability-Plasticity DilemmaContinual Learning

    School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China · School of Computer Science and Technology, Tongji University, Shanghai, China · Independent Researcher +1

  30. Feature Geometry of LoRA Adapters: A Sparse Autoencoder Analysis of Representational Divergence in Fine-Tuned Language Models

    May 27, 2026Prasanth K KLLM InterpretabilitySparse Autoencoders

    Independent AI Safety Researcher