Low-Rank Adaptation

Also known as LoRA

Ask about this topic

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

542 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

  1. Efficient Handwriting-Based Alzheimer,s Disease Diagnosis Using a Low-Rank Mixture of Experts Deep Learning Framework

    Apr 14, 2026Wu Wang, Yuang Cheng, Fouzi Harrou +1Alzheimer's Disease ClassificationMixture-of-Experts Models

    Center for Applied Statistics and School of Statistics Renmin University of China Beijing 100872, China · King Abdullah University of Science and Technology (KAUST) Computer, Electrical and Mathematical Sciences and Engineering (CEMSE) Division Thuwal 23955-6900, Saudi Arabia

  2. BID-LoRA: A Parameter-Efficient Framework for Continual Learning and Unlearning

    Apr 14, 2026Jagadeesh Rachapudi, Ritali Vatsi, Praful Hambarde +1Continual LearningMachine Unlearning

    Indian Institute of Technology Mandi, Mandi, India

  3. Orthogonal Subspace Projection for Continual Machine Unlearning via SVD-Based LoRA

    Apr 14, 2026Yogachandran Rahulamathavan, Nasir Iqbal, Juncheng Hu +1Machine UnlearningLow-Rank Adaptation

    Institute for Digital Technologies, Loughborough University, UK · School of Artificial Intelligent Medicine, Guilin Medical University, Guilin 541199, CN

  4. Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions

    Apr 12, 2026Wenhao Zhang, Lin Mu, Li Ni +2Nonlinear LoRALLM Fine-Tuning

    Anhui University · University of Science and Technology of China

  5. SpectralLoRA: Is Low-Frequency Structure Sufficient for LoRA Adaptation? A Spectral Analysis of Weight Updates

    Apr 12, 2026Rajveer SinghLow-Rank AdaptationParameter-Efficient Fine-Tuning

    Indian Institute of Technology Roorkee

  6. Mining Attribute Subspaces for Efficient Fine-tuning of 3D Foundation Models

    Apr 11, 2026Yu Jiang, Hanwen Jiang, Ahmed Abdelkader +4Parameter-Efficient Fine-TuningLow-Rank Adaptation

    The University of Texas at Austin · Shanghai Jiao Tong University · Adobe Research +1

  7. FREE-Switch: Frequency-based Dynamic LoRA Switch for Style Transfer

    Apr 11, 2026Shenghe Zheng, Minyu Zhang, Tianhao Liu +1Diffusion-Based Image GenerationDiffusion Models

    Harbin Institute of Technology

  8. Spectral Geometry of LoRA Adapters Encodes Training Objective and Predicts Harmful Compliance

    Apr 10, 2026Roi PaulLLM AuditingLow-Rank Adaptation

    Independent Researcher

  9. A Little Rank Goes a Long Way: Random Scaffolds with LoRA Adapters Are All You Need

    Apr 9, 2026Hananel Hazan, Yanbo Zhang, Benedikt Hartl +1Random Feature MethodsParameter-Efficient Fine-Tuning

    Allen Discovery Center at Tufts University, Medford, Massachusetts, USA · Institute for Theoretical Physics, TU Wien, Wien, Austria · Wyss Institute for Biologically Inspired Engineering, Harvard University, Boston, Massachusetts, USA

  10. SOLAR: Communication-Efficient Model Adaptation via Subspace-Oriented Latent Adapter Reparametrization

    Apr 9, 2026Seyed Mahmoud Sajjadi Mohammadabadi, Xiaolong Ma, Lei Yang +2Model CompressionNeural Network Compression

    University of Nevada, Reno, Reno, NV, USA · Argonne National Laboratory, Lemont, IL, USA · University of Houston, Houston, TX, USA +1

  11. Sensitivity-Positional Co-Localization in GQA Transformers

    Apr 9, 2026Manoj Chandrashekar RaoTransformerRotary Positional Embeddings

    Independent Researcher

  12. An empirical study of LoRA-based fine-tuning of large language models for automated test case generation

    Apr 8, 2026Milad Moradi, Ke Yan, David Colwell +1Automated Test GenerationLLM Fine-Tuning

    AI Research Lab, Tricentis, Vienna, Austria · AI Research Lab, Tricentis, Sydney, Australia

  13. TalkLoRA: Communication-Aware Mixture of Low-Rank Adaptation for Large Language Models

    Apr 7, 2026Lin Mu, Haiyang Wang, Li Ni +4Mixture-of-Experts Language ModelsLow-Rank Adaptation

    Anhui University · Hefei University · University of Science and Technology of China

  14. ALTO: Adaptive LoRA Tuning and Orchestration for Heterogeneous LoRA Training Workloads

    Apr 7, 2026Jingwei Zuo, Xinze Feng, Zien Liu +5LLM Fine-TuningLow-Rank Adaptation

    Rice University · Independent Researcher

  15. Aletheia: Gradient-Guided Layer Selection for Efficient LoRA Fine-Tuning Across Architectures

    Apr 4, 2026Abdulmalek SaketLLM Fine-TuningEfficient Language Model Training

    Royal Fenice Kft / ALETHEIA PROTOCOL research, Budapest, Hungary

  16. Scalable Variational Bayesian Fine-Tuning of LLMs via Orthogonalized Low-Rank Adapters

    Apr 3, 2026Haotian Xiang, Bingcong Li, Qin LuLLM Uncertainty EstimationBayesian LoRA

    School of Electrical and Computer Engineering, University of Georgia, Athens, GA, USA · Department of Computer Science, ETH Zurich, Zurich, Switzerland

  17. Ouroboros: Dynamic Weight Generation for Recursive Transformers via Input-Conditioned LoRA Modulation

    Apr 2, 2026Jaber Jaber, Osama JaberRecurrent TransformersDecoder-Only Language Models

    RightNow AI

  18. MiCA Learns More Knowledge Than LoRA and Full Fine-Tuning

    Apr 2, 2026Sten Rüdiger, Sebastian RaschkaLLM Fine-TuningParameter-Efficient Fine-Tuning

    Independent Researcher · RAIR Lab

  19. Sparse Spectral LoRA: Routed Experts for Medical VLMs

    Apr 1, 2026Omid Nejati Manzari, Hojat Asgariandehkordi, Taha Koleilat +2Medical VLMsVision-Language Models

    Concordia University, Montreal, Canada

  20. Matched-Learning-Rate Analysis of Attention Drift and Transfer Retention in Fine-Tuned CLIP

    Apr 1, 2026Ruize XiaVLM AdaptationLow-Rank Adaptation

    Independent Researcher

  21. CoLA: Cross-Modal Low-rank Adaptation for Multimodal Downstream Tasks

    Apr 1, 2026Wish Suharitdamrong, Tony Alex, Muhammad Awais +1Cross-Modal LearningVLM Adaptation

    Surrey Institute for People-Centred AI, University of Surrey, Guildford, GU2 7XH, UK · Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK

  22. Curvature-Guided LoRA: Matching Full Fine-Tuning in Function Space

    Mar 31, 2026Frédéric Zheng, Alexandre ProutièreLow-Rank AdaptationParameter-Efficient Fine-Tuning

    KTH, Stockholm, Sweden

  23. One-for-All: A Lightweight Stabilized and Parameter-Efficient Pre-trained LLM for Time Series Forecasting

    Mar 31, 2026Prasanjit Dey, Soumyabrata Dev, Bianca Schoen-PhelanLLM-Based Time Series ForecastingMultivariate Time Series Forecasting

    ADAPT SFI Research Centre, School of Computer Science, Technological University Dublin, Ireland · ADAPT SFI Research Centre, School of Computer Science, University College Dublin

  24. RecycleLoRA: Rank-Revealing QR-Based Dual-LoRA Subspace Adaptation for Domain Generalized Semantic Segmentation

    Mar 30, 2026Chanseul Cho, Seokju Yun, Jeaseong Jeon +2Domain Generalization for Semantic SegmentationDomain Generalization

    Machine Intelligence Laboratory, University of Seoul, Korea

  25. Low-Rank Adaptation Reduces Catastrophic Forgetting in Sequential Transformer Encoder Fine-Tuning: Controlled Empirical Evidence and Frozen-Backbone Representation Probes

    Mar 29, 2026Ashish PandeyCatastrophic ForgettingLow-Rank Adaptation

  26. Label-Free Cross-Task LoRA Merging with Null-Space Compression

    Mar 27, 2026Wonyoung Lee, Wooseong Jeong, Kuk-Jin YoonModel MergingLanguage Model Merging

    KAIST · *Equal contribution to this work.

  27. Preference-Aligned LoRA Merging: Preserving Subspace Coverage and Addressing Directional Anisotropy

    Mar 27, 2026Wooseong Jeong, Wonyoung Lee, Kuk-Jin YoonLow-Rank AdaptationLanguage Model Merging

    KAIST

  28. Low-Rank-Modulated Functa: Exploring the Latent Space of Implicit Neural Representations for Interpretable Ultrasound Video Analysis

    Mar 26, 2026Julia Wolleb, Cristiana Baloescu, Alicia Durrer +2Medical Image Representation LearningImplicit Neural Representations

    Dept. of Biomedical Informatics & Data Science, Yale University, New Haven, USA · Yale Biomedical Imaging Institute, Yale University, New Haven, USA · Department of Emergency Medicine, Yale School of Medicine, New Haven, USA +2

  29. MoE-Sieve: Routing-Guided LoRA for Efficient MoE Fine-Tuning

    Mar 25, 2026Andrea ManzoniMixture-of-Experts Fine-TuningFine-Tuning

  30. Scaling DoRA: High-Rank Adaptation via Factored Norms and Fused Kernels

    Mar 23, 2026Alexandra Zelenin, Alexandra ZhuravlyovaLow-Rank AdaptationMemory-Efficient Fine-Tuning