Low-Rank Adaptation

Also known as LoRA

Ask about this topic

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

542 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

  1. Low-Quality Face Recognition using Center Aligned Representations and Local Margin Constraints

    Sep 1, 2026Vedat Can Dilaver, Benjamin S. RigganMargin-Based LearningLow-Rank Adaptation

    University of Nebraska Lincoln

  2. Frozen Cores Need Task Signal: Fisher-Whitened Cross-Covariance for Low-Resource LLM Adaptation

    Sep 1, 2026Wentao Ye, Zhanming Shen, Zhiqing Xiao +3Memory-Efficient Fine-TuningLLM Fine-Tuning

    Zhejiang University · Hunyuan, Tencent

  3. Breaking the Structural Identity: Personalized Federated LoRA Fine-tuning under Rank Heterogeneity

    Sep 1, 2026Lei Wang, Jieming Bian, Letian Zhang +1Non-IID Federated LearningLow-Rank Adaptation

    University of Florida · Gainesville, FL 32611 · Middle Tennessee State University +1

  4. Lightweight Adaptation of EEG Foundation Models for Stroke Motor Imagery Decoding: Domain Shift and Subject-Level Robustness

    Aug 31, 2026Anh T. Nguyen, Zihua Sun, Michelle J. JohnsonCross-Subject EEG DecodingMI Classification

    School of Engineering and Applied Science, Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA · School of Engineering and Applied Science, Department of Electrical and Systems Engineering, University of Pennsylvania, Philadelphia, PA, USA · Department of Physical Medicine and Rehabilitation, Department of Bioengineering, and Department of Mechanical Engineering and Applied Mechanics, University of Pennsylvania, Philadelphia, PA, USA

  5. Normalized Low-Rank Adaptation

    Aug 31, 2026Jiale Kang, Ziyin Yue, Zheng Zhan +2Low-Rank AdaptationParameter-Efficient Fine-Tuning

    1Yuanshi Intelligence · 3The Chinese University of Hong Kong · 2Microsoft Research +1

  6. RW-LoRA: Communication-Efficient Decentralized LoRA Fine-Tuning via Random Walks

    Aug 31, 2026Xingran Chen, Rohit Bhagat, Ghadir Ayache +3Fine-TuningCommunication-Efficient Distributed Training

    Engineering Systems and Design Pillar, Singapore University of Technology and Design, Singapore 487372 · Department of Electrical and Computer Engineering, Rutgers University, Piscataway Township, NJ 08854, USA · LinkedIn, New York, NY 10118, USA +2

  7. CoRe-MoE: Compact Reusable MoE for Continual Multimodal Instruction Tuning

    Aug 28, 2026Runze Liu, Naibin Gu, Mingxu Ai +4Continual Learning for VLMsMultimodal Instruction Tuning

    Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China

  8. Geo-LoRA: Geometry-Aware Subspace Evolution for Low-Rank Adaptation in Continual Learning

    Aug 27, 2026Yibo FengContinual LearningContinual LoRA

    University of Electronic Science and Technology of China Chengdu, China

  9. FAN-LoRA: A Fourier-Adaptive Nonlinear Low-Rank Adaptor for Medical Foundation Model Domain Adaptation

    Aug 27, 2026Ziquan Liu, Zhewei Zhu, Xuyang ShiVision Foundation Model AdaptationLow-Rank Adaptation

    School of Information and Control Engineering, Southwest University of Science and Technology Mianyang, China

  10. How Much Rank Does LoRA Need? Rank-Error Bounds for Transformer Attention

    Aug 26, 2026Gerard Conangla PlanesAttention ApproximationLow-Rank Adaptation

    Aily Labs

  11. Learning New Facts with QLoRA: An Acquisition-Retention Frontier

    Aug 26, 2026Estelle Zheng, Sébastien Warichet, Emmanuel Helbert +1Factual Knowledge in Language ModelsLow-Rank Adaptation

    LORIA, CNRS, France · Alcatel-Lucent Enterprise, France

  12. Low-Rank Ternary Adaptation for Fine-Tuning Transformers

    Aug 25, 2026Alexandru-Dragos Manolache, Yunqiang Li, Jan van GemertParameter-Efficient Fine-TuningLow-Rank Adaptation

    Delft University of Technology, The Netherlands · Amazon Development Center

  13. The Emergence of Relevance Through Axiomatic Attention Patterns During LoRA Fine-Tuning

    Aug 24, 2026Matthew Perlman, Atharva Nijasure, James AllanLLM RerankingInformation Retrieval

    University of Massachusetts Amherst

  14. Reasoning-Oriented Post-Training and Inference-Time LoRA Rescaling for Audio-Dependent Question Answering

    Aug 24, 2026Weiteng Hu, Yin Cao, Jun YangAudio QAAudio-Language Models

    Institute of Acoustics, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · State Key Laboratory of Acoustics and Marine Information, Institute of Acoustics, Chinese Academy of Sciences, Beijing, China

  15. LiST: Local-Simplex Test-Time LoRA Fusion

    Aug 23, 2026Yihua Shao, Jia Li, Siyu Chen +12VLM AdaptationTest-Time Adaptation of VLMs

    PolyU · CASIA · GDUT +7

  16. C2^2Path: Class-Conditional Pathway Decoupling for Vision-Language Incremental Object Detection

    Aug 22, 2026Lecheng Xu, Feifei Shao, Ouyangzi Ye +5Continual Object DetectionObject Detection

    Zhejiang University · The Hong Kong University of Science and Technology · Zhejiang Tobacco Monopoly Administration

  17. CD-LoRA: Consistency-Driven Low-Rank Adaptation for Multi-Task Fine-Tuning

    Aug 22, 2026Qian Zha, Jinda Liu, Yuan Wu +1LLM Fine-TuningLow-Rank Adaptation

    School of Artificial Intelligence, Jilin University · Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China · International Center of Future Science, Jilin University

  18. FCPRAG: Fusion-Controller Parametric Retrieval-Augmented Generation for Stable Multi-Passage LoRA Injection

    Aug 22, 2026Jinchang Zhu, Jindong Li, Yi Ding +5Retrieval-Augmented GenerationRAG Robustness

    The Hong Kong University of Science and Technology (Guangzhou) · University of Macau · Institute of Artificial Intelligence (TeleAI), China Telecom

  19. LoRA-GA2^2: Low Rank Adaptation with Multi-step Gradient Adaptive Alignment

    Aug 20, 2026Haonan He, Xinyue FanLLM Fine-TuningLow-Rank Adaptation

    University of Science and Technology of China · Independent Researcher

  20. Frequency-Aware Continual Learning for Smart Contract Vulnerability Detection with Large Language Models

    Aug 20, 2026Tenghui Huang, Jiawen Kang, Dongning Liu +5Software Vulnerability DetectionContinual Learning for LLMs

    School of Automation, Guangdong University of Technology, Guangzhou 510006, China · School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China · College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China +4

  21. A Few Cases Are All You Need: An Empirical Study of Annotation-Efficient LoRA Fine-Tuning of MedSAM3

    Aug 19, 2026Sachin Dudda Nagaraju, Bendik Skarre Abrahamsen, Ashkan Moradi +1Few-Shot Medical Image Segmentation3D Medical Image Segmentation

    Department of Circulation and Medical Imaging, Norwegian University of Science and Technology, Trondheim, Norway · Central Staff, St. Olavs Hospital, Trondheim University Hospital, Trondheim, Norway

  22. Agents unlock new capabilities through Switching LoRA Adapters as a Tool (SLAaaT)

    Aug 17, 2026Kenneth GeLong-Horizon LLM AgentsLLM Agent Skill Learning

    Independent San Francisco, CA, United States of America

  23. Self-Routed Tensor Adapters for Parameter-Efficient Universal Visual Adaptation

    Aug 17, 2026Suraj YadavTucker DecompositionVision Foundation Model Adaptation

    Indraprastha Institute of Information Technology Delhi

  24. FedPA-LoRA: Product-Aligned Framework for Mitigating Aggregation and Initialization Errors in Heterogeneous Federated LoRA

    Aug 15, 2026Juseok Jeon, Ramy E. Ali, Doyun Kwon +3Federated Fine-TuningFederated Learning

    DGIST Daegu, Republic of Korea · Samsung San Diego, USA

  25. Spectral Rank Certification for Foundation Model Adapters

    Aug 15, 2026Mohammed Ahnouch, Lotfi ElaachakLow-Rank Adaptation

    Université Paris 1 Paris, France · Faculty of Science and Technology of Tangier Abdelmalek Essaadi University Tangier, Morocco

  26. PhaseLoRA: Control-Regime-Conditioned Low-Rank Adaptation for Continuous-Action Vision-Language-Action Policies

    Aug 15, 2026Yufei Guo, Yinan Wu, Haoran Duan +2Low-Rank AdaptationVision-Language-Action Models

    Tsinghua University

  27. Approximate Muon with low-rank adapters

    Aug 14, 2026Ben Anson, Conor Houghton, Edward MilsomMuon OptimizerParameter-Efficient Fine-Tuning

    University of Bristol School of Mathematics · University of Bristol School of Engineering Mathematics and Technology · University of Bath Department of Computer Science

  28. Market-Information-Aware Gated-LoRA of Foundation Models for Transferable Day-Ahead Electricity Price Forecasting

    Aug 11, 2026Hang Fan, Wei Wei, Shengwei MeiElectricity Price ForecastingTime Series Forecasting

  29. SeFoRA: Sketch-Aggregated Federated Low-Rank Adaptation with Heterogeneous Client Ranks

    Aug 10, 2026Yue Xia, Tayyebeh Jahani-Nezhad, Mayank Bakshi +1Federated Learning AggregationLow-Rank Adaptation

    Technical University of Munich Munich, 80333 Germany · Technische Universität Berlin Berlin, 10587 Germany · Northern Arizona University Flagstaff, AZ 86011 USA

  30. Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approach

    Aug 10, 2026Xinyi Xu, Bingnan Xiao, Shuang Qin +2LLM Fine-TuningLow-Rank Adaptation

    University of Electronic Science and Technology of China · Fudan University · Singapore University of Technology and Design