Low-Rank Adaptation

Also known as LoRA

Ask about this topic

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

542 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

  1. Contrastive Parameter Disentanglement for Multi-modal Remote Sensing Image Generation

    Jul 26, 2026Yu Zhang, Wenda Zhao, Haojun Tang +1T2I GenerationImage Generation

    Unit 92728 of PLA,Shanghai, 200436, China

  2. LARA: Lightweight Adapters in the Residual Stream for Composable Adaptation and Alignment

    Jul 25, 2026Pascal Ekin, Hyosun Choi, Wei JieLLM AlignmentLow-Rank Adaptation

    Royal Holloway, University of London, United Kingdom · University of West London, United Kingdom

  3. \k{appa}-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating

    Jul 24, 2026Jianghui Wang, Silong Yong, Francesco Orabona +3Fine-TuningMemory-Efficient Fine-Tuning

    King Abdullah University of Science and Technology · Carnegie Mellon University

  4. IFCLoRA: Topology-Aware Rank Allocation for Parameter-Efficient Fine-Tuning

    Jul 24, 2026Wei Zhang, Xinwu Liu, Yihang ChengLLM Fine-TuningLow-Rank Adaptation

    Faculty of Artificial Intelligence in Education Laboratory for Artificial Intelligence and New Forms of Education Central China Normal University Wuhan, Hubei, China

  5. MoE2^2-LoRA: When MoE Models Meet MoE-style Low-Rank Adaptation

    Jul 24, 2026Qingyu Yang, Haonan He, Minglei Li +4Mixture-of-Experts Language ModelsLow-Rank Adaptation

    Shanghai Artificial Intelligence Laboratory · KTH Royal Institute of Technology · University of Science and Technology of China +2

  6. On the Convergence of Stochastic Low-Rank Adaptation

    Jul 24, 2026Ru Wang, Chengchang Liu, John C. S. LuiNonconvex Stochastic OptimizationLow-Rank Adaptation

    The Chinese University of Hong Kong · Westlake University

  7. Data Quality over Capacity: Internalizing Documents into LoRA Adapters for Closed-Book QA

    Jul 23, 2026Joan Figuerola HurtadoKnowledge Augmentation for Language ModelsLLM Fine-Tuning

    Independent Researcher

  8. How Much Can a LoRA Adapter Memorize? Measuring Adapter Capacity in Bits

    Jul 23, 2026Kaizhen Tan, Heqing Du, Yang FengMemorization in Language ModelsPrivacy Leakage in Language Models

    Carnegie Mellon University, Pittsburgh, PA, USA · Columbia University, New York, NY, USA

  9. Encoding Invisible Causation for Bridge Diagnostic Agents: Triple-Guided Retrieval-Augmented Fine-Tuning with QLoRA

    Jul 23, 2026Takato YasunoCausal AttributionLow-Rank Adaptation

  10. Spectral Transformation for Layer-wise Global Rank Discovery in Federated LoRA for Vision Transformers

    Jul 23, 2026Hariharan Ramesh, Jyotikrishna DassFederated Learning AggregationCommunication-Efficient Distributed Training

    Department of Electrical and Computer Engineering University of Arizona, Tucson, AZ 85719

  11. Three-Pronged Spectral Control for Federated Parameter Efficient Fine Tuning

    Jul 23, 2026Shiva Raj Pokhrel, Dipsan Bhattarai, Anwar WalidFederated Learning AggregationNuclear Norm Minimization

    School of IT, Deakin University, Australia · Columbia University, USA

  12. The Blessing of Dimensionality: How Near-Orthogonality in High-Dimensional Spaces Explains Temporal Portability

    Jul 22, 2026Abigail Woodring, Adrian Chan, Rana Muhammad Shahroz Khan +3Continual Learning for LLMsLow-Rank Adaptation

    Department of Electrical and Computer Engineering NC State University, Raleigh, NC 27606, USA · Independent Researcher · Department of Computer Science University of North Carolina at Chapel Hill, Chapel Hill, NC 27599, USA

  13. Statistical Inference for Rank Allocation in Low-Rank Adaptation

    Jul 22, 2026Yihang Gao, Vincent Y. F. TanLLM Fine-TuningLow-Rank Adaptation

    Department of Mathematics, National University of Singapore · Department of Mathematics and Department of Electrical and Computer Engineering, National University of Singapore

  14. Selective State-Space Adaptation and Retrieval for Language Model Reasoning

    Jul 21, 2026Atahan Dokme, Larry HeckRetrieval-Augmented GenerationSelective SSMs

    AI Virtual Assistant (AVA) Lab Georgia Institute of Technology

  15. PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer

    Jul 20, 2026Nikhil Ghosh, Tetiana Parshakova, Robert M. GowerLLM Fine-TuningLow-Rank Adaptation

    Center for Computational Mathematics, Flatiron Institute, New York, NY

  16. DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation

    Jul 20, 2026Siobhan Reid, Zhixiang Chi, Li Gu +3Test-Time AdaptationFew-Shot Domain Adaptation

    Concordia University, Montreal QC H3G 1M8, Canada · University of Toronto, Toronto ON M5S 1A1, Canada · Mila – Quebec AI Institute, Montreal QC H2S 3H1, Canada

  17. Transferable Low-Rank Convolutional Bases for Onboarding Unseen Medical Imaging Modalities

    Jul 18, 2026Ranat Das Prangon, Istiaque Ahmed, Shajid Hasan Naim +2Multi-Source Domain AdaptationMedical Imaging

    Dept. of Chemical Engineering, Bangladesh University of Engineering and Technology (BUET), Dhaka, Bangladesh. · Graduate School of Informatics, Osaka Metropolitan University, Osaka, Japan. · Dept. of Mechanical Engineering, Chittagong University of Engineering and Technology (CUET), Chattogram, Bangladesh. +2

  18. DARA: Degradation-Aware Low-Rank Residual Adaptation with Original-to-Corrupted Distillation for Corruption-Robust Animal Re-Identification

    Jul 18, 2026Cynthia Xie, Talia XuAnimal Re-IdentificationImage Corruption Robustness

    The University of Auckland Auckland, New Zealand

  19. CT-Merging: Consensus Directions and Task-Level Scaling for LoRA Adapter Merging

    Jul 18, 2026Keumseo Ryum, Joonhyuk KangLow-Rank AdaptationModel Merging

    KAIST

  20. Model Merging for Medical LVLMs: A Benchmark and a Winner-Take-All Approach

    Jul 17, 2026Lichao Mou, Shilan Zhang, Chunlei Li +7Low-Rank AdaptationMedical VLMs

    MedAI Technology (Wuxi) Co. Ltd. · Wuhan University of Technology · Technical University of Munich +1

  21. Towards Human-like Physical Intelligence: LifelongVision-Language-Action Learning for Robotic Manipulation

    Jul 16, 2026Yao He, Gan Sun, Wenqi Liang +2Continual Robot LearningRobot Skill Learning

    South China University of Technology · University of Trento

  22. Dysco: Dynamic Subspace Boosting to Mitigate LoRA Interference in Federated Learning

    Jul 15, 2026Haobo Zhang, Jiankun Wang, Suraj Rajendran +5Subspace OptimizationFederated Learning Aggregation

    University of Michigan · Cornell University · University of Pennsylvania

  23. When Directional Accuracy Lies: A Base-Rate-Honest Benchmark for LoRA-Adapted TimesFM on Equity Forecasting

    Jul 14, 2026Taizhen CheungFinancial ForecastingTime Series Forecasting

  24. LoRA-Based Cascaded Multimodal Fusion for Action Recognition in Medical Training Environments

    Jul 13, 2026Divya Mereddy, Jeevan BeedareddyHuman Activity RecognitionLow-Rank Adaptation

    Vanderbilt University · Nashville, TN, USA · Quince +1

  25. CARE-LoRA: Compressed Activation REconstruction for Memory-Efficient LoRA

    Jul 11, 2026Gengyu Zhang, Haiyin Ran, Zhengbao He +4Fine-TuningMemory-Efficient Fine-Tuning

    Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University, Shanghai 200240, China

  26. On the Efficiency of LoRA Fine-Tuning for Vision-Language-Action Models in Industrial Robotic Manipulation

    Jul 11, 2026Finn Ferchau, Daniel Pommer, Cristian AxenieLow-Rank AdaptationVision-Language-Action Models

    Technische Hochschule Nürnberg Georg Simon Ohm, Nürnberg, Germany · Siemens AG, München, Germany · Fraunhofer Institute for Integrated Circuits (IIS), Erlangen, Germany

  27. TabLoRA: Parameter-Efficient Low-Rank Ensemble Learning for Large-Scale Tabular Data

    Jul 11, 2026Jiaqi Luo, Shixin XuEnsemble LearningLow-Rank Adaptation

    School of Mathematical Sciences, Soochow University, No.1 Shizi Street, Suzhou, 215006, Jiangsu Province, China · Digital innovation research center, Duke Kunshan University, No.8 Duke Avenue, Kunshan, 215000, Jiangsu Province, China

  28. Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning

    Jul 10, 2026Ivan Ilin, Philip Zmushko, Peter RichtárikMemory-Efficient Fine-TuningLow-Rank Adaptation

    KAUST

  29. Dive Into the Implicit Biases of Low-rank Vision-language Alignment

    Jul 9, 2026Mingjia Shi, Shuo Wang, Xiaobo Wang +7Vision-Language ModelsRepresentation Geometry in VLMs

    Mininglamp · Shenzhen University of Advanced Technology · National University of Singapore

  30. Distribution-Calibrated Candidate Selection for Personalized Text-to-Image Generation

    Jul 8, 2026Wenyan XuSubject-Driven Image GenerationImage Generation

    School of Computer Science, Guangdong University of Technology