Low-Rank Adaptation

Also known as LoRA

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

581 papers58 papers in the last four weeks, down 36% on the four weeks before. 0.4% of all new papers.

  1. Re-examining low-rank adaptation for private LLM fine-tuning

    Oct 1, 2025Ali Dadsetan, Frank RudziczPrivacy-Preserving Language ModelsLow-Rank Adaptation

  2. On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs

    Sep 22, 2025Rongguang Ye, Ming Tang, Edith C. H. NgaiMixed-Precision QuantizationLLM Quantization

  3. LoRAtorio: An intrinsic approach to LoRA Skill Composition

    Aug 15, 2025Niki Foteinopoulou, Ignas Budvytis, Stephan LiwickiT2I Diffusion ModelsLow-Rank Adaptation

  4. From Isolation to Alignment: Unified LoRA for Efficient Multi-Task Learning

    Aug 7, 2025Jinda Liu, Yi Chang, Yuan WuMulti-Task LearningLow-Rank Adaptation

  5. PLoRA: Efficient Concurrent LoRA Training for Large Language Models

    Aug 4, 2025Minghao Yan, Zhuang Wang, Zhen Jia +2LLM Fine-TuningEfficient Language Model Training

  6. WaRA: Wavelet Low-Rank Adaptation for Medical Image Classification

    Jun 25, 2025Moein Heidari, Yijin Huang, Yasamin Medghalchi +3Vision Foundation Model AdaptationLow-Rank Adaptation

  7. TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree

    Jun 12, 2025Yu-Yang Qian, Yuan-Ze Xu, Zhen-Yu Zhang +2Continual Learning for LLMsLow-Rank Adaptation

  8. COLORA: Efficient Fine-Tuning for Convolutional Models with a Study Case on Optical Coherence Tomography Image Classification

    May 23, 2025Mariano Rivera, Angello HoyosLow-Rank AdaptationConvolutional Neural Networks

  9. C-LoRA: Continual Low-Rank Adaptation for Pre-trained Visual Models

    Feb 25, 2025Xin Zhang, Liang Bai, Xian YangContinual LoRALow-Rank Adaptation

  10. A LoRA is Worth a Thousand Pictures

    Dec 16, 2024Chenxi Liu, Towaki Takikawa, Alec JacobsonWeight-Space AnalysisLow-Rank Adaptation

  11. BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models

    Aug 8, 2024Yupeng Chang, Yi Chang, Yuan WuSocial Bias in Language ModelsLow-Rank Adaptation