Low-Rank Adaptation

Also known as LoRA

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

581 papers58 papers in the last four weeks, down 36% on the four weeks before. 0.4% of all new papers.

  1. DCM-SAM: Defect-Conditioned Mixture of LoRA Experts for NPU-Deployed AM Defect Segmentation

    Sep 30, 2026Md Mushfiqur Rahaman, Md Mahedi Hasan, Imtiaz Ahmed +1Low-Rank AdaptationMixture of Experts

  2. Doc2LoRA Provides Decodable Representations of Scientific Ideas

    Sep 29, 2026Chand Sahil Mansuri, Joel Zachariah, Sadamori KojakuText EmbeddingsLow-Rank Adaptation

  3. FedLAFP: Low-Rank Aggregation Meets Full-Rank Personalization in Federated Fine-Tuning

    Sep 29, 2026Mengjun Yi, Huaian Gu, Yinghao Ai +2Low-Rank AdaptationFederated Fine-Tuning

  4. Beyond Low-Rank Parameterization: Narrowing the Gap Between LoRA and Full Fine-Tuning via Gradient Decomposition

    Sep 29, 2026Yihao Ouyang, Shiwei Li, Haozhao Wang +5Low-Rank AdaptationParameter-Efficient Fine-Tuning

  5. When Can Prefixes Compile LoRA? Exact Resource-Capped Tests for Frozen Attention

    Sep 29, 2026Joyanta Jyoti Mondal, Ibne Farabi ShihabPrefix TuningLow-Rank Adaptation

  6. Dual-Mode Low-Rank Learner with Bridge-Prototype Ensemble for Vision-Language Class-Incremental Learning

    Sep 29, 2026Chiyuan He, Zihuan Qiu, Fanman Meng +5Prototype-Based LearningLow-Rank Adaptation

  7. Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It

    Sep 29, 2026Zehao Jin, Ruixuan Deng, Junran WangTransformerLLM Fine-Tuning

  8. G3^3-LoRA: Organizing Reward-Weighted Video Data with Gradient-Guided Grouped LoRA

    Sep 28, 2026Jia Song, Wenhow Li, Lichen Bai +2Video Diffusion ModelsMulti-Task Learning

  9. Style-Driven Data Synthesis and Degradation-Aware Enhancement for Ultrasound Image Restoration

    Sep 28, 2026Yu-Kai Wang, Chun-Xin Tan, Manh-Hung Nguyen +1Synthetic Data AugmentationLow-Rank Adaptation

  10. Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Models

    Sep 28, 2026Jingdi lei, Junxian Li, Di Zhang +3Efficient Multimodal InferenceMultimodal Large Language Models

  11. SPACE-LoRA: Allocating Activation-Subspace Protection for Continual Learning

    Sep 28, 2026Seunghyun Yoo, Kiseok Kim, Hyeontae Joo +2Continual LearningLow-Rank Adaptation

  12. Rotated Manifold Optimization for Low-Rank Adaptation

    Sep 28, 2026Yuhui Ding, Javier Zazo, James HensmanRiemannian OptimizationLow-Rank Adaptation

  13. Resource-Aware Parameter-Efficient Model Adaptation for Onboard High-Dimensional Data

    Sep 27, 2026Qiyang Zhang, Xinhao Li, Lei Shi +4Fine-TuningLow-Rank Adaptation

  14. CFLoRA: Federated Fine-tuning of LLMs with Complementary Factors for Error-free Aggregation

    Sep 27, 2026Yanan Ma, Qiyuan Chen, Zihan Fang +2Federated Fine-TuningFederated Learning

  15. Stabilizing the Dynamic Low-Rank Training

    Sep 26, 2026Zhonghan Xu, Ling Wang, Junhao Chen +2Low-Rank ApproximationNeural Network Training Dynamics

  16. What Should Federated LoRA Share? FedSAIL via Input-aware Subspace Alignment

    Sep 26, 2026Junye Du, Shuaida He, Long FengFederated LearningLow-Rank Adaptation

  17. Certification Frontiers for Gaussian LoRA: Independent Priors, Posterior Risk, and Prediction-Preserving Balancing

    Sep 26, 2026Joyanta Jyoti Mondal, Ibne Farabi ShihabBayesian LoRAPAC-Bayesian Generalization Bounds

  18. New LoRA Skills Should Read but Never Write

    Sep 25, 2026Zeyan Li, Panqi Yang, Qirong Guo +5Low-Rank AdaptationContinual Learning for LLMs

  19. JevSoup: System-One Routing for Training-Free LoRA Composition

    Sep 25, 2026Xiuying Wang, Jiahua Cheng, Shuotian Li +4LLM RoutingParameter-Free Mixture-of-Experts Routing

  20. Diffu-LoRA: A Novel Low-Rank Adaptation for Personalized Diffusion Models

    Sep 24, 2026Tianjing Li, Wei ZhuNonlinear LoRADiffusion Model Adaptation

  21. Automatic Rank Allocation for Low-Rank Adaptation in Large Language Models via lp Regularization

    Sep 24, 2026Zebang Xie, Chuanyang Zheng, Yik-Chung Wu +1Low-Rank AdaptationAdaptive Rank Allocation

  22. LoRango: It Takes Two LoRAs to Unlock Hidden Behaviors in Diffusion Models

    Sep 22, 2026Jin Wei, Rundong Li, Ruihao Yang +10Adversarial Attacks on Diffusion ModelsBackdoor Attacks

  23. Reasoning-Preserving Fine-Tuning of Post-RL LLMs with Null-Basis LoRA

    Sep 22, 2026Wenzhi Fang, Nicholas Tzou, Lazar Valkov +1LLM Fine-TuningLow-Rank Adaptation

  24. LoRA-generating hypernetworks for efficient on-device LLM generative personalization

    Sep 21, 2026Sean Augenstein, Li Ding, Jihwan Lee +2On-Device Language Model InferenceLow-Rank Adaptation

  25. Muon Can Outperform Dedicated Continual Learning Methods

    Sep 21, 2026Sebastian George Sincari, Bogdan Alexandru Gheorghe, Antonio BarbalauContinual LearningMuon Optimizer