Low-Rank Adaptation

Also known as LoRA

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

581 papers58 papers in the last four weeks, down 36% on the four weeks before. 0.4% of all new papers.

  1. Which and When to Admit: Gradient Admission for Data-Centric Small Language Model Finetuning

    Oct 6, 2026Hongyu Cao, Yanchi Liu, Kunpeng Liu +5Gradient InterferenceLow-Rank Adaptation

  2. SkillFormer: Skill-Decomposed Adaptation for Audio Language Models

    Oct 6, 2026Lee Seung-woo, Bowen Qi, Kim Min-jun +1Multi-Task LearningAudio Understanding

  3. RACER: Residual-Adaptive Closed-Loop Estimation for Sampling-Based Planning in Wheeled-Quadruped Racing

    Oct 5, 2026Yuxiang Liu, Marla Eisman, Lizhi Yang +2Residual Dynamics LearningSim-to-Real Transfer

  4. Conditional Rank Allocation for Taxonomy-Aware Medical Language Model Adaptation

    Oct 5, 2026Guangyuan Dong, Ziwei Hong, Xuehao Zhou +6Low-Rank AdaptationDomain Adaptation for LLMs

  5. Frozen Factor or Spectral Band? Disentangling Two Choices in Low-Rank LoRA

    Oct 5, 2026Adnan Slimane Ali, Ayoub Belfatmi, David Ngwe PouthLow-Rank AdaptationParameter-Efficient Fine-Tuning

  6. A Fine-Grained Analysis of the LoRA Fine-Tuning Landscape with Implications for Data Selection

    Oct 5, 2026Bowen Zhang, Changrui Fang, Xinsong Ma +2Low-Rank AdaptationTraining Data Selection

  7. Adaptive Utilization of Low-Rank Adaptation via Conditioned Gating

    Oct 5, 2026Guang Yang, Changhao Guan, Chao Huang +2Low-Rank AdaptationParameter-Efficient Fine-Tuning

  8. Robust Parameter-Efficient LLM Adaptation on Analog Hardware

    Oct 4, 2026Jindan Li, Zhaoxian Wu, Tianyi ChenCompute-in-MemoryLLM Fine-Tuning

  9. LoRA's Second Descent Extends Beyond Parameter Parity

    Oct 3, 2026Yueran MaDouble DescentLow-Rank Adaptation

  10. Continual Humanoid Motion Learning

    Oct 3, 2026Zhewen He, Hao Huang, Geeta Chandra Raju Bethala +4Continual Robot LearningHumanoid Motion Tracking

  11. PaLoRA: Paced Low-Rank Adaptation for Continual Learning

    Oct 3, 2026Yuxuan Li, Fanhu Zeng, Hao TangContinual LoRAContinual Learning

  12. Predicting Steering Vectors and Adapter Weights for Few-Shot Author-Style Transfer

    Oct 2, 2026Leonard Popp, Danni Liu, Supriti Sinhamahapatra +1Text Style TransferText Generation

  13. Does Every User Need a Private LoRA? Decoupling Personalization from Per-User Adaptation

    Oct 1, 2026Songyuan Sui, Srikanth Malla, Chiho Choi +1LLM PersonalizationPersonalized Language Models

  14. Learn the Directions, Normalize the Gains: Post-Training Normalization for LoRA

    Oct 1, 2026Zailong Tian, Yanzhe Chen, Zhuoheng Han +2Low-Rank Adaptation

  15. Learning to Predict Distributions over Weight Updates for Test-Time Adaptation

    Oct 1, 2026Azal Ahmad Khan, Keshav Ramji, Tahira Naseem +2Test-Time AdaptationTest-Time Scaling

  16. Task-Oriented Rank Adaptation for Continual Learning in Text Classification

    Oct 1, 2026Rey Sanchez Lopez, Eduardo Morales Manzanares, Hugo Jair EscalanteContinual LearningLow-Rank Adaptation

  17. FedFit: Federated Fine-Tuning of LLMs via Vector-Bank Parameterization and Quantization

    Oct 1, 2026Hang Zou, Chao Zhang, Yuzhi Yang +3Federated Learning AggregationLow-Rank Adaptation

  18. CLASP: Continual Low-rank Adapters for Spatially Placed Concepts from One Hypernetwork

    Oct 1, 2026Wojciech Gromski, Patryk Krukowski, Jan Miksa +2Subject-Driven Image GenerationT2I Generation

  19. Adapter Thickets: Splitting an RLVR Budget Beats Concentrating It

    Oct 1, 2026Jonathan Williams, Esin Tureci, Karthik R. NarasimhanRL for Language Model ReasoningLow-Rank Adaptation

  20. Backdoor Purification for LoRA-Tuned LLMs via Null-Space Projection

    Sep 30, 2026Jianwei Li, Jung-Eun KimLLM Backdoor AttacksBackdoor Defense in LLMs

  21. Score the Update, Not the Token: Descent-Aligned Routing for Combinatorial LoRA Experts

    Sep 30, 2026Priya Nair, Lukas Brenner, Maya Lindqvist +5Mixture-of-Experts Language ModelsParameter-Free Mixture-of-Experts Routing

  22. ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs

    Sep 30, 2026Hang Yin, Haozhe Wang, Yuhua Luo +3Continual Learning for LLMsLow-Rank Adaptation

  23. LARC: Low-Rank Adaptive Residual Connections for Learning in Frozen Models

    Sep 30, 2026Junyi Zou, Avrova DonzMeta-LearningResidual Learning

  24. Dynamic LoRA-Experts and Prototype-Ensemble Matching for Class-Incremental Learning

    Sep 30, 2026Hongwei Zhao, Rui Liu, Yansong LiuClass-Incremental LearningPrototype-Based Classification

  25. Decoupled and Distilled: Task-Adaptive LoRA-Teachers with Ensemble Knowledge Transfer for Few-Shot Class-Incremental Learning

    Sep 30, 2026Hongwei Zhao, Rui Liu, Yansong Liu +2Class-Incremental LearningLow-Rank Adaptation

  26. A Dynamical Theory of LoRA in Continual Learning

    Sep 30, 2026Théo Marchetta, Filippo Alessandroni, Alessandro Breccia +2Continual LearningLow-Rank Adaptation