Low-Rank Adaptation

Also known as LoRA

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

579 papers58 papers in the last four weeks, down 36% on the four weeks before. 0.4% of all new papers.

  1. Unlocking the Potential of Vision Foundation Models for Multi-Task Dense Prediction via Mixture of Task Adapters

    Nov 2, 2026Yunzhi Zhuge, Xinzhuo Yu, Sitong Gong +3Multi-Task Dense PredictionVision Foundation Model Adaptation

  2. LoRA-Patch: Defending Against Backdoored LoRA Adapters via Directional Steering

    Nov 2, 2026Shengyan Huo, Peihai Jiang, Xixiang Lyu +1Backdoor Defense in LLMsLLM Backdoor Attacks

  3. LoRAForge: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-LoRA Serving

    Oct 31, 2026Shenghui Li, Thiemo Voigtmulti-lora inferenceLLM Serving

  4. ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning

    Oct 30, 2026Chenyu Zhou, Yuxuan Zeng, Qi Deng +2LLM Fine-TuningLow-Rank Adaptation

  5. AquaProbe: Efficient Adaptation of Frozen Vision Models for Underwater Images

    Oct 29, 2026Md Shihabul Islam Shovo, Nishi Kanta Paul, Trung Dong +1Vision Foundation Model AdaptationUnderwater Imaging

  6. Structure-retained low-rank adapters for weather synthesis

    Oct 28, 2026Shunxin Wang, Alexandros Stergiou, Luuk Spreeuwers +2Image GenerationSynthetic Data Augmentation

  7. SimplexLoRA: Dynamic Rank Updating via Alternating Minimization with Simplex Projection

    Oct 27, 2026Igor Shalygin, Grigorii Davydenko, Andrey Veprikov +4Low-Rank AdaptationFine-Tuning Language Models

  8. Hierarchical Low-Rank Reparameterization for Parameter-Efficient Fine-Tuning

    Oct 27, 2026Rong Huang, Yi Xiong, Yuan-Yuan Cheng +1LLM Fine-TuningParameter-Efficient Fine-Tuning

  9. LoRA on a Budget: Function-Aware Post-Hoc Compression for Adapter Fleets

    Oct 26, 2026Shenghui Li, Thiemo voigtLow-Rank CompressionAdaptive Rank Allocation

  10. CRLoMTL: Conflict-Resolution through Low-rank matrices for Multi-Task Learning

    Oct 26, 2026Ahmed Agiza, Tasneem Shaffee, Rahil Badkul +1Multi-Task LearningMulti-Task Optimization

  11. Improving JEV Entailment Reranking with Span Readouts and Selective Adaptation

    Oct 26, 2026Haocheng Mei, Yuchuan Tian, Ye Yuan +5LLM RerankingLearning to Rank

  12. Mechanistic Insights into LoRA: Layer Sparsity for Adaptive Fine-Tuning via Path Patching

    Oct 26, 2026Fei Zuo, Yizhou Huang, Kezhi WangLow-Rank AdaptationParameter-Efficient Fine-Tuning

  13. HARP-LoRA: Hierarchy-Conditioned Rank Projection for Low-Rank Adaptation in Scientific Table Understanding

    Oct 25, 2026Hossein Sholehrasa, Mohammadhossein Amini, Doina Caragea +8Low-Rank AdaptationTable QA

  14. In-Context Adapter Generation for Compositional Generalization in Large Audio Language Models

    Oct 25, 2026Yiwei Guo, Siqi Huang, Bohan Li +4Compositional GeneralizationAudio-Language Models

  15. Task Format as a Factor in Adaptation: A Controlled Comparison of LoRA and Full Fine-Tuning for Prompt-Safety Classification

    Oct 24, 2026John Alderete, Srirakshith Betageri, Jennifer Flynn +3Fine-Tuning Language ModelsLow-Rank Adaptation

  16. Singular Geometry of Low-Rank Adaptation

    Oct 23, 2026Kamillo Ferry, Anthea Monod, Keisuke YanoLow-Rank AdaptationSingular Learning Theory

  17. Mostly Reading: What Open-Book Scores Measure in Document-Level LoRA Adapters

    Oct 22, 2026Martino M. L. Pulici, Cuong Xuan Chu, Evgeny Kharlamov +1Low-Rank AdaptationLanguage Model Merging

  18. LoRA Superposition across Pretrained Models

    Oct 22, 2026Keigo Watanabe, Hiroshi Kera, Kazuhiko KawamotoLow-Rank AdaptationModel Merging

  19. Convergent Performance, Divergent Mechanisms in LoRA-Adapted Vision Foundation Models for Brain MRI

    Oct 20, 2026Cecilia Diana-Albelda, Fiona Kekwick, Raghav Mehta +6Vision Foundation Model AdaptationLow-Rank Adaptation

  20. Parameter-Efficient Adaptation of Time Series Foundation Models for Electrical Load Forecasting

    Oct 20, 2026Aliullah, Julian Oelhaf, Edouard Fouché +4Electricity Load ForecastingTime-Series Foundation Models

  21. Temporal Evidence-Parameter Co-Alignment for LLM Reasoning over Temporal Knowledge Graphs

    Oct 20, 2026Jiapu WangTemporal Reasoning in Language ModelsTemporal KGs

  22. LoRASpace: A Pool-Wide Shared Substrate for Static and Dynamic Multi-LoRA Composition

    Oct 20, 2026Thamirawaran Sathiyalogeswaran, Navindu De Silva, Chamikara Siriwardane +4Low-Rank AdaptationLLM Inference Acceleration

  23. WeAct: Dynamic Weight Switching as Agent Actions

    Oct 19, 2026Rui Shi, Xunjian Yin, Jiacheng Sang +2LLM Agent TrainingContinual Learning for LLM Agents

  24. Scaling Parameter-Efficient Fine-Tuning for Brain Encoding

    Oct 19, 2026Zhuofan Josh Ying, Guanglong Sun, Nolan Ho +6Neural EncodingParameter-Efficient Fine-Tuning

  25. An Empirical Study of Parameter-Efficient Fine-Tuning for Mixture-of-Experts LLMs

    Oct 19, 2026Minglei Li, Peng Ye, Haonan He +3Mixture-of-Experts Fine-TuningParameter-Efficient Fine-Tuning

  26. Residual-Guided Adaptive Tucker Rank Allocation for Parameter-Efficient MCI Conversion Prediction

    Oct 19, 2026Guanghua He, Hancan Zhu, Gaohang Yu +1Adaptive Rank AllocationParameter-Efficient Fine-Tuning

  27. PFedQFT: Personalized Federated Quantum-Inspired Fine-Tuning for Medical Large Language Models

    Oct 19, 2026Qihan Huang, Lingxia Zhang, Guye Zhang +2Federated Fine-TuningFederated Learning

  28. First-Order LoRA Training with Phone NPUs

    Oct 17, 2026Xu Zheng, Wei Cheng, Zhuomin Chen +4On-Device Language Model Fine-TuningOn-Device Training