Low-Rank Adaptation

Also known as LoRA

Overview

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

CardsList

581 papers58 papers in the last four weeks, down 36% on the four weeks before. 0.4% of all new papers.

  1. Safety Kernel: Mechanistic Alignment for Cross-Lingual Defense in LLMs

    Oct 17, 2026Chengyu Xie, Fei Shen, Shuyi Miao +5LLM SafetyLanguage Model Robustness

  2. ASR Personalization in Pathological Voice: Accuracy, Efficiency, and Duration Effects

    Oct 17, 2026Giuseppe Giaimis, Sergei Katkov, Muhammad Azfar Yaqub +1Automatic Speech RecognitionPersonalized Speech Recognition

  3. How Finite-Rank Bottlenecks Shape the Low-Rank Adaptation Landscape

    Oct 16, 2026Long Nguyen-Chi, Quynh Nguyen, Thanh Nguyen-Cung +1Low-Rank AdaptationNonconvex Optimization

  4. Ori: Drift-Aware Streaming kk-PCA for Very Low-Rank LLM Training

    Oct 16, 2026Xinyu Luo, Jincheng Zhou, Mengbo Wang +5LLM TrainingLow-Rank Adaptation

  5. Don't Retrain, Remix: Adaptation of Web Agents using Mixture Hypernetworks

    Oct 15, 2026Md Farhan Ishmam, Minh Pham Dinh, Kenneth MarinoWeb AgentsHypernetworks

  6. MorphAct: Robot Policies Morphing with the Worlds They Act In

    Oct 14, 2026Ziyan Liu, Jiting Liu, Weihong Zhong +7Robot Policy AdaptationVLMs for Robotics

  7. Multimodal Medical Image Parsing with Task-Specific Low-Rank Adapters

    Oct 14, 2026Taha Sabir, Alberto Montero Molina, Thinh Quang Ly +3Medical Image AnalysisVLM Adaptation

  8. ReGRA: Residual-Gradient Adaptation for Complementary Low-Rank Fine-Tuning

    Oct 14, 2026Fanyao Li, Yuxi Liu, Xiaohao Liu +3Low-Rank AdaptationParameter-Efficient Fine-Tuning

  9. How Many Ranks Does a Token Need? A Noise-Floor Squelch for Rank-One LoRA Experts

    Oct 13, 2026Mei Okonkwo, Owen Castellanos, Ingrid Solberg +1Low-Rank AdaptationAdaptive Rank Allocation

  10. Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models

    Oct 13, 2026Yupeng Qiu, Han Fang, Dhananjai Sharma +1T2I Diffusion ModelsDiffusion Models

  11. Frequency Routing and Identity Conditioning for Thermal-to-Visible Face Translation

    Oct 13, 2026Haojie Li, Xinlong Sun, Xiaohaoyang Lei +4Image-to-Image TranslationFrequency-Domain Feature Learning

  12. DT-LoRA: Reaching Beyond the Tangent Space of Low-Rank Adaptation

    Oct 13, 2026Yangming Zhang, Ling Yi, Datong Li +4Low-Rank AdaptationParameter-Efficient Fine-Tuning

  13. WIRE: Weight Space Implicit Representation Encoding of Frozen Video LLMs

    Oct 12, 2026Ameen Ali, Tamim Zoabi, Lior Wolf +1Efficient VLM InferenceVideo-Language Models

  14. Understanding Personalization Gains in Whisper ASR

    Oct 12, 2026Sergei Katkov, Antonio LiottaPersonalized Speech RecognitionAutomatic Speech Recognition

  15. Making LoRA Identifiable: Orthogonal Alignment for Continual Learning

    Oct 12, 2026Pratik Rakesh Singh, Mohammadi Zaki, Akash Saha +2Orthogonal Procrustes AlignmentContinual LoRA

  16. PGSB: Pretrained-Guided Shared Basis for LoRA Model Merging

    Oct 10, 2026Muqing Liu, Chongjie Si, Zhuoya Liu +1Language Model MergingLow-Rank Adaptation

  17. Monarch-MoE: Structured Experts for Multimodal Continual Instruction Tuning

    Oct 9, 2026Xi Zhang, Boyu Diao, Zesen Chen +3Continual Learning for VLMsMultimodal Instruction Tuning

  18. Static Gradient Attribution Underperforms a Density-Matched Random Mask on Loss-Based Forgetting Within LoRA B-Matrix

    Oct 8, 2026Yu Sun, Junwei Zhou, Zuodong Xiang +5Storage-Efficient Continual LearningContinual Learning for LLMs

  19. A Categorical Atlas of Parameter-Efficient Fine-Tuning: Cones, Orbits, and Gauges

    Oct 8, 2026Xiaoyu Li, Zhizhou Sha, Chiwun Yang +1Parameter-Efficient Fine-TuningLow-Rank Adaptation

  20. Where to Adapt Matters: Layer-Selective Fine-Tuning for Capability Retention

    Oct 8, 2026Zhiqiang Pang, Zihong Sun, Qi Xie +3LLM Fine-TuningSelective Fine-Tuning

  21. LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs

    Oct 8, 2026Xiaobing Yu, Peijie Qiu, Jin Yang +5Language Model EditingMemory-Efficient Fine-Tuning

  22. Are Parameter-Efficient Fine-tuning Methods Really Different?

    Oct 6, 2026Yikuan Li, Pinyan Lu, Fanghui LiuParameter-Efficient Fine-TuningLow-Rank Adaptation

  23. MemFLoRA: Memory-Floor LoRA for CNN Adaptation at the Edge

    Oct 6, 2026Mehmet Emre Akbulut, Johannes Geier, Ulf SchlichtmannOn-Device TrainingMemory-Efficient Fine-Tuning

  24. CoDe-LoRA: Mitigating the Orthogonality Dilemma in Continual Learning of LLMs via Knowledge Consolidation and Decoupling

    Oct 6, 2026Maoqi Liu, Quan Fang, Yufei HeContinual Learning for LLMsLow-Rank Adaptation

  25. A Riemannian Geometry for Low-rank Adaptation

    Oct 6, 2026Shoichiro Takeda, Shin'ya Yamaguchi, Satoshi Suzuki +1Riemannian OptimizationLow-Rank Adaptation