Low-Rank Adaptation

Also known as LoRA

Low-rank adaptation studies how to specialize pretrained models by representing weight changes as products of small matrices while keeping most backbone weights frozen. Methods choose where adapters are placed and how their rank is allocated, using approaches such as spectral or gradient-based initialization, adaptive rank selection, and quantized fine-tuning. For multiple tasks or users, adapter routing, expert mixtures, merging, and federated aggregation aim to share useful updates without interference. The central tension is how to gain enough capacity for new tasks while controlling memory and compute, preserving existing capabilities, and generalizing across data shifts; the best rank, placement, and composition strategy depend on the model and task.

Latest papers

542 papers92 papers in the last four weeks, up 59% on the four weeks before. 0.5% of all new papers.

Analyzing Quality-Latency-Resource Trade-offs in a Technical Documentation RAG Assistant Using LoRA Adaptation
May 27, 2026Evgenii Palnikov, Elizaveta GavrilovaLLM Inference Efficiency Retrieval-Augmented Generation
1HSE University
Adapting Automotive Aerodynamics Surrogates to New Vehicle Families via Transfer Learning
May 27, 2026Seunghwan Keum, Alok WareyAutomotive Aerodynamics PDE Surrogate Modeling
General Motors Research and Development
GRADE: Generalizable Reasoning-Aware Dialogue Evaluation for AI Tutors
May 27, 2026Parth Bhalerao, Jeromy Chang, David Chou +1Intelligent Tutoring Systems Low-Rank Adaptation
Santa Clara University
Energy-Structured Low-Rank Adaptation for Continual Learning
May 26, 2026Longhua Li, Lei Qi, Qi Tian +1Continual Learning Low-Rank Adaptation
School of Computer Science and Engineering, Southeast University, Nanjing, China · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China · Huawei Technologies, Shenzhen, China
L2Rec: Towards Dual-View Understanding of LLMs for Personalized Recommendation
May 26, 2026Pingjun Pan, Tingting Zhou, Peiyao Lu +3Mixture-of-Experts Language Models LLM Personalization
Netease Cloud Music Hangzhou, China
Rethinking VLM Representation for VLA Initialization
May 25, 2026Weifeng Lin, Siyuan Huang, Hao Li +5VLM Adaptation Low-Rank Adaptation
1CUHK · 4ACE Robotics · 3Peking University +1
RotMoLE: Enhancing Mixture of Low-Rank Experts through Rotational Gating Mechanism
May 25, 2026Mengyang Sun, Maochuan Dou, Tao Feng +5LLM Fine-Tuning Low-Rank Adaptation
Tsinghua University · Beijing Information Science and Technology University · National University of Singapore +2
CollectionLoRA: Collecting 50 Effects in 1 LoRA via Multi-Teacher On-Policy Distillation
May 25, 2026Fangtai Wu, Hailong Guo, Shijie Huang +7Diffusion Model Distillation Low-Rank Adaptation
Zhejiang University · Qwen Applications Business Group of Alibaba · Xi’an Jiaotong University
ReLoRA: Knowledge-Reusing Adaptation for Fast Rollout of Evolving LLM Services
May 23, 2026Yang Xu, Zihuai Xu, Hongli Xu +3LLM Serving LLM Fine-Tuning
Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China, 215123
PALoRA: Projection-Adaptive LoRA for Preserving Reasoning in Large Language Models
May 23, 2026Mustafa Hayri Bilgin, Mariam Barry, Albert Bifet +2Knowledge Editing Continual Learning for LLMs
IT Group, Research, BNP Paribas · LTCI, Télécom Paris, Institut Polytechnique de Paris · AI Institute, University of Waikato +1
Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning
May 22, 2026Yoshihiko Fujisawa, Yuma Ichikawa, Yudai Fujimoto +2LLM Quantization Memory-Efficient Fine-Tuning
Fujitsu Limited · Institute of Science Tokyo · RIKEN Center for AIP +1
SeqLoRA: Bilevel Orthogonal Adaptation for Continual Multi-Concept Generation
May 21, 2026Javad Parsa, Enis Simsar, Amir Joudaki +2Subject-Driven Image Generation Diffusion Model Fine-Tuning
ETH Zurich · Uppsala University, Sweden
HyLoVQA: Dynamic Hypernetwork-Generated Low-Rank Adaptation for Continual Visual Question Answering
May 21, 2026Yiran Wang, Chenyi Xiong, Ziyue Qin +3Visual Question Answering Vision-Language Models
School of Computer Science, Hubei University, Wuhan 430062, China · Hubei Key Laboratory of Big Data Intelligent Analysis and Application (Hubei University), Wuhan 430062, China · Key Laboratory of Intelligent Sensing System and Security (Hubei University), Ministry of Education, Wuhan 430062, China
Federated LoRA Fine-Tuning for LLMs via Collaborative Alignment
May 20, 2026Shuaida He, Liwen Chen, Long FengLLM Fine-Tuning Low-Rank Adaptation
School of Computing & Data Science, The University of Hong Kong
SMoA: Spectrum Modulation Adapter for Parameter-Efficient Fine-Tuning
May 20, 2026Yongkang Liu, Xing Li, Mengjie Zhao +7LLM Fine-Tuning Adapter Tuning
Northeastern University, China · Shandong University, China · CIS, LMU Munich, Germany +1
AR1-ZO: Topology-Aware Rank-1 Zeroth-Order Queries for High-Rank LoRA Fine-Tuning
May 19, 2026Ziye Chen, Hongbin Lin, Chenyu Zhang +3Memory-Efficient Fine-Tuning Zeroth-Order Optimization
Learning When to Adapt
May 18, 2026Ali Zindari, Xiaowen Jiang, Rotem Mulayoff +1LLM Fine-Tuning Low-Rank Adaptation
1Universität des Saarlandes · 2CISPA Helmholtz Center for Information Security
LoRA vs. Full Fine-Tuning: A Theoretical Perspective
May 18, 2026Ali Zindari, Rotem Mulayoff, Sebastian U. StichFine-Tuning Low-Rank Adaptation
Universität des Saarlandes · CISPA Helmholtz Center for Information Security
FIM-LoRA: Task-Informative Rank Allocation for LoRA via Calibration-Time Gradient-Variance Estimation
May 16, 2026Ramakrishnan SathyavageeswaranAdapter Tuning Low-Rank Adaptation
Intuit
GLT-PEFT: Gated Lie-Tucker Parameter-Efficient Fine-Tuning for Alzheimer's Disease Diagnosis with Hippocampal Segmentation Pretraining
May 16, 2026Guanghua He, Hancan Zhu, Gaohang Yu +1Medical Diagnosis Low-Rank Adaptation
Department of Mathematics, Hangzhou Dianzi University, Hangzhou, 310018, China · School of Mathematics, Physics and Information, Shaoxing2026 University, Shaoxing, 312000, China · Department of Mathematics, Zhejiang University of Science and Technology, Hangzhou, 310023, China
Strategic Over-Parameterization for Generalizable Low-Rank Adaptation
May 15, 2026Jing Gao, Zhong-Yi Lu, Pan Zhang +1LLM Fine-Tuning Low-Rank Adaptation
School of Fundamental Physics and Mathematical Sciences, Hangzhou Institute for Advanced Study, UCAS, Hangzhou 310024, China · School of Physical Sciences, University of Chinese Academy of Sciences, No. 19A Yuquan Road, Beijing 100049, China · CAS Key Laboratory of Theoretical Physics, Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing 100190, China +1
ACE-LoRA: Adaptive Orthogonal Decoupling for Continual Image Editing
May 14, 2026Yuehao Liu, Weijia Zhang, Xuanming Shang +4Continual Learning Diffusion Model Fine-Tuning
Shanghai Jiao Tong University · Nanjing University · VIVO
GPart: End-to-End Isometric Fine-Tuning via Global Parameter Partitioning
May 14, 2026Paolo Mandica, Michał Brzozowski, Zuzanna Dubanowska +1LLM Fine-Tuning Low-Rank Adaptation
Samsung AI Center, Warsaw, Poland · University of Warsaw, Poland
LoMETab: Beyond Rank-1 Ensembles for Tabular Deep Learning
May 14, 2026Changryeol Choi, Hyewon Park, Yujin Kwon +1Ensemble Learning Low-Rank Adaptation
CJ Logistics
MinT: Managed Infrastructure for Training and Serving Millions of LLMs
May 13, 2026Mind Lab, :, Song Cao +60LLM Serving Adapter Tuning
LoREnc: Low-Rank Encryption for Securing Foundation Models and LoRA Adapters
May 13, 2026Beomjin Ahn, Jungmin Kwon, Chanyong Jung +1Low-Rank Adaptation LLM Security
Samsung Research · Amazon Web Services · University of Michigan +1
Continual Fine-Tuning of Large Language Models via Program Memory
May 13, 2026Hung Le, Svetha VenkateshContinual Learning for LLMs Memory-Augmented Language Models
Deakin Applied AI Initiative, Deakin University, Australia
Low-Rank Adapters Initialization via Gradient Surgery for Continual Learning
May 12, 2026Joana Pasquali, Ramiro N. Barros, Arthur S. Bianchessi +7Gradient Interference Catastrophic Forgetting
MALTA, Machine Learning Theory and Applications Lab, PUCRS, Porto Alegre, Brazil · Kunumi Institute, Brazil
Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training
May 12, 2026Chengqian Zhang, Wei Zhu, Kyumin LeeLow-Rank Adaptation LLM Post-Training
Worcester Polytechnic Institute · University of Hong Kong
Not How Many, But Which: Parameter Placement in Low-Rank Adaptation
May 12, 2026Arijit Sehanobish, Charles LoveringAdapter Tuning Low-Rank Adaptation
Kensho Technologies

May 27, 2026J/K move · Enter open · S save

Analyzing Quality-Latency-Resource Trade-offs in a Technical Documentation RAG Assistant Using LoRA Adaptation

Evgenii Palnikov, Elizaveta Gavrilova

1HSE University

LLM Inference Efficiency Retrieval-Augmented Generation Question Answering Low-Rank Adaptation RAG Evaluation

Open paper AskPDF arXiv

We study quality-latency-resource trade-offs in a documentation-grounded retrieval-augmented generation (RAG) system that uses Low-Rank Adaptation (LoRA) of the generator. We build a manually verified benchmark of 5,144 question-answer pairs over the official Kubernetes documentation and combine it with a fixed hybrid-retrieval pipeline (BGE-M3 dense, BGE-M3 native sparse, Reciprocal Rank Fusion, cross-encoder reranking). Over this benchmark we ablate 20 LoRA configurations on Llama-3.2-3B-Instruct and Llama-3.1-8B-Instruct across rank and target-module choices, and evaluate each on token-level F1, LLM-judged groundedness and correctness (pass@4), inference latency, inference memory, and training cost, all reported with bootstrap 95% confidence intervals. Pareto analysis shows that LoRA adapters acting only on the q and v attention projections consistently dominate the front, while the 3B/8B choice mainly defines operating regime. A param-matched control comparison further indicates that the q/v advantage is structural rather than purely parametric. The benchmark, selected adapters, and code are available at https://github.com/EugPal/rag-lora-tradeoffs.