cs.AINov 5, 2026

LoRazor: Trimming Adapter Execution for Efficient Multi-LoRA Serving

Authors: Xiang Li, Pengcheng Wang, Azam Ikram, Saurabh Bagchi

Organizations: Purdue University

Abstract

LoRA offers a parameter-efficient adaptation strategy, supporting multi-user workloads via customized adapters on a shared backbone. However, serving concurrent requests with different adapters introduces substantial computation and memory-management overhead. We ask the research question: Do all requests need their adapters throughout the whole generation process? Our motivational study reveals that employing LoRA adapters at every decoding step is unnecessary, releasing them early has limited impact on accuracy while substantially accelerating generation. Although skipping adapters early offers theoretical workload reduction, translating these computational savings into concrete throughput gains requires tailored GPU kernels. We present LoRazor, a multi-LoRA serving framework that combines early adapter release with active-row aware GPU kernels. LoRazor determines when to permanently release a request's adapter and continue decoding with the backbone alone. By dynamically resizing LoRA kernel launches as adapters are released, LoRazor trims idle GPU blocks. Across two backbones and four tasks, LoRazor achieves up to 3.02x request throughput and 2.53x output-token throughput over vLLM multi-LoRA, with a mean relative task-accuracy decrease of 2.89%.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. EpiLoRA: Fusing Many LoRA Adapters into Low-Precision GEMMs for Serving and RL

    Nov 2, 2026Hieu Tran, Hong Yumulti-lora inferenceLLM Serving

  2. LoRAForge: Co-Designing Post-Hoc Compression and Execution for Efficient Multi-LoRA Serving

    Oct 31, 2026Shenghui Li, Thiemo Voigtmulti-lora inferenceLLM Serving

  3. PLoRA: Efficient Concurrent LoRA Training for Large Language Models

    Aug 4, 2025Minghao Yan, Zhuang Wang, Zhen Jia +2LLM Fine-TuningEfficient Language Model Training