Quantization-Aware LoRA

LoRA: Low-Rank Adaptation

Ask about this topic

Overview

Quantization-aware LoRA studies how to adapt pretrained models with small low-rank updates while storing or running their weights at reduced precision. Common approaches include QLoRA, which trains adapters over quantized weights, and methods that jointly choose layer-wise bit-widths and adapter ranks or compensate for quantization error. Researchers evaluate these methods on language, vision, and domain-specific tasks, measuring accuracy alongside memory use, speed, and deployment behavior. The central tension is that tighter memory and compute budgets can limit adaptation quality or capability retention, and gains on measures such as perplexity do not always translate to robust task performance.

Latest papers

CardsList

19 papers1 paper in the last four weeks, down 67% on the four weeks before. 0.0% of all new papers.

  1. Analysis of Quantized and Efficiently Adapted Protein Language Models

    Sep 30, 2026Ilan Yaniv Zeisler, Sebastian Clancy, Pouriya Bayat +8Protein Language ModelsQuantization-Aware LoRA

    Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada. · Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada. · Department of Computer Science, University of Toronto, Toronto, Ontario, Canada. +2

  2. Optimizing the Phi-2 Small Language Model for Real-time Chatbot Applications Using Parameter-Efficient Fine-Tuning (PEFT) with QLoRA Quantization

    Sep 27, 2026PhanTan Khanh Nguyen, Ashfaq Ali Shafin, Khandaker Mamun AhmedOn-Device Language Model InferenceSmall Language Models

    Beacom College of Computer and Cyber Sciences, Dakota State University · Mathematics and Computer Science, Augustana College

  3. Scale-QLoRA: Code-Invariant Adapter Merging for Native 4-bit Microscaling LLMs

    Sep 3, 2026Tung-Ling Li, Jiale Huang, Lee-Chi Wang +1Model MergingLLM Quantization

  4. Quality Recovery for Quantized KV Caches via Low-Rank Attention Adaptation

    Sep 2, 2026Seifeldin AbdellatifKV-Cache QuantizationQuantization-Aware LoRA

    College of Engineering, Al Ain University

  5. Learning New Facts with QLoRA: An Acquisition-Retention Frontier

    Aug 26, 2026Estelle Zheng, Sébastien Warichet, Emmanuel Helbert +1Factual Knowledge in Language ModelsLow-Rank Adaptation

    LORIA, CNRS, France · Alcatel-Lucent Enterprise, France

  6. Low-Rank Ternary Adaptation for Fine-Tuning Transformers

    Aug 25, 2026Alexandru-Dragos Manolache, Yunqiang Li, Jan van GemertParameter-Efficient Fine-TuningLow-Rank Adaptation

    Delft University of Technology, The Netherlands · Amazon Development Center

  7. AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning

    Aug 24, 2026Md Romyull IslamFine-TuningQuantization-Aware LoRA

    Kennesaw State University

  8. From Financial Sentiment Classification to Return Predictability: A QLoRA Benchmark of Large Language Models

    Aug 4, 2026Fusheng LuoLLM EvaluationQuantitative Finance

    Johns Hopkins University Baltimore, Maryland, USA

  9. The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs

    Jul 31, 2026Jiajia Tang, Sizhe Yuen, Francisco Gomez Medina +2Continual Learning for LLMsLow-Rank Adaptation

    The Alan Turing Institute, London, UK · King’s College London, London, UK · University of Southampton, Southampton, UK

  10. Encoding Invisible Causation for Bridge Diagnostic Agents: Triple-Guided Retrieval-Augmented Fine-Tuning with QLoRA

    Jul 23, 2026Takato YasunoCausal AttributionLow-Rank Adaptation

  11. TerraMARS: A Domain-Adapted Small-Language-Model Pipeline for Mars Terraforming Literature

    Jun 18, 2026Jyotsna Singh, Ash Black, Jeff Larsen +1Small Language ModelsQuantization-Aware LoRA

    College of Information Science, University of Arizona, Tucson, AZ, USA · Biosphere 2, University of Arizona, Tucson, AZ, USA · Department of Ecology and Evolutionary Biology, University of Arizona, Tucson, AZ, USA +1

  12. Small LLMs for Biomedical Claim Verification: Cost-Effective Fine-Tuning, Structural Dataset Shortcuts, and Cross-Domain Generalization

    Jun 11, 2026Gaurav KumarClaim VerificationLLM Fine-Tuning

    Moveworks AI · University of California San Diego

  13. LinguIUTics at PsyDefDetect: Iterative Imbalance-Aware Fine-tuning of Qwen3-8B for Psychological Defense Mechanism Classification

    May 30, 2026Shefayat E Shams Adib, Ahmed Alfey Sani, Md Hasibur Rahman Alif +1Class-Imbalanced LearningQuantization-Aware LoRA

    Department of Computer Science and Engineering, Islamic University of Technology, Dhaka, Bangladesh

  14. Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning

    May 22, 2026Yoshihiko Fujisawa, Yuma Ichikawa, Yudai Fujimoto +2LLM QuantizationMemory-Efficient Fine-Tuning

    Fujitsu Limited · Institute of Science Tokyo · RIKEN Center for AIP +1

  15. AdaLoRA-QAT: Adaptive Low-Rank and Quantization-Aware Segmentation

    Apr 1, 2026Prantik Deb, Srimanth Dhondy, N. Ramakrishna +3Medical Image SegmentationMedical Imaging Foundation Models

    International Institute of Information Technology (IIIT-H), Hyderabad, India · Nizam’s Institute of Medical Sciences (NIMS), Hyderabad, India · The Alan Turing Institute, London and University College London, United Kingdom

  16. Exploring parameter-efficient fine-tuning (PEFT) of billion-parameter vision models with QLoRA and DoRA: insights into generalization for limited-data image classification under a 98:1 test-to-train regime

    Mar 18, 2026Haiyu Yang, Sumit Sharma, Enhong Liu +1Precision Livestock FarmingParameter-Efficient Fine-Tuning

    Cornell University, College of Agriculture and Life Sciences, Ithaca, NY 14853

  17. SERQ: Saliency-Aware Low-Rank Error Reconstruction for LLM Quantization

    Mar 9, 2026Yeonsik Park, Hyeonseong Kim, Seungkyu ChoiLLM QuantizationPost-Training Quantization

    Kyung Hee University · Yonsei University

  18. AutoQRA: Joint Optimization of Mixed-Precision Quantization and Low-rank Adapters for Efficient LLM Fine-Tuning

    Feb 25, 2026Changhai Zhou, Shiyang Zhang, Yuhua Zhou +5LLM Fine-TuningQuantization-Aware LoRA

    Fudan University · Yale University · Zhejiang University +1

  19. On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs

    Sep 22, 2025Rongguang Ye, Ming Tang, Edith C. H. NgaiMixed-Precision QuantizationLLM Quantization

    Southern University of Science and Technology · The University of Hong Kong