Quantization-Aware LoRA
LoRA: Low-Rank Adaptation
Overview
Quantization-aware LoRA studies how to adapt pretrained models with small low-rank updates while storing or running their weights at reduced precision. Common approaches include QLoRA, which trains adapters over quantized weights, and methods that jointly choose layer-wise bit-widths and adapter ranks or compensate for quantization error. Researchers evaluate these methods on language, vision, and domain-specific tasks, measuring accuracy alongside memory use, speed, and deployment behavior. The central tension is that tighter memory and compute budgets can limit adaptation quality or capability retention, and gains on measures such as perplexity do not always translate to robust task performance.
Latest papers
20 papers0 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.