Analysis of Quantized and Efficiently Adapted Protein Language Models

Ilan Yaniv Zeisler; Sebastian Clancy; Pouriya Bayat; Saaim Raad; Ivan Kraskov; Matthew Xie; Vivian White; Spencer Perkins

cs.LGSep 30, 2026

Analysis of Quantized and Efficiently Adapted Protein Language Models

Authors: Ilan Yaniv Zeisler, Sebastian Clancy, Pouriya Bayat, Saaim Raad, Ivan Kraskov, Matthew Xie, Vivian White, Spencer Perkins, +3 more

Organizations: Leslie Dan Faculty of Pharmacy, University of Toronto, Toronto, ON, Canada. · Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada. · Department of Computer Science, University of Toronto, Toronto, Ontario, Canada. · Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, Ontario, Canada. · Department of Computing and Software, McMaster University, Hamilton, ON, Canada.

Abstract

Background: Protein language models (PLMs) are increasingly used for sequence generation and property prediction, but their size makes fine-tuning and deployment expensive. The effects of quantization and parameter efficient fine-tuning on performance, representations and generation remain insufficiently characterized. Results: We evaluated 4-bit quantization and low-rank adapter fine-tuning (QLoRA) across ESM-2, ESMC, ProtBERT, ProtT5, Ankh, Ankh3 and Profluent-E1. Across protein prediction tasks, many model-task pairs retained more than 90% of full fine-tuning performance. Peak GPU memory savings approached 90% for the largest models, although performance and efficiency varied by model, dataset and training configuration. QLoRA often preserved early-layer representations while inducing task-specific adaptations in middle and late layers, resembling full fine-tuning with smaller representational changes. Training speed and power effects were more varied. For unconditional generation with ProLLaMA, ProtGPT2, ProGen2, ProteinGLM and ESM3, 4-bit quantization largely preserved predicted structural and sequence-level properties, but token-level analysis revealed model-dependent shifts in autoregressive output distributions. Conclusion: QLoRA and 4-bit quantization reduce PLM computational requirements, particularly GPU memory usage. Our results support QLoRA as a first-pass strategy for memory limited adaptation, reserving full fine-tuning for challenging tasks, unstable architectures or low validation recovery. For generative PLMs, sequence-level and structural metrics should be complemented with distributional analysis, since downstream predictions alone may miss quantization-induced shifts. These approaches can broaden access to large-scale protein modelling while requiring model- and task-specific validation.

Figures & tables

Figure 1: Efficient use of pretrained protein language models for both property prediction and protein generation. a We evaluated PLMs using two different tasks. For fine-tuning (adaptation) to a specific dataset (left), the pretrained PLM backbone is quantized to 4-bit precision and augmented with trainable LoRA adapters, producing a QLoRA model. For protein generation (right), the PLM is quantized without added LoRA adapters. b To better understand the effects of quantization, we looked at several metrics for success, including performance (accuracy), training memory, power consumption, latency, and variability. Model performance was maintained, while memory and power improvements were observed. c The suggested practitioner workflow derived from benchmarking results. QLoRA is used as a first-pass strategy for memory-constrained supervised adaptation, with fallback to full fine-tuning for difficult tasks, unstable architectures or low-performance recovery. For quantized generation, aggregate sequence and structural metrics should be supplemented with distributional diagnostics to evaluate downstream use cases, falling back to the full precision model in some cases.

Figure 2: QLoRA fine-tuning performance ratios across models and tasks. Each value shows the relative performance (full model score / QLoRA model score) in large text, with standard deviation across three replicates ( $n=3$ seeds) shown below.

Figure 3: Relative QLoRA/full fine-tuning performance of ESM-2 models for different tasks by model size. Ratios are geometric means with 95% confidence interval error bars across three replicates ( $n=3$ seeds).

Figure 4: Layer-wise representational similarity between QLoRA and full fine-tuning, with higher mean CKA indicating higher similarity to the full model. a Mean CKA similarity averaged across datasets and three paired seeds for each model as a function of normalized layer depth. b Dataset-specific CKA profiles for ESM-2 8M and 3B, averaged across three paired seeds.

Figure 5: kNN task probe data averaged across six datasets. a Layer-wise task information by kNN probe scores across model depth for full fine-tuned models, averaged across three seeds. b Difference in probe scores between QLoRA and full fine-tuned models, averaged across three seed pairs. Red indicates QLoRA has lower task probe score, blue indicates full fine-tuning has lower task probe score.

Figure 6: Predicted properties of unconditionally generated sequences from full and quantized models. Profiles compare structural quality, structural similarity, secondary structure content and disorder for each model. Full lines indicate original models, dotted indicate quantized models.

Figure 7: Teacher-forced KL divergence between full and quantized autoregressive models. a Mean token-level divergence as a function of normalized sequence position and b full-model reference entropy. c Joint count density of tokens over per-token KL divergence and full-model reference entropy, showing where quantization-induced discrepancies are concentrated relative to the reference model’s confidence levels. Per-token KL is pooled over three generation seeds and three length bins per model. Lines: bin-wise mean $\pm$ 95% bootstrap CI (1,000 resamples); bins with $<$ 10 tokens omitted. Hexbins: all tokens, shared log-count scale, axes clipped at the 99.5th percentile.

Figure 8: Peak training memory allocation across models and batch sizes. Values show percent differences for QLoRA relative to full-fine-tuning, with negative values indicating reduced memory usage for QLoRA. Standard deviation across three replicates is listed in smaller text.

Figure 9: Training throughput and power draw across models and batch sizes. a Percent difference in average training steps per second and b percent difference in average GPU power draw for QLoRA relative to full fine-tuning. All values include standard deviation across three replicates.

Figure 10: Peak memory, inference latency and average power consumption for unconditional generation.

Use case	Recommended starting point	Validation check	Signals for caution
Supervised protein property prediction	Start with QLoRA (especially when GPU memory is limiting)	Compare QLoRA against full fine-tuning on representative validation subset (ratio and uncertainty)	QLoRA recovery below practical threshold, typically $<$ 90-95%*, large uncertainty introduced or latency degrades
Low signal or high variance tasks	Use QLoRA as initial screen, not as a default	Run multiple seeds and compare absolute performance and variance, not just QLoRA/full ratio	Baseline full fine-tuning performance is weak/highly variable
Hyperparameter sensitive models	Perform a small learning rate and batch size sweep before comparison	Check convergence stability across seeds	Results are unstable across random seeds, learning rates or batch sizes
Unconditional generation	Use 4-bit quantization for exploratory memory efficient generation	Compare sequence-level and structural metrics between full and quantized models	Token-level or entropy stratified KL shifts substantially; consider residue frequencies, length distribution, diversity metrics
Downstream experimental design	Consider quantization as a bonus, not as the only decision factor	Confirm quantized outputs preserve relevant design objectives	Generated candidates will be experimentally validated

Table 1: Suggested workflows for quantized PLM adaptation

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Workflow	Checklist item	Purpose/what to report
Supervised fine-tuning	Report pretrained model checkpoint and parameter scale	Enables comparison across models, parameter sizes
	Report quantization configuration	Bit width, quantization method, target modules, compute dtype
	Report LoRA configuration	Rank, alpha, dropout, target modules
	Report fine-tuning hyperparameters	Learning rate, optimizer, batch size, epoch count/early stopping criterion, random seeds
	Compare QLoRA with full fine-tuning when feasible	Use same train/validation/test split
	Report absolute performances	Avoid relying only on QLoRA/full performance ratio

Appendix

Table S1: Practical validation and reporting checklist for quantized PLM workflows.

Figure S1: Full fine-tuning and QLoRA performance across downstream tasks. Bars show absolute task performance, and points show QLoRA/full fine-tuning ratios with 95% confidence intervals.

Figure S2: Layer-wise representational similarity to the pretrained base model with a full fine-tuned models and b QLoRA fine-tuned models.

Figure S3: Layer-wise task information in pretrained and ESM-2 fine-tuned representations. a Mean kNN probe scores across normalized depth for pretrained base models and QLoRA fine-tuned models. b Dataset-specific kNN probe scores across normalized depth for ESM-2 8M and 3B after full fine-tuning and QLoRA fine-tuning.

Figure S4: Predicted properties of unconditionally generated sequences from full and quantized models stratified by sequence length bins. Bars compare full and quantized models across Foldseek, Boltz-2, secondary structure and disorder metrics.

Figure S5: Additional teacher-forced KL divergence analyses. a Cumulative distribution of per-token KL divergence for each autoregressive model. Dashed lines indicate 95% and 99% of accumulated tokens. b Mean per-token divergence by model and sequence-length bin. c Mean per-token divergence across normalized sequence position, stratified by model and sequence-length bin.

Fluorescence

Stability

Secondary Structure

Full

QLoRA

Full

QLoRA

Full

QLoRA

Model

ESM-2 8M

5E-05

ESM-2 35M

5E-05

ESM-2 150M

5E-05

5E-06

5E-05

ESM-2 650M

5E-05

5E-06

5E-05

Appendix

Table S2: Batch size and learning rate for full and QLoRA models fine-tuned on fluorescence, stability and secondary structure datasets. LoRA parameters were fixed with

r=8

\alpha=32

\mathrm{dropout}=0.05

. Runs were completed on NVIDIA GeForce RTX 4090s, RTX 5090s and a DGX Spark (GB10 Grace Blackwell) depending on model size and availability.

Explore similar work

ProjQ: Project-and-Quantize for Adapter-Aware LLM Compression
May 30, 2026Wenya Yu, Chao Zhang, Li Wang +2Large Language Model Quantization Post-Training Quantization
School of Mathematics and Statistics, Central South University, Changsha, China · Department of Computer Science, Central South University, Changsha, China · CRAN, Université de Lorraine, CNRS, Nancy, France +1
Loopy: Low-Bit Quantization Framework for Looped Language Models
Oct 4, 2026Zeyu LI, Yipu ZHANG, Jintao Chen +2Large Language Model Quantization
The Hong Kong University of Science and Technology · Duke Kunshan University
On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs
Sep 22, 2025Rongguang Ye, Ming Tang, Edith C. H. NgaiLarge Language Model Quantization Training-Side Stage-Aware Low-Rank Adaptation
Southern University of Science and Technology · The University of Hong Kong

May 30, 2026, cs.LGJ/K move · Enter open · S save

ProjQ: Project-and-Quantize for Adapter-Aware LLM Compression

Wenya Yu, Chao Zhang, Li Wang, Samson Lasaulce+1

School of Mathematics and Statistics, Central South University, Changsha, China · Department of Computer Science, Central South University, Changsha, China · CRAN, Université de Lorraine, CNRS, Nancy, France+1

Large Language Model Quantization Post-Training Quantization Large Language Model Compression Efficient Large Language Model Data Compression Methods Subspace

Open paper AskPDF arXiv

Post-Training Quantization (PTQ) and Low-Rank Adaptation (LoRA) constitute the standard pipeline for efficient Large Language Model (LLM) deployment. However, applying them sequentially poses a problem: PTQ often leaves behind random noise that is spread out (across the model's weights) in a way LoRA can't easily fix, meaning that LoRA ends up wasting its limited capacity trying to fix uncorrectable noise instead of improving task performance. In this paper, we propose \textbf{ProjQ}, a novel framework for constraining quantization noise to the low-rank manifold via orthogonal subspace projection. We derive an efficient alternating algorithm that shapes the quantization noise into a low-rank structure, effectively offloading dominant error components to the subsequent adapter while minimizing the residual error in the orthogonal "uncorrectable" subspace. Our theoretical analysis demonstrates that ProjQ preserves strictly greater model plasticity for downstream tasks compared to standard PTQ. Extensive experiments on LLaMA-2, Qwen2.5 and Qwen3 confirm that ProjQ consistently outperforms existing methods in both quantization error compensation and downstream task fine-tuning, achieving up to

2\times

lower evaluation loss for compensation and matching the performance of standard 4-bit baselines on language modeling tasks with only 3 bits. The code is available on https://github.com/yy9301/ProjQ .

Abstract

Explore similar work

ProjQ: Project-and-Quantize for Adapter-Aware LLM Compression

Loopy: Low-Bit Quantization Framework for Looped Language Models

On-the-Fly Adaptation to Quantization: Configuration-Aware LoRA for Efficient Fine-Tuning of Quantized LLMs