How to make the most of your masked language model for protein engineering

Calvin McCarter; Nick Bhattacharya; Sebastian W. Ober; Hunter Elliott

cs.LGMar 11, 2026

How to make the most of your masked language model for protein engineering

Authors: Calvin McCarter, Nick Bhattacharya, Sebastian W. Ober, Hunter Elliott

Organizations: BigHat Biosciences

Abstract

A plethora of protein language models have been released in recent years. Yet comparatively little work has addressed how to best sample from them to optimize desired biological properties. We fill this gap by proposing a flexible, effective sampling method for masked language models (MLMs), and by systematically evaluating models and methods both in silico and in vitro on actual antibody therapeutics campaigns. Firstly, we propose sampling with stochastic beam search, exploiting the fact that MLMs are remarkably efficient at evaluating the pseudo-perplexity of the entire 1-edit neighborhood of a sequence. Reframing generation in terms of entire-sequence evaluation enables flexible guidance with multiple optimization objectives. Secondly, we report results from our extensive in vitro head-to-head evaluation for the antibody engineering setting. This reveals that the choice of sampling method can have a substantial impact, motivating future research into this under-explored area.

Figures & tables

	Approximation		Approximation + Gumbel
Model	Pearson	Spearman	Pearson	Spearman
ESM2-650M	0.964	0.940	0.735	0.727
AbLang2	0.869	0.869	0.752	0.729
ProteinMPNN	0.944	0.934	0.721	0.724
AbMPNN	0.927	0.850	0.726	0.615

Table 1: Approximate-vs-true PLL correlation over all 190 one-edit CDR-H3 variants of Trastuzumab (

\tau=1

Figure 1: ESM2-650M on the Trastuzumab CDR-H3 in silico evaluation. Color groups MasonCNN -guided methods with their lighter-shaded strong-oracle counterparts; linestyle indicates methodology (solid: ours; dashed: previous). [Left:] Mean predicted p(binder) vs edits from the seed; error bars depict 95%-bootstrap CIs. [Middle:] OASis × p(binder) Pareto frontiers. Faded dots show the underlying generated variants. [Top Right:] Mean p(binder) at each OASis percentile bucket, pooling across all edits. [Bottom right:] Pairwise H3-diversity violins restricted to “high-quality” variants — those with p(binder) $>$ 0.5 AND OASis $>0.245$ , the Trastuzumab seed.

Panel of Figure 2 — Figure 2: In silico metrics of quality and diversity for generated sequences: (A) predicted probability of synthesizability, (B) predicted melting temperature, (C) inter-seed pairwise diversity, and (D) intra-seed pairwise diversity.

Figure 3: In vitro results. For error bars around the success rate, we show 95% CIs via binomial test inversion. We depict the seed’s binding affinity and yield as a star on the x-axis.

Figure 4: Additional developability-related in silico evaluations of the sequences in the in vitro experiment. For each method, observations are split by successful synthesizability and binding.

Figure 5: Additional analysis of mutational preferences of different methods for the in vitro experiment. or each method, observations are split by successful synthesizability and binding.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Antibody	Sapiens	AbLang2	ESM2-650M
c143	0.941	0.955	0.930
mab114	0.924	0.935	0.896
mab114-uca	0.946	0.734	0.928
medi8852	0.837	0.925	0.878
medi8852-uca	0.878	0.282	0.888
regn10987	0.910	0.933	0.942

Appendix

Table 2: Approximate-vs-true PLL Spearman correlation, one-edit full VH+VL variants, over the seven-antibody panel from Hie et al. [2024] .

Antibody	Sapiens-CDR	AbLang2-CDR	ESM2-CDR
c143	0.971	0.930	0.892
mab114	0.933	0.859	0.880
mab114-uca	0.946	0.855	0.946
medi8852	0.828	0.881	0.906
medi8852-uca	0.778	0.675	0.895
regn10987	0.887	0.927	0.919

Appendix

Table 3: Approximate-vs-true PLL Spearman correlation over the seven-antibody panel from Hie et al., scoring only Chothia-defined CDR positions of the one-edit variant (while still conditioning on the full VH+VL sequence in the forward pass).

Antibody	Sapiens	AbLang2	ESM2
c143	1.878	1.955	3.301
mab114	2.107	1.949	3.967
mab114-uca	1.256	1.256	2.862
medi8852	1.607	1.761	3.719
medi8852-uca	1.212	1.156	3.653
regn10987	1.407	1.447	2.838

Appendix

Table 4: Per-residue pseudo-perplexity of the full seed sequence, over the seven-antibody panel from Hie et al. AbLang2 uniquely assigns extremely low perplexity to the UCA seeds, revealing memorization.

Antibody	Sapiens	AbLang2	ESM2
c143	88.4	1123.0	124.0
mab114	70.6	1141.9	74.8
mab114-uca	175.8	6837.3	175.1
medi8852	119.3	1420.3	87.0
medi8852-uca	173.3	8668.2	94.4
regn10987	150.1	3189.8	191.5

Appendix

Table 5: Geometric-mean pseudo-perplexity ratio of the seed’s local one-edit neighborhood (neighbor residue vs. seed residue), over the seven-antibody panel from Hie et al. AbLang2’s UCA likelihood peaks are surrounded by sharp cliffs in the local landscape.

	Sapiens		AbLang2		ESM2
Antibody	Approx	True	Approx	True	Approx	True
c143	0.34	0.35	0.53	0.49	0.53	0.52
mab114	0.39	0.39	0.52	0.43	0.55	0.53
mab114-uca	0.08	0.08	0.33	-0.09	0.57	0.55
medi8852	0.24	0.19	0.45	0.30	0.52	0.51
medi8852-uca	0.11	-0.01	0.28	-0.35	0.54	0.52

Appendix

Table 6: Spearman correlation of each model’s approximate (

\approx

) and true PLL against the DASM germline-bias-free selection score, per antibody, over the seven-antibody panel from Hie et al. [2024] . On the UCA seeds where AbLang2’s approximate-vs-true correlation is weakest, DASM agrees more with the approximation than with the true PLL – and is negatively correlated with the AbLang2 true PLL, indicating the “true” landscape reflects germline bias rather than biological selection.

Approximation	Pos. $i$ sees at $i$	Pos. $i$ sees at $k$	Total cost
Exact PLL	MASK	$x^{\prime}_{k}$ (child)	$O(L^{4})$
Double-mask	MASK	MASK	$O(L^{4})$
Masked-marginal	MASK	$x_{k}$ (template)	$O(L^{3})$
No-masking (child)	$x^{\prime}_{i}$ (visible)	$x^{\prime}_{k}$ (child)	$O(L^{3})$
No-masking (template)	$x_{i}$ (visible)	$x_{k}$ (template)	$O(L^{2})$

Appendix

Table 7: Comparison of PLL approximations for scoring all single-substitution neighbors during one step of beam search. Position

k

is the substitution site; position

i\neq k

is any other position. Each forward pass has cost

O(L^{2})

Figure 6: AbLang2 on the Trastuzumab CDR-H3 in silico evaluation. Color groups MasonCNN -guided methods with their lighter-shaded strong-oracle counterparts; linestyle indicates methodology (solid: ours; dashed: previous). [Left:] Mean predicted p(binder) vs edits from the seed; error bars depict 95%-bootstrap CIs. [Middle:] OASis × p(binder) Pareto frontiers. Faded dots show the underlying generated variants. [Top Right:] Mean p(binder) at each OASis percentile bucket, pooling across all edits. [Bottom right:] Pairwise H3-diversity violins restricted to “high-quality” variants — those with p(binder) $>$ 0.5 AND OASis $>0.245$ , the Trastuzumab seed.

Panel of Figure 7 — Figure 7: ProteinMPNN on the Trastuzumab CDR-H3 in silico evaluation, with (A) and without (B) the bound antigen. Color groups MasonCNN -guided methods with their lighter-shaded strong-oracle counterparts; linestyle indicates methodology (solid: ours; dashed: previous). [Left:] Mean predicted p(binder) vs edits from the seed; error bars depict 95%-bootstrap CIs. [Middle:] OASis × p(binder) Pareto frontiers. Faded dots show the underlying generated variants. [Top Right:] Mean p(binder) at each OASis percentile bucket, pooling across all edits. [Bottom right:] Pairwise H3-diversity violins restricted to “high-quality” variants — those with p(binder) $>$ 0.5 AND OASis $>0.245$ , the Trastuzumab seed.

Panel of Figure 8 — Figure 8: AbMPNN on the Trastuzumab CDR-H3 in silico evaluation, with (A) and without (B) the bound antigen. Color groups MasonCNN -guided methods with their lighter-shaded strong-oracle counterparts; linestyle indicates methodology (solid: ours; dashed: previous). [Left:] Mean predicted p(binder) vs edits from the seed; error bars depict 95%-bootstrap CIs. [Middle:] OASis × p(binder) Pareto frontiers. Faded dots show the underlying generated variants. [Top Right:] Mean p(binder) at each OASis percentile bucket, pooling across all edits. [Bottom right:] Pairwise H3-diversity violins restricted to “high-quality” variants — those with p(binder) $>$ 0.5 AND OASis $>0.245$ , the Trastuzumab seed.

Configuration	Beam	Gibbs	Beam:Mason	Gibbs:Mason	Beam:Mason+OASis
ESM2-650M	0.98	0.44	0.92	0.49	0.98
AbLang2	0.26	0.40	0.99	0.49	0.64
ProteinMPNN (holo)	0.39	0.41	0.72	0.42	0.94
ProteinMPNN (apo)	0.22	0.35	0.51	0.43	0.98
AbMPNN (holo)	0.73	0.54	0.92	0.61	0.84
AbMPNN (apo)	0.57	0.40	0.88	0.47	0.78

Appendix

Table 8: Fraction of designs with oracle

p(\mathrm{binder})>0.5

E=6

edits, for the extended Trastuzumab models, both with (holo) and without (apo) the HER2 subdomain IV antigen.

Panel of Figure 9 — Figure 9: Results from sweeping beam size (A) and softmax temperature (B) on Trastuzumab experiment. Text annotations next to each marker denote the hyperparameter used for each experiment.

Figure 10: Wallclock time in seconds for Trastuzumab.

Figure 11: In silico predicted probability of synthesizability for AbLang2 generated sequences, as a function of number of edits.

Panel of Figure 12 — Figure 12: Additional in silico evaluation results for generated sequences: (A) OASis humanness percentile, (B) germline mutation bias, (C) isoelectric point, and (D) number of CDR mutations.

Explore similar work

Learning Task-Specific Antibody Representations via Function-Aware Masking
Sep 1, 2026Ayan Goel, Thomas A. Walton, Amirali AghazadehMasked Language Modeling Protein Language Models
School of Computer Science, Georgia Institute of Technology · School of Electrical and Computer Engineering, Georgia Institute of Technology
Unlocking Multimodal Protein Language Models at Inference Time
Aug 26, 2026Yi Zhou, Qipeng Wang, Yunqing Liu +3Inference-Time Search Protein Language Models
The Hong Kong Polytechnic University · The Hong Kong University of Science and Technology (Guangzhou)
Structure-Aware Masking for Protein Representation Learning
May 15, 2026Thomas Walton, Ayan Goel, Amirali AghazadehProtein Fitness Prediction Masked Language Modeling
Georgia Institute of Technology

Sep 1, 2026J/K move · Enter open · S save

Learning Task-Specific Antibody Representations via Function-Aware Masking

Ayan Goel, Thomas A. Walton, Amirali Aghazadeh

School of Computer Science, Georgia Institute of Technology · School of Electrical and Computer Engineering, Georgia Institute of Technology

Masked Language Modeling Protein Language Models Protein Representation Learning

Open paper AskPDF arXiv

Antibody-specific language models pretrained via masked language modeling (MLM) learn representations that are critical for downstream sequence design and property prediction tasks. Yet, the corruption process itself is rarely leveraged as a source of inductive bias during pretraining. While preferentially masking complementarity-determining regions (CDRs) improves binding-related predictions, antibodies possess diverse biological priors over a variety of functions. Herein, we introduce function-aware masking, a family of pretraining algorithms that align mask placement with specific functional priors (e.g., from IMGT annotations or structure predictions) to shape the learned representation space. We show that these specialist masking strategies significantly improve performance on their respective objectives, yielding up to a 14% gain on structure-related tasks and up to a 5.9x improvement on CDR-related tasks. To further improve performance across multiple functional axes, we develop hybrid masking strategies that integrate multiple priors, balancing reconstruction over binding, structural, and biophysical objectives. Our results demonstrate that informed mask placement provides a parameter-free mechanism for imposing functional inductive biases in antibody language model training.

Abstract

Explore similar work

Learning Task-Specific Antibody Representations via Function-Aware Masking

Unlocking Multimodal Protein Language Models at Inference Time

Structure-Aware Masking for Protein Representation Learning