HyPLM Technical Report
Abstract
Protein language models (PLMs) are standard tools for variant-effect prediction, protein annotation and design, but the compute needed to train and serve them keeps growing. We present HyPLM-Flash, an efficient and open PLM trained by Hyra 2.0, the research agent developed by our team. HyPLM-Flash is a 650-million-parameter masked language model trained on 726 billion tokens, 0.73 times the data of ESM-2 650M and a fifth to an eighth of the data of E1-600M and ESM C 600M. It runs on an inference engine with fused kernels and INT8 matrix multiplication, embedding about 130,000 residues per second on a single NVIDIA H20 GPU, 1.9 to 2.7 times faster than other 600M-class encoders. In a matched evaluation against 28 public checkpoints on 32 tasks, HyPLM-Flash reaches a ProteinGym zero-shot Spearman correlation of 0.409, statistically tied with ESM-2 650M, ESM C 600M and E1-600M, and the highest organismal-fitness score among self-supervised single-sequence models. It gives the best long-range contact precision of all models evaluated, including ESM-2 15B and ESM C 6B, and a lower masked-residue loss than ESM-2 650M, ESM C 600M and E1-600M on held-out sequences and on recently released PDB chains. Matching state-of-the-art accuracy at a fraction of the training and inference cost, HyPLM-Flash sets a new Pareto frontier for protein language models. We have released the model weights and the inference engine at https://github.com/Tencent-Hunyuan/hyra-results/tree/main.
Figures & tables
| Training FLOPs | |||||
| Model | Params | Tokens | 6ND | Published | Corpus |
| Encoders of similar size | |||||
| ESM-2 650M | 652M | 1.0T | a | UniRef50/90 (2021_04) | |
| ESM-1b | 652M | 66B | — | UniRef50 (2018_03) | |
| ESM C 600M | 575M | 6.2T | UniRef, MGnify, JGI (70% clusters) | ||
| E1-600M | 641M | 4.0T | — | Profluent Protein Atlas, UniRef | |
| Held-out UniRef proteins | Recent PDB chains | |||
| Model | NLL | PPL | NLL | PPL |
| Encoders of similar size | ||||
| ESM-2 650M | 1.820 | 6.17 | 1.591 | 4.91 |
| ESM C 600M | 1.720 | 5.59 | 1.478 | 4.39 |
| E1-600M | 1.811 | 6.11 | 1.560 | 4.76 |
| AMPLIFY-350M | 1.743 | 5.72 | 1.155 | 3.17 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Task | Metric | Train / valid / test | Source |
| Zero-shot | |||
| ProteinGym substitutions | Spearman | 217 assays | ProteinGym v1.3 [ Notin et al., 2023 ] |
| Validation set | NLL | – / – / 479 | held-out UniRef split |
| PDB time split | NLL | – / – / 887 | RCSB PDB entries released 9 Sep to 7 Oct 2026 |
| Structure | |||
| Secondary structure (Q3) | accuracy | 8,678 / 2,170 / 513 | TAPE [ Rao et al., 2019 ] ; CB513, TS115, CASP12 |
Protein Language Models
Protein language models learn patterns in amino-acid sequences to predict protein properties, interpret mutations, and design new proteins.
See all 74 papers