Protein Language Models

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 43

All topics

ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design
May 11, 2026Yulin Zhang, He Cao, Zihao Jiang +6Protein Language Models Multi-Teacher Knowledge Distillation
Tsinghua University · International Digital Economy Academy · Hong Kong University of2026 Science and Technology (Guangzhou) +1
Structural Interpretations of Protein Language Model Representations via Differentiable Graph Partitioning
May 9, 2026Siddhant Dutta, Edward Tan Beng Wai, Soumick Sarker +2Graph Neural Networks Protein Language Models
Nanyang Technological University, Singapore
ProteinJEPA: Latent prediction improves protein language model pretraining
May 8, 2026Dan Ofer, Dafna Shahaf, Michal LinialLanguage Model Pretraining Masked Language Modeling
Department of Biological Chemistry The Hebrew University of Jerusalem · School of Computer Science and Engineering The Hebrew University of Jerusalem
ProtSent: Protein Sentence Transformers
May 7, 2026Dan Ofer, Oriel Perets, Michal Linial +1Contrastive Learning Protein Language Models
Department of Computer and Information Science Ben-Gurion University of the Negev
Conditional generation of antibody sequences with classifier-guided germline-absorbing discrete diffusion
May 7, 2026Justin Sanders, Luca Giancardo, Lan Guo +4Diffusion Model Guidance Protein Language Models
Paul G. Allen School of Computer Science and Engineering, University of Washington · ∗Work performed during internship at Amazon Web Services · Life Sciences, Amazon Web Services
Retrieval and competition: how a protein foundation model starts a protein
May 5, 2026Piotr Jedryszek, Oliver M. CrookProtein Language Models Mechanistic Interpretability
Department of Biology, University of Oxford, Oxford, UK · Kavli Institute for Nanoscience Discovery, University of Oxford, Oxford, UK · Department of Chemistry, University of Oxford, Oxford, UK
Co-Generative De Novo Functional Protein Design
May 1, 2026Xinrui Chen, Yizhen Luo, Siqi Fan +1Protein Language Models Molecular Generation
Institute for AI Industry Research (AIR), Tsinghua University · Department of Computer Science and Technology, Tsinghua University · PharMolix Inc.
Towards A Generative Protein Evolution Machine with DPLM-Evo
Apr 30, 2026Xinyou Wang, Liang Hong, Jiasheng Ye +4Protein Language Models Molecular Generation
ByteDance Seed · Nanjing University · CUHK +1
Sampling at intermediate temperatures is optimal for training large language models in protein structure prediction
Mar 31, 2026L. Ghiringhelli, A. Zambon, G. TianaProtein Structure Prediction AI-Assisted Scientific Research
Department of Physics, via Celoria 16, 20133 Milano, Italy · Department of Physics and INFN, via Celoria 16, 20133 Milano, Italy
ProteinZero: Self-Improving Protein Generation via Online Reinforcement Learning
Jun 9, 2025Ziwen Wang, Jiajun Fan, Ruihan Guo +3Protein Language Models Molecular Generation
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Urbana, IL, USA · DOE Center for Advanced Bioenergy and Bioproducts Innovation, University of Illinois Urbana-Champaign, Urbana, IL, USA
Leveraging Natural Language Processing to Unravel the Mystery of Life: A Review of NLP Approaches in Genomics, Transcriptomics, and Proteomics
Jun 2, 2025Ella Rannon, David BursteinProtein Language Models Genomics
The Shmunis School of Biomedicine and Cancer Research, George S. Wise Faculty of Life Sciences, Tel-Aviv University, Tel-Aviv, Israel
PUMA: Learning a Mutation-Aware Vocabulary of Protein Units
Mar 11, 2025Burak Suyunu, Özdeniz Dolu, Ibukunoluwa Abigail Olaosebikan +2Protein Language Models Protein Representation Learning
Department of Computer Engineering, Boaziçi University, 34342, Bebek, Istanbul, Turkey · C. Eugene Bennett Department of Chemistry, West Virginia University, Morgantown, West Virginia 26505, United States · Department of Pharmaceutical Chemistry, Istanbul Medipol University, Faculty of Pharmacy, 34815 Beykoz, Istanbul, Turkey +1
When do cheap embeddings beat protein language models? A theoretically-grounded hashing sketch for biological sequence classification
Date pendingSarwan Ali, Taslim Murad, Imdadullah Khan +1Protein Language Models Randomized Sketching
Columbia University, NY, USA · Institute of Business Administration, Karachi, Pakistan · Lahore University of Management Sciences, Lahore, Pakistan +1

May 11, 2026, cs.LGJ/K move · Enter open · S save

ProteinOPD: Towards Effective and Efficient Preference Alignment for Protein Design

Yulin Zhang, He Cao, Zihao Jiang, Chenyi Zi+5

Tsinghua University · International Digital Economy Academy · Hong Kong University of2026 Science and Technology (Guangzhou)+1

Protein Language Models Multi-Teacher Knowledge Distillation Multi-Objective Language Model Alignment Molecular Generation On-Policy Distillation Preference Alignment+1

Open paper AskPDF arXiv

Designing proteins with desired functions or properties represents a core goal in synthetic biology and drug discovery. Recent advances in protein language models (PLMs) have enabled the generation of highly designable protein sequences, while preference alignment provides a promising way to steer designs toward desired functions and properties. Nevertheless, they often trigger catastrophic forgetting of pretrained knowledge, degrading basic designability and failing to balance multiple competing objectives. To address these issues, we draw inspiration from On-Policy Distillation (OPD), an advanced post-training method renowned for mitigating catastrophic forgetting through its mode-seeking nature. In this work, we propose ProteinOPD, a multi-objective preference alignment framework that can effectively balance multiple preference objectives while maintaining the inherent designability of PLMs. ProteinOPD adapts a pretrained PLM into preference-specific teachers and distills their knowledge into a shared student via token-level OPD on the student's own trajectories. During this process, the student is aligned to a unique normalized geometric consensus of weighted teachers while ensuring bounded optimization under conflicts. This bridges the gap for OPD in multi-objective/teacher alignment. Extensive experiments show that ProteinOPD achieves substantial gains on target preference objectives without compromising the designability, with an 8x training speedup over RL-based alignment competitors.

Momentum

Related topics

Latest papers 43