cs.AINov 6, 2026

Generative Latent Priors for Interpreting and Steering Protein Language Models

Authors: Daniel Hinjos, Lucía Urcelay

Organizations: Centre for Genomic Regulation

Abstract

Dominant methods for interpreting large models impose strong structural assumptions. Generative Latent Priors (GLP) are a complementary approach: a diffusion model that learns the distribution of a frozen model's activations directly, imposing no assumptions. We port GLP to ESM-2 activations (UniRef50) and evaluate it at two host scales (ESM-2-8M and -650M), both as a lens for protein representations and as a prior for editing them. Generation quality scales smoothly with depth and compute. As a lens, single GLP "meta-neurons" are the strongest probes of discrete Swiss-Prot concepts at both hosts, beating InterPLM's released SAE features, and lead on continuous properties at 8M though not at 650M; falsification tests show this is mostly concentration, not new information. As an editing prior, the on-manifold correction makes DiffMean steering of three biochemical properties effective and restores on-manifoldness, but only in isolation: re-embedding the sequences actually produced after several rounds of decoding, the structural plausibility is compromised. This is mainly reproduced in additional SAE-based clamping structural steering experiments, where only Helix motif succeeds. The naturalness failure is traced to the rewriting coverage and the decoder rather than the prior, highlighting what the method must clear before it can serve for both discovery-by-editing and discovery-by-exploration. Trained GLPs are released on huggingface.co/collections/HPAI-BSC/glprot.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Learning Latent Protein Languages for Autoregressive Generation

    Oct 2, 2026Mahdi Pourmirzaei, Farzaneh Esmaili, Amir Ziashahabi +2Protein Structure GenerationProtein Language Models

  2. Off-Manifold Collapse in Guided Protein Language Models

    Aug 19, 2026Shuibai Zhang, Xinchi Liu, Fred Zhangzhi Peng +4Protein Language ModelsLarge Language Model-Guided Optimization

  3. Structural Interpretations of Protein Language Model Representations via Differentiable Graph Partitioning

    May 9, 2026Siddhant Dutta, Edward Tan Beng Wai, Soumick Sarker +2Graph Neural NetworksProtein Language Models