ORBIT-FMIB: Tracking Order-Resolved Epistatic Information Through ESM-2

Maryam Rahimimovassagh; Ivan Garibay; Niloofar Yousefi

cs.LGSep 30, 2026

ORBIT-FMIB: Tracking Order-Resolved Epistatic Information Through ESM-2

Authors: Maryam Rahimimovassagh, Ivan Garibay, Niloofar Yousefi

Organizations: University of Central Florida

Abstract

Protein foundation models support mutation-effect and structural prediction, but predictive performance alone does not reveal which forms of biological interaction information remain accessible through model depth. We ask whether ESM-2 retains higher-order epistatic information as strongly as first- and second-order information across its representation hierarchy, introducing ORBIT-FMIB, a diagnostic framework combining Walsh-based interaction decomposition with subset-conditioned neural dependence estimation. The method is validated on synthetic landscapes with known interaction structure before being applied to the dense four-site GB1 fitness landscape using frozen ESM-2 representations. An initial production run suggested ESM-2 retains higher-order epistatic information less well than lower-order information ( $Δ_{\mathrm{HO-LO}}=-0.107$ ). An independent replication of the complete measurement grid, under matched GPU hardware and identical critic seeds, substantially reduced this contrast ( $Δ_{\mathrm{HO-LO}}=-0.017$ ), and its sign was unstable across otherwise-defensible evaluation-pairing choices applied to the same trained critics ( $-0.011$ to $+0.015$ ). We therefore do not currently have robust evidence that ESM-2 selectively loses higher-order epistatic information, nor that retention is equal across orders; the directional question remains open. The measurement protocol itself, including its documented removal of a positional-subset shortcut in pooled critics, remains validated and is unaffected by this finding. ORBIT-FMIB is offered as a diagnostic framework for probing interaction structure in protein foundation models; this study's own replication result illustrates why such probing requires adequately-powered reproducibility checks before its output is treated as a biological finding.

Figures & tables

Figure 1: Overview of the ORBIT-FMIB pipeline. The measurement framework is first validated on ORBIT -Synth, where interaction orders and imposed information-loss events are known, and is then applied to the four-site GB1 fitness landscape using frozen ESM-2 representations. Complete WT-anchored mutational cubes are decomposed using a common Walsh basis to obtain experimental epistatic coefficients

E_{C,S}

and representation-level interaction components

R_{\ell,C,S}

. Neural dependence

D_{\ell,S}

is estimated independently for each fixed positional subset across model depth, yielding layerwise trajectories and peak-to-final retention summaries.

Panel of Figure 2 — Figure 2: Order-resolved epistatic information dynamics across ESM-2 depth (production run). A, Order-level trajectories after normalization of each fixed subset to its own peak; shading summarizes across-subset variation ( $\pm 1$ SD) and is not a confidence interval. B, Peak-normalized trajectories for all 15 fixed positional subsets across L0–L30; stars mark each subset’s own peak layer. Neural dependence scores are comparative and are not interpreted as calibrated mutual information in nats.

$k$	Primary	Delete $41{=}L$	Replic., cyclic	Replic., joint-row	Replic., rand. perm.
1	0.960	0.940	0.933	0.951	0.912
2	0.971	0.955	0.963	0.953	0.967
3	0.910	0.848	0.908	0.961	0.960
4	0.808	0.679	0.955	0.922	0.948
$\Delta_{\mathrm{HO-LO}}$	-0.107	-0.184	$-0.017$	$-0.011$	$+0.015$

Table 1: Order-level retention: primary production run and whole-identity deletion sensitivity (single run), alongside independent replication under three evaluation pairings applied to the same replicated critics.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Scenario	Replicate mean $\Delta_{\text{HO}-\text{LO}}$ (SE)	$p$ -value	Old test floor
Flat, strong (gain $=1.0$ , all orders)	$-0.00088\pm 0.01455$	$0.949$	$\geq 1/3$
Flat, moderate (gain $=0.5$ , all orders)	$\phantom{-}0.00028\pm 0.00270$	$0.929$	$\geq 1/3$

Appendix

Table 2: Multi-replicate flat-scenario test, replacing the original 4-item exact permutation test (structural minimum

p=1/3

regardless of the data). 20 independent replicate instances per scenario, combined via a Monte Carlo permutation test over the product of each replicate’s own 24-outcome space (

n_{MC}=200,000

; resolution

\approx 5\times 10^{-6}

Quantity	Identity-strict split (original)	IID split (corrected)
$\tau$	$0.00677738$	$0.00763411$
Original statistic, $\Delta_{\text{HO}-\text{LO}}$	$-0.10702806$	$-0.10702806$
New statistic, pooled-subset gap (L30)	$0.23837734$	$0.23837734$
New statistic, equal-order gap (L30)	$0.2907419$	$0.2907419$
Layers significant ( $p<0.05$ ), of 31	$26$	$26$
Pooled/equal-order sign agreement	$30/30$	$30/30$

Appendix

Table 3: Downstream effect of correcting the null-derangement diagnostic’s cube partition from the legacy identity-strict split to the production IID split (seed 20260910). The floor threshold

\tau

(99th percentile of

|{\rm null\ score}|

) shifts modestly; every downstream statistic is unchanged.

Subset	$k$	Production	Cyclic (replic.)	Joint-row perm.	Random perm.
${39}$	1	0.994	0.899	0.935	0.934
${40}$	1	0.946	0.906	0.908	0.939
${39,40}$	2	0.959	0.987	0.885	0.958
${41}$	1	0.902	0.940	0.983	0.938
${39,41}$	2	0.966	0.896	0.956	0.959
${40,41}$	2	0.982	0.983	0.990	0.971

Appendix

Table 4: Per-subset peak-to-final retention: original production run versus independently replicated critics under three evaluation pairings.

k

is interaction order. Replication values are unfloored (the IID-split null-floor threshold has not been separately recomputed for the replicated critics).

Explore similar work

May 9, 2026cs.LG

Structural Interpretations of Protein Language Model Representations via Differentiable Graph Partitioning

Protein language models such as ESM-2 learn rich residue representations that achieve strong performance on protein function prediction, but their features remain difficult to interpret as structural

\&

evolutionary signals are encoded in dense latent spaces. We propose a plug-

\&

-play framework that projects ESM-2 representations onto protein contact graphs

\&

applies

\textbf{SoftBlobGIN}

, a lightweight Graph Isomorphism Network with differentiable Gumbel-softmax substructure pooling, to perform structure-aware message passing

\&

learn coarse functional substructures for downstream prediction tasks. Across enzyme classification, SoftBlobGIN achieves 92.8% accuracy

\&

0.898 macro-F1. Unlike post hoc analysis of protein language models alone, our method produces directly auditable structural explanations: GNNExplainer recovers biologically meaningful active-site residues, spatially localized functional clusters,

\&

catalytic contact patterns. On binding-site detection, SoftBlobGIN improves residue AUROC from

0.885

using an ESM-2 linear probe to

0.983

, indicating that these structural explanations are not recoverable from language-model features alone. Learned blob partitions provide an additional layer of interpretability by automatically grouping residues into functional substructures, with blobs containing annotated active-site residues showing

1.85\times

higher importance than other blobs (

ρ{=}0.339

p{=}0.009

), without any active-site supervision. Our framework requires no retraining of the language model, adds only

\sim

1.1M parameters,

\&

generalises across ProteinShake tasks, achieving

F_{\max}

0.733

on Gene Ontology prediction

\&

AUROC of

0.969

on binding-site detection. We position this as an interpretable structural companion to protein language models that makes their predictions more transparent

\&

auditable.

Siddhant Dutta, Edward Tan Beng Wai, Soumick Sarker +2

Nanyang Technological University, Singapore

Protein Structure Prediction Language Modeling Structured Context

Ask PDF

Sep 28, 2026cs.AI

A General Harness for Protein Foundation Model Fitness Prediction

Accurate fitness prediction is central to protein engineering and understanding sequence-function relationships. With advances in deep learning, protein foundation models (PFMs) have become widely used for this task. Recent analyses, however, show that these models share preferences reflecting their training corpora, while unreliable inputs can further distort fitness predictions. Family-specific evolutionary evidence and structural context can help address these limitations by providing complementary constraints on model scores, motivating VenusREM-Harness (VRH), a general, model-agnostic, training-free Retrieval-Enhanced Mutation harness. It fuses frozen model scores with multiple sequence alignment (MSA) evidence according to model uncertainty, then applies gated background correction and score shrinkage based on structural confidence and solvent exposure. Across 1,211 assays and 3.1 million measured variants from ProteinGym, VenusMutHub, and the newly curated viral benchmark VenusViroHub, all 71 configurations improve Spearman correlation on all 3 benchmarks by 0.073 on average, with broad gains across 5 metrics. Extended analyses relate retrieval gains to model-MSA preference differences, assess domain-level gains and immune-escape cases, and quantify computational speedups. Built with VRH, VenusREM2 is the first to rank highest in all function, taxon, MSA-depth, and mutation-depth categories, with a ProteinGym Average Spearman of 0.556, 0.038 above the prior best.

Yang Tan, Qijia Tian, Gangyu Sun +5

Shanghai Jiao Tong University · Shanghai Innovation Institute · Shanghai Artificial Intelligence Laboratory

Wireless Foundation Models Sirus Physical Activities Preference Alignment Learning

Ask PDF

Sep 30, 2026q-bio.BM

StabilityArc: Decoding Protein Sequence Embeddings into Generalizable Stability Landscapes

Every protein has a unique stability landscape, but the physical consequences of mutation are governed by recurring biochemical constraints. We test whether a shared decoder, trained on measurements from diverse proteins, can interpret these constraints in an unseen target, enabling cross-protein transfer for initial experimental round prescreening. We present StabilityArc , which maps frozen ESMC-600M residue representations through a shared RoPE transformer to an Lx20 matrix of substitution effects; a symmetric, contact-aware residual aids in predicting epistasis in simultaneous substitutions. In 66 strict leave-one-protein-out evaluations covering 134,794 ProteinGym variants, StabilityArc achieves 0.7134 Spearman correlation, exceeding the strongest zero-shot baseline, ProSST-2048 (0.6526), by 0.0608. We further explore the utility of this method by providing the score as a prior for Kermut, achieving Spearman correlation of 0.8280 across three supervised split schemes, improving on Kermut's reported 0.8167.

Aaron L. Feller, Andrew D. Ellington, Claus O. Wilke

The University of Texas at Austin Interdisciplinary Life Sciences · The University of Texas at Austin Integrative Biology

Protein Lattice Mutation Lateral String Stability Feature Alignment Optimization Landscape Physical Activities

Ask PDF