ORBIT-FMIB: Tracking Order-Resolved Epistatic Information Through ESM-2

Maryam Rahimimovassagh; Ivan Garibay; Niloofar Yousefi

cs.LGSep 30, 2026

ORBIT-FMIB: Tracking Order-Resolved Epistatic Information Through ESM-2

Authors: Maryam Rahimimovassagh, Ivan Garibay, Niloofar Yousefi

Organizations: University of Central Florida

Abstract

Protein foundation models support mutation-effect and structural prediction, but predictive performance alone does not reveal which forms of biological interaction information remain accessible through model depth. We ask whether ESM-2 retains higher-order epistatic information as strongly as first- and second-order information across its representation hierarchy, introducing ORBIT-FMIB, a diagnostic framework combining Walsh-based interaction decomposition with subset-conditioned neural dependence estimation. The method is validated on synthetic landscapes with known interaction structure before being applied to the dense four-site GB1 fitness landscape using frozen ESM-2 representations. An initial production run suggested ESM-2 retains higher-order epistatic information less well than lower-order information ( $Δ_{\mathrm{HO-LO}}=-0.107$ ). An independent replication of the complete measurement grid, under matched GPU hardware and identical critic seeds, substantially reduced this contrast ( $Δ_{\mathrm{HO-LO}}=-0.017$ ), and its sign was unstable across otherwise-defensible evaluation-pairing choices applied to the same trained critics ( $-0.011$ to $+0.015$ ). We therefore do not currently have robust evidence that ESM-2 selectively loses higher-order epistatic information, nor that retention is equal across orders; the directional question remains open. The measurement protocol itself, including its documented removal of a positional-subset shortcut in pooled critics, remains validated and is unaffected by this finding. ORBIT-FMIB is offered as a diagnostic framework for probing interaction structure in protein foundation models; this study's own replication result illustrates why such probing requires adequately-powered reproducibility checks before its output is treated as a biological finding.

Figures & tables

Figure 1: Overview of the ORBIT-FMIB pipeline. The measurement framework is first validated on ORBIT -Synth, where interaction orders and imposed information-loss events are known, and is then applied to the four-site GB1 fitness landscape using frozen ESM-2 representations. Complete WT-anchored mutational cubes are decomposed using a common Walsh basis to obtain experimental epistatic coefficients

E_{C,S}

and representation-level interaction components

R_{\ell,C,S}

. Neural dependence

D_{\ell,S}

is estimated independently for each fixed positional subset across model depth, yielding layerwise trajectories and peak-to-final retention summaries.

Panel of Figure 2 — Figure 2: Order-resolved epistatic information dynamics across ESM-2 depth (production run). A, Order-level trajectories after normalization of each fixed subset to its own peak; shading summarizes across-subset variation ( $\pm 1$ SD) and is not a confidence interval. B, Peak-normalized trajectories for all 15 fixed positional subsets across L0–L30; stars mark each subset’s own peak layer. Neural dependence scores are comparative and are not interpreted as calibrated mutual information in nats.

$k$	Primary	Delete $41{=}L$	Replic., cyclic	Replic., joint-row	Replic., rand. perm.
1	0.960	0.940	0.933	0.951	0.912
2	0.971	0.955	0.963	0.953	0.967
3	0.910	0.848	0.908	0.961	0.960
4	0.808	0.679	0.955	0.922	0.948
$\Delta_{\mathrm{HO-LO}}$	-0.107	-0.184	$-0.017$	$-0.011$	$+0.015$

Table 1: Order-level retention: primary production run and whole-identity deletion sensitivity (single run), alongside independent replication under three evaluation pairings applied to the same replicated critics.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Scenario	Replicate mean $\Delta_{\text{HO}-\text{LO}}$ (SE)	$p$ -value	Old test floor
Flat, strong (gain $=1.0$ , all orders)	$-0.00088\pm 0.01455$	$0.949$	$\geq 1/3$
Flat, moderate (gain $=0.5$ , all orders)	$\phantom{-}0.00028\pm 0.00270$	$0.929$	$\geq 1/3$

Appendix

Table 2: Multi-replicate flat-scenario test, replacing the original 4-item exact permutation test (structural minimum

p=1/3

regardless of the data). 20 independent replicate instances per scenario, combined via a Monte Carlo permutation test over the product of each replicate’s own 24-outcome space (

n_{MC}=200,000

; resolution

\approx 5\times 10^{-6}

Quantity	Identity-strict split (original)	IID split (corrected)
$\tau$	$0.00677738$	$0.00763411$
Original statistic, $\Delta_{\text{HO}-\text{LO}}$	$-0.10702806$	$-0.10702806$
New statistic, pooled-subset gap (L30)	$0.23837734$	$0.23837734$
New statistic, equal-order gap (L30)	$0.2907419$	$0.2907419$
Layers significant ( $p<0.05$ ), of 31	$26$	$26$
Pooled/equal-order sign agreement	$30/30$	$30/30$

Appendix

Table 3: Downstream effect of correcting the null-derangement diagnostic’s cube partition from the legacy identity-strict split to the production IID split (seed 20260910). The floor threshold

\tau

(99th percentile of

|{\rm null\ score}|

) shifts modestly; every downstream statistic is unchanged.

Subset	$k$	Production	Cyclic (replic.)	Joint-row perm.	Random perm.
${39}$	1	0.994	0.899	0.935	0.934
${40}$	1	0.946	0.906	0.908	0.939
${39,40}$	2	0.959	0.987	0.885	0.958
${41}$	1	0.902	0.940	0.983	0.938
${39,41}$	2	0.966	0.896	0.956	0.959
${40,41}$	2	0.982	0.983	0.990	0.971

Appendix

Table 4: Per-subset peak-to-final retention: original production run versus independently replicated critics under three evaluation pairings.

k

is interaction order. Replication values are unfloored (the IID-split null-floor threshold has not been separately recomputed for the replicated critics).

Explore similar work

Structural Interpretations of Protein Language Model Representations via Differentiable Graph Partitioning
May 9, 2026Siddhant Dutta, Edward Tan Beng Wai, Soumick Sarker +2Protein Structure Prediction Language Modeling
Nanyang Technological University, Singapore
A General Harness for Protein Foundation Model Fitness Prediction
Sep 28, 2026Yang Tan, Qijia Tian, Gangyu Sun +5Wireless Foundation Models Sirus
Shanghai Jiao Tong University · Shanghai Innovation Institute · Shanghai Artificial Intelligence Laboratory
StabilityArc: Decoding Protein Sequence Embeddings into Generalizable Stability Landscapes
Sep 30, 2026Aaron L. Feller, Andrew D. Ellington, Claus O. WilkeProtein Lattice Mutation
The University of Texas at Austin Interdisciplinary Life Sciences · The University of Texas at Austin Integrative Biology

May 9, 2026, cs.LGJ/K move · Enter open · S save

Structural Interpretations of Protein Language Model Representations via Differentiable Graph Partitioning

Siddhant Dutta, Edward Tan Beng Wai, Soumick Sarker, Pasan Gunawardane+1

Nanyang Technological University, Singapore

Protein Structure Prediction Language Modeling Structured Context

Open paper AskPDF arXiv

Protein language models such as ESM-2 learn rich residue representations that achieve strong performance on protein function prediction, but their features remain difficult to interpret as structural

\&

evolutionary signals are encoded in dense latent spaces. We propose a plug-

\&

-play framework that projects ESM-2 representations onto protein contact graphs

\&

applies

\textbf{SoftBlobGIN}

, a lightweight Graph Isomorphism Network with differentiable Gumbel-softmax substructure pooling, to perform structure-aware message passing

\&

learn coarse functional substructures for downstream prediction tasks. Across enzyme classification, SoftBlobGIN achieves 92.8% accuracy

\&

0.898 macro-F1. Unlike post hoc analysis of protein language models alone, our method produces directly auditable structural explanations: GNNExplainer recovers biologically meaningful active-site residues, spatially localized functional clusters,

\&

catalytic contact patterns. On binding-site detection, SoftBlobGIN improves residue AUROC from

0.885

using an ESM-2 linear probe to

0.983

, indicating that these structural explanations are not recoverable from language-model features alone. Learned blob partitions provide an additional layer of interpretability by automatically grouping residues into functional substructures, with blobs containing annotated active-site residues showing

1.85\times

higher importance than other blobs (

ρ{=}0.339

p{=}0.009

), without any active-site supervision. Our framework requires no retraining of the language model, adds only

\sim

1.1M parameters,

\&

generalises across ProteinShake tasks, achieving

F_{\max}

0.733

on Gene Ontology prediction

\&

AUROC of

0.969

on binding-site detection. We position this as an interpretable structural companion to protein language models that makes their predictions more transparent

\&

auditable.

Abstract

Explore similar work

Structural Interpretations of Protein Language Model Representations via Differentiable Graph Partitioning

A General Harness for Protein Foundation Model Fitness Prediction

StabilityArc: Decoding Protein Sequence Embeddings into Generalizable Stability Landscapes