Organizations: The Graduate Center, City University of New York · Hunter College, City University of New York · Tandon School of Engineering, New York University · Weill Cornell Medicine · Harvard University · American Museum of Natural History
Protein classifiers are highly sensitive to evaluation protocol: random train/test splits induce phylogenetic leakage, allowing models to exploit taxon-specific correlations and substantially overestimate out-of-distribution performance. This issue is particularly acute for venom compound classification by ion-channel target (Ca2+, K+, Na+), where reported accuracies of 80-90% do not reflect generalization to unseen taxa. We formulate cross-species prediction as a structured distribution shift in which taxonomic identity acts as a confounder, requiring representations that capture structural features invariant across phylogeny. Using a shared multimodal backbone that fuses ESM-2 sequence embeddings with a disulfide-aware graph attention network, we systematically compare five structural inductive biases (spectral, Riemannian, SE(3)-invariant, statistical-manifold, and persistent-homology representations) under a leave-one-taxon-out evaluation protocol on a benchmark of 1,373 ColabFold-predicted venom compounds spanning five taxa. The persistent-homology representation summarizes backbone geometry through topological descriptors that are robust to local structural perturbations, providing an inductive bias toward phylogenetically invariant learning. Under conventional i.i.d. splits, all models achieve similar accuracy (81-86%), masking differences in inductive bias. Under taxonomic shift, this parity collapses: a sequence-only baseline drops to approximately 30%, the remaining geometric priors cluster at 54-55%, while our persistent-homology model (PH-GNN) achieves 63.4%, outperforming the next-best method by over 8 percentage points. These results demonstrate that random-split evaluation obscures meaningful differences in representation quality and identify topological representations as an effective inductive bias for learning under structured biological distribution shift.
Figures & tables
Figure 1: PH-GNN Multimodal Architecture (M2). Our framework extracts three parallel representations from the ColabFold-predicted input data: (1) sequence embeddings via a pre-trained ESM-2 language model. The three modalities are concatenated into a joint representation and passed through a fusion Multi Layer Perceptron (MLP) to predict the target ion channel class ( Ca2+ , K+ , Na+ ), (2) structural embeddings via a disulfide-aware Graph Attention Network (GAT) operating on all-atom proximity graphs, and (3) topological features via Alpha complex filtration of Cα coordinates.
Rank
Model
Test Acc
Ca F1
K F1
Na F1
Macro F1
Best Epoch
1
(M1) Stat. Manifold
86.2%
0.73
0.93
0.86
0.84
9/50
2
(M2) PH-GNN
85.8%
0.69
0.90
0.88
0.82
15/50
3
(M3) Spectral
84.0%
0.68
0.90
0.85
0.81
11/50
4
Baseline (ESM+GAT)
82.9%
0.68
0.92
0.83
0.81
11/50
5
(M4) Riemannian
82.9%
0.57
0.90
0.84
0.77
4/50
6
(M5) Invariant SE(3)
81.1%
0.65
0.91
0.81
0.79
9/50
Table 1: In-Distribution Performance: Random split (80/20, stratified, seed=42). Results report overall accuracy, per-class F1 scores, and macro F1 based on the optimal checkpoint selected within a maximum 50-epoch budget using early stopping. Best results per column are bolded.
Rank
Model
Snake
Cone Snail
Spider
Scorpion
Sea Anemone
Mean
Std
pholm
1
(M2) PH-GNN
75.3%
51.2%
46.7%
74.1%
69.6%
63.4%
12.0%
—
2
(M5) Invariant SE(3)
49.4%
52.0%
41.4%
61.1%
73.3%
55.4%
10.9%
6.3×10−14
3
(M1) Stat. Manifold
46.8%
46.5%
43.1%
70.0%
69.6%
55.2%
12.0%
2.7×10−5
4
Baseline (ESM+GAT)
45.5%
52.0%
43.1%
66.7%
68.9%
55.2%
10.7%
4.8×10−17
5
(M3) Spectral
48.1%
48.8%
41.2%
62.8%
70.8%
54.3%
10.8%
2.6×10−9
6
(M4) Riemannian
45.5%
52.8%
41.4%
61.9%
68.9%
54.1%
10.2%
7.9×10−11
Table 2: Out-of-Distribution Performance: Taxa leave-one-out cross-validation accuracy (%) per taxa, mean among all taxa and the standard deviation for all the models. Last fold for baseline 7 was out of memory. McNemar exact test vs. PH-GNN on paired per-protein predictions pooled across folds ( N=1,373 ), Holm-corrected.
Figure 2: Visualization of the phylogenetic trap. UMAP projection of the learned 512-dimensional concatenated structure and sequence embeddings (prior to fusion) from the baseline multimodal GNN. Left: Colored by ion channel target. Right: Colored by organism taxon. The strong clustering by taxon rather than target class visually demonstrates the model’s reliance on evolutionary lineage, explaining the catastrophic collapse during out-of-distribution evaluation.
Figure 3: Residue-Level Explainable AI (XAI). Our interactive visualization framework maps attention weights directly onto the ColabFold-predicted 3D structure. The residue attention bar chart (top right) and structural heatmap (bottom right) reveal exactly which topological loops and explicit disulfide bonds drive the prediction, bridging the gap between abstract mathematical embeddings and actionable biophysics.
Classifying protein topology is essential for deciphering biological function, but progress is held back by the lack of large-scale benchmarks that avoid duplicates and by models that do not scale well. We introduce TEDBench, a large-scale, non-redundant benchmark for protein fold classification constructed from the Encyclopedia of Domains (TED) and Foldseek-clustered AlphaFold structures. We show that on TEDBench, current protein representation learning methods either require very large models or fail to deliver strong performance. To address this challenge, we propose Masked Invariant Autoencoders (MiAE), a self-supervised framework for protein structure representation learning. MiAE uses an extremely high masking ratio of up to 90% with an SE(3)-invariant encoder and a lightweight decoder that reconstructs backbone coordinates from the latent representation and mask tokens. MiAE scales well and outperforms supervised counterparts and state-of-the-art baselines on TEDBench, establishing a strong recipe for protein fold classification. To test transfer beyond AlphaFold structures, we further benchmark on a curated dataset from experimental structures of CATH v4.4. TEDBench is available at https://github.com/BorgwardtLab/TEDBench.
Dexiong Chen, Andrei Manolache, Mathias Niepert +1
Max Planck Institute of Biochemistry, Martinsried, Germany · Computer Science Department, University of Stuttgart, Germany · Bitdefender, Romania
Proteins are inherently multiscale physical systems whose functional properties emerge from coordinated structural organization across multiple spatial resolutions, ranging from atomic interactions to global fold topology. However, existing protein representation learning methods typically operate at a single structural level or treat different sources of structural information as parallel modalities, without explicitly modeling their hierarchical relationships. We introduce PRIME (Protein Representation via Physics-Informed Multiscale Equivariant Hierarchies), a unified framework that models proteins as a nested family of five physically grounded structural graphs spanning surface, atomic, residue, secondary-structure, and protein levels. Adjacent levels are connected through deterministic, physics-informed assignment operators, enabling bidirectional information exchange via bottom-up aggregation and top-down contextual refinement. Experiments on standard protein representation learning benchmarks demonstrate strong and competitive performance across diverse tasks, with particularly notable gains on the Fold Classification benchmark, where PRIME outperforms the strongest geometric GNN baseline by margins of 13.80 and 18.30 points on the harder Superfamily and Fold splits, and achieves a state-of-the-art accuracy of 84.10% on Reaction Class prediction, surpassing all baseline methods, including ESM. Ablation studies confirm that each structural level contributes complementary and non-redundant information, and adaptive cross-attention analysis reveals that PRIME autonomously identifies the most task-relevant structural resolutions at prediction time. Our source code is publicly available at https://github.com/HySonLab/PRIME
Viet Thanh Duy Nguyen, John K. Johnstone, Truong-Son Hy
University of Alabama at Birmingham Birmingham, Alabama, USA
This paper presents a novel approach to infer protein topology using the state-of-the-art graph neural network (GNN), SchNet. The model is trained on the same dataset used to develop the recent DeepTMHMM model with 5-fold cross-validation. Unlike the conventional approaches based on using only the protein sequences or the α-carbons as features, we have decoded our classifier in this way, so all atom-level embeddings are used. Without applying any pre-trained weight, the final results have shown great potential that GNNs can be used for topological predictions.