Ensemble-Aware Structural Encoding (EASE): Conditioning Protein Encoders on Peptide Conformational Ensembles

Ruoyao Zhang; Hongxuan Li; Jie Peng; Tianlong Chen

cs.LGOct 20, 2026

Ensemble-Aware Structural Encoding (EASE): Conditioning Protein Encoders on Peptide Conformational Ensembles

Authors: Ruoyao Zhang, Hongxuan Li, Jie Peng, Tianlong Chen

Organizations: University of Waterloo · University of North Carolina at Chapel Hill

Abstract

Protein foundation models provide transferable representations for downstream peptide prediction tasks. However, existing representations typically reduce a peptide to a single sequence or structure, which may miss conformational variability relevant to key peptide properties, including antibacterial activity, hemolysis, toxicity, stability, and binding. For example, protease resistance correlates more strongly with conformational dynamics measured by hydrogen exchange than with average helicity. Therefore, we introduce Ensemble-Aware Structural Encoding (EASE), a framework for effectively extracting structural information from conformational ensembles for peptide representation. To extract useful information from noisy conformers, EASE selects central conformers representing a peptide's major structural states and fuses their individual representations with a permutation-invariant operator to produce a peptide-level representation for downstream prediction. We then evaluate our framework on six peptide property-prediction tasks under leakage-controlled splits. Against matched single-structure baselines, EASE improves performance consistently on six tasks with a mean absolute gain of about 3.0% across two structural encoders. Gains keep growing as we add conformers through K=20. A single peptide conformer therefore provides a limited basis for structural representation, whereas our framework distilling and fusing multiple conformers yields a more informative representation.

Figures & tables

Figure 1: Ensemble-Aware Structural Encoding pipeline.

Task	Peptides	Metric	Application
Toxicity	4,056	AUROC	Safety
Hemolytic	4,259	AUROC	Blood compatibility
Hemolytic HC50	1,926	RMSE	Blood compatibility
E. coli MIC	3,161	RMSE	AMP activity
Blood stability	406	AUROC	Proteolytic stability
PepPI	5,931	AUROC	Target binding

Table 1: The six peptide property-prediction tasks.

Encoder	Arm	Toxicity $\uparrow$	Hemolytic $\uparrow$	HC50 $\downarrow$	E. Coli MIC $\downarrow$	Stability $\uparrow$	Interaction $\uparrow$	Avg. rank $\downarrow$
ESM-3	Sequence track only	0.7896 $\pm$ 0.0640	0.9010 $\pm$ 0.0245	0.5540 $\pm$ 0.0328	0.6667 $\pm$ 0.0294	0.6983 $\pm$ 0.0231	0.8308 $\pm$ 0.0121	3.33
	Reference conformer	0.7792 $\pm$ 0.0534	0.9013 $\pm$ 0.0209	0.5793 $\pm$ 0.0230	0.6842 $\pm$ 0.0225	0.7141 $\pm$ 0.0266	0.8329 $\pm$ 0.0118	3.08
	EASE, mean $K=10$	0.8058 $\pm$ 0.0387	0.9071 $\pm$ 0.0161	0.5554 $\pm$ 0.0334	0.6515 $\pm$ 0.0183	0.7032 $\pm$ 0.0314	0.8520 $\pm$ 0.0100	1.67
GearNet-Edge	ESM-2 (35M)	0.7652 $\pm$ 0.0597	0.8936 $\pm$ 0.0184	0.5505 $\pm$ 0.0254	0.6657 $\pm$ 0.0205	0.6558 $\pm$ 0.0694	0.8061 $\pm$ 0.0073	4.00
	Reference conformer	0.6893 $\pm$ 0.1008	0.8617 $\pm$ 0.0221	0.6460 $\pm$ 0.0191	0.6847 $\pm$ 0.0202	0.6737 $\pm$ 0.0458	0.8059 $\pm$ 0.0159	6.33
	EASE, mean $K=10$	0.7231 $\pm$ 0.0938	0.8733 $\pm$ 0.0225	0.6139 $\pm$ 0.0164	0.6651 $\pm$ 0.0275	0.7141 $\pm$ 0.0525	0.8327 $\pm$ 0.0101	3.92

Table 2: Absolute test performance. Mean

\pm

standard deviation over folds

\times

training seeds. AUROC for classification and RMSE for regression. Average rank is obtained by ranking all arms on each task and averaging their ranks across the six tasks.

Figure 2: Structural separation over conformer bank sizes $K$ . Mean within-cluster and between-cluster TM-scores.

Encoder	Fusion	Toxicity	Hemolytic	HC50	E. coli MIC	Stability	Interaction	Mean
GearNet-Edge	Mean	+0.0339 (5/5)	+0.0116 (5/5)	+0.0321 (5/5)	+0.0196 (5/5)	+0.0404 (3/5)	+0.0267 (5/5)	+0.0274
	Gated	+0.0331 (5/5)	+0.0133 (5/5)	+0.0345 (5/5)	+0.0208 (5/5)	+0.0396 (3/5)	+0.0294 (5/5)	+0.0285
	Attention	+0.0318 (5/5)	+0.0093 (4/5)	+0.0353 (5/5)	+0.0091 (3/5)	+0.0109 (3/5)	+0.0151 (5/5)	+0.0186
	Gated attn	+0.0333 (5/5)	+0.0136 (5/5)	+0.0350 (5/5)	+0.0200 (4/5)	+0.0317 (3/5)	+0.0271 (5/5)	+0.0268
ESM-3	Mean	+0.0266 (5/5)	+0.0058 (4/5)	+0.0238 (4/5)	+0.0327 (5/5)	$-$ 0.0108 (0/5)	+0.0191 (5/5)	+0.0162
	Gated	+0.0133 (5/5)	+0.0046 (4/5)	+0.0150 (4/5)	+0.0372 (5/5)	+0.0075 (4/5)	+0.0062 (4/5)	+0.0140

Table 3: EASE with different fusion operators, paired difference over same folds against reference conformer (folds improved).

Task	Len.	Conf.	Pairwise TM-score
Task	Len.	Conf.	Median [IQR]	$\%<0.2$	$\%\geq 0.5$
Toxicity	36	142	0.095 [.055-.307]	67.0	12.8
Hemolytic	20	145	0.054 [.033-.113]	86.0	3.5
HC50	17	146	0.052 [.036-.083]	93.8	0.9
E. coli MIC	20	144	0.051 [.033-.086]	92.5	1.0
Stability	21	142	0.054 [.033-.087]	89.9	0.7

Table 4: Structural diversity of the sampled conformer sets. Length (Len.) and number of conformers (Conf.) are reported as medians across peptides. Pairwise TM-scores are computed between conformers of the same peptide and summarized over each task’s sampled corpus.

Figure 3: Effect of ensemble size on task performance. Each panel reports the paired difference with the reference conformer on the same folds. Positive values indicates improvement for both classification (AUROC) and regression (RMSE) tasks.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Encoder	Fusion	Internal width	Params added
GearNet-Edge ( $d=512$ )	mean	512	0
	gated	512	513
	gated attn	512	131,457
	attention	64	99,584
ESM-3 ( $d=1536$ )	mean	1536	0
	gated	1536	1,537

Appendix

Table 5: Parameters added by each fusion operator.

Level	Quantity	Form	Definition
residue	fluctuation	both	RMS deviation of the C $\alpha$ position around the ensemble mean
residue	accessibility, mean and s.d.	both	per-residue solvent-accessible surface area across conformers
residue	dihedral variability	raw	circular variance of backbone $\phi$ and $\psi$ across conformers
pair	distance, mean and s.d.	both	C $\alpha$ –C $\alpha$ distance across conformers
pair	contact frequency	raw	fraction of conformers with C $\alpha$ –C $\alpha$ separation below 10 Å
pair	ensemble-contact relation	—	additional edge type for pairs with contact frequency $\geq 0.3$

Appendix

Table 6: Ensemble statistics available to the alternative mechanisms. Residue and pairwise quantities enter the encoder as node features, edge features or an added relation; the peptide-level descriptor enters at the readout. The form column indicates whether a quantity is injected in raw form, within-peptide z-scored form, or both.

Setting	GearNet-Edge	ESM-3
Optimizer	AdamW	AdamW
Trunk learning rate	$3\times 10^{-4}$	tuned, $\{10^{-4},10^{-5}\}$
Head learning rate	$3\times 10^{-4}$	$10^{-3}$
Weight decay	$0.1$	$0.01$
Batch size	16	tuned, $\{16,32\}$
LoRA rank	tuned, $\{4,8\}$	8

Appendix

Table 7: Training configuration for each encoder.

GearNet-Edge

ESM-2 (35M)

ChemBERTa

ESM-3 (trunk learning rate / batch size)

task

LoRA rank

head dropout

LoRA rank

head dropout

LoRA rank

head dropout

sequence

reference conformer

mean

gated

gated attn.

attention

toxicity

0.5

0.1

0.5

10^{-5}

/16

10^{-4}

/32

10^{-4}

/16

10^{-4}

/32

10^{-4}

/16

10^{-4}

/32

hemolytic

0.1

0.3

10^{-4}

/16

10^{-4}

/16

10^{-4}

/16

10^{-4}

/16

10^{-4}

/16

10^{-4}

/16

hemolytic HC50

0.1

0.5

0.3

10^{-4}

/16

10^{-4}

/16

10^{-4}

/32

10^{-4}

/32

10^{-4}

/16

10^{-4}

/16

E. coli MIC

0.5

10^{-4}

/16

10^{-5}

/16

10^{-4}

/16

10^{-4}

/32

10^{-4}

/16

10^{-4}

/32

blood stability

0.5

0.1

0.5

10^{-5}

/16

10^{-5}

/16

10^{-5}

/32

10^{-5}

/16

10^{-4}

/16

10^{-5}

/16

Appendix

Table 8: Selected configurations. GearNet-Edge, the ESM-2-35M sequence-only baseline, and the ChemBERTa chemical baseline each tune one configuration per task over LoRA rank

\{4,8\}

and head dropout

\{0.1,0.3,0.5\}

. ESM-3 tunes one configuration per (task, arm) over trunk learning rate

\{10^{-4},10^{-5}\}

and batch size

\{16,32\}

Encoder	Fusion	Toxicity $\uparrow$	Hemolytic $\uparrow$	HC50 $\downarrow$	E. coli MIC $\downarrow$	Stability $\uparrow$	Interaction $\uparrow$	Avg. rank $\downarrow$
GearNet-Edge	Ref. conformer	0.6893 $\pm$ 0.1008	0.8617 $\pm$ 0.0221	0.6460 $\pm$ 0.0191	0.6847 $\pm$ 0.0202	0.6737 $\pm$ 0.0458	0.8059 $\pm$ 0.0159	5.00
	Mean	0.7231 $\pm$ 0.0938	0.8733 $\pm$ 0.0225	0.6139 $\pm$ 0.0164	0.6651 $\pm$ 0.0275	0.7141 $\pm$ 0.0525	0.8327 $\pm$ 0.0101	2.50
	Gated	0.7224 $\pm$ 0.0952	0.8750 $\pm$ 0.0218	0.6115 $\pm$ 0.0182	0.6639 $\pm$ 0.0313	0.7134 $\pm$ 0.0413	0.8353 $\pm$ 0.0097	2.00
	Attention	0.7210 $\pm$ 0.0939	0.8710 $\pm$ 0.0208	0.6107 $\pm$ 0.0217	0.6755 $\pm$ 0.0232	0.6846 $\pm$ 0.0723	0.8210 $\pm$ 0.0110	3.50
	Gated attn	0.7226 $\pm$ 0.0956	0.8753 $\pm$ 0.0217	0.6110 $\pm$ 0.0191	0.6647 $\pm$ 0.0313	0.7055 $\pm$ 0.0495	0.8331 $\pm$ 0.0088	2.00
ESM-3	Ref. conformer	0.7792 $\pm$ 0.0534	0.9013 $\pm$ 0.0209	0.5793 $\pm$ 0.0230	0.6842 $\pm$ 0.0225	0.7141 $\pm$ 0.0266	0.8329 $\pm$ 0.0118	4.33

Appendix

Table 9: Test performance of EASE with different fusion operators. Mean

\pm

standard deviation over folds with training seeds averaged within each fold. AUROC is reported for classification and RMSE for regression. Average rank is obtained by ranking the five arms on each task and averaging their ranks across the six tasks. Bold indicates the best mean performance for each task within each encoder.

encoder	arm	toxicity $\uparrow$	hemolytic $\uparrow$	HC50 $\downarrow$	E. coli MIC $\downarrow$	stability $\uparrow$	interaction $\uparrow$
GearNet-Edge	reference conformer	0.6893 $\pm$ 0.1008	0.8617 $\pm$ 0.0221	0.6460 $\pm$ 0.0191	0.6847 $\pm$ 0.0202	0.6737 $\pm$ 0.0458	0.8059 $\pm$ 0.0159
GearNet-Edge	$K{=}5$	0.7231 $\pm$ 0.0953	0.8747 $\pm$ 0.0234	0.6051 $\pm$ 0.0231	0.6692 $\pm$ 0.0249	0.6770 $\pm$ 0.0496	0.8271 $\pm$ 0.0108
GearNet-Edge	$K{=}10$	0.7231 $\pm$ 0.0938	0.8733 $\pm$ 0.0225	0.6139 $\pm$ 0.0164	0.6651 $\pm$ 0.0275	0.7141 $\pm$ 0.0525	0.8327 $\pm$ 0.0101
GearNet-Edge	$K{=}20$	0.7384 $\pm$ 0.0876	0.8731 $\pm$ 0.0197	0.6101 $\pm$ 0.0256	0.6636 $\pm$ 0.0248	0.7126 $\pm$ 0.0604	0.8376 $\pm$ 0.0137
ESM-3	reference conformer	0.7792 $\pm$ 0.0534	0.9013 $\pm$ 0.0209	0.5793 $\pm$ 0.0230	0.6842 $\pm$ 0.0225	0.7141 $\pm$ 0.0266	0.8329 $\pm$ 0.0118
ESM-3	$K{=}5$	0.7944 $\pm$ 0.0498	0.9058 $\pm$ 0.0161	0.5608 $\pm$ 0.0400	0.6575 $\pm$ 0.0217	0.7022 $\pm$ 0.0525	0.8477 $\pm$ 0.0116

Appendix

Table 10: Ensemble sizes

K

, absolute test performance. Mean

\pm

standard deviation over folds with training seeds averaged within each fold. Mean fusion is used for every ensemble size. AUROC reported for classification and RMSE for regression.

encoder	$K$	toxicity	hemolytic	HC50	E. coli MIC	stability	interaction	mean	tasks $\uparrow$
GearNet-Edge	5	+0.0339 (5/5)	+0.0130 (5/5)	+0.0409 (5/5)	+0.0154 (4/5)	+0.0033 (2/5)	+0.0212 (5/5)	+0.0213	6/6
GearNet-Edge	10	+0.0339 (5/5)	+0.0116 (5/5)	+0.0321 (5/5)	+0.0196 (5/5)	+0.0404 (3/5)	+0.0267 (5/5)	+0.0274	6/6
GearNet-Edge	20	+0.0491 (5/5)	+0.0114 (5/5)	+0.0359 (5/5)	+0.0210 (5/5)	+0.0389 (3/5)	+0.0316 (5/5)	+0.0313	6/6
ESM-3	5	+0.0152 (5/5)	+0.0045 (4/5)	+0.0184 (4/5)	+0.0267 (5/5)	$-$ 0.0118 (2/5)	+0.0148 (5/5)	+0.0113	5/6
ESM-3	10	+0.0266 (5/5)	+0.0058 (4/5)	+0.0238 (4/5)	+0.0327 (5/5)	$-$ 0.0108 (0/5)	+0.0191 (5/5)	+0.0162	5/6
ESM-3	20	+0.0310 (5/5)	+0.0060 (4/5)	+0.0327 (4/5)	+0.0346 (5/5)	$-$ 0.0159 (0/5)	+0.0194 (5/5)	+0.0180	5/6

Appendix

Table 11: Ensemble sizes

K

, paired differences over same folds against the fixed reference-conformer baseline (folds improved out of five). Positive values indicate improvement.

#	Mechanism	Toxicity	Hemolytic	HC50	E. coli MIC	Stability	PepPI	Mean	Tasks $\uparrow$
1	Per-residue statistics	+0.0083 (4/5)	+0.0054 (4/5)	+0.0116 (3/5)	$-$ 0.0013 (3/5)	+0.0109 (5/5)	+0.0056 (4/5)	+0.0067	5/6
1	Pairwise statistics	+0.0127 (4/5)	+0.0068 (3/5)	+0.0044 (3/5)	$-$ 0.0047 (2/5)	+0.0155 (5/5)	+0.0037 (4/5)	+0.0064	5/6
1	$\hookrightarrow$ Extra edge type †	+0.0044 (2/5)	+0.0014 (3/5)	$-$ 0.0071 (2/5)	$-$ 0.0034 (1/5)	+0.0046 (2/5)	$-$ 0.0019 (2/5)	$-$ 0.0003	3/6
2	Edge distance histogram	+0.0095 (3/5)	+0.0059 (4/5)	+0.0029 (3/5)	$-$ 0.0039 (2/5)	+0.0159 (4/5)	+0.0027 (4/5)	+0.0055	5/6
3	Contact-frequency topology	+0.0055 (4/5)	+0.0032 (2/5)	+0.0260 (5/5)	$-$ 0.0025 (1/5)	+0.0499 (4/5)	+0.0102 (5/5)	+0.0154	5/6
4	Modulation, node stats	+0.0257 (5/5)	+0.0077 (4/5)	+0.0184 (4/5)	$-$ 0.0024 (3/5)	+0.0206 (4/5)	+0.0081 (5/5)	+0.0130	5/6

Appendix

Table 12: Alternative uses of the conformational ensemble. GearNet-Edge, mean paired difference against each mechanism’s own single-structure baseline over the same folds (folds improved out of five). The alternative mechanisms are paired against the base encoder with the mean structure and EASE is paired against the reference conformer. Positive is better, with mechanisms numbered as in § B.1 . † This row alone is an increment over the row above rather than a difference against a baseline.

Figure 4: Relationship between conformer embedding variation and ensemble gains. (a) Task-level comparison across three pretrained encoders and six peptide prediction tasks. The vertical axis shows the paired performance improvement of EASE (mean fusion, $K=10$ ) over the reference conformer baseline. (b) Encoder-level averages across the six tasks. Error bars indicate $\pm 1$ standard error (bootstrap over peptides for x-axis in (a), across folds for y-axis in (a), and across tasks in (b)).

Encoder	Arm	Toxicity $\uparrow$	Hemolytic $\uparrow$	HC50 $\downarrow$	E. coli MIC $\downarrow$	Stability $\uparrow$	PepPI $\uparrow$
ESM-GearNet	Reference conformer ( $K{=}1$ )	0.7832 $\pm$ 0.0700	0.8895 $\pm$ 0.0252	0.5635 $\pm$ 0.0358	0.7059 $\pm$ 0.0168	0.6895 $\pm$ 0.0447	0.8352 $\pm$ 0.0097
	EASE, mean $K{=}10$	0.7864 $\pm$ 0.0722	0.8896 $\pm$ 0.0285	0.5425 $\pm$ 0.0418	0.7022 $\pm$ 0.0157	0.7020 $\pm$ 0.0474	0.8410 $\pm$ 0.0057
GearNet-Edge	Reference conformer ( $K{=}1$ )	0.6893 $\pm$ 0.1008	0.8617 $\pm$ 0.0221	0.6460 $\pm$ 0.0191	0.6847 $\pm$ 0.0202	0.6737 $\pm$ 0.0458	0.8059 $\pm$ 0.0159
	EASE, mean $K{=}10$	0.7231 $\pm$ 0.0938	0.8733 $\pm$ 0.0225	0.6139 $\pm$ 0.0164	0.6651 $\pm$ 0.0275	0.7141 $\pm$ 0.0525	0.8327 $\pm$ 0.0101
ESM-3	Reference conformer ( $K{=}1$ )	0.7792 $\pm$ 0.0534	0.9013 $\pm$ 0.0209	0.5793 $\pm$ 0.0230	0.6842 $\pm$ 0.0225	0.7141 $\pm$ 0.0266	0.8329 $\pm$ 0.0118
	EASE, mean $K{=}10$	0.8058 $\pm$ 0.0387	0.9071 $\pm$ 0.0161	0.5554 $\pm$ 0.0334	0.6515 $\pm$ 0.0183	0.7032 $\pm$ 0.0314	0.8520 $\pm$ 0.0100

Appendix

Table 13: Absolute test performance. Mean

\pm

standard deviation over same folds. AUROC for classification, RMSE for regression. ESM-GearNet uses a frozen ESM-2-650M with LoRA on the GearNet trunk.

Encoder	Toxicity	Hemolytic	HC50	E. coli MIC	Stability	PepPI	Mean
ESM-GearNet	$+0.0032$ (3/5)	$+0.0000$ (3/5)	$+0.0210$ (5/5)	$+0.0037$ (3/5)	$+0.0125$ (2/5)	$+0.0058$ (4/5)	$+0.0077$
GearNet-Edge	$+0.0339$ (5/5)	$+0.0116$ (5/5)	$+0.0321$ (5/5)	$+0.0196$ (5/5)	$+0.0404$ (3/5)	$+0.0267$ (5/5)	$+0.0274$
ESM-3		$+0.0266$ (5/5)	$+0.0058$ (4/5)	$+0.0238$ (4/5)	$+0.0327$ (5/5)	$-0.0108$ (0/5)	$+0.0191$ (5/5)	$+0.0162$

Appendix

Table 14: Paired difference of EASE with mean

K{=}10

against the reference conformer over the same folds (folds improved). Training seeds are averaged within a fold before pairing, and the sign is set so positive always indicates improvement for both AUROC and RMSE. The Mean column averages the six tasks.

Protein Representation Learning

Protein representation learning builds computational features that capture sequence, structure, function, and molecular relationships.

See all 69 papers

Jul 23, 2026

EnsembleEGNN: Set-Based Graph Learning for Thermodynamic Ensembles of Cyclic Peptides

Molecular graph encoding often relies on a single, static structure, ignoring the thermodynamic ensemble of molecules that are present in solution. Here, we introduce EnsembleEGNN, a foundation model that encodes structural ensembles by processing individual conformers through shared equivariant graph neural network layers, pooled with a set attention block, to make property predictions from the whole ensemble. Pretrained on the CREMP cyclic peptide dataset using multi-task self-supervision, the model is trained to encode the conformational variability of each molecule. When predicting membrane permeability from the CycPeptMPDB benchmark, EnsembleEGNN achieves an

R^2

0.477

under random cross-validation, outperforming a sequence-only BERT baseline (

R^2=0.439

). This representation advantage persists under rigorous out-of-distribution Butina splits (

R^2=0.401

versus

0.354

). Finally, a hybrid architecture co-training EnsembleEGNN with the BERT model achieves the highest overall accuracy across both random (

R^2=0.538

) and structural holdout evaluations (

R^2=0.444

). These results demonstrate that encoding conformational ensembles into latent representations improves predictions for properties governed by thermodynamics.

Aaron L. Feller, Kris Deibler, Maxim Secor

Interdisciplinary Life Sciences, University of Texas at Austin, Austin, TX. · Molecular AI, Novo Nordisk, Lexington, MA.

Equivariant GNNs Molecular Property Prediction Molecular Representation Learning Conformer

Ask PDF

May 13, 2026

ENSEMBITS: an alphabet of protein conformational ensembles

Protein structure tokenizers (PSTs) are workhorses in protein language modeling, function prediction, and evolutionary analysis. However, existing PSTs only capture local geometry of static structures, and miss the correlated motions and alternative conformational states revealed by protein ensembles. Here we introduce Ensembits, the first tokenizer of protein conformational ensembles. Ensembits address challenges inherent to tokenizing dynamics: deriving informative geometric descriptors across conformations, permutation-invariance encoding of variable-size ensembles, and conquering sparsity in dynamics data. Trained with a Residual VQ-VAE using a frame distillation objective on a large molecular dynamics corpus, Ensembits outperforms all related methods on RMSF prediction, and is the strongest standalone structural tokenizer on an token-conditioned ANOVA test on per-residue motion amplitude. Ensembits further matches or exceeds static tokenizers on EC, GO, binding site/affinity prediction, and zero-shot mutation-effect prediction despite using far less pretraining data. Notably, the distillation objective enables Ensembits to predict dynamics token from one single predicted structure, which alleviates dynamics data sparsity. As the field moves from static structure prediction toward ensemble generation, Ensembits offer the discrete vocabulary needed to bring dynamics into protein language modeling and design.

Kaiwen Shi, Carlos Oliver

Department of Computer Science, Vanderbilt University · Center for AI in Protein Dynamics, Vanderbilt University · Department of Molecular Physiology and Biophysics, Vanderbilt University

Protein Language Models Protein Representation Learning Molecular Representation Learning

Ask PDF

May 21, 2026

Atom-level Protein Representation Learning Improves Protein Structure Prediction

Recent advances in generative modeling show that pretrained representations can improve generation as conditioning features or alignment targets. Motivated by this, we study protein representations for predicting structures beyond conventional function annotation. We propose TriProRep, a structure-aware pretraining method that jointly models three aligned residue-level views: amino-acid identity, backbone geometry, and local full-atom geometry, discretely encoded via VQ-VAE tokenizers. By pretraining to recover original tokens from generator-corrupted views, TriProRep learns to distinguish plausible but incorrect cross-view augmentations from the original protein. We further introduce RepSP, a benchmark for evaluating protein representations in structure-predictive settings. RepSP tests three uses of representations: homodimer co-folding from apo-chain representations, residue-level prediction of homodimer-derived interaction properties, and representation-aligned monomer structure prediction. Across these tasks, TriProRep improves over sequence-only and prior structure-aware representation models, while maintaining competitive performance on conventional benchmarks.

Taewon Kim, Hyosoon Jang, Hyunjin Seo +6

Graduate School of AI, KAIST · Department of Chemistry, KAIST · HITS Inc. +3

Protein Structure Prediction Self-Supervised Pre-Training AI-Assisted Scientific Research Protein-Protein Interaction Prediction Protein Representation Learning Molecular Representation Learning

Ask PDF

Abstract

Protein Representation Learning

Explore similar work

EnsembleEGNN: Set-Based Graph Learning for Thermodynamic Ensembles of Cyclic Peptides

ENSEMBITS: an alphabet of protein conformational ensembles

Atom-level Protein Representation Learning Improves Protein Structure Prediction