Beyond Expression Similarity: Contrastive Learning Recovers Functional Gene Associations from Protein Interaction Structure

Jason Dury

cs.LGMar 21, 2026

Beyond Expression Similarity: Contrastive Learning Recovers Functional Gene Associations from Protein Interaction Structure

Authors: Jason Dury

Organizations: Eridos AI

Abstract

The Predictive Associative Memory (PAM) framework posits that useful relationships often connect items that co-occur in shared contexts rather than items that appear similar in embedding space. A contrastive MLP trained on co-occurrence annotations--Contrastive Association Learning (CAL)--has improved multi-hop passage retrieval and discovered narrative function at corpus scale in text. We test whether this principle transfers to molecular biology, where protein-protein interactions provide functional associations distinct from gene expression similarity. Four experiments across two biological domains map the operating envelope. On gene perturbation data (Replogle K562 CRISPRi, 2,285 genes), CAL trained on STRING protein interactions achieves cross-boundary AUC of 0.908 where expression similarity scores 0.518. A second gene dataset (DepMap, 17,725 genes) confirms the result after negative sampling correction, reaching cross-boundary AUC of 0.947. Two drug sensitivity experiments produce informative negatives that sharpen boundary conditions. Three cross-domain findings emerge: (1) inductive transfer succeeds in biology--a node-disjoint split with unseen genes yields AUC 0.826 (Delta +0.127)--where it fails in text (+/-0.10), suggesting physically grounded associations are more transferable than contingent co-occurrences; (2) CAL scores anti-correlate with interaction degree (Spearman r = -0.590), with gains concentrating on understudied genes with focused interaction profiles; (3) tighter association quality outperforms larger but noisier training sets, reversing the text pattern. Results are stable across training seeds (SD < 0.001) and cross-boundary threshold choices.

Figures & tables

Threshold	Pairs	CB ( $\|$ cos $\|<0.2$ )	Mean pos. cosine	Frac. cos $>0.5$
Low ( $\geq$ 400)	93,562	34.9%	0.187	26.3%
Medium ( $\geq$ 700)	41,434	25.6%	0.306	39.9%
High ( $\geq$ 900)	23,268	20.2%	0.389	50.2%

Table 1: STRING pair statistics at each confidence threshold.

Metric	Low ( $\geq$ 400)	Medium ( $\geq$ 700)	High ( $\geq$ 900)
Training pairs	93,562	41,434	23,268
Cosine baseline AUC	0.570	0.644	0.692
CAL AUC (assoc-only)	0.800	0.877	0.910
$\Delta$ AUC	+0.230	+0.233	+0.218
CB cosine AUC	0.529	0.534	0.518
CB CAL AUC	0.783	0.861	0.908

Table 2: Confidence threshold sweep (Replogle K562 + STRING). See also Figure 1 .

Figure 1: Cross-boundary AUC across all four experiments. CAL provides large gains on gene perturbation data where latent associative signal exists (Replogle, DepMap). Drug experiments reveal failure modes: no latent signal (fingerprints) and degree confounding (L1000, where shuffled $>$ real).

Filter	Pairs	Overall AUC	CB AUC
Combined $\geq$ 700	41,434	0.877	0.861
Combined $\geq$ 900	23,268	0.910	0.908
Experimental $\geq$ 700	23,438	0.907	0.902

Table 3: Combined-score vs experimental-channel filtering. See also Figure 2 .

Figure 2: Confidence threshold sweep showing quality-over-quantity. (A) Overall AUC and (B) cross-boundary AUC both improve with tighter thresholds despite fewer training pairs. Experimental-only filtering (hatched) matches the highest combined-score threshold.

Condition	$\Delta$ AUC vs cosine	Interpretation
Full CAL	+0.233	Full model
Shuffled pairs	$-$ 0.096	Random pairings actively hurt
Similar positives	+0.018	Association $\neq$ similarity confirmed
Random negatives	+0.193	Positives carry signal
Inductive (edge-split 70/30)	+0.183	Compositional generalisation
Inductive (node-split 70/30)	+0.127	Cold-start generalisation (Section 6.1 )

Table 4: Ablation study (Replogle, medium confidence

\geq

700).

Setting	N pairs	CAL AUC	Cosine AUC	$\Delta$ AUC
In-batch negatives	200K	0.379	0.896	$-$ 0.517
Random negatives	200K	0.956	0.896	+0.060

Table 5: DepMap training results: in-batch vs random negatives.

Domain	Split type	$\Delta$ vs baseline	Notes
Text (HotpotQA)	Edge-split	+0.10 R@5	Not significant (CI includes zero)
Text (MuSiQue)	Edge-split	$-$ 7.63 R@5	Harmful
Biology (Replogle, med.)	Edge-split	+0.183 AUC	Genes overlap between train/test
Biology (Replogle, high)	Node-split	+0.127 AUC	Unseen genes, cold-start

Table 6: Inductive transfer across domains.

Experiment	Regime	Failure mode	Outcome
Gene (Replogle)	Latent signal present	—	+0.390 CB AUC
DepMap (random neg.)	Rescued	Negative sampling	+0.377 CB AUC
DepMap (in-batch)	Neg. sampling failure	In-batch neg. from pos. cluster	$-$ 0.517
Drug fingerprints	No latent signal	Embedding uninformative	+0.021
Drug L1000	Degree confounded	Low entity diversity	Shuffled $>$ real

Table 7: Operating envelope: when does CAL work?

Quintile	Degree range	N pairs	Cosine	Association	$\Delta$
Q1 (lowest)	5–184	950	0.003	0.224	+0.221
Q5 (highest)	521–1,110	933	$-$ 0.030	0.097	+0.127

Table 8: Cross-boundary improvement by degree quintile (Replogle, high confidence). See also Figure 3 .

Figure 3: Cross-boundary improvement by degree quintile (high confidence $\geq$ 900, cross-boundary pairs with $|\text{cosine}|<0.2$ ). Low-degree genes (Q1) benefit approximately twice as much as high-degree genes (Q5). The trend is approximately monotonic; Q4 and Q5 show near-equal improvement (+0.124 vs +0.127), suggesting the degree effect saturates for high-degree genes. Table 8 shows Q1 and Q5 from the same high-confidence cross-boundary data; the appendix (Table 17 ) uses medium-confidence all-pairs.

Figure 4: (A) Association score distributions by cosine similarity bin. In the cross-boundary regime (yellow shading), STRING-positive pairs (blue) show clear separation from non-associated pairs (grey). (B) Three illustrative gene pairs showing CAL lift: each arrow indicates the gain from cosine (grey dot on diagonal) to the actual association score (blue dot).

Cluster source	Total	Coherent	Partial	No signal	% coherent
PAM-specific (assoc. space)	22	22	0	0	100%
Expression-space HDBSCAN	36	35	1	0	97%
Random (matched sizes, 10 rep.)	34	$13.8\pm 1.4$	—	—	41%

Table 9: Cluster GO enrichment comparison.

Aspect	Text (HotpotQA)	Biology (Replogle)
Embeddings	BGE-large (1024d)	PCA-50 expression
Associations	Gold co-occurrence	STRING PPI
Cross-boundary gain	+28.5 R@5 (hard Qs)	+0.390 AUC
Inductive transfer (node-split)	Fails ( $\pm$ 0.10)	Succeeds (+0.127)
Quality vs quantity	More pairs better	Tighter pairs better
Cosine contribution	Useful ( $\lambda$ =0.6 optimal)	Uninformative (assoc-only best)

Table 10: Domain comparison: text vs biology.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

$\lambda$	Low ( $\geq$ 400)	Medium ( $\geq$ 700)	High ( $\geq$ 900)
0.0 (cosine only)	0.570	0.644	0.692
0.1	0.575	0.652	0.701
0.2	0.582	0.661	0.711
0.3	0.589	0.672	0.724
0.4	0.599	0.685	0.739
0.5	0.612	0.703	0.757

Appendix

Table 11: Overall AUC as a function of

\lambda

(Replogle).

$\lambda$	CB AUC
0.0	0.534
0.1	0.566
0.2	0.603
0.3	0.644
0.4	0.689
0.5	0.737

Appendix

Table 12: Cross-boundary AUC as a function of

\lambda

(medium confidence).

$\|$ cos $\|$ threshold	Pos. pairs	Neg. pairs	Cosine AUC	CAL AUC	$\Delta$
$<0.30$	6,805	27,671	0.540	0.907	+0.367
$<0.20$	4,706	19,580	0.518	0.908	+0.390
$<0.15$	3,643	15,109	0.516	0.909	+0.393
$<0.10$	2,368	10,327	0.510	0.914	+0.404
$<0.05$	1,211	5,293	0.491	0.915	+0.424

Appendix

Table 13: CAL AUC at varying cross-boundary thresholds (high confidence

\geq

900). See also Figure 5 .

Figure 5: Cross-boundary sensitivity analysis. As the threshold tightens (left to right), cosine AUC declines toward chance while CAL AUC remains stable or slightly increases. Shaded region shows the growing gap.

N pairs	CAL AUC	Cosine AUC	$\Delta$
25K	0.336	0.924	$-$ 0.588
50K	0.341	0.916	$-$ 0.576
100K	0.279	0.908	$-$ 0.628
200K	0.379	0.896	$-$ 0.517

Appendix

Table 14: DepMap in-batch training across pair counts.

N pairs	CAL AUC	Cosine AUC	$\Delta$
25K	0.305	0.565	$-$ 0.260
200K	0.554	0.534	+0.021
Shuffled	0.636	—	$>$ real

Appendix

Table 15: Drug structure training results.

Setting	CAL AUC	CB AUC	Shuffled AUC	Shuffled CB
In-batch neg.	0.674	0.679	—	—
Random neg.	0.850	0.842	0.909	0.908

Appendix

Table 16: Drug L1000 training results (200K pairs).

Quintile	Degree range	Cosine	Association	$\Delta$
Q1 (lowest)	5–184	0.364	0.252	$-$ 0.112
Q2	185–270	0.431	0.215	$-$ 0.216
Q3	271–386	0.437	0.180	$-$ 0.257
Q4	387–520	0.380	0.143	$-$ 0.238
Q5 (highest)	521–1,110	0.336	0.108	$-$ 0.228

Appendix

Table 17: Full degree quintile analysis (Replogle, medium confidence)—all pairs.

	Expr. cosine	Assoc. cosine	Ratio
Hub clusters ( $n$ =19)	$0.407\pm 0.181$	$0.692\pm 0.154$	1.70 $\times$
Non-hub clusters ( $n$ =3)	$0.728\pm 0.062$	$0.587\pm 0.053$	0.81 $\times$

Appendix

Table 18: Hub vs non-hub cluster properties.

Contrastive Learning

Contrastive learning trains representations by bringing related examples together and separating unrelated ones for recognition, retrieval, and transfer.

See all 322 papers

ProtAlign: Contrastive learning paradigm for Sequence and structure alignment
Mar 6, 2026Aditya Ranganath, Hasin Us Sami, Kowshik Thopalli +2Cross-Modal Alignment Protein Representation Learning
Center for Applied Scientific Computing, Lawrence Livermore National Laboratory
MKGR: Multimodal Knowledge-Graph Representation Learning for Cold-Start Protein-Protein Interaction Prediction
Jul 2, 2026Wenbo ZhangKG Embedding Protein-Protein Interaction Prediction
College of Computer and Information Science, Southwest University, Chongqing, Chongqing, China
Protein Thoughts: Interpretable Reasoning with Tree of Thoughts and Embedding-Space Flow Matching for Protein-Protein Interaction Discovery
May 19, 2026Kingsley Yeon, Xuefeng Liu, Promit GhosalInference-Time Search Tree of Thoughts
Department of Statistics and CCAM University of Chicago Chicago, IL 60637 · School of Medicine Stanford University Stanford, CA 94305 · Department of Statistics University of Chicago Chicago, IL 60637

Mar 6, 2026J/K move · Enter open · S save

ProtAlign: Contrastive learning paradigm for Sequence and structure alignment

Aditya Ranganath, Hasin Us Sami, Kowshik Thopalli, Bhavya Kailkhura+1

Center for Applied Scientific Computing, Lawrence Livermore National Laboratory

Cross-Modal Alignment Protein Representation Learning Sequence Alignment Contrastive Learning

Open paper AskPDF arXiv

Protein language models often take into consideration the alignment between a protein sequence and its textual description. However, they do not take structural information into consideration. Traditional methods treat sequence and structure separately, limiting the ability to exploit the alignment between the structure and protein sequence embeddings. In this paper, we introduce a sequence structure contrastive alignment framework, which learns a shared embedding space where proteins are represented consistently across modalities. By training on large-scale pairs of sequences and experimentally resolved or predicted structures, the model maximizes agreement between matched sequence structure pairs while pushing apart unrelated pairs. This alignment enables cross-modal retrieval (e.g., finding structural neighbors given a sequence), improves downstream prediction tasks such as function annotation and stability estimation, and provides interpretable links between sequence variation and structural organization. Our results demonstrate that contrastive learning can serve as a powerful bridge between protein sequences and structures, offering a unified representation for understanding and engineering proteins.

Abstract

Contrastive Learning

Explore similar work

ProtAlign: Contrastive learning paradigm for Sequence and structure alignment

MKGR: Multimodal Knowledge-Graph Representation Learning for Cold-Start Protein-Protein Interaction Prediction

Protein Thoughts: Interpretable Reasoning with Tree of Thoughts and Embedding-Space Flow Matching for Protein-Protein Interaction Discovery