IsoAlign: Isoform-Aware DNA-Protein Alignment via Joint-Embedding Predictive Architectures

Niousha Sadjadi; Arman Seyed-Ahmadi; Tony Tu; Micaela Elisa Consens; Kerr Ding; Alif Munim; Lila Kari; Zehui Li

cs.LGOct 30, 2026

IsoAlign: Isoform-Aware DNA-Protein Alignment via Joint-Embedding Predictive Architectures

Authors: Niousha Sadjadi, Arman Seyed-Ahmadi, Tony Tu, Micaela Elisa Consens, Kerr Ding, Alif Munim, Lila Kari, Zehui Li,

Organizations: University of Waterloo · Peter Munk Cardiac Centre, University Health Network · Georgia Institute of Technology · University of Toronto ·

Abstract

Genomic foundation models (GFMs) have emerged as a powerful paradigm for learning DNA sequence representations, yet existing self-supervised pretraining strategies overlook alternative splicing and underperform compared to specialized models. To address this gap, we introduce IsoAlign, a multimodal framework that grounds genomic pretraining in isoform-aware DNA-protein alignment to learn biologically meaningful representations. IsoAlign aligns transcripts to their corresponding protein isoforms, capturing both coding sequences and broader regulatory contexts, using two Joint-Embedding Predictive Architectures (JEPA): DNA-Protein JEPA (DP-JEPA), which predicts protein representations from DNA using contrastive alignment, and Isoform-Masked JEPA (IM-JEPA), which models relationships among alternative isoforms of the same gene through masked protein prediction. We evaluate the learned DNA representations of these architectures across a multi-metric evaluation suite spanning tasks, including the prediction of pathogenicity, fitness, gene expression, and TF binding. Our results demonstrate that isoform-aware cross-modal alignment using JEPA significantly improves functional representations and downstream performance, establishing a promising direction for GFM pretraining.

Figures & tables

Panel of Figure 1 — Figure 1: IsoAlign JEPA Architectures

Figure 2: DMS Fitness Prediction. Spearman correlation between embedding-based variant scores and fitness scores on Deep Mutational Scanning (DMS) datasets. Models are grouped by category: DNA FMs (gray), Protein FMs (red), and IsoAlign models (blue). IsoAlign consistently outperforms unimodal DNA baselines, demonstrating that cross-modal pretraining with protein-derived inductive biases improves variant effect prediction without task-specific fine-tuning.

		ClinVar Splicing		MFASS
Model	Params	Cosine	$\ell_{2}$	Cosine	$\ell_{2}$
DNABERT-2 ( Zhou et al., 2023 )	117M	0.14	0.15	0.04	0.04
JEPA-DNABERT-2 ( Larey et al., 2026 )	117M	0.14	0.15	0.04	0.04
HyenaDNA-tiny ( Nguyen et al., 2023 )	0.44M	0.16	0.15	0.04	0.04
JEPA-HyenaDNA-tiny	0.44M	0.15	0.16	0.04	0.04
Caduceus-PH ( Schiff et al., 2024 )	7.7M	0.20	0.18	0.04	0.04

Table 1: Variant effect prediction from DNA representations, evaluated by AUPRC using two metrics. Values in parentheses indicate changes relative to the parent DNA model; underlined values indicate the best performance per column, and bold values are the best IsoAlign model per backbone.

Model	Training	Params	AUROC	AUPRC
LucaOne ( He et al., 2025 )	LoRA	1.8B	0.785 $\pm$ 0.001	0.778 $\pm$ 0.002
BPNet ( Avsec et al., 2021 )	Full	0.12M	0.861 $\pm$ 0.004	0.851 $\pm$ 0.003
ChromBPNet ( Pampari et al., 2023 )	Full	6M	0.867 $\pm$ 0.005	0.851 $\pm$ 0.007
AlphaGenome ( Avsec et al., 2026 )	Full	450M	0.892 $\pm$ 0.002	0.879 $\pm$ 0.003
IsoAlign	LoRA	650M	0.934 $\pm$ 0.001	0.928 $\pm$ 0.001

Table 2: TFLink transcription factor binding prediction (mean

\pm

SD over 4 training seeds).

	Overall		Brain		Heart		Kidney		Liver		Stomach
Model	$\rho$	$r$	$\rho$	$r$	$\rho$	$r$	$\rho$	$r$	$\rho$	$r$	$\rho$	$r$
LucaOne	0.432	0.430	0.448	0.437	0.456	0.455	0.383	0.380	0.426	0.417	0.453	0.453
AlphaGenome	0.717	0.732	0.702	0.729	0.732	0.743	0.740	0.742	0.683	0.704	0.719	0.737
IsoAlign	0.725	0.738	0.716	0.739	0.735	0.744	0.745	0.747	0.697	0.715	0.722	0.736

Table 3: GTEx tissue-specific expression prediction.

Figure 3: Cross-modal retrieval (Avg MRR) by ablation category. Dashed line: CDS baseline.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Study	Protein	Samples	Protein (aa)	DNA (bp)	Value Range
Kotler et al. 2018	TP53	2,967	393	1,182	-4.99–1.51
Giacomelli et al. 2018	TP53	7,448	393	1,182	-4.37–2.84
Sun et al. 2020	CBS	6,757	551	1,656	0.00–1.00
Garvie et al. 2021	PDE3A	7,893	1,141	3,426	-3.83–2.47
Silverstein et al. 2021	GDI1	4,365	447	1,344	-7.33–3.09
Total		29,430

Appendix

Table 4: Deep Mutational Scanning dataset statistics.

Stratification	Categories	Samples
Binding Status	Positive	100,000
Binding Status	Negative (same-family)	100,000
Binding Site Length	Short ( $\leq$ 12 bp)	111,982
	Medium (13–20 bp)	78,807
	Long ( $>$ 20 bp)	9,211
TF Family	TF_bZIP	70,000

Appendix

Table 5: TFLink Evaluation Dataset

Baselines
		ClinVar Splicing				MFASS
Model	Params	Cosine	$\ell_{2}$	CLS	Mul	Cosine	$\ell_{2}$	CLS	Mul
DNABERT-2 ( Zhou et al., 2023 )	117M	0.14	0.15	0.18	0.18	0.04	0.04	0.05	0.04
JEPA-DNABERT-2 ( Larey et al., 2026 )	117M	0.14	0.15	0.27	0.19	0.04	0.04	0.08	0.04
HyenaDNA ( Nguyen et al., 2023 )	6.6M	0.17	0.18	0.25	0.18	0.04	0.04	0.05	0.04
HyenaDNA-tiny	0.44M	0.16	0.15	0.22	0.17	0.04	0.04	0.04	0.04

Appendix

Table 6: Variant effect prediction evaluation on ClinVar Splicing and MFASS using four metrics. Underlined values indicate the best performance per column, and bold values are the best IsoAlign model per backbone.

Model	VEP window ( $L$ )	Reason
LucaOne / LucaOne-gene	4,094	max_pos_embeddings =4096
AIDO.DNA-300M	3,998	max_pos_embeddings =4000
NTv3-650M / 100M, JEPA-NTv3-100M	4,096	U-Net 7-downsample (must be $\div 128$ )
Omni-DNA-116M / 300M	1,200	250 BPE-token context cap

Appendix

Table 7: Per-model VEP context-window deviations from the default.

Parameter	DP-JEPA	IM-JEPA	Contrastive	MLM
Batch size (per GPU)	16–32	8	16	16
Gradient accumulation	4	8	4	4
Effective batch size	256	256	256	256
Learning rate	1e-5	1e-5	1e-5	1e-4
Max epochs	150	150	150	150

Appendix

Table 8: Training hyperparameters by objective.

Objective	Runs	Wall-Clock (h)	GPU-Hours
DP-JEPA	34	274	1,097
IM-JEPA	12	136	543
Contrastive	48	368	1,474
MLM	11	63	254
Total	71	568	2,270

Appendix

Table 9: Total compute used for pretraining experiments.

Genomic Foundation Models

See all 21 papers

JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures
Feb 19, 2026Ariel Larey, Elay Dahan, Amit Bleiweiss +15Genomic Foundation Models Molecular Representation Learning
Applied AI Architecture, NVIDIA, Israel. · Worldwide Field Ops, NVIDIA, Israel. · Developer Programs, NVIDIA, Israel. +2
VANDAM: Viewing a nucleotide sequence with DNA molecular priors
Sep 30, 2026Jeremy Levy, Ariel Larey, Yury Nahshan +18Self-Supervised Pre-Training Genomic Foundation Models
Applied AI Architecture, NVIDIA, Israel. · Worldwide Field Ops, NVIDIA, Israel. · Developer Programs, NVIDIA, Israel. +3
What DNA Foundation Models Learn Beyond Sequence Composition
Oct 11, 2026Vincenzo Yuto Civale, Andrew David Bagdanov, Alberto MagiGenomic Foundation Models Molecular Foundation Models
University of Florence

Feb 19, 2026J/K move · Enter open · S save

JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures

Ariel Larey, Elay Dahan, Amit Bleiweiss, Raizy Kellerman+14

Applied AI Architecture, NVIDIA, Israel. · Worldwide Field Ops, NVIDIA, Israel. · Developer Programs, NVIDIA, Israel.+2

Genomic Foundation Models Molecular Representation Learning Genomics Joint-Embedding Predictive Architecture

Open paper AskPDF arXiv

Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature". While effective at capturing local syntax, these generative paradigms prioritize token-level reconstruction over high-level functional context. We introduce JEPA-DNA, a model-agnostic continual training framework that integrates a Joint-Embedding Predictive Architecture (JEPA) with traditional generative objectives. By supervising global sequence embeddings in a latent space, JEPA-DNA forces models to predict the functional representations of masked genomic segments, shifting the learning signal from token recovery to semantic alignment. We evaluate JEPA-DNA on 17 diverse genomic benchmark tasks, demonstrating consistent gains in linear probing and zero-shot performance regardless of the underlying GFM architecture or generative objective, and establishing a new state-of-the-art for GFMs under the linear-probing protocol. Through extensive ablation studies, we further characterize the synergistic interplay between generative and latent objectives. Our code is provided in the supplementary material.

Abstract

Genomic Foundation Models

Explore similar work

JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures

VANDAM: Viewing a nucleotide sequence with DNA molecular priors

What DNA Foundation Models Learn Beyond Sequence Composition