Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology

Junbo Shen; Jinying Gao; Bo Lei

cs.AISep 29, 2026

Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology

Authors: Junbo Shen, Jinying Gao, Bo Lei

Organizations: Department of Computer Science and Engineering, The Chinese University of Hong Kong · State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Beijing Academy of Artificial Intelligence, Beijing, China · University of Chinese Academy of Sciences, Beijing, China

Abstract

Integrating transcriptomic and electrophysiological data is essential for building multimodal foundation models for neuroscience. Patch-seq provides paired measurements of gene expression and intrinsic electrophysiology from the same neuron, establishing a basis for training cross-modal models. Here we introduce LangPatch, a foundation-model-based contrastive learning framework that uses paired Patch-seq data to align pretrained GenePT representations with electrophysiological phenotypes through a language-based interface. Gene descriptions and verbalized electrophysiological profiles are embedded by the same frozen text encoder. A context adapter and projection modules connect the modalities through paired contrastive learning. Across mouse visual, mouse motor, and human cortical cohorts, LangPatch achieves the highest mean transcriptome-to-electrophysiology prediction correlation among the evaluated foundation-model and representation-learning methods. It also improves held-out cross-modal alignment in the two mouse cohorts (FOSCTTM 0.107/0.135 vs. 0.208/0.222 for JAMIE, an existing cross-modal Patch-seq imputation method). It predicts transcriptomic family, type, cortical layer, and marker-gene expression from electrophysiology, exceeding other baselines on most endpoints. More importantly, the method transfers across brain areas and species: a model trained on mouse visual cortex predicts electrophysiology in motor cortex with approximately 70% correlation retention and in human cortex with 47% (58% on acute-slice recordings). Together, these results demonstrate alignment between molecular and functional representations of neurons, providing a building block for multimodal foundation models in neuroscience.

Figures & tables

Figure 1: Language as the interface. (a) Genes enter as text through a frozen embedder; a 0.4M-parameter adapter conditioned on training-cohort statistics adjusts the frozen gene prior, and a cell is the expression-weighted mean of its adapted genes followed by a linear map. (b) Electrophysiology is verbalized and embedded by the same frozen embedder, then linearly mapped. (c) A symmetric InfoNCE objective on paired cells aligns the two; the embedder is frozen. (d) The shared space has numerous downstream applications: predicting electrophysiological features from the transcriptome (Sec. 5.1 ), reading transcriptomic family, type, layer and marker genes from electrophysiology alone (Sec. 5.2 ), zero-shot transfer across cortical areas and species (Sec. 5.3 ), and gene-level attribution of each feature (Sec. 6 ).

	Mouse visual (39 feat.)			Mouse motor (29 feat.)			Human Lee (75 feat.)
Method	$r\uparrow$	AUROC $\uparrow$	MAE $\downarrow$	$r\uparrow$	AUROC $\uparrow$	MAE $\downarrow$	$r\uparrow$	AUROC $\uparrow$	MAE $\downarrow$
Ours	0.522 $\pm$ 0.018	0.779 $\pm$ 0.008	8.98 $\pm$ 0.12	0.520 $\pm$ 0.021	0.801 $\pm$ 0.005	7.72 $\pm$ 0.33	0.414 $\pm$ 0.030	0.725 $\pm$ 0.014	9.82 $\pm$ 0.27
Plain GenePT MLP	0.505 $\pm$ 0.016	0.768 $\pm$ 0.005	9.22 $\pm$ 0.04	0.502 $\pm$ 0.020	0.790 $\pm$ 0.002	8.02 $\pm$ 0.37	0.387 $\pm$ 0.021	0.713 $\pm$ 0.009	9.90 $\pm$ 0.19
Geneformer MLP	0.492 $\pm$ 0.017	0.760 $\pm$ 0.005	9.54 $\pm$ 0.16	0.146 $\pm$ 0.028	0.574 $\pm$ 0.010	10.44 $\pm$ 0.32	0.401 $\pm$ 0.034	0.719 $\pm$ 0.010	9.88 $\pm$ 0.27
UCE MLP	0.410 $\pm$ 0.016	0.714 $\pm$ 0.007	10.51 $\pm$ 0.14	0.462 $\pm$ 0.014	0.762 $\pm$ 0.009	8.57 $\pm$ 0.38	0.312 $\pm$ 0.035	0.670 $\pm$ 0.018	10.65 $\pm$ 0.37
Nicheformer MLP	0.486 $\pm$ 0.016	0.756 $\pm$ 0.005	9.56 $\pm$ 0.08	0.503 $\pm$ 0.018	0.784 $\pm$ 0.007	8.06 $\pm$ 0.27	0.386 $\pm$ 0.028	0.710 $\pm$ 0.008	10.13 $\pm$ 0.24

Table 1: Forward prediction on held-out cells (mean

\pm

s.d. over five folds): per-feature Pearson

r

, AUROC and raw MAE. Ours: context-adapted GenePT with alignment and per-feature heads whose width is selected on the validation fold (Appendix A ).

Figure 2: Shared space. (a) Reverse readout of transcriptomic family from electrophysiology (macro-F1). (b) Held-out FOSCTTM (lower is better; 0.5 is chance) and (c) family label-transfer accuracy for the methods that expose a joint space, JAMIE re-run on our folds. (d) Training time and peak GPU memory comparisons for one fold, cost-efficiency (Appendix C )

Figure 3: Zero-shot transfer without target training. (a) Per-concept Pearson correlations for visual

\leftrightarrow

motor transfer, evaluated using the cross-area matched electrophysiology features and permutation-based chance levels. (b) Correlation retention, defined relative to the corresponding in-domain performance, for cross-area and mouse-to-human transfer. (c,d) Few-shot calibration as a function of the number of paired target cells for cross-area and mouse-to-human transfer.

Figure 4: Interpretability of the shared space. (a, b) UMAP of the aligned space for the held-out cells of one fold: transcriptome embeddings (circles) and electrophysiology embeddings (triangles), coloured by transcriptomic family; grey segments join a random subset of pairs. (c, d) Gene $\times$ feature attribution of the forward heads (gradient $\times$ input, mean over five folds, $z$ -scored per feature) for the fifteen most attributed genes, feature columns grouped by physiological family.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Cohort	Cells	Genes	Ephys features	Labels	Role in the paper
Mouse visual cortex (Gouwens 2020)	3,654	1,293	39	6 families, 4 layers	In-domain benchmark; source cohort of every zero-shot transfer
Mouse motor cortex (Scala 2021)	1,208	1,277	29	9 families, 4 layers	In-domain benchmark; cross-area transfer, both directions
Human cortex (Lee 2023)	612 / 704	1,293	75 / 25 concepts	4 subclasses, 3 layers	In-domain benchmark (612 complete cases); zero-shot cross-species target (704 paired cells, 247 acute)
Human L1 (Chartrand 2023)	240	1,293	26 concepts	2 subclasses (Lamp5/Pax6, Vip)	Zero-shot target under composition shift

Appendix

Table 2: Cohorts. In-domain benchmarks use five-fold cross-validation; zero-shot targets are scored through pre-registered feature crosswalks against measured electrophysiology.

Stage 1: context adapter and alignment
Component	Setting
Context vector $C$	24 (visual), 30 (motor), 18 (human): 4 global + per-family and per-layer mean and detection rate
Adapter	LN $\to$ Linear( $C\to 128$ ) $\to$ GELU $\to$ Linear( $128\to 3072$ ); gated residual, gate scale $s=0.1$
Projections $P_{\mathrm{cell}},P_{\mathrm{ephys}}$	LN $\circ$ Linear( $3072\to 3072$ ) $\circ$ LN
Trainable parameters	adapter 0.40M; $P_{\mathrm{cell}}$ and $P_{\mathrm{ephys}}$ 9.45M each; auxiliary regression head 9.56M
Contrastive loss	symmetric InfoNCE, temperature $\tau=0.07$

Appendix

Table 3: Hyperparameters and trainable parameter counts of the two stages.

Mouse visual
Method	FOSCTTM $\downarrow$	LTA family $\uparrow$	LTA type $\uparrow$	LTA layer $\uparrow$	Silhouette $\uparrow$
Ours	0.107 $\pm$ 0.004	0.926 $\pm$ 0.032	0.298 $\pm$ 0.020	0.462 $\pm$ 0.012	0.178 $\pm$ 0.040
JAMIE	0.208 $\pm$ 0.006	0.777 $\pm$ 0.033	0.207 $\pm$ 0.022	0.407 $\pm$ 0.026	0.102 $\pm$ 0.020
GenePT–ephys VAE	0.265 $\pm$ 0.136	0.629 $\pm$ 0.189	0.113 $\pm$ 0.064	0.301 $\pm$ 0.138	0.124 $\pm$ 0.175
Nicheformer + CLIP	0.187 $\pm$ 0.004	0.850 $\pm$ 0.017	0.239 $\pm$ 0.028	0.434 $\pm$ 0.025	0.057 $\pm$ 0.007
Mouse motor

Appendix

Table 4: Integration metrics on held-out cells (FOSCTTM, label-transfer accuracy, silhouette) and aligner cost.

	Mouse visual				Mouse motor
Method	Family F1	Type F1	Layer F1	Marker $r$	Family F1	Type F1	Layer F1	Marker $r$
Ours (context-guided multitask head)	0.779 $\pm$ 0.028	0.382 $\pm$ 0.017	0.459 $\pm$ 0.019	0.539 $\pm$ 0.007	0.710 $\pm$ 0.075	0.572 $\pm$ 0.043	0.575 $\pm$ 0.035	0.572 $\pm$ 0.009
Direct supervised ephys	0.755 $\pm$ 0.018	0.397 $\pm$ 0.019	0.413 $\pm$ 0.017	0.453 $\pm$ 0.031	0.678 $\pm$ 0.047	0.557 $\pm$ 0.083	0.486 $\pm$ 0.020	0.509 $\pm$ 0.014
Ephys $\to$ GenePT PCA-ridge	0.665 $\pm$ 0.028	0.183 $\pm$ 0.031	0.404 $\pm$ 0.016	0.456 $\pm$ 0.032	0.625 $\pm$ 0.034	0.380 $\pm$ 0.055	0.477 $\pm$ 0.040	0.509 $\pm$ 0.015
Ephys $\to$ Geneformer MLP	0.745 $\pm$ 0.023	0.320 $\pm$ 0.026	0.429 $\pm$ 0.034	0.450 $\pm$ 0.111	0.363 $\pm$ 0.220	0.210 $\pm$ 0.154	0.313 $\pm$ 0.102	0.216 $\pm$ 0.154
Ephys $\to$ UCE MLP	0.659 $\pm$ 0.042	0.150 $\pm$ 0.007	0.344 $\pm$ 0.043	0.391 $\pm$ 0.021	0.672 $\pm$ 0.065	0.383 $\pm$ 0.042	0.496 $\pm$ 0.036	0.463 $\pm$ 0.030

Appendix

Table 5: Reverse readouts on held-out cells (mean

\pm

s.d. over five folds): macro-F1 for family, fine type and layer; Pearson

r

for marker genes and programs.

Concept	Visual feature (IPFX)	Motor feature (Scala)	Sign certain
resting potential	vrest	Resting.membrane.potential..mV.	yes
input resistance	ri	Input.resistance..MOhm.	yes
membrane time constant	tau	Membrane.time.constant..ms.	yes
sag	sag	Sag.ratio	no
rheobase	threshold_i_long_square	Rheobase..pA.	yes
AP threshold	threshold_v_long_square	AP.threshold..mV.	yes

Appendix

Table 6: Pre-registered visual

\leftrightarrow

motor crosswalk.

Method	Pearson $r$	AUROC	scaled MSE	raw MAE	$\Delta$ vs. GenePT	feat. better	$p$ (feat.)	$p$ (fold)
Ours (head selected on validation)	0.414 $\pm$ 0.030	0.725	1.182	9.82	+0.027	48/75	$7.2\times 10^{-5}$	0.062
Ours (mouse head config., no selection)	0.400 $\pm$ 0.024	0.722	1.221	10.00	+0.013	44/75	0.051	0.31
Mouse stage-1 frozen + human heads	0.395 $\pm$ 0.032	0.722	1.225	10.03	+0.008	36/75	0.55	0.62
Plain GenePT MLP	0.387 $\pm$ 0.021	0.713	1.196	9.90	ref.	—	—	—
Geneformer MLP	0.401 $\pm$ 0.034	0.719	1.206	9.88	+0.014	42/75	0.13	0.31
UCE MLP	0.312 $\pm$ 0.035	0.670	1.265	10.65	-0.075	11/75	$1.7\times 10^{-10}$	0.062

Appendix

Table 7: Human-Lee in-domain benchmark (612 cells, 75 features) and the nested-validation head selection.

Representation (+ ridge head)	$n=25$	$n=50$	$n=100$	$n=200$	all
Mouse stage 1, frozen	0.270	0.297	0.329	0.367	0.412
Plain GenePT	0.197	0.264	0.333	0.374	0.420
Geneformer	0.195	0.256	0.305	0.347	0.374
Human stage 1 (saw all pairs; reference only)	0.306	0.338	0.369	0.400	0.431

Appendix

Table 8: Low-resource human forward prediction: mean Pearson

r

over 75 features with a ridge head trained on

n

labelled Lee 2023 cells (median over 5 folds

\times

10 draws; “all” uses the full training fold, 5 folds). The last row’s stage 1 saw the electrophysiology of all training cells and is a reference, not a low-resource result.

Mouse visual cortex
Criterion	Ours	Plain GenePT MLP	JAMIE	null
Train-fold markers: recovery AUC ( $K\leq 400$ )	0.57 $\pm$ 0.01	0.57 $\pm$ 0.02	0.06 $\pm$ 0.03	0.16 μ
Train-fold markers among the top 200 (fraction)	0.64 $\pm$ 0.02	0.64 $\pm$ 0.03	0.05 $\pm$ 0.03	0.21
Curated inhibitory markers among the top 200 (fraction of 11)	0.40 $\pm$ 0.05 (4.4/11)	0.60 $\pm$ 0.05 (6.6/11)	0.00 $\pm$ 0.00 (0.0/11)	0.36
Train-fold markers: fold enrichment, top 50	6.60 $\pm$ 0.34	6.60 $\pm$ 0.44	0.35 $\pm$ 0.33	1.78
Inhibitory markers: fold enrichment, top 50	9.40 $\pm$ 0.00	8.93 $\pm$ 1.97	0.00 $\pm$ 0.00	4.70

Appendix

Table 9: Attribution benchmark: the proposed model, the Plain GenePT MLP and JAMIE, attributed with the same gradient

\times

input protocol on the same held-out cells and folds, scored on biological plausibility, stability across folds, faithfulness under gene deletion and keep-only, and gene removal in the joint space.

Target	cells / concepts	Method	mean $r$	$>$ null	$R$ (verdict)	$p$
Visual $\to$ motor	1,208 / 11	Ours	0.344	11/11	0.70 (A)	0.17
		Plain GenePT	0.317	11/11	0.66 (A)	ref.
Motor $\to$ visual	3,654 / 11	Ours	0.384	11/11	0.71 (A)	0.46
		Plain GenePT	0.371	11/11	0.69 (A)	ref.
Lee 2023, all	704 / 25	Ours	0.282	18/25	0.47 (B)	0.10
		Plain GenePT	0.271	18/25	0.44 (B)	ref.

Appendix

Table 10: Zero-shot transfer summary across cortical areas and species: mean

r

, concepts above the permutation null with the registered sign, retention and verdict.

Explore similar work

Oct 1, 2026cs.AI

CortexBridge: Cortical Alignment of EEG Montages for Foundation Models

Electroencephalography (EEG) foundation models are often pretrained with a fixed channel vocabulary or a limited set of montages, making transfer difficult when electrode layouts change. We propose CortexBridge, a lightweight adapter that combines EEG features with electrode and atlas coordinates to map arbitrary montages into a shared cortical latent space. Evaluated with three frozen foundation models on five brain-computer interface (BCI) datasets from the Mother of All BCI Benchmarks (MOABB), CortexBridge improves performance in 13 of 15 evaluations. The gains in balanced accuracy average 0.80% for EEGPT, 0.70% for LaBraM, and 3.26% for CBraMod, with a maximum gain of 13.02% on 12-class steady-state visual evoked potential (SSVEP) classification. Visualizations of the learned atlas representations reveal task-dependent spatial patterns, with SSVEP showing a more concentrated representation in the Yeo Visual network than auditory P300. These results establish cortical alignment as a learnable and anatomically grounded routing mechanism from heterogeneous EEG montages to pretrained foundation models.

Jiazhen Hong, Xiaotian Zhou, Zihao Ding +2

Department of Electrical and Computer Engineering, Rutgers University Piscataway, NJ 08854, USA

Electroencephalography Foundation Models Electroencephalography Brain-Computer Interface Human Visual Cortex Foundation Model Cortex+1

Ask PDF

Jun 18, 2026cs.LG

B[FM] $^2$ : Brain Foundation Model via Flow Matching with SplitUNet

EEG foundation models can learn generalizable representations from large-scale EEG corpora to enable single-backbone transfer across diverse clinical and brain-computer interface tasks. Existing models typically discretize the continuous multi-channel EEG waveform into patches or codebook tokens and train a transformer with masked self-supervision. Recognizing that this discretization fragments continuous brain rhythms and obscures fine-grained temporal dynamics, we present B[FM]

^2

(Brain Foundation Model via Flow Matching), whose inductive bias aligns with the data by pretraining directly on the raw signal using continuous-time flow matching without patches, tokenization, or masking. However, multi-channel EEG signals pose an architectural challenge for flow matching: time is densely sampled and highly autocorrelated (thousands of timepoints), while the electrode axis is short (tens of channels) at distinct scalp positions. To address this time-electrode asymmetry, we introduce SplitUNet, a velocity network that factorizes each block into separate 1D temporal and 1D electrode convolutions and downsamples only along time, preserving electrode topology throughout the hierarchy. B[FM]

^2

sets a new state of the art on 7 of 9 standard downstream EEG classification tasks, using a pretraining budget of only 36,895 segments (

\approx

307h), 1-2 orders of magnitude (

\approx

30x) less than required by existing EEG foundation models. Further, it generates synthetic EEGs that two board-certified neurologists cannot distinguish from brain data (Cohen's

κ=

-0.096). https://jd730.github.io/projects/BFM2

Jaedong Hwang, Kathleen Zhang, Wei Dai +5

Massachusetts Institute of Technology · KU Leuven

Electroencephalography Foundation Models Electrocardiogram Foundation Models Wireless Foundation Models U-Net Convolutional

Ask PDF

Aug 12, 2026cs.LG

Continuous-Latent Predictive Modeling with Semantic Alignment for EEG-Language Foundation Models

Recent advances in EEG foundation models have demonstrated the potential of large-scale pretraining to enable generalizable neural decoding across subjects, recording environments, and datasets. However, dominant pretraining paradigms face key challenges: masked autoencoding tends to prioritize low-level signal reconstruction over task-relevant semantics, while autoregressive modeling creates a mismatch between continuous neural dynamics and discrete token spaces. To address these challenges, new strategies are needed to effectively align continuous EEG representations with natural-language semantics and enable their integration with large language models. Accordingly, we propose Brain Latent Predictive Model (BLPM), an EEG-language foundation model that reformulates heterogeneous EEG decoding tasks as a continuous semantic embedding prediction problem. BLPM introduces a Continuous EEG Latent Predictive (CELP) encoder that learns transferable representations through latent target prediction. Building on these representations, a Multi-Query Semantic Decomposition (MQSD) module extracts task-relevant information and aligns continuous EEG representations with textual semantics within a shared latent space according to their semantic relationships. Experiments across multiple benchmarks demonstrate consistent generalization performance across diverse tasks, establishing continuous latent semantic prediction as an effective paradigm for EEG-language foundation models.

Myeong-Ju Cho, Hye-Bin Shin, Seo-Hyun Lee +1

Department of Brain and Cognitive Engineering, Korea University, Seoul, Republic of Korea · Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea

Electroencephalography Foundation Models Latent Prediction Electrocardiogram Foundation Models

Ask PDF

Abstract

Explore similar work

CortexBridge: Cortical Alignment of EEG Montages for Foundation Models

B[FM]2^22: Brain Foundation Model via Flow Matching with SplitUNet

Continuous-Latent Predictive Modeling with Semantic Alignment for EEG-Language Foundation Models

B[FM] $^2$ : Brain Foundation Model via Flow Matching with SplitUNet