Bridging the EHR Divide: Asymmetric Contrastive Learning for Cross-National Medical Representation Transfer

Qingyang Zhang

cs.LGOct 4, 2026

Bridging the EHR Divide: Asymmetric Contrastive Learning for Cross-National Medical Representation Transfer

Authors: Qingyang Zhang

Organizations: Information Networking Institute Carnegie Mellon University Pittsburgh, United States

Abstract

Cross-system transfer of longitudinal Electronic Health Record (EHR) representations is challenging because clinical coding, patient populations, and healthcare workflows differ substantially across institutions and countries. We introduce Asymmetric Supervised Contrastive Learning (Asymmetric SupCon), a task-specific pre-training objective motivated by the heterogeneity of negative clinical outcomes. The objective clusters patients sharing a target positive outcome without explicitly attracting negative trajectories toward one another. We pre-train temporal Transformer encoders on longitudinal records from 3.98 million patients in the Taiwanese National Health Insurance Research Database (NHIRD) and transfer them to two U.S. EHR datasets, MIMIC-IV and EHRSHOT. A hybrid semantic mapping pipeline combining direct mappings with embedding-based retrieval enables transfer across heterogeneous clinical vocabularies. On MIMIC-IV, NHIRD pre-training consistently improves over random initialization while substantially narrowing the performance gap to task-specific in-domain pre-training. On EHRSHOT, the transferred models show particularly strong few-shot performance for incident disease prediction. A controlled objective ablation under a matched pre-training scale shows that Asymmetric SupCon achieves higher mean AUPRC than direct supervised BCE transfer on all four evaluated tasks and Standard SupCon on three of four, with a 0.003 AUPRC deficit on readmission. These results support asymmetric contrastive pre-training as an effective approach for task-specific cross-national EHR representation transfer. Code is available at https://github.com/qingYzhang/Asymmetric_SupCon.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of the cross-national transfer pipeline. Phase 1 : Task-specific temporal encoders are pre-trained on Taiwanese NHIRD trajectories using Asymmetric SupCon. Phase 2 : Heterogeneous EHRSHOT clinical concepts are mapped to the NHIRD vocabulary using direct mappings when available and embedding-based semantic retrieval otherwise. Phase 3 : The transferred encoders are adapted to MIMIC-IV and EHRSHOT prediction tasks, including few-shot EHRSHOT settings.

Task	Metric	Train-from-Scratch	In-Domain (MIMIC $\rightarrow$ MIMIC)	Cross-Domain (NHIRD $\rightarrow$ MIMIC)
30-Day Readmission	AUROC	$0.6252\pm 0.0031$	$\mathbf{0.6372\pm 0.0024}$	$\underline{0.6331\pm 0.0027}$
	AUPRC	$0.4127\pm 0.0038$	$\mathbf{0.4205\pm 0.0030}$	$\underline{0.4195\pm 0.0032}$
	F1	$0.4665\pm 0.0042$	$\mathbf{0.4734\pm 0.0036}$	$\underline{0.4723\pm 0.0039}$
90-Day Mortality	AUROC	$0.8883\pm 0.0028$	$\mathbf{0.9237\pm 0.0021}$	$\underline{0.9142\pm 0.0025}$
	AUPRC	$0.5029\pm 0.0052$	$\mathbf{0.5933\pm 0.0041}$	$\underline{0.5693\pm 0.0048}$
	F1	$0.4799\pm 0.0049$	$\mathbf{0.5201\pm 0.0040}$	$\underline{0.5094\pm 0.0045}$

Table 1: Downstream performance on MIMIC-IV. Results are reported as mean

\pm

standard deviation over five downstream random seeds. All models use the same downstream architecture, data split, and hyperparameter-selection protocol. The in-domain and cross-domain models are initialized from task-specific pre-trained encoders, while the train-from-scratch model is randomly initialized. Best mean results are bolded and second-best results are underlined.

Task	Model	All	$\mathbf{k=1}$	$\mathbf{k=8}$	$\mathbf{k=16}$	$\mathbf{k=64}$	$\mathbf{k=128}$
Long LOS	Best Baseline	0.576	0.282	0.315	0.337	0.439	0.473
Long LOS	AsymSupCon	0.517	0.298	0.333	0.355	0.431	0.450
ICU Admission	Best Baseline	0.324	0.093	0.142	0.181	0.209	0.256
ICU Admission	AsymSupCon	0.138	0.055	0.080	0.090	0.108	0.129
30-Day Readmission	Best Baseline	0.412	0.150	0.237	0.297	0.368	0.350
30-Day Readmission	AsymSupCon	0.343	0.172	0.238	0.243	0.278	0.289

Table 2: EHRSHOT few-shot performance (AUPRC). Our results are averaged over three downstream random seeds for each setting. “Best Baseline” is the best published result among CLMBR, GBM, Logistic Regression, and Random Forest for each setting Wornow et al. (2023a) . Full baseline results across all

k

are provided in Appendix Table 5 . Bold indicates the better result between the best baseline and our task-specific NHIRD-pretrained model.

	MIMIC-IV		EHRSHOT ( $k=16$ )
Pre-training Objective	30-Day Readmission	90-Day Mortality	Acute MI	Hyperlipidemia
Random Initialization	$0.413\pm 0.004$	$0.503\pm 0.005$	$0.112\pm 0.008$	$0.172\pm 0.006$
Supervised BCE Transfer	$0.414\pm 0.004$	$0.510\pm 0.005$	$0.116\pm 0.007$	$0.185\pm 0.006$
Masked Language Modeling (MLM)	$0.414\pm 0.002$	$\underline{0.519\pm 0.004}$	$0.121\pm 0.007$	$0.194\pm 0.006$
Standard SupCon	$\mathbf{0.420\pm 0.004}$	$0.517\pm 0.006$	$\underline{0.132\pm 0.006}$	$\underline{0.205\pm 0.004}$
Asymmetric SupCon (Ours)	$\underline{0.417\pm 0.004}$	$\mathbf{0.538\pm 0.005}$	$\mathbf{0.144\pm 0.006}$	$\mathbf{0.228\pm 0.005}$

Table 3: Ablation of pre-training objectives using approximately 400K NHIRD patients (

\sim

10% of the full cohort). The four pre-trained variants use the same temporal encoder, NHIRD pre-training cohort, number of training epochs, batch size, optimizer settings, and downstream adaptation protocol. Objective-specific output heads differ as required by the corresponding pre-training loss. For task-specific supervised objectives (BCE, Standard SupCon, and Asymmetric SupCon), a separate encoder is pre-trained for each target outcome. The random-initialization baseline skips NHIRD pre-training while retaining the same downstream architecture and adaptation protocol. Results are reported as mean

\pm

standard deviation over five downstream random seeds. MIMIC-IV results use the full downstream training set, while EHRSHOT results use the

k=16

few-shot setting. Performance is measured by AUPRC.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Threshold	Model	SNOMED	HCPCS	RxNorm	CPT-4	LOINC	ICD-10-PCS	ICD-O-3
$\tau_{\mathrm{sim}}=0.7$	Qwen3-Embedding-8B	62.64% (4910/7838)	88.10% (37/42)	50.85% (1137/2236)	27.19% (143/526)	33.75% (243/720)	70.59% (24/34)	95.24% (20/21)
	SFR-Embedding-Mistral	60.18% (4789/7958)	82.00% (41/50)	55.95% (1449/2590)	30.50% (212/695)	35.67% (305/855)	71.43% (25/35)	90.00% (18/20)
	Linq-Embed-Mistral	66.32% (4780/7207)	93.33% (14/15)	62.44% (1350/2162)	26.85% (109/406)	41.68% (228/547)	68.75% (11/16)	95.00% (19/20)
	multilingual-e5-large-instruct	57.69% (4386/7603)	87.27% (48/55)	57.61% (1446/2510)	29.69% (198/667)	42.45% (346/815)	75.00% (27/36)	90.00% (18/20)
$\tau_{\mathrm{sim}}=0.8$	Qwen3-Embedding-8B †	72.12% (4661/6463)	90.91% (10/11)	54.13% (1010/1866)	41.62% (82/197)	32.39% (161/497)	71.43% (15/21)	95.24% (20/21)
	SFR-Embedding-Mistral	61.82% (4766/7709)	90.00% (27/30)	57.40% (1287/2242)	27.69% (167/603)	33.33% (250/750)	67.74% (21/31)	90.00% (18/20)

Appendix

Table 4: Collision-free target mapping rate (%) at different cosine-similarity thresholds. Parentheses report the number of distinct NHIRD target concepts receiving exactly one source mapping over the total number of distinct NHIRD target concepts receiving at least one source mapping. This statistic measures target-side mapping collisions rather than semantic mapping correctness. A dash indicates that no mapping survived the threshold. † denotes the configuration selected for the main experiments.

Task	Model	All	1	2	4	8	12	16	24	32	48	64	128
Long LOS	CLMBR	0.576	0.282	0.304	0.309	0.315	0.335	0.337	0.358	0.364	0.410	0.439	0.473
	GBM	0.532	0.255	0.253	0.265	0.295	0.292	0.296	0.305	0.323	0.349	0.380	0.403
	Logistic Regression	0.384	0.237	0.251	0.283	0.283	0.283	0.294	0.308	0.311	0.346	0.340	0.365
	Random Forest	0.483	0.249	0.250	0.279	0.284	0.312	0.324	0.311	0.334	0.347	0.383	0.424
	AsymSupCon (Ours)	0.517	0.298	0.306	0.295	0.333	0.337	0.355	0.387	0.399	0.415	0.431	0.450
ICU Admission	CLMBR	0.324	0.093	0.095	0.108	0.142	0.152	0.181	0.195	0.215	0.211	0.209	0.256

Appendix

Table 5: Few-shot performance (AUPRC) on the EHRSHOT benchmark. Our results are averaged over three downstream random seeds; baseline results are from [ 24 ] . Performance is evaluated using all available training data ( All ) and varying numbers of labeled examples per class (

k

). Our model denotes the task-specific NHIRD-pretrained Asymmetric SupCon encoder followed by downstream fine-tuning. Best results are bolded and second-best results are underlined.

Explore similar work

MiGHT-EHR: A Multi-task Graph Transformer for Heterogeneous Temporal Electronic Health Records
Aug 5, 2026Anirudh Rayas, Yuan Wang, Pavan TuragaElectronic Health Records Patient Trajectories
Arizona State University, Tempe, AZ, USA · University of South Carolina, Columbia, SC, USA
Harmonized Interpretable ECG Waveform Features for Robust Cross-Dataset Clinical Prediction
Jul 26, 2026Jie Lin, Weijie Sun, Sunil V. Kalmady +4Clinical Prediction Receiver Operating Characteristic Area Under The Curve
Department of Information Management, National Taiwan University, Taiwan · Department of Computing Science, University of Alberta, Canada · Canadian VIGOUR Center, University of Alberta, Canada +2
WISTERIA: Learning Clinical Representations from Noisy Supervision via Multi-View Consistency in Electronic Health Records
May 10, 2026Ruan Dong, Yuanyun Zhang, Shi LiElectronic Health Records Representation Learning
University of Science and Technology of China · Columbia University

Aug 5, 2026, cs.LGJ/K move · Enter open · S save

MiGHT-EHR: A Multi-task Graph Transformer for Heterogeneous Temporal Electronic Health Records

Anirudh Rayas, Yuan Wang, Pavan Turaga

Arizona State University, Tempe, AZ, USA · University of South Carolina, Columbia, SC, USA

Electronic Health Records Patient Trajectories Temporal Graph Neural Networks Multiplex Graph Transformers

Open paper AskPDF arXiv

Learning from Electronic Health Records (EHRs) has gained significant attention due to its potential to improve clinical prediction. However, effective learning remains challenging because EHRs encode heterogeneous, temporally ordered clinical interactions. In particular, EHRs contain: (i) heterogeneous clinical entities, including patients, visits, diagnoses, prescriptions, and procedures, together with their heterogeneous interactions, (ii) longitudinal patient trajectories across hospital visits and (iii) shared statistical dependencies across related clinical prediction tasks. Existing EHR learning methods capture only a subset of these properties. To bridge this gap, we propose Multi-task Graph transformer for Heterogeneous Temporal EHRs (MiGHT-EHR), which jointly models all three within a unified representation learning method. MiGHT-EHR constructs a heterogeneous graph from EHRs in which nodes represent clinical entities and edges connect statistically associated entities identified via normalized point-wise mutual information. Across MIMIC-III and MIMIC-IV datasets, MiGHT-EHR outperforms state-of-the-art methods on average across four tasks: drug recommendation, prediction of length-of-stay, mortality, and readmission, with particularly strong improvements in mortality and readmission prediction. Furthermore, a post-hoc analysis of the learned representations reveals that patient neighborhoods are organized by clinical outcomes, salient medical concepts are recoverable as linear directions in the representation space, and task probabilities are well calibrated. Collectively, these findings demonstrate that MiGHT-EHR representations support diverse prediction tasks while preserving clinically interpretable structure.

Abstract

Explore similar work

MiGHT-EHR: A Multi-task Graph Transformer for Heterogeneous Temporal Electronic Health Records

Harmonized Interpretable ECG Waveform Features for Robust Cross-Dataset Clinical Prediction

WISTERIA: Learning Clinical Representations from Noisy Supervision via Multi-View Consistency in Electronic Health Records