WASP: Weakly Aligned Spatiotemporal Pairs for Fetal Brain MRI-Ultrasound Learning

Francesco Correnti; Gabriele Magrini; Marco Mistretta; Niccolò Biondi; Pietro Pala; Alessandro Ramalli; Simona Fiori; Andrew D. Bagdanov

cs.CVOct 3, 2026

WASP: Weakly Aligned Spatiotemporal Pairs for Fetal Brain MRI-Ultrasound Learning

Authors: Francesco Correnti, Gabriele Magrini, Marco Mistretta, Niccolò Biondi, Pietro Pala, Alessandro Ramalli, Simona Fiori, Andrew D. Bagdanov, +1 more

Organizations: University of Florence, Italy · University of Trento, Italy · Meyer Children’s Hospital IRCCS, Florence, Italy

Abstract

Magnetic Resonance Imaging (MRI) is widely regarded as the optimal sensor for fetal brain analysis due to its superior soft-tissue contrast and anatomical detail. However, its high cost and operational burden make it invasive and difficult to obtain at scale. Ultrasound (US), in contrast, is cheap, safe, and routinely acquired, and as a result it has produced substantially larger datasets and a growing ecosystem of pretrained models. This asymmetry raises a natural question: Can we teach a US-only model to understand fetal MRI from only a limited set of examples? The standard recipe, training a foundation model on subject-to-subject paired MRI-US scans, is not viable since no such paired fetal dataset is publicly available. In this paper we address this gap with Weakly Aligned Spatiotemporal Pairs (WASP), a framework that formulates cross-modal correspondence as an entropic Optimal Transport problem driven by clinical metadata, in particular Gestational Age (GA) and diagnostic planes, enabling the fitting of a lightweight alignment module that lifts MRI representations into the US latent space, without fine-tuning the backbone. Empirically, WASP yields its largest gains when MRI is unseen by the model during pretraining (on USFM, GA estimation error drops from 21.9 to 17.4 days and standard plane classification accuracy climbs from 61.9% to 69.0%), while providing smaller, backbone-dependent refinements for backbones pretrained on both modalities (e.g., BioMedParse GA estimation error from 6.5 to 6.0 days and SAM-Med2d plane accuracy from 83.3% to 88.1%). Code is available at https://github.com/miccunifi/WASP.

Figures & tables

Figure 1: Motivation and overview of WASP. Left : Existing MRI–US alignment requires paired fetal scans from the same subject, but such data are not publicly available at scale. Right : WASP avoids exact pairing through two phases: Phase I builds an entropic Optimal Transport plan ${P}$ from unpaired cohorts using commonly available metadata such as Gestational Age and diagnostic plane. Phase II uses ${P}$ to project the target encoder space and train a lightweight alignment module $\mathcal{F}$ on the frozen source encoder’s features.

Figure 2: US and MRI data preprocessing. While in MRI we can directly get axis-aligned samples, for US images we also need to fix the orientation of the major and minor axes of the fetal brain.

MRI $\to$ US		Unimodal (US)				Multimodal
MRI $\to$ US		USFM [ 4 ]		Echocare [ 5 ]		SAM-Med2d [ 9 ]		BioMedParse [ 10 ]
Pairing	Alignment	GA Reg. $\downarrow$	Plane Cls. $\uparrow$	GA Reg. $\downarrow$	Plane Cls. $\uparrow$	GA Reg. $\downarrow$	Plane Cls. $\uparrow$	GA Reg. $\downarrow$	Plane Cls. $\uparrow$
Zero-Shot	—	21.9	61.9	10.3	81.0	7.7	83.3	6.5	85.7
—	CORAL [ 17 ]	22.4	54.8	29.7	69.0	41.2	50.0	24.9	40.5
Greedy	CCA	25.5	66.7	16.8	76.2	21.0	83.3	14.5	78.6
	Ridge	25.2	66.7	10.4	83.3	10.1	85.7	6.1	85.7

Table 1: Cross-modal alignment results in the MRI

\to

US direction across four backbones: two unimodal US models (USFM, Echocare) and two multimodal models (SAM-Med2d, BioMedParse). We compare zero-shot transfer, CORAL, and three pairing strategies combined with CCA, Ridge, or MLP alignment. Performance is measured by GA regression error (

\downarrow

) and diagnostic-plane classification accuracy (

\uparrow

). CORAL, CCA, and Ridge are deterministic; MLP rows report mean

\pm

std over 3 seeds. Best and second-best results are shown in bold and underlined, respectively.

US $\to$ MRI		Unimodal (US)				Unimodal (MRI)		Multimodal
US $\to$ MRI		USFM [ 4 ]		Echocare [ 5 ]		MRI-CORE [ 8 ]		SAM-Med2d [ 9 ]		BioMedParse [ 10 ]
Pairing	Alignment	GA Reg. $\downarrow$	Plane Cls. $\uparrow$	GA Reg. $\downarrow$	Plane Cls. $\uparrow$	GA Reg. $\downarrow$	Plane Cls. $\uparrow$	GA Reg. $\downarrow$	Plane Cls. $\uparrow$	GA Reg. $\downarrow$	Plane Cls. $\uparrow$
Zero-Shot	—	16.0	92.8	14.9	95.6	12.3	98.1	15.7	96.2	12.1	98.7
—	CORAL [ 17 ]	23.7	85.0	23.8	85.4	16.7	96.2	19.0	91.1	23.5	88.4
Greedy	CCA	23.2	89.7	21.5	92.2	24.9	93.5	26.1	93.0	18.6	95.8
	Ridge	22.8	90.1	20.6	92.6	25.3	93.5	22.0	96.4	17.9	98.5

Table 2: Cross-modal alignment results in the complementary US

\to

MRI direction across five backbones: two unimodal US models (USFM, Echocare), one unimodal MRI model (MRI-CORE), and two multimodal models (SAM-Med2d, BioMedParse). We report gestational-age regression (GA Reg.; lower is better) and plane classification accuracy (Plane Cls.; higher is better), comparing Zero-Shot performance against different pairing and alignment strategies. CORAL, CCA, and Ridge are deterministic; MLP rows report mean

\pm

std over 3 seeds. Best and second-best results within each backbone and task are shown in bold and underlined, respectively.

Student	Teacher	Setup	GA $\downarrow$	Cls. $\uparrow$
USFM [ 4 ]	Zero-Shot	-	21.9	61.9
	SAM-Med2d [ 9 ]	Raw	12.4	76.2
	SAM-Med2d [ 9 ]	+ WASP	12.0 $\pm$ 0.8	72.2 $\pm$ 1.4
	BioMedParse [ 10 ]	Raw	11.9	69.0
	BioMedParse [ 10 ]	+ WASP	11.1 $\pm$ 0.2	71.4 $\pm$ 0.0
Echocare [ 5 ]	Zero-Shot	-	10.3	81.0

Table 3: MRI

\to

US. Teacher-student distillation results on the MRI test set.

\mathcal{G}_{m\to m}

maps student embeddings to those of a teacher (raw or WASP-improved) using a Ridge regressor; the teacher alignment

\mathcal{F}

is WASP +MLP. Raw rows are deterministic; + WASP rows report mean

\pm

std over 3 seeds.

Active cost terms			USFM [ 4 ]		SAM-Med2d [ 9 ]
$D^{\text{time}}$	${\color[rgb]{0,0,0}D^{\text{proto}}}$	$D^{\text{space}}$	GA $\downarrow$	Cls. $\uparrow$	GA $\downarrow$	Cls. $\uparrow$
✗	✗	✗	21.9 –	61.9 –	7.7 –	83.3 –
✗	✗	✓	35.7 $\uparrow$ 13.8	76.2 $\uparrow$ 14.3	28.5 $\uparrow$ 20.8	90.5 $\uparrow$ 7.2
✗	✓	✗	20.3 $\downarrow$ 1.6	52.4 $\downarrow$ 9.5	8.9 $\uparrow$ 1.2	90.5 $\uparrow$ 7.2
✗	✓	✓	22.9 $\uparrow$ 1.0	50.0 $\downarrow$ 11.9	8.6 $\uparrow$ 0.9	95.2 $\uparrow$ 11.9
✓	✗	✗	12.6 $\downarrow$ 9.3	33.3 $\downarrow$ 28.6	6.8 $\downarrow$ 0.9	64.3 $\downarrow$ 19.0

Table 4: MRI

\to

US. Ablation on cost terms. The combination of all three terms results in the most consistent gains. USFM uses Ridge alignment; SAM-Med2d reports one MLP run per subset.

MRI $\to$ US		Unimodal (US)				Multimodal
MRI $\to$ US		USFM [ 4 ]		Echocare [ 5 ]		SAM-Med2d [ 9 ]		BioMedParse [ 10 ]
Pairing	Alignment	GA Reg. $\downarrow$	Plane Cls. $\uparrow$	GA Reg. $\downarrow$	Plane Cls. $\uparrow$	GA Reg. $\downarrow$	Plane Cls. $\uparrow$	GA Reg. $\downarrow$	Plane Cls. $\uparrow$
Zero-Shot	—	21.9	61.9	10.3	81.0	7.7	83.3	6.5	85.7
Random	CCA	28.9	35.7	36.3	38.1	28.8	40.5	31.3	47.6
	Ridge	32.0	42.9	22.2	66.7	29.2	54.8	20.4	61.9
	MLP	32.3 $\pm$ 3.1	45.2 $\pm$ 10.4	20.2 $\pm$ 0.7	81.0 $\pm$ 6.3	21.8 $\pm$ 1.6	71.4 $\pm$ 2.4	18.6 $\pm$ 1.8	63.5 $\pm$ 5.0

Table 5: MRI

\to

US. Ablation of pairing strategies without spatiotemporal metadata. Random pairs each source sample with a target sample drawn uniformly at random, Cosine greedily matches each source to its nearest target by cosine similarity in raw embedding space, and OT-Cosine uses the WASP entropic OT with the cosine distance between raw embeddings as the only cost. All three underperform Zero-Shot in most settings. MLP rows report mean

\pm

std over 3 seeds; for Random, the seed also changes the random pairing. WASP rows are reproduced from Tab. 1 .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Modality	Split	#Images	% Img	#Subjects	% Subj	GA (days)	TT / TC / TV
US	Overall	3,369	—	1,744	—	$163.3\pm 45.7$	2756 / 353 / 260
	$\mathcal{D}_{\text{pair}}$	961	28.5	613	35.1	$176.7\pm 39.7$	865 / 57 / 39
	$\mathcal{D}_{\text{train}}$	1,934	57.4	905	51.9	$157.0\pm 45.7$	1514 / 239 / 181
	$\mathcal{D}_{\text{test}}$	474	14.1	226	13.0	$161.8\pm 51.1$	377 / 57 / 40
MRI	Overall	687	—	229	—	$214.1\pm 30.6$	229 / 229 / 229
	$\mathcal{D}_{\text{pair}}$	480	69.9	160	69.9	$216.1\pm 29.7$	160 / 160 / 160

Appendix

Table 6: Dataset partition statistics for

\mathcal{D}^{\text{US}}

and

\mathcal{D}^{\text{MRI}}

. GA is reported in days as mean

\pm

std. % columns refer to the fraction of the respective modality total. Plane counts are TT / TC / TV.

	$\varepsilon=10^{-4}$		$\varepsilon=5\!\times\!10^{-4}$		$\varepsilon=10^{-3}$		$\varepsilon=5\!\times\!10^{-3}$
$\Delta$ (days)	GA $\downarrow$	Cls. $\uparrow$	GA $\downarrow$	Cls. $\uparrow$	GA $\downarrow$	Cls. $\uparrow$	GA $\downarrow$	Cls. $\uparrow$
$7$	21.8	64.4	21.4	66.0	21.4	66.5	20.7	66.6
$14$	20.4	64.3	20.2	60.0	20.4	59.1	20.9	67.7
$21$	20.8	65.7	20.0	65.6	19.8	65.3	18.3	68.3

Appendix

Table 7: MLP alignment hyperparameter sensitivity (USFM, MRI

\to

US, test set). Each cell reports GA Reg.

\downarrow

(days) / Plane Cls.

\uparrow

(%), averaged over weight decay

\in\{10^{-4},\ldots,5\!\times\!10^{-1}\}

and hidden dim

\in\{256,512,1024\}

. Bold = column-wise best.

		US $\to$ MRI		MRI $\to$ US
Pairing	Alignment	GA $\downarrow$	Cls. $\uparrow$	GA $\downarrow$	Cls. $\uparrow$
Zero-Shot	—	12.3	98.1	6.5	76.2
—	CORAL	16.7	96.2	19.8	57.1
Greedy	CCA	24.9	93.5	13.6	73.8
	Ridge	25.3	93.5	9.1	73.8
	MLP	12.3 $\pm$ 0.2	97.2 $\pm$ 0.1	9.0 $\pm$ 0.1	78.6 $\pm$ 2.4

Appendix

Table 8: MRI-CORE in both directions. US

\to

MRI is evaluated on the US test set, MRI

\to

US on the MRI test set. Best and second-best results in each metric column are shown in bold and underline , respectively.

Teacher	Setup	$\mathcal{G}$	GA $\downarrow$	Cls. $\uparrow$
Zero-Shot	—	—	12.3	98.1
SAM-Med2d	Raw	Ridge	13.3	97.3
	Raw	MLP	12.3 $\pm$ 0.1	97.5 $\pm$ 0.2
	+ WASP	Ridge	14.5 $\pm$ 0.4	95.2 $\pm$ 0.7
	+ WASP	MLP	11.6 $\pm$ 0.1	98.0 $\pm$ 0.1
BioMedParse	Raw	Ridge	13.2	97.9

Appendix

Table 9: MRI-CORE as student, US

\to

MRI. Teachers are improved with WASP in the US

\to

MRI direction, and

\mathcal{G}_{\text{US}\to\text{US}}

(Ridge or MLP) maps MRI-CORE’s US embeddings to the teacher’s; evaluation is on the US test set. Raw rows are deterministic; + WASP rows, including Ridge ones, report mean

\pm

std over the 3 seeds of the teacher alignment. Best and second-best results in each metric column are shown in bold and underline , respectively.

	Zero-Shot		MRI $\to$ US synthesis		Adversarial alignment
Backbone	GA $\downarrow$	Cls. $\uparrow$	GA $\downarrow$	Cls. $\uparrow$	GA $\downarrow$	Cls. $\uparrow$
USFM	21.9	61.9	20.8	71.4	21.8	64.3
Echocare	10.3	81.0	10.7	71.4	10.3	81.0
SAM-Med2d	7.7	83.3	9.5	81.0	7.7	83.3
BioMedParse	6.5	85.7	8.6	81.0	6.6	85.7

Appendix

Table 10: MRI

\to

US. Unpaired alignment baselines on the MRI test set. WASP results are in Tab. 1 . Best and second-best results within each backbone and metric are shown in bold and underline , respectively.

	MRI test set				US test set
	Zero-Shot		Fine-tuned		Before		After fine-tuning
Backbone	GA $\downarrow$	Cls. $\uparrow$	GA $\downarrow$	Cls. $\uparrow$	GA $\downarrow$	Cls. $\uparrow$	GA $\downarrow$	Cls. $\uparrow$
USFM	21.9	61.9	8.4 $\pm$ 1.4	81.7 $\pm$ 11.3	16.0	92.8	18.3 $\pm$ 0.3	95.1 $\pm$ 1.2
Echocare	10.3	81.0	7.9 $\pm$ 2.0	86.5 $\pm$ 3.6	14.9	95.6	26.0 $\pm$ 3.0	88.2 $\pm$ 4.2
SAM-Med2d	7.7	83.3	5.3 $\pm$ 0.5	88.1 $\pm$ 2.4	15.7	96.2	16.8 $\pm$ 0.5	98.6 $\pm$ 0.5
BioMedParse	6.5	85.7	4.3 $\pm$ 0.2	92.1 $\pm$ 1.4	12.1	98.7	15.0 $\pm$ 1.5	98.9 $\pm$ 0.9

Appendix

Table 11: Direct fine-tuning on MRI. Left: MRI test set. Right: US test set before and after fine-tuning. Best and second-best results within each backbone, test domain, and metric are shown in bold and underline , respectively.

Figure 3: Spatiotemporal Evolution of Fetal Brain Anatomy in MRI and US. Rows represent the three standard diagnostic planes (TV, TT, TC). Columns represent different Gestational Ages.

Explore similar work

Cross-Modal Ultrasound-MRI Learning for Fetal Brain Ventricular Volumetry and Abnormality Screening
Aug 14, 2026Yuhao Huang, Yuanji Zhang, Yuhuan Lu +3Ultrasound Imaging Medical Image Representation Learning
Department of Radiology, Boston Children’s Hospital, and Harvard Medical School, Boston, USA · Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China · Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong, China +3
AWR-Net: Decoupling Anatomy and Appearance for 3D Fetal Brain Ultrasound Synthesis
Sep 18, 2026Yuhuan Lu, Sergio Valencia, Yuanji Zhang +4Medical Image Synthesis Ultrasound Imaging
Department of Radiology, Boston Children’s Hospital and Harvard Medical School, Boston, MA 02115, USA · Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA
Benchmarking Ultrasound Foundation Models for Fetal Plane Classification
May 27, 2026Leya Barrientos, Yuexi Du, Nicha C. DvornekMedical Imaging Foundation Models Medical Image Classification
Radiology & Biomedical Imaging, Yale School of Medicine, USA · Department of Biomedical Engineering, Yale University, USA

Aug 14, 2026J/K move · Enter open · S save

Cross-Modal Ultrasound-MRI Learning for Fetal Brain Ventricular Volumetry and Abnormality Screening

Yuhao Huang, Yuanji Zhang, Yuhuan Lu, Dong Ni+2

Department of Radiology, Boston Children’s Hospital, and Harvard Medical School, Boston, USA · Medical Ultrasound Image Computing (MUSIC) Lab, Shenzhen University, Shenzhen, China · Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong, China+3

Ultrasound Imaging Medical Image Representation Learning

Open paper AskPDF arXiv

Assessment of ventriculomegaly (VM) on fetal brain ultrasound relies primarily on measuring lateral ventricular atrial width on standard planes, which is operator-dependent and may not fully reflect the overall ventricular enlargement. Fetal brain MRI provides more reliable volumetric information but is costly and less accessible for routine use. To address these limitations, we propose VIFBA, an ultrasound video-based framework for fetal brain assessment that predicts MRI-derived lateral ventricular volume, classifies VM severity, and identifies potential non-VM fetal brain abnormalities. Our contribution is three-fold. First, we introduce a joint-embedding predictive architecture (JEPA)-inspired tube latent prediction objective that leverages spatio-temporal coherence in ultrasound videos to enhance representation learning. Second, we develop a contrastive cross-modal alignment strategy that transfers structural information from MRI to ultrasound during training, while requiring ultrasound alone at inference. Third, we augment VIFBA with a training-free vision-language model and retrieval augmentation to verify uncertain predictions and identify potential non-VM fetal brain abnormalities. We validated VIFBA on a large dataset comprising 857 cases (3,196 videos) with paired fetal brain ultrasound and MRI examinations. On held-out test data, VIFBA achieved an MAE of 0.5909 mL and Pearson correlation coefficient of 0.9907 for ventricular volume regression, 0.9400 accuracy for VM severity classification, and an F1 score of 0.7764 for multi-abnormality classification, substantially outperforming single-task baselines, video-based strong competitors, and state-of-the-art foundation models. By enabling MRI-informed volumetric assessment from routine ultrasound alone, VIFBA offers a practical and potentially broadly deployable pathway toward accurate and affordable prenatal brain screening.

Abstract

Explore similar work

Cross-Modal Ultrasound-MRI Learning for Fetal Brain Ventricular Volumetry and Abnormality Screening

AWR-Net: Decoupling Anatomy and Appearance for 3D Fetal Brain Ultrasound Synthesis

Benchmarking Ultrasound Foundation Models for Fetal Plane Classification