SphereFlow: Missing Modality Imputation via Geometric Transport on Hypersphere

Huihan Wang; Juncheng Wang; Yuxiang Feng; Wenlong Hou; Guoqi Yu; Chao Xu; Yang Liu; Baigui Sun

cs.CVNov 3, 2026

SphereFlow: Missing Modality Imputation via Geometric Transport on Hypersphere

Authors: Huihan Wang, Juncheng Wang, Yuxiang Feng, Wenlong Hou, Guoqi Yu, Chao Xu, Yang Liu, Baigui Sun,

Organizations: Department of Biomedical Engineering and Sports Technology (BEST), The Hong Kong Polytechnic University, Hong Kong, China · Zhejiang University, Hangzhou, China · IROOTECH TECHNOLOGY, China · Wolf 1069 b Lab, Sany Group, China ·

Abstract

Missing modality imputation aims to synthesize unobserved modalities from observed ones. Existing methods treat each modality as living on its own latent manifold, forcing the generator to learn a long, unstructured jump between them. We present SphereFlow, which takes a fundamentally different view: by projecting all modalities into a shared hyperspherical latent space via a frozen self-supervised encoder, cross-modal imputation reduces to short-range geometric transport between nearby points on the sphere. Realizing this idea with flow matching, however, exposes two geometric conflicts: the standard Gaussian noise source is catastrophically mismatched with the unit-norm data manifold, and the linear interpolation path departs the sphere surface into a region where the decoder has no training signal. We resolve both with simple, geometry-aware modifications: replacing the noise source with the observed-modality latent so that the velocity field learns only the cross-modal displacement, and training the decoder with a time-weighted loss that extends its domain to the near-surface region traversed by the flow. We prove that this data-as-source formulation reduces the worst-case off-manifold deviation from a quantity that grows with the latent dimension to a small, dimension-independent bound governed by inter-modality similarity. Experiments on BraTS multi-modal MRI imputation and CT-MRI translation demonstrate state-of-the-art generation fidelity across all settings, while downstream brain tumor segmentation with our imputed data closes roughly eighty percent of the gap to the full-modality oracle. Code is available at https://github.com/WTHH031230/SphereFlow.

Figures & tables

Figure 1: Two views of cross-modal imputation. (a) Task: given a subset of observed modalities, synthesize the missing one. (b) Conventional view: modalities are encoded into disjoint latent manifolds (pixel space or per-modality VAE), so cross-modal generation is a long jump between them. (c) Our view: a frozen DINOv2 encoder maps every modality to a single shared hypersphere $\mathcal{S}^{d-1}$ , on which paired latents $(z_{s},z_{t})$ sit close together; imputation becomes short-range transport along the sphere.

Figure 2: Pilot study: tokenizer comparison on BraTS. (a) Cross-modal compactness: 2-D projection of paired $T_{1}$ (blue) and $T_{1c}$ (red) latents under three tokenizers, with thin lines connecting same-subject pairs; the title reports the ratio $\bar{d}_{\text{pair}}/\bar{d}_{\text{cross}}$ (lower is better, see § 2.1 ). (b) Geometric regularity: empirical distribution of per-token latent norms $\|z\|$ .

Figure 3: Overview of SphereFlow . (I) Tokenizer training (§ 3.4 ): a frozen DINOv2 encoder maps source and target images onto the hypersphere $\mathcal{S}^{d-1}$ . Modality-specific decoders $D^{(s)}\!,D^{(t)}$ are trained to reconstruct both on-manifold latents ( $z_{s},z_{t}$ ) and off-manifold interpolants ( $z_{\tau}$ ) via the time-weighted expansion loss (Eq. 9 ). (II) Generator training (§ 3.3 ): a Diffusion Transformer learns the cross-modal velocity field $v_{\theta}(z_{\tau},\tau)\mapsto u_{\tau}=z_{t}-z_{s}$ with $z_{\tau}$ as input, using the paired source latent as the flow origin instead of Gaussian noise. (III) Geometric conflict & SphereFlow : standard flow matching (gray trajectory) departs from Gaussian noise at $\|\varepsilon\|\!\approx\!\sqrt{d}\gg 1$ and must traverse a vast off-manifold void to reach the DINO hypersphere ( $\|z\|{=}1$ ). SphereFlow (orange trajectory) transports directly from $z_{s}$ to $z_{t}$ along a short near-surface chord, with bounded off-manifold deviation (Proposition 1 ).

Figure 4: Main-results overview. Top: PSNR/Dice (

\uparrow

); bottom: rFID/HD95 (

\downarrow

) across four generation tasks and segmentation. SphereFlow achieves the best or near-best score on every panel.

Modalities	PSNR ↑								SSIM ↑
Modalities	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours
$T_{1}\rightarrow T_{1c}$	30.50	29.93	30.75	31.38	29.84	32.79	3 2.85	33.53	0.9185	0.9075	0.9077	0.9235	0.8840	0.9384	0 .9405	0.9414
$T_{1}\rightarrow T_{2}$	27.91	29.68	29.62	29.29	28.87	3 1.76	30.77	32.68	0.8985	0.9035	0.8916	0.9010	0.8675	0 .9326	0.9289	0.9390
$T_{1}\rightarrow T_{f}$	33.16	34.92	34.72	33.81	32.88	3 6.06	35.36	37.35	0.9220	0.9317	0.9244	0.9271	0.8937	0 .9470	0.9458	0.9529
$T_{1c}\rightarrow T_{1}$	34.82	36.81	33.65	33.83	33.50	3 7.21	36.34	37.87	0.9421	0.9517	0.9235	0.9392	0.9067	0 .9586	0.9575	0.9612
$T_{1c}\rightarrow T_{2}$	28.86	31.15	29.45	28.80	29.05	3 1.80	30.58	32.67	0.9019	0.9183	0.8872	0.8952	0.8683	0 .9305	0.9252	0.9379

Table 1: 1-to-1 generation on BraTS (all 12 directions).

	Synthesis (N-to-N avg)				Segmentation (avg)
Method	PSNR $\uparrow$	SSIM $\uparrow$	rFID $\downarrow$	IS $\uparrow$	Dice $\uparrow$	HD95 $\downarrow$	IoU $\uparrow$	Sens $\uparrow$
Zero-fill	—	—	—	—	0.3185	169.40	0.2693	0.3260
MMSyn	31.81	0.9213	37.12	2.64	0.5546	57.73	0.4641	0.5553
MMGAN	33.81	0.9339	93.12	2.51	0.5657	55.39	0.4877	0.5524
MMT	32.40	0.9125	135.77	2.48	0.5450	4 6.27	0.4633	0.5374
ModImp	32.23	0.9226	123.87	2.42	0.5627	59.44	0.4832	0.5574

Table 2: N-to-N synthesis and segmentation averages on BraTS (14 input combinations). Full oracle = all ground-truth modalities. Per-combination results in Appendix Tables 6 , 7 .

Table 7

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: PSNR vs. Euler steps (

T_{1}{\to}T_{1c}

). Gaussian FM collapses at

N{=}1

; SphereFlow starts at

34.3

dB. Both saturate by

N{=}2

–

8

; SphereFlow retains a

{\sim}1.7

dB lead.

Modalities				PSNR $\uparrow$								SSIM $\uparrow$
$T_{1}$	$T_{2}$	$T_{1c}$	$T_{f}$	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours
$\bullet$	$\circ$	$\circ$	$\circ$	30.52	31.51	31.65	31.33	30.53	3 3.54	32.99	34.52	0.9130	0.9142	0.9076	0.9169	0.8818	0 .9393	0.9384	0.9444
$\circ$	$\bullet$	$\circ$	$\circ$	29.03	31.38	31.49	28.88	30.10	3 4.20	30.93	35.26	0.9003	0.9161	0.9015	0.8978	0.8792	0 .9441	0.9222	0.9498
$\circ$	$\circ$	$\bullet$	$\circ$	32.29	34.29	32.21	32.07	31.99	3 4.87	34.05	36.01	0.9196	0.9342	0.9083	0.9193	0.8907	0 .9454	0.9430	0.9509
$\circ$	$\circ$	$\circ$	$\bullet$	28.45	30.47	30.38	30.04	29.48	3 2.28	31.39	33.51	0.8959	0.9147	0.9003	0.9013	0.8778	0 .9336	0.9302	0.9414
$\circ$	$\circ$	$\bullet$	$\bullet$	32.23	34.63	32.28	32.48	32.19	3 5.72	34.18	36.35	0.9245	0.9407	0.9124	0.9259	0.8947	0 .9547	0.9475	0.9569

Appendix

Table 6: Full N-to-N synthesis results on BraTS (all 14 input combinations).

Modalities				Dice									HD95
$T_{1}$	$T_{2}$	$T_{1c}$	$T_{f}$	Zero	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours	Zero	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours
$\bullet$	$\circ$	$\circ$	$\circ$	0.0094	0.0686	0.1852	0.1910	0.0851	0.1558	0.1570	0 .2094	0.5710	316.87	237.74	167.35	9 4.73	232.55	140.21	114.11	169.75	35.75
$\circ$	$\bullet$	$\circ$	$\circ$	0.0133	0.3025	0.2740	0.2696	0.2679	0.1796	0.2662	0 .3637	0.6477	326.12	6 5.63	94.85	94.31	77.71	131.71	71.67	91.35	23.15
$\circ$	$\circ$	$\bullet$	$\circ$	0.0257	0.6412	0.6029	0.5389	0.6125	0.5467	0.3932	0 .6771	0.7811	301.14	3 5.79	66.93	58.71	43.21	62.99	75.44	38.97	22.09
$\circ$	$\circ$	$\circ$	$\bullet$	0.3695	0.4096	0.3583	0.3395	0 .4633	0.3508	0.4260	0.3946	0.6574	105.40	68.57	75.06	62.55	55.85	77.51	5 1.32	97.50	22.16
$\circ$	$\circ$	$\bullet$	$\bullet$	0.7386	0.8284	0.8348	0.8173	0.8254	0.7897	0.8195	0 .8421	0.8512	28.89	13.31	1 3.07	17.19	14.66	20.98	14.49	13.71	12.05

Appendix

Table 7: Full downstream segmentation results on BraTS (all 14 input combinations).

Figure 6: Qualitative comparison of synthesized images and error maps. Top two rows: $T_{1}{+}T_{2}{+}T_{f}\to T_{1c}$ on BraTS. Row 1 shows synthesized $T_{1c}$ images; row 2 shows the corresponding per-pixel absolute error (jet colormap, darker blue = lower error). Bottom two rows: $\mathrm{MRI}\to\mathrm{CT}$ on SynthRAD2023, with the same layout. Columns correspond to seven baselines, SphereFlow , and the ground truth (GT).

Figure 7: Qualitative comparison of downstream segmentation. Setting: $T_{1}{+}T_{2}{+}T_{f}\to T_{1c}$ on BraTS, then feeding all four modalities (three real + one imputed) to a 3D U-Net trained on full-modality data. Row 1: synthesized $T_{1c}$ images. Row 2: segmentation overlays showing whole tumor (WT, yellow), tumor core (TC, blue), and enhancing tumor (ET, red). The rightmost column is the ground truth.

Missing-Modality Learning

Missing-modality learning develops multimodal models that remain useful when input streams are absent, corrupted, or available only during training.

See all 181 papers

Jul 21, 2026

Posterior Samplings are Missing Modalities Generators for Medical Image Translation

Magnetic resonance imaging comes in various modality contrasts that provide complementary anatomical and pathological information. Complete multimodal acquisitions are often unavailable due to time and protocol constraints. This leads to real-world datasets with missing modalities, where conventional medical image translation methods are typically limited to fixed source-target settings or require retraining for each observed source-target pair. We propose a unified framework that formulates missing-modality generation as a linear inverse problem under a joint distribution and solves it via posterior sampling with a flow matching model. By learning a joint prior over the complete modality set, our method can reconstruct arbitrary missing modalities at inference time by guiding the sampling trajectory to enforce measurement consistency with observed modalities. We further mitigate inter-modality error propagation in multi-target generation by adopting a many-to-one sampling strategy. Experiments on BraTS and IXI datasets show that our method achieves the best performance over baselines across most missing-modality scenarios. In downstream tumor segmentation, synthesized images from our method result in higher segmentation performance, indicating better preservation of clinically relevant structures.

Jonghun Kim

Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea

Missing-Modality Learning Image-to-Image Translation Posterior Sampling Medical Image Synthesis Medical Image Modality Translation

Ask PDF

Oct 29, 2026

MOSAIC-Flow: Orthogonal-Axis and Status-Conditioned Fusion for Missing-Sequence Completion in Brain Tumor MRI

Missing-sequence completion is important for comprehensive brain tumor characterization, but cross-modal fusion can obscure sequence-specific contrast and lose fine-grained three-dimensional tumor detail. To address this challenge, we introduce Modality-aware Orthogonal-axis Status-conditioned Attentive Integration for Completion (MOSAIC-Flow), a unified framework for missing-sequence completion that learns a target-conditioned velocity field while keeping the seven acquired volumes fixed throughout the transport. Specifically, modality-aware stems independently encode each sequence, and orthogonal-axis refinement captures axial, coronal, and sagittal dependencies while retaining the three-dimensional representation. Status-conditioned prototype fusion uses compact modality and availability tokens to route and modulate pooled queries from the fused visual representation, enabling target-adaptive conditioning without dense voxel-to-voxel attention. A conditional flow matching objective transports target-channel noise toward the single missing sequence, while four Heun trajectories are averaged to produce the completion. Evaluation across structural, diffusion, and contrast-enhanced MRI in UCSF-PDGM demonstrates consistent whole-brain and tumor-region fidelity across all eight single-target completion tasks, with the strongest performance on structural sequences. Code is available at https://github.com/CUHK-AIM-Group/MOSAIC-FLOW.

Pengli Zhu, Zhihao Peng, Yixuan Yuan

Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China

Brain MRI 3D Medical Imaging Multimodal Fusion

Ask PDF

Apr 25, 2026

Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning

Multimodal Federated Learning (MMFL) enables privacy-preserving collaborative training, but real-world clinical applications often suffer from within-modality missingness caused by sensor intermittency or irregular sampling. Existing methods implicitly represent unobserved data via architectural alignment or missing embeddings, often failing to recover the true distribution and yielding sub-optimal performance. We propose CondI, a federated framework explicitly addressing this missingness using conditional diffusion models. CondI employs a two-phase training pipeline: first, imputing unobserved temporal components using available multimodal context and conditional embeddings; second, optimizing modality-specific extractors and joint embedding spaces. During inference, imputed raw data pass through trained extractors to generate robust features, providing a holistic representation for downstream tasks. Explicit data imputation ensures models operate on complete semantic structures, significantly enhancing resilience against severe data incompleteness. Experiments on three clinical datasets (PTB-XL, SLEEP-EDF, MIMIC-IV) demonstrate CondI achieves comparable results to state-of-the-art baselines. Code: https://github.com/ZhengWugeng/CondI

Wugeng Zheng, Ziwen Kan, Katie Wang +2

University of Central Florida · Independent Researcher

Multimodal Federated Learning Time Series Imputation Incomplete Data Imputation Learning with Missing Data Federated Learning

Ask PDF

Abstract

Missing-Modality Learning

Explore similar work

Posterior Samplings are Missing Modalities Generators for Medical Image Translation

MOSAIC-Flow: Orthogonal-Axis and Status-Conditioned Fusion for Missing-Sequence Completion in Brain Tumor MRI

Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning