SphereFlow: Missing Modality Imputation via Geometric Transport on Hypersphere

Huihan Wang; Juncheng Wang; Yuxiang Feng; Wenlong Hou; Guoqi Yu; Chao Xu; Yang Liu; Baigui Sun

cs.CVNov 3, 2026

SphereFlow: Missing Modality Imputation via Geometric Transport on Hypersphere

Authors: Huihan Wang, Juncheng Wang, Yuxiang Feng, Wenlong Hou, Guoqi Yu, Chao Xu, Yang Liu, Baigui Sun,

Organizations: Department of Biomedical Engineering and Sports Technology (BEST), The Hong Kong Polytechnic University, Hong Kong, China · Zhejiang University, Hangzhou, China · IROOTECH TECHNOLOGY, China · Wolf 1069 b Lab, Sany Group, China ·

Abstract

Missing modality imputation aims to synthesize unobserved modalities from observed ones. Existing methods treat each modality as living on its own latent manifold, forcing the generator to learn a long, unstructured jump between them. We present SphereFlow, which takes a fundamentally different view: by projecting all modalities into a shared hyperspherical latent space via a frozen self-supervised encoder, cross-modal imputation reduces to short-range geometric transport between nearby points on the sphere. Realizing this idea with flow matching, however, exposes two geometric conflicts: the standard Gaussian noise source is catastrophically mismatched with the unit-norm data manifold, and the linear interpolation path departs the sphere surface into a region where the decoder has no training signal. We resolve both with simple, geometry-aware modifications: replacing the noise source with the observed-modality latent so that the velocity field learns only the cross-modal displacement, and training the decoder with a time-weighted loss that extends its domain to the near-surface region traversed by the flow. We prove that this data-as-source formulation reduces the worst-case off-manifold deviation from a quantity that grows with the latent dimension to a small, dimension-independent bound governed by inter-modality similarity. Experiments on BraTS multi-modal MRI imputation and CT-MRI translation demonstrate state-of-the-art generation fidelity across all settings, while downstream brain tumor segmentation with our imputed data closes roughly eighty percent of the gap to the full-modality oracle. Code is available at https://github.com/WTHH031230/SphereFlow.

Figures & tables

Figure 1: Two views of cross-modal imputation. (a) Task: given a subset of observed modalities, synthesize the missing one. (b) Conventional view: modalities are encoded into disjoint latent manifolds (pixel space or per-modality VAE), so cross-modal generation is a long jump between them. (c) Our view: a frozen DINOv2 encoder maps every modality to a single shared hypersphere $\mathcal{S}^{d-1}$ , on which paired latents $(z_{s},z_{t})$ sit close together; imputation becomes short-range transport along the sphere.

Figure 2: Pilot study: tokenizer comparison on BraTS. (a) Cross-modal compactness: 2-D projection of paired $T_{1}$ (blue) and $T_{1c}$ (red) latents under three tokenizers, with thin lines connecting same-subject pairs; the title reports the ratio $\bar{d}_{\text{pair}}/\bar{d}_{\text{cross}}$ (lower is better, see § 2.1 ). (b) Geometric regularity: empirical distribution of per-token latent norms $\|z\|$ .

Figure 3: Overview of SphereFlow . (I) Tokenizer training (§ 3.4 ): a frozen DINOv2 encoder maps source and target images onto the hypersphere $\mathcal{S}^{d-1}$ . Modality-specific decoders $D^{(s)}\!,D^{(t)}$ are trained to reconstruct both on-manifold latents ( $z_{s},z_{t}$ ) and off-manifold interpolants ( $z_{\tau}$ ) via the time-weighted expansion loss (Eq. 9 ). (II) Generator training (§ 3.3 ): a Diffusion Transformer learns the cross-modal velocity field $v_{\theta}(z_{\tau},\tau)\mapsto u_{\tau}=z_{t}-z_{s}$ with $z_{\tau}$ as input, using the paired source latent as the flow origin instead of Gaussian noise. (III) Geometric conflict & SphereFlow : standard flow matching (gray trajectory) departs from Gaussian noise at $\|\varepsilon\|\!\approx\!\sqrt{d}\gg 1$ and must traverse a vast off-manifold void to reach the DINO hypersphere ( $\|z\|{=}1$ ). SphereFlow (orange trajectory) transports directly from $z_{s}$ to $z_{t}$ along a short near-surface chord, with bounded off-manifold deviation (Proposition 1 ).

Figure 4: Main-results overview. Top: PSNR/Dice (

\uparrow

); bottom: rFID/HD95 (

\downarrow

) across four generation tasks and segmentation. SphereFlow achieves the best or near-best score on every panel.

Modalities	PSNR ↑								SSIM ↑
Modalities	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours
$T_{1}\rightarrow T_{1c}$	30.50	29.93	30.75	31.38	29.84	32.79	3 2.85	33.53	0.9185	0.9075	0.9077	0.9235	0.8840	0.9384	0 .9405	0.9414
$T_{1}\rightarrow T_{2}$	27.91	29.68	29.62	29.29	28.87	3 1.76	30.77	32.68	0.8985	0.9035	0.8916	0.9010	0.8675	0 .9326	0.9289	0.9390
$T_{1}\rightarrow T_{f}$	33.16	34.92	34.72	33.81	32.88	3 6.06	35.36	37.35	0.9220	0.9317	0.9244	0.9271	0.8937	0 .9470	0.9458	0.9529
$T_{1c}\rightarrow T_{1}$	34.82	36.81	33.65	33.83	33.50	3 7.21	36.34	37.87	0.9421	0.9517	0.9235	0.9392	0.9067	0 .9586	0.9575	0.9612
$T_{1c}\rightarrow T_{2}$	28.86	31.15	29.45	28.80	29.05	3 1.80	30.58	32.67	0.9019	0.9183	0.8872	0.8952	0.8683	0 .9305	0.9252	0.9379

Table 1: 1-to-1 generation on BraTS (all 12 directions).

	Synthesis (N-to-N avg)				Segmentation (avg)
Method	PSNR $\uparrow$	SSIM $\uparrow$	rFID $\downarrow$	IS $\uparrow$	Dice $\uparrow$	HD95 $\downarrow$	IoU $\uparrow$	Sens $\uparrow$
Zero-fill	—	—	—	—	0.3185	169.40	0.2693	0.3260
MMSyn	31.81	0.9213	37.12	2.64	0.5546	57.73	0.4641	0.5553
MMGAN	33.81	0.9339	93.12	2.51	0.5657	55.39	0.4877	0.5524
MMT	32.40	0.9125	135.77	2.48	0.5450	4 6.27	0.4633	0.5374
ModImp	32.23	0.9226	123.87	2.42	0.5627	59.44	0.4832	0.5574

Table 2: N-to-N synthesis and segmentation averages on BraTS (14 input combinations). Full oracle = all ground-truth modalities. Per-combination results in Appendix Tables 6 , 7 .

Table 7

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: PSNR vs. Euler steps (

T_{1}{\to}T_{1c}

). Gaussian FM collapses at

N{=}1

; SphereFlow starts at

34.3

dB. Both saturate by

N{=}2

–

8

; SphereFlow retains a

{\sim}1.7

dB lead.

Modalities				PSNR $\uparrow$								SSIM $\uparrow$
$T_{1}$	$T_{2}$	$T_{1c}$	$T_{f}$	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours
$\bullet$	$\circ$	$\circ$	$\circ$	30.52	31.51	31.65	31.33	30.53	3 3.54	32.99	34.52	0.9130	0.9142	0.9076	0.9169	0.8818	0 .9393	0.9384	0.9444
$\circ$	$\bullet$	$\circ$	$\circ$	29.03	31.38	31.49	28.88	30.10	3 4.20	30.93	35.26	0.9003	0.9161	0.9015	0.8978	0.8792	0 .9441	0.9222	0.9498
$\circ$	$\circ$	$\bullet$	$\circ$	32.29	34.29	32.21	32.07	31.99	3 4.87	34.05	36.01	0.9196	0.9342	0.9083	0.9193	0.8907	0 .9454	0.9430	0.9509
$\circ$	$\circ$	$\circ$	$\bullet$	28.45	30.47	30.38	30.04	29.48	3 2.28	31.39	33.51	0.8959	0.9147	0.9003	0.9013	0.8778	0 .9336	0.9302	0.9414
$\circ$	$\circ$	$\bullet$	$\bullet$	32.23	34.63	32.28	32.48	32.19	3 5.72	34.18	36.35	0.9245	0.9407	0.9124	0.9259	0.8947	0 .9547	0.9475	0.9569

Appendix

Table 6: Full N-to-N synthesis results on BraTS (all 14 input combinations).

Modalities				Dice									HD95
$T_{1}$	$T_{2}$	$T_{1c}$	$T_{f}$	Zero	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours	Zero	MMSyn	MMGAN	MMT	ModImp	M2DN	Zhang	CodeBrain	Ours
$\bullet$	$\circ$	$\circ$	$\circ$	0.0094	0.0686	0.1852	0.1910	0.0851	0.1558	0.1570	0 .2094	0.5710	316.87	237.74	167.35	9 4.73	232.55	140.21	114.11	169.75	35.75
$\circ$	$\bullet$	$\circ$	$\circ$	0.0133	0.3025	0.2740	0.2696	0.2679	0.1796	0.2662	0 .3637	0.6477	326.12	6 5.63	94.85	94.31	77.71	131.71	71.67	91.35	23.15
$\circ$	$\circ$	$\bullet$	$\circ$	0.0257	0.6412	0.6029	0.5389	0.6125	0.5467	0.3932	0 .6771	0.7811	301.14	3 5.79	66.93	58.71	43.21	62.99	75.44	38.97	22.09
$\circ$	$\circ$	$\circ$	$\bullet$	0.3695	0.4096	0.3583	0.3395	0 .4633	0.3508	0.4260	0.3946	0.6574	105.40	68.57	75.06	62.55	55.85	77.51	5 1.32	97.50	22.16
$\circ$	$\circ$	$\bullet$	$\bullet$	0.7386	0.8284	0.8348	0.8173	0.8254	0.7897	0.8195	0 .8421	0.8512	28.89	13.31	1 3.07	17.19	14.66	20.98	14.49	13.71	12.05

Appendix

Table 7: Full downstream segmentation results on BraTS (all 14 input combinations).

Figure 6: Qualitative comparison of synthesized images and error maps. Top two rows: $T_{1}{+}T_{2}{+}T_{f}\to T_{1c}$ on BraTS. Row 1 shows synthesized $T_{1c}$ images; row 2 shows the corresponding per-pixel absolute error (jet colormap, darker blue = lower error). Bottom two rows: $\mathrm{MRI}\to\mathrm{CT}$ on SynthRAD2023, with the same layout. Columns correspond to seven baselines, SphereFlow , and the ground truth (GT).

Figure 7: Qualitative comparison of downstream segmentation. Setting: $T_{1}{+}T_{2}{+}T_{f}\to T_{1c}$ on BraTS, then feeding all four modalities (three real + one imputed) to a 3D U-Net trained on full-modality data. Row 1: synthesized $T_{1c}$ images. Row 2: segmentation overlays showing whole tumor (WT, yellow), tumor core (TC, blue), and enhancing tumor (ET, red). The rightmost column is the ground truth.

Missing-Modality Learning

Missing-modality learning develops multimodal models that remain useful when input streams are absent, corrupted, or available only during training.

See all 181 papers

Posterior Samplings are Missing Modalities Generators for Medical Image Translation
Jul 21, 2026Jonghun KimMissing-Modality Learning Image-to-Image Translation
Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea
MOSAIC-Flow: Orthogonal-Axis and Status-Conditioned Fusion for Missing-Sequence Completion in Brain Tumor MRI
Oct 29, 2026Pengli Zhu, Zhihao Peng, Yixuan YuanBrain MRI 3D Medical Imaging
Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong, China
Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning
Apr 25, 2026Wugeng Zheng, Ziwen Kan, Katie Wang +2Multimodal Federated Learning Time Series Imputation
University of Central Florida · Independent Researcher

Jul 21, 2026J/K move · Enter open · S save

Posterior Samplings are Missing Modalities Generators for Medical Image Translation

Jonghun Kim

Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea

Missing-Modality Learning Image-to-Image Translation Posterior Sampling Medical Image Synthesis Medical Image Modality Translation

Open paper AskPDF arXiv

Magnetic resonance imaging comes in various modality contrasts that provide complementary anatomical and pathological information. Complete multimodal acquisitions are often unavailable due to time and protocol constraints. This leads to real-world datasets with missing modalities, where conventional medical image translation methods are typically limited to fixed source-target settings or require retraining for each observed source-target pair. We propose a unified framework that formulates missing-modality generation as a linear inverse problem under a joint distribution and solves it via posterior sampling with a flow matching model. By learning a joint prior over the complete modality set, our method can reconstruct arbitrary missing modalities at inference time by guiding the sampling trajectory to enforce measurement consistency with observed modalities. We further mitigate inter-modality error propagation in multi-target generation by adopting a many-to-one sampling strategy. Experiments on BraTS and IXI datasets show that our method achieves the best performance over baselines across most missing-modality scenarios. In downstream tumor segmentation, synthesized images from our method result in higher segmentation performance, indicating better preservation of clinically relevant structures.

Abstract

Missing-Modality Learning

Explore similar work

Posterior Samplings are Missing Modalities Generators for Medical Image Translation

MOSAIC-Flow: Orthogonal-Axis and Status-Conditioned Fusion for Missing-Sequence Completion in Brain Tumor MRI

Conditional Imputation for Within-Modality Missingness in Multi-Modal Federated Learning