Learning a Motion Manifold to Constrain Ambiguity in Unsupervised 4D Medical Image Interpolation

Haojin Li; Huazhu Fu; Heng Li; Jiang Liu

cs.CVOct 29, 2026

Learning a Motion Manifold to Constrain Ambiguity in Unsupervised 4D Medical Image Interpolation

Authors: Haojin Li, Huazhu Fu, Heng Li, Jiang Liu

Organizations: Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China · Institute of Advanced Intelligence and Computing, Agency for Science, Technology and Research (A*STAR), Singapore 138632 · Shenzhen University of Advanced Technology, Shenzhen 518107, China

Abstract

Unsupervised 4D medical image interpolation reconstructs intermediate anatomical states from temporally sparse observations without intermediate supervision. Distinct motion fields may attain comparable endpoint agreement while producing different intermediate anatomy, giving rise to motion ambiguity. This ambiguity is difficult to constrain when many independently adjustable motion variables must be estimated from the observed endpoints. We develop a framework that learns a motion manifold to restrict candidate motion fields while retaining sample-specific variation. Cross-Sample Spectral Basis Learning extracts recurring motion structure from endpoint-derived dense estimates and encodes it in a shared spectral basis. Sample-Conditioned Cayley Basis Reorientation uses predicted motion coordinates to rotate the effective motion directions for each endpoint pair without increasing the number of independent coordinates. Their composition defines a nonlinear family of complete velocity fields across endpoint pairs, accommodating anatomical and motion-state differences within a restricted representation. Integrating each predicted field supports bidirectional reconstruction at arbitrary intermediate times. Theoretical analysis establishes sufficient conditions under which endpoint observations distinguish nearby motions within the learned family. Experiments on ACDC and 4D-Lung demonstrate state-of-the-art performance among the evaluated methods trained without intermediate volumes and competitive results against methods using intermediate supervision.

Figures & tables

Fig. 1: Constraining motion ambiguity with a learned motion manifold. (a) The same endpoint volumes can admit different intervening motions. (b) Learning shared motion structure restricts the candidate motions available for endpoint fitting while accommodating sample-specific variation.

Fig. 2: Overview of the proposed framework for unsupervised 4D medical image interpolation. (a) Endpoint-derived dense estimates are aggregated in the DCT domain to learn a shared spectral reference $(\mu,B,\Lambda_{\epsilon})$ . (b) Predicted motion coordinates condition the reorientation of shared directions and their combination into a complete velocity field. (c) Velocity integration, endpoint warping, and time-weighted blending generate intermediate volumes. The shared reference is fixed at inference; dashed arrows indicate training-only paths.

Fig. 3: Conditional reorientation of motion coordinates. The motion coordinates

z

generate a Cayley rotation

Q(z)

that acts on their embedding

Ez

. This changes the effective motion directions for each pair while preserving coordinate length and the number of independent coordinates.

Method	Venue	Sup.	ACDC			4D-Lung			#P (M)
			PSNR ${}_{\text{dB}}^{\uparrow}$	NMI ${}_{\times 10^{-2}}^{\uparrow}$	SSIM ${}_{\times 10^{-2}}^{\uparrow}$	PSNR ${}_{\text{dB}}^{\uparrow}$	NMI ${}_{\times 10^{-2}}^{\uparrow}$	SSIM ${}_{\times 10^{-2}}^{\uparrow}$
VM [ 9 ]	TMI’19	✗	25.694 ±0.074	53.543 ±0.367	94.678 ±0.104	26.077 ±0.066	43.982 ±0.355	86.055 ±0.292	10.044
TM [ 10 ]	MedIA’22	✗	27.125 ±0.200	64.628 ±0.671	96.644 ±0.123	26.683 ±0.051	48.945 ±0.104	84.779 ±0.125	10.804
SVIN [ 5 ]	CVPR’20	✗	28.434 ±0.443	60.178 ±0.335	97.352 ±0.152	27.312 ±0.280	43.171 ±0.392	85.442 ±0.286	12.393
UVI-Net [ 6 ]	CVPR’24	✗	31.903 ±0.118	72.252 ±0.166	98.618 ±0.027	28.948 ±0.277	55.157 ±0.841	88.750 ±0.668	14.273
TTT4MII [ 25 ]	IJCNN’25	✗	32.136 ±0.140	72.514 ±0.174	98.671 ±0.031	28.322 ±0.529	50.590 ±1.563	83.495 ±1.280	12.006

TABLE I: Quantitative comparison on ACDC and 4D-Lung. Results are mean

\pm

standard deviation over five training runs. Sup. indicates intermediate-volume supervision. Model size (#P) is also reported. Best reconstruction scores are in bold.

Fig. 4: Qualitative comparison on ACDC and 4D-Lung. Enlarged regions highlight the ventricular blood-pool boundary and internal low-signal structures in ACDC, and fine elongated intrapulmonary structures in 4D-Lung.

Fig. 5: Shared-basis analyses on ACDC. (a) Retained energy fraction for learned, PCA, frequency-ordered DCT, and random directions. (b) Reconstruction versus

R

d=40

, with and without projection consistency

P

. (c) Change in intermediate-volume MSE after projecting the dense field, plotted against its original MSE; positive

\Delta\mathrm{MSE}

favors projection.

Fig. 6: Reorientation analyses on ACDC. (a) Fixed embedding versus sample-conditioned reorientation at $R=80$ . (b) Global versus sample-conditioned reorientation across low, medium, and high image-change groups at $d=40$ . The $\alpha=0$ column gives absolute PSNR; remaining columns show gains over this reference. (c) Cayley, matrix-exponential, QR-based, and normalized-MLP mappings, with PSNR expressed as deviations from each case’s mean across the four mappings.

$\mathcal{L}_{\mathrm{proj}}$	$\mathcal{L}_{\mathrm{vel}}$	$\mathcal{L}_{\mathrm{coord}}$	PSNR ${}_{\text{dB}}^{\uparrow}$	NMI ${}_{\times 10^{-2}}^{\uparrow}$	SSIM ${}_{\times 10^{-2}}^{\uparrow}$
			30.855	66.625	98.295
✓			31.520	68.754	98.507
	✓	✓	31.907	72.416	98.614
✓		✓	32.030	72.672	98.645
✓	✓		32.045	72.666	98.648
✓	✓	✓	32.205	72.734	98.685

TABLE II: Loss ablation on ACDC.

	Ours	MPVF	Fourier-Net+	DDM
Difference	0.03989	0.04572	0.14694	0.23164

TABLE III: Mean pairwise displacement difference across training runs on ACDC at

t=0.5

(

\mathrm{voxel}

\downarrow

Fig. 7: Temporal reconstruction on ACDC. (a) Ventricular anatomy and enlarged details across intermediate query times. (b) Relative change error across normalized time intervals, evaluating agreement with the reference sequence’s image evolution. Lower values indicate more faithful temporal reconstruction.

Fig. 8: Segmentation consistency on ACDC. (a) Dice agreement between masks predicted from reconstructed and reference volumes, averaged over test volumes including endpoints. RV, MYO, and LV denote the right ventricular cavity, left ventricular myocardium, and left ventricular cavity. (b) Reference and reconstructed images with their model-predicted masks.

Method	#P (M)	Est. train time (s/epoch) $\downarrow$	QPS $\uparrow$
Ours	$15.100$	$19.18$	$40.52$
MPVF	$14.536$	$105.14$	$4.61$
DDM	$23.164$	$18.42$	$14.04$
LDDM	$15.653$	$51.52$	$2.72$
Fourier-Net+	$12.414$	$1.84$	$121.16$

TABLE IV: Computational cost on ACDC. QPS counts complete 3D midpoint queries. MPVF training time includes all training stages.

4D Reconstruction

4D reconstruction recovers changing 3D scenes, shapes, and motion over time from images, videos, or other sparse measurements.

See all 140 papers

Low-Rank Velocity Fields as a Structural Prior for Unsupervised 4D Medical Image Interpolation
Aug 25, 2026Haojin Li, Hengzhuo Wang, Chang Liu +3Medical Image Synthesis 4D Reconstruction
Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China · Faculty of Biomedical Engineering, Shenzhen University of Advanced Technology, Shenzhen, China · School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China +2
Phase-Aligned Finite-Fourier Periodic Deformation for 4D Medical Image Interpolation
Aug 25, 2026Haojin Li, Hengzhuo Wang, Zhiheng Ma +34D Reconstruction Deformable Image Registration
School of Computer Science and Engineering Beihang University Beijing, China
UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images
Feb 27, 2026Junhwa Hur, Charles Herrmann, Songyou Peng +44D Reconstruction Dynamic Scene Reconstruction
Google · Princeton University · Harvard University

Aug 25, 2026J/K move · Enter open · S save

Low-Rank Velocity Fields as a Structural Prior for Unsupervised 4D Medical Image Interpolation

Haojin Li, Hengzhuo Wang, Chang Liu, Zhiheng Ma+2

Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China · Faculty of Biomedical Engineering, Shenzhen University of Advanced Technology, Shenzhen, China · School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China+2

Medical Image Synthesis 4D Reconstruction Deformable Image Registration

Open paper AskPDF arXiv

Endpoint-only unsupervised 4D medical image interpolation synthesizes intermediate volumes from sparsely sampled sequences with only the start and end volumes available for training; however, this weakly constrained setting often yields intermediates with unstable boundaries and non-physiological motion, limiting interpretability and downstream analysis. We propose low-rank velocity fields as a structural prior, constraining motion to a structured Tucker low-rank velocity field space that decomposes motion into globally shared spatial bases and a compact sample-specific core, thereby encouraging spatially correlated, anatomy-consistent deformation while suppressing voxel-wise high-frequency artifacts. To capture global coordination and local non-rigid details, we model motion in a coarse-to-fine multi-scale scheme and compose scale-wise deformations at inference to synthesize volumes at arbitrary times. We further provide a theoretical analysis showing that, under Tucker parameterization, low-rank parameters control the smoothness energy of the velocity field, explaining why low-rank modeling promotes smoother motion. Experiments on ACDC and 4D-Lung demonstrate state-of-the-art performance, remaining competitive with methods trained with intermediate-frame supervision, and producing intermediates with improved structural coherence and more stable anatomical contours.

Abstract

4D Reconstruction

Explore similar work

Low-Rank Velocity Fields as a Structural Prior for Unsupervised 4D Medical Image Interpolation

Phase-Aligned Finite-Fourier Periodic Deformation for 4D Medical Image Interpolation

UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images