Learning a Motion Manifold to Constrain Ambiguity in Unsupervised 4D Medical Image Interpolation

Haojin Li; Huazhu Fu; Heng Li; Jiang Liu

cs.CVOct 29, 2026

Learning a Motion Manifold to Constrain Ambiguity in Unsupervised 4D Medical Image Interpolation

Authors: Haojin Li, Huazhu Fu, Heng Li, Jiang Liu

Organizations: Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China · Institute of Advanced Intelligence and Computing, Agency for Science, Technology and Research (A*STAR), Singapore 138632 · Shenzhen University of Advanced Technology, Shenzhen 518107, China

Abstract

Unsupervised 4D medical image interpolation reconstructs intermediate anatomical states from temporally sparse observations without intermediate supervision. Distinct motion fields may attain comparable endpoint agreement while producing different intermediate anatomy, giving rise to motion ambiguity. This ambiguity is difficult to constrain when many independently adjustable motion variables must be estimated from the observed endpoints. We develop a framework that learns a motion manifold to restrict candidate motion fields while retaining sample-specific variation. Cross-Sample Spectral Basis Learning extracts recurring motion structure from endpoint-derived dense estimates and encodes it in a shared spectral basis. Sample-Conditioned Cayley Basis Reorientation uses predicted motion coordinates to rotate the effective motion directions for each endpoint pair without increasing the number of independent coordinates. Their composition defines a nonlinear family of complete velocity fields across endpoint pairs, accommodating anatomical and motion-state differences within a restricted representation. Integrating each predicted field supports bidirectional reconstruction at arbitrary intermediate times. Theoretical analysis establishes sufficient conditions under which endpoint observations distinguish nearby motions within the learned family. Experiments on ACDC and 4D-Lung demonstrate state-of-the-art performance among the evaluated methods trained without intermediate volumes and competitive results against methods using intermediate supervision.

Figures & tables

Fig. 1: Constraining motion ambiguity with a learned motion manifold. (a) The same endpoint volumes can admit different intervening motions. (b) Learning shared motion structure restricts the candidate motions available for endpoint fitting while accommodating sample-specific variation.

Fig. 2: Overview of the proposed framework for unsupervised 4D medical image interpolation. (a) Endpoint-derived dense estimates are aggregated in the DCT domain to learn a shared spectral reference $(\mu,B,\Lambda_{\epsilon})$ . (b) Predicted motion coordinates condition the reorientation of shared directions and their combination into a complete velocity field. (c) Velocity integration, endpoint warping, and time-weighted blending generate intermediate volumes. The shared reference is fixed at inference; dashed arrows indicate training-only paths.

Fig. 3: Conditional reorientation of motion coordinates. The motion coordinates

z

generate a Cayley rotation

Q(z)

that acts on their embedding

Ez

. This changes the effective motion directions for each pair while preserving coordinate length and the number of independent coordinates.

Method	Venue	Sup.	ACDC			4D-Lung			#P (M)
			PSNR ${}_{\text{dB}}^{\uparrow}$	NMI ${}_{\times 10^{-2}}^{\uparrow}$	SSIM ${}_{\times 10^{-2}}^{\uparrow}$	PSNR ${}_{\text{dB}}^{\uparrow}$	NMI ${}_{\times 10^{-2}}^{\uparrow}$	SSIM ${}_{\times 10^{-2}}^{\uparrow}$
VM [ 9 ]	TMI’19	✗	25.694 ±0.074	53.543 ±0.367	94.678 ±0.104	26.077 ±0.066	43.982 ±0.355	86.055 ±0.292	10.044
TM [ 10 ]	MedIA’22	✗	27.125 ±0.200	64.628 ±0.671	96.644 ±0.123	26.683 ±0.051	48.945 ±0.104	84.779 ±0.125	10.804
SVIN [ 5 ]	CVPR’20	✗	28.434 ±0.443	60.178 ±0.335	97.352 ±0.152	27.312 ±0.280	43.171 ±0.392	85.442 ±0.286	12.393
UVI-Net [ 6 ]	CVPR’24	✗	31.903 ±0.118	72.252 ±0.166	98.618 ±0.027	28.948 ±0.277	55.157 ±0.841	88.750 ±0.668	14.273
TTT4MII [ 25 ]	IJCNN’25	✗	32.136 ±0.140	72.514 ±0.174	98.671 ±0.031	28.322 ±0.529	50.590 ±1.563	83.495 ±1.280	12.006

TABLE I: Quantitative comparison on ACDC and 4D-Lung. Results are mean

\pm

standard deviation over five training runs. Sup. indicates intermediate-volume supervision. Model size (#P) is also reported. Best reconstruction scores are in bold.

Fig. 4: Qualitative comparison on ACDC and 4D-Lung. Enlarged regions highlight the ventricular blood-pool boundary and internal low-signal structures in ACDC, and fine elongated intrapulmonary structures in 4D-Lung.

Fig. 5: Shared-basis analyses on ACDC. (a) Retained energy fraction for learned, PCA, frequency-ordered DCT, and random directions. (b) Reconstruction versus

R

d=40

, with and without projection consistency

P

. (c) Change in intermediate-volume MSE after projecting the dense field, plotted against its original MSE; positive

\Delta\mathrm{MSE}

favors projection.

Fig. 6: Reorientation analyses on ACDC. (a) Fixed embedding versus sample-conditioned reorientation at $R=80$ . (b) Global versus sample-conditioned reorientation across low, medium, and high image-change groups at $d=40$ . The $\alpha=0$ column gives absolute PSNR; remaining columns show gains over this reference. (c) Cayley, matrix-exponential, QR-based, and normalized-MLP mappings, with PSNR expressed as deviations from each case’s mean across the four mappings.

$\mathcal{L}_{\mathrm{proj}}$	$\mathcal{L}_{\mathrm{vel}}$	$\mathcal{L}_{\mathrm{coord}}$	PSNR ${}_{\text{dB}}^{\uparrow}$	NMI ${}_{\times 10^{-2}}^{\uparrow}$	SSIM ${}_{\times 10^{-2}}^{\uparrow}$
			30.855	66.625	98.295
✓			31.520	68.754	98.507
	✓	✓	31.907	72.416	98.614
✓		✓	32.030	72.672	98.645
✓	✓		32.045	72.666	98.648
✓	✓	✓	32.205	72.734	98.685

TABLE II: Loss ablation on ACDC.

	Ours	MPVF	Fourier-Net+	DDM
Difference	0.03989	0.04572	0.14694	0.23164

TABLE III: Mean pairwise displacement difference across training runs on ACDC at

t=0.5

(

\mathrm{voxel}

\downarrow

Fig. 7: Temporal reconstruction on ACDC. (a) Ventricular anatomy and enlarged details across intermediate query times. (b) Relative change error across normalized time intervals, evaluating agreement with the reference sequence’s image evolution. Lower values indicate more faithful temporal reconstruction.

Fig. 8: Segmentation consistency on ACDC. (a) Dice agreement between masks predicted from reconstructed and reference volumes, averaged over test volumes including endpoints. RV, MYO, and LV denote the right ventricular cavity, left ventricular myocardium, and left ventricular cavity. (b) Reference and reconstructed images with their model-predicted masks.

Method	#P (M)	Est. train time (s/epoch) $\downarrow$	QPS $\uparrow$
Ours	$15.100$	$19.18$	$40.52$
MPVF	$14.536$	$105.14$	$4.61$
DDM	$23.164$	$18.42$	$14.04$
LDDM	$15.653$	$51.52$	$2.72$
Fourier-Net+	$12.414$	$1.84$	$121.16$

TABLE IV: Computational cost on ACDC. QPS counts complete 3D midpoint queries. MPVF training time includes all training stages.

4D Reconstruction

4D reconstruction recovers changing 3D scenes, shapes, and motion over time from images, videos, or other sparse measurements.

See all 140 papers

Aug 25, 2026

Low-Rank Velocity Fields as a Structural Prior for Unsupervised 4D Medical Image Interpolation

Endpoint-only unsupervised 4D medical image interpolation synthesizes intermediate volumes from sparsely sampled sequences with only the start and end volumes available for training; however, this weakly constrained setting often yields intermediates with unstable boundaries and non-physiological motion, limiting interpretability and downstream analysis. We propose low-rank velocity fields as a structural prior, constraining motion to a structured Tucker low-rank velocity field space that decomposes motion into globally shared spatial bases and a compact sample-specific core, thereby encouraging spatially correlated, anatomy-consistent deformation while suppressing voxel-wise high-frequency artifacts. To capture global coordination and local non-rigid details, we model motion in a coarse-to-fine multi-scale scheme and compose scale-wise deformations at inference to synthesize volumes at arbitrary times. We further provide a theoretical analysis showing that, under Tucker parameterization, low-rank parameters control the smoothness energy of the velocity field, explaining why low-rank modeling promotes smoother motion. Experiments on ACDC and 4D-Lung demonstrate state-of-the-art performance, remaining competitive with methods trained with intermediate-frame supervision, and producing intermediates with improved structural coherence and more stable anatomical contours.

Haojin Li, Hengzhuo Wang, Chang Liu +3

Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China · Faculty of Biomedical Engineering, Shenzhen University of Advanced Technology, Shenzhen, China · School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China +2

Medical Image Synthesis 4D Reconstruction Deformable Image Registration

Ask PDF

Aug 25, 2026

Phase-Aligned Finite-Fourier Periodic Deformation for 4D Medical Image Interpolation

4D medical image interpolation aims to recover missing volumes from sparsely observed time points and is important for dynamic anatomical analysis in applications such as cardiac MRI and thoracic CT, where motion is often repetitive or near-periodic over clinically relevant intervals. A key challenge is that this structure is not always encoded directly in deformation representations for interpolation. In addition, physiological motion is often non-uniform, so equal temporal intervals do not necessarily correspond to equal amounts of anatomical change. To address these issues, we formulate interpolation as learning a continuous deformation process with a phase-structured prior. Given two endpoint volumes, we parameterize a phase-conditioned velocity field with a finite Fourier basis, which embeds near-periodic motion patterns directly into the deformation space and supports continuous querying at arbitrary target times. We further introduce a phase-aligned temporal reparameterization that maps normalized within-interval time to a latent motion phase according to deformation variation intensity, thereby better modeling non-uniform motion progression. Intermediate volumes are then synthesized by continuously warping both endpoints, followed by bidirectional fusion and lightweight residual refinement. Experiments on ACDC and 4D-Lung show that the proposed method achieves state-of-the-art performance over existing baselines while producing anatomically plausible and coherent intermediate volumes from sparse observations.

Haojin Li, Hengzhuo Wang, Zhiheng Ma +3

School of Computer Science and Engineering Beihang University Beijing, China

4D Reconstruction Deformable Image Registration

Ask PDF

Feb 27, 2026

UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images

Dense 4D reconstruction from unposed images remains a critical challenge, with current methods relying on slow test-time optimization or fragmented, task-specific feedforward models. We introduce UFO-4D, a unified feedforward framework to reconstruct a dense, explicit 4D representation from just a pair of unposed images. UFO-4D directly estimates dynamic 3D Gaussian Splats, enabling the joint and consistent estimation of 3D geometry, 3D motion, and camera pose in a feedforward manner. Our core insight is that differentiably rendering multiple signals from a single Dynamic 3D Gaussian representation offers major training advantages. This approach enables a self-supervised image synthesis loss while tightly coupling appearance, depth, and motion. Since all modalities share the same geometric primitives, supervising one inherently regularizes and improves the others. This synergy overcomes data scarcity, allowing UFO-4D to outperform prior work by up to 3 times in joint geometry, motion, and camera pose estimation. Our representation also enables high-fidelity 4D interpolation across novel views and time. Please visit our project page for visual results: https://ufo-4d.github.io/

Junhwa Hur, Charles Herrmann, Songyou Peng +4

Google · Princeton University · Harvard University

4D Reconstruction Dynamic Scene Reconstruction Camera Pose Estimation 3D Gaussian Splatting

Ask PDF

Abstract

4D Reconstruction

Explore similar work

Low-Rank Velocity Fields as a Structural Prior for Unsupervised 4D Medical Image Interpolation

Phase-Aligned Finite-Fourier Periodic Deformation for 4D Medical Image Interpolation

UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images