ViT3Flow: A Test-Time Training Transformer MeanFlow for Postoperative Radiograph Synthesis in Scoliosis

Rui Tang; Sicheng Yang; Moxin Zhao; Hongqiu Wang; Guankun Wang; Lei Zhu; Hongliang Ren; Menglin Cong

cs.CVSep 4, 2026

ViT3Flow: A Test-Time Training Transformer MeanFlow for Postoperative Radiograph Synthesis in Scoliosis

Authors: Rui Tang, Sicheng Yang, Moxin Zhao, Hongqiu Wang, Guankun Wang, Lei Zhu, Hongliang Ren, Menglin Cong,

Abstract

Predicting postoperative spinal morphology from preoperative radiographs could provide valuable support for scoliosis surgical planning, but remains challenging because surgical correction induces large spatial changes while anatomical structures must be faithfully retained. We formulate this problem as postoperative scoliosis radiograph synthesis and construct ScoliSurg, the first paired dataset for this task, comprising 632 preoperative--postoperative whole-spine radiograph pairs with structured morphology information. We further propose ViT $^{3}$ Flow, a single-NFE conditional MeanFlow framework for efficient postoperative radiograph synthesis. ViT $^{3}$ Flow models surgical correction as finite-interval generative transport and replaces conventional self-attention with test-time-training token mixers that perform sample-specific inner adaptation to the anatomy and deformity pattern of each case. In addition, a Spinal Morphology Extraction Agent extracts distributions of dominant-curve region and direction from the preoperative radiograph. These distributions guide Diagnosis-Routed Interval Cross-Attention (DRICA), which performs interval-dependent vertical, horizontal, joint, and global retrieval from a separate preoperative token stream. This design enables the evolving postoperative representation to incorporate spatially corresponding anatomical evidence throughout the transport process. Extensive experiments on ScoliSurg demonstrate that ViT $^{3}$ Flow achieves the best performance among the compared methods in perceptual image quality, anatomical fidelity, and clinically relevant geometric accuracy, while requiring only a single network evaluation. These results highlight the potential of ViT $^{3}$ Flow for efficient and anatomically faithful postoperative radiograph synthesis in scoliosis surgical planning.

Figures & tables

Figure 1: Comparisons of generative paradigms for preoperative-to-postoperative scoliosis radiograph synthesis. Direct conditional translation (a) predicts the postoperative radiograph through a single feed-forward mapping, while iterative conditional generation (b) progressively transforms noise into the target image using multiple network function evaluations. Our ViT 3 Flow (c) performs patient-conditioned finite-interval generation with a single network evaluation, where the Spinal Morphology Extraction Agent provides dominant-curve region and direction distributions to guide Diagnosis-Routed Interval Cross-Attention in retrieving spatially organized evidence from the preoperative radiograph.

Figure 2: Overview of the proposed ViT 3 Flow framework. (a) The preoperative radiograph $x_{\mathrm{pre}}$ provides the image condition and is analyzed by SMEA to obtain the dominant-curve region probability vector $\bm{q}_{\mathrm{reg}}$ , image-coordinate curve-direction probability vector $\bm{q}_{\mathrm{dir}}$ , and joint region–direction distribution $\bm{q}_{\mathrm{joint}}$ . Starting from $z_{1}\sim\mathcal{N}(0,I)$ , ViT 3 Flow predicts an interval-averaged velocity and synthesizes the postoperative radiograph under the principal single-NFE setting. The dashed box illustrates the general finite-interval update $z_{r}=z_{t}-(t-r)u_{\theta}$ . (b) The preoperative radiograph and current flow state are independently encoded into preoperative tokens $H_{\mathrm{pre}}$ and current tokens $H_{t}$ . Both streams are supplied to the unified ViT 3 Flow backbone: $H_{t}$ is sequentially processed by the ViT 3 blocks, whereas the fixed $H_{\mathrm{pre}}$ is accessed by DRICA at selected depths. The interval embeddings condition the ViT 3 blocks and provide interval context for DRICA, whose outputs update the evolving current representation before the conditional interval-averaged velocity $u_{\theta}(z_{t},t,r\mid\mathcal{C})$ is predicted.

Figure 3: Overview of Diagnosis Routed Interval Cross Attention (DRICA). The evolving current tokens $H_{t}$ retrieve evidence from the fixed preoperative tokens $H_{\mathrm{pre}}$ through region-guided vertical retrieval, direction-guided horizontal retrieval, joint cross-axis coupling, and diagnosis-independent global retrieval. The MeanFlow interval context controls the route scales and residual write-back before producing the updated current tokens $H_{t}^{\prime}$ .

Method	NFE	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Cobb MAE ( ∘ ) $\downarrow$	CR Error (pp) $\downarrow$
Multi-NFE Methods
Palette [ 21 ]	1000	22.18	0.7845	0.2189	9.38 $\pm$ 7.61	16.81 $\pm$ 8.01
$I^{2}$ SB [ 15 ]	100	25.54	0.7926	0.1958	8.83 $\pm$ 2.49	13.13 $\pm$ 4.25
Diffusion [ 8 ]	1000	25.81	0.8237	0.2416	2.85 $\pm$ 3.95	6.43 $\pm$ 8.55
MOTFM [ 27 ]	10	24.40	0.8031	0.2223	4.79 $\pm$ 3.25	6.33 $\pm$ 5.08
FM [ 14 ]	100	25.42	0.8319	0.2524	1.52 $\pm$ 3.57	2.98 $\pm$ 4.76

Table 1: Quantitative comparison with image-translation, diffusion-based, and flow-based methods on ScoliSurg. NFE denotes the number of network function evaluations during inference. Higher PSNR and SSIM are better, while lower LPIPS, Cobb MAE, and CR error are better.

Figure 4: Quality efficiency comparison on ScoliSurg. Reconstruction quality is measured by PSNR, and inference throughput is reported in frames per second (FPS) on a logarithmic scale. Multi-NFE diffusion- and flow-based methods are concentrated in the low-throughput region, whereas single-NFE image-translation methods achieve higher throughput but lower reconstruction quality. ViT 3 Flow reaches the highest PSNR while remaining in the high-efficiency regime.

Figure 5: Qualitative comparison of postoperative scoliosis radiograph synthesis on three representative cases. From left to right: preoperative input, postoperative ground truth (GT), ViT 3 Flow prediction, Pix2Pix, ResViT, diffusion-based model, and flow-matching model (FM). The yellow boxes highlight the principal correction and fixation regions. ViT 3 Flow more closely reproduces the corrected spinal trajectory and the continuity of postoperative fixation structures while preserving the surrounding anatomical appearance.

Variant	MeanFlow	ViT 3	DRICA	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Cobb MAE ( ∘ ) $\downarrow$	CR Error (pp) $\downarrow$
Base				25.42	0.8319	0.2524	1.52 $\pm$ 3.57	2.98 $\pm$ 4.76
M1	✓			25.62	0.8495	0.2311	1.56 $\pm$ 2.47	2.69 $\pm$ 3.50
M2	✓	✓		26.20	0.8596	0.1961	1.67 $\pm$ 3.34	2.87 $\pm$ 2.83
M3	✓		✓	25.59	0.8468	0.2097	1.11 $\pm$ 2.44	2.84 $\pm$ 3.01
Ours	✓	✓	✓	26.79	0.8641	0.1828	1.07 $\pm$ 2.30	2.23 $\pm$ 4.71

Table 2: Contribution analysis of the core components in ViT 3 Flow on ScoliSurg. Higher PSNR and SSIM are better, while lower LPIPS, Cobb MAE, and CR error are better.

Variant	Global Ctx.	Axial	Morph.	Interval	Joint	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	Cobb MAE ( ∘ ) $\downarrow$	CR Error (pp) $\downarrow$
R1	✓					25.23	0.8394	0.2102	2.58 $\pm$ 2.81	3.88 $\pm$ 2.75
R2	✓	✓				25.41	0.8355	0.2035	2.17 $\pm$ 3.29	3.77 $\pm$ 2.82
R3	✓	✓		✓	✓	26.46	0.8461	0.1975	2.01 $\pm$ 3.73	2.55 $\pm$ 2.58
R4	✓	✓	✓		✓	26.42	0.8544	0.1969	2.03 $\pm$ 3.02	3.56 $\pm$ 3.06
R5	✓	✓	✓	✓		26.28	0.8505	0.1893	1.46 $\pm$ 4.08	2.47 $\pm$ 2.47
R6	✓	✓	✓	✓	✓	26.79	0.8641	0.1828	1.07 $\pm$ 2.30	2.23 $\pm$ 4.71

Table 3: Analysis of the routing mechanisms in Diagnosis Routed Interval Cross Attention (DRICA). Higher PSNR and SSIM are better, while lower LPIPS, Cobb MAE, and CR error are better.

Figure 6: Clinical correlation and agreement analysis of postoperative Cobb angle measurements for Pix2Pix, ResViT, Diffusion, FM, and ViT 3 Flow. For each method, the correlation plot compares the predicted and ground-truth postoperative Cobb angles. The dashed diagonal line denotes the identity line, while the solid colored line denotes the fitted regression line. Pearson correlation coefficient (

r

), coefficient of determination (

R^{2}

), and Spearman correlation coefficient (

\rho

) are reported in each plot, and the corresponding absolute angular error is shown below each method. The adjacent Bland–Altman plot presents the difference between the predicted and ground-truth Cobb angles against their paired mean. The solid horizontal line indicates the mean difference, and the dashed horizontal lines indicate the 95% limits of agreement.

Medical Image Synthesis

Medical image synthesis creates, translates, reconstructs, and edits clinical images to fill missing data and support analysis.

See all 310 papers

Aug 6, 2026

Patient Pose Assessment Using a CT-Based Framework for Synthetic Data Generation

An adequate diagnostic quality of radiographs is essential for reliable diagnoses and treatment planning. The patient's pose during radiography is one of the most important factors determining the diagnostic quality. Since patient positioning is difficult and not standardized, an automated AI-based approach using depth images to automatically assess the patient's pose before the radiograph has been taken would be helpful. Due to regulatory hurdles, however, it is difficult in practice to acquire the required depth images and corresponding radiographs. In this paper, we present a framework that can generate such training data synthetically from Computed Tomography scans. We further show that by pretraining on our generated synthetic dataset consisting of 3077 image pairs of upper ankle joints, the pose assessment of real upper ankle joints can be improved by up to 11 percentage points.

Manuel Laufer, Dominik Mairhöfer, Malte Sieren +7

Institute for Neuro- and Bioinformatics, University of Lübeck, Germany · University Medical Center Schleswig-Holstein, Lübeck, Germany · IMAGE Information Systems Europe GmbH, Rostock, Germany +1

Synthetic Data Pretraining Medical Imaging Synthetic Data Generation Medical Image Synthesis

Ask PDF

Jun 17, 2026

Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers

We introduce the first generative foundation model for chest radiograph synthesis trained from scratch at the billion-parameter scale. Existing radiographic AI models often suffer from poor generalisation across patient subpopulations, institutions, and acquisition settings, resulting in limited real-world clinical utility. Controlled, high-fidelity synthesis of chest radiographs is a promising path toward diversifying clinical datasets and evaluating the robustness of diagnostic models. Therefore, we present the largest specialist generative foundation model for chest radiographs to date, with over 1.3B parameters, trained for 1.6T tokens on a curated, heterogeneous dataset comprising 1.2M radiographs and clinical expert-guided metadata. Our model supports controllable radiograph generation and editing across multiple demographic subgroups, acquisition views, and a dozen pathologies. Moreover, we significantly advance the state of the art in radiograph synthesis fidelity, producing images that are indistinguishable from real radiographs to clinical experts.

Fabio De Sousa Ribeiro, Emma A. M. Stanley, Charles Jones +7

Imperial College London · Causality in Healthcare AI Hub · Cleveland Clinic London +4

Medical Imaging Foundation Models Medical Image Synthesis Conditional Generative Modeling

Ask PDF

Oct 27, 2026

MRI-to-CT Translation of Spinal Images via Anatomy-Guided Pseudo-Pairing and Adversarial Diffusion

Magnetic Resonance Imaging (MRI) is routinely used to diagnose spinal disorders, but surgical planning often requires Computed Tomography (CT) for better visualization of vertebral bone anatomy. Synthesizing CT from MRI could avoid additional radiation exposure and streamline preoperative assessment. We propose a three-stage framework for spine MRI-to-CT synthesis. First, an anatomy-guided vertebral matching system identifies similar sagittal MRI and CT slices using spine segmentations and aligns them through rigid registration, creating pseudo-paired data from unpaired scans. Second, these pairs train an adversarial diffusion model for MRI-to-CT translation. Third, synthesized slices are assembled into 3D volumes and refined by a network that improves inter-slice continuity and structural coherence. The framework was trained on over 10,000 unpaired sagittal MRI and CT slices and evaluated on 1,911 slices from 83 MRI volumes spanning all spinal regions. Assessment combined distribution-based measures with vertebral segmentation metrics. Compared with diffusion alone, the proposed approach increased binary 3D vertebral Dice from 0.666 to 0.739. Volumetric post-processing reduced visible discontinuities between stacked slices and improved vertebral coherence in axial and coronal views. Combining anatomy-guided pseudo-pairing, adversarial diffusion and 3D refinement improves spine MRI-to-CT synthesis. Vertebral matching primarily enhanced anatomical alignment and fidelity, while volumetric refinement improved 3D consistency. These findings support further development of MRI-derived CT-like visualization for preoperative planning, particularly to preserve fine intensity details while maintaining volumetric coherence.

Eduardo Nieto, Ibai Genua, Davide Scorza +1

Fundación Vicomtech, Basque Research and Technology Alliance (BRTA), Donostia - San Sebastián, Spain · R&D Department, Cyber Surgery, Donostia - San Sebastián, Spain · eHealth Group, Bioengineering Area, Biogipuzkoa Health Research Institute, Donostia - San Sebastián, Spain

Medical Image Synthesis Medical Imaging Unpaired Image-to-Image Translation

Ask PDF

Abstract

Medical Image Synthesis

Explore similar work

Patient Pose Assessment Using a CT-Based Framework for Synthetic Data Generation

Scaling Generative Foundation Models for Chest Radiography with Rectified Flow Transformers

MRI-to-CT Translation of Spinal Images via Anatomy-Guided Pseudo-Pairing and Adversarial Diffusion