Vessel2Brain: Edge-Aware Multi-Scale Flow Matching for 3D Brain Image Generation from Vessel Masks
Authors: Zhi Chen, Tianyang Zhang, Theodoros N. Arvanitis, Le Zhang
Organizations: School of Engineering, College of Engineering and Physical Sciences, University of Birmingham, Birmingham, UK · Institute of Biomedical Engineering, University of Oxford, Oxford, UK
Generating realistic 3D brain MRI volumes from sparse anatomical priors is a highly ill-posed yet clinically valuable problem, with important applications in data augmentation, simulation, and downstream neuroimaging analysis. In this work, we investigate the challenging task of synthesising full 3D brain MRI volumes conditioned solely on vascular label maps. While vascular structures provide strong anatomical and topological cues, they occupy only a small fraction of the brain volume and offer no direct information about surrounding tissue appearance. To address this challenge, we propose Vessel2Brain, an edge-aware multi-scale conditional flow matching framework that explicitly incorporates vascular structure information into the generative process. Our approach formulates vessel-guided brain image synthesis as a conditional continuous flow, enabling stable training and flexible generation while preserving structural correspondence. A multi-scale flow matching strategy is introduced to jointly model global anatomical coherence and fine-grained local details, with auxiliary velocity predictions providing additional supervision across spatial resolutions. Furthermore, we design an edge-aware objective that emphasises vascular boundaries during training, encouraging anatomically faithful tissue synthesis around vessel structures. We evaluate Vessel2Brain on two public 3D brain MRI datasets with paired vascular annotations. Extensive quantitative and qualitative experiments demonstrate that our method consistently outperforms state-of-the-art generative models in terms of reconstruction accuracy, structural consistency, and distributional realism.
Figures & tables
Figure 1: Overview of the proposed Vessel2Brain framework. Given a 3D vascular label map and a time step t , a UNet-style FlowNet conditioned on timestep and label embeddings learns a velocity field for conditional flow matching. Velocity fields are predicted at multiple spatial resolutions through a main velocity head and an auxiliary velocity head. Edge-aware weighting derived from Sobel operators on the vascular volume emphasises vessel boundaries during training. The predicted velocity fields are integrated to generate the final 3D brain MRI volume, while auxiliary outputs provide additional multi-scale supervision.
Figure 2: Qualitative comparison on the CereVessMRA dataset. From left to right: vascular mask, ground-truth brain MRI, 3D Pix2Pix, LDM, 3D VAE, Med-DDPM, and the proposed Vessel2Brain. Results are visualized on representative sagittal, coronal, and axial slices. The highlighted regions and corresponding zoomed-in views illustrate fine-grained anatomical details. Compared with baseline methods, Vessel2Brain generates more anatomically consistent brain structures while better preserving vascular-aligned tissue appearance under sparse vessel conditioning.
Method
SSIM (%)↑
PSNR ↑
NMI ↑
MAE (%)↓
3D Pix2Pix [ 5 ]
57.69(±8.13)
23.74(±2.82)
1.138(±0.017)
4.57(±0.78)
3D Medical VAE [ 6 ]
63.17(±8.70)
26.40(±2.10)
1.133(±0.020)
3.13(±0.89)
LDM [ 11 ]
61.83(±9.46)
26.16(±2.91)
1.129(±0.018)
3.23(±0.97)
Med-DDPM [ 3 ]
63.81(±8.57)
26.79(±2.59)
1.142(±0.018)
2.99(±1.09)
Vessel2Brain(w/o Aug.)
63.99(±9.02)
26.40(±2.72)
1.139(±0.018)
3.28(±0.89)
Vessel2Brain(w/o Aux.)
64.83(±8.32)
26.53(±2.31)
1.138(±0.017)
3.41(±0.91)
Table 1: Quantitative comparison on the CereVessMRA dataset. Results are reported as mean ± standard deviation. ↑ / ↓ denote that higher/lower values indicate better performance. Vessel2Brain consistently outperforms representative GAN-, VAE-, and diffusion-based baselines across all metrics, highlighting its effectiveness for vessel-guided 3D brain MRI generation under sparse conditioning.
Figure 3: Qualitative comparison on the TopBrain Challenge dataset. The visualization setting and column order are the same as in Fig. 2 . Vessel2Brain produces more anatomically coherent brain structures and clearer local details than competing methods under sparse vascular conditioning.
Method
SSIM (%)↑
PSNR ↑
NMI ↑
MAE (%)↓
3D Pix2Pix [ 5 ]
17.41(±3.54)
13.78(±1.19)
1.052(±0.016)
15.56(±3.40)
3D Medical VAE [ 6 ]
59.40(±2.45)
26.61(±0.97)
1.071(±0.014)
3.12(±0.30)
LDM [ 11 ]
60.97(±2.72)
27.15(±0.87)
1.074(±0.011)
2.94(±0.33)
Med-DDPM [ 3 ]
59.70(±2.45)
26.64(±0.69)
1.072(±0.008)
3.11(±0.28)
Vessel2Brain(w/o Aug.)
56.72(±2.74)
25.72(±1.32)
1.066(±0.015)
3.07(±0.42)
Vessel2Brain(w/o Aux.)
58.92(±2.85)
26.81(±1.03)
1.071(±0.017)
2.97(±0.44)
Table 2: Quantitative comparison on the TopBrain Challenge dataset. The evaluation protocol, metric definitions ( ↑ / ↓ ), and reporting format (mean ± std.) are the same as in Table 1 . Vessel2Brain achieves the best performance across all metrics, indicating strong generalization under increased dataset heterogeneity.
High-fidelity 3D MRI synthesis requires both globally coherent anatomy and fine-grained voxel-level detail. Although latent diffusion makes volumetric generation tractable, its image autoencoder introduces a reconstruction bottleneck that can limit the fine detail recoverable in the final volume. We present VoxStruct3D, a voxel-space flow-matching framework that directly models full-resolution MRI volumes using a clean-data prediction objective. Its Volumetric Voxel Generator (VVG) combines factorized 3D patch embedding with overlapping upsampling, time-modulated residual refinement, and skip fusion, enabling neighboring tokens to jointly reconstruct shared voxel regions and suppress patch-boundary artifacts. To complement direct voxel-space modeling with an explicit anatomical prior, we further introduce a Structure-First, Image-Follows (SFIF) strategy. A frozen pretrained 3D medical encoder and a StructVAE extract compact structure tokens that preserve dominant anatomy, while a structure-leading schedule keeps their trajectory ahead of the image trajectory. Patch-Aligned RoPE spatially aligns the unequal token grids, and asymmetric attention enforces one-way guidance from structure to image. Experiments on pathological and healthy T1-weighted brain MRI datasets show that VoxStruct3D achieves the strongest overall performance across feature-distribution alignment, sample diversity, and perceptual quality, producing anatomically coherent and visually realistic volumes.
Fang Li, Yang Gao, Shihao Zou +5
State Key Laboratory of Virtual Reality Technology and Systems, Beihang University · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · Faculty of Computer Science and Control Engineering, Shenzhen University of Advanced Technology
Large and demographically balanced datasets are essential for reliable neuroimaging biomarkers. Full-resolution 3D brain MRI synthesis can support data augmentation in this setting, but existing approaches either incur prohibitive computational cost at volumetric scale or rely on lossy latent compression that may compromise anatomical detail. As a result, practical 3D generative augmentation often requires specialized compute infrastructure. We propose WaveDiT, a conditional flow matching framework operating in the coefficient space of a 3D Haar Discrete Wavelet Transform. The model combines factorized spatio-depth attention with band-wise heteroscedastic uncertainty modeling derived from higher-order wavelet statistics. Predicted log-variance is integrated directly into both the flow objective and conditioning pathway, enabling adaptive precision consistent with the heavy-tailed and input-dependent variance structure of anatomical detail. This formulation supports full-resolution 3D synthesis under practical memory and time constraints on a single modern GPU. Evaluation on a multi-site cohort demonstrates improved alignment between generated and real MRI distributions, together with enhanced downstream brain age prediction and region-level anatomical agreement relative to diffusion, latent, and wavelet-based baselines. Code is available at https://github.com/sisinflab/WaveDiT
Danilo Danese, Angela Lombardi, Giuseppe Fasano +2
Politecnico di Bari, Italy · Sapienza University of Rome, Italy
Generative inpainting of brain MRI volumes is essential for synthesizing healthy tissue in pathological regions, improving the accuracy and reliability of automated downstream brain analysis applications such as image registration, brain extraction, and segmentation. However, standard 3D approaches are computationally prohibitive, while efficient 2D slice-wise methods suffer from severe inter-slice discontinuities. Furthermore, traditional models rely on conditional training, requiring task-specific learning of masked inputs. We propose a zero-shot brain MRI inpainting framework utilizing 2.5D unconditional flow priors to capture spatial context along the superior-inferior axis without the overhead of full 3D convolutions. During training, our flow matching model learns the joint distribution of adjacent axial slice triplets, modeling the manifold of healthy brain anatomy while explicitly excluding pathological regions from the loss function. At inference, the model processes the input triplets autoregressively along the depth axis. We employ the Restora-Flow solver to constrain the unconditional prior using the input mask, achieving accurate zero-shot inpainting. Evaluations show our 2.5D strategy resolves the structural discontinuities of 2D baselines, synthesizing plausible healthy tissue while maintaining volumetric consistency across the axial, sagittal, and coronal planes. As a final step, we generate and average an ensemble of multiple stochastic reconstructions to form the final prediction. Quantitative results benchmarked on the official BraTS 2026 Inpainting Challenge validation set demonstrate the effectiveness of our proposed approach, yielding an SSIM of 0.816 ± 0.112, MSE of 0.007 ± 0.005, and PSNR of 22.923 ± 4.343. Code is available at https://github.com/imigraz/brats2026-inpainting.
Arnela Hadzic, Franz Thaler, Simon Johannes Joham +1
Institute for Medical Informatics, Statistics and Documentation, Medical University of Graz, Graz, Austria