Optimizing 3D Diffusion Models for Medical Imaging via Multi-Scale Reward Learning

Yueying Tian; Xudong Han; Meng Zhou; Rodrigo Aviles-Espinosa; Rupert Young; Philip Birch

cs.CVMar 6, 2026

Optimizing 3D Diffusion Models for Medical Imaging via Multi-Scale Reward Learning

Authors: Yueying Tian, Xudong Han, Meng Zhou, Rodrigo Aviles-Espinosa, Rupert Young, Philip Birch

Organizations: University of Sussex · University of Toronto

Abstract

Diffusion models have emerged as powerful tools for 3D medical image generation, yet bridging the gap between standard training objectives and clinical relevance remains a challenge. This paper presents a method to enhance 3D diffusion models using Reinforcement Learning (RL) with multi-scale feedback. We first pretrain a 3D diffusion model on MRI volumes to establish a robust generative prior. Subsequently, we fine-tune the model using Proximal Policy Optimization (PPO), guided by a novel reward system that integrates both 2D slice-wise assessments and 3D volumetric analysis. This combination allows the model to simultaneously optimize for local texture details and global structural coherence. We validate our framework on the BraTS 2019 and OASIS-1 datasets. Our results indicate that incorporating RL feedback effectively steers the generation process toward higher quality distributions. Quantitative analysis reveals significant improvements in Fréchet Inception Distance (FID) and, crucially, the synthetic data demonstrates enhanced utility in downstream tumor and disease classification tasks compared to non-optimized baselines.

Figures & tables

Figure 1: Illustration for the proposed framework. Stage I involves pretraining a 3D-VQGAN and a latent diffusion model. Stage II (Middle) shows the self-supervised reward generation strategy. We utilize synthetic trajectories (FID $\approx$ 50) and noised-reconstruction trajectories (FID $\approx$ 25) to fill the fidelity gap. Reward values are computed using the objective $R=\exp(-(FID-25)/15)$ . Stage III (Bottom) illustrates the RL fine-tuning phase where the policy $\pi_{\theta}$ is optimized via PPO based on 3D volumetric and 2D slice-wise feedback.

Split	BraTS 2019			OASIS-1
Split	HGG	LGG	All	AD	CN	All
Training	229	46	275	28	208	240
Testing	30	30	60	30	30	60
Total	259	76	335	58	242	300

Table 1: Overview of dataset splits, including the total (All) count for each.

Method	BraTS 2019			OASIS-1
Method	HGG $\downarrow$	LGG $\downarrow$	All $\downarrow$	AD $\downarrow$	CN $\downarrow$	All $\downarrow$
3D-VQGAN Reconstruction	25.82	35.14	24.64	33.81	30.51	29.42
Standard Diffusion	53.98	62.28	50.38	68.21	60.70	57.45
Ours	40.11	51.86	38.05	61.88	55.84	52.92

Table 2: Fréchet Inception Distance (FID) values across multiple datasets. The All columns represent the aggregate FID scores across the full spectrum of each dataset.

BraTS 2019 (HGG/LGG)
Method	Accuracy $\uparrow$	F1 Score $\uparrow$	AUC $\uparrow$
Real Data Only	0.59 $\pm$ 0.07	0.67 $\pm$ 0.04	0.65 $\pm$ 0.07
Standard Synthetic	0.62 $\pm$ 0.05	0.68 $\pm$ 0.04	0.65 $\pm$ 0.08
Ours	0.71 $\pm$ 0.02	0.71 $\pm$ 0.01	0.74 $\pm$ 0.03
OASIS-1 (AD/CN)
Real Data Only	0.76 $\pm$ 0.03	0.75 $\pm$ 0.04	0.81 $\pm$ 0.01

Table 3: Downstream classification performance. For each method, we conduct three independent runs, and report mean

\pm

standard deviation.

Method	Accuracy $\uparrow$	F1 Score $\uparrow$	AUC $\uparrow$
3D- $\alpha$ WGAN [ 11 ]	0.59 $\pm$ 0.06	0.59 $\pm$ 0.09	0.70 $\pm$ 0.09
3D-Med-DDPM [ 6 ]	0.61 $\pm$ 0.06	0.64 $\pm$ 0.01	0.69 $\pm$ 0.03
TAMT [ 20 ]	0.67 $\pm$ 0.04	0.71 $\pm$ 0.02	0.77 $\pm$ 0.03
Ours	0.71 $\pm$ 0.02	0.71 $\pm$ 0.01	0.74 $\pm$ 0.03

Table 4: Downstream classification performance on BraTS 2019. For each method, we conduct three independent runs and report mean

\pm

standard deviation.

Configuration	FID value $\downarrow$	Accuracy $\uparrow$
Full Framework	38.05	0.71
Sparse Steps (1, 50, 99)	47.05	0.67
3D Reward Only	43.92	0.69

Table 5: Ablation study on reward scale and step efficiency. Values reflect performance on the BraTS 2019 dataset.

Medical Image Synthesis

Medical image synthesis creates, translates, reconstructs, and edits clinical images to fill missing data and support analysis.

See all 310 papers

3D Diffusion Models for Medical Imaging
Oct 19, 2026Frank-Peter Schilling, Daniel Barco, Philipp Denzel +13D Diffusion Models Medical Image Synthesis
Centre for AI (CAI), Zurich University of Applied Sciences (ZHAW), Technikumstrasse 9, Winterthur, 8400, Zurich, Switzerland
TCAM-Diff: Triplane-Aware Cross-Attention Medical Diffusion Model
Jul 15, 2026Zhenkai Zhang, Krista A. Ehinger, Tom Drummond3D Medical Imaging Medical Image Synthesis
School of Computing and Information Systems, The University of Melbourne · The University of Melbourne
Flow3D-OPD: Multi-Teacher On-Policy Distillation for 3D Geometry Generation with Flow-Matching Diffusion Transformer
Sep 7, 2026Zhiwei Ning, Zhen Zhou, Puhua Jiang +73D Mesh Generation Diffusion Model Distillation
Shanghai Jiao Tong University · Tencent Hunyuan3D · Zhejiang Normal University +1

Oct 19, 2026J/K move · Enter open · S save

3D Diffusion Models for Medical Imaging

Frank-Peter Schilling, Daniel Barco, Philipp Denzel, Marc Stadelmann

Centre for AI (CAI), Zurich University of Applied Sciences (ZHAW), Technikumstrasse 9, Winterthur, 8400, Zurich, Switzerland

3D Diffusion Models Medical Image Synthesis 3D Medical Imaging

Open paper AskPDF arXiv

Generative deep learning methods such as Variational Autoencoders, Generative Adversarial Networks or Diffusion Models have achieved remarkable results in image synthesis. In particular, Diffusion Models have demonstrated superior image fidelity and sample diversity compared to other approaches on standard benchmarks, while also offering more flexible training dynamics as evidenced both in natural image domains and in medical applications. In medical imaging including Computed Tomography, Magnetic Resonance Imaging or Positron Emission Tomography, such models have various important use-cases including data augmentation, domain transfer, image reconstruction, inpainting, image augmentation and artifact correction. Further, generated images can be conditioned on patient characteristics, medical reports, segmentation masks or other images. Specific challenges arise due to the 3D volumetric nature of medical images, which translate to substantial demands in terms of memory, compute, and training data. This article provides a comparative overview of 3D Diffusion Models which have been developed for medical images, with a focus on current performance, limitations, and an outlook for future work.

Abstract

Medical Image Synthesis

Explore similar work

3D Diffusion Models for Medical Imaging

TCAM-Diff: Triplane-Aware Cross-Attention Medical Diffusion Model

Flow3D-OPD: Multi-Teacher On-Policy Distillation for 3D Geometry Generation with Flow-Matching Diffusion Transformer