Authors: Chenhui Wang, Boyun Zheng, Liuxin Bao, Zhihao Peng, Peter Y. M. Woo, Hongming Shan, Yixuan Yuan
Organizations: Institute of Science and Technology for Brain-inspired Intelligence, Fudan University, Shanghai 200433, China · Department of Electronic Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China · School of Automation, Hangzhou Dianzi University, Hangzhou 310018, China · Department of Neurosurgery, Prince of Wales Hospital, Hong Kong, SAR, China
Precise prognostic modeling of glioblastoma (GBM) under varying treatment interventions is essential for optimizing clinical outcomes. While generative AI has shown promise in simulating GBM evolution, existing methods typically treat interventions as static conditional inputs rather than dynamic decision variables. Consequently, they fail to capture the complex, reciprocal interplay between tumor evolution and treatment response. To bridge this gap, we present Brain-WM, a pioneering brain GBM world model that unifies next-step treatment prediction and future MRI generation, thereby capturing the co-evolutionary dynamics between tumor and treatment. Specifically, Brain-WM encodes spatiotemporal dynamics into a shared latent space for joint autoregressive treatment prediction and flow-based future MRI generation. Then, instead of a conventional monolithic framework, Brain-WM adopts a novel Y-shaped Mixture-of-Transformers (MoT) architecture. This design structurally disentangles heterogeneous objectives, successfully leveraging cross-task synergies while preventing feature collapse. Finally, a synergistic multi-timepoint mask alignment objective explicitly anchors latent representations to anatomically grounded tumor structures and progression-aware semantics. Extensive validation on internal and external multi-institutional cohorts demonstrates the superiority of Brain-WM, achieving 91.5% accuracy in treatment planning and SSIMs of 0.8524, 0.8581, and 0.8404 for FLAIR, T1CE, and T2W sequences, respectively. Ultimately, Brain-WM offers a robust clinical sandbox for optimizing patient healthcare. The source code is made available at https://github.com/thibault-wch/Brain-GBM-world-model.
Figures & tables
Fig. 1: Conceptual overview of Brain-WM. Brain-WM establishes the first brain GBM world model by encoding spatiotemporal dynamics from treatment history, demographics, and multi-modal MRI to unify next-step clinical planning and future tumor morphology synthesis, thereby capturing the co-evolutionary dynamics between treatment and tumor.
Fig. 2: Overview of our Brain-WM. ( a ) Brain-WM features a unified Y-shaped Mixture-of-Transformers (MoT) backbone that bifurcates to optimize distinct objectives: autoregressive treatment planning and flow-based future MRI generation. ( b ) A multi-timepoint mask alignment objective ensures the learned representations remain anatomically grounded and task-consistent across time.
Fig. 3: Illustration of the volumetric MRI tokenizer.
Fig. 4: Qualitative results of ours and other competing methods on the internal cohort.
Fig. 5: Qualitative results of ours and other competing methods on the external cohort.
Fig. 6: Qualitative visualization of the masks predicted by the lightweight mask aligner during treatment planning.
Fig. 7: Correlation between the MM-Align focused-region volume in future MRI generation and the ground-truth tumor volume.
Technical University of Munich (TUM), Germany · Munich Center for Machine Learning (MCML) · Chair for AI in Healthcare and Medicine, TUM and TUM University Hospital, Munich, Germany +3
Amsterdam UMC, Department of Radiation Oncology, De Boelelaan 1117, 1081 HV Amsterdam, The Netherlands · Cancer Center Amsterdam, Imaging and Biomarkers, De Boelelaan 1117, 1081 HV Amsterdam, The Netherlands · Department of Radiology and Nuclear Medicine, De Boelelaan 1117, 1081 HV Amsterdam, The Netherlands +1