CoRe-WM: Conditioned Response-State World Modeling for Longitudinal Treatment-Response Prediction in Breast DCE-MRI

Lin Zhao; Bowen Chen; Qiaohui Gao; Shaowen Wan; Xiang Li; Wei Liu; Tianming Liu

cs.CVOct 18, 2026

CoRe-WM: Conditioned Response-State World Modeling for Longitudinal Treatment-Response Prediction in Breast DCE-MRI

Authors: Lin Zhao, Bowen Chen, Qiaohui Gao, Shaowen Wan, Xiang Li, Wei Liu, Tianming Liu

Organizations: Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA. · College of Engineering, Northeastern University, Boston, MA 02115, USA. · Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA. · Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA. ·

Abstract

Pathologic complete response (pCR) after neoadjuvant chemotherapy is a key prognostic indicator in aggressive breast cancer, yet existing prediction approaches either rely on pretreatment imaging alone, missing the patient's evolving response, or optimize multi-visit classifiers under direct outcome supervision, coupling representation learning with the clinical endpoint. We formulate longitudinal pCR prediction as a world-modeling problem and propose Conditioned Response-state World Modeling (CoRe-WM), which learns a latent state-transition model that predicts the next imaging state from the observed sequence. The transition function is conditioned on patient clinical profile and assigned therapy through additive shift and feature-wise modulation, enabling patient- and treatment-specific dynamics. A Factorized Response State pathway further decouples treatment-response dynamics from imaging appearance by encoding the structural prior that response manifests primarily in lesion morphology. After pretraining, all parameters are frozen and a linear readout predicts pCR at each clinical decision point. We evaluate CoRe-WM on I-SPY2 under five-fold cross-validation and assess cross-cohort transfer on I-SPY1. CoRe-WM achieves the highest discrimination among all evaluated methods at every prospective decision point, and demonstrates robust cross-cohort transfer. Ablation studies identify patient-profile conditioning and the factorized response pathway as the principal contributors. These results establish world-model-based representation learning as a promising framework for outcome prediction in oncology and beyond.

Figures & tables

Fig. 1 : Overview of CoRe-WM . (A) Outcome-free pretraining. The upper pathway encodes DCE-MRI visits into latent states and predicts the next latent via a conditioned causal Transformer; the lower pathway operates on lesion geometry descriptors to produce a factorized response state

\hat{\mathbf{s}}_{t+1}

, which corrects the latent prediction and is supervised by the IRG loss. Patient profile, treatment, and temporal embeddings condition both pathways through FiLM modulation [ 43 ] . The EMA target encoder [ 44 ] provides the stable prediction target. (B) pCR prediction. All parameters are frozen; accumulated response states at each decision point are passed to a linear readout for progressively updated pCR estimates.

	Method	Avail. MRI	AUROC	AUPRC	Acc.	Sens.	Spec.
Pretreatment methods	ViT [ 23 ]	T0	$0.5591\pm 0.0320$	$0.4064\pm 0.0156$	$0.5657\pm 0.0507$	$0.5164\pm 0.1935$	$0.5917\pm 0.1741$
	ViT + two-stage fusion [ 23 ]	T0	$0.5950\pm 0.0130$	$0.4230\pm 0.0194$	$0.5521\pm 0.0629$	$0.5745\pm 0.2431$	$0.5412\pm 0.2194$
	3-D ResNet	T0	$0.5444\pm 0.0391$	$0.3685\pm 0.0242$	$0.4977\pm 0.0617$	$0.6836\pm 0.1546$	$0.4022\pm 0.1661$
Longitudinal methods	PrefixGRU	T0–T2	$0.5784\pm 0.0668$	$0.4157\pm 0.0799$	$0.5718\pm 0.0514$	$0.4982\pm 0.2075$	$0.6098\pm 0.1709$
	Temp. repr. learning [ 34 ]	T0–T2	$0.5179\pm 0.0516$	$0.3710\pm 0.0559$	$0.5606\pm 0.0476$	$0.3527\pm 0.2079$	$0.6678\pm 0.1748$
	Two-stage dual-task [ 33 ]	T0–T1	$0.6958\pm 0.0193$	$0.5412\pm 0.0378$	$0.6435\pm 0.0323$	$0.5745\pm 0.0539$	$0.6792\pm 0.0600$

TABLE I : Five-fold I-SPY2 comparison using the same patient splits and seed for all methods. Values are fold mean

\pm

standard deviation. The “Available MRI” column lists the examinations used by the pCR predictor; T3 may additionally be used as an outcome-free pretraining target ( CoRe-WM and BSTNet temporal-order pretraining). ViT two-stage fusion uses the complete BreastDCEDL clinical and geometry feature set. Accuracy, sensitivity, and specificity use a validation-selected threshold.

	Method	Avail. MRI	AUROC (95% CI)	AUPRC	Acc.	Sens.	Spec.
Pretreatment methods	ViT [ 23 ]	T0	$0.5584\,[0.4549,0.6609]$	$0.3159$	$0.4872$	$0.6829$	$0.4174$
	ViT + two-stage fusion [ 23 ]	T0	$0.5750\,[0.4764,0.6746]$	$0.3276$	$0.4487$	$0.6585$	$0.3739$
	3-D ResNet	T0	$0.6738\,[0.5698,0.7709]$	$0.4213$	$0.2949$	$0.9512$	$0.0609$
Longitudinal methods	PrefixGRU	T0–T2	$0.6494\,[0.5510,0.7428]$	$0.3798$	$0.6282$	$0.6341$	$0.6261$
	Temp. repr. learning [ 34 ]	T0–T2	$0.5548\,[0.4519,0.6579]$	$0.3433$	$0.7308$	$0.0732$	$0.9652$
	Two-stage dual-task [ 33 ]	T0–T1	$0.7635\,[0.6780,0.8402]$	$0.5391$	$0.7372$	$0.2439$	$0.9130$

TABLE II : Cross-cohort transfer from I-SPY2 to 156 I-SPY1 patients (41 pCR). AUROC brackets are patient-bootstrap 95% confidence intervals. All pCR supervision, model selection, and operating-threshold selection use I-SPY2 only.

Cumulative configuration	Profile	Tx	Geo	FRS/IRG	AUROC	$\Delta$ AUROC
Image-only JEPA	–	–	–	–	$0.5816\pm 0.0316$	–
$+$ patient profile	$\checkmark$	–	–	–	$0.7135\pm 0.0117$	$+0.1319\pm 0.0350$
$+$ treatment conditioning	$\checkmark$	$\checkmark$	–	–	$0.7115\pm 0.0393$	$-0.0019\pm 0.0467$
$+$ ROI geometry	$\checkmark$	$\checkmark$	$\checkmark$	–	$0.7380\pm 0.0138$	$+0.0264\pm 0.0451$
$+$ factorized response state	$\checkmark$	$\checkmark$	$\checkmark$	FRS	$0.7506\pm 0.0473$	$+0.0126\pm 0.0442$
$+$ image-derived response guidance ( CoRe-WM )	$\checkmark$	$\checkmark$	$\checkmark$	FRS+IRG	$\mathbf{0.7545\pm 0.0468}$	$+0.0039\pm 0.0082$

TABLE III : Cumulative five-fold ablation at T0–T1. Values are fold mean

\pm

standard deviation.

\Delta

AUROC is relative to the preceding row. Patient profile includes HR, HER2, MammaPrint, and age; Tx denotes treatment conditioning; Geo denotes ROI geometry. FRS and IRG denote factorized response state and image-derived response guidance, respectively.

Panel of Figure 5 — Fig. 2 : Five-fold diagnostic analyses. (a) OOF calibration curves at three prospective decision points; points show equal-frequency bins with Wilson 95% CIs. (b) Decision-curve analysis for non-pCR identification; shaded regions are bootstrap 95% CIs. (c) Condition-dependence analysis; MRI latents and geometry are fixed while treatment, clinical profile, or their joint context is permuted; shaded regions show $\pm 1$ SD across folds. (d) Response-state grounding; a ridge probe maps the frozen T0-to-T2 state update to observed log-volume shrinkage in 791 OOF patients.

Subtype	$n$ (pCR%)		T0	T0–T1	T0–T2 (95% CI)
Overall	808	(34.0)	0.733	0.755	0.762
HR+/HER2 $-$	320	(17.8)	0.683	0.689	0.705 [0.616, 0.784]
HER2+	201	(46.8)	0.712	0.741	0.731 [0.658, 0.801]
TNBC	287	(43.2)	0.640	0.669	0.707 [0.646, 0.765]

TABLE IV : Receptor-subtype stratification of CoRe-WM five-fold OOF AUROC at each prospective decision point. Values are pooled out-of-fold AUROC; brackets are patient-bootstrap 95% confidence intervals.

Cancer Drug Response Prediction

See all 21 papers

Aug 7, 2026

Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI

Pathologic complete response (pCR) is an important endpoint in neoadjuvant chemotherapy (NAC) for breast cancer, and predicting pCR from imaging during treatment could support treatment response assessment. Many existing imaging-based approaches rely on a single static timepoint, which fails to capture changes that occur during treatment. In this work, we present a longitudinal framework that combines a frozen 3D foundation encoder (Pillar-0) with our Temporal Dynamics Network (TDN) to predict treatment response from serial Dynamic Contrast-Enhanced (DCE) MRI acquired across four clinical timepoints from pre-treatment to pre-surgery. The TDN combines time-aware volumetric embeddings with clinical and treatment data to predict pCR. Evaluated on 982 patients from the combined I-SPY2 and ACRIN-6698 cohort, the proposed model achieves strong performance across all reported metrics when longitudinal 3D imaging is fused with clinical data (test AUROC: 73.6%, balanced accuracy: 69.1%). While clinical variables provide the strongest individual predictive signal, longitudinal 3D imaging contributes complementary information when fused with clinical data, improving pCR prediction. Our source code is available at: https://github.com/omarftt/longitudinal_temporal_pillar.

Fidel Omar Tito Cruz, Neda Ghafouri, Zengyan Wang +3

University of Central Florida, Orlando, FL, USA

Breast Cancer Longitudinal Medical Image Analysis Clinical Outcome Prediction Magnetic Resonance Imaging Oncology Cancer Drug Response Prediction

Ask PDF

Jul 6, 2026

Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction

In patients with breast cancer, pathological complete response (pCR) has been established as a clinically meaningful surrogate marker for long-term outcomes. While commonly treated with neoadjuvant chemotherapy (NACT), effective treatment decision-making remains challenging, as therapeutic response can vary substantially across patients, calling for predictive models capable of accurately estimating individualized treatment response. To address this, we propose an imaging-based 3D spatio-temporal framework for treatment response prediction that integrates a state-of-the-art graph neural network with relational modeling of temporal interactions across timepoints alongside three novel complementary self-supervised treatment trajectory representation learning objectives. Experiments across a cohort of 585 patients from the public ISPY-2 dataset demonstrate that our method substantially outperforms both vision and self-supervised learning baselines across several classification metrics. Alongside establishing a breast cancer pCR prediction benchmark, we include a principled ablation of our method and further introduce and empirically assess the impact of the available number of DCE-MRI timepoints per patient trajectory and the inclusion of inter-scan time-differences. Overall, our study substantiates the utility of clinically meaningful longitudinal medical imagaging modeling for predicting NACT-induced pCR. We will publicly share our code repository and a user-friendly PyPI library for dataset curation upon publication, effectively promoting reproducible open-source research.

Johannes Kiechle, Richard Osuala, Daniel M. Lang +5

Technical University of Munich · TUM University Hospital Rechts der Isar · Helmholtz Munich +4

Graph Representation Learning 3D Medical Imaging Longitudinal Medical Image Analysis Clinical Outcome Prediction Oncology Medical Image Representation Learning+2

Ask PDF

Aug 18, 2026

Neoadjuvant chemotherapy response prediction using pretreatment diffusion and contrast-enhanced magnetic resonance imaging with clinical variables

Prediction of pathological complete response before neoadjuvant chemotherapy may facilitate more tailored therapeutic planning for breast cancer patients. This work proposes a deep-learning model for pretreatment data only, combining apparent diffusion coefficient maps, dynamic contrast-enhanced magnetic resonance imaging, and clinical variables. The study uses the public ACRIN 6698/I-SPY2 multicenter dataset. The architecture employs EfficientNet-B0 pretrained encoders for image feature extraction and late fusion with clinical information. Multiple clinical variables were evaluated, including age, race, histological type, HR/HER2 subtype, SBR grade, and maximum diameter. Only HR/HER2 subtype improved the average area under the receiver operating characteristic curve (AUC) and was retained in the final model. Using stratified five-fold cross-validation, standalone apparent diffusion coefficient maps achieved a mean AUC of 0.79, whereas dynamic contrast-enhanced magnetic resonance imaging achieved 0.74. Adding HR/HER2 subtype improved performance to 0.83 and 0.81, respectively. The final configuration, using both imaging modalities and HR/HER2 subtype, achieved an AUC of 0.86. These results support pretreatment multimodal learning for response prediction, although external validation is required before clinical use.

Pablo García Marcos, Paula Puerta González, Guillermo Lorenzo +6

Computer Science Department, University of Oviedo, Asturias, Spain · Biomedical Engineering Center (BME), University of Oviedo, Asturias, Spain · Electrical Engineering Department, University of Oviedo, Asturias, Spain +7

Cancer Drug Response Prediction Clinical Outcome Prediction Oncology Multimodal Learning

Ask PDF

Abstract

Cancer Drug Response Prediction

Explore similar work

Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI

Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction

Neoadjuvant chemotherapy response prediction using pretreatment diffusion and contrast-enhanced magnetic resonance imaging with clinical variables