CoRe-WM: Conditioned Response-State World Modeling for Longitudinal Treatment-Response Prediction in Breast DCE-MRI

Lin Zhao; Bowen Chen; Qiaohui Gao; Shaowen Wan; Xiang Li; Wei Liu; Tianming Liu

cs.CVOct 18, 2026

CoRe-WM: Conditioned Response-State World Modeling for Longitudinal Treatment-Response Prediction in Breast DCE-MRI

Authors: Lin Zhao, Bowen Chen, Qiaohui Gao, Shaowen Wan, Xiang Li, Wei Liu, Tianming Liu

Organizations: Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA. · College of Engineering, Northeastern University, Boston, MA 02115, USA. · Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA. · Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA. ·

Abstract

Pathologic complete response (pCR) after neoadjuvant chemotherapy is a key prognostic indicator in aggressive breast cancer, yet existing prediction approaches either rely on pretreatment imaging alone, missing the patient's evolving response, or optimize multi-visit classifiers under direct outcome supervision, coupling representation learning with the clinical endpoint. We formulate longitudinal pCR prediction as a world-modeling problem and propose Conditioned Response-state World Modeling (CoRe-WM), which learns a latent state-transition model that predicts the next imaging state from the observed sequence. The transition function is conditioned on patient clinical profile and assigned therapy through additive shift and feature-wise modulation, enabling patient- and treatment-specific dynamics. A Factorized Response State pathway further decouples treatment-response dynamics from imaging appearance by encoding the structural prior that response manifests primarily in lesion morphology. After pretraining, all parameters are frozen and a linear readout predicts pCR at each clinical decision point. We evaluate CoRe-WM on I-SPY2 under five-fold cross-validation and assess cross-cohort transfer on I-SPY1. CoRe-WM achieves the highest discrimination among all evaluated methods at every prospective decision point, and demonstrates robust cross-cohort transfer. Ablation studies identify patient-profile conditioning and the factorized response pathway as the principal contributors. These results establish world-model-based representation learning as a promising framework for outcome prediction in oncology and beyond.

Figures & tables

Fig. 1 : Overview of CoRe-WM . (A) Outcome-free pretraining. The upper pathway encodes DCE-MRI visits into latent states and predicts the next latent via a conditioned causal Transformer; the lower pathway operates on lesion geometry descriptors to produce a factorized response state

\hat{\mathbf{s}}_{t+1}

, which corrects the latent prediction and is supervised by the IRG loss. Patient profile, treatment, and temporal embeddings condition both pathways through FiLM modulation [ 43 ] . The EMA target encoder [ 44 ] provides the stable prediction target. (B) pCR prediction. All parameters are frozen; accumulated response states at each decision point are passed to a linear readout for progressively updated pCR estimates.

	Method	Avail. MRI	AUROC	AUPRC	Acc.	Sens.	Spec.
Pretreatment methods	ViT [ 23 ]	T0	$0.5591\pm 0.0320$	$0.4064\pm 0.0156$	$0.5657\pm 0.0507$	$0.5164\pm 0.1935$	$0.5917\pm 0.1741$
	ViT + two-stage fusion [ 23 ]	T0	$0.5950\pm 0.0130$	$0.4230\pm 0.0194$	$0.5521\pm 0.0629$	$0.5745\pm 0.2431$	$0.5412\pm 0.2194$
	3-D ResNet	T0	$0.5444\pm 0.0391$	$0.3685\pm 0.0242$	$0.4977\pm 0.0617$	$0.6836\pm 0.1546$	$0.4022\pm 0.1661$
Longitudinal methods	PrefixGRU	T0–T2	$0.5784\pm 0.0668$	$0.4157\pm 0.0799$	$0.5718\pm 0.0514$	$0.4982\pm 0.2075$	$0.6098\pm 0.1709$
	Temp. repr. learning [ 34 ]	T0–T2	$0.5179\pm 0.0516$	$0.3710\pm 0.0559$	$0.5606\pm 0.0476$	$0.3527\pm 0.2079$	$0.6678\pm 0.1748$
	Two-stage dual-task [ 33 ]	T0–T1	$0.6958\pm 0.0193$	$0.5412\pm 0.0378$	$0.6435\pm 0.0323$	$0.5745\pm 0.0539$	$0.6792\pm 0.0600$

TABLE I : Five-fold I-SPY2 comparison using the same patient splits and seed for all methods. Values are fold mean

\pm

standard deviation. The “Available MRI” column lists the examinations used by the pCR predictor; T3 may additionally be used as an outcome-free pretraining target ( CoRe-WM and BSTNet temporal-order pretraining). ViT two-stage fusion uses the complete BreastDCEDL clinical and geometry feature set. Accuracy, sensitivity, and specificity use a validation-selected threshold.

	Method	Avail. MRI	AUROC (95% CI)	AUPRC	Acc.	Sens.	Spec.
Pretreatment methods	ViT [ 23 ]	T0	$0.5584\,[0.4549,0.6609]$	$0.3159$	$0.4872$	$0.6829$	$0.4174$
	ViT + two-stage fusion [ 23 ]	T0	$0.5750\,[0.4764,0.6746]$	$0.3276$	$0.4487$	$0.6585$	$0.3739$
	3-D ResNet	T0	$0.6738\,[0.5698,0.7709]$	$0.4213$	$0.2949$	$0.9512$	$0.0609$
Longitudinal methods	PrefixGRU	T0–T2	$0.6494\,[0.5510,0.7428]$	$0.3798$	$0.6282$	$0.6341$	$0.6261$
	Temp. repr. learning [ 34 ]	T0–T2	$0.5548\,[0.4519,0.6579]$	$0.3433$	$0.7308$	$0.0732$	$0.9652$
	Two-stage dual-task [ 33 ]	T0–T1	$0.7635\,[0.6780,0.8402]$	$0.5391$	$0.7372$	$0.2439$	$0.9130$

TABLE II : Cross-cohort transfer from I-SPY2 to 156 I-SPY1 patients (41 pCR). AUROC brackets are patient-bootstrap 95% confidence intervals. All pCR supervision, model selection, and operating-threshold selection use I-SPY2 only.

Cumulative configuration	Profile	Tx	Geo	FRS/IRG	AUROC	$\Delta$ AUROC
Image-only JEPA	–	–	–	–	$0.5816\pm 0.0316$	–
$+$ patient profile	$\checkmark$	–	–	–	$0.7135\pm 0.0117$	$+0.1319\pm 0.0350$
$+$ treatment conditioning	$\checkmark$	$\checkmark$	–	–	$0.7115\pm 0.0393$	$-0.0019\pm 0.0467$
$+$ ROI geometry	$\checkmark$	$\checkmark$	$\checkmark$	–	$0.7380\pm 0.0138$	$+0.0264\pm 0.0451$
$+$ factorized response state	$\checkmark$	$\checkmark$	$\checkmark$	FRS	$0.7506\pm 0.0473$	$+0.0126\pm 0.0442$
$+$ image-derived response guidance ( CoRe-WM )	$\checkmark$	$\checkmark$	$\checkmark$	FRS+IRG	$\mathbf{0.7545\pm 0.0468}$	$+0.0039\pm 0.0082$

TABLE III : Cumulative five-fold ablation at T0–T1. Values are fold mean

\pm

standard deviation.

\Delta

AUROC is relative to the preceding row. Patient profile includes HR, HER2, MammaPrint, and age; Tx denotes treatment conditioning; Geo denotes ROI geometry. FRS and IRG denote factorized response state and image-derived response guidance, respectively.

Panel of Figure 5 — Fig. 2 : Five-fold diagnostic analyses. (a) OOF calibration curves at three prospective decision points; points show equal-frequency bins with Wilson 95% CIs. (b) Decision-curve analysis for non-pCR identification; shaded regions are bootstrap 95% CIs. (c) Condition-dependence analysis; MRI latents and geometry are fixed while treatment, clinical profile, or their joint context is permuted; shaded regions show $\pm 1$ SD across folds. (d) Response-state grounding; a ridge probe maps the frozen T0-to-T2 state update to observed log-volume shrinkage in 791 OOF patients.

Subtype	$n$ (pCR%)		T0	T0–T1	T0–T2 (95% CI)
Overall	808	(34.0)	0.733	0.755	0.762
HR+/HER2 $-$	320	(17.8)	0.683	0.689	0.705 [0.616, 0.784]
HER2+	201	(46.8)	0.712	0.741	0.731 [0.658, 0.801]
TNBC	287	(43.2)	0.640	0.669	0.707 [0.646, 0.765]

TABLE IV : Receptor-subtype stratification of CoRe-WM five-fold OOF AUROC at each prospective decision point. Values are pooled out-of-fold AUROC; brackets are patient-bootstrap 95% confidence intervals.

Cancer Drug Response Prediction

See all 21 papers

Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI
Aug 7, 2026Fidel Omar Tito Cruz, Neda Ghafouri, Zengyan Wang +3Breast Cancer Longitudinal Medical Image Analysis
University of Central Florida, Orlando, FL, USA
Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction
Jul 6, 2026Johannes Kiechle, Richard Osuala, Daniel M. Lang +5Graph Representation Learning 3D Medical Imaging
Technical University of Munich · TUM University Hospital Rechts der Isar · Helmholtz Munich +4
Neoadjuvant chemotherapy response prediction using pretreatment diffusion and contrast-enhanced magnetic resonance imaging with clinical variables
Aug 18, 2026Pablo García Marcos, Paula Puerta González, Guillermo Lorenzo +6Cancer Drug Response Prediction Clinical Outcome Prediction
Computer Science Department, University of Oviedo, Asturias, Spain · Biomedical Engineering Center (BME), University of Oviedo, Asturias, Spain · Electrical Engineering Department, University of Oviedo, Asturias, Spain +7

Aug 7, 2026J/K move · Enter open · S save

Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI

Fidel Omar Tito Cruz, Neda Ghafouri, Zengyan Wang, Pegah Khosravi+2

University of Central Florida, Orlando, FL, USA

Breast Cancer Longitudinal Medical Image Analysis Clinical Outcome Prediction Magnetic Resonance Imaging Oncology Cancer Drug Response Prediction

Open paper AskPDF arXiv

Pathologic complete response (pCR) is an important endpoint in neoadjuvant chemotherapy (NAC) for breast cancer, and predicting pCR from imaging during treatment could support treatment response assessment. Many existing imaging-based approaches rely on a single static timepoint, which fails to capture changes that occur during treatment. In this work, we present a longitudinal framework that combines a frozen 3D foundation encoder (Pillar-0) with our Temporal Dynamics Network (TDN) to predict treatment response from serial Dynamic Contrast-Enhanced (DCE) MRI acquired across four clinical timepoints from pre-treatment to pre-surgery. The TDN combines time-aware volumetric embeddings with clinical and treatment data to predict pCR. Evaluated on 982 patients from the combined I-SPY2 and ACRIN-6698 cohort, the proposed model achieves strong performance across all reported metrics when longitudinal 3D imaging is fused with clinical data (test AUROC: 73.6%, balanced accuracy: 69.1%). While clinical variables provide the strongest individual predictive signal, longitudinal 3D imaging contributes complementary information when fused with clinical data, improving pCR prediction. Our source code is available at: https://github.com/omarftt/longitudinal_temporal_pillar.

Abstract

Cancer Drug Response Prediction

Explore similar work

Longitudinal 3D Foundation Modeling for Neoadjuvant Breast Cancer Response Prediction from Serial DCE-MRI

Graph Representation Learning of Longitudinal Medical Imaging Trajectories for Treatment Response Prediction

Neoadjuvant chemotherapy response prediction using pretreatment diffusion and contrast-enhanced magnetic resonance imaging with clinical variables