A Unified Model for Highly Accurate ECG-Free Dynamic Coronary Roadmapping Using Spatio-Temporal Transformers
Authors: Saahil Islam, Sebastian Piat, Venkatesh N. Murthy, Serkan Cimen, Puneet Sharma, Andreas Maier, Florin C. Ghesu
Organizations: Pattern Recognition Lab, Friedrich Alexander University, Erlangen, Germany · Digital Technology and Innovation, Siemens Healthineers, Erlangen, Germany · Digital Technology and Innovation, Siemens Healthineers, Princeton, USA
Abstract
Percutaneous Coronary Intervention (PCI) is a minimally invasive procedure used to restore coronary blood flow obstructed by atherosclerotic plaque. During PCI, repeated injections of iodine-based contrast agents are required to visualize the coronary arteries and guide interventional devices. However, frequent contrast injections increase radiation exposure and the risk of contrast-induced nephropathy, with acute kidney injury reported in up to 30% of patients with renal impairment. Dynamic Coronary Roadmapping (DRM) reduces these risks by overlaying a precomputed angiographic vessel map onto live fluoroscopy and continuously updating it throughout the procedure. Accurate DRM relies on precise cardiac phase matching between angiography and fluoroscopy, together with reliable catheter tip tracking for motion compensation. These tasks remain challenging in ECG-free settings and when only limited manual annotations are available. We present a unified DRM framework that simultaneously performs cardiac phase matching and catheter tip tracking for accurate real-time guidance. Our method employs a large-scale spatio-temporal encoder pretrained on 16 million X-ray frames to learn cardiac motion dynamics. To the best of our knowledge, this is the first application of large-scale spatio-temporal pretraining for motion compensation in DRM. We further introduce auxiliary tasks based on ECG R-peak detection and catheter tip tracking, improving optimization while eliminating the need for extensive catheter mask annotations. Finally, a majority-voting postprocessing strategy aggregates temporal predictions, improving robustness and providing a confidence score that correlates with phase-matching error. Comprehensive evaluation on clinical X-ray datasets demonstrates state-of-the-art performance, achieving low temporal misalignment and robust phase-matching accuracy suitable for real-time DRM.
Accurate 4D whole-heart mesh reconstruction from sparse cine MRI is critical for creating cardiac digital twins, but remains challenging due to limited 2D slice coverage and the complex coupling between cardiac shape and motion. Existing methods often rely on intermediate contour fitting and typically reconstruct static, single-phase, or partial cardiac geometries, limiting their ability to capture full-chamber dynamics. We propose a novel end-to-end framework for reconstructing temporally resolved whole-heart meshes from multi-view 2D cine MRI sequences by learning an image-to-mesh mapping. The framework incorporates a differentiable contour renderer inspired by the Beer-Lambert attenuation principle, enabling anatomy-aware supervision of 3D+t mesh deformation through contour-based projection losses. To improve temporal consistency across the cardiac cycle, we further introduce a multi-scale temporal modeling module that integrates global cycle-level dynamics with local inter-frame coherence to generate smooth and physiologically plausible mesh trajectories. The proposed method achieved a whole-heart mean absolute error of 1.68 ± 0.31 mm and a motion jitter of 0.77 ± 0.17 mm/frame3, outperforming existing methods with lower reconstruction error and substantially improved motion smoothness. It also improved 2D contour alignment across multiple cine MRI views and supported downstream proof-of-concept electrophysiological simulation. The code will be released publicly upon acceptance of the manuscript for publication.
Real-time cardiac cine MRI enables visualization of the beating heart during free breathing, but severe undersampling and motion make reconstruction highly challenging. A central challenge for reconstruction is incorporating powerful priors of cardiac anatomy while remaining computationally efficient. We propose Piecewise Dynamic Diffusion Regularization (PDDR), a reconstruction method that integrates a spatiotemporal diffusion model as a generative prior within a variational reconstruction framework for cine MRI. The model employs dedicated spatial layers to encode anatomical structure and temporal layers to capture cardiac motion learned from gated cine data. PDDR leverages the dynamic prior in a piecewise manner, enabling the efficient use of spatiotemporal diffusion models for processing of long real-time sequences. Experiments on retrospectively accelerated and prospective real-time cine MRI demonstrate that PDDR outperforms classical, unsupervised, and diffusion-based methods, delivering high-quality reconstructions with substantially reduced computation time compared to state-of-the-art baselines. These results highlight PDDR as a practical and scalable solution for free-breathing, real-time cardiac MRI. Code is available at https://github.com/MLI-lab/pddr.
Accurate correspondence matching across multiple angiographic views is the prerequisite for 3D coronary reconstruction and interventional guidance. However, the development of robust deep learning models for this task has been stifled by a fundamental data bottleneck. Obtaining ground truth for matching tasks in angiography pairs is prohibitively expensive and hard to scale. To overcome this barrier, we introduce a physically-grounded data generation framework that synthesizes high-fidelity Digital Reconstructed Radiographs (DRRs) from 3D Coronary CT Angiography (CCTA) volumes. Our framework generates dense, highly accurate 3D-to-2D projection labels by simulating realistic C-arm acquisition geometry on patient anatomy at zero human cost. Leveraging this dense supervision, we propose a Geometry-Informed Matching Module (GIMM) that integrates global feature and anatomical structure into correspondence learning. Unlike real angiography where assessment relies on subjective human annotation, our dataset provides 2D correspondence labels with paired images, allowing human-free evaluation. We comprehensively evaluate our method on the proposed CT-derived DRR dataset and demonstrate improvements over other matching baseline models.