ConvFormer3D-TAP: Phase/Uncertainty-Aware Front-End Fusion for Cine CMR View Classification Pipelines
Authors: Nafiseh Ghaffar Nia, Vinesh Appadurai, Suchithra V., Chinmay Rane, Daniel Pittman, James Carr, Adrienne Kline
Organizations: Center for Artificial Intelligence, Bluhm Cardiovascular Institute, Northwestern Medicine, Chicago, IL, USA · Department of Electrical and Computer Engineering, Northwestern University, Chicago, IL, USA · Department of Surgery, Northwestern University, Chicago, IL, USA · Department of Radiology, Northwestern University, Chicago, IL, USA; and Xtasis Inc., Chicago, IL, USA · Department of Cardiology, The Prince Charles Hospital; and School of Medicine, The University of Queensland, Queensland, Australia · Quantiphi Inc., USA
Reliable recognition of standard cine cardiac MRI views is essential because each view determines which cardiac anatomy is visualized and which quantitative analyses can be performed. Incorrect view identification, whether by a human reader or an automated deep learning system, can propagate errors into segmentation, volumetric assessment, strain analysis, and valve evaluation. However, accurate view classification remains challenging under routine clinical variability in scanner vendor, acquisition protocol, motion artifacts, and plane prescription. We present ConvFormer3D-TAP, a cine-specific spatiotemporal architecture that integrates 3D convolutional tokenization with multiscale self-attention. The model is trained using masked spatiotemporal reconstruction and uncertainty-weighted multi-clip fusion to enhance robustness across cardiac phases and ambiguous temporal segments. The design captures complementary cues: local anatomical structure through convolutional priors and long-range cardiac-cycle dynamics through hierarchical attention. On a cohort of 150,974 clinically acquired cine sequences spanning six standard cine cardiac MRI views, ConvFormer3D-TAP achieved 96% validation accuracy with per-class F1-scores >= 0.94 and strong calibration (ECE = 0.025; Brier = 0.040). Error analysis shows that residual confusions are concentrated in anatomically adjacent long-axis and LVOT/AV view pairs, consistent with intrinsic prescription overlap. These results support ConvFormer3D-TAP as a scalable front-end for view routing, filtering and quality control in end-to-end cMRI workflows.
Figures & tables
Figure 1: Overview of the six standardized cine CMR views used in this study. Representative frames illustrate the anatomical orientation for: Cine2 (two-chamber), Cine3 (three-chamber), Cine4 (four-chamber), Cine-SAX (short-axis), Cine-LVOT (left ventricular outflow tract), and Cine-AV (aortic valve). These views constitute the target label set for the proposed cine view-classification framework.
Figure 2: Architecture of ConvFormer3D-TAP. Phase-aware sampling selects 16-frame clips from 25-frame cine loops, which are encoded into spatiotemporal tokens by a 3D convolutional stem and refined by a multiscale Transformer bottleneck. A masked reconstruction decoder regularizes training, while global pooling and uncertainty-weighted multi-clip fusion yield stable study-level view predictions.
Metric
Train
Validation
Accuracy
0.99
0.96
Balanced Accuracy
0.99
0.96
MCC
0.99
0.95
Cohen’s κ
0.99
0.95
Log Loss
0.10
0.25
AUROC (macro OvR)
0.998
0.990
Table 1: Overall train and validation performance across all classes. Values summarize final model performance using standard multi-class metrics, including discrimination (AUROC/AUPRC), ranking (Top- k accuracy), and probability calibration (ECE, Brier score).
Class
Precision
Recall
F1
Sensitivity
Specificity
CINE_2
0.96
0.96
0.96
0.96
0.990
CINE_3
0.95
0.95
0.95
0.95
0.990
CINE_4
0.97
0.97
0.97
0.97
0.993
CINE_AV
0.95
0.94
0.94
0.94
0.990
CINE_LVOT
0.94
0.94
0.94
0.94
0.990
CINE_SAX
0.98
0.98
0.98
0.98
0.995
Table 2: Per-class validation performance for the six cine-CMR view categories. Metrics are reported per class using one-vs-rest evaluation, where sensitivity corresponds to recall and specificity corresponds to the true-negative rate across all other classes.
Figure 3: Row-normalized training confusion matrix produced by the evaluation pipeline. Diagonal concentration indicates strong within-class separability across cine view categories.
Figure 4: Row-normalized validation confusion matrix for the best-performing checkpoint. Most errors occur between anatomically similar long-axis views and between LVOT/AV outflow-tract views, reflecting clinically plausible ambiguity.
Figure 5: Learning curves for cine cMRI view classification. Training and validation accuracy across epochs show rapid early improvement followed by stable convergence, reaching approximately 99% training accuracy and 96% validation accuracy by the final epoch.
Department of Ultrasound, The First Affiliated Hospital of Chengdu Medical College, School of Clinical Medicine, Chengdu Medical College, Chengdu, China · College of Computer Science, Sichuan University, Chengdu, China · Department of Medical Ultrasound, West China Hospital of Sichuan University, Chengdu, China +1
Digital Technology and Innovation, Siemens Healthineers, Princeton, NJ, USA · Chair for Al in Healthcare and Medicine, Technical University of Munich (TUM) and TUM University Hospital, Munich, Germany · Department of Computing, Imperial College London, UK