Organizations: State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · DAMO Academy, Alibaba Group, Hangzhou 310023, China · Hupan Lab, Hangzhou 310023, China · Joint Laboratory of Intelligence Science and Technology, Institute of Systems Engineering, Macau University of Science and Technology, Taipa, Macao, China
Objective: X-ray angiography is the gold standard imaging modality for cardiovascular diseases. However, current deep learning approaches for X-ray angiogram analysis are severely constrained by the scarcity of annotated data. While large-scale self-supervised learning (SSL) has emerged as a promising solution, its potential in this domain remains largely unexplored, primarily due to the lack of effective SSL frameworks and large-scale datasets. Methods: To bridge this gap, we introduce a vascular anatomy-aware masked image modeling (VasoMIM) framework that explicitly integrates domain-specific anatomical knowledge. Specifically, VasoMIM comprises two key designs: an anatomy-guided masking strategy and an anatomical consistency loss. The former strategically masks vessel-containing patches to compel the model to learn robust vascular semantics, while the latter preserves structural consistency of vessels between the original and reconstructed images, enhancing the discriminability of the learned representations. In conjunction with VasoMIM, we curate XA-170K, the largest X-ray angiogram pre-training dataset to date. Results: We validate VasoMIM on four downstream tasks across six datasets, where it demonstrates superior transferability and achieves state-of-the-art performance compared to existing methods. Conclusion: Incorporating vascular anatomical knowledge into self-supervised pre-training enables more discriminative and transferable representations for X-ray angiogram analysis. Significance: These findings highlight the significant potential of VasoMIM as a foundation model for advancing a wide range of X-ray angiogram analysis tasks. VasoMIM and XA-170K are available on GitHub and Hugging Face, respectively.
Figures & tables
Fig. 1 : Conceptual comparison. (a) Conventional MIM masks patches based on generic rules and utilizes standard supervision to reconstruct the original image. (b) VasoMIM employs anatomical guidance to selectively mask vessel-relevant regions (highlighted in yellow) and enforces structural consistency via anatomical supervision , enabling the model to learn richer vascular representations.
Fig. 2 : Generic vs. anatomy-guided masking strategies. While generic methods like AMT [ 11 ] and HPM [ 12 ] fail to prioritize vascular structures, VasoMIM effectively focuses on vessel-relevant regions. Red denotes a higher masking probability, while blue indicates the opposite.
Aspect
VasoMIM-v1 [ 13 ]
VasoMIM
Main Contribution
Anatomical Guidance
Instance-level guidance from Frangi filter
Incorporating population-level guidance learned from large-scale X-ray angiograms
More reliable anatomical guidance by complementing handcrafted vesselness cues with learned anatomical knowledge
Pre-training Dataset
20 K X-ray angiograms
170 K X-ray angiograms
The largest publicly available dataset for X-ray angiogram pre-training
Downstream Validation
1 task (vessel segmentation) on 3 datasets
4 tasks (segmentation and detection) on 6 datasets
More comprehensive evaluation across diverse X-ray angiogram analysis tasks
Experimental Analysis
Basic ablation studies
Extended ablations and analyses of scalability, label efficiency, pre-training datasets, and robustness
More in-depth understanding of the effectiveness and generalizability of VasoMIM
TABLE I : Comparison between the preliminary conference version VasoMIM-v1 [ 13 ] and this extended version.
Fig. 3 : Overview of VasoMIM. Given an input X-ray angiogram, VasoMIM constructs anatomical guidance from two complementary sources: instance-level vesselness cues extracted by Frangi filter and population-level anatomical knowledge learned by a lightweight segmentor from the large-scale XA- 170 K. Based on the co-guidance from these two sources, a patch-wise vascular anatomical distribution f(gi) is computed to guide the masking strategy, thereby prioritizing vessel-relevant regions during pre-training. The masked image is then reconstructed by an encoder-decoder model. Finally, the model is optimized by minimizing the total objective LMIM , which combines the standard pixel-level reconstruction loss Lrec. and the anatomical consistency loss Lcons. to learn discriminative vascular representations.
Usage
Dataset
#train
#test
Pre-training
CADICA [ 47 ]
6,594
−
SYNTAX [ 48 ]
2,943
−
XCAD [ 49 ]
1,621
−
CoronaryDominance [ 50 ]
160,320
−
Total
171,478
−
Segmentation
ARCADE-V [ 51 ]
1,000
300
TABLE II : Statistics of XA- 170 K and downstream datasets.
Fig. 4 : Overview of downstream tasks. We evaluate VasoMIM on four distinct tasks: vessel segmentation, vessel segment segmentation, stenosis segmentation, and stenosis detection.
Method
Pre-training Data
ARCADE-V
CAXF
XACV
ARCADE-S
ARCADE-VS
Avg. Rank
DSC (%)
clDice (%)
DSC (%)
clDice (%)
DSC (%)
clDice (%)
DSC (%)
DSC (%)
Traditional
Frangi Filter [ 19 ] [ MICCAI ’98]
−
41.30
40.91
64.01
65.73
58.46
57.15
−
−
−
From Scratch
UNet [ 37 ] [ MICCAI ’15]
−
71.44±0.32
70.67±0.40
82.76±0.75
84.95±0.56
78.18±0.34
74.71±0.44
27.04±3.65
38.77±0.82
23.00±0.93
nnUNet [ 53 ] [ Nat. Methods ’21]
−
75.85±0.40
77.82±0.16
87.26±0.18
90.31±0.26
84.05±0.17
82.63±0.15
43.14±1.31
47.86±2.37
13.88±4.55
TABLE III : Quantitative comparison on segmentation tasks. We report “ mean±std ” over five random seeds for all learning-based methods. Since Frangi filter is deterministic, we report only “ mean ”. The best results are highlighted in bold , and the second-best are underlined . Paired t -tests are conducted between VasoMIM-v1 and VasoMIM based on five seed-level results. ∗ : p<0.05 , ∗∗ : p<0.01 , ∗∗∗ : p<0.001 .
Fig. 5 : Qualitative analysis of the anatomy-guided masking strategy. Left: The proportion of vessel-containing patches among all masked patches across training epochs. Right: Visual comparison. We highlight failure cases of Frangi filter where UNeXt-S [ 20 ] , leveraging a strong inductive bias, extracts vascular anatomy more accurately to provide precise guidance. The heatmaps display the patch-wise masking ratio, i.e. , \frac{1}{E}\sum_{j=1}^{E}\mathbb{I}\left(\text{Patch x_{i}ismaskedinepochj}\right) .
Method
Pre-training Data
mAP 50 (%)
mAP 75 (%)
mAP (%)
Avg. Rank
From Scratch
Faster R-CNN [ 38 ] [ TPAMI ’16]
−
88.37±0.90
19.01±1.77
36.63±0.63
20.33±0.58
DETR [ 67 ] [ ECCV ’20]
−
89.30±0.79
19.35±2.00
36.51±0.57
19.00±2.65
General SSL
MoCo v3 [ 24 ] [ ICCV ’21]
XA- 170 K
91.54±1.04
21.74±0.67
38.74±0.32
14.00±1.00
DINO [ 25 ] [ ICCV ’21]
XA- 170 K
88.83±0.46
21.12±0.78
37.65±0.41
17.00±1.00
TABLE IV : Quantitative results for the stenosis detection task. We report “ mean±std ” over five random seeds. The best and second-best results are highlighted in bold and underlined , respectively. Paired t -tests are conducted between VasoMIM-v1 and VasoMIM based on five seed-level results. ∗ : p<0.05 , ∗∗ : p<0.01 , n.s. : not significant.
Guidance
Lcons.
ARCADE-V
XACV
DSC (%)
clDice (%)
DSC (%)
clDice (%)
−
−
79.31±0.16 ∗∗∗
81.05±0.25 ∗∗∗
84.52±2.12 n.s.
82.25±2.32 n.s.
−
✓
79.85±0.14 ∗
81.59±0.16 ∗
85.79±0.52 n.s.
83.66±0.77 n.s.
↑0.54
↑0.54
↑1.27
↑1.41
✓
−
79.87±0.09 ∗∗
81.41±0.11 ∗∗
85.92±0.04 ∗
83.78±0.15 ∗
↑0.56
↑0.36
↑1.40
↑1.53
TABLE V : Ablation study of the anatomy-guided masking strategy and the anatomical consistency loss. Paired t -tests are conducted between each variant and VasoMIM using five seed-level results. ∗ : p<0.05 , ∗∗ : p<0.01 , ∗∗∗ : p<0.001 , n.s. : not significant.
Loss
Params. (M)
FLOPs (G)
ARCADE-V
XACV
Baseline
−
111.91
19.01
79.87±0.09
85.92±0.04
Feature Distillation
DINO [ 25 ] [ ICCV ’21]
197.95
52.90
79.81±0.12
85.03±2.27
+86.04
+33.89
↓0.06
↓0.89
CLIP [ 68 ] [ ICML ’21]
169.43
41.66
79.79±0.13
85.29±1.47
TABLE VI : Comparison with alternative reconstruction objectives. GFLOPs are calculated on an NVIDIA H 20 GPU using a single 224×224 masked RGB image. DSC is reported in this table.
Loss
SS ↑
CHI ↑
DBI ↓
Baseline
−
0.27
1.39
1.37
Feature Distillation
DINO [ 25 ] [ ICCV ’21]
0.03
0.06
6.42
CLIP [ 68 ] [ ICML ’21]
0.08
0.18
3.71
DINOv3 [ 56 ] [ arXiv ’25]
0.29
1.65
1.24
TABLE VII : Comparison of clustering metrics on XACV training set .
Guidance
ARCADE-V
XACV
DSC (%)
clDice (%)
DSC (%)
clDice (%)
−
79.85±0.14 ∗
81.59±0.16 ∗
85.79±0.52 n.s.
83.66±0.77 n.s.
Frangi-only [ 13 ] [ AAAI ’26]
79.90±0.16 ∗∗
81.57±0.28 ∗∗
85.80±0.27 ∗
83.75±0.37 ∗
↑0.05
↓0.02
↑0.01
↑0.09
Segmentor-only
79.98±0.13 ∗∗∗
81.77±0.19 ∗∗
85.98±0.08 ∗∗
83.91±0.13 ∗
↑0.13
↑0.18
↑0.19
↑0.25
TABLE VIII : Comparison between instance-level and population-level anatomical guidance. Paired t -tests are conducted between each variant and VasoMIM using five seed-level results. ∗ : p<0.05 , ∗∗ : p<0.01 , ∗∗∗ : p<0.001 , n.s. : not significant.
Fig. 6 : Qualitative comparison between VasoMIM-v1 [ 13 ] and VasoMIM on vessel segmentation (ARCADE-V), stenosis segmentation (ARCADE-S), vessel segment segmentation (ARCADE-VS), and stenosis detection (Stenosis). Yellow boxes highlight improved structural continuity, lesion delineation, or localization.
Fig. 7 : Hyper-parameter sensitivity of the mixing coefficient η on ARCADE-V and XACV. A larger η assigns a higher relative weight to Frangi-derived instance-level guidance, with η=0 and η=1 corresponding to segmentor-only and Frangi-only guidance, respectively. All results are averaged over five random seeds.
Guidance
Format
Sampling
ARCADE-V
XACV
−
−
Random
79.85±0.14 ∗
85.79±0.52 n.s.
Co-guidance
Soft
Uniform
79.99±0.14 ∗
85.94±0.06 ∗
↑0.14
↑0.15
Hard
f(gi)
80.03±0.04 ∗
85.98±0.03 ∗
↑0.18
↑0.19
80.25±0.12
86.09±0.09
TABLE IX : Ablation of patch sampling strategy and guidance format. DSC is reported in this table. “Format” refers to the segmentor output used in guidance: “Hard” denotes a binarized mask, “Soft” denotes a probability map, and “ − ” indicates that no segmentor output is used. Paired t -tests are conducted between each variant and VasoMIM using five seed-level results. ∗ : p<0.05 , n.s. : not significant.
Case
Manner
β0
βE
ARCADE-V
XACV
Random
−
0
0
79.85±0.14
85.79±0.52
Guided
Weak-to-Strong
80.25±0.12
86.09±0.09
0
0.5
↑0.40
↑0.30
0
0.75
79.91±0.13
85.99±0.07
↑0.06
↑0.20
0
1
79.96±0.06
85.94±0.13
TABLE X : Evaluation of different masking strategies. DSC is reported in this table.
Fig. 8 : Data scaling analysis. We evaluate the impact of increasing pre-training data size from 10 K to 170 K on downstream performance. Left: ARCADE-V. Right: XACV.
Fig. 9 : Scalability analysis of VasoMIM. We evaluate the impact of scaling the model from ViT-B to ViT-L and ViT-H on downstream performance. Left: ARCADE-V. Right: XACV.
Fig. 10 : Label-efficient fine-tuning on ARCADE-V. VasoMIM consistently outperforms competing SSL alternatives, notably achieving superior performance with only 10% of labels compared with nnUNet [ 53 ] and TransUNet [ 41 ] trained from scratch using 100% of labels. ∗ : p<0.05 , ∗∗ : p<0.01 , ∗∗∗ : p<0.001 (paired t -test).
Method
Pre-training Data
ARCADE-V
XACV
MAE [ 14 ] [ CVPR ’22]
ImageNet- 1 K
78.10±0.21
83.69±1.48
XA- 170 K
79.39±0.15
84.84±1.79
Δ
↑1.29
↑1.15
LocalMIM [ 54 ] [ CVPR ’23]
ImageNet- 1 K
77.55±0.66
83.00±0.61
XA- 170 K
78.79±0.20
83.18±1.84
Δ
↑1.24
↑0.18
TABLE XI : Impact of pre-training dataset. We compare models pre-trained on generic, medical-specific, and our XA- 170 K datasets. DSC is reported in this table.
Fig. 11 : Segmentor-based Pre-training (SPT) vs. MIM. VasoMIM significantly outperforms SPT, achieving DSC improvements of +3.77% on ARCADE-V and +4.09% on XACV. ∗∗∗ : p<0.001 (paired t -test).
Fig. 12 : Hyper-parameter sensitivity of the masking ratio γ on ARCADE-V. All results are averaged over five random seeds.
α
Method
ARCADE-V
XACV
DSC (%)
clDice (%)
DSC (%)
clDice (%)
50
Frangi
12.06
8.20
32.16
22.58
VasoMIM
79.71±0.12
81.26±0.21
86.02±0.08
84.05±0.12
80
Frangi
27.06
22.34
66.70
62.51
VasoMIM
79.67±0.18
81.31±0.22
86.03±0.06
84.03±0.18
92
Frangi
41.30
40.91
58.46
57.15
TABLE XII : Sensitivity analysis of VasoMIM to the percentile threshold α of Frangi filter.
Filter
ARCADE-V
XACV
DSC (%)
clDice (%)
DSC (%)
clDice (%)
−
79.31±0.16
81.05±0.25
84.52±2.12
82.25±2.32
Canny † [ 73 ] [ TPAMI ’86]
79.32±0.12
80.86±0.24
85.73±0.06
83.60±0.09
↑0.01
↓0.19
↑1.21
↑1.35
Sato ‡ [ 74 ] [ MedIA ’98]
79.99±0.14
81.53±0.31
85.84±0.09
83.75±0.09
{\color[rgb]{0,0.707,0}\uparrow 0.68}
↑0.48
↑1.32
↑1.50
TABLE XIII : Evaluation of different filters. † : General edge detector. ‡ : Vesselness filter. − denotes the vanilla baseline, i.e., MAE ( γ=0.5 ), without anatomy-guided masking or anatomical consistency loss.
L
ARCADE-V
XACV
DSC (%)
clDice (%)
DSC (%)
clDice (%)
−
79.87±0.09
81.41±0.11
85.92±0.04
83.78±0.15
L 1
79.64±0.17
81.21±0.24
85.87±0.08
83.85±0.07
↓0.23
↓0.20
↓0.05
↑0.07
Dice
79.94±0.11
81.64±0.19
85.93±0.08
84.04±0.14
↑0.07
↑0.23
↑0.01
↑0.26
TABLE XIV : Evaluation of different metric functions.
School of Mathematical Sciences, Ocean University of China, China · School of Haide, Ocean University of China, China · School of Intelligent Science and Technology, Nanjing University, China +2
Department of Electrical and Computer Engineering, Concordia University, Montreal, Canada · Department of Computer Science and Software Engineering, Concordia University, Montreal, Canada