ReG-SAM: Reference Graph-Driven SAM for 2D Foundational Vessel Segmentation

Donghang Lyu; Zichen Zhang; Oleh Dzyubachyk; Marius Staring

cs.CVSep 25, 2026

ReG-SAM: Reference Graph-Driven SAM for 2D Foundational Vessel Segmentation

Authors: Donghang Lyu, Zichen Zhang, Oleh Dzyubachyk, Marius Staring

Organizations: Division of Image Processing, Department of Radiology, Leiden University Medical Center, Leiden, the Netherlands

Abstract

Vessel segmentation in medical images is essential for many clinical tasks, ranging from diagnosis to treatment planning. However, it remains challenging due to complex vascular morphology and diverse imaging conditions. Existing deep learning methods rarely aim at building a generalizable vessel segmentor across anatomies and modalities. While the Seg- ment Anything Model (SAM) has shown promise for med- ical image segmentation, its original design does not fully exploit vascular morphology and struggles with fine-grained vascular structures, leading to suboptimal performance. In this paper, we propose ReG-SAM, a SAM-based framework tailored to 2D vessel segmentation that leverages reference graph set for enhancing vascular representations. Specifically, we introduce two modality-aware representations derived from the reference masks: graph prompt embeddings (GPEs) that encode global spatial features from graphs, and vascu- lar prototype embeddings (VPEs) that capture fine-grained modality-specific vessel characteristics from multi-scale fea- ture maps and vascular masks. Since both require vascular masks that are unavailable during inference and require robust modality-aware vascular feature representations, we construct a modality-wise vascular database and develop two reference graph-guided representation learning schemes for estimating GPEs and VPEs using samples from the database rather than ground-truth masks. Extensive experiments across 19 datasets demonstrate that ReG-SAM consistently outperforms existing baselines, even those using manual prompts, particularly on challenging thin vessels.

Figures & tables

Figure 1: An overview of the proposed ReG-SAM. Given a query image, reference graphs are randomly sampled from a vascular database, and their combined embeddings are aligned with the embedding representations of the query image during training.

Dataset	Modality
FS-CAD ( Zeng et al. 2024 )	X-Ray
XACV ( Wu et al. 2025a )	X-Ray
XCAD ( Ma et al. 2021 )	X-Ray
Dr-SAM ( Zohranyan et al. 2024 )	X-Ray
DIAS ( Liu et al. 2024 )	DSA
DSCA ( Zhang et al. 2025 )	DSA

Table 1: A summary of the datasets. DSA: digital subtraction angiography, PF: placental fetoscopy, OCTA: optical coherence tomography angiography, SLO: scanning laser ophthalmoscopy.

Method	Fundus		DSA		X-Ray		PF
Method	Dice $\uparrow$	IoU $\uparrow$	Dice $\uparrow$	IoU $\uparrow$	Dice $\uparrow$	IoU $\uparrow$	Dice $\uparrow$	IoU $\uparrow$
ASPS (auto)	85.17 $\pm$ 7.72	74.83 $\pm$ 9.87	79.34 $\pm$ 4.75	66.00 $\pm$ 6.45	91.51 $\pm$ 5.90	84.89 $\pm$ 9.78	80.79 $\pm$ 11.35	69.12 $\pm$ 14.22
MedSAM (box)	87.04 $\pm$ 8.54	77.89 $\pm$ 11.34	79.22 $\pm$ 4.85	65.86 $\pm$ 6.57	91.28 $\pm$ 6.28	84.56 $\pm$ 10.33	84.59 $\pm$ 8.14	74.08 $\pm$ 11.29
OVS-Net (point)	87.44 $\pm$ 8.19	78.46 $\pm$ 10.87	80.89 $\pm$ 4.57	68.15 $\pm$ 6.33	91.86 $\pm$ 5.75	85.46 $\pm$ 9.59	82.02 $\pm$ 10.66	70.75 $\pm$ 13.70
SAM-HQ (box)	80.80 $\pm$ 8.58	68.51 $\pm$ 10.29	73.74 $\pm$ 5.29	58.69 $\pm$ 6.60	87.82 $\pm$ 8.75	79.32 $\pm$ 13.33	71.77 $\pm$ 18.08	58.61 $\pm$ 19.06
SAM-HQ (point)	80.41 $\pm$ 8.08	67.89 $\pm$ 9.65	72.78 $\pm$ 5.54	57.51 $\pm$ 6.79	87.18 $\pm$ 9.07	78.38 $\pm$ 13.75	71.95 $\pm$ 18.80	58.97 $\pm$ 19.34

Table 2: Modality-wise, average, and zero-shot segmentation performance. The best results are highlighted in bold and the second-best in italic .

Figure 2: Radar chart comparing dataset-wise Dice scores of different methods.

Figure 3: Qualitative comparison of different methods. Challenging regions are highlighted by green boxes and enlarged in at the bottom left corner. Evaluation metrics for each case are shown at the top, with the highest values highlighted in yellow.

Configurations	Regular
Configurations	Dice $\uparrow$	IoU $\uparrow$
w/o 4 Upsampling	88.08 $\pm$ 8.46	79.61 $\pm$ 12.23
w/o GPEs	88.34 $\pm$ 8.32	80.00 $\pm$ 11.99
w/o VPEs & VPA	85.10 $\pm$ 10.36	75.32 $\pm$ 14.05
w/o $\mathcal{L}_{g}$	88.52 $\pm$ 8.24	80.28 $\pm$ 11.98
w/o $\mathcal{L}_{p}$	86.92 $\pm$ 8.09	77.67 $\pm$ 11.46

Table 3: Experimental results of the ablation studies on architectures and loss functions. The best results are highlighted in bold .

Figure 4: Performance comparison on the regular and zero-shot test sets using random and top- $k$ sampling strategies.

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 788 papers

Jun 23, 2026

MorVess: Morphology-Aware Pulmonary Vessel Segmentation Network

Accurate pulmonary vessel segmentation remains challenging due to the sparse, tortuous, and multi-scale nature of vascular structures, where small branches are easily lost and topology integrity is difficult to preserve under voxel-wise supervision. Existing deep segmentation models primarily optimize binary masks, lacking explicit geometric constraints, thus struggling to recover continuous tubular morphology and fine vascular connectivity. In this study, we introduce MorVess, a morphology-aware segmentation framework that integrates differentiable geometric priors with large-scale foundation model adaptation to achieve fine-grained vascular parsing. MorVess jointly predicts vessel masks, distance maps, and thickness maps, providing explicit supervision for vascular boundaries, centerline consistency, and smooth diameter transitions. A lightweight 2.5D adapter bridges 3D spatial context and 2D SAM representations, while a global-local fusion block aggregates multi-level semantics and geometric cues for high-fidelity topology reconstruction. Across two challenging pulmonary CT benchmarks, MorVess delivers superior Dice, clDice, and HD95 scores, substantially improving small-vessel recovery and global connectivity. These results demonstrate that embedding geometric intelligence into pretrained vision models offers a principled and scalable pathway toward precise vessel analysis and clinically reliable structural quantification. Our source code is available at https://github.com/MaoFuyou/MorVess.

Fuyou Mao, Yifei Chen, Beining Wu +9

Central South University, 410083 Changsha, China · Tsinghua University, 100084 Beijing, China · Hangzhou Dianzi University, 310018 Hangzhou, China +1

Vision Foundation Model Adaptation Topology-Preserving Image Segmentation Medical Image Segmentation 3D Medical Image Segmentation

Ask PDF

May 19, 2026

Uncertainty-Guided Conservative Propagation for Structured Inference in Vessel Segmentation

Accurate vessel segmentation is essential for medical image analysis, yet remains challenging due to complex vascular patterns and imaging ambiguity. Most deep models rely on single-pass prediction, limiting their ability to refine uncertain or disconnected regions during inference. To address this limitation, we propose Uncertainty-Guided Conservative Propagation (UGCP), a general plug-in module for vessel segmentation. Instead of directly using a one-shot output as the final prediction, UGCP performs a small number of logit-space update steps to refine the segmentation through local predictions interaction. Predictive uncertainty guides reliable regions to support ambiguous regions, while structure-aware modulation and source-based stabilization reduce unreliable propagation and excessive drift. The module is differentiable and can be trained end-to-end with different segmentation networks. We evaluate UGCP on four public vessel segmentation datasets covering 2D and 3D tasks, including retinal vessel, coronary artery, and cerebral vessel segmentation. Experiments with convolutional neural network-based and Transformer-based backbones show consistent improvements in Dice similarity coefficient, centerline Dice, and 95th percentile Hausdorff distance. Further analysis demonstrates that UGCP reduces vessel disconnections and improves structural consistency with limited additional computation. The code will be made available at https://github.com/chenzhao2023/UGC_PR.

Huan Huang, Michele Esposito, Chen Zhao

Department of Computer Science, Kennesaw State University, Marietta, 30060, GA, USA · Department of Cardiology, Medical University of South Carolina, Charleston, SC, USA

Image Segmentation Medical Image Segmentation 3D Medical Image Segmentation

Ask PDF

May 25, 2026

VesselSim: learning 3D blood vessel segmentation without expert annotations

Blood vessel segmentation is a core task in medical image analysis for the care of vascular diseases and surgical planning, yet the challenges of providing expert vascular annotations pose a major obstacle for the progress of related deep learning techniques. To address this, we propose VesselSim, a two-stage framework for universal 3D blood vessel segmentation that eliminates the need for real annotated data during training. First, we introduce a stochastic, geometry-driven vascular simulation framework that models recursive branching, curvature-controlled growth, and collision-aware topology, followed by domain-randomized intensity synthesis to generate 16,500 anatomically plausible 3D angiographic volumes. Second, a 3D U-Net is trained solely on this synthetic data. To bridge the domain gap from synthetic to real images at inference time, we introduce a test-time adaptation strategy via a self-supervised mask reconstruction decoder, enabling adaptation to unseen clinical scans without prior domain knowledge. We evaluate VesselSim in a zero-shot setting on multiple real-world datasets spanning MR and CT across several anatomical regions, including the brain and kidneys. Despite being trained exclusively on synthetic data, VesselSim achieves performance competitive with state-of-the-art vascular segmentation foundation models. These findings suggest that learning vessel geometry from synthetic tubular structures is effective for robust cross-domain generalization, substantially reducing the reliance on acquired medical imaging data and more importantly, expert annotations.

Erin Rainville, Melissa Ananian, Tristan Mirolla +2

Department of Electrical and Computer Engineering, Concordia University, Montreal, Canada · Department of Computer Science and Software Engineering, Concordia University, Montreal, Canada

Synthetic-to-Real Domain Adaptation Synthetic Data Augmentation Image Segmentation Medical Image Segmentation 3D Medical Image Segmentation

Ask PDF

Abstract

Medical Image Segmentation

Explore similar work

MorVess: Morphology-Aware Pulmonary Vessel Segmentation Network

Uncertainty-Guided Conservative Propagation for Structured Inference in Vessel Segmentation

VesselSim: learning 3D blood vessel segmentation without expert annotations