Mask-HybridGNet: Graph-based segmentation with emergent anatomical correspondence from pixel-level supervision
Authors: Nicolás Gaggion, Maria J. Ledesma-Carbayo, Stergios Christodoulidis, Maria Vakalopoulou, Enzo Ferrante
Organizations: APOLO Biotech, Ciudad Autónoma de Buenos Aires, Argentina · Departamento de Computación, Universidad de Buenos Aires, Ciudad Autónoma de Buenos Aires, Argentina · Instituto de Ciencias de la Computación (ICC), CONICET-Universidad de Buenos Aires, Ciudad Autónoma de Buenos Aires, Argentina · Biomedical Image Technologies, ETSI Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain · Centro de Investigación Biomédica en Red de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Instituto de Salud Carlos III, Madrid, Spain · IHU PRISM National Precision Medicine Center in Oncology, Cancer Data Science Unit, CentraleSupélec, Gustave Roussy, INSERM, Université Paris-Saclay, Gif-sur-Yvette, France · MICS, CentraleSupélec, Université Paris-Saclay, Gif-sur-Yvette, France
Graph-based medical image segmentation represents anatomical structures using boundary graphs, providing fixed-topology landmarks and inherent population-level correspondences. However, their clinical adoption has been hindered by a major requirement: training datasets with manually annotated landmarks that maintain point-to-point correspondences across patients rarely exist in practice. We introduce Mask-HybridGNet, a framework that trains graph-based models directly using standard pixel-wise masks, eliminating the need for manual landmark annotations. Our approach aligns variable-length ground truth boundaries with fixed-length landmark predictions by combining Chamfer distance supervision and edge-based regularization to ensure local smoothness and regular landmark distribution, further refined via differentiable rasterization. A significant emergent property of this framework is that predicted landmark positions become consistently associated with specific anatomical locations across patients without explicit correspondence supervision. This implicit atlas learning enables temporal tracking, cross-slice reconstruction, and morphological population analyses. Beyond direct segmentation, Mask-HybridGNet can extract correspondences from existing segmentation masks, allowing it to generate stable anatomical atlases from any high-quality pixel-based model. Experiments across chest radiography, cardiac ultrasound, cardiac MRI, and fetal imaging demonstrate that our model achieves competitive results against state-of-the-art pixel-based methods, while ensuring anatomical plausibility by enforcing boundary connectivity through a fixed graph adjacency matrix. This framework leverages the vast availability of standard segmentation masks to build structured models that maintain topological integrity and provide implicit correspondences.
Figures & tables
Figure 1: Framework overview. Mask-HybridGNet enables training graph-based segmentation models with standard pixel-level supervision, without manual landmark annotations. (Left) Given a set of medical imaging datasets with potentially heterogeneous pixel-level masks, we first automatically extract variable-length contour pixels from the masks. (Middle) Graph structures are generated from dataset statistics, which determine fixed landmark counts and construct adjacency matrices. This allows for training our Mask-HybridGNet models. (Right) Our model produces fixed-topology boundary graphs where landmark indices implicitly represent consistent anatomical locations across patients, without explicit correspondence supervision.
Figure 2: Architectural overview of the Mask-HybridGNet framework. (Left) The standard Mask-HybridGNet architecture couples a CNN encoder with a graph decoder via Image-to-Graph Skip Connections (IGSC). (Right) The Dual variant introduces an auxiliary CNN decoder following a U-Net topology. In this setup, the IGSC layers are redirected to sample feature maps from the auxiliary decoder instead of the encoder. Both variants share the same graph decoder, utilize a variational bottleneck for shape modeling, and are trained using a combination of Chamfer distance, edge-based regularization, and differentiable rasterization. The Dual model additionally incorporates a pixel-wise loss for the auxiliary decoder branch.
Figure 3: Graph representations for chest X-ray and echocardiograph anatomical structures. The figure illustrates the two graph representation strategies employed in our framework. Left : Independent graph representations treat each anatomical structure separately with circular graphs. For chest X-ray: left lung (blue), right lung (green), heart (orange), left clavicle (cyan), and right clavicle (purple). For echocardiography: left ventricular endocardium (red), left ventricular epicardium (green), and left atrium (blue). Right : Unified graph representations model share anatomical boundaries, where nodes can belong to multiple organs. Cyan contour segments indicate shared interfaces between adjacent structures, enabling joint modeling of anatomically connected regions.
Lungs
Heart
Clavicles
Model
Graph Type
Shenzhen
Montgomery
Padchest
JSRT
Padchest
JSRT
JSRT
Baseline HybridGNet (MSE)
Independent
0.963 ± 0.015
0.964 ± 0.031
0.956 ± 0.023
0.973 ± 0.010
0.939 ± 0.022
0.936 ± 0.030
0.816 ± 0.068
nnUNet
-
0.969 ± 0.014
0.982 ± 0.009
0.965 ± 0.017
0.979 ± 0.009
0.947 ± 0.016
0.953 ± 0.025
0.951 ± 0.018
Mask-HybridGNet
Independent
0.961 ± 0.015
0.968 ± 0.018
0.953 ± 0.016
0.967 ± 0.014
0.925 ± 0.038
0.928 ± 0.038
0.801 ± 0.067
Mask-HybridGNet Dual
Independent
0.965 ± 0.014
0.974 ± 0.019
0.956 ± 0.022
0.975 ± 0.011
0.939 ± 0.019
0.941 ± 0.027
0.860 ± 0.077
Mask-HybridGNet
Unified
0.963 ± 0.014
0.974 ± 0.015
0.954 ± 0.017
0.972 ± 0.011
0.940 ± 0.022
0.935 ± 0.034
0.836 ± 0.068
Table 1: Dice Coefficient Scores on the test-set of the Chest-xray-landmark database (mean ± std).
Figure 4: Emergent anatomical correspondences learned through mask supervision. The figure demonstrates that our framework successfully learns anatomically meaningful landmark correspondences without explicit point-level supervision. The left column shows the ground-truth contours, middle column results from independent graph representations, while the right column show unified graph representations. Each row displays a subset of organs for two different subjects, revealing consistent anatomical locations on a subset of equidistant nodes.
LV Endo
LV Epi
LA
Model
Graph Type
DC ↑
HD ↓
ASSD ↓
DC ↑
HD ↓
ASSD ↓
DC ↑
HD ↓
ASSD ↓
nnUNet
-
0.940 ± 0.027
13.686 ± 5.890
4.575 ± 1.807
0.873 ± 0.048
17.203 ± 6.698
4.889 ± 1.740
0.900 ± 0.076
16.782 ± 12.305
5.491 ± 4.048
Mask-HybridGNet
Independent
0.925 ± 0.042
17.285 ± 7.151
5.819 ± 3.021
0.840 ± 0.076
22.181 ± 8.755
6.405 ± 2.557
0.884 ± 0.083
19.105 ± 11.609
6.563 ± 4.837
Mask-HybridGNet Dual
Independent
0.929 ± 0.039
16.665 ± 7.997
5.452 ± 2.764
0.850 ± 0.065
21.279 ± 8.182
5.951 ± 2.213
0.889 ± 0.074
17.939 ± 11.155
6.192 ± 4.065
Mask-HybridGNet
Unified
0.926 ± 0.038
18.678 ± 8.240
5.699 ± 2.625
0.850 ± 0.058
21.713 ± 9.529
5.896 ± 2.262
0.875 ± 0.094
18.984 ± 10.305
6.836 ± 4.638
Mask-HybridGNet Dual
Unified
0.930 ± 0.035
17.296 ± 7.777
5.382 ± 2.453
0.855 ± 0.061
20.182 ± 8.487
5.645 ± 2.184
0.886 ± 0.085
18.234 ± 11.393
6.223 ± 4.133
Table 2: Performance comparison on CAMUS dataset showing Dice coefficients (DC), Hausdorff distances (HD), and Average Symmetric Surface Distances (ASSD) for cardiac structure segmentation.
Figure 5: Temporal cardiac analysis demonstrating landmark correspondence tracking using a unified graph representation. The figure displays comprehensive cardiac segmentation and tracking for a representative test patient with normal systolic function (EF = 54%) across two-chamber (top two rows) and four-chamber (bottom two rows) views. For each view, the first row compares the model prediction against the ground truth at end-diastole and end-systole. The second row visualizes the temporal motion tracking of landmarks for the left ventricular endocardium (LV Endo), epicardium (LV Epi), and left atrium (LA).
Dice Coefficient ↑
HD (pixels) ↓
ASSD (pixels) ↓
Organ
Graph Type
GT Input
nnUNet Input
GT Input
nnUNet Input
GT Input
nnUNet Input
LA
Independent
0.988 ± 0.005
0.986 ± 0.004
3.06 ± 1.425
2.834 ± 1.216
0.709 ± 0.204
0.746 ± 0.131
LA
Unified
0.99 ± 0.004
0.989 ± 0.003
2.852 ± 1.053
2.532 ± 0.707
0.613 ± 0.183
0.595 ± 0.084
LV Endo
Independent
0.99 ± 0.002
0.989 ± 0.002
3.096 ± 0.978
2.774 ± 0.694
0.769 ± 0.133
0.748 ± 0.087
LV Endo
Unified
0.991 ± 0.002
0.99 ± 0.002
6.224 ± 3.407
4.472 ± 1.866
0.7 ± 0.102
0.683 ± 0.063
LV Epi
Independent
0.979 ± 0.005
0.981 ± 0.004
5.181 ± 2.622
3.663 ± 1.871
0.842 ± 0.152
0.734 ± 0.09
Table 3: Agreement between graph segmentation outputs and input masks. Metrics evaluate Dice, HD and ASSD between the output of the Mask-HybridGNet model and the specific input mask (GT vs. nnUNet) provided.
Figure 6: Learned landmark correspondences across cardiac MRI slices demonstrate both cross-slice intra-subject and inter-subject tracking capabilities. The figure shows two different slices from the same subject in the Sunnybrook Cardiac Data (left), two different slices from different subjects (right).
LV Endo
LV Myo
Model
Graph Type
DC ↑
HD ↓
ASSD ↓
DC ↑
HD ↓
ASSD ↓
nnUNet
-
0.919
3.559
1.186
0.734
3.979
0.997
Mask-HybridGNet
Independent
0.893
4.014
1.535
0.655
4.508
1.385
Mask-HybridGNet Dual
Independent
0.903
3.744
1.417
0.673
4.553
1.305
Table 4: Performance comparison on Sunnybrook Cardiac Data showing Dice coefficients (DC), Hausdorff distances (HD), and Average Symmetric Surface Distances (ASSD) for left ventricular segmentation.
Tested on HC18
Tested on JNU-IFM
Tested on PSFHS
Model
DC ↑
HD ↓
ASSD ↓
N
DC ↑
HD ↓
ASSD ↓
N
DC ↑
HD ↓
ASSD ↓
N
Fetal Head Segmentation
Mask-HybridGNet Dual
Multi-centric
0.970
14.942
5.440
101
0.951
31.664
11.695
406
0.924
12.302
4.393
510
HC18-only
0.975
11.375
4.416
101
0.827
97.057
40.554
406
0.802
28.611
11.154
510
JNU-IFM-only
0.884
50.873
21.854
101
0.931
43.841
16.642
406
0.884
17.408
7.051
510
Table 5: Multi-dataset fetal head and pubic symphysis segmentation results. Metrics (Dice, HD, ASSD) are reported for training on individual datasets (HC18, JNU-IFM, PSFHS) versus a combined training set (Multi-centric). Models were evaluated across all test sets to assess cross-dataset generalization. We compare our Mask-HybridGNet Dual (independent contours) against nnUNet. N indicates the number of test images for each evaluation set. N/A indicates that no ground truth data for pubic symphisis is available in this dataset.
Figure 7: Temporal landmark correspondence demonstration across multiple patients. Color-coded landmark visualization showing consistent anatomical correspondences for fetal head (autumn colormap) and pubic symphysis (cool colormap) structures across four representative patients from the JNU-IFM dataset. Each row represents a different patient’s temporal sequence, demonstrating the learned anatomical atlas where corresponding landmark indices represent the same anatomical locations across the population. This capability enables clinical applications including standardized measurement, temporal pattern analysis, and population-based shape studies.
Figure 8: Anatomical atlas and organ grouping strategy for PAX-Ray++ chest radiograph segmentation. The figure displays a representative chest X-ray atlas with colored anatomical structures organized into eight functional categories. Top row shows unified graph representations: (1) Diaphragm Complex including diaphragm, left and right hemidiaphragms; (2) Aortic System comprising aorta, ascending aorta, descending aorta, and aortic arch; (3) Left Lung Zones encompassing left lung and its four anatomical zones; (4) Right Lung Zones including right lung and its four anatomical zones. Bottom row presents independent contour groupings: (5) Clavicles, Scapulae & Sternum; (6) Stomach, Liver & Kidneys; (7) Heart structures including chambers and myocardium; (8) Other soft tissues comprising trachea, breast tissue, esophagus, and inferior vena cava.
Figure 9: Emergent anatomical correspondences learned through mask supervision on PAX-Ray++ dataset. The figure demonstrates that our framework successfully learns anatomically meaningful landmark correspondences without explicit point-level supervision across two representative chest X-ray subjects. Each row shows a different organ group with Subject 1 (left) and Subject 2 (right) displayed side-by-side. Yellow dashed lines connect corresponding landmarks between subjects, revealing that landmark indices represent consistent anatomical locations across the population.
Model
Graph Type
DC ↑
HD ↓
ASSD ↓
Mask-HybridGNet
Independent
0.837 ± 0.142
25.269 ± 20.081
7.391 ± 9.170
Mask-HybridGNet
Unified
0.855 ± 0.134
23.185 ± 19.914
6.617 ± 9.236
Mask-HybridGNet Dual
Independent
0.864 ± 0.133
22.894 ± 19.777
6.299 ± 9.063
Mask-HybridGNet Dual
Unified
0.869 ± 0.132
22.321 ± 20.192
6.148 ± 9.409
Table 6: Performance comparison on PAX-Ray++ dataset showing Dice coefficients (DC), Hausdorff distances (HD), and Average Symmetric Surface Distances (ASSD) for 37-organ chest radiograph segmentation across 737 test images.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Lungs
Heart
Clavicles
Architecture
Graph
Rast.
Shenzhen
Montgomery
Padchest
JSRT
Padchest
JSRT
JSRT
Single Decoder
Independent
No
0.958 ± 0.014
0.964 ± 0.021
0.946 ± 0.020
0.966 ± 0.012
0.935 ± 0.029
0.935 ± 0.032
0.825 ± 0.078
Yes
0.961 ± 0.015
0.968 ± 0.018
0.953 ± 0.016
0.967 ± 0.014
0.925 ± 0.038
0.928 ± 0.038
0.801 ± 0.067
Dual Decoder
Independent
No
0.961 ± 0.014
0.969 ± 0.019
0.950 ± 0.020
0.970 ± 0.011
0.930 ± 0.027
0.936 ± 0.032
0.841 ± 0.062
Yes
0.965 ± 0.014
0.974 ± 0.019
0.956 ± 0.022
0.975 ± 0.011
0.939 ± 0.019
0.941 ± 0.027
0.860 ± 0.077
Single Decoder
Unified
No
0.960 ± 0.014
0.966 ± 0.020
0.948 ± 0.019
0.968 ± 0.012
0.931 ± 0.028
0.936 ± 0.033
0.827 ± 0.059
Appendix
Table 7: Complete ablation study showing the effect of differentiable rasterization on all architectural variants on the Chest-xray-landmark dataset in terms of Dice score.
Mask-HybridGNet
Mask-HybridGNet Dual
Structure
Res.
DC ↑
HD ↓
ASSD ↓
DC ↑
HD ↓
ASSD ↓
LV Myo
2
0.925
17.589
5.819
0.924
17.299
5.871
3
0.926
17.679
5.738
0.931
15.768
5.309
4
0.921
18.403
6.099
0.933
15.911
5.118
LV Endo
2
0.841
23.248
6.182
0.843
21.457
5.968
3
0.836
22.724
6.216
0.849
20.839
5.711
Appendix
Table 8: Performance comparison on CAMUS dataset showing Dice coefficients (DC), Hausdorff distances (HD), and Average Symmetric Surface Distances (ASSD) across different resolution levels.
Mask-HybridGNet
Mask-HybridGNet Dual
Structure
Res.
DC ↑
HD ↓
ASSD ↓
DC ↑
HD ↓
ASSD ↓
LV Endo
2
0.882
4.302
1.690
0.904
3.654
1.396
3
0.893
4.014
1.535
0.903
3.744
1.417
4
0.900
3.664
1.416
0.887
3.887
1.601
LV Myo
2
0.598
5.557
1.589
0.684
4.434
1.295
3
0.655
4.508
1.385
0.673
4.553
1.305
Appendix
Table 9: Performance comparison on Sunnybrook Cardiac Data showing Dice coefficients (DC), Hausdorff distances (HD), and Average Symmetric Surface Distances (ASSD) across different resolution levels.
Figure 10: Temporal cardiac analysis demonstrating landmark correspondence tracking using independent graphs representation. The figure displays comprehensive cardiac segmentation and tracking for a representative test patient with normal systolic function (EF = 54%) across two-chamber (top two rows) and four-chamber (bottom two rows) views. For each view, the first row compares the model prediction against the ground truth at end-diastole and end-systole. The second row visualizes the temporal motion tracking of landmarks for the left ventricular endocardium (LV Endo), epicardium (LV Epi), and left atrium (LA). Unlike the unified approach, this independent representation fails to enforce shared boundaries, resulting in anatomically implausible overlaps and gaps at the interface between the left atrium and left ventricle.
Mask-HybridGNet
Mask-HybridGNet Dual
Structure
Independent
Unified
Independent
Unified
Diaphragm
0.961 ± 0.027
0.967 ± 0.027
0.945 ± 0.029
0.966 ± 0.024
Left Hemidiaphragm
0.947 ± 0.044
0.946 ± 0.044
0.923 ± 0.049
0.943 ± 0.045
Right Hemidiaphragm
0.964 ± 0.023
0.967 ± 0.025
0.950 ± 0.027
0.965 ± 0.023
Aorta
0.886 ± 0.072
0.897 ± 0.067
0.867 ± 0.070
0.883 ± 0.068
Ascending Aorta
0.856 ± 0.090
0.852 ± 0.091
0.831 ± 0.097
0.841 ± 0.093
Appendix
Table 10: Detailed per-organ performance comparison on PAX-Ray++ dataset. The table presents Dice coefficients (mean ± standard deviation) for all 37 anatomical structures across four model configurations. Results demonstrate the consistent benefits of the dual decoder architecture and unified graph representations across diverse anatomical systems.
Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.
Graph-based cardiac segmentation with implicit anatomical correspondences provides topological guarantees and population-level analysis capabilities, but models trained on independent frames of image sequences exhibit temporal discontinuities that affect reliable clinical measurements, particularly in cardiac ultrasound. In this work, we introduce self-supervised temporal regularization as a post-training refinement stage that exploits the temporal coherence in image sequences to enforce consistent cardiac segmentation and motion estimation over time, without requiring per-frame annotations. By penalizing velocity and acceleration discontinuities across consecutive frames, our method achieves temporally consistent segmentations while maintaining the learned anatomical correspondences. We further leverage these correspondences to automatically map landmarks to the AHA 17-segment clinical standard, enabling standardized regional assessment and detection of pathological myocardial motion patterns. Validation on CAMUS dataset demonstrates the clinical utility of combining temporal consistency with automatic regional mapping. The code is publicly available at https://github.com/david-montalvoo/MaskHybridGNet-TempReg
David Montalvo-García, Nicolás Gaggion, María J. Ledesma-Carbayo +1
Biomedical Image Technologies, ETSI Telecomunicación, Universidad Politécnica de Madrid, Madrid, Spain · Centro de Investigación Biomédica en Red de Bioingeniería, Biomateriales y Nanomedicina (CIBER-BBN), Instituto de Salud Carlos III, Madrid, Spain · APOLO Biotech, Ciudad Autónoma de Buenos Aires, Argentina +2
Medical image classification models are ideally expected to identify diagnostically relevant regions while making predictions, yet standard classification losses rarely provide spatial supervision. Explicit supervision via anatomical shape information, such as segmentation masks of task-relevant anatomy, has been shown to guide the network toward regions relevant to the target prediction. However, obtaining such masks incurs substantial manual annotation effort and computational overhead. With the advent of segmentation foundation models that exhibit strong localization of anatomical structures across diverse imaging modalities, we leverage this capability to extract anatomical shape priors without the burden of training a dedicated segmentation model. In this paper, we propose a new framework, Locus, an anatomical attention regularization framework that leverages pretrained segmentation foundation models to guide a classifier's attention toward diagnostically meaningful anatomical structures across diverse imaging modalities. Instead of enforcing pixel-wise alignment with the foundation-model-derived mask, we introduce a regularization term that adaptively balances attention between anatomical (foreground) and background regions, penalizing the classifier when background attention dominates. We validate Locus on eight diverse medical imaging datasets spanning dermoscopy, X-ray, histopathology, and cardiac MRI, showing consistent gains in classification performance alongside improved anatomically grounded attention.
Department of Computer Science, University of Virginia, Virginia, USA · Ahsanullah University of Science and Technology, Dhaka, Bangladesh · Kahlert School of Computing, University of Utah, Utah, USA +1
Vision Transformers (ViTs) excel in 3D medical segmentation but require massive annotated datasets. While Self-Supervised Learning (SSL) mitigates this using unlabeled data, it still faces strict privacy and logistical barriers. Formula-Driven Supervised Learning (FDSL) offers a privacy-preserving alternative by pre-training on synthetic mathematical primitives. However, a critical semantic gap limits its efficacy: generic shapes lack the morphological fidelity, fixed spatial layouts, and inter-organ relationships of real anatomy, preventing models from learning essential global structural priors. To bridge this gap, we propose an Anatomy-Informed Synthetic Supervised Pre-training framework unifying FDSL's infinite scalability with anatomical realism. We replace basic primitives with a lightweight shape bank with de-identified, label-only segmentation masks from 5 subjects. Furthermore, we introduce a structure-aware sequential placement strategy to govern the patch synthesis process. Instead of random placement, we enforce physiological plausibility using spatial anchors for correct localization and a topological graph to manage inter-organ interactions (e.g., preventing impossible overlaps). Extensive experiments on BTCV and MSD datasets demonstrate that our method significantly outperforms state-of-the-art FDSL baselines and SSL methods by 1.74% and up to 1.66%, while exhibiting a robust scaling effect where performance improves with increased synthetic data volume. This provides a data-efficient, privacy-compliant solution for medical segmentation. The code will be made publicly available upon acceptance.
Jiaqi Tang, Mengyan Zheng, Shu Zhang +2
Peking University · Beihang University · Deepwise Ltd.