Towards High-Quality Image Segmentation: Improving Topology Accuracy by Penalizing Neighbor Pixels

Juan Miguel Valverde; Dim P. Papadopoulos; Rasmus Larsen; Anders Bjorholm Dahl

cs.CVMar 19, 2026

Towards High-Quality Image Segmentation: Improving Topology Accuracy by Penalizing Neighbor Pixels

Authors: Juan Miguel Valverde, Dim P. Papadopoulos, Rasmus Larsen, Anders Bjorholm Dahl

Organizations: Technical University of Denmark

Abstract

Standard deep learning models for image segmentation cannot guarantee topology accuracy, failing to preserve the correct number of connected components or structures. This, in turn, affects the quality of the segmentations and compromises the reliability of the subsequent quantification analyses. Previous works have proposed to enhance topology accuracy with specialized frameworks, architectures, and loss functions. However, these methods are often cumbersome to integrate into existing training pipelines, they are computationally very expensive, or they are restricted to structures with tubular morphology. We present SCNP, an efficient method that improves topology accuracy by penalizing the logits with their poorest-classified neighbor, forcing the model to improve the prediction at the pixels' neighbors before allowing it to improve the pixels themselves. We show the effectiveness of SCNP across 13 datasets, covering different structure morphologies and image modalities, and integrate it into three frameworks for semantic and instance segmentation. Additionally, we show that SCNP can be integrated into several loss functions, making them improve topology accuracy. Our code can be found at https://jmlipman.github.io/SCNP-SameClassNeighborPenalization.

Figures & tables

Figure 1: SCNP improves topology accuracy. Arrows indicate breakage in the structures.

Figure 2: Logits (left) and logits after applying SCNP (right). SCNP propagates small values across the foreground-class logits, and large values across the background-class logits.

Figure 3: Logits after applying SCNP and softmax normalization.

Figure 4:

\beta_{0e}

obtained by Cross Entropy Dice loss after optimizing the standard logits (CEDice), the SCNP-penalized logits (

\overline{CEDice}

), both (CEDice+

\overline{CEDice}

), and the top two loss functions across the 13 datasets. Datasets are grouped into four categories (① ② ③ ④) based on their image type (medical, non-medical) and structures’ morphologies (tubular, non-tubular, rounded). First panel: Number of datasets in which each loss function led the most accurate segmentations topologically.

$\downarrow w$	$2048^{2}$ ( 9.69)	$1024^{2}$ ( 7.23)	$512^{2}$ ( 4.70)
3	6.94 $\pm$ 1.47	7.12 $\pm$ 1.80	8.57 $\pm$ 1.98
5	5.64 $\pm$ 1.34	5.96 $\pm$ 1.51	7.79 $\pm$ 1.63
7	4.87 $\pm$ 1.20	5.45 $\pm$ 1.61	7.78 $\pm$ 1.90
9	4.79 $\pm$ 1.31	7.14 $\pm$ 2.86	7.89 $\pm$ 2.34
11	7.80 $\pm$ 1.87	11.02 $\pm$ 2.55	18.65 $\pm$ 3.72

Table 1: Sensitivity analysis on SCNP’s hyperparameter (

w

) across three versions of FIVES. Top row: pixel size 2 , : median thickness of the blood vessels.

Loss	Dice	$\beta_{0e}$	clDice
CE	0.92 $\pm$ 0.0	11.93 $\pm$ 2.92	0.92 $\pm$ 0.0
$\overline{CE}$ ( ours )	0.92 $\pm$ 0.0	7.53 $\pm$ 1.74	0.92 $\pm$ 0.0
Dice	0.92 $\pm$ 0.0	12.03 $\pm$ 3.4	0.92 $\pm$ 0.0
$\overline{Dice}$ ( ours )	0.92 $\pm$ 0.0	7.88 $\pm$ 2.03	0.92 $\pm$ 0.0
CEDice	0.92 $\pm$ 0.0	10.82 $\pm$ 2.71	0.92 $\pm$ 0.0
CEDice+ $\overline{CEDice}$ ( ours )	0.92 $\pm$ 0.0	6.94 $\pm$ 1.47	0.92 $\pm$ 0.0

Table 2: Ablation comparing several loss functions optimizing the standard logits vs. the SCNP-penalized (ours) logits.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Loss	Epoch time (sec.)	GPU (MiB)
CEDice	54.85	5350
clDice ( $i=24$ )	181.78	19340
SkelRecall	115.59	5334
TopoLoss	1505	5334
RWLoss	56.59	5778
Tversky ( $\beta=0.7$ )	54.76	5326

Appendix

Table B.1 : Resources required on FIVES (2D nnUNetv2).

Loss	Iteration time (sec.)	GPU (MiB)
CEDice	72.79	6644
clDice ( $i=16$ )	199.10	38672
SkelRecall	157.17	7784
TopoLoss	12960	6632
RWLoss	79.42	6864
Tversky ( $\beta=0.6$ )	73.02	6644

Appendix

Table B.2 : Resources required on PulmonaryVA (3D nnUNetv2).

Figure D.1 : Left: Convergence plot when optimizing Cross entropy over the standard logits, the SCNP logits, and both. Right: Same scenario with Dice loss. The small plots inside the two panels are a zoomed in on the first 20 epochs.

Dataset	CEDice	•(CEDice)	SCNP ( $\overline{CEDice}$ )
FIVES	10.82 $\pm$ 2.71	8.79 $\pm$ 2.47	6.94 $\pm$ 1.47
DeepGlobeRoads	11.77 $\pm$ 3.17	9.92 $\pm$ 2.83	6.56 $\pm$ 2.25
CirrMRI600	0.27 $\pm$ 0.26	0.19 $\pm$ 0.13	0.12 $\pm$ 0.05
NuInsSeg	8.18 $\pm$ 2.39	8.17 $\pm$ 2.40	7.53 $\pm$ 2.20

Appendix

Table E.3 : Average

\beta_{0e}

in four datasets when optimizing CEDice (second column) and after applying binary closing (third column), and when optimizing CEDice with the SCNP logits (fourth column). •: Binary closing.

Figure F.2 : Visual comparison between clCE and $\overline{CEDice}$ . Red circle: topological errors.

FIVES	• Blood vessels in fundus retina images • Semantic segmentation • R,G,B • 2048 $\times$ 2048 pixels	Splits: 480-120-200 (train, validation, test)
PulmonaryVA	• Pulmonary arteries and veins in chest CT scans • Semantic segmentation • CT • 184 $\times$ 235 $\times$ 262.5 voxels (median)	Dataset size: 106 (five-fold cross validation)
Axons	• Axons in mouse brain • Semantic segmentation • Electron-microscopy • 1062.5 $\times$ 1067 $\times$ 285 voxels (median)	Dataset size: 10 (five-fold cross validation) Sham_HM_25_contra—ipsi, Sham_HM_49_contra—ipsi, TBI_HM_24_contra—ipsi, TBI_HM_28_contra—ipsi, TBI_HM_2_contra—ipsi

Appendix

Table G.4 : Medical datasets with tubular structures (nnUNetv2 framework, Group ①)

TopoMortar	• Mortar in red brick wall images • Semantic segmentation • R,G,B • 512 $\times$ 512 pixels	Splits: 50-20-350 (train, validation, test)
Crack500	• Cracks in concrete images • Semantic segmentation • R,G,B • 640 $\times$ 896 Pixels (median)	Splits: 250-50-200 (train, validation, test)
DeepGlobeRoads	• Roads in satellite images • Semantic segmentation • R,G,B • 1024 $\times$ 1024 pixels (median)	Splits: 4357-623-1246 (train, validation, test)

Appendix

Table G.5 : Non-medical datasets with tubular structures (Detectron2+DeepLabv3+ framework, Group ②)

CirrMRI600	• Cirrhotic liver in MRI • Semantic segmentation • T2w • 320 $\times$ 260 $\times$ 80 voxels (median)	Splits: 198-50-31 (train, validation, test)
ATLAS2	• Lesion in brain MRI • Semantic segmentation • T1w • 197 $\times$ 233 $\times$ 189 voxels (median)	Dataset size: 655 (five-fold cross validation)
ISLES24	• Ischemic lesion in brain MRI • Semantic segmentation • ADC, DWI • 512 $\times$ 581 $\times$ 69 voxels (median)	Dataset size: 149 (five-fold cross validation)
MSLesSeg	• Multiple sclerosis lesion in brain MRI • Semantic segmentation • T1w, T2w, FLAIR • 182 $\times$ 218 $\times$ 182 voxels (median)	Splits: 74-19-22 (train, validation, test)

Appendix

Table G.6 : Medical datasets with non-tubular structures (nnUNetv2 framework, Group ③)

NuInsSeg	• Cell nuclei in H&E-stained histological images • Instance segmentation • R,G,B • 512 $\times$ 512 pixels (median)	Splits: 532-66-67 (train, validation, test)
IHC_TMA	• Cell nuclei in immunohistochemical tissue microarray images • Instance segmentation • R,G,B • 256 $\times$ 256 pixels (median)	Splits: 212-27-27 (train, validation, test)
LyNSeC	• Lymphoma nuclei in H&E-stained histological images • Instance segmentation • R,G,B • 512 $\times$ 512 pixels (median)	Splits: 559-70-70 (train, validation, test)

Appendix

Table G.7 : Medical (cells) datasets with rounded structures (InstanSeg framework, Group ④)

Dataset	Eps. w/ TopoLoss	% of train time
FIVES	38 (1000)	44.20%
PulmonaryVA	3 (1000)	34.30%
Axons	5 (1000)	51.88%
TopoMortar	3600 (12000)	87.86%
Crack500	3600 (12000)	62.65%
DeepGlobeRoads	3600 (12000)	41.50%

Appendix

Table H.8 : Second column: Epochs optimizing TopoLoss and total epochs. Third column: Percentage of training time optimizing TopoLoss relative to the total training time (average across the five runs).

	Loss	Dice	$\beta_{0e}$	clDice
FIVES	CEDice	0.92 $\pm$ 0.0	10.82 $\pm$ 2.71	0.92 $\pm$ 0.0
	clDice ( $i=24$ )	0.91 $\pm$ 0.0	36.55 $\pm$ 10.70	0.91 $\pm$ 0.01
	SkelRecall	0.91 $\pm$ 0.0	12.45 $\pm$ 4.25	0.91 $\pm$ 0.0
	TopoLoss	0.92 $\pm$ 0.0	8.09 $\pm$ 2.45	0.92 $\pm$ 0.0
	RWLoss	0.92 $\pm$ 0.0	15.99 $\pm$ 5.35	0.92 $\pm$ 0.0
	Tversky ( $\beta=0.7$ )	0.92 $\pm$ 0.0	12.22 $\pm$ 3.45	0.92 $\pm$ 0.0

Appendix

Table I.9 : Group ① Medical datasets with tubular structures (nnUNetv2).

\overline{CEDice}

: CEDice applied to SCNP. Bold (Dice): Highest and significantly significant. Bold (

\beta_{0e}

, clDice): Best scores with Dice equal or better and statistically significant.

	Loss	Dice	$\beta_{0e}$	clDice
TopoMortar	CEDice	0.67 $\pm$ 0.03	25.06 $\pm$ 9.26	0.75 $\pm$ 0.03
	clDice ( $i=4$ )	0.68 $\pm$ 0.02	21.40 $\pm$ 5.86	0.75 $\pm$ 0.02
	SkelRecall	0.68 $\pm$ 0.02	23.64 $\pm$ 7.30	0.76 $\pm$ 0.02
	TopoLoss	0.67 $\pm$ 0.02	27.22 $\pm$ 7.64	0.74 $\pm$ 0.02
	RWLoss	0.54 $\pm$ 0.30	19.40 $\pm$ 13.80	0.76 $\pm$ 0.03
	Tversky ( $\beta=0.8$ )	0.68 $\pm$ 0.02	22.59 $\pm$ 6.46	0.75 $\pm$ 0.02

Appendix

Table I.10 : Group ② Non-medical datasets with tubular structures (Detectron2+DeepLabv3+).

	Loss	Dice	$\beta_{0e}$
CirrMRI600	CEDice	0.97 $\pm$ 0.01	0.27 $\pm$ 0.26
	TopoLoss	0.97 $\pm$ 0.01	0.32 $\pm$ 0.28
	RWLoss	0.96 $\pm$ 0.01	0.43 $\pm$ 0.31
	Tversky ( $\beta=0.7$ )	0.95 $\pm$ 0.0	1.39 $\pm$ 1.49
	Focal	0.97 $\pm$ 0.01	0.51 $\pm$ 0.30
	CEDice+ $\overline{CEDice}$	0.97 $\pm$ 0.01	0.17 $\pm$ 0.09

Appendix

Table I.11 : Group ③ Medical datasets with non-tubular structures (nnUNetv2).

	Loss	Dice	$\beta_{0e}$	$(\mathcal{R}_{e})$
NuInsSeg	CEDice	0.76 $\pm$ 0.04	8.18 $\pm$ 2.39	0.91 $\pm$ 0.01
	TopoLoss	0.28 $\pm$ 0.01	13.35 $\pm$ 9.23	0.76 $\pm$ 0.04
	RWLoss	0.72 $\pm$ 0.04	8.42 $\pm$ 3.04	0.90 $\pm$ 0.01
	Tversky ( $\beta=0.6$ )	0.77 $\pm$ 0.03	7.94 $\pm$ 2.91	0.90 $\pm$ 0.01
	Focal	0.71 $\pm$ 0.03	9.25 $\pm$ 2.05	0.89 $\pm$ 0.01
	CEDice+ $\overline{CEDice}$	0.76 $\pm$ 0.04	7.95 $\pm$ 2.21	0.91 $\pm$ 0.01

Appendix

Table I.12 : Group ④ Medical datasets with rounded structures.

Figure J.3 : Results on medical datasets with tubular structures (Group ①).

Figure J.4 : Results on non-medical datasets with tubular structures (Group ②).

Figure J.5 : Results on medical datasets with non-tubular structures (Group ③).

Figure J.6 : Results on medical datasets with rounded structures (Group ④).

$\downarrow w$	9.69	7.23	4.70
3	0.92 $\pm$ 0.0	0.92 $\pm$ 0.0	0.92 $\pm$ 0.0
5	0.92 $\pm$ 0.0	0.92 $\pm$ 0.0	0.92 $\pm$ 0.0
7	0.92 $\pm$ 0.0	0.91 $\pm$ 0.0	0.91 $\pm$ 0.0
9	0.9 $\pm$ 0.0	0.89 $\pm$ 0.0	0.9 $\pm$ 0.0
11	0.92 $\pm$ 0.0	0.91 $\pm$ 0.0	0.92 $\pm$ 0.0

Appendix

Table K.13 : Dice coefficients of the sensitivity analysis on SCNP’s hyper-parameter

w

$\downarrow w$	9.69	7.23	4.70
3	0.92 $\pm$ 0.0	0.92 $\pm$ 0.0	0.92 $\pm$ 0.0
5	0.91 $\pm$ 0.0	0.91 $\pm$ 0.0	0.92 $\pm$ 0.0
7	0.9 $\pm$ 0.0	0.89 $\pm$ 0.0	0.9 $\pm$ 0.0
9	0.88 $\pm$ 0.0	0.87 $\pm$ 0.0	0.88 $\pm$ 0.0
11	0.91 $\pm$ 0.0	0.91 $\pm$ 0.0	0.92 $\pm$ 0.0

Appendix

Table K.14 : clDice metric of the sensitivity analysis on SCNP’s hyper-parameter

w

Topology-Preserving Image Segmentation

Topology-preserving image segmentation models image regions while maintaining structural properties such as connectivity, branches, and holes.

See all 43 papers

Disconnect to Connect: A Data Augmentation Method for Improving Topology Accuracy in Image Segmentation
Mar 7, 2025Juan Miguel Valverde, Maja Østergaard, Adrian Rodriguez-Palomo +4Topology-Preserving Image Segmentation Thin-Structure Segmentation
DTU Compute, Technical University of Denmark, Denmark · A.I. Virtanen Institute, University of Eastern Finland, Finland · Department of Chemistry and iNANO, Aarhus University, Denmark
Continuous-tone Simple Points: An $\ell_0$ -Norm of Cyclic Gradient for Topology-Preserving Data-Driven Image Segmentation
Apr 30, 2026Wenxiao Li, Faqiang Wang, Yuping Duan +3Image Segmentation Topology-Preserving Image Segmentation
Laboratory of Mathematics and Complex Systems (Ministry of Education), School of Mathematical Sciences, Beijing Normal University, Beijing, 100875, People’s Republic of China · State Key Laboratory of Remote Sensing Science, Faculty of Geographical Science, Beijing Normal University, Beijing, 100875, People’s Republic of China
Deep Image Segmentation via Discriminant Feature Learning
May 14, 2026Adam Dawid Sztamborski, Raül Pérez-Gonzalo, Antonio AgudoImage Segmentation Representation Learning
Institut de Robòtica i Informàtica Industrial, CSIC-UPC, Barcelona, Spain · Politechnika Łódzka, Łód´z, Poland

Mar 7, 2025J/K move · Enter open · S save

Disconnect to Connect: A Data Augmentation Method for Improving Topology Accuracy in Image Segmentation

Juan Miguel Valverde, Maja Østergaard, Adrian Rodriguez-Palomo, Peter Alling Strange Vibe+3

DTU Compute, Technical University of Denmark, Denmark · A.I. Virtanen Institute, University of Eastern Finland, Finland · Department of Chemistry and iNANO, Aarhus University, Denmark

Topology-Preserving Image Segmentation Thin-Structure Segmentation Data Augmentation

Open paper AskPDF arXiv

Accurate segmentation of thin, tubular structures (e.g., blood vessels) is challenging for deep neural networks. These networks classify individual pixels, and even minor misclassifications can break the thin connections within these structures. Existing methods that focus on improving topology accuracy, such as topology loss functions, require very precise training labels with the correct topology, which is difficult to obtain. Low image resolution and contrast can further complicate the annotation by causing these structures to appear disconnected. We present CoLeTra, a data augmentation method that improves topology accuracy, even on datasets with topologically inaccurate training labels. CoLeTra achieves this by artificially breaking the structures in the images while maintaining the original labels, incorporating to the models the prior knowledge that structures can be continuous even if they appear disconnected. Our extensive experiments, involving different architectures, loss functions, and datasets, demonstrate that CoLeTra leads to segmentations topologically more accurate while often improving the Dice coefficient and clDice. We also release a dataset specifically suited for image segmentation methods with a focus on topology accuracy. CoLetra's code can be found at https://jmlipman.github.io/CoLeTra.

Abstract

Topology-Preserving Image Segmentation

Explore similar work

Disconnect to Connect: A Data Augmentation Method for Improving Topology Accuracy in Image Segmentation

Continuous-tone Simple Points: An ℓ0\ell_0ℓ0​-Norm of Cyclic Gradient for Topology-Preserving Data-Driven Image Segmentation

Deep Image Segmentation via Discriminant Feature Learning

Continuous-tone Simple Points: An $\ell_0$ -Norm of Cyclic Gradient for Topology-Preserving Data-Driven Image Segmentation