Benchmarking and Improvement of Artifact Segmentation Methods in H&E-Stained Whole Slide Imaging
Authors: Julia Merta, Daniel Mendelsohn, Benedikt Schoser, Joanna Polanska, Michal Marczyk
Organizations: Department of Data Science and Engineering, Silesian University of Technology, Akademicka 16, Gliwice, 44-100, Poland · Friedrich-Baur-Institute, Department of Neurology, LMU Medizin, Munich, 80336, Germany · Yale Cancer Center, Yale University, New Haven, 06511, CT, USA
Technical artifacts introduced during tissue preparation or slide scanning remain a major obstacle to developing robust computational pathology algorithms for hematoxylin and eosin (HE)-stained whole-slide images (WSIs). Despite the growing number of quality-control tools, their real-world utility remains hard to assess because of the limited number of publicly available implementations and the lack of systematic, multi-dataset comparisons. This paper presents a benchmark of artifact segmentation methods for HE-stained WSIs. After a literature review, six publicly available algorithms were selected and compared in terms of their ability to segment artifact-free tissue and selected artifact classes. Two publicly available datasets and a newly introduced dataset released to support further research were used for evaluation. Tissue samples from multiple medical centers and different organs are included, enabling evaluation across heterogeneous data sources. Segmentation performance was assessed using Dice Index, pixel-level recall, and false discovery rate for artifact-free tissue segmentation. For algorithms supporting multiclass artifact segmentation, class-wise Dice Index and inter-class misclassification were evaluated. In addition to segmentation quality, processing time was assessed. Statistical analysis revealed significant differences in segmentation performance between methods. HistoKit, which incorporates a modified version of the GrandQC, was identified as the best-performing algorithm, while HistoQC was the worst. In terms of artifact types, dark spots and out of focus regions were harder to segment than others. Finally, a simple post-processing step was introduced for HistoKit, which further improved segmentation performance. The benchmark establishes a reference for evaluating and developing artifact segmentation methods for HE-stained WSIs.
Figures & tables
Figure 1: Overview of the study design and algorithm comparison workflow. Six publicly available algorithms: GrandQC [ 51 ] , HistoKit [ 32 ] , HistoQC [ 23 ] , HistoROI [ 34 ] , Patil et al. [35] , and Schreiber et al. [39] , were evaluated based on segmentation quality, artifact classification, and processing time. Three datasets comprising multiple artifact classes were used in this study. The annotated classes included dark spots and foreign objects (DS), air bubbles and other edge-like artifacts (EDGE), tissue folds (FOLD), out-of-focus regions (OOF), and pen-marker traces of different colors (PEN), as well as artifact-free tissue (TIS) and background (BG). For Dataset A, background and artifact regions were considered jointly. Additionally, the parameters of the boundary discordance reduction for the best-performing method were examined. Moreover, a post-processing method for the artifact mask for the top-performing algorithm to further enhance segmentation quality.
Dataset A
Dataset B
Dataset C
Number of Annotated Regions
13
567
23
Source
TCGA, CAMELYON16
Medical Center in Austria
Compass NMD, TCGA
Formats
SVS / TIF
PNG
SVS
Organs
Breast, Head and Neck, Nervous system, Prostate, Small Intestine
Breast, Colon, Kidney, Prostate
Brain, Head and Neck, Heart, Kidney, Muscle, Uterus
Preservation Method
FFPE, Unknown
FFPE
FFPE, Fresh Frozen Tissue, Unknown
Annotated Classes
Artifact-Free Tissue, Background and Artifacts
Air Bubbles and Edges, Artifact-free Tissue, Dark spots and Foreign Objects, Out-of-Focus Regions, Tissue Folds
Air Bubbles and Edges, Artifact-Free Tissue, Background, Dark Spots and Foreign Objects, Out-of-Focus Regions, Pen, Tissue Folds
Table 2: Summary of datasets used to benchmark artifact segmentation algorithms.
Figure 2: Overview of the datasets used in the study. (a) Percentage of images containing each selected artifact type across the datasets. In Dataset A, the background class includes pixels corresponding to both the image background and other artifact types, such as markers, tissue folds, and foreign objects. (b) Plot showing the size of the annotated tissue regions in pixels for each dataset. The y-axis is shown on a logarithmic scale. (c) Representative images from each dataset and their corresponding ground-truth annotations.
Figure 3: Exemplary segmentation results for each method across all datasets. The dataset name (A, B, or C) is indicated in the leftmost panel. For each example, the original tissue image and the corresponding ground-truth mask are shown, with colors representing the annotated artifact classes. The segmentation results obtained with each method are visualized using classification outcome maps: true positive (TP) pixels are shown in green, true negative (TN) pixels in black, false positive (FP) pixels in red, and false negative (FN) pixels in blue.
Figure 4: Violin plots showing the distributions of the Dice index, sensitivity, and FDR across color-coded methods and datasets.
Figure 5: Tissue regions from Datasets B and C with the corresponding ground-truth multiclass artifact annotation masks and segmentation results for selected images from each dataset. Colors indicate different artifact classes: background (BG), artifact-free tissue (TIS), dark spots and foreign objects (DS), edges and air bubbles (EDGE), tissue folds (FOLD), pen markings (PEN), and out-of-focus regions (OOF).
Figure 6: Violin plots showing the distributions of Dice Index values between methods across datasets, stratified by individual artifact classes. In Dataset B, the Background and Pen classes are unavailable. Since the method developed by Patil et al. does not support detecting artifacts in the Edge and Dark Spot classes, it is not shown in the corresponding plots.
Figure 7: Heatmaps showing the percentage of pixels classified into each artifact class for datasets B and C. Although the Pen class is not present in dataset B, some algorithms incorrectly classify pixels as belonging to this class. Therefore, we include it in the heatmap. Gray indicates that a given artifact class is absent from the ground-truth image.
Method
Segmentation Stage
Mean (CI) [min]
GrandQC
Tissue vs Background
0.01 (0.00, 0.01)
Artifact
0.77 (0.75, 0.79)
All
0.78 (0.76, 0.80)
HistoKit
Tissue vs Background
4.16 (4.15, 4.16)
Artifact
Overlap 0%
0.41 (0.39, 0.43)
Overlap 25%
0.51 (0.49, 0.53)
Table 3: Comparison of segmentation times for the selected algorithms. For GrandQC and HistoKit, the processing times of individual stages are also reported. For HistoKit, the relationship between processing time and patch overlap is additionally presented. The mean processing time was calculated from 10 segmentation runs performed on an image from dataset C.
Figure 8: Selected images before and after post-processing HistoKit segmentation masks. From left to right: tissue region, ground-truth artifact segmentation, HistoKit segmentation without post-processing, and HistoKit segmentation after the proposed post-processing.
Hematoxylin and eosin (H&E) staining is the cornerstone of histopathology, yet scalable, quantitative analysis of H&E whole-slide images (WSIs) remains a central challenge in computational pathology. We present Atlas H&E-TME, an AI-based system built on the Atlas family of pathology foundation models that predicts tissue quality, tissue region, and cell type labels across multiple cancer types, yielding over 4,500 quantitative readouts per slide at cell-level resolution. A key challenge to validating such systems is overcoming morphological ambiguity inherent to H&E-only ground truth and the limited scalability of more informed references drawing on modalities such as immunohistochemistry (IHC). We address this with a dual validation framework combining biologically grounded depth with technical and morphological breadth. For depth, we propose an IHC-informed multi-pathologist consensus protocol that substantially improves inter-rater agreement over conventional H&E-only annotation. This yields a molecularly grounded reference against which we compare Atlas H&E-TME and pathologists working from H&E alone. For breadth, we benchmark Atlas H&E-TME on over 200,000 high-confidence H&E-only pathologist annotations across 1,500+ cases spanning eight cancer types and their most common metastatic sites, with subtypes covering >90% of clinical cases per cancer type, drawn from 25+ sources and 8+ scanner models. Benchmarked against the IHC-informed consensus, Atlas H&E-TME matches or exceeds pathologist H&E-only performance and generalizes consistently and robustly across this broad morphological and technical scope. In doing so, Atlas H&E-TME turns the H&E slide -- the most ubiquitous data in pathology -- into a scalable, quantitative window into the tumor and its microenvironment, laying a foundation for the next generation of tissue-based biomarkers in translational and clinical research.
Kai Standvoss, Miriam Hägele, Rosemarie Krupar +25
Aignostics, Germany · Institute of Pathology, Charité – Universitätsmedizin Berlin, Germany · Berlin Institute of Health, Charité – Universitätsmedizin Berlin, Germany +9
Background and Objective: Quality control is a prerequisite for whole-slide image analysis, yet the benchmarks on which quality-control methods are compared share four properties that make their reported differences hard to interpret: few independent slides, annotation concentrated in a minority of them, pooled ratio metrics with no closed-form standard error, and a single inherited train/test partition. We propose a reliability protocol for such benchmarks. Methods: The protocol quantifies four sources of variability - test-set sampling, training stochasticity, partition composition, and undocumented preprocessing - a claim is reportable only if it survives all four; three of the four cost minutes of compute. We apply it to an independent reconstruction of a published diffusion-based artifact detector, evaluated on the original 24-slide partition and against a supervised baseline. Results: The method's central mechanism reproduces: the auxiliary contrastive term improves pooled F1 from 0.673 to 0.688 and replicates under a second seed (+0.0156, p = 0.031; +0.0190, p = 0.005), although it acts on pen marking rather than the artifact types cited to motivate it. Its comparative claims do not: differences between design variants, and against the supervised baseline, fall inside the uncertainty of the evaluation. Four of 24 slides carry 70% of scored annotated pixels, giving an effective sample size of 6.2, and the inherited partition sits at the 7th percentile. An unreported tissue-restriction step excludes 41.4% of out-of-focus annotation against 2.6% of air bubble; such a gate is confounded with blur by construction. Conclusions: Small-cohort benchmarks support far weaker conclusions than current reporting implies. The four checks are cheap enough to accompany any evaluation on such a resource and separate reproducible effects from differences the evaluation cannot resolve.
Konstantinos Moutselos, Ilias Maglogiannis
Department of Digital Systems, University of Piraeus, Piraeus, Greece
Foreground segmentation is the critical first step of every computational pathology pipeline, yet existing methods rely on hand-tuned heuristics or supervised models that overfit to narrow stain and scanner distributions, failing silently on specialised stains such as Jones silver or Elastica van Gieson. We propose a coarse-to-fine approach that recasts foreground segmentation as a visual perception task and leverages general-purpose vision-language models (VLMs) as zero-annotation oracles. Our key insight is that tissue-versus-background discrimination is a natural-image recognition problem, not a histopathological one, so VLMs trained on internet-scale corpora generalise where domain-specific models cannot. We introduce Leica-75, a benchmark of 75 renal transplant whole-slide images spanning three stain families. On Leica-75, our method achieves the highest segmentation quality on out-of-distribution stains (Dice 0.858 +/- 0.027 on Jones, 0.853 +/- 0.041 on EVG) with 7x lower cross-stain variance than the best supervised baseline, while remaining competitive on in-distribution H&E. Few-shot prompting with automatically curated exemplars (Auto-context) rescues hard cases on Stress-32 (n=32), a curated stress-test subset (Dice 0.470 to 0.819 for the 2B model). VLM-based annotation review matches human expert consensus (kappa=0.989 for blur detection; mean precision/recall grading accuracy 0.708 vs. human 0.646 for segmentation mask review). The resulting pseudo-labels are used to distil lightweight student models that are as performant as the teacher model while running for a fraction of the cost. Our framework provides a principled, scalable solution to a persistent infrastructure bottleneck in digital pathology.
Vishal Jain, Giorgio Buzzanca, Sarah Cechnicka +6
Imperial College London, London, UK · Leiden University Medical Center, Leiden, The Netherlands · KU Leuven, Leuven, Belgium +3