cs.CVOct 7, 2026

Performance at What Cost? A Sustainability-Aware Performance Index for Cell and Nucleus Instance Segmentation

Authors: Eiram Mahera Sheikh, Alaa Tharwat, Wolfram Schenck

Organizations: Department of Engineering and Mathematics, Bielefeld University of Applied Sciences and Arts, Interaktion 1, 33619 Bielefeld

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Pretrained models for cell and nuclear instance segmentation differ substantially in architecture, pretraining data and objectives, parameter count, inference strategy, adaptation requirements, postprocessing pipeline, and computational demand. Large pretrained and foundation models are increasingly adopted because of their strong zero-shot capabilities, but their use also imposes greater energy consumption, memory requirements, computational demands, adaptation costs, and operational carbon emissions. Whether these additional demands are justified by meaningful gains in segmentation performance remains unclear. We address this question by introducing the Sustainability-Aware Performance Index (SAPI), a configurable metric that combines segmentation performance, energy consumption, and model size. We benchmark 19 pretrained and foundation models across six CellBinDB datasets under zero-shot inference and evaluate 16 fine-tunable models using few-shot adaptation with both frozen encoder and full-model fine-tuning. We estimate energy consumption for GPU, CPU, and RAM using software-based monitoring tools. Our results show that larger and more computationally demanding models do not consistently achieve proportionate improvements in segmentation quality. While few-shot adaptation benefits several models, the gains and resource costs vary considerably across architectures, datasets, and adaptation strategies, causing SAPI-based rankings to differ from rankings based on performance alone. This study provides a practical framework for comparing segmentation models more comprehensively and supports more computationally accessible and environmentally responsible model selection in biomedical image analysis.

Figures & tables

DatasetImagesCell instancesTissuesImage size
mIF606,0133256×256256\times 256
10xGenomics_HE1007,0873256×256256\times 256
10xGenomics_DAPI1007,7452256×256256\times 256
DAPI20316,6576256×256256\times 256
ssDNA27623,86720256×256256\times 256
HE30541,1116512×512512\times 512
Table 1 : Details of the six CellBinDB datasets.
Figure 1 : Sample images and their corresponding instance segmentation masks from the six CellBinDB datasets. Each column shows one dataset. The top row shows the microscopy image and the bottom row shows the reference instance masks.
Figure 2 : Model size and dataset compatibility of the 19 evaluated models. Bar height represents the total number of parameters ( NθN_{\theta} ) on a logarithmic scale, with exact counts reported in millions above each bar. Hatch patterns indicate whether a model was used with the fluorescence datasets, the H&E datasets, or all six CellBinDB datasets.
Figure 3 : Zero-shot segmentation performance across compatible model–dataset combinations. Each cell reports the mean performance score PP , calculated as the harmonic mean of AJI + , PQ, and NSD. Grey cells denote model–dataset combinations that were not evaluated because the model is not intended to be used with that dataset modality.
Figure 4 : Mean zero-shot inference energy per test image, separated into GPU, CPU, and RAM contributions. Energy values were averaged across experimental seeds and the datasets compatible with each model. The total height of each stacked bar represents the total measured IT energy per image.
Figure 5 : Relationship between zero-shot segmentation performance, inference energy consumption, and model size. Each point represents one model. Performance and energy were averaged across ten runs and then across the datasets compatible with that model. The horizontal axis shows total inference energy per image on a logarithmic scale, and marker area represents the total number of model parameters.
Figure 6 : Zero-shot sustainability-aware performance across models and datasets.
Figure 7 : Segmentation performance PP on the mIF dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of PP across ten runs for each compatible model with error bars indicating standard deviation.
Figure 8 : Segmentation performance PP on the 10xGenomics_DAPI dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of PP across ten runs for each compatible model with error bars indicating standard deviation.
Figure 9 : Segmentation performance PP on the DAPI dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of PP across ten runs for each compatible model with error bars indicating standard deviation.
Figure 10 : Segmentation performance PP on the ssDNA dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of PP across ten runs for each compatible model with error bars indicating standard deviation.
Figure 11 : Segmentation performance PP on the 10xGenomics_HE dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of PP across ten runs for each compatible model with error bars indicating standard deviation.
Figure 12 : Segmentation performance PP on the HE dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of PP across ten runs for each compatible model with error bars indicating standard deviation.
Figure 13 : Mean fine-tuning energy per support image under full-model and frozen-encoder adaptation. The total bar height represents the combined GPU, CPU, and RAM energy consumption averaged across compatible datasets.
Figure 14 : Sustainability-aware performance after few-shot adaptation. The results combine segmentation performance with fine-tuning and post-fine-tuning inference energy and model size. Scores are averaged across ten independent runs.
Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Model nameSoftware nameSoftware versionCheckpoint name
StarDist (Fluo)stardist0.9.22D_versatile_fluo
StarDist (HE)stardist0.9.22D_versatile_he
InstanSeginstanseg-torch0.1.2single_channel_nuclei
Cellpose cyto2cellpose3.1.1cyto2torch_0
Cellpose cyto3cellpose3.1.1cyto3
Cellpose-SAMcellpose4.0.8cpsam
Appendix
Table S1 : Models, pretrained checkpoints, and software versions used in the benchmark.
Figure S1 : Zero-shot AJI+ scores across compatible model–dataset combinations. Grey cells denote model–dataset combinations that were not evaluated because the model is not intended to be used with that image modality.
Figure S2 : Zero-shot Panoptic Quality (PQ) scores across compatible model–dataset combinations. Grey cells denote model–dataset combinations that were not evaluated because the model is not intended to be used with that image modality.
Figure S3 : Zero-shot Normalized Surface Dice (NSD) scores across compatible model–dataset combinations. Grey cells denote model–dataset combinations that were not evaluated because the model is not intended to be used with that image modality.
Figure S4 : Qualitative zero-shot comparison for one image sampled from the mIF dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S5 : Qualitative zero-shot comparison for one image sampled from the DAPI dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S6 : Qualitative zero-shot comparison for one image sampled from the ssDNA dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S7 : Qualitative zero-shot comparison for one image sampled from the 10xGenomics_DAPI dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S8 : Qualitative zero-shot comparison for one image sampled from the 10xGenomics_HE dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S9 : Qualitative zero-shot comparison for one image sampled from the HE dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S10 : AJI + results for few-shot adaptation on the mIF dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S11 : AJI + results for few-shot adaptation on the 10xGenomics_DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S12 : AJI + results for few-shot adaptation on the DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S13 : AJI + results for few-shot adaptation on the ssDNA dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S14 : AJI + results for few-shot adaptation on the 10xGenomics_HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S15 : AJI + results for few-shot adaptation on the HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S16 : PQ results for few-shot adaptation on the mIF dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S17 : PQ results for few-shot adaptation on the 10xGenomics_DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S18 : PQ results for few-shot adaptation on the DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S19 : PQ results for few-shot adaptation on the ssDNA dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S20 : PQ results for few-shot adaptation on the 10xGenomics_HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S21 : PQ results for few-shot adaptation on the HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S22 : NSD results for few-shot adaptation on the mIF dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S23 : NSD results for few-shot adaptation on the 10xGenomics_DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S24 : NSD results for few-shot adaptation on the DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S25 : NSD results for few-shot adaptation on the ssDNA dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S26 : NSD results for few-shot adaptation on the 10xGenomics_HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S27 : NSD results for few-shot adaptation on the HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.

Explore similar work

CardsList
Jun 8, 2026cs.CV

Maximum Matching Accuracy: An Instance Segmentation Evaluation Metric Utilizing Globally Optimal Matching

Reliable evaluation of instance segmentation models requires metrics that accurately and consistently reflect segmentation quality. However, the metrics most widely used in biological imaging carry fundamental mathematical weaknesses: hard Intersection-over-Union (IoU) thresholds that produce discontinuous, low sensitivity scoring; per-object normalization that distorts scores under object size variation; and greedy or one-to-many matching procedures that yield non-optimal, order-dependent correspondences. Together, these properties produce unintuitive and unreliable model rankings under common failure modes such as split cells, merged cells, and cell boundary imprecision. We propose Maximum Matching Accuracy (MMA), a threshold-free continuous metric that finds a globally optimal one-to-one matching between predicted and ground truth objects and aggregates total overlap using per-pixel normalization. We evaluate MMA against AP@50, PQ, SEG, and AJI across three experiments: synthetic failure cases, progressive corruption tests, and a model ranking comparison. MMA produces scores that are more stable, more sensitive, and more interpretable than existing alternatives, providing a principled foundation for fair instance segmentation benchmarking in biological cell imaging.

Kaden Stillwagon, Alexandra D. VandeLoo, Craig R. Forest

College of Computing, Georgia Institute of Technology, Atlanta, 30332, Georgia, United States · School of Materials Science and Engineering, Georgia Institute of Technology, Atlanta, 30332, Georgia, United States · Wallace H. Coulter Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, 30332, Georgia, United States +1

Multiple Instance LearningMetric Spaces
AskPDF
Jun 9, 2026cs.CV

Don't waste SAM

Meta AI has recently released the Segment Anything Model (SAM), which demonstrates exceptional zero-shot image segmentation performance across various tasks with remarkable accuracy. Despite its inability to provide accurate segmentation across multiple research fields, SAM still serves as a valuable starting point for supporting the segmentation pipeline process, particularly for tasks that require extensive and senior skills annotations. This study aims to evaluate the generalization of SAM and fine-tuning SAM models using three waste segmentation datasets. Although they are captured from real scenes as SAM was pretrained on, these datasets present several challenges, including occlusions, deformable objects, transparency, and objects easily confused with backgrounds. In our findings, the fine-tuned SAM-ViT-H model outperforms the state-ofthe-art Zerowaste, and TACO datasets with a significant increase of +30 in IoU, and it closely approaches performance levels of TrashCan 1.0, with only a -1.44 difference. After evaluating these popular waste datasets, it became evident that fine-tuning SAM as a foundational model is a crucial step for providing better generalization for downstream waste segmentation tasks. Therefore, SAM should not be disregarded or wasted.

Nermeen Abou Baker, Uwe Handmann

1- Ruhr West University of Applied Sciences - Dept of Computer Science Lutzowstrassse 5, 46236 Bottrop - Germany

Segment Anything ModelFew-Shot SegmentationWasteArtificial Intelligence Systems
AskPDF
May 28, 2026cs.CV

One Click per Cell Type Suffices: Training-free Group Interaction for Cell Instance Segmentation

Cell instance segmentation models trained on cell-specific datasets suffer severe performance drops on out-of-distribution cell types, while interactive foundation models overcome this through per-instance prompting at a cost that is prohibitively expensive for histopathology images containing hundreds to thousands of densely packed instances. We introduce \textbf{Group Prompting}, a new paradigm that shifts interactive segmentation from per-instance O(N)O(N) to per-type O(T)O(T), where a single click per cell type suffices to segment all instances of that type. Our key observation is that the frozen image encoder of the Segment Anything Model (SAM) already clusters same-type cells in its feature space before any prompt is given, and that this clustering holds across staining modalities without any training. Exploiting this property, we propose \textbf{Chain-of-Prompts (CoP)}, a training-free framework that recursively expands a single user click by (1) identifying reliable same-type locations through non-parametric gating of multi-scale encoder features, and (2) selecting the most spatially distant reliable point as the next prompt to maximize coverage. On eleven benchmarks, CoP generalizes to both unseen cell types and unseen imaging modalities without any adaptation: with one click per type it retains over 90% of per-instance performance on three cell-type-annotated datasets while surpassing fully-supervised methods, and with one click per image it retains over 95% on eight datasets spanning both H&E and non-H&E imaging. Project Page: https://shjo-april.github.io/Chain-of-Prompts/

Sanghyun Jo, Seo Jin Lee, Seohyung Hong +4

1OGQ, Korea · 2Seoul National University, Korea · 3LG CNS, Korea

CellsMultiple Instance LearningMultiple Prompt Learning FrameworksDigital PathologyPrompt Engineering
AskPDF