cs.CVOct 7, 2026
SavePerformance at What Cost? A Sustainability-Aware Performance Index for Cell and Nucleus Instance Segmentation
Organizations: Department of Engineering and Mathematics, Bielefeld University of Applied Sciences and Arts, Interaktion 1, 33619 Bielefeld
Abstract
Pretrained models for cell and nuclear instance segmentation differ substantially in architecture, pretraining data and objectives, parameter count, inference strategy, adaptation requirements, postprocessing pipeline, and computational demand. Large pretrained and foundation models are increasingly adopted because of their strong zero-shot capabilities, but their use also imposes greater energy consumption, memory requirements, computational demands, adaptation costs, and operational carbon emissions. Whether these additional demands are justified by meaningful gains in segmentation performance remains unclear. We address this question by introducing the Sustainability-Aware Performance Index (SAPI), a configurable metric that combines segmentation performance, energy consumption, and model size. We benchmark 19 pretrained and foundation models across six CellBinDB datasets under zero-shot inference and evaluate 16 fine-tunable models using few-shot adaptation with both frozen encoder and full-model fine-tuning. We estimate energy consumption for GPU, CPU, and RAM using software-based monitoring tools. Our results show that larger and more computationally demanding models do not consistently achieve proportionate improvements in segmentation quality. While few-shot adaptation benefits several models, the gains and resource costs vary considerably across architectures, datasets, and adaptation strategies, causing SAPI-based rankings to differ from rankings based on performance alone. This study provides a practical framework for comparing segmentation models more comprehensively and supports more computationally accessible and environmentally responsible model selection in biomedical image analysis.
Figures & tables
| Dataset | Images | Cell instances | Tissues | Image size |
|---|---|---|---|---|
| mIF | 60 | 6,013 | 3 | |
| 10xGenomics_HE | 100 | 7,087 | 3 | |
| 10xGenomics_DAPI | 100 | 7,745 | 2 | |
| DAPI | 203 | 16,657 | 6 | |
| ssDNA | 276 | 23,867 | 20 | |
| HE | 305 | 41,111 | 6 |
Table 1 : Details of the six CellBinDB datasets.
Figure 1 : Sample images and their corresponding instance segmentation masks from the six CellBinDB datasets. Each column shows one dataset. The top row shows the microscopy image and the bottom row shows the reference instance masks.
Figure 2 : Model size and dataset compatibility of the 19 evaluated models. Bar height represents the total number of parameters ( ) on a logarithmic scale, with exact counts reported in millions above each bar. Hatch patterns indicate whether a model was used with the fluorescence datasets, the H&E datasets, or all six CellBinDB datasets.
Figure 3 : Zero-shot segmentation performance across compatible model–dataset combinations. Each cell reports the mean performance score , calculated as the harmonic mean of AJI + , PQ, and NSD. Grey cells denote model–dataset combinations that were not evaluated because the model is not intended to be used with that dataset modality.
Figure 4 : Mean zero-shot inference energy per test image, separated into GPU, CPU, and RAM contributions. Energy values were averaged across experimental seeds and the datasets compatible with each model. The total height of each stacked bar represents the total measured IT energy per image.
Figure 5 : Relationship between zero-shot segmentation performance, inference energy consumption, and model size. Each point represents one model. Performance and energy were averaged across ten runs and then across the datasets compatible with that model. The horizontal axis shows total inference energy per image on a logarithmic scale, and marker area represents the total number of model parameters.
Figure 6 : Zero-shot sustainability-aware performance across models and datasets.
Figure 7 : Segmentation performance on the mIF dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of across ten runs for each compatible model with error bars indicating standard deviation.
Figure 8 : Segmentation performance on the 10xGenomics_DAPI dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of across ten runs for each compatible model with error bars indicating standard deviation.
Figure 9 : Segmentation performance on the DAPI dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of across ten runs for each compatible model with error bars indicating standard deviation.
Figure 10 : Segmentation performance on the ssDNA dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of across ten runs for each compatible model with error bars indicating standard deviation.
Figure 11 : Segmentation performance on the 10xGenomics_HE dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of across ten runs for each compatible model with error bars indicating standard deviation.
Figure 12 : Segmentation performance on the HE dataset under zero-shot inference, full-model fine-tuning and frozen-encoder fine-tuning. The plot reports the mean value of across ten runs for each compatible model with error bars indicating standard deviation.
Figure 13 : Mean fine-tuning energy per support image under full-model and frozen-encoder adaptation. The total bar height represents the combined GPU, CPU, and RAM energy consumption averaged across compatible datasets.
Figure 14 : Sustainability-aware performance after few-shot adaptation. The results combine segmentation performance with fine-tuning and post-fine-tuning inference energy and model size. Scores are averaged across ten independent runs.
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
| Model name | Software name | Software version | Checkpoint name |
|---|---|---|---|
| StarDist (Fluo) | stardist | 0.9.2 | 2D_versatile_fluo |
| StarDist (HE) | stardist | 0.9.2 | 2D_versatile_he |
| InstanSeg | instanseg-torch | 0.1.2 | single_channel_nuclei |
| Cellpose cyto2 | cellpose | 3.1.1 | cyto2torch_0 |
| Cellpose cyto3 | cellpose | 3.1.1 | cyto3 |
| Cellpose-SAM | cellpose | 4.0.8 | cpsam |
Table S1 : Models, pretrained checkpoints, and software versions used in the benchmark.
Figure S1 : Zero-shot AJI+ scores across compatible model–dataset combinations. Grey cells denote model–dataset combinations that were not evaluated because the model is not intended to be used with that image modality.
Figure S2 : Zero-shot Panoptic Quality (PQ) scores across compatible model–dataset combinations. Grey cells denote model–dataset combinations that were not evaluated because the model is not intended to be used with that image modality.
Figure S3 : Zero-shot Normalized Surface Dice (NSD) scores across compatible model–dataset combinations. Grey cells denote model–dataset combinations that were not evaluated because the model is not intended to be used with that image modality.
Figure S4 : Qualitative zero-shot comparison for one image sampled from the mIF dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S5 : Qualitative zero-shot comparison for one image sampled from the DAPI dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S6 : Qualitative zero-shot comparison for one image sampled from the ssDNA dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S7 : Qualitative zero-shot comparison for one image sampled from the 10xGenomics_DAPI dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S8 : Qualitative zero-shot comparison for one image sampled from the 10xGenomics_HE dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S9 : Qualitative zero-shot comparison for one image sampled from the HE dataset using random seed 0. The figure shows the input image, the ground-truth instance mask, the ground-truth overlay, and predictions from all models compatible with the image modality. Each prediction is overlaid on the input image.
Figure S10 : AJI + results for few-shot adaptation on the mIF dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S11 : AJI + results for few-shot adaptation on the 10xGenomics_DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S12 : AJI + results for few-shot adaptation on the DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S13 : AJI + results for few-shot adaptation on the ssDNA dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S14 : AJI + results for few-shot adaptation on the 10xGenomics_HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S15 : AJI + results for few-shot adaptation on the HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S16 : PQ results for few-shot adaptation on the mIF dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S17 : PQ results for few-shot adaptation on the 10xGenomics_DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S18 : PQ results for few-shot adaptation on the DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S19 : PQ results for few-shot adaptation on the ssDNA dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S20 : PQ results for few-shot adaptation on the 10xGenomics_HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S21 : PQ results for few-shot adaptation on the HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S22 : NSD results for few-shot adaptation on the mIF dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S23 : NSD results for few-shot adaptation on the 10xGenomics_DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S24 : NSD results for few-shot adaptation on the DAPI dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S25 : NSD results for few-shot adaptation on the ssDNA dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S26 : NSD results for few-shot adaptation on the 10xGenomics_HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.
Figure S27 : NSD results for few-shot adaptation on the HE dataset. The figure compares compatible, fine-tunable models under zero-shot inference, frozen-encoder and full-model fine-tuning. Each value reports the mean across ten runs.