Bayesian Optimization for Design Parameters of 3D Image Data Analysis
Authors: David Exler, Joaquin Eduardo Urrutia Gómez, Martin Krüger, Maike Schliephake, John Jbeily, Mario Vitacolonna, Rüdiger Rudolf, Markus Reischl
Organizations: Institute for Automation and Applied Informatics, Karlsruhe Institute of Technology, Hermann-von-Helmholtz-Platz 1, 76344 Eggenstein-Leopoldshafen, Germany. · Institute of Biological and Chemical Systems, Karlsruhe Institute of Technology, Hermann-von-Helmholtz-Platz 1, 76344 Eggenstein-Leopoldshafen, Germany. · CeMOS Research and Transfer Center, Technische Hochschule Mannheim, Paul-Wittsack-Straße 10, 68163 Mannheim, Germany.
Deep learning-based segmentation and classification are crucial to large-scale biomedical imaging, particularly for 3D data, where manual analysis is impractical. Although many methods exist, selecting suitable models and tuning parameters remains a major bottleneck in practice. Hence, we introduce the 3D data Analysis Optimization Pipeline, a method designed to facilitate the design and parameterization of segmentation and classification using two Bayesian Optimization stages. First, the pipeline selects a segmentation model and optimizes postprocessing parameters using a domain-adapted syntactic benchmark dataset. To ensure a concise evaluation of segmentation performance, we introduce a segmentation quality metric that serves as the objective function. Second, the pipeline optimizes design choices of a classifier, such as encoder and classifier head architectures, incorporation of prior knowledge, and pretraining strategies. To reduce manual annotation effort, this stage includes an assisted class-annotation workflow that extracts predicted instances from the segmentation results and sequentially presents them to the operator, eliminating the need for manual tracking. In four case studies, the 3D data Analysis Optimization Pipeline efficiently identifies effective model and parameter configurations for individual datasets.
Figures & tables
Fig. 1: Progression of the introduced pipeline. 3D data (1) is used for data synthesis (2), combining instance simulation and domain adaptation. The synthetic data serves as input for segmentation optimization (3), where a pretrained model and postprocessing parameters are optimized with respect to the ipq metric. The optimized segmentation model (4) predicts instances, which are annotated by an operator (5). Classifier optimization (6) selects encoder and classifier head architectures, preprocessing, and pretraining strategies based on validation accuracy. The optimized classifier (7) predicts class labels, enabling the final analysis (8), which yields statistics such as class counts and instance volume distributions.
Fig. 2: Examples of idealized error types quantified by the ipq metric. rq detects hallucinations, iq captures instance splitting, and sq reflects oversegmentation.
Fig. 3: 3D depiction of the synthetic segmentation benchmark. a) Original reference, b) synthetic image, and c) synthetic image after domain adaptation via cycleGAN.
Name
Abbrev.
Acc@5
Parameters (M)
ResNet18 [ 16 ]
RN18
89.08%
11.7
ResNet101 [ 16 ]
RN101
93.55%
44.5
Swin V2 [ 22 ]
Swin
96.87%
87.9
EfficientNet V2 [ 38 ]
EN
97.79%
118.5
ConvNeXt-L V1 [ 23 ]
CN
96.98%
197.8
CellposeSAM [ 30 ]
CSAM
-
305
TABLE I: Comparison of the six encoders included.
model
abbreviation
parameters
StarDist 2D Versitile Fluo [ 41 , 40 , 34 ]
SD
1.4B
PlantSeg 3D Nuc Platinum [ 44 ]
PSP
1B
PlantSeg 3D Confocal UNet [ 42 ]
PSG
4B
CellposeSAM [ 30 ]
CSAM
304B
TABLE II: Segmentation models included in all experiments.
Parameter
Values
Description
Morphology
θED
[−10,10]
Radius of the spherical structuring element for erosion (negative) or dilation (positive).
θCO
[−5,5]
Radius of the spherical structuring element for an opening (negative) or closing (positive) operation.
θMc
[0,1]
Weight of contour continuity at intersections (from 1D profiles along the connecting axis) for the adjacency graph.
Merging
θMs
[0,1]
Weight of connecting-surface smoothness for the adjacency graph.
θMr
[0,1]
Weight of relative intersecting area (normalized by smaller-instance surface area) for the adjacency graph.
Splitting
θSσ
[0,1]
Gaussian smoothing strength for watershed.
TABLE III: Parameters included in the optimization of postprocessing methods.
Fig. 4: ipq results of every experiment. Full bars present the baseline, thick-striped bars the random search, and thin-striped bars the model optimized by the pipe .
Fig. 5: 2D projection of a subset of the parameter space with resulting ipq values. The dots denote exploration steps, and the cross denotes the found optimum. Red numbers next to dots indicate that a specific parameter value combination was evaluated multiple times.
Fig. 6: Results of the classification optimization processes per encoder, classifier head, pretraining method, and preprocessing method with four highlights: 1) On average worst combination performs best, 2) second to worst combination performs best, 3) slice classifier outperforms volume classifier , and 4) best pretraining strategy is outperformed.
Name
Duration one image (s)
ResNet18
0.42
ResNet101
0.47
Swin V2
0.61
EfficientNet V2
0.59
ConvNeXt-L V1
0.73
CellposeSAM
2.23
TABLE IV: Comparison of inference durations per encoder.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Fig. 7: Architectures of the classifier heads. The slice classifier performs a self-attention mechanism over the encoders’ representations of the 2D slices of a volume. The volume classifier applies 3D convolutions on the stacked 2D representations.
Morphological Operators
Merging Parameters
Splitting Parameters
Result
Experiment
Model
θED
θCO
θMc
θMr
θMs
θSσ
θSt
ipq
C3DH-H157
CSAM
-4
4
0.073
0.584
0.194
0.915
0.000
0.699
SD
5
3
0.472
0.125
0.977
0.317
0.587
0.244
pSG
1
2
0.862
0.321
0.882
0.695
0.823
0.237
pSP
10
2
0.306
0.479
0.254
0.744
0.496
0.488
C3DL-MDA231
CSAM
1
2
1.000
1.000
0.000
0.746
0.000
0.637
Appendix
TABLE V: ipq results of every segmentation model with its optimal postprocessing parameterization per experiment.
Medical image segmentation models typically rely on large amounts of densely annotated volumetric data, limiting their scalability across tasks and imaging modalities. This work addresses the challenge of predicting entire 3D anatomical structures from extreme annotation sparsity. An annotation-efficient student-teacher framework is proposed for automatic 3D medical segmentation that requires only a set of point prompts on a single 2D slice per volume, as input. A foundation model serves as an offline teacher, utilizing the provided point prompts from the selected slice to full-volume pseudo-annotations alongside their corresponding spatial confidence scores prior to student training. To mitigate the error propagation of noisy pseudo-annotations, a task-specific 3D student network is trained using a confidence-aware optimization strategy. By leveraging the teacher's pre-computed confidence scores, this strategy explicitly excludes statically uncertain regions of the pseudo-annotations from the loss calculation, while simultaneously emphasizing regions with higher confidence. Evaluated on 3D cardiac MRI datasets, our framework outperforms state-of-the-art semi-supervised methods, improving segmentation performance by up to 43.6%. Furthermore, it drastically reduces the manual annotation burden to just a few positive point prompts per volume, while improving surface boundary precision by up to 14.7% over the teacher and successfully recovering up to 34.1% of the performance gap toward the fully supervised upper bound.
Georgios Triantafyllou, Dimitris K. Iakovidis
Department of Computer Science and Biomedical Informatics, University of Thessaly, 2-4 Papasiopoulou st. 35131, Lamia, Greece
Accurate and efficient 3D medical image segmentation is essential for clinical AI, where models must remain reliable under stringent memory, latency, and data availability constraints. Transformer-based methods achieve strong accuracy but suffer from excessive parameters, high FLOPs, and limited generalization. We propose LightMedSeg, a modular UNet-style segmentation architecture that integrates anatomical priors with adaptive context modeling. Anchor-conditioned FiLM modulation enables anatomy-aware feature calibration, while a local structural prior module and texture-aware routing dynamically allocate representational capacity to boundary-rich regions. Computational redundancy is minimized through ghost and depthwise convolutions, and multi-scale features are adaptively fused via a learned skip router with anchor-relative spatial position bias. Despite requiring only 0.48M parameters and 14.64~GFLOPs, LightMedSeg achieves segmentation accuracy within a few Dice points of heavy transformer baselines. Therefore, LightMedSeg is a deployable and data-efficient solution for 3D medical image segmentation. Code will be released publicly upon acceptance.
Kavyansh Tyagi, Vishwas Rathi, Puneet Goyal
National Institute of Technology Kurukshetra, India · Indian Institute of Technology Ropar, India
Accurate and computationally efficient 3D medical image segmentation remains a critical challenge in clinical workflows. Transformer-based architectures often demonstrate superior global contextual modeling but at the expense of excessive parameter counts and memory demands, restricting their clinical deployment. We propose RefineFormer3D, a lightweight hierarchical transformer architecture that balances segmentation accuracy and computational efficiency for volumetric medical imaging. The architecture integrates three key components: (i) GhostConv3D-based patch embedding for efficient feature extraction with minimal redundancy, (ii) MixFFN3D module with low-rank projections and depthwise convolutions for parameter-efficient feature extraction, and (iii) a cross-attention fusion decoder enabling adaptive multi-scale skip connection integration. RefineFormer3D contains only 2.94M parameters, substantially fewer than contemporary transformer-based methods. Extensive experiments on ACDC and BraTS benchmarks demonstrate that RefineFormer3D achieves 93.44% and 85.9% average Dice scores respectively, outperforming or matching state-of-the-art methods while requiring significantly fewer parameters. Furthermore, the model achieves fast inference (8.35 ms per volume on GPU) with low memory requirements, supporting deployment in resource-constrained clinical environments. These results establish RefineFormer3D as an effective and scalable solution for practical 3D medical image segmentation.
Kavyansh Tyagi, Vishwas Rathi, Puneet Goyal
National Institute of Technology Kurukshetra Department of Electronics and Communication Engineering · National Institute of Technology Kurukshetra Department of Computer Engineering · Indian Institute of Technology Ropar Department of Computer Science and Engineering