A 211K Parameter Foundation Model: Genetic Prompting in Neural Cellular Automata

John Kalkhof; Niklas Ihm; Daniel Kirste; Anirban Mukhopadhyay

cs.CVOct 28, 2026

A 211K Parameter Foundation Model: Genetic Prompting in Neural Cellular Automata

Authors: John Kalkhof, Niklas Ihm, Daniel Kirste, Anirban Mukhopadhyay

Organizations: Darmstadt University of Technology, Karolinenplatz 5, 64289 Darmstadt, Germany

Abstract

The scale of modern foundation models enables broad task generalization. In medical imaging, where segmentation demands vary widely across clinical contexts, such flexibility is invaluable. But current models, such as SAM trade adaptability for scale, locking their utility behind high-end infrastructure and leaving behind the low-resource environments most in need of diagnostic support. To bridge this gap, we introduce Foundational-NCA, a lightweight foundation model capable of adapting to a wide range of medical segmentation tasks without retraining. Built on Neural Cellular Automata (NCA), Foundational-NCA incorporates gradient-based learning with genetic optimization over a latent task space, using a compact task vector to guide predictions. This enables dynamic adaptation via a genetic prompt mechanism, allowing the model to generalize across tasks while keeping the parameter count magnitudes lower at just 211 thousand, compared to SAM-based models with 38.9 to 636 million parameters. We evaluate Foundational-NCA on 15 trained and 6 entirely unseen segmentation tasks from the TotalSegmentator dataset, achieving an average Dice score of 84.5% on known tasks and 60.4% on unseen ones. These results demonstrate that task generalization is possible in ultra-compact models, laying the groundwork for resource-efficient, deployable foundation models in clinical environments where infrastructure is limited. By reducing model size by over 180 times, Foundational-NCA takes a step toward a more equitable future for medical AI, one where powerful tools are within reach of every healthcare setting.

Figures & tables

Figure 1: (A) A genetic tree visualizes how Foundational-NCA organizes and separates task prompts for different segmentation targets. (B) These task prompts, combined with positional encoding and the input image, guide the model’s dynamic prediction process. (C) Unlike SAM variants, which require millions of parameters, Foundational-NCA achieves strong generalization with just 211K parameters.

Figure 2: (A) A single input image is paired with a task-specific latent vector, enabling the model to produce different segmentation outputs for the same input. (B) The task vector serves as a multiplicative conditioning signal within the NCA architecture, modulating the update dynamics. (C) Training alternates between two phases: model updates are performed via gradient descent, while task vector optimization is carried out using genetic optimization.

t_{j}^{(g+1)}=\alpha t_{a}^{(g)}+(1-\alpha)t_{b}^{(g)}+\epsilon,\quad\epsilon\sim[-1,1]

Algorithm 1 Genetic Task Population Optimization (Single Step)

Task Set A	Foundational-NCA	SAM	SAM 2	Med-SAM
Seen	211K param.	93.7M param.	38.9M param.	93.7M param.
Gluteus Max. L	0.885 $\pm$ 0.077	0.054 $\pm$ 0.014	0.073 $\pm$ 0.019	0.814 $\pm$ 0.057
Gluteus Max. R	0.874 $\pm$ 0.073	0.076 $\pm$ 0.021	0.075 $\pm$ 0.020	0.796 $\pm$ 0.064
Liver	0.911 $\pm$ 0.105	0.343 $\pm$ 0.076	0.369 $\pm$ 0.092	0.891 $\pm$ 0.068
Lung Lower Lobe R	0.832 $\pm$ 0.162	0.638 $\pm$ 0.163	0.552 $\pm$ 0.174	0.809 $\pm$ 0.116
Lung Middle Lobe R	0.808 $\pm$ 0.114	0.463 $\pm$ 0.125	0.312 $\pm$ 0.109	0.842 $\pm$ 0.080

Table 1: Dice accuracy comparison of Foundational-NCA against SAM, SAM2 and MedSAM on Task Set A (tasks seen during training) and Task Set B (unseen during training).

Figure 3: Exemplary segmentation results comparing Foundational-NCA with Med-SAM.

Task Set A	Foundational-NCA	No Location	No Task
Average	0.845	0.753	0.848
Task Set B	Foundational-NCA	No Location	No Task
Autochthon L	0.678 $\pm$ 0.159	0.002 $\pm$ 0.012	0.661 $\pm$ 0.148
Iliopsoas L	0.568 $\pm$ 0.174	0.020 $\pm$ 0.041	0.593 $\pm$ 0.141
Lung Lower Lobe L	0.730 $\pm$ 0.153	0.299 $\pm$ 0.168	0.746 $\pm$ 0.158
Sternum	0.661 $\pm$ 0.088	0.018 $\pm$ 0.038	0.645 $\pm$ 0.079

Table 2: Ablation study of Foundational-NCA on unseen tasks, evaluating the impact of removing the task vector and the positional prompt. Results for seen tasks are provided in the supplementary.

Figure 4: The four globes illustrate how automatically optimized task vectors align with the tasks they are intended to solve. Each globe presents the task sphere from a different viewing angle to highlight the spatial distribution of task representations.

Figure 5: Interpolation between task vectors on the spherical task space. In both liver-to-kidney and kidney-to-lung transitions, the model gradually morphs the predicted segmentation shape to match the evolving task prompt while adapting it to the fixed anatomical position.

Figure 6: Grid of predictions generated by applying a fixed input image to a 6×6 grid of task vectors sampled uniformly from the spherical task space. Each output is colored according to the closest training class, revealing how different regions of the task space influence the segmentation behavior.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: The mean task vector history shows early separation between tasks, which becomes increasingly refined with additional genetic optimization steps.

Figure 9: Loss trajectory of Foundational-NCA. Clear improvements are observed following each genetic optimization step, especially during early training phases.

Figure 10: Genetic optimization trajectories for each label show consistent convergence toward an optimum, with some fluctuations remaining for more challenging segmentation targets.

Task Set A	Foundational-NCA	No Location	No Task	Single Stage
Gluteus Max. L	0.885 $\pm$ 0.077	0.867 $\pm$ 0.114	0.879 $\pm$ 0.085	0.879 $\pm$ 0.090
Gluteus Max. R	0.874 $\pm$ 0.073	0.807 $\pm$ 0.131	0.876 $\pm$ 0.089	0.877 $\pm$ 0.098
Liver	0.911 $\pm$ 0.105	0.874 $\pm$ 0.170	0.892 $\pm$ 0.116	0.904 $\pm$ 0.122
Lung Lower Lobe R	0.832 $\pm$ 0.162	0.763 $\pm$ 0.188	0.886 $\pm$ 0.139	0.813 $\pm$ 0.173
Lung Middle Lobe R	0.808 $\pm$ 0.114	0.736 $\pm$ 0.143	0.837 $\pm$ 0.084	0.825 $\pm$ 0.106
Lung Upper Lobe L	0.870 $\pm$ 0.170	0.786 $\pm$ 0.205	0.887 $\pm$ 0.156	0.841 $\pm$ 0.186

Appendix

Table 3: Full Table of Ablation study of Foundational-NCA on unseen tasks, evaluating the impact of removing the task vector and the positional prompt

Figure 11: Task space visualization for the single-stage setup shows that introducing location and task information simultaneously leads to a disorganized and unstructured task representation.

Figure 12: UMAP projection comparing single- and two-stage training. The two-stage setup leads to a more organized and interpretable task space, confirming earlier observations.

Task Set A	SAM-H	SAM2.1-L
Parameters	641.1M	224.4M
Gluteus Max. L	0.076 $\pm$ 0.020	0.074 $\pm$ 0.021
Gluteus Max. R	0.074 $\pm$ 0.019	0.077 $\pm$ 0.022
Liver	0.367 $\pm$ 0.089	0.443 $\pm$ 0.124
Lung Lower Lobe R	0.583 $\pm$ 0.176	0.532 $\pm$ 0.171
Lung Middle Lobe R	0.349 $\pm$ 0.122	0.305 $\pm$ 0.106

Appendix

Table 4: Additional Baseline results of the high parameter variants of SAM and SAM2.

Figure 13: Additional task vector interpolations further confirm that the model adapts the segmentation to fit the task prompt as well as possible within the spatial constraints given by the positional prompt.

Figure 14: Evenly sampled task prompts from the unit sphere, applied to two input images with corresponding positional prompts. Each output is labeled according to the closest mean class in the task space.

Figure 15: Qualitative results for all 15 training tasks (Task Set A) and 6 unseen tasks (Task Set B). Green: ground truth, Red: prediction, Yellow: overlap.

Medical Imaging Foundation Models

Medical imaging foundation models learn reusable representations for tasks across clinical images, modalities, and institutions.

See all 235 papers

SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentation
Feb 22, 2026Yujie Lu, Jingwen Li, Sibo Ju +5Medical Image Segmentation Vision Foundation Model Adaptation
Sichuan University · Xinjiang University · Fuzhou University +1
Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM
Apr 25, 2026Jingxuan Kang, Ziqi Zhang, Shaoming Zheng +9Medical Imaging Foundation Models Promptable Image Segmentation
Imperial College London · Beihang University · National Health Service +1
Deep Reprogramming Distillation for Medical Foundation Models
May 6, 2026Siyuan Du, Yuhang Zhou, Haolin Li +5Medical Imaging Foundation Models Vision Foundation Model Adaptation
School of Computer Science, Fudan University and with the Shanghai AI Laboratory, Shanghai, China · Shanghai AI Laboratory, Shanghai, China · Cooperative Medianet Innovation Center, Shanghai Jiao Tong University and with the Shanghai AI Laboratory, Shanghai, China +3

Feb 22, 2026J/K move · Enter open · S save

SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentation

Yujie Lu, Jingwen Li, Sibo Ju, Yanzhou Su+4

Sichuan University · Xinjiang University · Fuzhou University+1

Medical Image Segmentation Vision Foundation Model Adaptation Promptable Image Segmentation Mixture of Experts

Open paper AskPDF arXiv

Medical image segmentation is vital for clinical diagnosis and quantitative analysis, yet remains challenging due to the heterogeneity of imaging modalities and the high cost of pixel-level annotations. Although general interactive segmentation models like SAM have achieved remarkable progress, their transfer to medical imaging still faces two key bottlenecks: (i) the lack of adaptive mechanisms for modality- and anatomy-specific tasks, which limits generalization in out-of-distribution medical scenarios; and (ii) current medical adaptation methods fine-tune on large, heterogeneous datasets without selection, leading to noisy supervision, higher cost, and negative transfer. To address these issues, we propose SegMoTE, an efficient and adaptive framework for medical image segmentation. SegMoTE preserves SAM's original prompt interface, efficient inference, and zero-shot generalization while introducing only a small number of learnable parameters to dynamically adapt across modalities and tasks. In addition, we design a progressive prompt tokenization mechanism that enables fully automatic segmentation, significantly reducing annotation dependence. Trained on MedSeg-HQ, a curated dataset less than 1% of existing large-scale datasets, SegMoTE achieves SOTA performance across diverse imaging modalities and anatomical tasks. It represents the first efficient, robust, and scalable adaptation of general segmentation models to the medical domain under extremely low annotation cost, advancing the practical deployment of foundation vision models in clinical applications.

Abstract

Medical Imaging Foundation Models

Explore similar work

SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentation

Learning from Noisy Prompts: Saliency-Guided Prompt Distillation for Robust Segmentation with SAM

Deep Reprogramming Distillation for Medical Foundation Models