Segment Anything for Dendrites from Electron Microscopy
Authors: Zewen Zhuo, Ilya Belevich, Ville Leinonen, Eija Jokitalo, Tarja Malm, Alejandra Sierra, Jussi Tohka
Organizations: A.I. Virtanen Institute for Molecular Sciences University of Eastern Finland Kuopio, Finland · Electron Microscopy Unit Institute of Biotechnology University of Helsinki Helsinki, Finland · Department of Medicine Faculty of Health Sciences University of Eastern Finland Kuopio, Finland
Segmentation of cellular structures in electron microscopy (EM) images is fundamental to analyzing the morphology of neurons and glial cells in the healthy and diseased brain tissue. Current neuronal segmentation applications are based on convolutional neural networks (CNNs) and do not effectively capture global relationships within images. Here, we present DendriteSAM, a vision foundation model based on Segment Anything, for interactive and automatic segmentation of dendrites in EM images. The model is trained on high-resolution EM data from healthy rat hippocampus and is tested on diseased rat and human data. Our evaluation results demonstrate better mask quality compared to the original and other fine-tuned models, leveraging the features learned during training. This study introduces the first implementation of vision foundation models in dendrite segmentation, paving the path for computer-assisted diagnosis of neuronal anomalies.
Figures & tables
Dataset
Pixel Size
Width × Height
Slices
Employed
A
15 × 15 nm2
3100 × 3100
1044
Training & Evaluation
B
15 × 15 nm2
3100 × 3100
698
Evaluation
C
10 × 10 nm2
4096 × 4096
697
Evaluation
TABLE I: Overview of Data Utilized
Fig. 1: Concavity Distribution: The x-axis and y-axis represent the value of concavity and the corresponding percentage of masks in sampled images, respectively. The green star and red arrow in the subplot indicate the dendrite and spine structure, respectively.
Fig. 2: Model Architecture
Fig. 3: Impact of Image Tiling vs Resizing: On the left, segmentation quality with the model trained with the tiling method (cropping with a sliding window size 1024 and step size 512), and on the right with the model trained with the resizing method (resizing the whole image to 1024 × 1024). In the legend, the digits after ”p” and ”n” stand for the number of foreground points and background points used, respectively.
Fig. 4: Quantitative Evaluation: The blue and black legends present ViT-B and ViT-L models from SAM. ViT-B-EM-organelles [ 32 ] and ViT-L-EM-organelles [ 33 ] stand for the models from Micro_SAM trained on EM organelles images, and ViT-B-resize-EM-dendrite and ViT-L-resize-EM-dendrite are specialist models that we trained.
Scale
Quality
1
Poor Quality
2
Major Part of the Object Missing
3
Object Moderately Segmented
4
Major Part of the Object Segmented
5
Acceptable Quality
TABLE II: Qualitative Grading Criteria
Fig. 5: Qualitative Analysis: Expert graded quality of randomly sampled masks predicted by ViT-L-resize-EM-dendrite with bbox_p4_n8 prompts.
Fig. 6: Quality Comparison in User Study and Efficiency Improvements: (a) masks from fully manual annotation, (b) masks from model-assisted annotation, (c) average time spent per object under different annotation modes.
Fig. 7: Mask Similarities in Different Annotation Modes: (a) & (b) are examples from dataset A, (c) & (d) are examples from dataset B.
Fig. 8: Quantitative Evaluation of Automatic Inference
Fig. 9: Qualitative Analysis for Automatic Inference
Vision foundation models have substantially advanced computer vision, enabling state-of-the-art performance in zero- and few-shot settings. They have been successfully applied to biomedical imaging tasks ranging from organ segmentation in computed tomography to cell segmentation in light microscopy. Electron microscopy (EM) is a central modality for analyzing cellular ultrastructure due to its nanometer-scale resolution. However, the application of foundation models in EM has so far been limited to specific organelles, such as mitochondria, largely due to the diversity of segmentation tasks and the scarcity of comprehensively annotated data. As a result, EM segmentation still predominantly relies on supervised learning, requiring extensive manual annotation and limiting ultrastructural analysis. To address this gap, we propose μMatch, a framework for semi-supervised learning and domain adaptation that leverages foundation models. We implement state-of-the-art student-teacher-based methods and evaluate multiple foundation models (SAM, SAM2, μSAM, DINOv2/v3) on challenging EM tasks, including mitochondrion, nucleus, and neurite segmentation. Our results demonstrate consistent improvements over strong baselines and highlight a path toward substantially reducing the annotation effort in EM.
Establishing large-scale, high-resolution neural connectivity maps is fundamental to elucidating the structural basis of brain function. However, when processing terabyte- or petabyte-scale electron microscopy data, over-segmentation inherent in automated reconstruction algorithms remains a critical bottleneck, requiring extensive manual proofreading spanning person-years. To alleviate the heavy reliance on annotated data and the limited flexibility of conventional tracing methods, we propose a training-free, targeted neuron tracing framework. Specifically, we introduce a skeleton-guided Heuristic Spatial Search paradigm that leverages geometric priors to iteratively reconstruct neuronal morphologies through a probing-verification cycle. To achieve robust zero-shot semantic verification, we further develop a Dimension-Aware Semantic Verification strategy built upon the foundation model NeuroSAM 2. This strategy resolves intra-slice splits via Planar Ensemble Consensus and inter-slice splits via Axial Spatio-Temporal Propagation. Notably, we integrate the proposed workflow into the Neuroglancer visualization platform, enabling an interactive human-in-the-loop proofreading system. Experimental results demonstrate that the proposed method outperforms supervised baselines and reduces manual proofreading time by 33.4%. The source code is publicly available at https://github.com/HeadLiuYun/Probe-EM.
Liuyun Jiang, Yanchao Zhang, Jinyue Guo +5
State Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Future Technology, University of Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences +1
Neuron counting and segmentation in microscopy images of neuronal cultures is a routine and time-consuming task in neuroscience research, traditionally performed through manual inspection or semi-automatic tools. We present NeuroAdaptTrainer, an open-source Fiji/ImageJ plugin that integrates a YOLO instance-segmentation model directly into the microscopist's workflow. The plugin allows a user to run automatic neuron detection on a single image or a batch of images, manually correct the resulting detections from within Fiji, and use those corrections to adapt the model to new imaging conditions via transfer learning. A built-in external validation module allows the base and adapted models to be compared quantitatively on a held-out annotated set. NeuroAdaptTrainer lowers the barrier for non-specialist users to benefit from deep-learning-based segmentation while keeping expert supervision at the center of the workflow.
aComputer Sciences Department, University of Oviedo, Asturias, Spain · bElectrical Engineering Department, University of Oviedo, Asturias, Spain · cBiomedical Engineering Center (BME), University of Oviedo, Asturias, Spain +3