Ca2+ transient detection and segmentation with the Astronomically motivated algorithm for Background Estimation And Transient Segmentation (Astro-BEATS)
Authors: Bolin Fan, Anthony Bilodeau, Frederic Beaupre, Theresa Wiesner, Christian Gagne, Flavie Lavoie-Cardinal, Renee Hlozek
Organizations: Dunlap Institute for Astronomy and Astrophysics, University of Toronto, Toronto, Canada · David A. Dunlap Department for Astronomy and Astrophysics, University of Toronto, Toronto, Canada · CERVO Brain Research Center, Québec, Canada · Institute Intelligence and Data, Université Laval, Québec, Canada · Department of Electrical Engineering and Computer Engineering, Université Laval, Québec, Canada · Canada CIFAR AI Chair, affiliated to Mila · Department of Psychiatry and Neuroscience, Université Laval, Québec, Canada
Fluorescence-based Ca2+-imaging is a powerful tool for studying localized neuronal activity, including miniature Synaptic Calcium Transients, providing real-time insights into synaptic activity. These transients induce only subtle changes in the fluorescence signal, often barely above baseline, which poses a significant challenge for automated synaptic transient detection and segmentation. Detecting astronomical transients similarly requires efficient algorithms that will remain robust over a large field of view with varying noise properties. We leverage techniques used in astronomical transient detection for miniature Synaptic Calcium Transient detection in fluorescence microscopy. We present Astro-BEATS, an automatic miniature Synaptic Calcium Transient segmentation algorithm that incorporates image estimation and source-finding techniques used in astronomy and designed for Ca2+-imaging videos. Astro-BEATS outperforms current threshold-based approaches for synaptic Ca2+ transient detection and segmentation. The produced segmentation masks can be used to train a supervised deep learning algorithm for improved synaptic Ca2+ transient detection in Ca2+-imaging data. The speed of Astro-BEATS and its applicability to previously unseen datasets without re-optimization makes it particularly useful for generating training datasets for deep learning-based approaches.
Figures & tables
Figure 1: Overview of Astro-BEATS in the context of mSCTs analysis. Raw Ca 2+ -imaging videos are processed using Astro-BEATS , an unsupervised source-finding algorithm used to generate segmentation masks of transients in these videos. The RHT [ 10 ] is used to generate a dendritic foreground mask, a mask of pixels that contain the foreground GCaMP fluorescence signal (top row, described in Supplementary Material A ). A dynamic estimate of the GCaMP6f fluorescence signal in the dendrites is used to estimate the average foreground in each frame. We take the difference image between this weighted foreground and our raw Ca 2+ video to isolate mSCTs from the foreground (middle row, described in Supplementary Material B ). Finally, density-based spatial clustering of applications with noise (DBSCAN) source-finding is used on pixels in the difference image to identify and segment transients (bottom row, described in Section 2.2.2 ).
Figure 2: A) Precision, Recall and F 1 Score for mSCT detection by Astro-BEATS , IDT and AQuA (N=9 videos). The center line represents the median value while, the boxes represents the 2 nd and 3 rd quartile range. Whiskers represent the maximum and minimum value and raw data points are plotted beside each box plot. B) Comparison of transients detected by Astro-BEATS compared to manual ground truth annotations (True positive (TP); gray, FN; red, FP; blue). Volume ( V ) is expressed in units of voxels (1 voxel = 0.00256 μ m 2 ms) and maximum brightness ( Bmax ) is obtained from the transient signal map ( σ ). See Supplementary Table 2 for descriptions of these variables. C) Comparison of transients detected by Astro-BEATS and IDT (detected by both algorithms: green, detected by Astro-BEATS only: grey, detected by IDT only: black cross, missed by both algorithms: red). D) Example images of out-of-focus, synaptic and dendritic transients during their temporal maximum. We also show the image 1s before the temporal peak of each transient. E) The V and Bmax of each transient types. F) The proportion of manually annotated Ca 2+ transients recovered by Astro-BEATS and IDT , for various types of Ca 2+ transients.
Figure 3: A) Segmentation DiCE scores calculated over manually segmented transients, using expert segmentation masks as ground truth. Since there is variation between expert annotations, we use inter-expert agreement DiCE scores as a baseline of comparison for segmentation performance ( Section 3.1.1 ). When comparing segmentation DiCE score across transient types described in Section 2.1.1 , we find that Astro-BEATS has comparable DiCE scores to IDT for synaptic transients, while having improved DiCE scores for dendritic and out-of-focus events. C) Example Annotations generated by Astro-BEATS compared to expert annotations. We chose example transients with Astro-BEATS DiCE segmentation scores closest to the median.
Parameter
Astro-BEATS
IDT
AQuA
Computation time
∼0.6 s/frame
∼30 s/transient
∼30 min/video
( ∼5 min/video)
( ∼3 hr/video)
Annotation Precision
0.92 ± 0.06
0.94 ± 0.10
0.56 ± 0.34
Annotation Recall
0.75 ± 0.14
0.38 ± 0.18
0.34 ± 0.22
Annotation F 1 score
0.81 ± 0.11
0.50 ± 0.17
0.39 ± 0.26
Segmentation DiCE score
0.66 ± 0.18
0.58 ± 0.21
0.41 ± 0.24
Table 1: Comparison of performance of mSCT segmentation and detection methods. In addition to improvements in annotation and comparable performance in segmentation, Astro-BEATS is significantly faster than other approaches.
Figure 4: A) Precision, Recall, and F 1 score for the optimal performing (see Section 2.5 ) 3D U-Net method trained using Astro-BEATS and IDT annotations as ground truth. Despite similar mean values, the variance in both Precision and Recall is reduced for the 3D U-Net AB compared to the 3D U-Net IDT and Astro-BEATS only. With the exception of a single video, 3D U-Net AB annotations yielded scores of F1>0.8 . B) DiCE score distribution for Astro-BEATS , the 3D U-Net AB , and the 3D U-Net IDT . A cutoff of DiCE > 0.45 is assigned to shown the fraction of segmentations with shapes vastly different from the expert segmentation. Some examples of these non-representative segmentation generated using Astro-BEATS and improved using 3D U-Net AB are shown in C).
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Supplementary Fig 1: Large-scale fluctuation in the brightness of GCaMP6f on the dendritic shaft. The dendrite appears brighter at t=0s compared to t=30s . These brightness fluctuations are accounted for in Astro-BEATS by taking a separate time-dependent background estimate for each frame.
Supplementary Fig 2: A) Optimization of the DBSCAN parameters. The range of DBSCAN parameters that yield good F 1 scores are shown in Table 1 . We find that for a range of 400<Minpts<700 and 5σ<μ<9σ , we obtain F 1>0.7 , which we consider to be acceptable. B) Precision, Recall and F 1 scores obtained by finding the DBSCAN parameters that achieve the highest F 1 score for one video, and applied over the remaining 9 test videos.
Supplementary Fig 3: A) Comparison of the performance of AQuA , IDT , and Astro-BEATS before and after manual expert reverification. B) We see that, prior to manual expert reverification, many TP detections made by Astro-BEATS are falsely labeled as FP or FN, despite their high brightness and volume due to misalignment error. C) Reverification allows us to better quantify the true number of FN and FP detections.
Supplementary Fig 4: A) TP detections made by Astro-BEATS that were classified as FP or FN due to misalignment error. B) Small and dim transients detected using Astro-BEATS but missed through manual annotation of the original testing dataset.
Supplementary Fig 5: A) Precision and Recall of Astro-BEATS on the out-of-distribution dataset described in Ref. [ 5 ] . Astro-BEATS parameters were not re-optimized for this new dataset and parameters listed in Supplementary Table 1 were used. This represents a hypothetical use case for applying Astro-BEATS to a new dataset without further fine tuning. B) Example raw data and transient segmentation maps taken from a Ca 2+ -imaging videos of this out-of-distribution dataset. Red circles in the raw video indicate detected transients in the segmentation map.
Parameter
Description
Range Tested
Value used here
μ
Initial brightness threshold
[3,11]σ
7σ
Minpts
Minimum points per cluster
[100,1000]
500
ϵ
Maximum spatial separation
[3,11] pixels
7 pixels
ϵt
Maximum time separation
[1,3] frames
2 Frames
Appendix
Supplementary Table 1: Parameters of the DBSCAN algorithm. Values used in this work are obtained using the procedure outlined in Supplementary Material C (see Supplementary Figure 2 ).
Parameter
Description
A
Area of 2D projection of detection within aperture
in number of square pixels
(1px 2 = 0.0256 μ m 2 ).
Note that detections with footprint area larger
than our aperture ( Section 2.2.3 ) will be
recorded as filling our entire aperture, having
Appendix
Supplementary Table 2: Properties of mSCTs calculated for each segmented event in the Ca 2+ -imaging video.
Parameter
Range Tested
Optimal Value Found
Smoothing
[0.5,2]σ
0.5 σ
Minimum size of groups of
connected tentative active voxels
[5,15] voxels
10 voxels
Appendix
Supplementary Table 3: Range of AQuA parameters used to optimize algorithm performance.
Synapses are densely packed submicron structures that dynamically reorganize during learning and memory formation. Longitudinal in vivo imaging of fluorescently tagged synaptic receptors offers a promising opportunity to study large-scale synaptic dynamics and how these processes are disrupted in neurological disease. However, in vivo imaging with 2-photon microscopy uses low laser power and therefore suffers from low signal-to-noise ratio (SNR) and high shot noise, nonlinear tissue motion between days, nonstationary fluctuations in synaptic fluorescence, and significant blur induced by the microscope point spread function (PSF). Together, these factors make it challenging to detect and track synapses, especially in regions with high synaptic density. This paper presents a novel template-based framework for modeling synapses as varying luminance point sources that move under a nonlinear tissue deformation. Taking a unified Bayesian approach, we apply this model to microscopy data by deriving a posterior that incorporates a diffeomorphic mapping for domain warping, a Gaussian point spread function for the imaging process, and a Poisson observation model for raw photon counts. The Bayesian solution simultaneously: (1) Constructs a probabilistic template of synapse locations, (2) denoises and deconvolves the image data, (3) infers fluorescence intensities, (4) performs diffeomorphic image registration to correct for tissue motion, and (5) provides confidence regions for these parameter estimates. We demonstrate the framework on both a 2D+t simulated dataset and a 3D+t longitudinal in vivo microscopy dataset of fluorescent synapses imaged in a mouse over two weeks.
Shashwat Kumar, Dominic M. Padova, Binish Narang +6
Department of Biomedical Engineering, Johns Hopkins University · Department of Neuroscience, Johns Hopkins University · Kavli Neuroscience Discovery Institute, Johns Hopkins University +2
Neuron counting and segmentation in microscopy images of neuronal cultures is a routine and time-consuming task in neuroscience research, traditionally performed through manual inspection or semi-automatic tools. We present NeuroAdaptTrainer, an open-source Fiji/ImageJ plugin that integrates a YOLO instance-segmentation model directly into the microscopist's workflow. The plugin allows a user to run automatic neuron detection on a single image or a batch of images, manually correct the resulting detections from within Fiji, and use those corrections to adapt the model to new imaging conditions via transfer learning. A built-in external validation module allows the base and adapted models to be compared quantitatively on a held-out annotated set. NeuroAdaptTrainer lowers the barrier for non-specialist users to benefit from deep-learning-based segmentation while keeping expert supervision at the center of the workflow.
aComputer Sciences Department, University of Oviedo, Asturias, Spain · bElectrical Engineering Department, University of Oviedo, Asturias, Spain · cBiomedical Engineering Center (BME), University of Oviedo, Asturias, Spain +3
In time-lapse fluorescence imaging, single-particle-tracking is a powerful tool to monitor the dynamics of objects of interest, and extract information about biological processes. However, tracked particles can be subject to occlusion and intermittent detectability. When these phenomena persist over a few frames, tracking algorithms tend to produce multiple tracklets for the same particle. In this work, we introduce self-supervised learning of visual features to compare tracked particles, and we exploit both visual and positional distances to robustly stitch tracklets representing the same particle. We demonstrate the performance of our stitching framework on time-lapse fluorescence sequences of Hydra vulgaris neurons. Results show high stitching precision, and reduction of errors made by previous algorithms on the same data by a factor of two.
Raphael Reme, Victor Piriou, Alison Hanson +5
IP Paris, BIA, IDS, IMAGES · Institut Pasteur, Universit´e de Paris-Cit´e, CNRS UMR 3691, BioImage Analysis Unit F-75015 Paris, France · LTCI, Telecom Paris, Institut Polytechnique de Paris, France +3