Electron Microscopy

Follow this topic Ask about this topic

Latest papers

36 papers6 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Oct 13, 2026

CryoTracer: A Synthetic-to-Real System for Filament Instance Tracing in Cryo-EM

Particle picking in cryo-electron microscopy is commonly formulated as an object-detection problem in which individual particles are represented by bounding boxes, a natural representation for globular particles but a poor match for filaments that can extend across an entire micrograph. Existing methods reconstruct filament trajectories by linking local predictions in a separate post-processing stage, decoupling detection from instance formation and creating potential ambiguities at gaps and crossings. Directly preserving filament identity is particularly useful for polymorphic specimens, where segments originating from the same filament should remain associated. However, learning such instance-level representations is challenging because expert centerline annotations are scarce and costly to obtain. We introduce CryoTracer, a synthetic-to-real framework that addresses this limitation through procedural micrograph generation, transformer-based polyline prediction, and specimen-specific fine-tuning. Procedural pretraining exposes the model to diverse filament geometries, intersections, and imaging conditions before adaptation using a limited number of fully annotated experimental micrographs. CryoTracer represents filaments as ordered centerline polylines and optimizes their geometry using an orientation-invariant discrete Fréchet objective, enabling consistent learning regardless of tracing direction. Evaluation on established and newly expert-annotated datasets shows that CryoTracer improves instance-level filament recovery over existing particle-detection approaches. These results establish end-to-end polyline prediction as a practical alternative to detection-and-linking pipelines for recovering complete filament instances with limited micrograph annotation budgets. Code and data are available at https://github.com/cryotracer/cryotracer

Radosław Kuczbański, Wojciech Kozłowski, Mateusz Grzesiuk +4

WUST, Poland · ETH Zurich, Switzerland · Tooploox, Poland

Electron Microscopy Cryo-Electron Microscopy Instance Segmentation

Oct 5, 2026

Patch-based Querying Identifies Structures of Interest in Electron Microscopy

Volume electron microscopy (vEM) has emerged as an essential sensing technique in biomedical research, allowing the three-dimensional imaging of biological cells and tissues at nanometer-scale resolution. The ability to generate extensive datasets has reached the limitations of downstream analysis processes, which depend significantly on the intervention of human experts for preprocessing and annotation. We propose an efficient and reliable patch-based retrieval framework based on self-supervised learning of local image descriptors to locate self-similar structures in vEM datasets. Given a few manual annotations of a given cellular structure, our method can retrieve similar structures across the EM volume. Our framework is interactive, allowing the human expert to refine the search queries and retrieve relevant image patches quickly and using little labeled data. Experiments on real-world vEM images of biological tissues demonstrate that our framework can reliably identify relevant cellular structures, generalize across different organelles and acquisition modalities, and substantially reduce the search space for downstream analysis.

Niels Vyncke, Nicolas Nadisic, Yvan Saeys +1

Department of Telecommunications and Information Processing, Sint-Pietersnieuwstraat 41, Ghent, 9000, Belgium · Royal Institute for Cultural Heritage (KIK-IRPA), Jubelpark 1, Brussels, 1000, Belgium · Department of Mathematics, Computer Science and Statistics, Krijgslaan 281, Ghent, 9000, Belgium

Image Retrieval Microscopy Image Analysis Electron Microscopy

Oct 1, 2026

Fold'EM: Direct atomic structure inference from Cryo-EM particles

Single-particle cryo-electron microscopy (cryo-EM) has become a widely adopted technique for biomolecular structure determination. The conventional cryo-EM computational pipeline first combines many particle images to reconstruct an electrostatic potential (ESP) map and then fits an atomic model to the recovered map. Density reconstruction has high sample complexity, requiring large numbers of particle images and making structure determination high-cost and low-throughput, particularly for heterogeneous samples. Downstream atomic model building, in turn, becomes increasingly difficult as the resolution of the reconstructed map deteriorates. Protein structure prediction models provide strong sequence-derived priors on atomic structure, and experiment-guided approaches can use these priors to recover structures consistent with experimental measurements. Yet, in cryo-EM, such priors are typically integrated only after density reconstruction during atomic model fitting. We introduce Fold'EM, an inference-time framework that combines priors from protein generative models directly with cryo-EM particle images to determine atomic models from a small number of single particle images, bypassing both intermediate density reconstruction and downstream model building against the reconstructed map. Across synthetic and experimental cryo-EM datasets, Fold'EM recovers accurate atomic structures both with known particle orientations and in an ab-initio setting where orientations are inferred jointly with structure. In heterogeneous datasets, Fold'EM further resolves distinct conformational states from mixed particle populations without separately reconstructing a density map and building an atomic model for each state. We believe these results open new avenues for structure determination in the low-sample regime and for characterizing low-population conformational states directly from cryo-EM particles.

Advaith Maddipatla, Märt-Erik Mäeots, Marco Pegoraro +5

Institute of Science and Technology Austria · University of Zurich · Frankfurt Institute for Advanced Studies +2

Cryo-Electron Microscopy Electron Microscopy

Sep 24, 2026

Atelier: Learning Local Self-Supervised Features for CryoEM Volumes via Hypernetworks

CryoEM map interpretation requires features that are spatially localized, consistent across samples, and informative across spatial scales. Most deep learning methods for map annotation extract features from fixed voxel grids. However, implicit neural representations (INRs) are able to model volumetric data as scale-agnostic, coordinate-conditioned functions. INRs are therefore attractive for cryoEM, but fitting a separate INR for each map is too expensive for large-scale feature extraction and produces representations that are not aligned across samples. We introduce Atelier, a self-supervised framework that amortizes INR fitting for reconstructed cryoEM maps. Pretrained on 5,439 Electron Microscopy Data Bank maps, Atelier is a transformer-based hypernetwork that generates high-fidelity reconstructions across a wide range of protein structures, including large multi-subunit assemblies. Beyond reconstruction, the INR generated by the pretrained transformer exposes a continuous, local feature field through its intermediate activations at any spatial query point, a property that voxel grid and patch-tokenizer architectures do not naturally provide. Used as auxiliary channels to a 3D nested U-Net annotation head trained from scratch, these coordinate-conditioned features improve performance on eight voxel-level property prediction tasks over a volume-only baseline. Our results demonstrate that amortized implicit neural representations are an effective primitive for geometry-aware analysis of cryoEM data.

Phillip Lo, Sudarshan Babu, Dari Kimanius +1

Biohub Chicago, IL · University of Chicago, Biohub Chicago, IL

3D Representation Learning Cryo-Electron Microscopy Electron Microscopy Implicit Neural Representations Self-Supervised Learning Hypernetworks

Sep 22, 2026

Real-Time Atomic-Resolution Electron Phase Imaging without Probe Calibration via Ptychography-Supervised Learning

Atomic-scale phase imaging is central to resolving defects, interfaces, and weakly scattering atoms that govern the behavior of nanoscale materials. Electron ptychography delivers sub-ångström phase sensitivity but remains an offline technique, because its iterative reconstruction is computationally expensive and sensitive to experimental calibration, preventing live use during data acquisition. Here, a ptychography-supervised local inference framework is presented that converts four-dimensional scanning transmission electron microscopy (4D-STEM) into an acquisition-compatible phase-imaging workflow. Physics-constrained reference phase maps reconstructed from a single experimental AuPd dataset serve as teacher labels for a compact model that predicts local phase patches directly from diffraction measurements, without explicit probe input or online iterative optimization. Full-field images are assembled by deterministic overlap stitching. The workflow reaches an online latency of about 0.27 ms per probe position and a throughput of about 20,000 positions per second, an approximately 1,000-fold speed-up over GPU-accelerated ePIE, while preserving atomic-scale lattice contrast and reciprocal-space fidelity. Without fine-tuning, the same model transfers across materials (WS2), defocus conditions (high-entropy alloy nanoparticles), and instruments (hBN at 300 kV). The approach amortizes ptychographic redundancy into a fast, generalizable workflow that enables real-time atomic-scale phase imaging for materials microscopy.

H. Yue, C. -C. Chen, C. -N. Hsiao +4

College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai 200433, China. · Department of Engineering and System Science, National Tsing Hua University, Hsinchu 300044, Taiwan · National Center for Instrumentation Research, National Institutes of Applied Research, Hsinchu 300092, Taiwan +3

Supervised Learning Electron Microscopy Computational Imaging Image Reconstruction Scanning Transmission Electron Microscopy

Sep 19, 2026

SomaNet: Weakly Supervised Learning for Instance Soma Segmentation in 3D Electron Microscopy with Partial Annotations

Soma instance segmentation, i.e., identifying and delineating individual cell somas as distinct instances, is crucial for cellular analysis and connectomic reconstruction. Three-dimensional electron microscopy (3D EM) provides nanometer-scale resolution for capturing fine-grained soma morphology. However, dense instance-level manual annotation is prohibitively costly, limiting the scalability of fully supervised methods. To address this challenge, we propose SomaNet, a weakly supervised framework for 3D EM soma instance segmentation under partial annotation constraints. SomaNet adopts a teacher--student learning paradigm tailored to partial labels. The teacher is trained using partially annotated data to generate pseudo-labels, while the student jointly learns from the partial ground-truth annotations and the generated pseudo-labels, progressively recovering dense instance segmentations. To accurately delineate cell somas under limited supervision with varying instance counts, SomaNet incorporates affinity learning, which encourages high similarity within instances and low similarity across instance boundaries. Semantic-guided affinity decoding and 2D-to-3D reconstruction then produce volumetrically consistent 3D soma instances while preserving the large receptive fields of 2D backbones. The framework is architecture-flexible and supports diverse backbones, including vision transformers and foundation models, enabling direct transfer of pretrained visual representations to volumetric EM segmentation. Experiments on 3D EM brain datasets demonstrate that SomaNet achieves accurate and robust soma instance segmentation across regions with diverse soma morphologies under partial annotation. Code is available at https://github.com/mkhateri/SomaNet.

Mohammad Khateri, Morteza Ghahremani, Jussi Tohka +1

A. I. Virtanen Institute for Molecular Sciences, Faculty of Health Sciences, University of Eastern Finland, Finland

Electron Microscopy Instance Segmentation 3D Instance Segmentation Microscopy Image Analysis Weakly Supervised Learning

Sep 18, 2026

Catena: A Comprehensive Software Suite for Large-Scale Connectomics

The gold standard datasets for mapping connectomes are electron microscopy volumes of densely labeled neural tissue at nanometer resolution. Yet reconstructing and proofreading neuronal arbors and annotating all synapses requires pipelining multiple software tools that are often fragmented, inconsistently maintained, or proprietary, hindering reproducibility and automation. Here, we introduce Catena, an open-source, comprehensive, developer-centric software suite for connectomics that integrates modules for 3D neuron and organelle segmentation, synapse detection, microtubule tracking, and neurotransmitter inference. Catena organizes its modules in composable, chunk-wise processing pipelines in a completely documented, extensible, and adaptable design. We further reduce compute and ground-truth data requirements with pretrained machine learning models, facilitating fine-tuning. Catena ships fully containerized modules that encapsulate evolving dependencies for consistent execution across workstations and clusters. By consolidating open components, shareable models, and containerized runtimes, Catena delivers a reproducible and scalable approach to mapping cellular connectomes from electron microscopy volumes. Code and documentation: https://github.com/Mohinta2892/catena.git

Samia Mohinta, Pedro Gómez-Gálvez, Shi Yan Lee +5

Dept. of Physiology, Development and Neuroscience, University of Cambridge, UK · MRC Laboratory of Molecular Biology, UK · HHMI Janelia, USA +1

Electron Microscopy Microscopy Image Analysis Scientific Workflow Automation 3D Medical Image Segmentation

Sep 12, 2026

Adapting Open-Weight MLLMs to Generate Point Prompts for Electron Microscopy Segmentation

Promptable models such as microSAM segment electron microscopy (EM) images from point prompts, but automation requires generating prompts without user input. We ask whether open-weight multimodal large language models (MLLMs) can generate them from natural-language requests by returning coordinates to a frozen segmenter. To that end, we convert masks from three mitochondria datasets into training examples, pairing images and instructions with centroid coordinates, then train LoRA adapters while freezing the MLLM backbone and microSAM. We find that Qwen3-VL reaches segmentation AP

_{50}

0.736

after supervised fine-tuning and reward optimization, up from

0.247

without adaptation, while automatic prompt generation (APG) achieves

0.773

. In addition, two other MLLMs improve, reaching or exceeding APG. When compared with a supervised centroid-heatmap detector that reaches AP

_{50}

0.904

for this mitochondria task, Qwen3-VL more closely matches the annotated point set and instance counts. Moreover, training on two public datasets transfers to an unseen third, while training on all three transfers to an independent EM volume. Robustness tests show stable performance under unseen formulations of the natural-language request, while the coordinates can be reused by a second segmenter. To our knowledge, this is the first feasibility study of open-weight MLLMs as EM point generators, providing an inspectable, language-directed link between localization and mask decoding.

Samia Mohinta, Albert Cardona

University of Cambridge, Cambridge, UK · MRC Laboratory of Molecular Biology, Cambridge, UK

Electron Microscopy VLM Adaptation Microscopy Image Analysis Multimodal Large Language Models Promptable Image Segmentation Prompt Engineering

Sep 8, 2026

FPicker: Topology-Guided Evolution for Filament Tracing in Low-SNR Microscopy

Automating filament tracing in Cryo-Electron Microscopy (Cryo-EM) is essential for 3D helical reconstruction but challenged by intersecting topologies and extremely low Signal-to-Noise Ratios (

\text{SNR} = σ_s^2/σ_n^2

< 0.1 or -10 dB). Existing paradigms fail: pixel-wise segmenters suffer from severe topological fracturing, box-based detectors face ghost center drift, sequential trackers derail due to error accumulation, and traditional active contours collapse under artificial closed-curve constraints. To resolve these bottlenecks, we present FPicker, the first topology-guided framework reconciling these incompatibilities. It unifies perception via a center-endpoint representation and an open-curve evolution module to explicitly model non-cyclic connectivity. On simulated benchmarks, FPicker outperforms top baselines by over

40\%

relative gain in mean spatio-angular precision (mSAP) and reduces topological gap rates by over

60\%

under extreme noise (

-20\text{ dB}

). By learning intrinsic physical geometry rather than local texture, FPicker demonstrates strong potential as a resilient geometric backbone. Its zero-shot performance on the real-world EMPIAR dataset exhibits robust topological resistance, achieving a state-of-the-art 82.9% mSAP upon fine-tuning. Our results also suggest modeling physical priors is a highly robust path toward bridging the sim-to-real gap in signal-starved scientific imaging. The code is publicly available at: https://github.com/tomzhaosky/FPicker.

Tingyin Zhao, Mingtao Huang, Yuan Shen

Department of Electronic Engineering, Tsinghua University, Beijing, China · Beijing National Research Center for Information Science and Technology, Beijing, China

Cryo-Electron Microscopy Microscopy Image Analysis Electron Microscopy

Sep 2, 2026

Scalable Bayesian Optimization of Composite Functions for Image-Based Inverse Problems in Materials Characterization

Estimating physical parameters from scientific images is a common inverse problem in materials characterization that often relies on expensive physics-based simulations. In electron microscopy, specimen thickness and crystal mistilt are critical parameters that govern how electrons scatter through the sample, and therefore the accuracy of any atomic-scale structure recovered from it. They are commonly inferred by matching experimental position-averaged convergent-beam electron diffraction (PACBED) patterns to simulated ones, but grid searches scale poorly and neural-network methods require extensive pretraining that may not transfer to new conditions. Here, we propose scalable Bayesian optimization of composite functions (SBOCF), a simulation-efficient method that exploits the known composite structure of the image-matching objective and the intermediate information contained in simulated images. By representing PACBED images with patch-level summaries and two correction terms, SBOCF preserves the original pixel-wise objective while reducing the number of modeled outputs from 24,649 to 11. Under a budget of 50 simulator evaluations, SBOCF outperformed standard Bayesian optimization with expected improvement on synthetic SrTiO3 benchmarks with thick and thin specimens, reducing the median final SSE by up to 290x in the thick-sample case. On experimental data, SBOCF produced parameter estimates consistent with previously reported values without task-specific pretraining. For a simulated mistilted specimen, using the SBOCF estimates in a downstream ptychographic reconstruction recovered sharp atoms that were otherwise blurred. These results establish SBOCF as a promising approach for inverse problems involving expensive simulators and high-dimensional structured outputs.

Dasol Yoon, Poompol Buathong, Chia-Hao Lee +3

School of Applied and Engineering Physics, Cornell University, Ithaca, NY 14853, USA · Department of Materials Science and Engineering, Cornell University, Ithaca, NY 14853, USA · Center for Applied Mathematics, Cornell University, Ithaca, NY 14853, USA +2

Electron Microscopy Bayesian Optimization Inverse Problems Microscopy Image Analysis

Aug 31, 2026

TopoFuse: Topology-Aware Tri-Planar Fusion for 3D Cryo-Electron Tomography Segmentation

Automated segmentation of cryo-electron tomograms routinely produces masks that are voxel-accurate but topologically broken: membranes fragment, organelles merge into one another, and enclosed cavities collapse. Existing topology-aware losses reduce these violations but cannot eliminate them, because topology is encouraged through gradient pressure rather than structurally enforced. We introduce TopoFuse, which reframes topology as a differentiable projection operator rather than a loss penalty. At each forward pass, the projection operator

\mathrm{Proj}_T

(a PH-guided sparse edit) identifies the critical voxels responsible for topological violations via bottleneck matching and applies sparse edits to satisfy a specified topology target (diagram feature counts and lifetime budgets) for dimensions

d \in \{0,2\}

. If the projection converges, the output satisfies those constraints on the downsampled grid (

s=2

); when it does not, a repair certificate exposes this explicitly, enabling downstream filtering. A topology prior head predicts the correction target directly from input features, removing any dependence on ground-truth topology at inference. Across three cryo-ET benchmarks, TopoFuse reduces Betti number error by 54% over the strongest soft-loss baseline (

p < 0.001

), improves Dice by 4.6 points, and edits only 3.1% of voxels to achieve this.

Rohit Kumar Salla, Neelesh Gupta, Xingjian Li +1

Virginia Tech, Blacksburg, VA, USA · National Institute of Technology, Kurukshetra, India · Carnegie Mellon University, Pittsburgh, PA, USA

Topology-Preserving Image Segmentation Image Segmentation 3D Medical Image Segmentation Cryo-Electron Microscopy Electron Microscopy Topological Data Analysis

Aug 28, 2026

Reconstruction-Aware Cryo-EM Particle Picking

Cryo-electron microscopy (cryo-EM) determines the structures of proteins and macromolecular assemblies at near-atomic resolution, and the final 3D reconstruction depends on extracting a clean particle stack from noisy micrographs. This extraction decomposes into three sub-tasks, namely particle picking, contamination removal, and 2D class selection. Each of them, however, is trained and evaluated in isolation, and none is optimized for the reconstruction. We instead integrate the three sub-tasks into a single pipeline posed against downstream reconstruction quality. We instantiate the pipeline with a state-of-the-art component for each sub-task, CryoTransformer picking permissively, MicrographCleaner masking contamination, and CryoSift selecting 2D classes by a continuous quality score, and close the loop with a fine-tuning step that returns the surviving particles to the picker. The pipeline achieves a better 3D resolution than every picker we compare. We also show that the best 2D F1 is not the best resolution, so particle selection is better treated as one reconstruction-aware pipeline judged by the map it delivers.

Riku Itsuji, Yuanhao Wang, Xingjian Li +3

Keio University, Yokohama, Japan · Keio AI Research Center, Yokohama, Japan · Carnegie Mellon University, Pittsburgh, PA, USA

Cryo-Electron Microscopy Electron Microscopy 3D Reconstruction

Aug 26, 2026

A meta-algorithm for ab initio reconstruction of complex mixtures in cryo-EM

We describe a systematic approach for spawning and aggregating multi-class cryo-EM reconstruction jobs. This approach formalizes standard ad hoc strategies of iterative classification and filtering typically used by practitioners to sort impure, heterogeneous samples. To our knowledge, this is the first method that can successfully perform ab initio reconstruction on datasets containing dozens of distinct species. We obtain 97% accuracy on ab initio reconstruction of a 45-class subset of Tomotwin-100, 75% accuracy on the full Tomotwin-100 dataset, and demonstrate recovery of ribosomal assembly states from an unfiltered experimental cryo-EM dataset. Our approach's capability scales with compute and lays the foundation for automated cryo-EM workflows in modern experimental settings.

Alkin Kaz, Arda Kaz, Ellen D. Zhong

Princeton University · University of California, Berkeley

Cryo-Electron Microscopy Electron Microscopy 3D Reconstruction Scientific Workflow Automation

Aug 13, 2026

An Interactive, Automated 4D-STEM data acquisition and analysis routine for Scanning Electron Nanobeam Diffraction and Ptychography experiments

Modern transmission electron microscopes are versatile instruments which have become indispensable tools for understanding structure and chemical composition at the nano- and atomic scale. In the physical sciences these instruments are still largely manually controlled, requiring significant operator expertise, limiting throughput, and precluding statistical analysis of large datasets. Recent technical advances in both hardware and in control software now allow for the interaction with almost every functionality of the microscope through a programming interface. This enables better experimental design and data collection automation while also reducing operator collection bias and required expertise. In this study, we present an automated data collection routine with machine-driven decision-making to enable the collection of hundreds of 4D-STEM nanobeam diffraction and ptychography data from a large distribution of size-selectively deposited Pt nanoparticles. We present a semi-automated data analysis workflow to extract pertinent information from the large volumes of collected data. For the nanobeam diffraction data, reducing each dataset to its azimuthal variance profile and combining automated crystal orientation mapping with per-particle morphology descriptors reveals the orientation, shape and phase distributions across the ensemble, including a weak {110} texture. For the ptychographic data, an automated screening pipeline identifies on-zone-axis particles and enables atomic-resolution phase imaging and lattice-strain mapping of individual grains. Together these demonstrate how automation turns instrument throughput into statistically meaningful, atomic-scale microstructural information.

Mohsen Danaie, Max England, Yiming Xu +11

Electron Microscopy Scientific Workflow Automation Laboratory Automation Materials Science

Jul 30, 2026

Physics-Aligned Self-Supervised Learning for Scientific Imaging

Data augmentations define the invariances learned by self-supervised learning (SSL). Standard augmentation pipelines were designed for natural images, yet scientific imaging modalities are governed by physical measurement processes with distinct symmetry and acquisition constraints. Enforcing invariances that contradict these constraints can distort learned representations and limit downstream performance, but practitioners moving from machine learning into a new scientific modality currently have little guidance beyond transferring natural-image pipelines unexamined. We address this gap with a principled, reproducible procedure for augmentation design in scientific SSL: we formalise the physics-aligned augmentation set as a union of measurement-consistent symmetries and acquisition-driven perturbations, and we give a concrete, largely label-free workflow---enumerate candidates, label each by the measurement operator, validate with representation-geometry diagnostics, and confirm by single-factor ablation---for selecting them. We instantiate the procedure for real-space electron microscopy and reciprocal-space 4D-STEM diffraction, and evaluate it across five SSL paradigms (DINOv2, SimCLR, MAE, VICRegL, I-JEPA) on classification and crystal-orientation regression. Physics-aligned augmentations substantially improve downstream performance for objectives relying on cross-view consistency, reduce geodesic error and improve robustness under realistic acquisition variability (detector gain, resolution loss), and systematically reshape representation geometry. While our experiments use electron microscopy, the procedure is modality-agnostic and applies to other measurement-driven domains such as medical and remote-sensing imaging. These results position augmentation design as a primary, and controllable, source of inductive bias in scientific self-supervised learning.

Bashir Kazimi, Stefan Sandfeld

Institute for Materials Data Science and Informatics (IAS-9), Forschungszentrum Jülich GmbH, 52425, Jülich, Germany · Chair of Materials Data Science and Materials Informatics, Faculty 5 – Georesources and Materials Engineering, RWTH Aachen University, 52056, Aachen, Germany

Electron Microscopy Data Augmentation Scientific ML Self-Supervised Visual Representation Learning Microscopy Image Analysis

Jul 24, 2026

Frequency-Aware Dual-Stream Learning for Balanced Realism and Fidelity in Electron Microscopy Imaging

Electron microscopy enables nanoscale cellular visualization but faces a trade-off between imaging resolution and acquisition speed. Existing learning-based methods rely on single-stream architectures that struggle to balance perceptual realism and quantitative fidelity, either over-smoothing details or generating unrealistic hallucinations. This work introduces a frequency-adaptive dual-stream architecture to resolve this conflict. Using discrete wavelet transform, we decompose images into low-frequency structures and high-frequency details, then employ a conditional diffusion model for realistic global synthesis and a transformer network for precise detail recovery. Experiments on the EMDiffuse dataset show the method achieves superior LPIPS and resolution ratio, substantially outperforming existing approaches. The method also shows strong generalization across diverse biological samples, supporting fast and reliable electron microscopy imaging for structural biology and nanotechnology applications. The source code and associated dataset are publicly available to facilitate further research.

Longmi Gao, Zhengkai Zhao, Pan Gao +1

College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, China. · School of computing, Mathematics, and Engineering, Charles Sturt University, NSW, Australia.

Diffusion Models for Image Restoration Microscopy Image Analysis Image Super-Resolution Electron Microscopy

Jul 18, 2026

Mapping Order in Semicrystalline Polymers using Machine Learning of Nanobeam Electron Diffraction

Organic mixed ionic electronic conductors (OMIECs) are a promising class of polymer materials for applications spanning neuromorphic computation to energy efficient electronics and bioelectronics. Despite being highly tunable, the relationship between structural features and key performance properties such as charge carrier mobility is poorly understood. Scanning nanodiffraction in the transmission electron microscope (TEM) is a powerful probe for elucidating this structure-property relationship, but produces large, noisy datasets that are difficult to interpret because polymer reflections exhibit several distinct morphologies. To address the complexity, we trained a machine learning (ML) model to detect these polymer diffraction peaks and their intensities from synthetic data. Compared to correlative peak detection algorithms, the conventional method for analyzing nanobeam 4D scanning transmission electron microscopy (4DSTEM) data, we show that the ML model is significantly faster and outperforms correlative algorithms in almost all cases, opening up the possibility of near-live visualization of 4DSTEM experiments.

Nicholas Marchese, Arthur R. C. McCray, Yael Tsarfati +4

Department of Materials Science and Engineering, Stanford University, 496 Lomita Mall, Stanford, CA 94305, USA · SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, CA 94025, USA · National Center for Electron Microscopy, Molecular Foundry, Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA 94720, USA

Materials Science Electron Microscopy Scanning Transmission Electron Microscopy

Jul 11, 2026

The evolution of AI from image interpretation toward scientific inference in nanoparticle electron microscopy

Artificial intelligence (AI) is transforming electron microscopy by enabling quantitative analysis of increasingly large and complex datasets for nanoparticle characterization. Recent advances in machine learning (ML) and deep learning (DL) have expanded microscopy from a descriptive imaging technique into a data-driven platform for structural interpretation, dynamic analysis, and scientific inference. This review examines AI methodologies for nanoparticle electron microscopy, focusing on transmission electron microscopy (TEM), high-resolution transmission electron microscopy (HRTEM), scanning transmission electron microscopy (STEM), and in situ TEM. The discussion is organized around the principal challenges in nanoparticle characterization, including particle detection, segmentation, morphology quantification, atomic-resolution restoration, defect identification, two-dimensional-to-three-dimensional structural inference, and analysis of dynamic processes in situ. We review computational approaches from conventional ML and convolutional neural networks to transformer architectures, self-supervised learning, foundation models, multimodal AI, and physics-informed learning. We further discuss integrating microscopy data with simulations, metadata, and autonomous experimentation to relate nanoparticle structure, dynamics, synthesis conditions, and functional properties. The advantages, limitations, benchmarking, and data requirements of current methodologies are critically assessed. Finally, emerging opportunities for foundation models, AI-guided microscopy, closed-loop experimentation, and autonomous materials discovery are discussed. By integrating advances across computer vision, materials informatics, and electron microscopy, this review highlights the role of AI in next-generation nanoparticle characterization and accelerated materials discovery.

Evropi Toulkeridou, Jiafei Li, Leonardo Lari +1

Statistical Analysis and Optimal Design of Experiments Group, Department of Mathematics, University of Castilla-La Mancha, 45071 Toledo, Spain · Materials Science and Engineering, Guangdong Technion – Israel Institute of Technology, Shantou, Guangdong 515063, China · LAME Laboratory, Area Science Park Campus di Basovizza, SS14, KM163.5, 34149, Trieste, Italy +2

Materials Science Microscopy Image Analysis Computational Materials Science Electron Microscopy

Jun 28, 2026

STEMGym: Benchmarking Sequential Decision-Making under Dose Budgets in Autonomous Electron Microscopy

A central premise of autonomous scientific imaging is that smarter navigation, whether Bayesian, RL-based, or otherwise adaptive, is the principal lever for sample-efficient acquisition. We present evidence to the contrary in scanning transmission electron microscopy (STEM), an atomic-resolution imaging modality whose every measurement deposits damaging electron dose. We introduce STEMGym, an open-source Gymnasium benchmark of 15 physics-simulated STEM worlds spanning five materials, three difficulty levels, and four characterisation tasks, scored by the Dose-Efficiency Curve area (DEC-AUC), a single scalar capturing the information-vs-dose Pareto frontier. Across 33 agent configurations under realistic dose budgets, the dominant determinant of dose efficiency is the analyst (perception) pipeline, not the navigator: pairing a trained CNN analyst with naïve raster scanning raises DEC-AUC by 5.5x over a CNN-free raster baseline (0.287 vs.\ 0.052), while substituting Bayesian or adaptive finite-state-machine navigation for raster yields no statistically significant further gain. Production-tier vision-language models further underperform task-specific CNNs by {\sim}13x on crystallographic defect analysis. By decoupling perception, navigation, and planning under a unified dose budget, STEMGym reframes where ML effort should be invested in autonomous electron microscopy and provides the measurement infrastructure to test it.

Can Polat, Erchin Serpedin, Mustafa Kurban +1

Dept. of Electrical & Computer Eng. Texas A&M Univ. College Station, TX, USA · Dept. of Prosthetics & Orthotics College of Science & Eng. Ankara Univ., Ankara, Turkey · Hamad Bin Khalifa Univ. and Texas A&M Univ. at Qatar Doha, Qatar

Scanning Transmission Electron Microscopy Autonomous Experimentation Microscopy Image Analysis Electron Microscopy

Jun 19, 2026

$μ$ Match: Foundation Models for Semi-supervised Learning and Domain Adaptation in EM

Vision foundation models have substantially advanced computer vision, enabling state-of-the-art performance in zero- and few-shot settings. They have been successfully applied to biomedical imaging tasks ranging from organ segmentation in computed tomography to cell segmentation in light microscopy. Electron microscopy (EM) is a central modality for analyzing cellular ultrastructure due to its nanometer-scale resolution. However, the application of foundation models in EM has so far been limited to specific organelles, such as mitochondria, largely due to the diversity of segmentation tasks and the scarcity of comprehensively annotated data. As a result, EM segmentation still predominantly relies on supervised learning, requiring extensive manual annotation and limiting ultrastructural analysis. To address this gap, we propose

μ

Match, a framework for semi-supervised learning and domain adaptation that leverages foundation models. We implement state-of-the-art student-teacher-based methods and evaluate multiple foundation models (SAM, SAM2,

μ

SAM, DINOv2/v3) on challenging EM tasks, including mitochondrion, nucleus, and neurite segmentation. Our results demonstrate consistent improvements over strong baselines and highlight a path toward substantially reducing the annotation effort in EM.

Marei Freitag, Olesia Korchevaia, Luca Freckmann +2

Life and Medical Sciences Institute (LIMES), University of Bonn, Germany · Institute of Computer Science, Georg-August-University Göttingen, Germany

Electron Microscopy Unsupervised Domain Adaptation Vision Foundation Model Adaptation Semi-Supervised Medical Image Segmentation Microscopy Image Analysis

Jun 9, 2026

Unsupervised Deep Learning for Limited-Angle STEM-EDX Tomography -- Application to 3D Chemical Analysis of Phase-Change Memory Devices

Energy Dispersive X-ray (EDX) tomography in Scanning Transmission Electron Microscopy (STEM) enables 3D compositional and elemental mapping at the nanoscale, but its use is limited by restricted tilt ranges and low-dose conditions required to avoid beam damage. Limited-angle acquisition introduces missing-wedge artefacts such as elongation and anisotropic resolution, while noisy low-dose data further degrade reconstruction quality and quantitative reliability. Here, we introduce an unsupervised deep learning framework based on Deep Image Prior with total variation regularization (DIP-TV) for limited-angle STEM-EDX tomography. We extend it to a multi-channel formulation (DIPm-TV) that jointly reconstructs multiple elemental maps by exploiting spatial correlations. Using a synthetic 3-channel phantom, we show that the method compensates for severe missing-wedge artefacts corresponding to approximately

100^\circ

of missing angular range under moderate noise, outperforming simultaneous iterative reconstruction technique and compressed sensing approaches. We apply the method to 3D chemical analysis of Ge-Sb-Te (GST) memory devices in virgin (as-fabricated) and SET (crystalline) operational states. Samples were prepared as cross-sectional focused ion beam lamellae and acquired under a limited-angle tilt range from

-40^\circ

to

+40^\circ

with

5^\circ

steps and a dose of

2.0\times10^5

e^-/Ang^2

. The multi-channel approach enables voxel-by-voxel elemental reconstruction using only EDX signals without external structural priors such as high-angle annular dark-field imaging. The reconstructed volumes show near-isotropic spatial resolution and reveal compositional heterogeneities associated with device operation. This approach enables 3D chemical characterization in experimentally accessible sample geometries where conventional methods fail due to severe angular limitations.

Daniel del Pozo Bueno, Serge Brosset, Theo Monniez +3

1CEA, LETI, Univ. Grenoble Alpes, Grenoble, 38000, France. · 2CEA, Neurospin, Paris-Saclay University, Gif-sur-Yvette, F-91191, France. · 3Inria, MIND, Palaiseau, F-91210, France.

Deep Image Prior Image Reconstruction Computed Tomography Electron Microscopy Scanning Transmission Electron Microscopy

Jun 8, 2026

POPSICLE: Benchmark Datasets for Segmentation and Localization in CryoET

Cryo-electron tomography (cryoET) has emerged as a powerful tool in structural and cellular biology by enabling direct visualization of macromolecular structures within intact cells, thereby linking molecular architecture to cellular organization in a native context. Realizing the full potential of cryoET, however, increasingly depends on advances in computational analysis, particularly machine learning (ML), to interpret its complex and information-rich data. Despite rapid progress, ML development for cryoET remains bottlenecked by the lack of standardized, well-annotated benchmarks. Existing evaluations are typically small, task-specific, and are assembled in isolation, limiting robust comparisons across methods. Here, we present POPSICLE, a benchmark suite for cryoET segmentation and macromolecular localization built from the CryoET Data Portal - an open, ML-ready repository of tomographic data, metadata, and annotations. POPSICLE spans eukaryotic and prokaryotic systems, both purified and fully in situ samples, and dense voxel-wise segmentation as well as sparse localization tasks. Built on a living data resource, it can expand as new datasets and annotations become available. Baseline experiments reveal substantial variation in model rankings across tasks, underscoring the need for benchmarks tailored to the unique characteristics of cryoET rather than evaluation practices adapted from adjacent biomedical imaging domains. POPSICLE thus provides an open and extensible foundation for reproducible ML evaluation in cryoET.

Jonathan Schwartz, Utz Heinrich Ermel, C. Braxton Owens +6

1Biohub, Redwood City, CA 94063, USA · 2Brigham Young University, Provo, UT, 84602, USA

Image Segmentation Cryo-Electron Microscopy Microscopy Image Analysis Electron Microscopy

May 26, 2026

Unsupervised Deep Image Prior for Sparse-View and Limited-Angle Electron Tomography

Electron tomography (ET) plays an important role in the three-dimensional (3D) characterization of nanomaterials. However, under limited-angle and sparse-view conditions, conventional algorithms produce degraded reconstructions, which compromise the quality and interpretability of resulting 3D data. In this paper, we present deep image prior (DIP), an unsupervised deep learning (DL) approach, for highly degraded tomography acquisitions and demonstrate, using simulated data, that its performance is comparable to that of supervised approaches requiring training datasets, even for tilt ranges as limited as 60° and tilt increments of 10°. We then apply it to experimental data and show that it enables reliable 3D quantification under both sparse-view and limited-angle conditions, highlighting its potential for a wide range of materials and acquisition modalities.

Serge Brosset, Daniel del Pozo Bueno, Thomas David +3

a Univ. Grenoble Alpes, CEA, Leti, F-38000 Grenoble, France. · b Univ. Grenoble Alpes, CEA, Liten, F-38000 Grenoble, France. · c CEA, Joliot, NeuroSpin, Gif-sur-Yvette Cedex 91191, France. +1

Deep Image Prior Sparse-View CT Reconstruction Computed Tomography Image Inverse Problems Electron Microscopy

Apr 27, 2026

Contrastive Image-Metadata Pre-Training for Materials Transmission Electron Microscopy

The transmission electron microscope facilitates the highest-resolution imaging of any instrument ever created, and its limiting factor is no longer spatial resolution but dose efficiency. Low electron doses avoid sample damage but produce noisy images for which, unlike in classical computer vision, there is no ground truth. Autonomous materials experimentation poses a related problem, since closed-loop instruments need representations grounded in the microscope state at acquisition. Both demand representations grounded in how an image was acquired. We release 7,330 paired high-angle annular dark-field scanning-TEM (HAADF-STEM) images and their seven-dimensional acquisition metadata, and propose Contrastive Image-Metadata Pre-training (CIMP), a CLIP-style encoder that aligns the two modalities and reaches 84.4% Top-1 cross-modal retrieval on a held-out split. All seven parameters are individually recoverable from the frozen visual embedding through a linear probe, and we use the embedding to condition a metadata-conditioned style-transfer model that re-renders experimental images under different acquisition parameters. Virtually scaling dwell time and beam current of low-dose images turns this model into a physics-informed denoiser; in a blind user study, experimental microscopists prefer it over the current state-of-the-art denoiser for STEM imagery on 70.2% of trials.

Georgia Channing, Debora Keller, Marta D. Rossell +3

Hugging Face · University of Oxford · Empa Dübendorf +1

Cross-Modal Alignment Multimodal Contrastive Learning Scanning Transmission Electron Microscopy Microscopy Image Analysis Image Denoising Electron Microscopy

Apr 20, 2026

Score-Based Matching with Target Guidance for Cryo-EM Denoising

Cryo-electron microscopy (cryo-EM) enables single-particle analysis of biological macromolecules under strict low-dose imaging conditions, but the resulting micrographs often exhibit extremely low signal-to-noise ratios and weak particle visibility. Image denoising is therefore an important preprocessing step for downstream cryo-EM analysis, including particle picking, 2D classification, and 3D reconstruction. Existing cryo-EM denoising methods are commonly trained with pixel-wise or Noise2Noise-style objectives, which can improve visual quality but do not explicitly account for structural consistency required by downstream analysis. In this work, we propose a score-based denoising framework for cryo-EM that learns the clean-data score to recover particle signals while better preserving structural information. Building on this formulation, we further introduce a target-guided variant that incorporates reference-density guidance to stabilize score learning under weak and ambiguous signal conditions. Rather than simply amplifying particle-like responses, our framework better suppresses structured low-frequency background, which improves particle--background separability for downstream analysis. Experiments on multiple cryo-EM datasets show that our score-based methods consistently improve downstream particle picking and produce more structure-consistent 3D reconstructions. Experiments on multiple cryo-EM datasets show that our methods improve downstream particle picking and produce more structure-consistent reconstructions.

Xiaoqi Wu, Xueying Zhan, Wen Li +3

Carnegie Mellon University · Towson University

Cryo-Electron Microscopy Score-Based Generative Modeling Image Denoising Electron Microscopy

Apr 7, 2026

A Mixture of Experts Foundation Model for Scanning Electron Microscopy Image Analysis

Scanning Electron Microscopy (SEM) is indispensable in modern materials science, enabling high-resolution imaging across a wide range of structural, chemical, and functional investigations. However, SEM imaging remains constrained by task-specific models and labor-intensive acquisition processes that limit its scalability across diverse applications. Here, we introduce the first foundation model for SEM images, pretrained on a large corpus of multi-instrument, multi-condition scientific micrographs, enabling generalization across diverse material systems and imaging conditions. Leveraging a self-supervised transformer architecture, our model learns rich and transferable representations that can be fine-tuned or adapted to a wide range of downstream tasks. As a compelling demonstration, we focus on defocus-to-focus image translation-an essential yet underexplored challenge in automated microscopy pipelines. Our method not only restores focused detail from defocused inputs without paired supervision but also outperforms state-of-the-art techniques across multiple evaluation metrics. This work lays the groundwork for a new class of adaptable SEM models, accelerating materials discovery by bridging foundational representation learning with real-world imaging needs.

Sk Miraj Ahmed, Yuewei Lin, Chuntian Cao +9

Computing and Data Sciences, Brookhaven National Laboratory, Upton, 11973, New York, USA. · Center for Functional Nanomaterials, Brookhaven National Laboratory, Upton, 11973, New York, USA. · Department of Material Sciences and Engineering, The University of Texas at Dallas, Richardson, 75080, Texas, USA.

Microscopy Image Analysis Electron Microscopy Self-Supervised Visual Representation Learning Unpaired Image-to-Image Translation

Apr 5, 2026

PATHFINDER: Multi-objective discovery in structural and spectral spaces

Automated decision-making is becoming key for automated characterization including electron and scanning probe microscopies and nano indentation. Most machine learning driven workflows optimize a single predefined objective and tend to converge prematurely on familiar responses, overlooking rare but scientifically important states. More broadly, the challenge is not only where to measure next, but how to coordinate exploration across structural, spectral, and measurement spaces under finite experimental budgets while balancing target-driven optimization with novelty discovery. Here we introduce PATHFINDER, a framework for autonomous microscopy that combines novelty driven exploration with optimization, helping the system discover more diverse and useful representations across structural, spectral, and measurement spaces. By combining latent space representations of local structure, surrogate modeling of functional response, and Pareto-based acquisition, the framework selects measurements that balance novelty discovery in feature and object space and are informative and experimentally actionable. Benchmarked on pre acquired STEM EELS data and realized experimentally in scanning probe microscopy of ferroelectric materials, this approach expands the accessible structure property landscape and avoids collapse onto a single apparent optimum. These results point to a new mode of autonomous microscopy that is not only optimization-driven, but also discovery-oriented, broad in its search, and responsive to human guidance.

Kamyar Barakati, Boris N. Slautin, Utkarsh Pratiush +2

Department of Materials Science and Engineering, University of Tennessee, Knoxville, TN, USA · Department of Materials Science and Engineering, Institute of Science Tokyo, Yokohama, Japan

Autonomous Experimentation Multi-Objective Optimization Electron Microscopy Novelty Search

Apr 4, 2026

SAGE-GAN: Towards Realistic and Robust Segmentation of Spatially Ordered Nanoparticles via Attention-Guided GANs

Precise analysis of nanoparticles for characterization in electron microscopy images is essential for advancing nanomaterial development. Yet it remains challenging due to the time-consuming nature of manual methods and the shortcomings of traditional automated segmentation techniques, especially when dealing with complex shapes and imaging artifacts. While conventional methods yield promising results, they depend on a large volume of labeled training data, which is both difficult to acquire and highly time-consuming to generate. In order to overcome these challenges, we have developed a two-step solution: Firstly, our system learns to segment the key features of nanoparticles from a dataset of real images using a self-attention driven U-Net architecture that focuses on important physical and morphological details while ignoring background features and noise. Secondly, this trained Attention U-Net is embedded in a cycle-consistent generative adversarial network (CycleGAN) framework, inspired by the cGAN-Seg model introduced by Abzargar et al. This integration allows for the creation of highly realistic synthetic electron microscopy image-mask pairs that naturally reflect the structural patterns learned by the Attention U-Net. Consequently, the model can accurately detect features in a diverse array of real-world nanoparticle images and autonomously augment the training dataset without requiring human input. Cycle consistency enforces a direct correspondence between synthetic images and ground-truth masks, ensuring realistic features, which is crucial for accurate segmentation training.

Anindya Pal, Varun Ajith, Saumik Bhattacharya +1

National Institute of Technology Durgapur, Department of Physics · Indian Institute of Technology Kharagpur

Electron Microscopy Image Segmentation Microscopy Image Analysis Synthetic Data Augmentation Convolutional Neural Networks

Mar 19, 2026

Statistical Characteristic-Guided Denoising for Rapid High-Resolution Transmission Electron Microscopy Imaging

High-Resolution Transmission Electron Microscopy (HRTEM) enables atomic-scale observation of nucleation dynamics, which boosts the studies of advanced solid materials. Nonetheless, due to the millisecond-scale rapid change of nucleation, it requires short-exposure rapid imaging, leading to severe noise that obscures atomic positions. In this work, we propose a statistical characteristic-guided denoising network, which utilizes statistical characteristics to guide the denoising process in both spatial and frequency domains. In the spatial domain, we present spatial deviation-guided weighting to select appropriate convolution operations for each spatial position based on deviation characteristic. In the frequency domain, we present frequency band-guided weighting to enhance signals and suppress noise based on band characteristics. We also develop an HRTEM-specific noise calibration method and generate a dataset with disordered structures and realistic HRTEM image noises. It can ensure the denoising performance of models on real images for nucleation observation. Experiments on synthetic and real data show our method outperforms the state-of-the-art methods in HRTEM image denoising, with effectiveness in the localization downstream task. Code will be available at https://github.com/HeasonLee/SCGN.

Hesong Li, Ziqi Wu, Ruiwen Shao +1

Beijing Institute of Technology

Scanning Transmission Electron Microscopy Image Denoising Microscopy Image Analysis Electron Microscopy

Mar 17, 2026

Novelty-Driven Target-Space Discovery in Automated Electron and Scanning Probe Microscopy

Modern automated microscopy faces a fundamental discovery challenge: in many systems, the most important scientific information does not reside in the immediately visible image features, but in the target space of sequentially acquired spectra or functional responses, making it essential to develop strategies that can actively search for new behaviors rather than simply optimize known objectives. Here, we developed a deep-kernel-learning BEACON framework that is explicitly designed to guide discovery in the target space by learning structure-property relationships during the experiment and using that evolving model to seek diverse response regimes. We first established the method through demonstration workflows built on pre-acquired ground-truth datasets, which enabled direct benchmarking against classical acquisition strategies and allowed us to define a set of monitoring functions for comparing exploration quality, target-space coverage, and surrogate-model behavior in a transparent and reproducible manner. This benchmarking framework provides a practical basis for evaluating discovery-driven algorithms, not just optimization performance. We then operationalized and deployed the workflow on STEM, showing that the approach can transition from offline validation to real experimental implementation. To support adoption and extension by the broader community, the associated notebooks are available, allowing users to reproduce the workflows, test the benchmarks, and adapt the method to their own instruments and datasets.

Utkarsh Pratiush, Kamyar Barakati, Boris N. Slautin +4

Department of Materials Science and Engineering, University of Tennessee, Knoxville, TN 37996, USA · Department of Chemistry, University of Washington, Seattle, WA 98195, USA · Pacific Northwest National Laboratory, Richland, WA 99354, USA

Deep Kernel Learning Scanning Transmission Electron Microscopy Autonomous Experimentation Electron Microscopy Microscopy Image Analysis