physics.ins-detAug 13, 2026

An Interactive, Automated 4D-STEM data acquisition and analysis routine for Scanning Electron Nanobeam Diffraction and Ptychography experiments

Authors: Mohsen Danaie, Max England, Yiming Xu, Ruomu Zhang, Ed Darnbrough, Josh Willem De Boer, Frederick Allars, Zaeem Najeeb, +6 more

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Modern transmission electron microscopes are versatile instruments which have become indispensable tools for understanding structure and chemical composition at the nano- and atomic scale. In the physical sciences these instruments are still largely manually controlled, requiring significant operator expertise, limiting throughput, and precluding statistical analysis of large datasets. Recent technical advances in both hardware and in control software now allow for the interaction with almost every functionality of the microscope through a programming interface. This enables better experimental design and data collection automation while also reducing operator collection bias and required expertise. In this study, we present an automated data collection routine with machine-driven decision-making to enable the collection of hundreds of 4D-STEM nanobeam diffraction and ptychography data from a large distribution of size-selectively deposited Pt nanoparticles. We present a semi-automated data analysis workflow to extract pertinent information from the large volumes of collected data. For the nanobeam diffraction data, reducing each dataset to its azimuthal variance profile and combining automated crystal orientation mapping with per-particle morphology descriptors reveals the orientation, shape and phase distributions across the ensemble, including a weak {110} texture. For the ptychographic data, an automated screening pipeline identifies on-zone-axis particles and enables atomic-resolution phase imaging and lattice-strain mapping of individual grains. Together these demonstrate how automation turns instrument throughput into statistically meaningful, atomic-scale microstructural information.

Figures & tables

Figure 1: Hardware components and their interactions to enable automated data acquisition. The automated data collection GUI communicates with three main hardware components: the JEOL ARM300F microscope (via PyJEM ), the Quantum Detectors Scan Engine (via pyscanengine ), and the Quantum Detectors MerlinEM pixellated detector (via TCP/IP commands). Solid lines represent software communication with each component. The scan engine receives the STEM signal from the microscope and delivers a TTL trigger signal to the MerlinEM detector to synchronise frame capture with the scan.
Figure 2: Flowchart of the automated data collection software workflow. Pre-defined parameters (stage positions, scan and correction parameters, analysis settings) are set by the user prior to acquisition. A survey image is acquired using ADF or BF STEM (sparse 4D-STEM is also in development), and analysed via thresholding, shape detection, or user-defined ROI selection to define the 4D-STEM scan regions. An optional stage movement step (in development) enables stitching across larger areas. Within the acquisition loop, 4D-STEM scans are interleaved with drift and focus corrections. The collected data feed directly into an automated analysis pipeline.
Figure 3: Screenshots of the GUI at successive stages of automated data collection for a Pt nanoparticle sample: A survey HAADF-STEM image, B binarised image after intensity thresholding to isolate nanoparticles, C DBSCAN-clustered binary image with individual particle regions highlighted in red, D 4D-STEM acquisition grid populated over the segmented particle regions with user-defined overlap.
Figure 4: NBED dataset overview and diffraction analysis. A Survey HAADF-STEM image with all 153 numbered 4D-STEM acquisition regions overlaid as red boxes. B Gallery of segmentation masks for the first 36 nanoparticles, showing ADF signal (blue-green) and the automated particle mask (overlaid in red). C Azimuthal variance 1D diffraction profile summed over all valid datasets (red), overlaid with reference Pt peak positions (black bars). Arrows mark additional peaks at 5.52 Å and 4.78 Å d-spacings, attributed to an oxide phase.
Figure 5: Automated crystal orientation mapping (ACOM) and morphology analysis from 153 NBED datasets. A Inverse pole figure (IPF) scatter plot of all particle orientations coloured by dataset acquisition index (left) and the corresponding pooled pole density function (right), revealing a weak {110} texture. B Per-particle IPF bubble chart: position encodes mean crystallographic orientation, marker size is proportional to particle area, and colour encodes eccentricity (0 = circular, 1 = rod-like). C Violin plots of particle area (left) and eccentricity (right) grouped by the nearest low-index pole ({100}, {110}, {111}); violin width is proportional to grain count in each group ( nn shown above each violin). Note that some particles contained multiple crystallographic grains so the total number of grains analysed exceeds the number of particles detected.
Figure 6: Ptychographic phase reconstructions of Pt nanoparticles from automated data collection, showing the first 30 datasets sorted by acquisition time. Acquisition timestamps are shown above each panel. Scale bars are shown in each panel. The variation in particle morphology, size, and number of visible nanoparticles per acquisition reflects the diversity of the sample region sampled by the automated routine.
Figure 7: Atomic-resolution ptychographic analysis of Pt nanoparticles. (A) Fast Fourier transform (FFT) of the ptychographic reconstruction showing {111} and {200} Bragg reflections, confirming the [110] zone-axis orientation. (B)–(D) Multi-slice ptychographic phase reconstructions from three different nanoparticles, showing resolved atomic columns. Scale bars: 2 nm. (E) Atomic-resolution phase image a nanoparticle, with fitted atomic column positions coloured by the local radial lattice strain (%). Red arrows indicate grain boundary regions.
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Complete list of metadata items captured with each automated 4D-STEM dataset, covering microscope state, scan parameters, detector settings, stage coordinates, and the images used for automated data collection decisions.
Figure 9: Segmentation masks for all 153 automatically acquired NBED datasets, numbered by acquisition order. Each panel shows the ADF signal (blue-green) overlaid with the automated particle mask, illustrating the range of nanoparticle sizes, shapes, and the occasional presence of multiple particles per acquisition.
Figure 10: Particle morphology scatter plots from NBED analysis. Left : particle area (nm 2 ) vs. aspect ratio, with colour encoding aspect ratio magnitude. Right : particle area vs. eccentricity (0 = circular, 1 = rod-like). The majority of particles fall in the 100–500 nm 2 area range with aspect ratios between 1 and 2.5.
Figure 11: Overview of NBED 1D diffraction profiles for all 150 datasets. Left : survey image with numbered acquisition boxes and stacked bar chart of dominant scattering angle per dataset. Right : gallery of 1D azimuthal variance profiles for all 150 particles, coloured by peak scattering angle, enabling rapid identification of datasets with anomalous or oxide-phase diffraction signal.
Figure 12: Spatial maps of the dominant scattering vector for all 150 NBED datasets. Each panel shows, for every pixel of the segmented particle, the scattering angle at which the azimuthal variance profile peaks, colour-coded as in Figure 11 . Datasets are ordered by acquisition sequence.
Figure 13: Summed mean diffraction signal from all the NP’s, overlaid with ideal fcc Pt powder peak positions and intensities. Table on the right compares the observed peak intensities with the untextured pattern.
Figure 14: Comparison of conventional HAADF-STEM ( left ) and multi-slice ptychographic phase reconstruction ( right ) of the same Pt nanoparticle cluster. The ptychographic reconstruction (40 nm defocus, 4 MX camera length) reveals significantly improved phase contrast and signal-to-noise compared with the incoherent HAADF image. Inset in the right panel shows the reconstructed probe amplitude. Scale bars: 10 nm. The same sampling conditions were matched in the automated routine.
Figure 15: GUI screenshots for the ptychography automated data collection session. A Survey HAADF-STEM image of the Pt nanoparticle sample. B Thresholded image with a user-defined circular ROI (red dashed circle) restricting data collection to the central field of view; the cropped binary image is shown to the right. C DBSCAN-clustered image with individual particle clusters in red. D Populated 4D-STEM acquisition grid overlaid on the segmented regions. Region showed in the upper right is used for focus and drift correction.
Figure 16: Estimated defocus of the probe for each dataset using the parallax algorithm.
Figure 17: Ptychographic phase reconstructions for all automatically acquired datasets, sorted by acquisition time (earliest top-left). Timestamps are shown above each panel. A shared phase shift colour scale (rad) is shown on the right. Reconstructions collected later in the session show progressively reduced quality, consistent with gradual build-up of probe aberrations over the course of the multi-hour automated acquisition.
Figure 18: Reconstructed probe intensity ( ∣probe∣2|\text{probe}|^{2} , incoherent sum of all modes) for all ptychographic datasets, sorted by acquisition time. The estimated probe FWHM (nm) is annotated below each timestamp. Probe intensity and size remain relatively stable during the early acquisitions and degrade progressively over time, providing a direct measure of beam quality evolution during the automated session. A shared colour scale (probe intensity, a.u.) is shown on the right.

Explore similar work

CardsList
Sep 22, 2026

Real-Time Atomic-Resolution Electron Phase Imaging without Probe Calibration via Ptychography-Supervised Learning

Atomic-scale phase imaging is central to resolving defects, interfaces, and weakly scattering atoms that govern the behavior of nanoscale materials. Electron ptychography delivers sub-ångström phase sensitivity but remains an offline technique, because its iterative reconstruction is computationally expensive and sensitive to experimental calibration, preventing live use during data acquisition. Here, a ptychography-supervised local inference framework is presented that converts four-dimensional scanning transmission electron microscopy (4D-STEM) into an acquisition-compatible phase-imaging workflow. Physics-constrained reference phase maps reconstructed from a single experimental AuPd dataset serve as teacher labels for a compact model that predicts local phase patches directly from diffraction measurements, without explicit probe input or online iterative optimization. Full-field images are assembled by deterministic overlap stitching. The workflow reaches an online latency of about 0.27 ms per probe position and a throughput of about 20,000 positions per second, an approximately 1,000-fold speed-up over GPU-accelerated ePIE, while preserving atomic-scale lattice contrast and reciprocal-space fidelity. Without fine-tuning, the same model transfers across materials (WS2), defocus conditions (high-entropy alloy nanoparticles), and instruments (hBN at 300 kV). The approach amortizes ptychographic redundancy into a fast, generalizable workflow that enables real-time atomic-scale phase imaging for materials microscopy.

H. Yue, C. -C. Chen, C. -N. Hsiao +4

College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai 200433, China. · Department of Engineering and System Science, National Tsing Hua University, Hsinchu 300044, Taiwan · National Center for Instrumentation Research, National Institutes of Applied Research, Hsinchu 300092, Taiwan +3

Computational ImagingImage Reconstruction
AskPDF
Jul 18, 2026

Mapping Order in Semicrystalline Polymers using Machine Learning of Nanobeam Electron Diffraction

Organic mixed ionic electronic conductors (OMIECs) are a promising class of polymer materials for applications spanning neuromorphic computation to energy efficient electronics and bioelectronics. Despite being highly tunable, the relationship between structural features and key performance properties such as charge carrier mobility is poorly understood. Scanning nanodiffraction in the transmission electron microscope (TEM) is a powerful probe for elucidating this structure-property relationship, but produces large, noisy datasets that are difficult to interpret because polymer reflections exhibit several distinct morphologies. To address the complexity, we trained a machine learning (ML) model to detect these polymer diffraction peaks and their intensities from synthetic data. Compared to correlative peak detection algorithms, the conventional method for analyzing nanobeam 4D scanning transmission electron microscopy (4DSTEM) data, we show that the ML model is significantly faster and outperforms correlative algorithms in almost all cases, opening up the possibility of near-live visualization of 4DSTEM experiments.

Nicholas Marchese, Arthur R. C. McCray, Yael Tsarfati +4

Department of Materials Science and Engineering, Stanford University, 496 Lomita Mall, Stanford, CA 94305, USA · SLAC National Accelerator Laboratory, 2575 Sand Hill Road, Menlo Park, CA 94025, USA · National Center for Electron Microscopy, Molecular Foundry, Lawrence Berkeley National Laboratory, 1 Cyclotron Road, Berkeley, CA 94720, USA

Materials Science
AskPDF
Jun 28, 2026

STEMGym: Benchmarking Sequential Decision-Making under Dose Budgets in Autonomous Electron Microscopy

A central premise of autonomous scientific imaging is that smarter navigation, whether Bayesian, RL-based, or otherwise adaptive, is the principal lever for sample-efficient acquisition. We present evidence to the contrary in scanning transmission electron microscopy (STEM), an atomic-resolution imaging modality whose every measurement deposits damaging electron dose. We introduce STEMGym, an open-source Gymnasium benchmark of 15 physics-simulated STEM worlds spanning five materials, three difficulty levels, and four characterisation tasks, scored by the Dose-Efficiency Curve area (DEC-AUC), a single scalar capturing the information-vs-dose Pareto frontier. Across 33 agent configurations under realistic dose budgets, the dominant determinant of dose efficiency is the analyst (perception) pipeline, not the navigator: pairing a trained CNN analyst with naïve raster scanning raises DEC-AUC by 5.5x over a CNN-free raster baseline (0.287 vs.\ 0.052), while substituting Bayesian or adaptive finite-state-machine navigation for raster yields no statistically significant further gain. Production-tier vision-language models further underperform task-specific CNNs by {\sim}13x on crystallographic defect analysis. By decoupling perception, navigation, and planning under a unified dose budget, STEMGym reframes where ML effort should be invested in autonomous electron microscopy and provides the measurement infrastructure to test it.

Can Polat, Erchin Serpedin, Mustafa Kurban +1

Dept. of Electrical & Computer Eng. Texas A&M Univ. College Station, TX, USA · Dept. of Prosthetics & Orthotics College of Science & Eng. Ankara Univ., Ankara, Turkey · Hamad Bin Khalifa Univ. and Texas A&M Univ. at Qatar Doha, Qatar

Autonomous ExperimentationMicroscopy Image Analysis
AskPDF