cs.LGApr 13, 2026
SaveAutonomous Diffractometry Enabled by Visual Reinforcement Learning
Organizations: Physik-Institut, Universität Zürich, Winterthurerstrasse 190, CH-8057 Zürich, Switzerland · Jeremiah Horrocks Institute for Mathematics, Physics and Astronomy, University of Central Lancashire, Preston PR1 2HE, United Kingdom · Department of Physics, Clarendon Laboratory, University of Oxford, Oxford OX1 RPU, United Kingdom · State Key Laboratory of Surface Physics and Department of Physics, Fudan University, Shanghai 200433, China ·
Abstract
Automation underpins progress across scientific and industrial disciplines. Yet, automating tasks requiring interpretation of abstract visual information remain challenging. For example, crystal alignment strongly relies on humans with the ability to comprehend diffraction patterns. Here we introduce an autonomous system that aligns single crystals without access to crystallography and diffraction theory. Using a model-free reinforcement learning framework, an agent learns to identify and navigate towards high-symmetry orientations directly from Laue diffraction patterns. Despite the absence of human supervision, the agent develops human-like strategies to achieve time-efficient alignment across different crystal symmetry classes. With this, we provide a computational framework for intelligent diffractometers. As such, our approach advances the development of automated experimental workflows in materials science.
Figures & tables
Figure 1: Schematic of agent-environment interaction for Laue single crystal alignment. The environment consists of an x-ray Laue back-reflection setup yielding a two-dimensional diffraction pattern (state , linear false color scale), here shown for a cubic crystal structure. Given , an agent — described by a deep neural network — is trained to predict an action consisting of two rotation angles around two perpendicular axes, executed by a robotic arm. This results in a reward signal and a new state . The state-action-reward loop ends when a predefined high-symmetry orientation — here chosen as (001) (yellow crosses) — is achieved within an angular tolerance (red circles) after steps.
Figure 2: Agent training curves for different crystal structures. (a-c) Success rate, episode length, and episode reward as a function of percentage training cycles for selected crystal structures (space groups) as indicated. Solid lines represent the mean, while shaded areas indicate the 95% confidence interval calculated from five training runs with different random seeds.
Figure 3: Evaluation of agent performance . (a-c) Stereographic projection along the (001) direction for crystal structures (space groups) as indicated, together with 10000 (3000) agent alignment trajectories for cubic (hexagonal and tetragonal) systems. To illustrate the agent’s behavior, the step size is here limited to two degrees and a false color scale is used to indicate distance to the closest target. Schematics of the respective crystal structure are given in the top right with high-symmetry (001) axis highlighted. Alignment is achieved when this or equivalent axes are – within an angular tolerance – parallel to the x-ray beam. Histograms of number of actions to reach the (001) high-symmetry target (episode length) are shown at the bottom left, obtained by evaluating the agents for 10000 episodes. Insets at the bottom right display examples of the simulated target Laue pattern with angular tolerance (red circles).
Figure 4: Agent evaluation on real experimental data . (a) Representative alignment trajectory obtained on a cubic SrTiO 3 single crystal with space group 221. The top row shows Laue diffraction patterns with examples of high-symmetry lines highlighted in light colors. The patterns are preprocessed and downsampled (lower left images) before being passed as inputs to the agent, which in turn predicts two rotation angles ( ). Executing those angles using a robotic arm results in a momentum space shift as indicated by the yellow arrows. This sequence repeats until an end-of-episode (EOE) flag is triggered, indicating that the crystal has been aligned along a specific high-symmetry direction (yellow star) within an angular distance of five degrees (red circle). (b, c) Example trajectories for hexagonal CsV 3 Sb 5 and tetragonal La 1.5 Sr 0.5 NiO 4 single crystals with space groups 191 and 139 respectively. In both cases, the agent was trained to target the (001) high-symmetry orientation. (d) Histogram comparing performance of simulation and experiment in terms of average episode length. Error bars represent a single standard deviation. (e) Pie charts of percentage of cubic trajectories targeting specific high-symmetry orientations for simulation (top) and experiment (bottom). For simulation and experiment, statistics in (d,e) are derived from 10000 and 50/25/25 trajectories for cubic/hexagonal/tetragonal crystal structures, respectively.
| Parameter | Setting |
|---|---|
| Observation shape | 84 x 84 |
| Frame stack | 1 |
| Action repeat | 1 |
| Augmentation padding | 4 |
| Discount rate | 0.99 |
| Seed frames | 2000 |
Table 1: DrM [ 70 ] hyperparameters for the RL training.
| Layer | Input shape | Output shape |
|---|---|---|
| Conv2D(32, 3, 2, 1) | (1, 84, 84) | (32, 42, 42) |
| Conv2D(32, 3, 2, 1) | (32, 42, 42) | (32, 21, 21) |
| Conv2D(32, 3, 1, 1) | (32, 21, 21) | (32, 21, 21) |
| Conv2D(32, 3, 1, 1) | (32, 21, 21) | (32, 21, 21) |
Table 2: Encoder structure used for extracting features from the Laue patterns. Order of convolutional layer parameters are (filters, kernel size, stride, padding). After each convolutional layer we apply a ReLU activation function to introduce non-linearity.
| Parameter | Fixed | Domain randomization |
|---|---|---|
| Detector-film distance (cm) | 5 | 4 – 6 |
| Number of spots (cubic) | 60 | 30 – 60 |
| Number of spots (hexagonal) | 90 | 40 – 90 |
| Number of spots (tetragonal) | 120 | 60 – 120 |
| Cubic lattice constants (Å) | 9 ( ) | 3 – 15 ( ) |
| Hexa. lattice constants (Å) | 4 ( ), 8 ( ) | 4 – 7 ( ), 8 – 11 ( ) |
Table 3: Simulation parameters. The second column indicates values for a fixed setting while the third column displays value ranges for domain randomization.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Impact of different angular solution tolerances. (a-c) Training curves for a cubic system showing episode reward, episode length, and success rate as a function of percentage training cycles for different angular solution tolerances in degrees as indicated. 100% training progress corresponds to 100000 training steps. Solid lines represent the mean, while shaded areas indicate the 95% confidence interval calculated from five training runs with different random seeds.
Figure S2: Impact of different maximum angular action step sizes . (a-c) Training curves for a cubic system showing episode reward, episode length, and success rate as a function of percentage training cycles for different maximum angular action step sizes in degrees as indicated. 100% training progress corresponds to 100000 training steps. Solid lines represent the mean, while shaded areas indicate the 95% confidence interval calculated from five training runs with different random seeds.
Figure S3: Effect of curriculum learning on training performance . Performance comparison of agents trained on tetragonal (a-d) and hexagonal (e-h) crystal structures with domain randomization. Solid lines in (a-c) and (e-g) represent the mean, while shaded areas indicate the 95% confidence interval calculated from five training runs with different random seeds. The faint vertical lines mark the positions at which the curriculum learning step has occurred. Gray lines represent the non-curriculum learning baseline. (d,h) Histograms of episode lengths for the respective agents evaluated for 10000 episodes.
Figure S4: Supervised high-symmetry identification . (a) Validation categorical cross entropy loss (top) and validation accuracy (bottom) as a function of training epochs of a supervised CNN trained on the cubic crystal structure with space groups 221, 225, and 229. Solid lines represent the mean, while the (barely visible) shaded area the 95% confidence interval obtained using 5-fold cross validation. (b) Multiclass confusion matrix, evaluated on a separate simulated test set. Class 1, 2, and 3 refer to (001), (101), and (111) high-symmetry axes respectively, within a solution tolerance of five degrees. Class 0 encompasses any other (non high-symmetry) orientation. (c) Trained network applied to experimental Laue patterns recorded on a SrTiO 3 single crystal with predicted (true) class labels and probabilities as indicated. A green tick (red cross) mark in the top right indicates a successful (false) prediction.
Figure S5: Procedure for high-accuracy crystal alignment . (a-b) A reinforcement-learning (RL) agent, described in the main text, is aligning a single crystal along a given high-symmetry orientation within steps and a distance tolerance of five degrees (indicated by the red circle). (c) Once a high-symmetry orientation is found, the Hough transform (HT) is used to calculate the angular offsets by selecting the (angle, distance) parameters with highest probabilities in the Hough parameter space, indicated by red arrows in the inset. The high-symmetry lines detected by the HT are highlighted in yellow on top of the Laue pattern. (d) The angular offsets determined in (c) are then used to achieve a high-accuracy alignment after a single step.
Figure S6: Approaches for high-confidence action prediction . (a) Experimental Laue alignment trajectory on a hexagonal CsV 3 Sb 5 single crystal as predicted by a single agent. The yellow arrows indicate the predicted action while the yellow star highlights the target state within the solution tolerance (red circle). (b,c) Trajectory achieved with the same starting pattern (blue border) as in (a) where the action taken for the current image is calculated as the average over predictions of six unique transformations or over predictions of five agents trained with different random seeds. (d) Trajectory achieved using a combination of approaches in (b) and (c).
Figure S7: Single agent trajectories with and without test-time data augmentation (TTA). Top panels (above dashed line) display trajectories of a single agent trained to operate in cubic, hexagonal, or tetragonal reciprocal space with space groups as indicated and evaluated for a fixed orientation. Bottom panels (below dashed line) display the same agents but with decision trajectories obtained through an average of mirror and rotational augmentation of the Laue patterns.
Figure S8: Comparison of different reinforcement-learning algorithms . (a-c) Training curves for a cubic system as a function of percentage training cycles for different algorithms with and without data augmentation. Solid lines represent the mean, while shaded areas indicate the 95% confidence interval calculated from five training runs with different random seeds.
Figure S9: Comparison of different reward functions. (a-c) Training curves for a cubic system as a function of percentage training cycles for dense and sparse reward functions as indicated. Solid lines represent the mean, while shaded areas indicate the 95% confidence interval calculated from five training runs with different random seeds.
Figure S10: Agent generalization to different cubic space groups . (a-c) Laue agent, trained to align a crystal with simple cubic (221) space group along the (001) high-symmetry orientation, evaluated on simple cubic (221), face-centered cubic (255), and body-centered cubic (229) crystal structures for 10000 episodes using stereographic projections. A high performance can be observed for identifying the (001) orientation in space group 221. For space groups 225 and 229 the agent seems to be directed towards the (111)- and (101)-style directions respectively. Examples of high-symmetry Laue patterns are shown at the bottom of each stereographic projection. (d-f) Laue agent, trained to align a crystal with any of the space groups (221, 225, 229) along the (001) high-symmetry orientation, evaluated on space group 221, 255, and 229 cubic crystal structures for 10000 episodes. In this case, the training produced a robust agent for different cubic crystal structures. To better illustrate the agent’s behavior, the step size is limited to two degrees.
Figure S11: Agent generalization to different cubic space groups . (a-c) Training curves for a single seed showing success rate, episode length, and success rate for training an agent on either simple cubic (221) or a mixture of simple cubic, face-centered cubic, and body-centered cubic crystal structures (221,225,229). The training with purely simple cubic crystal structure (mixture model) was performed over 200000 (400000) training steps.
Figure S12: Experimental control setup . (a) Mecademic Meca500 six-axis robot arm, equipped with a MEGP 25E end effector and custom aluminium grippers. The crystal sample is mounted on a goniometer head (highlighted in red and enlarged in (b)). The sample center of rotation is manually positioned in front of the x-ray source before the angular alignment is initiated. The robotic control commands are executed using the Mecademic Python API. (b) Conventional goniometer head (model 1002) from Huber with indicated angles , used to create the initial angular offsets of the sample as described in the Methods section of the main text.
More in this topic
Autonomous Experimentation
Autonomous experimentation uses AI, active learning, and robotics to design, run, and interpret experiments with less human intervention.
See all 148 papers