Uncertainty-Aware Continual Learning for 2D XRD Interpretation Using Simulated and Experimental Data
Authors: Zihan Wang, Hyun Sang Park, Ali Rachidi, David Elbert, Todd C. Hufnagel, Wei Chen
Organizations: Department of Mechanical Engineering, Northwestern University, 2145 Sheridan Road, Evanston, Illinois 60076, USA. · Department of Materials Science and Engineering, Johns Hopkins University, 3400 N. Charles Street, Baltimore, Maryland 21218, USA. · Hopkins Extreme Materials Institute, Johns Hopkins University, 3400 N. Charles Street, Baltimore, Maryland 21218, USA.
Autonomous experimentation systems require characterization models that can interpret heterogeneous measurements and maintain predictive performance as data accumulate. Developing such models involves addressing domain mismatch between data sources, overconfident predictions on unfamiliar inputs, and catastrophic forgetting during sequential updates. In this work, we present a proof-of-concept framework for uncertainty-aware continual interpretation of two-dimensional X-ray diffraction (XRD) data. The framework integrates established components: maximum mean discrepancy-based domain adaptation to reduce simulation-to-experiment discrepancies, a convolutional neural network with a spectral-normalized neural Gaussian process (CNN--SNGP) for predictive uncertainty estimation, and replay with knowledge distillation for sequential model updating. We evaluate the framework using simulated and previously-collected experimental data for crystal structure classification of single-phase body-centered cubic, face-centered cubic, and hexagonal close-packed patterns. Data are organized into predefined sequential blocks to assess adaptation and retention of previously learned knowledge. Within this setting, the results show accurate classification across simulated and experimental data, improved alignment between the two domains, and reduced forgetting compared with naïve fine-tuning. Predictive uncertainty is also evaluated on unfamiliar and noisy inputs. The contribution is the integration and evaluation of these capabilities for sequential XRD interpretation. The framework provides a basis for future integration into autonomous experimentation systems, with experimental planning, instrument control, and closed-loop deployment remaining areas for future development.
Figures & tables
Fig. 1: Conceptual framework for uncertainty-aware continual learning from heterogeneous scientific data. The framework integrates predictive uncertainty estimation, domain adaptation, and sequential model updating. This study evaluates an offline implementation using simulated and experimental 2D XRD data; applications to other characterization tasks and integration with experimental control remain future directions.
Crystal structure
Simulated materials
Experimental materials
FCC
Ag, Al, Cu, Ni, Pb, Pt
Al, Cu, Ni
BCC
Mo, Nb, Fe, Ta, V, Zr
Mo, Nb, Fe ( α -Fe)
HCP
Be, Hf, Mg, Sc, Ti, Zn
Ti, Zn
4H single crystal
–
SiC
4H polycrystalline
–
SiC
Table 1: Simulated and experimental materials used in this study
Fig. 2: Inner level: Uncertainty-aware simulation-to-experiment learning. The inner level employs SNGP for uncertainty-aware classification and aligns simulated and experimental XRD feature distributions using maximum mean discrepancy. Here, Conv. denotes convolution, while BCC, FCC, and HCP denote body-centered cubic, face-centered cubic, and hexagonal close-packed, respectively.
381,024→2,048 ; linear, without spectral normalization
Table 2: Layers and specifications of the CNN–SNGP classifier. CNN denotes convolutional neural network; SNGP denotes spectral-normalized neural Gaussian process; SN–Dense denotes a spectrally normalized dense layer; and ReLU denotes the rectified linear unit activation. The CNN–SNGP–adapt model uses the same architecture with additional domain-adaptation losses.
Fig. 3: Outer-level sequential model updating with replay and distillation. The frozen teacher from the preceding stage and the student process the same replayed inputs. Logit distillation uses Kullback–Leibler divergence to match the student’s temperature-scaled class probabilities to the teacher’s. Feature distillation uses a normalized squared error between their 256-dimensional pre-head features. The student is updated using the combined objective in Eq. (14).
Fig. 4: Sequential organization of simulated and experimental XRD datasets. Each dashed box represents one model-update stage and contains one simulated data block and one experimental data block. The known BCC, FCC, and HCP label space remains fixed across stages, while new material identities and experimental measurements are introduced over time to create sequential distribution shifts. The illustrated sequence represents one block order; three orders are evaluated in the repeated-training comparison.
Model
Train Acc.
Val. Acc.
Sil. Score
Align. Dist.
Sim.
Exp.
Sim.
Exp.
Val.
Val.
CNN
0.9439±0.0483
0.9423±0.0447
0.9431±0.0398
0.9414±0.0480
0.215±0.059
12.696±0.614
CNN–SNGP
0.9464±0.0443
0.9454±0.0473
0.9409±0.0456
0.9474±0.0456
0.533±0.007
6.838±0.641
CNN–SNGP–adapt
0.9472±0.0455
0.9569±0.0385
0.9518±0.0341
0.9569±0.0417
0.664±0.078
3.113±1.203
Table 3: Training and validation performance and latent-space organization on the D(1) – D(3) subsets. Train Acc. and Val. Acc. denote training and validation accuracy, respectively, reported as proportions. Sil. Score and Align. Dist. denote the silhouette score and simulation-to-experiment centroid alignment distance, both calculated from validation pre-head embeddings. Values are mean ± sample standard deviation across three repeats. Higher accuracy and silhouette scores and lower alignment distances indicate better performance. Sim. and Exp. denote simulated and experimental data, respectively.
Fig. 5: t-SNE visualization of latent spaces learned by different models. (a) A conventional CNN produces a weakly organized embedding with class overlap, (b) CNN–SNGP improves class separation but retains domain-specific separation between simulated and experimental samples, and (c) CNN–SNGP-adapt yields compact, class-consistent clusters with improved simulation-to-experiment alignment.
(A) Material-held-out classification
Model
Simulated
Experimental
Acc. (%)
Bal. Acc. (%)
F1
Acc. (%)
Bal. Acc. (%)
F1
CNN
59.1±11.4
56.4±11.3
0.541±0.135
29.1±10.2
28.8±8.7
0.290±0.069
CNN–SNGP
64.9±9.2
63.0±9.4
0.621±0.105
44.3±4.9
42.6±6.5
0.421±0.059
CNN–SNGP–adapt
61.6±11.9
58.4±13.3
0.562±0.170
50.3±6.9
48.2±6.6
0.499±0.067
Table 4: Classification and predictive-probability quality on material-held-out data. Within each repeat, predictions are pooled across the eight material holdouts before calculating metrics. Values are mean ± sample standard deviation across the three repeat-level estimates. Acc., Bal. Acc., and F1 denote accuracy, balanced accuracy, and macro-F1. NLL denotes negative log-likelihood, Brier denotes the multiclass Brier score, and ECE denotes expected calibration error. Higher classification metrics and lower NLL, Brier, and ECE indicate better performance. CNN denotes convolutional neural network, SNGP denotes spectral-normalized neural Gaussian process, and adapt denotes the domain-adapted pipeline.
Input
Model
AUROC
AUPR
FPR95 (%)
OOD false acceptance (%)
Noise
CNN
0.721±0.098
0.512±0.274
37.320±4.221
100.000±0.000
Noise
CNN–SNGP
0.998±0.001
0.989±0.006
0.816±0.019
0.000±0.000
Noise
CNN–SNGP–adapt
1.000±0.000
1.000±0.000
0.000±0.000
0.000±0.000
SiC
CNN
0.749±0.103
0.541±0.202
32.165±6.332
100.000±0.000
SiC
CNN–SNGP
0.997±0.001
0.992±0.003
0.712±0.114
0.000±0.000
SiC
CNN–SNGP–adapt
1.000±0.000
1.000±0.000
0.000±0.000
0.000±0.000
Table 5: Detection performance for noise-only and silicon carbide (SiC) inputs. Out-of-distribution (OOD) inputs are treated as the positive class. AUROC and AUPR denote the areas under the receiver operating characteristic and precision–recall curves. FPR95 is the false-positive rate on known-class reference inputs at an operating point attaining at least 95% OOD detection. OOD false acceptance is the fraction of OOD inputs accepted at the evaluated rejection operating point. Higher AUROC and AUPR and lower error rates indicate better performance. Values are mean ± standard deviation across repeated runs. CNN denotes convolutional neural network, SNGP denotes spectral-normalized neural Gaussian process, and adapt denotes the domain-adapted pipeline.
Fig. 6: Pre-head latent-space evolution during sequential learning. Rows show (a) joint retraining, (b) naïve fine-tuning, (c) replay-only updating, (d) learning-without-forgetting (LwF)-style distillation, and (e) the proposed continual-learning strategy; columns show stages 1–4. At each stage, all methods are visualized using the same ordered set of accumulated training, validation, and test scans. Blue, orange, and green indicate body-centered cubic (BCC), face-centered cubic (FCC), and hexagonal close-packed (HCP) structures, respectively. Larger black-edged circles denote simulated scans, and small dots denote experimental scans. Annotations identify selected material subsets and the earlier blocks in which they appeared. Each t-SNE panel is fitted independently; absolute positions and distances should not be compared across panels.
Fig. 7: Stage-wise test accuracy under five sequential training strategies. Panels show (a) joint retraining, (b) naïve fine-tuning, (c) replay-only updating, (d) learning-without-forgetting (LwF)-style distillation, and (e) the proposed continual-learning strategy. Rows indicate model-update stages, and columns indicate introduced-block positions (P1–P4) within each evaluated sequence. Block identities at these positions depend on the introduction order. Each cell displays the mean test accuracy, with the standard deviation beneath it. Future-block cells are blank. Accuracies are proportions rounded to two decimals, and all panels use a common 0–1 color scale.
Strategy
Sim. Acc. (%)
Exp. Acc. (%)
Sraw↑
Dalign,std↓
Training time (h)
Joint retraining
99.88±0.22
99.44±0.97
0.904±0.020
1.156±0.078
8.49±1.99
Naïve fine-tuning
95.88±3.20
94.23±3.05
0.408±0.116
6.068±1.013
2.32±0.27
Replay-only
99.75±0.43
97.47±2.19
0.616±0.190
4.165±1.754
4.69±0.37
LwF-style distillation
93.51±2.49
89.31±0.24
0.353±0.049
5.787±0.834
1.63±0.24
Proposed CL
99.63±0.37
98.31±1.93
0.761±0.066
2.788±0.594
4.36±0.65
Table 6: Final-stage classification performance, latent-space diagnostics, and cumulative training time for five primary strategies. Results average orders within each seed and report mean ± sample standard deviation across three seeds. Accuracy pools test scans from all introduced blocks. Sim. and Exp. denote simulated and experimental data. Sraw denotes the raw validation silhouette score; Dalign,std denotes the standardized source–target class-centroid gap. Arrows indicate the preferred direction. CL denotes continual learning. Training time includes the complete sequence, validation, GP precision reconstruction, and retries, excluding test exports and separate plotting.
Strategy
AA (%)
Forgetting (pp)
BWT (pp)
FWT (pp)
Old (%)
New (%)
Joint retraining
99.56±0.77
−0.15±1.68
0.89±0.77
29.12±23.63
99.41±1.03
100.00±0.00
Naïve fine-tuning
93.11±3.40
4.00±4.00
−3.41±3.83
26.13±20.95
95.85±4.22
84.89±16.88
Replay-only
98.00±1.73
2.52±2.19
−1.63±3.37
20.97±19.57
97.33±2.31
100.00±0.00
LwF-style distillation
88.63±2.65
−0.63±1.67
3.39±3.34
14.55±23.04
90.33±6.20
83.56±14.26
Proposed CL
98.63±1.52
0.84±2.41
−0.55±2.78
22.38±18.72
98.32±2.08
99.56±0.77
Table 7: Final experimental-domain retention and transfer metrics, using the aggregation in Table 6 . Average accuracy (AA) weights blocks equally. Forgetting, backward transfer (BWT), and forward transfer (FWT) are in percentage points (pp). Old and New denote task-average accuracy on earlier blocks and accuracy on the latest introduced block, respectively. Negative signed forgetting indicates improvement beyond the best earlier post-introduction accuracy. BWT references accuracy at introduction; FWT references the untrained model. CL denotes continual learning.
Strategy
NLL
ECE (%)
Joint retraining
0.017±0.025
0.88±1.12
Naïve fine-tuning
0.228±0.103
12.03±3.97
Replay-only
0.100±0.045
6.82±2.15
LwF-style distillation
0.391±0.035
19.38±5.37
Proposed CL
0.088±0.024
6.58±1.67
Table 8: Final experimental predictive-probability quality for five primary strategies using the same CNN–SNGP–adapt backbone. Results use the aggregation in Table 6 . NLL denotes negative log-likelihood, ECE denotes expected calibration error, and CL denotes continual learning. ECE uses 15 equal-width confidence bins; NLL uses natural logarithms. Lower values indicate better performance. Probabilities use the analytical mean-field calculation without post-hoc temperature scaling.
Fig. A1: Resolution-convergence analysis for XRD image resizing. (a) Representative original and resized-restored XRD images. (b) Mean radial peak-position error as a function of the resized image size. Error bars indicate the standard deviation across images.
Dataset
Domain
Materials
Total
Train
Val.
Test/Eval.
Known-class inventory
Simulation
Ag, Al, Be, Cu, Fe, Hf, Mg, Mo, Nb, Ni, Pb, Pt, Sc, Ta, Ti, V, Zn, Zr
900
–
–
–
Known-class inventory
Experiment
Al, Cu, Fe ( α -iron), Mo, Nb, Ni, Ti, Zn
800
–
–
–
Per-fold development
Simulation
Listed simulated materials excluding m
850
680
85
85
Per-fold outer test
Simulation
m : Al, Cu, Fe, Mo, Nb, Ni, Ti, or Zn
50
0
0
50
Per-fold development
Experiment
Al, Cu, Fe, Mo, Nb, Ni, Ti, Zn, excluding m
700
560
70
70
Per-fold outer test
Experiment
m : Al, Cu, Fe, Mo, Nb, Ni, Ti, or Zn
100
0
0
100
Table A1: Material inventory and per-fold dataset roles in the eight-fold leave-one-material-out evaluation. The experimental material set is Mexp={Al,Cu,Fe,Mo,Nb,Ni,Ti,Zn} . Each fold holds out one material m from this set, reserving all its simulated and experimental scans for outer testing. The remaining data in each domain are partitioned into training, validation, and internal test subsets using an 80%/10%/10% split. Experimental training contains 56 labeled anchors (10% of the training subset) and 504 unlabeled scans per fold. Validation data are used for early stopping and model selection; validation and test scans are excluded from anchor selection. The Test/Eval. column reports internal test counts for development rows and independent evaluation counts for outer-test and OOD rows. SiC and noise-only images belong to a separate uncertainty evaluation.
Partition
Domain
Materials
Total
Train
Val.
Test
Dsim(1)
Simulation
Al, Be, Cu, Fe, Hf, Mg, Mo, Pb, Pt, Sc, Ta, Zn, Zr
247
205
20
22
Dexp(1)
Experiment
Al, Fe, Mo, Ti, Zn
233
186
22
25
Dsim(2)
Simulation
Al, Be, Cu, Hf, Mg, Mo, Nb, Pb, Pt, Sc, Ta, Zn, Zr
252
200
26
26
Dexp(2)
Experiment
Al, Mo, Nb, Ti, Zn
233
192
22
19
Dsim(3)
Simulation
Al, Be, Cu, Hf, Mg, Mo, Ni, Pb, Pt, Sc, Ta, Zn, Zr
251
197
28
26
Dexp(3)
Experiment
Al, Mo, Ni, Ti, Zn
234
184
25
25
Table A2: Dataset partitions used in the main domain-incremental sequential-update evaluation. Block identities remain fixed when their introduction order changes. Image-level training, validation, and test roles are preserved. Experimental blocks D1 – D3 share Al, Mo, Ti, and Zn, with Fe, Nb, and Ni occurring in D1 , D2 , and D3 , respectively; experimental D4 contains FCC Cu only. Every simulated block contains all three crystal classes. Recurring materials use distinct image identities but may include correlated measurements of the same specimen.
Element
ICSD record number
Ag
ICSD 64706
Al
ICSD 64700
Cu
ICSD 136042
Ni
ICSD 37502
Pb
ICSD 96501
Pt
ICSD 64919
Table A3: Record numbers for ICSD CIF files, accessed from FIZ Karlsruhe on September 12, 2025
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Type
Material
Structure
Exp. (s)
CL block: Tr/V/Te
CL anchors / hidden
Fixed / grouped roles
Sim.
Ag
FCC
N/A
D4: 42/4/4
All training labels
L: full material holdout (50) G: development only
Sim.
Al
FCC
N/A
D1: 14/2/1 D2: 13/3/0 D3: 14/2/1
All training labels
L Tr/V/Te: 41/7/2 G: held out / development
Sim.
Be
HCP
N/A
D1: 13/1/2 D2: 13/2/2 D3: 13/2/2
All training labels
L Tr/V/Te: 39/5/6 G: development only
Sim.
Cu
FCC
N/A
D1: 14/1/1 D2: 15/1/1 D3: 14/1/2
All training labels
L Tr/V/Te: 43/3/4 G: held out / development
Sim.
Fe
BCC
N/A
D1: 43/1/6
All training labels
L Tr/V/Te: 43/1/6 G: held out / development
Sim.
Hf
HCP
N/A
D1: 13/3/1 D2: 13/2/2 D3: 12/3/1
All training labels
L Tr/V/Te: 38/8/4 G: development only
Appendix
Table A4: Dataset composition and evaluation assignments. CL denotes continual learning; Tr/V/Te gives training/validation/test image counts. The experimental anchors/hidden entries identify the requested overall training-label budget of 10%; they do not give realized counts for each row or block. The remaining training labels are hidden. The fixed and grouped comparisons also have a requested 10% experimental training-label budget. L denotes the fixed D(1) – D(3) comparison, and G denotes the eight-material held-out evaluation with three repeats. “Held out / development” means that a material is the outer test set in its own holdout fold and development data in other folds. Exp. gives exposure time; N/A denotes not applicable.
Fig. A2: Experimental validation accuracy of CNN–SNGP–adapt versus the requested training-anchor fraction. Markers and error bars show the mean and sample standard deviation across three initializations. All configurations retain the same 77 labeled experimental validation scans.
Replay capacity (images)
0
32
64
128
256
Exp. Acc. (%)
88.19±0.73
94.94±7.70
95.78±7.31
97.05±1.46
97.89±0.73
Appendix
Table A5: Sensitivity to replay-memory capacity. Proposed continual learning is evaluated using the reference order D1 – D2 – D3 – D4 . Values are mean ± sample standard deviation across three random seeds. Exp. Acc. denotes final-stage experimental accuracy pooled across test scans from all four blocks. These reference-order results are reported separately from the order-averaged primary comparison. Zero capacity disables replay supervision and replay-based distillation.
Evaluation set
CNN–DKL
CNN–SNGP -adapt
D(1) – D(3) , Sim. train
100.00
100.00
D(1) – D(3) , Exp. train
100.00
100.00
D(1) – D(3) , Sim. test
100.00
100.00
D(1) – D(3) , Exp. test
100.00
100.00
D(4) , Sim. holdout
98.00
98.00
D(4) , Exp. Cu holdout
43.00
61.00
Appendix
Table A6: Classification accuracy (%) of CNN–DKL and CNN–SNGP-adapt. CNN denotes convolutional neural network, DKL denotes deep kernel learning, and SNGP denotes spectral-normalized neural Gaussian process. The suffix “adapt” indicates domain adaptation. Sim. and Exp. denote simulated and experimental data. D(4) is evaluated separately from D(1) – D(3) .
X-ray diffraction (XRD) is a experimental technique for determining the phase composition and structure of crystalline materials. However, interpreting XRD patterns is challenging, particularly in high-throughput materials discovery, where many novel materials may need to be characterized and no reference patterns are available. Consequently, machine learning is increasingly used to accelerate and automate the analysis while reducing errors associated with human interpretation. We propose a multi-decision framework for XRD phase analysis that integrates representation learning, similarity-based retrieval, and explainable decision support within a unified reference database. A convolutional autoencoder learns compact latent representations of XRD patterns that preserve structural similarity while remaining robust to variations arising from experimental noise and measurement conditions. By integrating multiple decision pathways within a shared latent space, the framework moves beyond single-label prediction toward ranked and interpretable phase analysis that mirrors expert practice. During inference, complementary decision mechanisms are applied, including latent-space classification and retrieval, explanation-guided similarity using Integrated Gradients, and composition-based similarity search. These mechanisms generate ranked candidate phase lists that are aggregated into a final prediction with an associated confidence score. Experiments on synthetic datasets demonstrate strong predictive performance, achieving 98.85,% accuracy for crystal system classification and 95.82,% accuracy for space group prediction on the test set, while maintaining robustness under realistic perturbations. The framework supports reliable, analyst-friendly identification of crystal phases and structures in high-throughput and exploratory materials discovery settings.
Doaa Mohamed, Markus Stricker
Materials Informatics and Data Science Interdisciplinary Centre for Advanced Materials Simulation (ICAMS) Ruhr University Bochum, Bochum, Germany
Autonomous crystal-structure determination from powder X-ray diffraction (pXRD) remains a key bottleneck for self-driving materials laboratories. We formulate the task of pXRD structure determination as a verifiable agentic search problem: candidate structures can be assessed during inference through forward-simulated profile agreement and during evaluation through structural matching to a known reference. To benchmark progress, we introduce pXRD-Bench, a suite of 200 simulated single-phase problems spanning in-distribution structure determination, challenging and polymorphic structures, and controlled compositional and topological distribution shifts. Agent adaptation raises selected structural recovery from 27% to 69%, with further gains from repeated sampling and verifier-guided inference-time scaling. We extend this approach to experimental multiphase pXRD through a reflexive evolution framework that learns from self-generated synthetic problems. A replaceable laboratory specification defines the sample family and measurement conditions. An evolver agent uses the known generating structures and measurement parameters to diagnose solver failures and develop persistent guidance and executable tools, which are then provided to agents on measured patterns. This supplies verifiable training experience without requiring a collection of solved experimental measurements. On 15 measured patterns from solid-state synthesis, the evolved package improves average profile agreement and transfers an analogue-substitution procedure that recovers a phase whose structure is absent from the supplied reference library. These results demonstrate how learning from verifiable synthetic experience can improve agentic characterization of experimental samples whose structures are not known in advance.
Samuel Chen, Jiaru Bai, Austin Henry Cheng +2
Stanford University, Stanford, CA, USA · University of Toronto, Toronto, ON, Canada · Vector Institute for Artificial Intelligence, Toronto, ON, Canada +3
Automation underpins progress across scientific and industrial disciplines. Yet, automating tasks requiring interpretation of abstract visual information remain challenging. For example, crystal alignment strongly relies on humans with the ability to comprehend diffraction patterns. Here we introduce an autonomous system that aligns single crystals without access to crystallography and diffraction theory. Using a model-free reinforcement learning framework, an agent learns to identify and navigate towards high-symmetry orientations directly from Laue diffraction patterns. Despite the absence of human supervision, the agent develops human-like strategies to achieve time-efficient alignment across different crystal symmetry classes. With this, we provide a computational framework for intelligent diffractometers. As such, our approach advances the development of automated experimental workflows in materials science.
J. Oppliger, M. Stifter, A. Rüegg +7
Physik-Institut, Universität Zürich, Winterthurerstrasse 190, CH-8057 Zürich, Switzerland · Jeremiah Horrocks Institute for Mathematics, Physics and Astronomy, University of Central Lancashire, Preston PR1 2HE, United Kingdom · Department of Physics, Clarendon Laboratory, University of Oxford, Oxford OX1 RPU, United Kingdom +3