Modelling magnetic material properties with uncertainty-aware neural networks
Authors: Clemens Wager, Heisam Moustafa, Alexander Kovacs, Qais Ali, Harald Oezelt, Hayate Yamano, Masao Yano, Noritsugu Sakuma, +5 more
Organizations: Christian Doppler Laboratory for magnet design through physics informed machine learning,Jun University for Continuing Education Krems, Wr. Neustadt, 2700, Austria · Department for integrated sensor systems, University for Continuing Education Krems, Wr.10 Neustadt, 2700, Austria · Vienna Doctoral School in Physics, Vienna, Austria · Advanced Materials Engineering Division, Toyota Motor Corporation, Susono, Japan
Machine learning is increasingly applied to accelerate the discovery of novel materials by exploring large compositional and structural design spaces. Yet, the scarcity of high-quality data and the frequent need for out-of-distribution prediction introduce substantial uncertainty, making the assessment of model reliability essential. In this work, we investigate uncertainty quantification as a means to evaluate model confidence in the context of permanent magnet research. In a first study, we benchmark classical and modern machine learning models for predicting intrinsic magnetic properties, focusing on the quality of their uncertainty estimates. We apply Gaussian negative log-likelihood loss and dropout-based Bayesian approximation as practical strategies for estimating predictive uncertainty. In a second study, we transfer these architectural features for uncertainty estimation to a more complex task: predicting coercivity from microstructural information using a graph neural network. Together, these studies demonstrate that uncertainty quantification not only enhances the trustworthiness of predictions but is also transferable across different modeling tasks.
Existing methods for quantifying predictive uncertainty in neural networks are either computationally intractable for large language models or require access to training data that is typically unavailable. We derive a lightweight alternative through two approximations: a first-order Taylor expansion that expresses uncertainty in terms of the gradient of the prediction and the parameter covariance, and an isotropy assumption on the parameter covariance. Together, these yield epistemic uncertainty as the squared gradient norm and aleatoric uncertainty as the Bernoulli variance of the point prediction, from a single forward-backward pass through an unmodified pretrained model. We justify the isotropy assumption by showing that covariance estimates built from non-training data introduce structured distortions that isotropic covariance avoids, and that theoretical results on the spectral properties of large networks support the approximation at scale. Validation against reference Markov Chain Monte Carlo estimates on synthetic problems shows strong correspondence that improves with model size. We then use the estimates to investigate when each uncertainty type carries useful signal for predicting answer correctness in question answering with large language models, revealing a benchmark-dependent divergence: the combined estimate achieves the highest mean AUROC on TruthfulQA, where questions involve genuine conflict between plausible answers, but falls to near chance on TriviaQA's factual recall, suggesting that parameter-level uncertainty captures a fundamentally different signal than self-assessment methods.
The reliable identification of magnetic ground states remains a major challenge in high-throughput materials databases, where density functional theory (DFT) workflows often converge to ferromagnetic (FM) solutions. Here, we partially address this challenge by developing machine-learning classifiers trained on experimentally validated MAGNDATA magnetic materials, leveraging a limited number of simple compositional, structural, and electronic descriptors sourced from the Materials Project Database. Our propagation-vector classifiers achieve accuracies above 92%, outperforming a recent equivariant-neural-network study on a differently constructed dataset in reliably distinguishing between zero and nonzero propagation-vector structures, and exposing a systematic ferromagnetic bias inherent to the Materials Project database for more than 6840 candidate materials. In parallel, LightGBM and XGBoost models trained directly on the Materials Project labels achieve accuracies of 82% and 85%, respectively (with macro-F1 average scores of 66% and 63%), proving useful for large-scale screening for magnetic classes, when refined by MAGNDATA-trained classifiers. These results underscore the role of machine-learning techniques as corrective and exploratory tools, enabling more trustworthy databases and accelerating progress toward the identification of materials with various properties.
Magnetic order is a fundamental property of materials, governing collective behavior and enabling a broad range of functionalities. Yet magnetic structure remains difficult to determine: experiments are costly and specialized, while first-principles methods often struggle with the noncollinear and incommensurate orders found in real materials. Here we introduce magnetic structure network (MSN), an E(3) equivariant graph neural network that predicts both collinear and non-collinear magnetic structures directly from atomic crystal structures, trained directly on experimentally determined structures from MAGNDATA. By proposing the primitive modulated structure representation (PMSR), we are able to encode commensurate and incommensurate structures in a unified way without symmetry assumptions. The model achieves strong performance across all modulation components and reconstructs experimental magnetic structures with high fidelity. Our approach provides a scalable framework for rapid magnetic structure prediction and opens a route to data-driven discovery of magnetic materials.