PYTHIA, a transferable machine learning framework for simultaneous defect and adsorption site optimization
Authors: Ioannis Kouroudis, Zhaosu Gu, Poonam, Leon Muller, Alessio Gagliardi
Organizations: Chair of Simulation of Nanosystems for Energy Conversion, Department of Electrical Engineering, TUM School of Computation, Information and Technology, Atomistic Modeling Center (AMC), Munich Data Science Institute (MDSI), Technical University of Munich, Hans-Piloty-Straße 1, 85748 Garching, Germany · Fritz-Haber-Institut der Max-Planck-Gesellschaft, Faradayweg 4-6, Berlin, Germany
Machine learning has permeated material optimization, providing high value. Nevertheless, its re- liance on a high number of samples limits its applicability to relatively explored material families. In this paper we extend the applicability of machine learning to novel and understudied materials through the PYTHIA (Probabilistic Yet Transferable Hierarchical Interaction Architecture) framework. Pythia utilizes two of the most powerful machine learning concepts, foundational models and transfer learning. We employ a complex foundational model trained on a large dataset of chemically similar molecules, on a target that is physically close to our prediction target. Then, we freeze the weights and perform subspace alignment with the use of a transformer layer and add uncertainty quantification with gaussian processes. These two layers can be trained on a limited dataset yet allow us to leverage the gained knowl- edge and the studied materials and complex architectures. The resulting model is used as a surrogate for a Bayesian Optimization scheme. To showcase the robustness of the framework, we applied it to the investigation of adsorption and defect optimization of two distinct systems. Not only were we able to correctly and efficiently identify the optimal adsorption sites of even complex a
Figures & tables
Figure 1: Pythia Pipeline. The top row corresponds to the predictive part of the framework while the bottom is the permutation of defects and placements, along with the optimization. Both together consitute the total optimization process.
Figure 2: Comparison of Zn 2+ adsorption energy predictions for doped Chini clusters a) a GNN with GP as the final layer, b) a MACE model trained fully with GP as a final layer c) PYTHIA. The points predicted correspond to systems with defect configuration not present in the training set.
Figure 3: Energy landscape (left) and corresponding uncertainty (right) for the chini cluster (Pd 3 , center) that exhibits the best Zn adsorption energy. As can be seen, the model uncertainty is uniformly low, which implies the model has acquired an understanding of the entire energy surface.
Figure 4: Comparison of adsorption energy prediction accuracy for a defective Silicene- CHX2O system for a) a GNN with GP as the final layer, b) a MACE model trained fully with GP as a final layer c) PYTHIA
Figure 5: Energy landscape (left) and corresponding uncertainty (right) for the silicene that exhibits the best adsorption properties (Ga defect). The position of each energy point is that of the first Carbon of formaldehyde. Silicene is periodic so the energy surface is non closed. Further, formaldehyde can freely rotate and therefore each position can have multiple energies associated with it. As can be seen, the model uncertainty is uniformly low, which implies the model has acquired an understanding of the entire energy surface.
Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Jun Kowloon, Hong Kong 999077, China · Department of Chemistry, Hong Kong University of Science and Technology, Kowloon, Hong Kong 999077, China · Laboratory of Artificial Chemical Intelligence (LIAC), EPFL, Lausanne, Switzerland +2
Department of Computer Science, Rice University, Houston, TX, USA · Thayer School of Engineering, Dartmouth College, Hanover, NH, USA · Department of Materials Science and Nanoengineering, Rice University, Houston, TX, USA +2