cond-mat.mtrl-sciOct 20, 2026

Learning the Geometry of Materials Space with Trained Neural Tangent Kernels

Authors: Daniel Kaplan, Gabriel Kotliar, Anirvan M Sengupta

Organizations: Center for Materials Theory, Department of Physics and Astronomy, Rutgers University, Piscataway 08854 New Jersey USA · Department of Physics and Astronomy, Rutgers University, Piscataway 08854 New Jersey USA · Department of Physics, University of Central Florida, Orlando, Florida 32816, USA · NanoScience and Technology Center, University of Central Florida, Orlando, Florida 32816, USA · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Neural networks trained to predict materials properties learn representations whose physical content can be difficult to extract. We use the neural tangent kernel (NTK), evaluated at trained parameters, to recover the geometry learned by a network that predicts superconducting transition temperatures from elemental composition alone. Kernel ridge regression with the trained NTK closely reproduces the network's predictive performance and substantially outperforms regression with the initial kernel. Spectral analysis and multidimensional scaling reveal a geometry dominated by a small number of directions. Clustering and local neighborhood analysis organize compounds into recognizable superconducting families and connect materials with related chemical and correlated-electron properties despite substantial differences in TcT_c. These results establish trained tangent kernels as practical probes of learned materials representations, connecting property prediction with the exploration of materials space.

Figures & tables

Figure 1: Schematic overview of the workflow. Minimal compositional descriptors are used to train a neural network predictor for superconducting properties. The neural tangent kernel (NTK) evaluated at trained parameters induces a task-adaptive similarity geometry on materials space. Dimensional reduction and clustering of this geometry reveal semantically meaningful structure learned through supervised training.
Methodmean ∣Tc−Tc0∣|T_{c}-T_{c}^{0}|rms (Tc−Tc0)(T_{c}-T_{c}^{0})Tˉc\bar{T}_{c}
Neural Network2.11K7.20K17.5K
Kernel Regression (trained)2.24K7.61K–
Kernel Regression (untrained)21.7K31.9K–
Table 1: Comparison between NN results and Kernel Regression: TcT_{c} is the prediction Tc0T_{c}^{0} , being the original data.
Figure 2: Properties and structure of the NTK. (a) The Shepard plot, i.e., embedding distance ∣∣zi−zj∣∣||z_{i}-z_{j}|| vs Euclidean distance of an untrained NTK realization, derived first the first N=1N=1 step of training. No embedding is generated. (b) Eigenvalue spectrum of the train NTK. Approximately two principal eigenvalues can be discerned. (Note the log scale). (c) Kernel Ridge Regression (KRR) properties: rapid convergence of the KRR with respect to the regularization λ\lambda confirms the relative number of useful degrees of freedom (non-null eigenvalues in (b)). (d) Shepard plot of the trained NTK, indicating a significantly different embedding of the data.
Figure 3: Low-dimensional embedding of superconducting materials obtained from classical multidimensional scaling (MDS) applied to trained NTK distances. Colors correspond to clusters identified through unsupervised k-means clustering in the embedded representation space. Several clusters align with known superconducting families, including cuprates, iron-based superconductors, undoped or otherwise superconductors with “simple” chemical formulae, and non-superconducting compounds. As this information is not present during training, we manually assign labels to the generated clusters; each cluster taxonomy is determined by the the first 200 compounds, measured by distance to the centroid.
ClusterMean Silhouette
Mixed 10.175
High Entropy0.415
High TcT_{c}0.450
Iron Based0.350
Mixed 20.200
Mixed 30.140
Table 2: Mean Silhouette Scores per Cluster
Figure 4: Neighborhood structure induced by the trained NTK geometry. Materials are ordered according to radial distance from representative compounds in the learned NTK metric (top: YBa 2 Cu 4 O 8 -based cuprate; bottom: CeCoIn 5 heavy-fermion superconductor). Blue points show NTK distance, while red points indicate the superconducting transition temperature TcT_{c} . Gray dashed lines denote distance gaps used to identify neighborhood shells in the learned geometry. Nearby compounds frequently share chemical motifs, doping structure, or correlated-electron phenomenology despite substantial variation in TcT_{c} , indicating that the trained NTK captures transferable physical structure beyond the original supervised learning task.
Figure 5: Measure of K(x,x)K(x,x) as uncertainty in clustering. We take a cut along a fixed z2z_{2} value z2=−25z_{2}=-25 and average over a window of ±6\pm 6 in that range. We then traverse z1z_{1} . We find a correlation between boundaries of the identified clusters and increases in K(x,x)K(x,x) . In areas of low explainability/silhouette, such corresponding changes in K(x,x)K(x,x) were not observed.
More in this topic

Neural Representation Geometry

Neural representation geometry studies how learned features are arranged and how that structure relates to model behavior and performance.

See all 256 papers

Explore similar work

CardsList

  1. Mathematical Invariant-Enabled Topological Neural Networks for Molecular and Materials Property Prediction

    Oct 6, 2026Yiming Ren, Xiang Liu, Mustafa Hajij +2Materials Property PredictionMolecular Property Prediction

    Department of Mathematics, University of Georgia, Athens, GA 30602, USA. · Department of Data Science, University of San Francisco, CA 94117, USA. · Department of Computer Science and Technology, Cambridge University, Cambridge, United Kingdom. +2

  2. A Function-Space Dichotomy for Compositional Learning: Exponential Sub-Optimality of the Neural Tangent Kernel

    Jul 7, 2026Arkaprabha Ganguli, Emil ConstantinescuNeural Network GeneralizationKernel Regression

    Mathematics & Computer Science Division, Argonne National Laboratory, Lemont, IL, USA

  3. Measuring trainable degrees of freedom in materials graph neural networks: a random-subspace intrinsic dimension analysis

    Sep 28, 2026Shehroz Ahmad Shoaib, Kangming LiIntrinsic DimensionalityGraph Neural Networks

    Physical Science and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia. · Department of Electrical Engineering, King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia.