Surprisingly High Redundancy in Electronic Structure Data Across Materials Explained by Low Intrinsic Dimensionality

Sazzad Hossain; Ponkrshnan Thiagarajan; Shashank Pathrudkar; Stephanie Taylor; Abhijeet S. Gangan; Amartya S. Banerjee; Susanta Ghosh

cond-mat.mtrl-sciJul 11, 2025

Surprisingly High Redundancy in Electronic Structure Data Across Materials Explained by Low Intrinsic Dimensionality

Authors: Sazzad Hossain, Ponkrshnan Thiagarajan, Shashank Pathrudkar, Stephanie Taylor, Abhijeet S. Gangan, Amartya S. Banerjee, Susanta Ghosh

Organizations: Department of Mechanical and Aerospace Engineering, Michigan Technological University, Houghton, MI, USA. · Hopkins Extreme Materials Institute, Johns Hopkins University, Baltimore, MD, USA. · Department of Materials Science and Engineering, University of California, Los Angeles, CA, USA. · Center for Artificial Intelligence, Michigan Technological University, Houghton, MI, USA.

Abstract

Machine learning (ML) models for electronic structure typically rely on large datasets generated by computationally expensive Kohn-Sham density functional theory calculations, as it is not known a priori which portions of the data are essential for accurate learning. Here, we reveal significant redundancies in electronic structure datasets across diverse material systems and attribute them to the low intrinsic dimensionality of the underlying data. We show that even random pruning can substantially reduce dataset size with minimal degradation in predictive accuracy. Moreover, a state-of-the-art coverage-based pruning strategy that samples data across all learning difficulties almost always preserves chemical accuracy and maintains model generalizability while using up to two orders of magnitude less data and reducing training time by a factor of three or more. We further demonstrate that the essential electronic structure information lies on a low-dimensional, non-linear manifold, providing a potential geometric explanation for the observed prunability. These observations are consistent with the predominance of local atomic environments in determining electronic properties, as suggested by nearsightedness arguments, and indicate that large-scale datasets may contain highly overlapping information. Our findings challenge the prevailing assumption that such extensive datasets are necessary for accurate ML-based electronic structure predictions and open a path toward identifying minimal, representative datasets for each material class.

Figures & tables

Figure 1 : Error in ML-predicted electron density for the original dataset, 90% and 99% randomly, and CCS-based pruned datasets. Each ML model was trained three times and the mean error is reported.

Figure 2 : Error in energy with respect to KS-DFT, as obtained from the ML-predicted electron density for the original dataset, 90% CCS, and 99% CCS-based pruned datasets. The electron density prediction from one of the ML models was postprocessed in each case.

Figure 3 : Two-dimensional (2D) slices showing electron density obtained by KS-DFT, ML model trained on the original dataset, 90% CCS, and 99% CCS-based pruned dataset for (a) Aluminum at 1500K (b) Water at 600K (c) SiGeSn at 2400K (d) CrFeCoNi for 5000K. The electron density( $\rho$ ) along the solid line is also compared with the KS-DFT and the ML-predicted electron densities show remarkable agreement with the KS-DFT, true for all three ML models across four systems. The unit of electron density is $\text{Bohr}^{-3}$ in atomic units.

Figure 4 : Comparison of pruning strategies across material systems. (a-d) The error in the electron density prediction for various pruning factors for all systems. The shaded region represents the range (maximum to minimum) of three ML models around the mean, shown by a solid line. (e-h)The error in the total energy obtained via postprocessing from the predicted electron density for various pruning factors for all systems

Figure 5 : Model generalizability to unseen configurations: 2D slices of electron density obtained by KS-DFT, ML model trained on the original dataset, 90% CCS, and 99% CCS-based pruned datasets. The electron density( $\rho$ ) along the solid line is also compared with KS-DFT, showing remarkable agreement. The energy errors(Ha/atom) are also shown for each case. (Top) Mono-vacancy defect for 32-atoms aluminum systems. (Middle) 216-atoms SiGeSn checkerboard systems (Bottom) Di-vacancy defect for 32-atoms CrFeCoNi systems. The unit of electron density is $\text{Bohr}^{-3}$ in atomic units.

System	Generalization test	Original	90% CCS	99% CCS
Aluminum	Structural defects (mono-vacancies)	below	below	below
Aluminum	Temperature extrapolation (2100 K)	below	below	below
SiGeSn	Checkerboard systems (64 and 216 atoms)	below	below	below
SiGeSn	Size extrapolation (216 atoms)	below	below	near
CrFeCoNi	Structural defects (mono-/di-vacancies)	near	near	above
CrFeCoNi	Unseen compositions	below	below	above

Table 1 : Summary of energy error for generalization tests relative to the chemical-accuracy threshold (

\sim 1.6

milli Ha/atom or 1 kcal/mole) for models trained on the original, 90% CCS-, and 99% CCS-pruned datasets. “below”, “near”, and “above” indicate that the energy error is below the chemical-accuracy threshold, above but in its immediate vicinity(around

2-3

milli Ha/atom), and above the threshold, respectively. Detailed results are provided in the Supplementary Section VIII .

Figure 6 : Intrinsic-dimensionality of electronic-structure descriptors. (a) Intrinsic dimension (ID) estimates for four materials using MLE with varying neighbor counts on 2 million randomly sampled points; PCA-based ID for 99% variance are shown. (b) Cumulative variance for PCA.

Figure 7 : Error in smoothness of the predicted electron density: (a) $H^{1}$ seminorm and (b) $H^{1}$ norm of the error for the ML-predicted electron density obtained from the original dataset, $90\%$ , and $99\%$ randomly and CCS based pruned dataset.

Figure 8 : Relative

L_{\infty}

error in the ML-predicted electron density for the original dataset, 90% and 99% randomly, and CCS-based pruned datasets.

Figure 9 : Comparison among the error distribution from the ML-predicted density obtained from the original dataset, $90\%$ , and $99\%$ randomly and CCS based dataset. (left) Absolute Error Distribution for four systems. (right) Probability of a pointwise error exceeding a given threshold.

Figure 10 : Choice of score computation epoch: The error in electron density prediction for (a) Aluminum(20% pruning), (b) Water(40% pruning) (c) SiGeSn(20% pruning) and (d) CrFeCoNi(40% pruning) after computing the GraNd score at different times in training.

Figure 11 : Choice of number of strata(

k

): The error in electron density prediction with various numbers of strata for Aluminum system for various pruning factors for CCS method.

Figure 12 : Choice of hard cut-off rate(

\beta

): The error in electron density prediction with varying

\beta

for (a) Aluminum, (b) water (c) SiGeSn (d) CrFeCoNi for the CCS based pruning. It is observed that the obtained NRMSE with pruned dataset varies with the choice of

\beta

. For higher pruning rate, a higher

\beta

may be optimal. The points with

\times

markers failed to beat random pruning.

Figure 13 : Temporal correlation analysis for a representative system (equiatomic Cr 25 Fe 25 Co 25 Ni 25 , 32 atoms, 5000 K) : Dataset A contains snapshots at every timestep (1 fs spacing) whereas dataset B contains snapshots taken at 100-timestep(100 fs spacing). (Left) Distribution of the descriptor displacement over one sampling step, i.e.

\Delta t

= 1 fs (dataset A) and 100 fs (dataset B). (Right) Descriptor displacement as a function of time separation

\Delta t

averaged over all

t

and grid points.

Aluminum
Pruning	$\text{AUC}_{pr}$		NRMSE
Rate	GraNd	CCS	GraNd	CCS
90%	8.396	8.30	$1.58\times 10^{-2}$	$1.15\times 10^{-2}$
99%	8.727	8.584	$2.55\times 10^{-2}$	$1.3\times 10^{-2}$
Water
Pruning	$\text{AUC}_{pr}$		NRMSE

Table 2 : Comparison of

\text{AUC}_{pr}

and NRMSE between GraNd and CCS method for different pruning percentages

Figure 14 : Structural Generalizability tests for Aluminum system: The error in the electron density prediction for various pruning factors for monovacancies for 32-atoms Aluminum system. The shaded region represents the range (maximum to minimum) of three ML models around the mean, shown by a solid line.

Figure 15 : Structural Generalizability tests for SiGeSn system: The error in the electron density prediction for various pruning factors for SiGeSn handcrafted checkerboard systems. The shaded region represents the range (maximum to minimum) of three ML models around the mean, shown by a solid line.

Figure 16 : Structural Generalizability tests for CrFeCoNi system: The error in the electron density prediction for various pruning factors for (a) mono- (b) di-vacancy defects for 32-atoms CrFeCoNi system. The shaded region represents the range (maximum to minimum) of three ML models around the mean, shown by a solid line. Figures (c) and (d) show the magnified views of Figures (a) and (b).

Figure 17 : Structural Generalizability tests for CrFeCoNi Monovacancy: The error in the energy from the ML-predicted electron density for the original dataset, 90% and 99% CCS-based dataset.

Figure 20 : Generalization tests for temperature extrapolation for aluminum: (left) The error in electron density prediction and (right) the energy error for Aluminum 32-atoms system at 2100K, for models trained on the original, 90% CCS, and 99% CCS pruned datasets.

Figure 21 : Generalization tests for size extrapolation for SiGeSn: (left) The error in electron density prediction and (right) the energy error for SiGeSn 216-atoms system at 2400K, for models trained on the original, 90% CCS, and 99% CCS pruned datasets.

Figure 22 : Model generalizability to unseen configurations: 2D slices of electron density obtained by KS-DFT, ML model trained on the original dataset, 90% CCS, and 99% CCS based pruned datasets. The electron density( $\rho$ ) along the solid line is also compared with KS-DFT, showing remarkable agreement. (First row) Temperature extrapolation test for 32-atoms aluminum systems. (Second row) System size extrapolation test for SiGeSn system. (Third row) Unseen compositions test for the CrFeCoNi system (Fourth row) Mono-vacancy defect for 32-atoms CrFeCoNi systems. The unit of electron density is $\text{Bohr}^{-3}$ in atomic units.

Figure 23 : Time comparison between the ML model trained on the original dataset, 90% CCS and 99% CCS based pruned dataset. The coreset selection time is small and hence is not visible.

Figure 24 : Spatial distribution of pruned and retained grid points for Aluminum, SiGeSn, and CrFeCoNi at 90% CCS pruning. For each material, the top figure shows the KS-DFT electron density on a representative 2D slice, and the bottom figure shows the corresponding spatial map of pruned (red) and retained (blue) grid points. The unit of electron density is $\text{Bohr}^{-3}$ in atomic units.

Figure 25 : Error(RMSE) in atomic forces, as obtained from the ML-predicted electron density for the original dataset, 90% CCS, and 99% CCS-based pruned datasets.

Figure 26 : Redundancy using the SOAP descriptor for the SiGeSn system : Error in electron density prediction for the ML models trained on the original, 90% and 99% CCS pruned datasets.

System	MLE	IID	GRIDE	MiND ML	MID	PCA (99%)	Ambient
Aluminum	60 - 65	72	55 - 60	68 - 70	131	206	236
Water	83 - 92	95	75 - 83	90 - 93	200	247	295
SiGeSn	78 - 89	97	69 - 78	89 - 94	196	243	295
CrFeCoNi	78 - 88	96	68 - 80	89 - 93	201	259	295

Table 3 : Intrinsic dimension (ID) estimates for the four material systems using several estimators, compared with the number of principal components required to capture 99% of the variance (PCA) and the ambient descriptor dimension. The neighbourhood-based estimators (MLE, GRIDE, MiND ML ) are reported as the range obtained by considering the following number of neighbors:

100,250,500,750,1000

. The IID as well as the MID sweep across 100 – 1000 nearest neighbors for their estimation.

Figure 27 : Water system: (a) Error in the predicted electron density for various pruning factors. (b) Error in energy obtained from the predicted electron density for various pruning factors.

Figure 28 : SiGeSn system: (a) Error in the predicted electron density for various pruning factors. (b) Error in energy obtained from the predicted electron density for various pruning factors.

Figure 29 : CrFeCoNi system: (a) Error in the predicted electron density for various pruning factors. (b) Error in energy obtained from the predicted electron density for various pruning factors.

Figure 30 : Aluminum system at 1500 K. Iso-surfaces of electron density obtained by KS-DFT (a) and ML model for original (unpruned) (b), $90$ % CCS (c), and $99$ % CCS (d).

Figure 31 : Aluminum system at 1500 K. Iso-surfaces of errors in electron density obtained by ML model for original (a), 90% CCS (b), and 99% CCS(c) respectively.

Figure 32 : Water system at 600K : Iso-surfaces of electron density (at 0.1 e) obtained by DFT (a) and ML model for original (b), 90% CCS (c), and 99% CCS (d). The red and pink spheres represent the Oxygen and Hydrogen atoms.

Figure 33 : Water system at 600K : Iso-surfaces of errors in electron density (at 0.0075 e) obtained by ML model for original (unpruned) (a), 90% CCS (b), and 99% CCS (c) respectively.

Figure 34 : $\text{Si}_{50}\text{Ge}_{12.5}\text{Sn}_{37.5}$ system at 2400K : Iso-surfaces of electron density obtained by DFT (a) and ML model for original (unpruned) (b), 90% CCS (c), and 99% CCS (d). The blue, purple, and green spheres represent the Si, Ge, and Sn atoms.

Figure 35 : $\text{Si}_{50}\text{Ge}_{12.5}\text{Sn}_{37.5}$ system at 2400K : Iso-surfaces of errors in electron density obtained by ML model for original (a), 90% CCS (b), and 99% CCS (c) respectively.

Figure 36 : $\text{Cr}_{25}\text{Fe}_{25}\text{Co}_{25}\text{Ni}_{25}$ system at 5000K : Iso-surfaces of electron density obtained by DFT (a) and ML model for original (unpruned) (b), 90% CCS (c), and 99% CCS (d). The green, orange, blue, and gray spheres represent the Cr, Fe, Co, and Ni atoms.

Figure 37 : $\text{Cr}_{25}\text{Fe}_{25}\text{Co}_{25}\text{Ni}_{25}$ system at 5000K : Iso-surfaces of errors in electron density obtained by ML model for original (a), 90% CCS (b), and 99% CCS (c) respectively.

Explore similar work

Multitask learning with semiempirical orbital charges enables sample-efficient MLIPs
May 22, 2026Ihor Neporozhnii, Sjoerd Hoogland, Oleksandr VoznyyInteratomic Potentials Large-Scale Electronic Structure Calculations
Department of Physical and Environmental Sciences, University of Toronto, 1065 Military Trail, Toronto, M1C 1A4, Ontario, Canada. · The Alliance for AI-Accelerated Materials Discovery, 10 King’s College Road, Toronto, M5S 3G4, Ontario, Canada. · Department of Electrical and Computer Engineering, University of Toronto, 10 King’s College Road, Toronto, M5S 3G4, Ontario, Canada.
ElemeNet: Multiscale Molecular Machine Learning with Uncertainty Quantification Across the Periodic Table
Jun 29, 2026Jacob W. Toney, Samir Darouich, Yiran Wang +3Molecular Property Prediction Atoms
Department of Chemical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Center for Computational Science and Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Institute for Theoretical Chemistry, University of Stuttgart, Germany +2
MALOQ: Massively Accelerated Learning of Operators for Quantum Transport
Jun 27, 2026Manasa Kaniselvan, Alexander Maeder, Denghui Lu +2Large-Scale Electronic Structure Calculations Quantum Machine Learning
D-ITET, ETH Zurich, Zurich, Switzerland

May 22, 2026, physics.chem-phJ/K move · Enter open · S save

Multitask learning with semiempirical orbital charges enables sample-efficient MLIPs

Ihor Neporozhnii, Sjoerd Hoogland, Oleksandr Voznyy

Department of Physical and Environmental Sciences, University of Toronto, 1065 Military Trail, Toronto, M1C 1A4, Ontario, Canada. · The Alliance for AI-Accelerated Materials Discovery, 10 King’s College Road, Toronto, M5S 3G4, Ontario, Canada. · Department of Electrical and Computer Engineering, University of Toronto, 10 King’s College Road, Toronto, M5S 3G4, Ontario, Canada.

Interatomic Potentials Large-Scale Electronic Structure Calculations Machine Learning Multi--Task Learning Near-Earth Objects

Open paper AskPDF arXiv

Machine learning interatomic potentials (MLIPs) require generating computationally expensive, large-scale training datasets to accurately simulate materials and molecules. Incorporating electronic structure information using multitask learning improves sample efficiency, however, training on full Hamiltonian matrices, which scale quadratically with the number of atoms, is intractable for large datasets. In this work, we show that multitask learning utilizing orbitally resolved semiempirical charges significantly improves sample efficiency and accuracy in MLIPs. To efficiently predict orbital charges, we implement a specialized equivariant model, reducing charge prediction error compared to an invariant baseline. By augmenting training with computationally inexpensive GFN1-xTB orbital charges, which scale linearly with the number of atoms, our model achieves a 46% reduction in energy mean absolute error and requires five times less data to match the performance of energy-only models. Furthermore, our approach outperforms models trained on expensive density functional theory (DFT) atomic charges, capturing orbitally resolved electronic complexity and forcing the network to learn a physically accurate latent space that spontaneously clusters metals by shared chemical properties. Because orbital charges are only required during training, this approach preserves inference efficiency, providing a scalable recipe for developing accurate, data-efficient foundation models for complex chemical systems.

Abstract

Explore similar work

Multitask learning with semiempirical orbital charges enables sample-efficient MLIPs

ElemeNet: Multiscale Molecular Machine Learning with Uncertainty Quantification Across the Periodic Table

MALOQ: Massively Accelerated Learning of Operators for Quantum Transport