Out-of-training properties in frozen interatomic potential embeddings are decodable and ordered by physical scale
Organizations: Altrove, 25 rue de Ponthieu, 75008 Paris, France
Abstract
Machine-learned interatomic potentials (MLIPs) are trained on energies, forces and stresses, yet their frozen representations can predict other materials properties. We probe ORB-v3 and UMA-S across layers, read-outs and equivariant channels for nine properties of 154,875 Materials Project crystals. Both models outperform composition-only descriptors on all targets, including five outside their training objectives; errors in physical units help assess potential screening applications. Optimal depth varies by property, consistent with a physical-scale ordering. Random-weight controls reveal strongly property-dependent benefits of pretraining. UMA-S's rotation-invariant angular channels improve crystal-system classification, while higher scores from its full tensor disappear under rotation; matched-model controls do not isolate equivariance as the explanation. Accuracy varies across different probes, but nearest-neighbour prediction often approaches XGBoost performance, even though materials do not form distinct clusters according to their properties. We also test whether a linear predictor trained on one model's embeddings can predict properties using the other model's embeddings after a learned linear transformation. Predictive accuracy is largely preserved, despite differences in which atoms the two models place close together. These results guide the reuse of frozen MLIP representations for materials prediction.
Figures & tables
| Target | Comp.-only | ORB config | ORB score | ORB cut | UMA config | UMA score | UMA cut |
|---|---|---|---|---|---|---|---|
| band gap (R²) | 0.698 | concat · g4 | 0.890 | 40 % | concat · ang-norm · b2 | 0.907 | 44 % |
| bulk modulus (R²) | 0.882 | concat · g0 | 0.951 | 35 % | concat · ang-norm · emb | 0.958 | 41 % |
| metal/insulator (F1) | 0.845 | concat · g4 | 0.909 | 41 % | node · ang-norm · bb | 0.914 | 44 % |
| magnetic label (F1) | 0.860 | concat · g4 | 0.890 | 22 % | node · ang-norm · bb | 0.903 | 31 % |
| mag. density (R²) | 0.816 | concat · g0 | 0.846 | 9 % | concat · ang-norm · emb | 0.839 | 6 % |
| crystal system (F1) | 0.520 | concat · g2 | 0.622 | 21 % | concat · ang-norm · b0 | 0.817 | 62 % |
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Read-out | Per-atom node [N, ·] | What it captures | Pooled per material [·] |
|---|---|---|---|
| flat / full | flatten([9,128]) → 1152 | all channels; changes under rotation | mean.flatten() → 1152 |
| L=0 | [:,0,:] → 128 | composition / chemistry | L0.mean → 128 |
| L=1 † | ‖L1‖ → 128 | how directional the bonding is | ‖L1.mean‖ → 128 |
| L=2 † | ‖L2‖ → 128 | how elongated or flattened the coordination shell is | ‖L2.mean‖ → 128 |
| ang-norm | cat(L0, ‖L1‖, ‖L2‖) → 384 | atom identity and how asymmetric its environment is, without the direction | cat(L0, ‖L1‖, ‖L2‖).mean → 384 |
| geom-norm | cat(‖L1‖, ‖L2‖) → 256 | geometry with composition removed | cat(‖L1‖, ‖L2‖).mean → 256 |
| Component | Setting | Value |
| Split | scheme | fixed 80/20 train/test, per target |
| seed | 42 | |
| stratification | by class (classification targets only) | |
| Input | dimension | the pooled vector at the selected read-out and layer, 256–768 at the Table 1 configurations (node 256 or 384; concat, node edge at one layer, 512 or 768; L0 concat 256) |
| XGBoost (regressor / classifier) | n_estimators | 300 |
| max_depth | 6 |
| Stage | Wall-clock | Basis |
| (a) Embedding extraction | ||
| ORB-v3, 154,875 structures | ≈ 7 h | measured (0.16 s/structure) |
| UMA-S, 154,875 structures | ≈ 13 h | measured (0.30 s/structure) |
| Random-weight networks, both models | ≈ 16.5 h | measured (4.7 + 11.8 h) |
| Matched models on the 20,000-material subsample (ORB-v3 mpa, MACE-OMAT-0) | ≈ 1.3 h | measured (0.8 + 0.5 h) |
| External databases, both models | ≈ 13 h | JARVIS-DFT 3D estimated from the per-structure rates ((0.16 + 0.30) s × 93,902); OQMD measured (1.4 h) |
| MP reference | Crystals absent from MP | Crystals shared with MP (structure / label) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Target | MP test | n | External refit | MP-trained | Transfer gap | n | S 0 MP / MP | S 1 Ext. / MP | S 3 MP / Ext. | S 2 Ext. / Ext. |
| formation E/atom | 0.990 | 30,346 | 0.969 ± 0.002 | 0.896 ± 0.004 | −0.073 ± 0.003 | 12,769 | 0.989 | 0.962 | 0.938 | 0.968 |
| energy above hull | 0.948 | 30,335 | 0.951 ± 0.004 | −1.117 ± 0.015 | −2.068 ± 0.014 | 12,769 | 0.918 | 0.536 | −1.585 | −1.599 |
| band gap | 0.890 | 30,335 | 0.848 ± 0.016 | 0.693 ± 0.008 | −0.155 ± 0.015 | 12,769 | 0.882 | 0.872 | 0.817 | 0.824 |
| density | 0.982 | 30,335 | 0.978 ± 0.001 | 0.955 ± 0.002 | −0.023 ± 0.001 | 12,769 | 0.980 | 0.926 | 0.917 | 0.977 |
| bulk modulus | 0.951 | 14,500 | 0.914 ± 0.008 | 0.884 ± 0.009 | −0.030 ± 0.001 | 1,536 | 0.942 | 0.935 | 0.924 | 0.927 |
| MP reference | Crystals absent from MP | Crystals shared with MP (structure / label) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Target | MP test | n | External refit | MP-trained | Transfer gap | n | S 0 MP / MP | S 1 Ext. / MP | S 3 MP / Ext. | S 2 Ext. / Ext. |
| formation E/atom | 0.990 | 37,797 | 0.976 ± 0.004 | 0.931 ± 0.004 | −0.045 ± 0.000 | 439 | 0.988 | 0.987 | 0.976 | 0.975 |
| energy above hull | 0.948 | 37,804 | 0.301 ± 0.122 | 0.011 ± 0.009 | −0.290 ± 0.129 | 439 | 0.860 | 0.839 | 0.861 | 0.864 |
| band gap | 0.890 | 37,693 | 0.818 ± 0.034 | 0.722 ± 0.031 | −0.096 ± 0.011 | 436 | 0.929 | 0.921 | 0.864 | 0.875 |
| density | 0.982 | 37,804 | 0.971 ± 0.001 | 0.940 ± 0.002 | −0.031 ± 0.002 | 439 | 0.981 | 0.978 | 0.975 | 0.982 |
| crystal system | 0.623 | 37,804 | 0.806 ± 0.005 | 0.454 ± 0.004 | −0.351 ± 0.005 | 418 | 0.556 | 0.521 | 0.530 | 0.529 |
| MP reference | Crystals absent from MP | Crystals shared with MP (structure / label) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Target | MP test | n | External refit | MP-trained | Transfer gap | n | S 0 MP / MP | S 1 Ext. / MP | S 3 MP / Ext. | S 2 Ext. / Ext. |
| formation E/atom | 0.990 | 30,346 | 0.969 ± 0.003 | 0.919 ± 0.004 | −0.050 ± 0.001 | 12,768 | 0.988 | 0.960 | 0.935 | 0.967 |
| energy above hull | 0.960 | 30,346 | 0.948 ± 0.003 | −1.071 ± 0.026 | −2.018 ± 0.026 | 12,768 | 0.937 | 0.534 | −1.587 | −1.598 |
| band gap | 0.907 | 30,346 | 0.875 ± 0.014 | 0.708 ± 0.029 | −0.167 ± 0.022 | 12,768 | 0.895 | 0.886 | 0.832 | 0.845 |
| density | 0.983 | 30,346 | 0.980 ± 0.001 | 0.953 ± 0.001 | −0.027 ± 0.001 | 12,768 | 0.982 | 0.926 | 0.919 | 0.979 |
| bulk modulus | 0.959 | 14,500 | 0.921 ± 0.008 | 0.896 ± 0.008 | −0.025 ± 0.003 | 1,536 | 0.955 | 0.948 | 0.927 | 0.932 |
| MP reference | Crystals absent from MP | Crystals shared with MP (structure / label) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Target | MP test | n | External refit | MP-trained | Transfer gap | n | S 0 MP / MP | S 1 Ext. / MP | S 3 MP / Ext. | S 2 Ext. / Ext. |
| formation E/atom | 0.990 | 37,797 | 0.976 ± 0.005 | 0.958 ± 0.005 | −0.018 ± 0.001 | 439 | 0.987 | 0.985 | 0.979 | 0.977 |
| energy above hull | 0.960 | 37,805 | 0.158 ± 0.146 | 0.014 ± 0.010 | −0.144 ± 0.155 | 439 | 0.859 | 0.832 | 0.857 | 0.848 |
| band gap | 0.907 | 37,694 | 0.814 ± 0.023 | 0.732 ± 0.033 | −0.082 ± 0.015 | 436 | 0.939 | 0.936 | 0.871 | 0.890 |
| density | 0.983 | 37,805 | 0.973 ± 0.001 | 0.937 ± 0.002 | −0.036 ± 0.002 | 439 | 0.977 | 0.971 | 0.966 | 0.975 |
| crystal system | 0.817 | 37,805 | 0.908 ± 0.007 | 0.647 ± 0.005 | −0.261 ± 0.004 | 418 | 0.804 | 0.668 | 0.734 | 0.692 |
Materials Property Prediction
Materials property prediction uses data and physical models to estimate how composition, structure, and processing determine a material’s behavior.
See all 200 papers