Tensor-reduced atomic density representations

James P. Darby; Dávid P. Kovács; Ilyes Batatia; Miguel A. Caro; Gus L. W. Hart; Christoph Ortner; Gábor Csányi

physics.chem-phOct 2, 2022

Tensor-reduced atomic density representations

Authors: James P. Darby, Dávid P. Kovács, Ilyes Batatia, Miguel A. Caro, Gus L. W. Hart, Christoph Ortner, Gábor Csányi

Organizations: Warwick Centre for Predictive Modelling, School of Engineering, University of Warwick, Coventry, CV4 7AL, UK · Engineering Laboratory, University of Cambridge, Cambridge, CB2 1PZ UK · ENS Paris-Saclay, Université Paris-Saclay, 91190 Gif-sur-Yvette, France · Department of Electrical Engineering and Automation, Aalto University, FIN-02150 Espoo, Finland ·

Abstract

Density based representations of atomic environments that are invariant under Euclidean symmetries have become a widely used tool in the machine learning of interatomic potentials, broader data-driven atomistic modelling and the visualisation and analysis of materials datasets.The standard mechanism used to incorporate chemical element information is to create separate densities for each element and form tensor products between them. This leads to a steep scaling in the size of the representation as the number of elements increases. Graph neural networks, which do not explicitly use density representations, escape this scaling by mapping the chemical element information into a fixed dimensional space in a learnable way. We recast this approach as tensor factorisation by exploiting the tensor structure of standard neighbour density based descriptors. In doing so, we form compact tensor-reduced representations whose size does not depend on the number of chemical elements, but remain systematically convergeable and are therefore applicable to a wide range of data analysis and regression tasks.

Figures & tables

Name	Product Basis	Index Notation	Basis size
ACE	${\bm{A}}_{i}=\vec{A}_{i}\otimes\vec{A}_{i}\dots\otimes\vec{A}_{i}$	${\bm{A}}_{i,{\bm{z}}{\bm{n}}{\bm{l}}{\bm{m}}}=\prod_{t=1}^{\nu}A_{i,z_{t}n_{t}l_{t}m_{t}}$	$\mathcal{O}(NS)^{\nu}$
[ 10 ]
Embedding	$\bar{{\bm{A}}}_{i}=(\bm{W}\vec{A}_{i})\otimes(\bm{W}\vec{A}_{i})\dots\otimes(\bm{W}\vec{A}_{i})$	$\bar{{\bm{A}}}_{i,{\bm{k}}{\bm{l}}{\bm{m}}}=\prod_{t=1}^{\nu}\bar{A}_{i,k_{t}l_{t}m_{t}}$	$\mathcal{O}(K^{\nu})$
[ 14 , 19 , 15 , 10 , 30 ]		$\bar{A}_{i,k_{t}l_{t}m_{t}}=\sum_{zn}W^{k_{t}}_{zn}A_{i,znl_{t}m_{t}}$
Tensor decomposition	$\tilde{{\bm{A}}}_{i}=(\bm{W}\vec{A}_{i}){\begin{smallmatrix}\otimes^{lm}\vskip-1.70709pt\\ \odot_{k\hskip 4.41014pt}\end{smallmatrix}}(\bm{W}\vec{A}_{i})\dots{\begin{smallmatrix}\otimes^{lm}\vskip-1.70709pt\\ \odot_{k\hskip 4.41014pt}\end{smallmatrix}}(\bm{W}\vec{A}_{i})$	$\tilde{{\bm{A}}}_{i,k{\bm{l}}{\bm{m}}}=\prod_{t=1}^{\nu}\tilde{A}_{i,kl_{t}m_{t}}$	$\mathcal{O}(K)$
[ 25 ]		$\tilde{A}_{i,kl_{t}m_{t}}=\sum_{zn}W^{kl_{t}}_{zn}A_{i,znl_{t}m_{t}}$

Table 1: The density projection

\vec{A}_{i}

are viewed as vectors with a composite index

(z,n,l,m)

whereas the embedded density projections

\bar{A}_{i}=(\bm{W}\vec{A}_{i})

etc. are indexed by

(k,l,m)

. The most general tensor reduced many body density projection (“Tensor sketch”) and its special case (“Tensor decomposition”) are shown together with their scaling with the number of radial basis functions,

N

, number of chemical elements,

S

and number of embedding channels,

K

. The symbol

{\begin{smallmatrix}\otimes^{lm}\vskip-1.70709pt\\ \odot_{k\hskip 4.41014pt}\end{smallmatrix}}

means full tensor product across

l

and

m

but element-wise product across

k

whereas

\otimes

indicates a full tensor product across all indices.

Figure 1: The row rank of the design matrix as a function of basis set on a dataset of all symmetry inequivalent fcc lattices of 5 chemical elements and unit cell sizes of up to 6 atoms. The inset zooms in on the

x=y

region. The colours denote the size of the rattling around the crystal sites of an ideal fcc lattice, the markers correspond to increasing body-order of the basis and the line-style compares the fully-coupled and tensor-reduced ACE versions.

Figure 2: Convergence of the energy errors of organic molecules on the independent test set with respect to the number of basis functions is shown for linear models using standard and tensor-reduced (tensor decomposition) Atomic Cluster Expansion features. Error bars show the standard error in the mean, computed across 5 fits using independently chosen random weights.

Figure 3: Convergence of force RMSE on the high-entropy alloy test set as a function of descriptor length for various GAP models. Error bars show the standard error in the mean, computed across 5 fits using independently chosen random weights. The tensor reduction is denoted using TD for tensor-decomposition ( 12 ), TS for tensor-sketching ( 15 ) and the reduction is performed over the indices after the hyphen.

	MACE $(1)$	MACE $(1)$	MACE $(1)$	MACE $(2)$	TeaNet [ 56 ]	NequIP [ 23 ]
$U_{zn}$	opt	rand	rand	opt
$V^{kl_{t}}_{n}$	opt	opt	rand	opt
$E$	53.8	54.7	254.8	16.5	19.6	47.8
$F$	0.194	0.193	0.451	140.2	0.174	0.199

Table 2: Mean absolute errors on the HME21 dataset. Energy (

E

, meV) and force (

F

, eV/Å) errors of models. The labels, (1) or (2), of the MACE models indicate the number of message passing layers. MACE errors are an average over three seeds.

Figure 4: Convergence of linear Tensor-reduced ACE force error to the fully coupled with increasing number of channels. Error bars show the standard error in the mean, computed across 5 fits using independently chosen random weights

Figure 5: Convergence of the information imbalance, relative to the full power spectrum, for 1000 randomly chosen environments from liquid configurations in a high-entropy alloy dataset [ 49 ] , S=5. Element and element-radial mixing are indicated using red and blue respectively. Fully coupled channels are indicated with a dotted line whilst uncoupled channels are indicated with a solid (or dashed) line. The black dotted line corresponds to a random projection of the full power spectrum whilst the grey dotted line corresponds to

p^{zz^{\prime}}_{nn^{\prime}l}

computed using fewer basis functions (while keeping the

n_{\mathrm{max}}=2l_{\mathrm{max}}

relation). Error bars show the standard error in the mean across 10 randomly chosen sets of weights and the vertical grey line indicates the length of the reference power spectrum,

n_{\mathrm{max}}=8,l_{\mathrm{max}}=4

Figure 6: Convergence of the information imbalance with respect to descriptor length for 2000 carbon environments randomly selected from the revised MD17 dataset [ 58 ] . Element and element-radial mixing are indicated using red and blue respectively. Fully coupled channels are indicated with a dotted line whilst uncoupled channels are indicated with a solid (or dashed) line. The black dotted line is Random Projection on the full power spectrum. Error bars show the standard error in the mean computed across 10 (20 for the red lines) randomly chosen sets of weights. The grey line corresponds to using the full power spectrum truncated using

n_{\mathrm{max}}=2l_{\mathrm{max}}

for

l_{\mathrm{max}}=1,2,3

where

l_{\mathrm{max}}

=4 is the reference used for all comparisons.

Figure 7: Convergence of the information imbalance, relative to the full Smooth Overlap of Atomic Positions power spectrum, is shown as a function of descriptor length for a dataset of P environments. The dataset was constructed by extracting 2000 environments from the training dataset of ref. [ 59 ] using CUR decomposition and the same Smooth Overlap of Atomic Positions parameters as the original work [ 57 ] were used. Fully coupled channels are indicated with a dotted line whilst uncoupled channels are indicated with a solid (or dashed) line. The black dotted line corresponds to a random projection of the full power spectrum. Error bars show the standard error in the mean computed across 10 randomly chosen sets of weights. The grey line corresponds to using the full power spectrum truncated using

n_{\mathrm{max}}=2l_{\mathrm{max}}

for

l_{\mathrm{max}}=1,2,3,4,5

where

l_{\mathrm{max}}=6

is the reference used for all comparisons.

Figure 8: Convergence of energy errors on the test set as a function of descriptor length for various GAP models. The grey line is full Smooth Overlap of Atomic Positions using

n_{\mathrm{max}}=2l_{\mathrm{max}}

, for

l_{\mathrm{max}}=1,2,3,4

. The black dotted line is Random Projection on the full power spectrum whilst the red and blue lines use a tensor-reduced features with element and element-radial mixing respectively. Error bars show the standard error in the mean, computed across 5 fits using independently chosen random weights.

Molecular Representation Learning

Molecular representation learning encodes chemical structures and context for prediction, generation, and discovery across chemistry and biology.

See all 207 papers

MatRIS: Toward Reliable and Efficient Pretrained Machine Learning Interatomic Potentials
Mar 2, 2026Yuanchang Zhou, Siyu Hu, Xiangyu Zhang +3Machine Learning Interatomic Potentials Computational Materials Science
State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, UCAS
QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space
May 11, 2026Pablo Martínez Crespo, Stefano Ribes, Martin Rahm +6Graph Neural Networks Quantum Chemistry
Department of Computer Science and Engineering Chalmers University of Technology and University of Gothenburg · Department of Chemistry and Chemical Engineering Chalmers University of Technology · Technology Research Intel Corporation +3
TriForces: Augmenting Atomistic GNNs for Transferable Representations
May 20, 2026Ali Ramlaoui, Alexandre Duval, Hannah Bull +4Disentangled Representation Learning Machine Learning Interatomic Potentials
Entalpic, Paris, France · Universit´e Paris-Saclay, CentraleSup´elec, Inria, Gif-sur-Yvette, France

Mar 2, 2026J/K move · Enter open · S save

MatRIS: Toward Reliable and Efficient Pretrained Machine Learning Interatomic Potentials

Yuanchang Zhou, Siyu Hu, Xiangyu Zhang, Hongyu Wang+2

State Key Lab of Processors, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, UCAS

Machine Learning Interatomic Potentials Computational Materials Science Invariant Representation Learning Materials Property Prediction

Open paper AskPDF arXiv

Foundation MLIPs demonstrate broad applicability across diverse material systems and have emerged as a powerful and transformative paradigm in chemical and computational materials science. Equivariant MLIPs achieve state-of-the-art accuracy in a wide range of benchmarks by incorporating equivariant inductive bias. However, the reliance on tensor products and high-degree representations makes them computationally costly. This raises a fundamental question: as quantum mechanical-based datasets continue to expand, can we develop a more compact model to thoroughly exploit high-dimensional atomic interactions? In this work, we present MatRIS (Materials Representation and Interaction Simulation), an invariant MLIP that introduces attention-based modeling of three-body interactions. MatRIS leverages a novel separable attention mechanism with linear complexity

O(N)

, enabling both scalability and expressiveness. MatRIS delivers accuracy comparable to that of leading equivariant models on a wide range of popular benchmarks (Matbench-Discovery, MatPES, MDR phonon, Molecular dataset, etc). Taking Matbench-Discovery as an example, MatRIS achieves an F1 score of up to 0.847 and attains comparable accuracy at a lower training cost. The work indicates that our carefully designed invariant models can match or exceed the accuracy of equivariant models at a fraction of the cost, shedding light on the development of accurate and efficient MLIPs.

Abstract

Molecular Representation Learning

Explore similar work

MatRIS: Toward Reliable and Efficient Pretrained Machine Learning Interatomic Potentials

QT-Net: Rethinking Evaluation of AI Models in Atomic Chemical Space

TriForces: Augmenting Atomistic GNNs for Transferable Representations