We develop a theoretical foundation for designing group-equivariant neural networks that align the choice of symmetries with the underlying probability distributions of the data. Utilising the general structure of fibre decompositions on the domain under group equivariant maps and its relation to that of the likelihood ratio, we present a theoretical framework for identifying group actions that maintain optimal classification performance via the Neyman-Pearson lemma. This provides a unified methodology for improving classification accuracy especially in fundamental applications where one has knowledge of the inherent symmetries of the distributions and how they are broken by measurement. As an application to jet classification at the Large Hadron Collider, we find that there can be performance gains when one utilises smaller permutation symmetries within the constituents. This work offers insights and practical guidelines for constructing more effective group equivariant architectures in diverse machine-learning contexts.
Figures & tables
Sym.
Uniform
Normal
AUC
ϵˉ0(ϵ1=0.95)
AUC
ϵˉ0(ϵ1=0.95)
E(3)
0.974 ± 0.001
0.868 ± 0.007
0.920 ± 0.003
0.652 ± 0.010
O(3)
0.981 ± 0.000
0.905 ± 0.002
0.998 ± 0.000
0.994 ± 0.000
O(2)
1.000 ± 0.000
1.000 ± 0.000
0.999 ± 0.000
0.999 ± 0.000
Table 1: The mean and standard deviation of the background rejection at 95% signal acceptance ϵˉ0(ϵ1=0.95)=1−ϵ0(ϵ1=0.95) and the AUC over ten training runs for the two toy scenarios with added noise. The best performing values are highlighted in bold.
Task
Particle type
Value
Quark/Gluon tagging
Negatively charged mesons and leptons
1
Positively charged mesons and leptons
2
Neutral mesons and the photon
3
Rest
0
Top tagging
Bottom
1
Rest
0
Table 2: Scalar node attribute for particle type utilised in the jet classification experiments.
Cont. Sym.
Discrete Sym.
ΔΓ=30 GeV
ΔΓ=50 GeV
AUC
ϵˉ0(ϵ1=0.95)
AUC
ϵˉ0(ϵ1=0.95)
O(1,3)
Sn
0.972 ± 0.001
0.891 ± 0.008
0.979 ± 0.000
0.915 ± 0.002
⊗tSmt
0.977 ± 0.000
0.933 ± 0.002
0.983 ± 0.000
0.937 ± 0.001
O(1,1) l⊗ O(2) t
Sn
0.977 ± 0.001
0.933 ± 0.006
0.984 ± 0.001
0.941 ± 0.003
⊗tSmt
0.978 ± 0.000
0.938 ± 0.002
0.984 ± 0.000
0.942 ± 0.002
Table 3: The mean and standard deviation of the background rejection at 95% signal acceptance ϵˉ0(ϵ1=0.95)=1−ϵ0(ϵ1=0.95) and the AUC over five training runs for top tagging datasets with different mass windows ΔΓ . The best performing values are highlighted in bold.
Cont. Sym.
Discrete Sym.
AUC
ϵˉ0(ϵ1=0.95)
O(1,3)
Sn
0.875 ± 0.000
0.450 ± 0.002
⊗tSmt
0.879 ± 0.001
0.463 ± 0.002
O(1,1) l⊗ O(2) t
Sn
0.900 ± 0.003
0.531 ± 0.010
⊗tSmt
0.901 ± 0.001
0.532 ± 0.005
Table 4: The mean and standard deviation of the background rejection at 95% signal acceptance ϵˉ0(ϵ1=0.95)=1−ϵ0(ϵ1=0.95) and the AUC over five training runs for quark vs gluon classification. The best performing values are highlighted in bold.
Parameter space symmetries are important for understanding neural networks' loss landscape, training dynamics, and generalization. However, systematically identifying these symmetries remains a challenge. In this paper, we formalize data-dependent parameter symmetries and characterize loss invariance and the group-action axioms through infinitesimal conditions, which provide objectives for jointly learning group generators and nonlinear action maps. Our framework systematically uncovers parameter symmetries, including previously unknown ones. To study larger networks, we establish conditions under which subnetwork symmetries extend to the full model. The same construction gives an explicit family of finite-batch symmetries, providing both analytical examples and a foundation for discovery through small subnetworks. Using the infinitesimal characterization and subnetwork construction, we implement a framework for automated discovery of parameter symmetries, and successfully uncovered symmetries in various architectures, including pretrained transformer models.
Bo Zhao, Nima Dehmamy, Robin Walters +1
Harvard University · IBM Research · Northeastern University +1
Equivariant Neural Networks (ENNs) have empowered numerous applications in scientific fields. Despite their remarkable capacity for representing geometric structures, ENNs suffer from degraded expressivity when processing symmetric inputs: the output representations are invariant to transformations that extend beyond the input's symmetries. The mathematical essence of this phenomenon is that a symmetric input, after being processed by an equivariant map, experiences an increase in symmetry. While prior research has documented symmetry increase in specific cases, a rigorous understanding of its underlying causes and general reduction strategies remains lacking. In this paper, we provide a detailed and in-depth characterization of symmetry increase together with a principled framework for its reduction: (i) For any given feature space and input symmetry group, we prove that the increased symmetry admits an infimum determined by the structure of the feature space; (ii) Building on this foundation, we develop a computable algorithm to derive this infimum, and propose practical guidelines for feature design to prevent harmful symmetry increases. (iii) Under standard regularity assumptions, we demonstrate that for most equivariant maps, our guidelines effectively reduce symmetry increase. To complement our theoretical findings, we provide visualizations and experiments on both synthetic datasets and the real-world QM9 dataset. The results validate our theoretical predictions.
Ning Lin, Jiacheng Cen, Anyi Li +2
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
Symmetry is everywhere in nature and society. Geometric deep learning exploits symmetries in data to improve the performance and efficiency of deep learning systems. In this paper, we extend geometric deep learning to utilize richer symmetry structures. Specifically, we develop order-equivariant neural networks (OENN), which generalize standard graph message passing and sheaf neural networks via the theory of equivariant bundles over face posets (face categories). We (i) characterize all linear order-equivariant maps, (ii) build OENN layers, and (iii) prove universal approximation theorems (UATs) for continuous order-equivariant maps, which are new results even when restricted to sheaf neural networks (for which no UAT was known before). We illustrate the framework on graph and sheaf models. Our results can also be seen as extending the known UAT for graph neural networks to a more general setting that subsumes sheaf neural networks as well. In addition, we show that OENN can be extended further to CENN, Category-Equivariant Neural Network, which gives the general form of equivariant neural networks as well as of equivariant universal approximation theorems, allowing us to leverage categorical symmetry in data (e.g., non-invertible symmetries on multiple objects with compositional relations on those symmetries).
Yoshihiro Maruyama
Department of Mathematical and Information Sciences, Kyoto University, Kyoto, Japan.