Optimal Symmetries in Binary Classification

Vishal S. Ngairangbam; Michael Spannowsky

cs.LGAug 16, 2024

Optimal Symmetries in Binary Classification

Authors: Vishal S. Ngairangbam, Michael Spannowsky

Organizations: Department of Physics, Durham University Durham DH1 3LE, United Kingdom

Abstract

We develop a theoretical foundation for designing group-equivariant neural networks that align the choice of symmetries with the underlying probability distributions of the data. Utilising the general structure of fibre decompositions on the domain under group equivariant maps and its relation to that of the likelihood ratio, we present a theoretical framework for identifying group actions that maintain optimal classification performance via the Neyman-Pearson lemma. This provides a unified methodology for improving classification accuracy especially in fundamental applications where one has knowledge of the inherent symmetries of the distributions and how they are broken by measurement. As an application to jet classification at the Large Hadron Collider, we find that there can be performance gains when one utilises smaller permutation symmetries within the constituents. This work offers insights and practical guidelines for constructing more effective group equivariant architectures in diverse machine-learning contexts.

Figures & tables

Sym.	Uniform		Normal
	AUC	$\bar{\epsilon}_{0}(\epsilon_{1}=0.95)$	AUC	$\bar{\epsilon}_{0}(\epsilon_{1}=0.95)$
E(3)	0.974 $\pm$ 0.001	0.868 $\pm$ 0.007	0.920 $\pm$ 0.003	0.652 $\pm$ 0.010
O(3)	0.981 $\pm$ 0.000	0.905 $\pm$ 0.002	0.998 $\pm$ 0.000	0.994 $\pm$ 0.000
O(2)	1.000 $\pm$ 0.000	1.000 $\pm$ 0.000	0.999 $\pm$ 0.000	0.999 $\pm$ 0.000

Table 1: The mean and standard deviation of the background rejection at 95% signal acceptance

\bar{\epsilon}_{0}(\epsilon_{1}=0.95)=1-\epsilon_{0}(\epsilon_{1}=0.95)

and the AUC over ten training runs for the two toy scenarios with added noise. The best performing values are highlighted in bold.

Task	Particle type	Value
Quark/Gluon tagging	Negatively charged mesons and leptons	1
	Positively charged mesons and leptons	2
	Neutral mesons and the photon	3
	Rest	0
Top tagging	Bottom	1
Top tagging	Rest	0

Table 2: Scalar node attribute for particle type utilised in the jet classification experiments.

Cont. Sym.	Discrete Sym.	$\Delta\Gamma=30$ GeV		$\Delta\Gamma=50$ GeV
		AUC	$\bar{\epsilon}_{0}(\epsilon_{1}=0.95)$	AUC	$\bar{\epsilon}_{0}(\epsilon_{1}=0.95)$
O(1,3)	$S_{n}$	0.972 $\pm$ 0.001	0.891 $\pm$ 0.008	0.979 $\pm$ 0.000	0.915 $\pm$ 0.002
O(1,3)	$\otimes_{t}\;S_{m_{t}}$	0.977 $\pm$ 0.000	0.933 $\pm$ 0.002	0.983 $\pm$ 0.000	0.937 $\pm$ 0.001
O(1,1) ${}_{l}\otimes$ O(2) t	$S_{n}$	0.977 $\pm$ 0.001	0.933 $\pm$ 0.006	0.984 $\pm$ 0.001	0.941 $\pm$ 0.003
O(1,1) ${}_{l}\otimes$ O(2) t	$\otimes_{t}\;S_{m_{t}}$	0.978 $\pm$ 0.000	0.938 $\pm$ 0.002	0.984 $\pm$ 0.000	0.942 $\pm$ 0.002

Table 3: The mean and standard deviation of the background rejection at 95% signal acceptance

\bar{\epsilon}_{0}(\epsilon_{1}=0.95)=1-\epsilon_{0}(\epsilon_{1}=0.95)

and the AUC over five training runs for top tagging datasets with different mass windows

\Delta\Gamma

. The best performing values are highlighted in bold.

Cont. Sym.	Discrete Sym.	AUC	$\bar{\epsilon}_{0}(\epsilon_{1}=0.95)$
O(1,3)	$S_{n}$	0.875 $\pm$ 0.000	0.450 $\pm$ 0.002
O(1,3)	$\otimes_{t}\;S_{m_{t}}$	0.879 $\pm$ 0.001	0.463 $\pm$ 0.002
O(1,1) ${}_{l}\otimes$ O(2) t	$S_{n}$	0.900 $\pm$ 0.003	0.531 $\pm$ 0.010
O(1,1) ${}_{l}\otimes$ O(2) t	$\otimes_{t}\;S_{m_{t}}$	0.901 $\pm$ 0.001	0.532 $\pm$ 0.005

Table 4: The mean and standard deviation of the background rejection at 95% signal acceptance

\bar{\epsilon}_{0}(\epsilon_{1}=0.95)=1-\epsilon_{0}(\epsilon_{1}=0.95)

and the AUC over five training runs for quark vs gluon classification. The best performing values are highlighted in bold.

Explore similar work

Discovering Symmetries in Neural Network Parameter Spaces
Sep 27, 2026Bo Zhao, Nima Dehmamy, Robin Walters +1Symmetry Equivariant Neural Networks
Harvard University · IBM Research · Northeastern University +1
Reducing Symmetry Increase in Equivariant Neural Networks
Aug 12, 2026Ning Lin, Jiacheng Cen, Anyi Li +2Equivariant Neural Networks Equivariance
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
Foundations of Equivariant Deep Learning: Unifying Graph and Sheaf Neural Networks
Jul 4, 2026Yoshihiro MaruyamaEquivariant Neural Networks Sheaf Neural Networks
Department of Mathematical and Information Sciences, Kyoto University, Kyoto, Japan.

Sep 27, 2026, cs.LGJ/K move · Enter open · S save

Discovering Symmetries in Neural Network Parameter Spaces

Bo Zhao, Nima Dehmamy, Robin Walters, Rose Yu

Harvard University · IBM Research · Northeastern University+1

Symmetry Equivariant Neural Networks Neural Network

Open paper AskPDF arXiv

Parameter space symmetries are important for understanding neural networks' loss landscape, training dynamics, and generalization. However, systematically identifying these symmetries remains a challenge. In this paper, we formalize data-dependent parameter symmetries and characterize loss invariance and the group-action axioms through infinitesimal conditions, which provide objectives for jointly learning group generators and nonlinear action maps. Our framework systematically uncovers parameter symmetries, including previously unknown ones. To study larger networks, we establish conditions under which subnetwork symmetries extend to the full model. The same construction gives an explicit family of finite-batch symmetries, providing both analytical examples and a foundation for discovery through small subnetworks. Using the infinitesimal characterization and subnetwork construction, we implement a framework for automated discovery of parameter symmetries, and successfully uncovered symmetries in various architectures, including pretrained transformer models.

Abstract

Explore similar work

Discovering Symmetries in Neural Network Parameter Spaces

Reducing Symmetry Increase in Equivariant Neural Networks

Foundations of Equivariant Deep Learning: Unifying Graph and Sheaf Neural Networks