cs.LGOct 20, 2026

A General Topological Symmetry Index for Critical Orbits In Neural Network Loss Landscapes

Authors: Jingzhou Liu

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

A systematic understanding of the global structure of neural-network loss landscapes remains challenging due to their high dimensionality. Symmetries arising from network architectures and learning objectives provide a way to investigate aspects of this structure. Although a growing number of studies have drawn attention to these symmetries, a systematic characterization of them, as well as of the critical orbits they give rise to, remains incomplete. To address this gap, in this work we establish a topological symmetry index, based on the methodologies from the family of equivariant degree theories, to decode these symmetries. The frameworks apply to arbitrary network architectures and include methods applicable to degenerate critical orbits. The index detects the isotropy of the critical points and quantifies the corresponding net topological contributions. Several examples, including softplus MLPs, Transformers, and equivariant neural networks, are presented to illustrate the computations.

Figures & tables

NotationDescription
N(K)N(K)Normalizer of KK in GG
W(K)=N(K)/KW(K)=N(K)/KWeyl group of KK
(K)={gKg−1:g∈G}(K)=\{gKg^{-1}:g\in G\}Conjugacy class of KK in GG
Φ(G)={(K):K≤G}\Phi(G)=\{(K):K\leq G\}Collection of subgroups’ conjugacy classes of GG
Φ0(G)={(K)∈Φ(G):W(K) finite}\Phi_{0}(G)=\{(K)\in\Phi(G):W(K)\text{ finite}\}Elements (K)∈Φ(G)(K)\in\Phi(G) for which dim⁡W(K)=0\dim W(K)=0
Gθ={g∈G:gθ=θ}G_{\theta}=\{g\in G:g{\theta}={\theta}\}Isotropy group of θ∈V\theta\in V
Table 1. Notations
Rep.Character(1)(1)(1,2)(1,2)(1,2)(3,4)(1,2)(3,4)(1,2,3)(1,2,3)(1,2,3,4)(1,2,3,4)
W0\mathcal{W}_{0}χ0\chi_{0}1111111111
W1\mathcal{W}_{1}χ1\chi_{1}11−1-11111−1-1
W2\mathcal{W}_{2}χ2\chi_{2}220022−1-100
W3\mathcal{W}_{3}χ3\chi_{3}33−1-1−1-10011
W4\mathcal{W}_{4}χ4\chi_{4}3311−1-100−1-1
R4\mathbb{R}^{4}χR4\chi_{\mathbb{R}^{4}}4422001100
Table 2. Character table of S4S_{4} .
EigenvalueEigenspaceDimension
22W0\mathcal{W}_{0}11
22W2\mathcal{W}_{2}22
22W4\mathcal{W}_{4}33
11W4∗\mathcal{W}_{4}^{*}33
−1-1W0∗\mathcal{W}_{0}^{*}11
−1-1W2∗\mathcal{W}_{2}^{*}22
Table 3. δ(∇2L(θo))\delta\big(\nabla^{2}\mathscr{L}(\theta_{o})\big) and the corresponding eigenspaces
Figure 1. S3×S3S_{3}\times S_{3} Subgroup Lattice
KK∣K∣|K|W(K)W(K)∣W(K)∣|W(K)|
Z1×Z1\mathbb{Z}_{1}\times\mathbb{Z}_{1}1S3×S3S_{3}\times S_{3}36
Z1×D1\mathbb{Z}_{1}\times D_{1}2S3S_{3}6
D1×Z1D_{1}\times\mathbb{Z}_{1}2S3S_{3}6
D1×D1D_{1}\times D_{1}4Z1\mathbb{Z}_{1}1
Z1×S3\mathbb{Z}_{1}\times S_{3}6S3S_{3}6
S3×Z1S_{3}\times\mathbb{Z}_{1}6S3S_{3}6
Table 4. W(K)W(K) in S3×S3S_{3}\times S_{3}
K\HK\backslash HZ1×Z1\mathbb{Z}_{1}\times\mathbb{Z}_{1}Z1×D1\mathbb{Z}_{1}\times D_{1}D1×Z1D_{1}\times\mathbb{Z}_{1}D1×D1D_{1}\times D_{1}Z1×S3\mathbb{Z}_{1}\times S_{3}S3×Z1S_{3}\times\mathbb{Z}_{1}S3×D1S_{3}\times D_{1}D1×S3D_{1}\times S_{3}S3×S3S_{3}\times S_{3}
Z1×Z1\mathbb{Z}_{1}\times\mathbb{Z}_{1}133911331
Z1×D1\mathbb{Z}_{1}\times D_{1}010310131
D1×Z1D_{1}\times\mathbb{Z}_{1}001301311
D1×D1D_{1}\times D_{1}000100111
Z1×S3\mathbb{Z}_{1}\times S_{3}000010031
S3×Z1S_{3}\times\mathbb{Z}_{1}000001301
Table 5. The number of n(K,H)n(K,H) .
KKdim⁡VK\dim V^{K}deg⁡(∇VL∣VK,ΩK)\deg(\nabla_{V}\mathscr{L}|_{V^{K}},\Omega^{K})
S3×S3S_{3}\times S_{3}212111
S3×D1S_{3}\times D_{1}272700
D1×S3D_{1}\times S_{3}272700
D1×D1D_{1}\times D_{1}333300
S3×Z1S_{3}\times\mathbb{Z}_{1}33336464
Z1×S3\mathbb{Z}_{1}\times S_{3}33336464
Table 6. The Brouwer degree deg⁡(∇VL∣VK,ΩK)\deg(\nabla_{V}\mathscr{L}|_{V^{K}},\Omega^{K})
More in this topic

Loss Landscape Geometry

Loss landscape geometry studies how the shape of learning objectives affects optimization, convergence, and generalization.

See all 34 papers

Explore similar work

CardsList

  1. Parameter-Level Attribution of Symmetry in Trained Networks Though Parameter-Wise Functional Sensitivity

    Aug 25, 2026Alan Muriithi, Vedanta Thapar, Torben BerndtEquivariant Neural NetworksNeural Network Interpretability

    Mathematical Institute, University of Oxford · Heidelberg Institute for Theoretical Studies

  2. Discovering Symmetries in Neural Network Parameter Spaces

    Sep 27, 2026Bo Zhao, Nima Dehmamy, Robin Walters +1Scientific DiscoverySymmetry Discovery

    Harvard University · IBM Research · Northeastern University +1

  3. Symmetries and Singularities

    Sep 13, 2026Vishnu Varadarajan, Mihir More, Aritra Das +1Singular Learning TheoryGraph Attention Networks

    Truth Audit Labs