Equivariant Neural Networks

Follow this topic Ask about this topic

Latest papers 146

18 papers in the last four weeks, up 38% on the four weeks before. 0.1% of all new papers.

May 14, 2026

Discretizing Group-Convolutional Neural Networks for 3D Geometry in Feature Space

Group-convolutional neural networks (GCNNs) are among the most important methods for introducing symmetry as an inductive bias in deep learning: In each linear layer, GCNNs sample a transformation group

G

densely and correlate data and filters in different poses (with suitable anti-aliasing for steerable GCNNs) to maintain equivariance with respect to

G

. Unfortunately, applying filters to many data items resulting from this sampling is expensive (even for translations alone, i.e., in ordinary CNNs), and costs grow exponentially with increasing degrees of freedom (such as translations and rotations in 3D), which often hinders practical applications. In this paper, we propose sampling in feature space, i.e., replacing geometrically dense samples with representative samples selected by feature similarity. This decouples geometric resolution from memory and processing costs during training and inference, providing a novel way to trade off computational effort and accuracy. Our main empirical finding is that a coarse feature-space sampling already preserves classification accuracy remarkably well, which permits precomputation based on geometric similarity, accelerating the training of equivariant 3D classifiers substantially.

Daniel Franzen, Jean Philip Filling, Michael Wand

Johannes Gutenberg University Mainz, Germany

Equivariant Neural Networks 3D Representation Learning

May 14, 2026

Spontaneous symmetry breaking and Goldstone modes for deep information propagation

In physical systems, whenever a continuous symmetry is spontaneously broken, the system possesses excitations called Goldstone modes, which allow coherent information propagation over long distances and times. In this work, we study deep neural networks whose internal layers are equivariant under a continuous symmetry and may therefore support analogous Goldstone-like degrees of freedom. We demonstrate, both analytically and empirically, that these degrees of freedom enable coherent signal propagation across depth and recurrent iterations, providing a mechanism for stable information flow without relying on architectural stabilizers such as residual connections or normalization. In feedforward networks, this results in improved trainability and representational diversity across layers. In recurrent settings, we demonstrate the same mechanism is valuable for long-term memory by propagating information over recurrent iterations, thereby improving performance of RNNs and GRUs on long-sequence modeling tasks.

Nabil Iqbal, T. Anderson Keller, Yue Song +2

Dept. of Mathematical Sciences, Durham University · Kempner Institute, Harvard University · College of AI, Tsinghua University +2

Symmetry Breaking Equivariant Neural Networks Recurrent Neural Networks

May 13, 2026

Aligning Network Equivariance with Data Symmetry: A Theoretical Framework and Adaptive Approach for Image Restoration

Image restoration is an inherently ill posed inverse problem. Equivariant networks that embed geometric symmetry priors can mitigate this ill posedness and improve performance. However, current understanding of the relationship between network equivariance and data symmetry remains largely heuristic. Particularly for real world data with imperfect symmetry, existing research lacks a systematic theoretical framework to quantify symmetry, select transformation groups, or evaluate model data alignment. To bridge this gap, we conduct an analysis from an optimization perspective and formalize the intrinsic relationship among data symmetry priors, model equivariance, and generalization capability. Specifically, we propose for the first time a quantifiable definition of non strict symmetry at the dataset level (rather than sample level) and use it as a constraint to formulate the restoration inverse problem. We then show that the equivariance for restoration models can be naturally derived from this inverse problems incorporated the proposed symmetry constraints, and that the equivariance error of the optimal restoration operator is strictly bounded by the data symmetry error and the discretization mesh size. Furthermore, by analyzing the network's empirical risk, we demonstrate that aligning equivariance with data symmetry optimizes the bias variance trade off, minimizing the total expected risk. Guided by these insights, we propose a Sample Adaptive Equivariant Network that uses a hypernetwork and transformation learnable equivariant convolutions to dynamically align with each sample's inherent symmetry. Extensive experiments on super resolution, denoising, and deraining validate our theoretical findings and show significant superiority over standard baselines and traditional equivariant models. Our code and supplementary material are available at https://github.com/tanfy929/SA-Conv.

Feiyu Tan, Qi Xie, Zongben Xu +1

School of Mathematics and Statistics, Xi’an Jiaotong University, Xi’an, Shaanxi, China

Equivariant Neural Networks Image Restoration Image Inverse Problems Image Denoising

May 12, 2026

When Does Equivariance Help? Canonical Alignment in Neural Fluid Surrogates

Neural surrogates can accelerate computational fluid dynamics (CFD) simulations by orders of magnitude, but practical deployment in engineering and healthcare applications requires architectures that scale to high-resolution meshes and learn effectively from limited data. Explicit equivariance offers a principled inductive bias, yet its accuracy benefits may depend on the prediction task and the distribution of anatomical orientations. We investigate this dependence across three hemodynamic benchmarks with different degrees of natural canonical alignment. To support this study, we introduce the Anchored-Branched Geometric Algebra Transformer (AB-GATr), an

E(3)

-equivariant surrogate that efficiently predicts coupled surface and volume quantities. Across these benchmarks, AB-GATr consistently outperforms the evaluated non-equivariant models, including variants trained with rotational augmentation, while achieving accuracy competitive with

E(3)

-equivariant LaB-GATr at substantially lower training cost. In comparison, rotational augmentation provides inconsistent benefits across architectures and can reduce accuracy. A controlled experiment on ShapeNet-Car shows that strong canonical alignment can favor non-equivariant models, but their accuracy generally deteriorates as training orientations broaden and can decline sharply under broader test rotations. We further investigate these patterns using extended symmetry-breaking diagnostics and probes of the predictive information associated with canonical alignment across all benchmarks. Together, these results support explicit equivariance for the evaluated hemodynamic tasks with natural orientation variation, while showing that its accuracy benefits depend on the task and orientation distribution.

Patryk Rygiel, Julian Suk, Kak Khee Yeung +2

Department of Applied Mathematics Technical Medical Centre Cardiovascular Health Technology Centre University of Twente · Department of Computer Science Munich Center for Machine Learning Technical University of Munich · Department of Surgery Amsterdam UMC, Location University of Amsterdam Atherosclerosis & Aortic diseases Amsterdam Cardiovascular Sciences

Neural Surrogate Modeling Equivariant Neural Networks Computational Fluid Dynamics

May 11, 2026

Steerable Neural ODEs on Homogeneous Spaces

We introduce steerable neural ordinary differential equations on homogeneous spaces

M=G/H

. These models constitute a novel geometric extension of manifold neural ordinary differential equations (NODEs) that transport associated feature vectors transforming under the local symmetry group

H

. We interpret features as sections of associated vector bundles over

M

, and describe their evolution as parallel transport. This results in a coupled system of ODEs consisting of a flow equation on

M

and a steering equation acting on features. We show that steerable NODEs are

G

-equivariant whenever the vector field generating the flow and the connection governing parallel transport are both

G

-invariant. Furthermore, we demonstrate how steerable NODEs incorporate existing NODE models and continuous normalizing flows on Lie groups. Our framework provides the geometric foundation for learning continuous-time equivariant dynamics of general vector-valued features on homogeneous spaces.

Emma Andersdotter, Daniel Persson, Fredrik Ohlsson

Department of Mathematics and Mathematical Statistics Umeå University Umeå SE-901 87, Sweden · Department of Mathematical Sciences2026 Chalmers University of Technology and Gothenburg University Gothenburg SE-412 96, Sweden · Department of Mathematics and Mathematical Statistics 11 Umeå University Umeå SE-901 87, Sweden

Equivariant Neural Networks Geometric Representation Learning Neural ODEs

May 11, 2026

Equivariant Reinforcement Learning for Clifford Quantum Circuit Synthesis

We consider the problem of synthesizing Clifford quantum circuits for devices with all-to-all qubit connectivity. We approach this task as a reinforcement learning problem in which an agent learns to discover a sequence of elementary Clifford gates that reduces a given symplectic matrix representation of a Clifford circuit to the identity. This formulation permits a simple learning curriculum based on random walks from the identity. We introduce a novel neural network architecture that is equivariant to qubit relabelings of the symplectic matrix representation, and which is size-agnostic, allowing a single learned policy to be applied across different qubit counts without circuit splicing or network reparameterization. On six-qubit Clifford circuits, the largest regime for which optimal references are available, our agent finds circuits within one two-qubit gate of optimality in milliseconds per instance, and finds optimal circuits in 99.2% of instances within seconds per instance. After continued training on ten-qubit instances, the agent scales to unseen Clifford tableaus with up to thirty qubits, including targets generated from circuits with over a thousand Clifford gates, where it achieves lower average two-qubit gate counts than Qiskit's Aaronson-Gottesman and greedy Clifford synthesizers.

Richie Yeung, Aleks Kissinger, Rob Cornish

Department of Computer Science, University of Oxford · Quantinuum · College of Computing and Data Science, Nanyang Technological University, Singapore

Reinforcement Learning Quantum Circuit Compilation Equivariant Neural Networks Program Synthesis Quantum RL

May 8, 2026

Diagnosing Spectral Ceilings in Equivariant Neural Force Fields

We introduce a spectral-injection diagnostic for measuring which angular frequencies a trained equivariant force-field backbone preserves: inject a controlled angular-frequency perturbation into a molecular force field, attach a lightweight Spectral Prediction Network (SPN) to the frozen backbone, and read off which frequencies are recoverable. On aspirin, a quadratic SPN attached to an L = 2 NequIP backbone recovers the boundary signal at l = 4 but collapses at l = 5: a 11.7x cliff at the predicted drL boundary, with p dropping from 0.913 to 0.078. The same boundary-vs-above contrast persists across n = 4 independently trained backbones (raw-gain delta contrast, hierarchical cluster bootstrap) and is corroborated by a denominator-free injected-residual metric (R2_inj(4) = 0.374 versus R2_inj(5) = 0.006). A finite-degree span theorem calibrates the diagnostic: for a single marked direction, degree-d polynomials of degree-L spherical-harmonic features span exactly H less than or equal to dL with multiplicity-one saturation at the boundary (scoped to single-direction degree-bounded probes, not a function-class upper bound on multi-atom MPNNs). A synthetic C5 calibration plus capacity, activation, and cross-architecture controls rule out parameter count alone as the explanation.

Hyunmog Kim

Independent Researcher.

Machine Learning Interatomic Potentials Spectral Bias Equivariant Neural Networks Spectral Methods Neural Network Expressivity

May 5, 2026

Normalization Equivariance for Arbitrary Backbones, with Application to Image Denoising

Normalization Equivariance (NE) is a structural prior that improves robustness to distribution shift in image-to-image tasks. A function

f

is normalization equivariant iff

f(a y + b\mathbf{1}) = a f(y) + b\mathbf{1}

for all

a>0

and

b\in\mathbb{R}

. Existing NE methods constrain every internal layer to NE-compatible operations. These constraints add runtime cost and exclude standard transformer components such as softmax attention and LayerNorm. We introduce Wrapped Normalization Equivariance (WNE), a parameter-free wrapper that normalizes the input, applies any backbone, and denormalizes the output. We prove every NE function admits this factorization, so the wrapper exactly parameterizes the class of NE functions. On blind denoising, wrapping CNN and transformer architectures improves robustness under noise-level mismatch with no measurable GPU overhead, while architectural NE baselines are up to

1.6\times

slower.

Youssef Saied, François Fleuret

Department of Computer Science, University of Geneva, Switzerland · Meta FAIR.

Image Corruption Robustness Neural Network Robustness Equivariant Neural Networks Image Denoising

May 5, 2026

Symmetry, Defects, and Diffusion in Continuous-memory Recurrent Networks

Continuous-memory recurrent networks must preserve phase, position, or orientation despite model imperfections and state noise. We develop a geometric framework that separates three questions: how many memory coordinates are neutrally transported, how deterministic perturbations alter their finite-horizon stability, and how ambient noise is decoded along them. Exact per-input equivariance transports analytical group tangents pathwise and, on a compact nondegenerate orbit stratum, yields at least

q=\dim(G/H)

zero group-tangent Lyapunov exponents under stationary ergodic driving. For imperfect dynamics, a four-block tangent/normal decomposition gives local and finite-horizon bounds on tangent growth and subspace rotation, distinguishing first-order direct damage from second-order leakage through contracting normal directions. For noisy dynamics, a specified decoder maps ambient covariance

Q

to coordinate covariance

ZQZ^\top

; under isotropic noise and fixed tangent energy, least-squares decoding and scaled-isometric action geometry minimize local diffusion. Local and finite-horizon evaluations include cases both within and outside the sufficient conditions. In a fresh twenty-seed

T^2

replication, a decoder-covariance objective improves noisy horizon-256 memory in every pair while meeting a prespecified clean-error equivalence margin. Direct noise training also improves noisy memory but incurs a clean-error tradeoff. In coupled

T^4/T^8

integrators, the advantage of anisotropic-covariance over isotropic regularization reverses when evaluation noise becomes isotropic. These results connect continuous symmetry to measurable limits and design choices for recurrent memory under explicit dynamical, decoder, and noise assumptions.

Hanson Hanxuan Mo

Equivariant Neural Networks Recurrent Neural Networks

May 4, 2026

Augmented Equivariant Mesh Networks for Anatomical Segmentation

Anatomical mesh segmentation requires models that operate directly on irregular surface geometry while remaining robust to changes in coordinate pose across meshes of varying resolution. Existing task-specific mesh and point-cloud methods are not equivariant, and can degrade sharply under test-time perturbation, for example dropping by 28-30 IoU points on 3D-IOSSeg at

40^\circ

rotation even with matched training augmentation. We present EAMS, an Equivariant Anatomical Mesh Segmentor built on Equivariant Mesh Neural Networks (EMNN), and evaluate it in four dataset settings across three clinical application areas, spanning edge-, vertex-, and face-level supervision. We combine intrinsic mesh descriptors with anatomy-aware priors, including PCA-derived frames for dental arches and liver surfaces, and augment message passing to provide lightweight global context. Across intracranial aneurysm and intraoral segmentation, EAMS variants are competitive with specialized baselines on unperturbed inputs while remaining stable under geometric perturbations, and on liver surfaces they expose a favorable trade-off between canonical-pose accuracy and rotation robustness. These results show that a lightweight (

<2

M parameters) equivariant framework can deliver robust anatomical mesh segmentation across diverse label types.

Daniel Saragih

Department of Pathology and Molecular Medicine Queen’s University Kingston, ON K7L 3N6, Canada

Equivariant Neural Networks Medical Image Segmentation 3D Medical Image Segmentation

May 4, 2026

Isotropic Fourier Neural Operators

Fourier Neural Operators are deep learning models that learn mappings between function spaces and can be used to learn and solve partial differential equations (PDEs), in some cases significantly faster than traditional PDE solvers. Within the model are Fourier layers, which apply linear transformations directly to the Fourier modes, with parameters depending on the wave numbers. However, most physical systems are isotropic, with the results being independent of the coordinate system chosen, but the linear transformations do not necessarily respect these symmetries. We propose a modification to the linear transformations that ensures spatial symmetries are respected, called the Isotropic Fourier Neural Operator, which both improves model performance and reduces the number of parameters by up to a factor of 16 in 2D and 96 in 3D.

Michael F. Staddon

Independent Researcher

Fourier Neural Operator Neural PDE Solvers Equivariant Neural Networks PDE Operator Learning Neural Operators

Apr 29, 2026

Towards Accelerated SCF Workflows with Equivariant Density-Matrix Learning and Analytic Refinement

We present dm-PhiSNet, a physically constrained PhiSNet-based equivariant model that predicts one-electron reduced density matrices (1-RDMs) directly from molecular geometries in an atomic-orbital (AO) basis for accelerated self-consistent field (SCF) workflows. Training follows a two-stage schedule with progressively introduced physically motivated objectives, and the resulting predictions are refined by a lightweight analytic block. This block enforces electron-number conservation, drives the 1-RDM toward generalized idempotency in the AO metric, and regularizes the occupation spectrum of the Löwdin-orthogonalized density. Across six closed-shell systems -- H

_2

O, CH

_4

, NH

_3

, HF, ethanol, and NO

_3^-

-- the refined 1-RDMs provide SCF initial guesses that substantially reduce iteration steps by 49--81% relative to standard initializations. Beyond SCF acceleration, the learned 1-RDMs yield accurate one-shot total energies and Hellmann--Feynman atomic forces without force supervision, indicating that the model captures chemically meaningful electronic structure. These results demonstrate that combining equivariant learning with analytic constraint enforcement provides a simple, general route to solver-ready density-matrix initializations and accelerated SCF workflows.

Zuriel Y. Yescas-Ramos, Andrés Álvarez-García, Huziel E. Sauceda

Instituto de Física, Universidad Nacional Autónoma de México, Cd. de México C.P. 04510, Mexico

Quantum Chemistry Equivariant Neural Networks Molecular Representation Learning

Apr 27, 2026

Monocular Depth Estimation via Neural Network with Learnable Algebraic Group and Ring Structures

Monocular depth estimation (MDE) has witnessed remarkable progress driven by Convolutional Neural Networks and transformer-based architectures. However, these approaches typically treat the problem as a generic image-to-image regression on Euclidean grids, thereby overlooking the intrinsic algebraic and geometric structures induced by perspective projection. To address this limitation, we propose LAGRNet, a novel framework that fundamentally grounds MDE in algebraic geometry by explicitly embedding learnable group, ring, and sheaf structures into the deep learning pipeline. Modeling feature maps as sections of a sheaf over an approximated image manifold, our method first establishes a Group-defined Feature Manifold (GFM) parameterized by a learned algebraic group action to enforce projective equivariance and robustness against view changes. To facilitate algebraically consistent cross-scale interactions, we subsequently introduce a Ring Convolution Layer (RCL) that formulates feature fusion as a graded ring homomorphism. Furthermore, to ensure global topological consistency, a Sheaf-based Module (SM) aggregates local depth cues via Čech nerve on the image topology. Extensive zero-shot evaluations across the KITTI, NYU-Depth V2, and ETH3D benchmarks demonstrate that LAGRNet significantly outperforms state-of-the-art methods in both accuracy and generalization capabilities.

Qianlei Wang, Kexun Chen, Shaolin Zhang +3

Chengdu Institute of Computer Applications, Chinese Academy of Sciences, Chengdu 610213 China. · School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu 611756 China. · School of Mechanical Engineering, Southwest Jiaotong University, Chengdu 611756 China. +1

Monocular Depth Estimation Equivariant Neural Networks Sheaf Neural Networks

Apr 26, 2026

Quasi-Equivariant Metanetworks

Metanetworks are neural architectures designed to operate directly on pretrained weights to perform downstream tasks. However, the parameter space serves only as a proxy for the underlying function class, and the parameter-function mapping is inherently non-injective: distinct parameter configurations may yield identical input-output behaviors. As a result, metanetworks that rely solely on raw parameters risk overlooking the intrinsic symmetries of the architecture. Reasoning about functional identity is therefore essential for effective metanetwork design, motivating the development of equivariant metanetworks, which incorporate equivariance principles to respect architectural symmetries. Existing approaches, however, typically enforce strict equivariance, which imposes rigid constraints and often leads to sparse and less expressive models. To address this limitation, we introduce the novel concept of quasi-equivariance, which allows metanetworks to move beyond the rigidity of strict equivariance while still preserving functional identity. We lay down a principled basis for this framework and demonstrate its broad applicability across diverse neural architectures, including feedforward, convolutional, and transformer networks. Through empirical evaluation, we show that quasi-equivariant metanetworks achieve good trade-offs between symmetry preservation and representational expressivity. These findings advance the theoretical understanding of weight-space learning and provide a principled foundation for the design of more expressive and functionally robust metanetworks.

Viet-Hoang Tran, An Nguyen, Benoît Guérand +2

Department of Mathematics National University of Singapore

Equivariant Neural Networks

Apr 22, 2026

Neuro-evolutionary stochastic architectures in gauge-covariant neural fields

We extend our gauge-covariant stochastic neural-field framework by promoting architecture-level parameters to slow stochastic variables evolving in function space. Our effective theory is formulated in terms of classical commuting fields and provides symmetry-constrained diagnostics of marginality and finite-width effects through the maximal Lyapunov exponent, the amplification factor, and dressed spectral kernels. On top of this dynamics, we introduce a Markovian evolutionary scheme compatible with the local

U(1)

structure of the effective model. By using a minimal implementation, the genotype is reduced to the weight-variance parameter

σ_w^2

, and the fitness functional combines spectral agreement, marginal stability, and a symmetry-constrained critical anchor. Comparing three evolutionary models, we find that only the fully symmetry-constrained Ginibre

U(1)

version robustly approaches a narrow near-marginal regime and reproduces the predicted low-frequency finite-width spectral behavior. These results support the use of symmetry-guided effective stability diagnostics as practical principles for stochastic architecture search in controlled settings.

Rodrigo Carmo Terin

King Juan Carlos University, Facuty of Experimental Sciences and Technology, Department of Applied Physics, Av. del Alcalde de M´ostoles, 28933, Madrid, Spain

Evolutionary Optimization Equivariant Neural Networks Neural Architecture Search Neural Network Training Dynamics Neuroevolution

Apr 16, 2026

Learning Affine-Equivariant Proximal Operators

Proximal operators are fundamental across many applications in signal processing and machine learning, including solving ill-posed inverse problems. Recent work has introduced Learned Proximal Networks (LPNs), providing parametric functions that compute exact proximals for data-driven and potentially non-convex regularizers. However, in many settings it is important to include additional structure to these regularizers--and their corresponding proximals--such as shift and scale equivariance. In this work, we show how to obtain learned functions parametrized by neural networks that provably compute exact proximal operators while being equivariant to shifts and scaling, which we dub Affine-Equivariant Learned Proximal Networks (AE-LPNs). We demonstrate our results on synthetic, constructive examples, and then on real data via denoising in out-of-distribution settings. Our equivariant learned proximals enhance robustness to noise distributions and affine shifts far beyond training distributions, improving the practical utility of learned proximal operators

Oriel Savir, Zhenghan Fang, Jeremias Sulam

Mathematical Institute for Data Science, Johns Hopkins University, Baltimore, MD 21218, USA

Equivariant Neural Networks Operator Learning

Apr 14, 2026

Image-to-Image Translation Framework Embedded with Rotation Symmetry Priors

Image-to-image translation (I2I) is a fundamental task in computer vision, focused on mapping an input image from a source domain to a corresponding image in a target domain while preserving domain-invariant features and adapting domain-specific attributes. Despite the remarkable success of deep learning-based I2I approaches, the lack of paired data and unsupervised learning framework still hinder their effectiveness. In this work, we address the challenge by incorporating transformation symmetry priors into image-to-image translation networks. Specifically, we introduce rotation group equivariant convolutions to achieve rotation equivariant I2I framework, a novel contribution, to the best of our knowledge, along this research direction. This design ensures the preservation of rotation symmetry, one of the most intrinsic and domain-invariant properties of natural and scientific images, throughout the network. Furthermore, we conduct a systematic study on image symmetry priors on real dataset and propose a novel transformation learnable equivariant convolutions (TL-Conv) that adaptively learns transformation groups, enhancing symmetry preservation across diverse datasets. We also provide a theoretical analysis of the equivariance error of TL-Conv, proving that it maintains exact equivariance in continuous domains and provide a bound for the error in discrete cases. Through extensive experiments across a range of I2I tasks, we validate the effectiveness and superior performance of our approach, highlighting the potential of equivariant networks in enhancing generation quality and its broad applicability. Our code is available at https://github.com/tanfy929/Equivariant-I2I

Feiyu Tan, Heran Yang, Qihong Duan +3

School of Mathematics and Statistics, Xi’an Jiaotong University, Shaanxi, P.R.China · MOE Key Lab for Intelligent Networks and Networks Security, Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Shaanxi, P.R.China

Image-to-Image Translation Unpaired Image-to-Image Translation Equivariant Neural Networks Equivariant Representation Learning

Apr 9, 2026

Rotation Equivariant Convolutions in Deformable Registration of Brain MRI

Image registration is a fundamental task that aligns anatomical structures between images. While CNNs perform well, they lack rotation equivariance - a rotated input does not produce a correspondingly rotated output. This hinders performance by failing to exploit the rotational symmetries inherent in anatomical structures, particularly in brain MRI. In this work, we integrate rotation-equivariant convolutions into deformable brain MRI registration networks. We evaluate this approach by replacing standard encoders with equivariant ones in three baseline architectures, testing on multiple public brain MRI datasets. Our experiments demonstrate that equivariant encoders have three key advantages: 1) They achieve higher registration accuracy while reducing network parameters, confirming the benefit of this anatomical inductive bias. 2) They outperform baselines on rotated input pairs, demonstrating robustness to orientation variations common in clinical practice. 3) They show improved performance with less training data, indicating greater sample efficiency. Our results demonstrate that incorporating geometric priors is a critical step toward building more robust, accurate, and efficient registration models.

Arghavan Rezvani, Kun Han, Anthony T. Wu +2

University of California, Irvine

Medical Image Registration Deformable Image Registration Equivariant Neural Networks Magnetic Resonance Imaging Equivariant Representation Learning

Apr 7, 2026

Pixel-Translation-Equivariant Quantum Convolutional Neural Networks via Fourier Multiplexers

Convolutional neural networks owe much of their success to hard-coding translation equivariance. Quantum convolutional neural networks (QCNNs) have been proposed as near-term quantum analogues, but the relevant notion of translation depends on the data encoding. For address/amplitude encodings such as FRQI, a pixel shift acts as modular addition on an index register, whereas many MERA-inspired QCNNs are equivariant only under cyclic permutations of physical qubits. We formalize this mismatch and construct QCNN layers that commute exactly with the pixel cyclic shift (PCS) symmetry induced by the encoding. Our main technical result is a constructive characterization of all PCS-equivariant unitaries: conjugation by the quantum Fourier transform (QFT) diagonalizes translations, so any PCS-equivariant layer is a Fourier-mode multiplexer followed by an inverse QFT (IQFT). Building on this characterization, we introduce a deep PCS-QCNN with measurement-induced pooling, deferred conditioning, and inter-layer QFT cancellation. We also analyze trainability at random initialization and prove a lower bound on the expected squared gradient norm that remains constant in a depth-scaling regime, ruling out a depth-induced barren plateau in that sense.

Dmitry Chirkov, Igor Lobanov

School of Physics and Engineering, ITMO University, St. Petersburg, Russia

Quantum Convolutional Neural Networks Equivariant Neural Networks Quantum Circuit Trainability

Apr 1, 2026

Efficient Equivariant Transformer for Self-Driving Agent Modeling

Accurately modeling agent behaviors is an important task in self-driving. It is also a task with many symmetries, such as equivariance to the order of agents and objects in the scene or equivariance to arbitrary roto-translations of the entire scene as a whole; i.e., SE(2)-equivariance. The transformer architecture is a ubiquitous tool for modeling these symmetries. While standard self-attention is inherently permutation equivariant, explicit pairwise relative positional encodings have been the standard for introducing SE(2)-equivariance. However, this approach introduces an additional cost that is quadratic in the number of agents, limiting its scalability to larger scenes and batch sizes. In this work, we propose DriveGATr, a novel transformer-based architecture for agent modeling that achieves SE(2)-equivariance without the computational cost of existing methods. Inspired by recent advances in geometric deep learning, DriveGATr encodes scene elements as multivectors in the 2D projective geometric algebra

\mathbb{R}^*_{2,0,1}

and processes them with a stack of equivariant transformer blocks. Crucially, DriveGATr models geometric relationships using standard attention between multivectors, eliminating the need for costly explicit pairwise relative positional encodings. Experiments on the Waymo Open Motion Dataset demonstrate that DriveGATr is comparable to the state-of-the-art in traffic simulation and establishes a superior Pareto front for performance vs computational cost.

Scott Xu, Dian Chen, Kelvin Wong +3

Waabi · University of Toronto

Autonomous Driving Equivariant Neural Networks Autonomous Driving Simulation Autonomous Driving Perception

Mar 31, 2026

The Geometry of Polynomial Group Convolutional Neural Networks

We study polynomial group convolutional neural networks (PGCNNs) for an arbitrary finite group

G

. In particular, we introduce a new mathematical framework for PGCNNs using the language of graded group algebras. This framework yields two natural parameterizations of the architecture, based on Hadamard and Kronecker products, related by a linear map. We compute the dimension of the associated neuromanifold, verifying that it depends only on the number of layers and the size of the group. Furthermore, we show the general fiber of both parameterizations is trivial up to the regular group action and rescaling. Hence both parametrization maps are identifiable.

Yacoub Hendi, Daniel Persson, Magdalena Larfors

Uppsala University · Chalmers University of Technology

Convolutional Neural Networks Polynomial Neural Networks Equivariant Neural Networks Parameter Identifiability Neural Representation Geometry

Mar 27, 2026

Tunable Soft Equivariance with Guarantees

Equivariance is a fundamental property in computer vision models, yet strict equivariance is rarely satisfied in real-world data, which can limit a model's performance. Controlling the degree of equivariance is therefore desirable. We propose a general framework for constructing soft equivariant models by projecting the model weights into a designed subspace. The method applies to any pre-trained architecture and provides theoretical bounds on the induced equivariance error. Empirically, we demonstrate the effectiveness of our method on multiple pre-trained backbones, including ViT and ResNet, across image classification, semantic segmentation, and human-trajectory prediction tasks. Notably, our approach improves the performance while simultaneously reducing equivariance error on the competitive ImageNet benchmark.

Md Ashiqur Rahman, Lim Jun Hao, Jeremiah Jiang +2

Purdue University · DSO National Laboratories

Equivariant Neural Networks

Mar 25, 2026

Amortized Inference for Correlated Discrete Choice Models via Equivariant Neural Networks

Discrete choice models are fundamental tools in management science, economics, and marketing for understanding and predicting decision-making. Logit-based models are dominant in applied work, largely due to their convenient closed-form expressions for choice probabilities. However, they impose restrictive assumptions on the stochastic utility component, constraining our ability to capture realistic substitution patterns. We propose an amortized inference approach that relies on a neural network emulator to approximate choice probabilities for general error distributions, including those with correlated errors. We develop a specialized neural network architecture designed to respect the invariance properties of discrete choice models. We provide group-theoretic foundations for the architecture, including a proof of universal approximation given a minimal set of invariant features. Once trained, the emulator enables rapid likelihood evaluation and gradient computation. We use Sobolev training, augmenting the likelihood loss with a gradient-matching penalty, so that the emulator learns both choice probabilities and their derivatives. We show that emulator-based maximum likelihood estimators are consistent and asymptotically normal under mild approximation conditions, and we provide sandwich standard errors that remain valid for a psuedo-true parameter even with imperfect likelihood approximation. Simulations show significant gains over the GHK simulator in accuracy and speed.

Easton Huch, Michael Keane

Postdoctoral Fellow, Johns Hopkins Carey Business School · Carey Distinguished Professor, Johns Hopkins Carey Business School

Neural Surrogate Modeling Amortized Inference Equivariant Neural Networks Discrete Choice Modeling

Mar 22, 2026

ReDiffuse: Rotation Equivariant Diffusion Model for Multi-focus Image Fusion

Diffusion models have achieved impressive performance on multi-focus image fusion (MFIF). However, a key challenge in applying diffusion models to the ill-posed MFIF problem is that defocus blur can make common symmetric geometric structures (e.g., textures and edges) appear warped and deformed, often leading to unexpected artifacts in the fused images. Therefore, embedding rotation equivariance into diffusion networks is essential, as it enables the fusion results to faithfully preserve the original orientation and structural consistency of geometric patterns underlying the input images. Motivated by this, we propose ReDiffuse, a rotation-equivariant diffusion model for MFIF. Specifically, we carefully construct the basic diffusion architectures to achieve end-to-end rotation equivariance. We also provide a rigorous theoretical analysis to evaluate its intrinsic equivariance error, demonstrating the validity of embedding equivariance structures. ReDiffuse is comprehensively evaluated against various MFIF methods across four datasets (Lytro, MFFW, MFI-WHU, and Road-MF). Results demonstrate that ReDiffuse achieves competitive performance, with improvements of 0.28-6.64% across six evaluation metrics. The code is available at https://github.com/MorvanLi/ReDiffuse.

Bo Li, Tingting Bao, Lingling Zhang +3

School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, Shaanxi, China · School of Information Engineering, Chang’an University, Xi’an, Shaanxi, China

Multi-Source Image Fusion Diffusion Models Equivariant Neural Networks

Mar 19, 2026

Any-Subgroup Equivariant Networks via Symmetry Breaking

The inclusion of symmetries as an inductive bias, known as equivariance, often improves generalization on geometric data (e.g. grids, sets, and graphs). However, equivariant architectures are usually highly constrained, designed for symmetries chosen a priori, and not applicable to datasets with other symmetries. This precludes the development of flexible, multi-modal foundation models capable of processing diverse data equivariantly. In this work, we build a single model -- the Any-Subgroup Equivariant Network (ASEN) -- that can be simultaneously equivariant to several groups, simply by modulating a certain auxiliary input feature. In particular, we start with a fully permutation-equivariant base model, and then obtain subgroup equivariance by using a symmetry-breaking input whose automorphism group is that subgroup. However, finding an input with the desired automorphism group is computationally hard. We overcome this by relaxing from exact to approximate symmetry breaking, leveraging the notion of 2-closure to derive fast algorithms. Theoretically, we show that our subgroup-equivariant networks can simulate equivariant MLPs, and their universality can be guaranteed if the base model is universal. Empirically, we validate our method on symmetry selection for graph and image tasks, as well as multitask and transfer learning for sequence tasks, showing that a single network equivariant to multiple permutation subgroups outperforms both separate equivariant models and a single non-equivariant model.

Abhinav Goel, Derek Lim, Hannah Lawrence +2

MIT · TUM · Flatiron Institute

Equivariant Neural Networks Equivariant GNNs Symmetry Breaking

Mar 19, 2026

An SO(3)-equivariant reciprocal-space neural potential for long-range interactions

Long-range electrostatic and polarization interactions play a central role in molecular and condensed-phase systems, yet remain fundamentally incompatible with locality-based machine-learning interatomic potentials. Although modern SO(3)-equivariant neural potentials achieve high accuracy for short-range chemistry, they cannot represent the anisotropic, slowly decaying multipolar correlations governing realistic materials, while existing long-range extensions either break SO(3) equivariance or fail to maintain energy-force consistency. Here we introduce EquiEwald, a unified neural interatomic potential that embeds an Ewald-inspired reciprocal-space formulation within an irreducible SO(3)-equivariant framework. By performing equivariant message passing in reciprocal space through learned equivariant k-space filters and an equivariant inverse transform, EquiEwald captures anisotropic, tensorial long-range correlations without sacrificing physical consistency. Across periodic and aperiodic benchmarks, EquiEwald captures long-range electrostatic behavior consistent with ab initio reference data and consistently improves energy and force accuracy, data efficiency, and long-range extrapolation. These results establish EquiEwald as a physically principled paradigm for long-range-capable machine-learning interatomic potentials.

Lingfeng Zhang, Taoyong Cui, Dongzhan Zhou +6

The Chinese University of Hong Kong, Hong Kong, 999077, China. · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, 999077, China. · Shanghai Artificial Intelligence Laboratory, Shanghai, 200232, China. +1

Machine Learning Interatomic Potentials Computational Materials Science Equivariant Neural Networks

Mar 18, 2026

Learning Coordinate-based Convolutional Kernels for Continuous SE(3) Equivariant and Efficient Point Cloud Analysis

A symmetry on rigid motion is one of the salient factors in efficient learning of 3D point cloud problems. Group convolution has been a representative method to extract equivariant features, but its realizations have struggled to retain both rigorous symmetry and scalability simultaneously. We advocate utilizing the intertwiner framework to resolve this trade-off, but previous works on it, which did not achieve complete SE(3) symmetry or scalability to large-scale problems, necessitate a more advanced kernel architecture. We present Equivariant Coordinate-based Kernel Convolution, or ECKConv. It acquires SE(3) equivariance from the kernel domain defined in a double coset space, and its explicit kernel design using coordinate-based networks enhances its learning capability and memory efficiency. The experiments on diverse point cloud tasks, e.g., classification, pose registration, part segmentation, and large-scale semantic segmentation, validate the rigid equivariance, memory scalability, and outstanding performance of ECKConv compared to state-of-the-art equivariant methods.

Jaein Kim, Hee Bin Yoo, Dong-Sig Han +1

Interdisciplinary Program in Neuroscience, Seoul National University · Département d’Informatique, École Normale Supérieure (ENS) · Department of Computing, Imperial College London +1

Equivariant Neural Networks Point Cloud Learning

Mar 17, 2026

Accurate Shift Invariant Convolutional Neural Networks Using Gaussian-Hermite Moments

The convolutional neural networks (CNNs) are not inherently shift invariant or equivariant. The downsampling operation, used in CNNs, is one of the key reasons which breaks the shift invariant property of a CNN. Conversely, downsampling operation is important to improve computational efficiency and increase the area of the receptive field for more contextual information. In this work, we propose Gaussian-Hermite Sampling (GHS), a novel downsampling strategy designed to achieve accurate shift invariance. GHS leverages Gaussian-Hermite polynomials to perform shift-consistent sampling, enabling CNN layers to maintain invariance to arbitrary spatial shifts prior to training. When integrated into standard CNN architectures, the proposed method embeds shift invariance directly at the layer level without requiring architectural modifications or additional training procedures. We evaluate the proposed approach on CIFAR-10, CIFAR-100, and MNIST-rot datasets. Experimental results demonstrate that GHS significantly improves shift consistency, achieving 100% classification consistency under spatial shifts, while also improving classification accuracy compared to baseline CNN models.

Jaspreet Singh, Petra Bosilj, Grzegorz Cielniak

Convolutional Neural Networks Equivariant Neural Networks Image Classification

Mar 16, 2026

Gauge-Equivariant Intrinsic Neural Operators for Geometry-Consistent Learning of Elliptic PDE Maps

Learning solution operators of partial differential equations (PDEs) from data has emerged as a promising route to fast surrogate models in multi-query scientific workflows. However, for geometric PDEs whose inputs and outputs transform under changes of local frame (gauge), many existing operator-learning architectures remain representation-dependent, brittle under metric perturbations, and sensitive to discretization changes. We propose Gauge-Equivariant Intrinsic Neural Operators (GINO), a class of neural operators that parameterize elliptic solution maps primarily through intrinsic spectral multipliers acting on geometry-dependent spectra, coupled with gauge-equivariant nonlinearities. This design decouples geometry from learnable functional dependence and enforces consistency under frame transformations. We validate GINO on controlled problems on the flat torus (

\mathbb{T}^2

), where ground-truth resolvent operators and regularized Helmholtz--Hodge decompositions admit closed-form Fourier representations, enabling theory-aligned diagnostics. Across experiments E1--E6, GINO achieves low operator-approximation error, near machine-precision gauge equivariance, robustness to structured metric perturbations, strong cross-resolution generalization with small commutation error under restriction/prolongation, and structure-preserving performance on a regularized exact/coexact decomposition task. Ablations further link the smoothness of the learned spectral multiplier to stability under geometric perturbations. These results suggest that enforcing intrinsic structure and gauge equivariance yields operator surrogates that are more geometry-consistent and discretization-robust for elliptic PDEs on form-valued fields.

Pengcheng Cheng

School of Mathematics, Jilin University, Changchun, 130012, China

PDE Operator Learning Neural Operators PDE Solving Equivariant Neural Networks

Mar 12, 2026

Bases of Steerable Kernels for Equivariant CNNs: From 2D Rotations to the Lorentz Group

We present an alternative way of solving the steerable kernel constraint that appears in the design of steerable equivariant convolutional neural networks. We find explicit real and complex bases which are ready to use, for different symmetry groups and for feature maps of arbitrary tensor type. A major advantage of this method is that it bypasses the need to numerically or analytically compute Clebsch-Gordan coefficients and works directly with the representations of the input and output feature maps. The strategy is to find a basis of kernels that respect a simpler invariance condition at some point

x_0

, and then steer it with the defining equation of steerability to move to some arbitrary point

x=g\cdot x_0

. This idea has already been mentioned in the literature before, but not advanced in depth and with some generality. Here we describe how it works with minimal technical tools to make it accessible for a general audience.

Alan Garbarz

Universidad de Buenos Aires, Facultad de Ciencias Exactas y Naturales, Departamento de F´ısica. Ciudad Universitaria, pabell´on 1, 1428, Buenos Aires, Argentina. · CONICET - Universidad de Buenos Aires, Instituto de F´ısica de Buenos Aires (IFIBA). Ciudad Universitaria, pabell´on 1, 1428, Buenos Aires, Argentina.

Equivariant Neural Networks Kernel Methods