Equivariant Representation Learning

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 30

All topics

How Do Transformers Learn to Represent Symmetries?
Oct 7, 2026Eduardo Santos-Escriche, Valerie Engelmayer, Ya-Wei Eileen Lin +1Invariant Representation Learning Transformer
Technical University of Munich Munich Center for Machine Learning · Massachusetts Institute of Technology
PEACE: Covariant learning of nonadiabatic manifolds with parity-resolved Hamiltonians
Oct 7, 2026Rongzhi Gao, Shuguang Chen, Yang Zhou +3Molecular Dynamics Simulation Quantum Chemistry
Department of Chemistry, The University of Hong Kong, Pok Fu Lam, Hong Kong SAR, China · Hong Kong Quantum AI Lab, Pak Shek Kok, Hong Kong SAR, China · MattVerse Limited, Pak Shek Kok, Hong Kong SAR, China +1
Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models
Oct 6, 2026Jivan Waber, Vanessa Piccolo, Yatin Dandi +1Representation Learning Data Augmentation
Information, Learning and Physics Laboratory École Polytechnique Fédérale de Lausanne (EPFL) · Statistical Physics of Computation Laboratory École Polytechnique Fédérale de Lausanne (EPFL)
REViT-v2: Hierarchical Windowed Roto-reflection Equivariant ViT for Equivariant Feature Extraction
Oct 6, 2026Sheir A. Zaheer, Jihwan Moon, Chan Y. ParkVision Transformer Equivariant Neural Networks
KC Machine Learning Lab, Rep. of Korea · NFOCZ Inc and Seoul National University, Rep. of Korea
From Isolated Feature to Orbits: Discovering Music Concepts via Multi-SAE Alignment
Oct 1, 2026Liwei Lin, Gus XiaRepresentation Learning Sparse Autoencoders
Computer Science Department, NYU Shanghai · Machine Learning Department, MBZUAI
Reformulation-Contrastive Learning for Mixed Integer Programs
Sep 30, 2026Ousema Bouaneni, Mathis Le Bail, Clément Elliker +2Contrastive Learning Representation Learning
LIX (École Polytechnique, IP Paris, CNRS) · AMIAD (Agence Ministérielle pour l’IA de Défense)
Graded Representation Theory of Equivariant Neural Networks
Sep 22, 2026Mani ShayestehfarEquivariant Neural Networks Equivariant Representation Learning
BiRoAD: Learning Shared and Role-Adaptive Representations for Bimanual Manipulation
Sep 20, 2026Yan Shen, Yuchen Liu, Feng Jiang +5Representation Learning Bimanual Robotic Manipulation
CFCS, School of Computer Science, Peking University
Equivariance Breaks the Learning Rate
Sep 8, 2026Andrei Manolache, Mathias NiepertEquivariant Neural Networks Equivariant Representation Learning
University of Stuttgart, Germany · Bitdefender, Romania
Pix2Rep-v2: Data-Efficient Representation Learning for Dense Medical Imaging Applications
Sep 1, 2026S. Sifaoui, E. Angelini, S. Toupin +2Representation Learning Visual Representation Learning
LTCI, Télécom Paris, Institut Polytechnique de Paris, Palaiseau, France · MIRACL.ai Laboratory, Hôpital Universitaire Lariboisière (AP-HP), Paris, France · Université Paris Cité, Inserm MASCOT, Paris, France
Rotational Equivariance in Machine Learning: A Comprehensive Tutorial
Aug 31, 2026Peter Lippmann, Fred A. HamprechtEquivariant Neural Networks Geometric Representation Learning
Interdisciplinary Center for Scientific Computing (IWR), Heidelberg University, 69120 Heidelberg, Germany
No Equivariant Architecture Covers All Equivariant Attention
Aug 31, 2026Tīkun ÔngAttention Head Analysis Equivariant Neural Networks
Equivariant Sheaf Neural Networks: Learning Geometric Transport on Graphs
Aug 28, 2026Alessio Borgi, Mario Severino, Fabrizio Silvestri +1Equivariant Neural Networks Sheaf Neural Networks
Department of Computer Science and Technology, University of Cambridge · Department of Computer, Control and Management Engineering, Sapienza University of Rome · Department of Information Engineering, University of Padua
Reducing Symmetry Increase in Equivariant Neural Networks
Aug 12, 2026Ning Lin, Jiacheng Cen, Anyi Li +2Equivariant Neural Networks Equivariant Representation Learning
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
Equivariant Music Transformer
Aug 4, 2026Zixun Guo, Simon DixonRepresentation Learning Transformer
Centre for Digital Music, Queen Mary University of London, United Kingdom
Generator-Aligned Representation Interfaces for Diagnostic Soft Equivariance
Jul 28, 2026Weitao Li, Gong ChengEquivariant Neural Networks Equivariant Representation Learning
School of Mathematical Sciences Tongji University Shanghai 200092, China · School of Mathematical Sciences Key Laboratory of Intelligent Computing and Applications (Ministry of Education) Tongji University Shanghai 200092, China
GEqTrain: A Configuration-Driven Framework for Retargeting Equivariant Graph Neural Networks Across 3D Scientific Tasks
Jul 21, 2026Daniele Angioletti, Marco Nobile, Vittorio LimongelliFlow Matching Generative Modeling
Faculty of Biomedical Sciences, Euler Institute, Universit´a della Svizzera italiana, 6900 Lugano, Switzerland
MIDI-RAE-JEPA: Hierarchical Representation Learning and Generation for Symbolic Music
Jul 16, 2026Scott H. HawleyRepresentation Learning Self-Supervised Pre-Training
Department of Chemistry & Physics, Belmont University, Nashville, TN, USA
When Do Geometric Algebra Layers Beat Scalarization? A Controlled Study on SO(3)-Equivariant Vector Laws
Jul 7, 2026Fabien PollyEquivariant Neural Networks Equivariant Representation Learning
Foundations of Equivariant Deep Learning: Unifying Graph and Sheaf Neural Networks
Jul 4, 2026Yoshihiro MaruyamaGraph Neural Networks Equivariant Neural Networks
Department of Mathematical and Information Sciences, Kyoto University, Kyoto, Japan.
A Unified Framework for Vision Transformers Equivariant to Discrete Subgroups of $\mathrm{O}(2)$
Jun 26, 2026Tīkun Ông, Georg BökmanVision Transformer Equivariant Neural Networks
Independent Researcher · University of Amsterdam
Equivariant Neural Belief Propagation
Jun 4, 2026Zehua Cheng, Wei Dai, Jiahao SunFactor Graph Optimization Gaussian Mixture Models
Department of Computer Science, University of Oxford, Oxford, United Kingdom · FLock.io, London, United Kingdom
Exact equivariance, kept through training, buys zero-shot generalisation across the symmetry group
Jun 2, 2026Hongbo WangWorld Models for Robotics World Models
Department of Mathematics, Stony Brook University, Stony Brook, NY 11794, USA
SSR: Scaling Surefooted and Symmetric Humanoid Traversal to the Open World
May 29, 2026Ruiqi Yu, Yiwen Wang, Yuan Hao +2Humanoid Robot Locomotion Terrain-Aware Robot Locomotion
Zhejiang University
On the Equivariant Learning of the $Q$ -tensor Order Parameter
May 26, 2026Julia Navarro, Mark WilkinsonEquivariant Neural Networks Materials Science
Nottingham Trent University, UK · Berea College, USA
How the Optimizer Shapes Learned Solutions in Equivariant Neural Networks
May 26, 2026Teodor-Mihai Stupariu, Andrei ManolacheNeural Representation Geometry Neural Network Optimization
1Bitdefender, Romania · 2Tudor Vianu High School of Com · University of Stuttgart, Germany +1
Group-Algebraic Tensors: Provably-optimal Equivariant Learning and Physical Symmetry Discovery
May 19, 2026Paulina Hoyos, Shashanka Ubaru, Dongsung Huh +5Scientific Discovery Equivariant Neural Networks
UT Austin · IBM Research · Independent +2
PHALAR: Phasors for Learned Musical Audio Representations
May 5, 2026Davide Marincione, Michele Mancusi, Giorgio Strano +4Audio Representation Learning Representation Learning
Department of Computer Science, Sapienza University of Rome, Italy · Moises Systems, Inc. · Paradigma, Inc.
L-GTA: Latent Generative Modeling for Time Series Augmentation
Jul 31, 2025Luis Roque, Vitor Cerqueira, Carlos Soares +1Variational Autoencoders Synthetic Data Augmentation
Faculdade de Engenharia da Universidade do Porto, Porto, Portugal · University of Coimbra, Coimbra, Portugal · Fraunhofer Portugal AICOS, Portugal +1
Representation Learning for Equivariant Inference with Guarantees
May 26, 2025Daniel Ordoñez-Apraez, Vladimir Kostić, Alek Fröhlich +3Disentangled Representation Learning Representation Learning
CSML, Italian Institute of Technology, Genova, Italy · Universit`a di Genova, Genova, Italy · University of Novi Sad, Novi Sad, Serbia +2

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

How Do Transformers Learn to Represent Symmetries?

Eduardo Santos-Escriche, Valerie Engelmayer, Ya-Wei Eileen Lin, Stefanie Jegelka

Technical University of Munich Munich Center for Machine Learning · Massachusetts Institute of Technology

Invariant Representation Learning Transformer Equivariant Representation Learning Data Augmentation

Open paper AskPDF arXiv

Training Transformer-based architectures with finite data augmentation has become an increasingly popular approach in geometric machine learning. Despite its empirical success, the interplay between the Transformer architecture, invariance to different symmetries, and augmentation budgets remains underexplored. In this paper, we study the ability of a vanilla Transformer to learn various symmetries through finite data augmentation for point cloud datasets. We identify an ordering of increasing learnability across the following symmetry groups: (i) non-angle-preserving symmetries, (ii) angle-preserving symmetries, and (iii) base angle-preserving subgroups, such as translation, rotation, and scale. For the base angle-preserving groups, we further investigate the Transformer's extrapolation behavior and conduct a structural analysis of the trained models, allowing us to identify interpretable mechanisms that induce invariance. Finally, we extend our analysis to equivariant functions and show that the detected mechanisms for approximate invariance can also provide a key building block for learned equivariance. Our project page is available at https://transformers-learn-symmetries.github.io/

Momentum

Related topics

Latest papers 30