Modular Addition

Latest papers

12 papers2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Nonconformal Distortion and Local Orientation Reversal in the Fourier Circuit of a Modular-Addition Transformer
Oct 30, 2026Eduardo Fuentes, Gonzalo Riera, Héctor Allende-CidNeural Representation Geometry Mechanistic Interpretability
AI Mechanisms Lab, Chile · WriteWise, Chile · Facultad de Matemáticas, Pontificia Universidad Católica de Chile, Santiago, Chile +2
Probability-Signature Dynamics: Unpacking Modular Addition Learning Within Two-Layer Networks
Oct 8, 2026Yunji Wang, Junjie Yao, Linyu Liu +2Neural Network Training Dynamics Frequency-Domain Feature Learning
School of Mathematical Sciences, Shanghai Jiao Tong University · Huawei Technologies Ltd · Shanghai University of Finance and Economics +1
A Spectral Theory of Grokking: Weight Decay induces Feature Learning
Sep 22, 2026Lenz Pracher, Pascal de Jong, Oskar Lieshaus +2Neural Network Generalization Weight Decay
Arnold-Sommerfeld-Center for Theoretical Physics, Ludwig-Maximilians-Universität München, Munich, Germany · Department of Applied Physics, Standford University, Standford, CA, USA · Department of Mathematics, University of Cambridge, Cambridge, UK
Symmetry without a manifold: intrinsic dimension on orbits
Sep 15, 2026Chon-Fai Kam, Miloud Bessafi, Frédéric CadetIntrinsic Dimensionality Modular Addition
Dipartimento di Fisica e Chimica, Universit`a degli Studi di Palermo, via Archirafi 36, I-90123 Palermo, Italy · EnergyLab, Universit´e de La R´eunion, F-97715 Saint-Denis, France · Universit´e Paris Cit´e and Universit´e de La R´eunion, BIGR, INSERM UMR S1134, F-75014 Paris, France +1
A Counterexample to Fourier Alignment in Single-Neuron Modular Addition
Aug 5, 2026Gautam Neelakantan MemanaShallow Neural Networks ReLU Neural Networks
Beyond Neural Collapse: Task-Intrinsic Geometry Governs Neural Representations in Modular Arithmetic
Jun 8, 2026Hu Tan, Kuo Gai, Shihua ZhangNeural Representation Geometry Classification
Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; and School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China · Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS), Shanghai, China · Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China; and Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Chinese Academy of Sciences, Hangzhou 310024, China
BRo-JEPA: Learning Modular Arithmetic in Latent Space
May 31, 2026Divyansh Jha, Yuanfang Xie, Varan Mehra +1Zero-Shot Learning Joint-Embedding Predictive Architecture
1Georgia Institute of Technology · 2NYU Langone Health
Assign and Add: A Mechanistic Study of Compositional Arithmetic
May 29, 2026Brady Exoo, Alberto Bietti, John SousTransformer Interpretability Transformer
Yale University · Flatiron Institute
Learning Large-Scale Modular Addition with an Auxiliary Modulus
May 8, 2026Hanato Kikuchi, Ryosuke Masuya, Kazuhiko Kawamoto +1Boolean Function Learning Distribution Shift
Chiba University · National Institute of Informatics
A Mechanistic Study of Algorithmic Generalization in Diffusion Models on Modular Arithmetic
Apr 20, 2026Joon Hyeok Kim, Yong-Hyun Park, Mattis Dalsætra Østby +1Pixel-Space Diffusion Models Mechanistic Interpretability
Department of Computer and Information Science, University of Pennsylvania, Philadelphia, PA 19104, USA
On the Mechanism and Dynamics of Modular Addition: Fourier Features, Lottery Ticket, and Grokking
Feb 18, 2026Jianliang He, Leda Wang, Siyu Chen +1Lottery Ticket Hypothesis Modular Addition
Department of Statistics and Data Science, Yale University
A Mechanistic Study of Transformers Training Dynamics
Oct 31, 2024Ambroise Odonnat, Wassim Bouaziz, Vivien CabannesTransformer Interpretability Attention Mechanisms
Inria, Univ. Rennes 2 · Mistral AI; work done while at Meta. · FAIR at Meta.

Oct 30, 2026J/K move · Enter open · S save

Nonconformal Distortion and Local Orientation Reversal in the Fourier Circuit of a Modular-Addition Transformer

Eduardo Fuentes, Gonzalo Riera, Héctor Allende-Cid

AI Mechanisms Lab, Chile · WriteWise, Chile · Facultad de Matemáticas, Pontificia Universidad Católica de Chile, Santiago, Chile+2

Neural Representation Geometry Mechanistic Interpretability Modular Addition

Open paper AskPDF arXiv

We study the local differential geometry of a Fourier-based modular-addition circuit in a one-layer transformer whose algorithm has previously been reverse-engineered. For each of the five key Fourier frequencies, we restrict the model's multilayer perceptron (MLP) contribution to a corresponding Fourier input plane and readout, and measure its directional Beltrami coefficient under continuous input perturbations. Whereas nondegenerate restrictions of ideal complex multiplication are conformal, the trained maps exhibit nonconformal distortion (median

|μ|=0.304

0.446

), spatial organization in the Beltrami argument, and regions of local orientation reversal. The median across frequencies decreases from

0.939

at initialization to

0.390

after training. The key-frequency planes also show lower median distortion than most random-plane controls, although their distributions overlap and the coordinate frames differ in conditioning. Across 500 checkpoints in one training run, most of the aggregate reduction in Beltrami magnitude occurs before the principal increase in the measured Fourier-circuit formation. At the individual-input level, Beltrami magnitude shows negligible association with the correct-logit margin (

|ρ|\leq0.026

). Late-checkpoint measurements across six runs show comparable distortion for most selected frequencies. These results provide a quantitative characterization of local deformation in a frequency-projected component of an identified neural circuit, complementary to its algorithmic reconstruction.