Modular Arithmetic

Ask about this topic

Overview

This area examines how neural networks learn operations such as addition and multiplication on residues modulo an integer, offering a controlled setting for studying symbolic reasoning and generalization. Common approaches use Fourier and character transforms, cyclic or algebraically structured embeddings, and circuit analysis to identify how models encode and compute these operations. Small modular-arithmetic benchmarks are also used to study grokking, where test performance can emerge after a period of memorization. A central tension is that generalization depends on data coverage, architecture, and optimization, while changes in representations may lag behind accuracy or fail to explain readout-level failures.

Latest papers

CardsList

14 papers0 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

  1. Post-Grokking Collapse at the Representation-Readout Interface in Muon-Trained Transformers

    Aug 7, 2026Ali Janati, Kaoutar El Maghraoui, Anass BelfatmiTransformerMuon Optimizer

    Data Science Institute, Columbia University, New York, NY, USA · Department of Computer Science, Columbia University, New York, NY, USA · CentraleSupélec, Gif-sur-Yvette, France

  2. Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations

    Jul 15, 2026Chon-Fai Kam, Xavier Cadet, Miloud Bessafi +1Feedforward Neural NetworksNeural Network Expressivity

    University Paris City & University of Reunion, Paris, France · Dipartimento di Fisica e Chimica Emilio Segrè, Università degli Studi di Palermo, Palermo, Italy · Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, USA +2

  3. Multiplication Beyond Groups: Stratified Fourier Mechanisms in Transformer Circuits

    Jul 8, 2026Zitong Andrew Chen, Junaid Hasan, Akhil Srinivasan +2Transformer InterpretabilityTransformer Attention

    1UW Math AI Lab, University of Washington.

  4. At-Grok Is Not Converged:A Measurement-Validity Audit for Grokking Representation Metrics

    Jul 7, 2026Truong Xuan KhanhTransformerModular Arithmetic

    H&K Research Studio / Clevix LLC, Hanoi, Vietnam

  5. Grokking Is Conditional and Fragile: A Fully-Tractable, Multi-Seed Study at 12K Parameters

    Jul 6, 2026Yoshiyuki OotaniPhase TransitionsML Reproducibility

    Independent Researcher

  6. Prime Fourier Embeddings: A Principled Basis for Modular Arithmetic

    Jun 22, 2026Hyunsang Hwang, Suhyun Bae, Donghun LeeFourier Feature EmbeddingsRepresentation Learning

    Department of Mathematics, Korea University, Seoul, South Korea.

  7. The Discrete-Log Clock: How a Transformer Learns Modular Multiplication

    Jun 16, 2026Huu Danh NguyenTransformer InterpretabilityFrequency-Domain Feature Learning

    Stanford University

  8. Beyond Neural Collapse: Task-Intrinsic Geometry Governs Neural Representations in Modular Arithmetic

    Jun 8, 2026Hu Tan, Kuo Gai, Shihua ZhangNeural Representation GeometryClassification

    Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; and School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China · Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS), Shanghai, China · Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100049, China; and Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Chinese Academy of Sciences, Hangzhou 310024, China

  9. BRo-JEPA: Learning Modular Arithmetic in Latent Space

    May 31, 2026Divyansh Jha, Yuanfang Xie, Varan Mehra +1Zero-Shot LearningJoint-Embedding Predictive Architecture

    1Georgia Institute of Technology · 2NYU Langone Health

  10. Unveiling Memorization-Generalization Coexistence: A Case Study on Arithmetic Tasks with Label Noise

    May 18, 2026Linyu Liu, Pinyan LuNeural Network GeneralizationNeural Network Memorization

    1Taylor Lab, Huawei Technologies Co., Ltd. · 2Key Laboratory of Interdisciplinary Research of Computation and Economics, Shanghai University of Finance and Economics

  11. Learning Large-Scale Modular Addition with an Auxiliary Modulus

    May 8, 2026Hanato Kikuchi, Ryosuke Masuya, Kazuhiko Kawamoto +1Boolean Function LearningDistribution Shift

    Chiba University · National Institute of Informatics

  12. Universal Quantum Transformer

    Apr 29, 2026Sungyong Chung, Alireza TalebpourVariational Quantum CircuitsModular Arithmetic

    Grainger College of Engineering, Department of Civil and Environmental Engineering, University of Illinois Urbana-Champaign

  13. Feature Repulsion and Spectral Lock-in: An Empirical Study of Two-Layer Network Grokking

    Apr 28, 2026Yongzhong XuNeural Network Training DynamicsModular Arithmetic

  14. Convergent Evolution: How Different Language Models Learn Similar Number Representations

    Apr 22, 2026Deqing Fu, Tianyi Zhou, Mikhail Belkin +2Fourier Feature EmbeddingsFrequency-Domain Feature Learning

    University of Southern California · UC San Diego