Neural Network Expressivity

Ask about this topic

Momentum

7 papers in the last four weeks, up 40% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 92

All topics
CardsList

  1. Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity

    Sep 28, 2026Zilan Cheng, Li-Lian Wang, Zhongjian WangNeural Network Approximation TheoryFeedforward Neural Networks

    Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University, 637371, Singapore.

  2. The Computational Value of Sensory-Aligned Receptive Fields Depends on Neuronal Expressivity

    Sep 22, 2026Agnese Adorante, Aaron Spieler, Anna LevinaRecurrent Neural NetworksNeural Network Expressivity

    Department of Computer Science, University of Tübingen, Tübingen, Germany · Max Planck Institute for Biological Cybernetics, Tübingen, Germany

  3. Neural Approximation by Function Composition: Rigidity and Doubly Exponential Convergence

    Sep 22, 2026Wentao Huang, Haizhang ZhangNeural Network Approximation TheoryNeural Network Expressivity

    Department of Mathematics, The University of Hong Kong, Hong Kong, P.R. China. · School of Mathematics, Sun Yat-sen University, Guangzhou, P.R. China.

  4. Graded Representation Theory of Equivariant Neural Networks

    Sep 22, 2026Mani ShayestehfarEquivariant Neural NetworksEquivariant Representation Learning

  5. Complex KDA: Understanding and Enhancing the Expressivity of Kimi Delta Attention

    Sep 21, 2026Julien Siems, Riccardo Grazzi, Korbinian Pöppel +8Length GeneralizationDelta-Rule Linear Attention

  6. Polyhedral Geometry of Time-to-First-Spike Neural Networks

    Sep 11, 2026Manjot Singh, Guido Montúfar, Gitta KutyniokSpiking Neural NetworksFeedforward Neural Networks

    Department of Mathematics, Ludwig-Maximilians-Universität München · Munich Center for Machine Learning · Departments of Mathematics and Statistics & Data Science, University of California, Los Angeles +2

  7. Diversity of EML-type operators

    Sep 10, 2026Andrzej OdrzywołekNeural Network Activation FunctionsNeural Network Expressivity

    Institute of Theoretical Physics, Jagiellonian University, Łojasiewicza 11, 30-348 Kraków, Poland

  8. Sharp Approximation Rates for Neural Networks with Affine Latent Parameterizations

    Aug 31, 2026Shijun ZhangHypernetworksNeural Network Approximation Theory

    Department of Applied Mathematics Hong Kong Polytechnic University

  9. Every Layer Counts: An Exponential L2L_2 Depth Hierarchy for ReLU Networks

    Aug 24, 2026Itay SafranShallow Neural NetworksReLU Neural Networks

    Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev

  10. The Boolean Power of ReLU

    Aug 12, 2026Pablo Barceló, Floris Geerts, Matthias Lanzinger +2ReLU Neural NetworksGNN Expressivity

    Pontifical Catholic University · IMFD & CENIA, Chile · University of Antwerp +4

  11. Reducing Symmetry Increase in Equivariant Neural Networks

    Aug 12, 2026Ning Lin, Jiacheng Cen, Anyi Li +2Equivariant Neural NetworksEquivariant Representation Learning

    Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China

  12. Three Tokens Force Exponential Feature Rank in Nonnegative Kernel Attention

    Aug 11, 2026Vicente OpazoLow-Rank AttentionAttention Mechanisms

    CENIA

  13. The Spectral Neuron

    Aug 8, 2026Alex ShtoffNeural Network RobustnessNeural Network Interpretability

    Technology Innovation Institute

  14. Neural network realization of binary refinement iterates via a two-chart atlas selector

    Jul 21, 2026Tsogtgerel GantumurReLU Neural NetworksNeural Network Approximation Theory

    McGill University, Montr´eal, QC, Canada · National University of Mongolia, Ulaanbaatar, Mongolia · Mongolian Academy of Sciences, Institute of Mathematics and Digital Technology

  15. Spectral Higher-Order Neural Networks Have Sharp Expressivity Bounds

    Jul 21, 2026Gianluca Peri, Diego Febbe, Duccio FanelliHypergraph Neural NetworksNeural Network Expressivity

    Department of Physics, University of Florence & INFN, Italy

  16. Beyond the Edge of Chaos: Stability-Expressivity Transfer in Reservoir Forecasting

    Jul 20, 2026Yao Du, Xingang WangLyapunov StabilityReservoir Computing

    School of Physics and Information Technology, Shaanxi Normal University, Xi’an 710062, China

  17. Expressivity of Shallow Neural Networks Over Finite Fields

    Jul 19, 2026Maksym Zubkov, Carol Wu, Shiwei Yang +2Shallow Neural NetworksPolynomial Neural Networks

    Department of Mathematics, University of British Columbia, 1984 Mathematics Rd, Vancouver, V6T 1Z2, BC, Canada

  18. Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations

    Jul 15, 2026Chon-Fai Kam, Xavier Cadet, Miloud Bessafi +1Feedforward Neural NetworksNeural Network Expressivity

    University Paris City & University of Reunion, Paris, France · Dipartimento di Fisica e Chimica Emilio Segrè, Università degli Studi di Palermo, Palermo, Italy · Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, USA +2

  19. Tropical Circuits with Scalar Multiplication Gates

    Jul 13, 2026Christoph Hertrich, Moritz StargallaNeural Network Expressivity

    University of Technology Nuremberg

  20. All you need is SAMPAT

    Jul 10, 2026Jayadeva, Madhur AswaniShallow Neural NetworksInterpretable ML

    Department of Electrical Engineering, Indian Institute of Technology, Delhi

  21. On the Principles of Deep Feedforward ReLU Networks

    Jul 8, 2026Changcun HuangReLU Neural NetworksNeural Network Interpretability

    Shuitu Institute of Applied Mathematics, Chongqing 400700, P.R.C

  22. On Explicit Super-Expressive Approximation for Neural Networks

    Jul 7, 2026Feng-Lei Fan, Ze-Yu Li, Chen-Yu Wang +1Neural Network Approximation TheoryNeural Network Expressivity

    Department of Data Science, City University of Hong Kong, 83 Tat Chee Avenue, Kowloon Tong, Kowloon, Hong Kong. · Department of Mathematics, Chinese University of Hong Kong, University Avenue, Shatin, N.T., Hong Kong. · School of Civil Engineering and Architecture, Guangxi Minzu University, Nanning 530006, China.

  23. When Does Tool Use Increase the Expressive Power of Finite-Precision Recurrent Models?

    Jul 7, 2026Nikola Zubić, Qian Li, Yuyi Wang +1Recurrent Neural NetworksNeural Network Expressivity

    Robotics and Perception Group, University of Zurich · Shenzhen International Center for Industrial and Applied Mathematics, Shenzhen Research Institute of Big Data · Tengen Intelligence Institute, CRRC Zhuzhou Institute

  24. Separation Capacity of Scattering Networks on Low-Dimensional Datasets

    Jul 7, 2026Konstantin Häberle, Helmut BölcskeiNeural Network Expressivity

    ETH Zurich

  25. Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity

    Jul 6, 2026Julia Nakhleh, Robert D. NowakReLU Neural NetworksNeural Network Approximation Theory

    Department of Computer Science · University of Wisconsin-Madison · Department of Electrical & Computer Engineering

  26. Compressed Computation under L4L^4 Loss is likely Computation in Superposition

    Jul 6, 2026Francisco Ferreira da Silva, Stefan HeimersheimReLU Neural NetworksFeedforward Neural Networks

  27. Minimum Block Width for Universal Approximation by Residual Neural Networks with Inner Width One

    Jul 6, 2026Qi Zhou, Xuan Zhou, Xiao-Song YangReLU Neural NetworksResidual Learning

    School of Mathematics and Statistics, Huazhong University of Science and Technology, Wuhan, 430074, Hubei, P.R. China. · Hubei Key Laboratory of Engineering Modeling and Scientific Computing, Huazhong University of Science and Technology, Wuhan, 430074, Hubei, P.R. China.

  28. Low-dimensional topology of deep neural networks

    Jun 30, 2026Junyu Ren, Lek-Heng LimTransformerFeedforward Neural Networks

    University of Chicago, Chicago, IL, USA.

  29. Separation Capacity of Scattering Networks

    Jun 29, 2026Konstantin Häberle, Helmut BölcskeiConvolutional Neural NetworksNeural Network Expressivity

    ETH Zurich

  30. Algorithmic Foundations of Deep Learning: Complexity-Theoretic Rates and a Characterization of Universal Approximation

    Jun 25, 2026Anastasis Kratsios, Simone Brugiapaglia, Bum Jun Kim +2Universal ApproximationNeural Network Approximation Theory

    McMaster University & Vector Institute, Toronto, ON, Canada · Concordia University, Montréal, QC, Canada · University of Tokyo, Tokyo, Japan +2

  31. Identifying structural design principles shaping the computational abilities of recurrent neural networks

    Jun 22, 2026Tom Talpir, Elad SchneidmanRecurrent Neural NetworksNeural Network Expressivity

    Department of Brain Sciences, Weizmann Institute of Science, Rehovot, Israel

  32. EML Trees Are Universal Approximators

    Jun 22, 2026Joe Germany, Elie Abdo, Joseph BakarjiUniversal ApproximationNeural Network Approximation Theory

    Department of Mathematics American University of Beirut Beirut, Lebanon · Department of Mechanical Engineering American University of Beirut Beirut, Lebanon

  33. On the Expressive Power of Weight Quantization in Large Language Models

    Jun 20, 2026Shao-Qun ZhangLLM QuantizationWeight-Only Quantization

    National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210063, China. · School of Intelligent Science and Technology, Nanjing University, Suzhou 215163, China.

  34. Expressivity Saturation: Reduced Affine Region Usage Under Increasing Task Complexity

    Jun 19, 2026Xuan Qi, Yi Wei, Fanqi Yu +1Multilayer PerceptronsNeural Network Approximation Theory

    AI for Good (AIGO), Istituto Italiano di Tecnologia · DITEN, University of Genoa · State Key Laboratory of Novel Software Technology +1

  35. Effects of sparsity and superposition on loss in simple autoencoders

    Jun 16, 2026Mriganka Basu Roy Chowdhury, Eric McLaughlin WeinerFeature SuperpositionActivation Sparsity

    Department of Statistics, UC Berkeley · Department of Materials Science, UC Berkeley

  36. Neuro-Relational Programs: Unifying Queries and Neural Computation over Structured Data

    Jun 10, 2026Arie Soeteman, Balder ten Cate, Maurice Funk +3Graph Neural NetworksRelational Reasoning

    ILLC, University of Amsterdam, Amsterdam, Netherlands · Leipzig University, Leipzig, Germany · ScaDS.AI Center Dresden/Leipzig, Germany +2

  37. Characterizing the Discrete Geometry of ReLU Networks

    Jun 5, 2026Blake B. Gaines, Jinbo BiNeural Representation GeometryReLU Neural Networks

    Department of Computer Science University of Connecticut

  38. Rethinking Neural Width for Alternating Current Optimal Power Flow Proxies

    Jun 2, 2026Dhruvi Khandelwal, Anurag Basistha, Ayushi Jolotia +1Neural Surrogate ModelingOptimal Power Flow

    Department of Electrical Engineering, National Institute of Technology Kurukshetra, India. · Indraprastha Institute of Information Technology Delhi, India. · Department of Electrical Engineering, Indian Institute of Technology Roorkee, India.

  39. Expressivity of congruence-based architectures for DNNs on positive-definite matrices

    Jun 1, 2026Antonin Oswald, Estelle MassartNeural Network Expressivity

    ICTEAM, UCLouvain · Louvain-la-Neuve, Belgium

  40. An Algebraic View of the Expressivity of Recurrent Language Models

    Jun 1, 2026Franz Nowak, Ryan Cotterell, Reda BoumasmoudRecurrent Neural NetworksNeural Network Expressivity

    ETHZürich.

  41. Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't

    May 28, 2026Anej Svete, William Merrill, Ryan Cotterell +1Transformer ExpressivityTransformer

    1. ETH Zürich · 2. Allen Institute for AI

  42. Expressive Power of Floating-Point Neural Networks with Arbitrary Reduction Orders and Inexact Activation Implementations

    May 27, 2026Yeachan Park, Geonho Hwang, Wonyeol Lee +1Neural Network Activation FunctionsUniversal Approximation

  43. More Expressive Feedforward Layers: Part I. Token-Adaptive Mixing of Activations

    May 26, 2026Mingze Wang, Jinbo Wang, Yikuan Xia +2Transformer FFNsTransformer Expressivity

    ByteDance Seed · Peking University

  44. Random Neural Network Expressivity for Non-Linear Partial Differential Equations

    May 24, 2026Muhammed Ali Mehmood, Lukas GononNeural PDE SolversPDE Solving

    Department of Mathematics Imperial College London UK · School of Computer Science University of St. Gallen Switzerland

  45. Characterizing the Representational Capacity of Neural Processes

    May 22, 2026Robin YoungNeural ProcessesNeural Network Expressivity

    University of Cambridge

  46. Preisach Attention: A Hysteretic Model of Sequential Memory

    May 22, 2026Piotr FrydrychTransformerEfficient Attention

    The Metrology and Biomedical Engineering Institute · Faculty of Mechatronics, Warsaw University of Technology

  47. The Deterministic Horizon: Impossibility Results as Design Specifications for Trustworthy AI Systems

    May 21, 2026Dongxin GuoTransformer ExpressivityTrust in AI

    UNIVERSITY OF HONG KONG

  48. Holographic functions and neural networks

    May 21, 2026Balazs SzegedyNeural Network Approximation TheoryNeural Network Expressivity

  49. Lost in Tokenization: Fundamental Trade-offs in Graph Tokenization for Transformers

    May 21, 2026Maya Bechler-Speicher, Gilad Yehudai, Gil Harari +3Transformer ExpressivityGraph Representation Learning

    Meta AI · John A. Paulson School of Engineering and Applied Sciences, Harvard University · Google Research +2

  50. How Many Different Outputs Can a Transformer Generate?

    May 21, 2026Maxime Meyer, Mario Michelessa, Caroline Chaux +1Transformer ExpressivityTransformer

    Department of Mathematics, National University of Singapore, Singapore, 117543 · IPAL, IRL2955, Singapore · School of Computing, National University of Singapore, Singapore, 117543 +2

  51. Approximation Theory for Neural Networks: Old and New

    May 20, 2026Soumendu Sundar Mukherjee, Himasish TalukdarUniversal ApproximationNeural Network Approximation Theory

    Statistics and Mathematics Unit, Indian Statistical Institute, Kolkata

  52. A Measure-Theoretic Analysis of Reasoning: Structural Generalization and Approximation Limits

    May 19, 2026Yuyang Zhang, Yifu Zhang, Xuehai Zhou +1Transformer ExpressivityOOD Generalization

    McGill University · Mila - Quebec AI Institute · Université de Montréal

  53. Updating the standard neuron model in artificial neural networks

    May 19, 2026Raul Mohedano, Thomas Batard, Erik Velasco-Salido +4Neural Network RobustnessNeural Network Expressivity

    1Spanish National Research Council (CSIC), Madrid, Spain. · Center for Research in Mathematics (CIMAT), Guanajuato, Mexico. · 3Universidad Aut´onoma de Madrid (UAM), Madrid, Spain. +1

  54. The Expressive Power of Low Precision Softmax Transformers with (Summarized) Chain-of-Thought

    May 18, 2026Moritz Brösamle, Stephan EcksteinTransformer ExpressivityTransformer

    Department of Mathematics, University of Tübingen, Germany.

  55. Beyond Accuracy: Robustness, Interpretability and Expressiveness of EEG Foundation Models

    May 17, 2026Urban Širca, Maryam Alimardani, Stefanos Zafeiriou +1EEG DecodingNeural Network Interpretability

    Vrije Universiteit Amsterdam · Imperial College London

  56. The WidthWall: A Strict Expressivity Hierarchy for Hypergraph Neural Networks

    May 13, 2026Fengqing Jiang, Yuetai Li, Yichen Feng +6GNN ExpressivityHypergraph Neural Networks

    University of Washington · Western Washington University · King Abdulaziz City for Science and Technology +1

  57. Lower bounds for one-layer transformers that compute parity

    May 12, 2026Daniel HsuTransformer ExpressivitySelf-Attention

  58. Scaling Laws and Tradeoffs in Recurrent Networks of Expressive Neurons

    May 12, 2026Aaron Spieler, Georg Martius, Anna LevinaRecurrent Neural NetworksNeural Scaling Laws

    University of Tübingen, Germany · Max Planck Institute for Intelligent Systems, Tübingen, Germany · Max Planck Institute for Biological Cybernetics, Tübingen, Germany

  59. The Polynomial Counting Capabilities of Message Passing Neural Networks

    May 11, 2026Marco Sälzer, Pascal Bergsträßer, Anthony W. LinGraph Neural NetworksMessage Passing Neural Networks

    1RPTU University Kaiserslautern-Landau · 2Max Planck Institute for Software Systems (MPI-SWS)

  60. Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defects

    May 10, 2026Kevin Dao, Jose Israel RodriguezPolynomial Neural NetworksNeural Network Approximation Theory

    Department of Mathematics, University of Wisconsin-Madison, Wisconsin, USA.