Feedforward Neural Networks

Ask about this topic

Overview

Feedforward neural networks pass inputs through a sequence of layers to produce outputs, forming the basis of multilayer perceptrons and the feed-forward blocks used in larger models. Research studies what functions these networks can represent and learn, how depth, width, and activation choices affect approximation and generalization, and how to make computation more efficient or interpretable. Common approaches include ReLU network theory and geometric analysis, gradient-based optimization, structured layers and pruning, and local or forward-only learning rules. A central tension is that theoretical guarantees often depend on assumptions that do not capture practical scale, while empirical gains in efficiency, robustness, and interpretability can depend strongly on the task and training setup.

Latest papers

CardsList

53 papers3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

  1. MorphingNet: Symbolic Regression Where the Network Is the Formula

    Oct 14, 2026Jincheng Gao, Jiahe Yi, Shuo SunSymbolic RegressionFeedforward Neural Networks

    Thrust of Financial Technology, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China.

  2. Approximation Property of Dropout Neural Networks: Sobolev Rates and Confidence Bounds

    Sep 30, 2026Jia-He YaoNeural Network Approximation TheoryFeedforward Neural Networks

  3. Arbitrary-Accuracy Neural Approximation with Optimal Neuron Count and Near-Optimal Bit Complexity

    Sep 28, 2026Zilan Cheng, Li-Lian Wang, Zhongjian WangNeural Network Approximation TheoryFeedforward Neural Networks

    Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University, 637371, Singapore.

  4. Polyhedral Geometry of Time-to-First-Spike Neural Networks

    Sep 10, 2026Manjot Singh, Guido Montúfar, Gitta KutyniokSpiking Neural NetworksFeedforward Neural Networks

    Department of Mathematics, Ludwig-Maximilians-Universität München · Munich Center for Machine Learning · Departments of Mathematics and Statistics & Data Science, University of California, Los Angeles +2

  5. Wave Function Backpropagation with Explicit Temporal-Interval Dynamics

    Sep 1, 2026Byunggu Yu, Justin KimBackpropagationFeedforward Neural Networks

    Dept. of Computer Science and Information Technology University of the District of Columbia Washington, DC USA

  6. A concentration result for multilayer feedforward neural networks

    Aug 15, 2026Vera KoponenMultilayer PerceptronsFeedforward Neural Networks

  7. Convergence Guarantees of Gradient Descent for Neural Networks via Generalized Lipschitz Smoothness

    Aug 11, 2026Siqiao Mu, Diego KlabjanGradient DescentNeural Network Optimization

  8. Machine Learning-Based Inter-Crystal Scatter Recovery for Ultra-High Resolution PET Imaging

    Aug 7, 2026Alexandre Bernier, Roger Lecomte, Jean-Baptiste MichaudMedical ImagingImage Reconstruction

    Department of Electrical & Computer Engineering, Universit´e de Sherbrooke, Sherbrooke, QC, Canada · Interdisciplinary Institute for Technological Innovation, Sherbrooke, QC, Canada · Sherbrooke Molecular Imaging Center of CRCHUS, Sherbrooke, QC, Canada +2

  9. Threshold-Based Early Stopping of Accumulations in Neural Networks with Binary Activation

    Aug 6, 2026Quentin Luquet de Saint-Germain, Massil Ait Abdeslam, Jean Pierre DavidEfficient Neural Network InferenceBinary Neural Networks

    Polytechnique Montréal, Department of Electrical Engineering Montréal, Canada

  10. Non-Destructive Quantification of Urea Adulteration in Bovine Milk Using Transmittance Multispectral Imaging

    Aug 4, 2026Sharukshan Niranjan, Iresha Ranaweera, Tharindu Chandrarathne +5Feedforward Neural Networks

    Dept. of Electrical and Electronic Eng. University of Peradeniya Peradeniya, Sri Lanka · Dept. of Food Science and Technology University of Peradeniya Peradeniya, Sri Lanka · Dept. of Animal Science University of Peradeniya Peradeniya, Sri Lanka

  11. Algorithmic Separation between Constant-Depth and Logarithmic-Depth Neural Networks

    Jul 28, 2026Yunwei Ren, Zihao Wang, Jason D. LeeBoolean Function LearningNeural Network Approximation Theory

    Princeton University · Stanford University · University of California, Berkeley

  12. Searching the Space of Feed-Forward Neural-Network Weight-Update Rules with Fixed Depth Symbolic Regression

    Jul 23, 2026Charles Brum, Edward FinkelsteinNeural Network OptimizationLearning to Optimize

    San Diego, California, USA · UCI, USA

  13. Kolmogorov--Arnold Networks for Small Language Models

    Jul 17, 2026Felippe Alves, Renato VicenteSmall Language ModelsNeural Network Interpretability

    TELUS Digital Research Hub – CIAAM Institute of Mathematics, Statistics and Computer Science University of São Paulo

  14. Algebraic Representability as the Limiting Regime of Grokking: An Exactly Solvable Model with Holomorphic Activations

    Jul 15, 2026Chon-Fai Kam, Xavier Cadet, Miloud Bessafi +1Feedforward Neural NetworksNeural Network Expressivity

    University Paris City & University of Reunion, Paris, France · Dipartimento di Fisica e Chimica Emilio Segrè, Università degli Studi di Palermo, Palermo, Italy · Thayer School of Engineering, Dartmouth College, Hanover, NH 03755, USA +2

  15. Backpropagation as a Nilpotent Linear System

    Jul 13, 2026Ahmed BoughammouraBackpropagationNeural Network Training Dynamics

    Higher Institute of Informatics and Mathematics of Monastir, University of Monastir, 5000 Monastir, Tunisia.

  16. Statistically Undetectable Backdoors in Deep Neural Networks

    Jul 10, 2026Andrej Bogdanov, Alon Rosen, Neekon VafaBackdoor AttacksAdversarial Examples

    University of Ottawa. · Bocconi University. · Massachusetts Institute of Technology.

  17. On the Principles of Deep Feedforward ReLU Networks

    Jul 8, 2026Changcun HuangReLU Neural NetworksNeural Network Interpretability

    Shuitu Institute of Applied Mathematics, Chongqing 400700, P.R.C

  18. Compressed Computation under L4L^4 Loss is likely Computation in Superposition

    Jul 6, 2026Francisco Ferreira da Silva, Stefan HeimersheimReLU Neural NetworksFeedforward Neural Networks

  19. Radial Suppression Accelerates Algorithmic Generalization: A Geometric Analysis of Delayed Generalization

    Jun 30, 2026Srijan Tiwari, Aditya Chauhan, Manjot SinghNeural Network GeneralizationNeural Representation Geometry

    Indian Institute of Technology Roorkee

  20. Low-dimensional topology of deep neural networks

    Jun 30, 2026Junyu Ren, Lek-Heng LimTransformerFeedforward Neural Networks

    University of Chicago, Chicago, IL, USA.

  21. Explicit Fuzzy Logic in the Feed-Forward Layer: Self-Forgetting Quantifiers Discover Legible Grammatical-Licensing Detectors

    Jun 30, 2026Mark OskinTransformer InterpretabilityTransformer FFNs

    School of Computer Science and Engineering University of Washington

  22. Algorithmic Foundations of Deep Learning: Complexity-Theoretic Rates and a Characterization of Universal Approximation

    Jun 25, 2026Anastasis Kratsios, Simone Brugiapaglia, Bum Jun Kim +2Universal ApproximationNeural Network Approximation Theory

    McMaster University & Vector Institute, Toronto, ON, Canada · Concordia University, Montréal, QC, Canada · University of Tokyo, Tokyo, Japan +2

  23. How Linear Is a Transformer Feed-Forward Block? Per-Block Linear Recoverability Is Learned, Not Architectural

    Jun 12, 2026Stuart WhippTransformer FFNsTransformer Expressivity

    Independent Research

  24. HalfNet: Randomized Neural Networks with Learned Subspace Geometry

    Jun 3, 2026Ethem AlpaydinNeural Representation GeometryRandom Feature Methods

    Independent Researcher Istanbul, Turkey

  25. Flatness and Generalization: Learning Multi-Index Models with Homogeneous Neural Networks

    Jun 3, 2026Harsh Vardhan, Hossein Taheri, Arya MazumdarNeural Network GeneralizationFlat Minima

  26. FFR: Forward-Forward Learning for Regression

    Jun 2, 2026Xinyang Liu, Xuanyu Liang, Shiqi Ding +4Neural Network OptimizationFeedforward Neural Networks

    University of Bristol · University College London · University of Cambridge

  27. Predicting the Neutrino Mass Ordering Using Neural Networks

    Jun 2, 2026T. J. C. Bezerra, L. Asquith, E. Bannister +1High-Energy PhysicsFeedforward Neural Networks

    Department of Physics and Astronomy, University of Sussex, Brighton, BN1 9QH, United Kingdom · †Now at Universal Quantum Ltd., Haywards Heath, RH16 1XQ, United Kingdom

  28. Do Deep Networks Forget Initialization? A Forgetting-Time View of Practical Inductive Bias

    May 27, 2026Mohua Das, Pierfrancesco Beneventano, Shibshankar Dey +2Neural Network GeneralizationNeural Network Initialization

    MIT · Northwestern University

  29. The Hamilton-Jacobi Theory of Deep Learning

    May 27, 2026Jose Marie Antonio Miñoza, Erika Fille T. Legara, Christopher P. MonterolaNeural Network GeneralizationNeural Network Training Dynamics

    Center for AI Research PH · Asian Institute of Management

  30. More Expressive Feedforward Layers: Part I. Token-Adaptive Mixing of Activations

    May 26, 2026Mingze Wang, Jinbo Wang, Yikuan Xia +2Transformer FFNsTransformer Expressivity

    ByteDance Seed · Peking University