Rectified Linear Unit Networks

Ask about this topic

Momentum

10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 61

All topics
CardsList

  1. The Conflict Between Logic and Memory: Learning Higher-Order Interactions in Shallow MLPs

    Sep 30, 2026Gongyue Zhang, Honghai LiuMultilayer PerceptronsRectified Linear Unit Networks

  2. Minimax rates for learning spectral Barron functions by deep ReLU neural networks

    Sep 30, 2026Songqiu Ma, Yunfei YangRectified Linear Unit NetworksNeural Network

    School of Mathematics (Zhuhai), Sun Yat-sen University, Zhuhai, China. · School of Mathematics (Zhuhai) and Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University, Zhuhai, China.

  3. Let the Neurons Die: Exploiting ReLU-Induced Model Degradation

    Sep 28, 2026Kexin Li, Wenjun Qiu, Joshua Abraham +2Rectified Linear Unit NetworksNeurons

    University of Toronto, Toronto, Ontario, Canada

  4. Two-Timescale Fine-tuning Provably Learns New Features for Two-Layer ReLU Networks

    Sep 28, 2026Etienne Boursier, Nicolas FlammarionFeature LearningModel Fine-Tuning

    INRIA, LMO, Universit´e Paris-Saclay, Orsay, France · EPFL, Lausanne, Switzerland

  5. Minimal-Norm Univariate Two-Layer ReLU Classification: Exact Solutions and Global Optimality with Skip Connections

    Sep 23, 2026Karolina Drabik, Ben Lewis, Antoni Puch +4Rectified Linear Unit NetworksRegularization

    Department of Mathematics, Informatics and Mechanics, University of Warsaw, Poland · Department of Computer Science, University of Warwick, UK · INRIA & LMO, Université Paris-Saclay, France

  6. Generalized Deep Regression for Repeated Measurements

    Sep 22, 2026Kexuan LiRectified Linear Unit NetworksConvex Loss

    Bristol Myers Squibb

  7. Error bounds in Sobolev norms for approximations with norm constrained ReLU neural networks

    Sep 17, 2026Xianjun Li, Yunfei YangLogarithmic Sobolev InequalitiesRectified Linear Unit Networks

    School of Statistics and Data Science, Capital University of Economics and Business, Beijing, China. · School of Mathematics (Zhuhai) and Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University, Zhuhai, China.

  8. Rank and computation of the pathlifting Jacobian of a DAG ReLU network

    Sep 16, 2026Manon VerbockhavenRectified Linear Unit NetworksJacobian

    OCKHAM

  9. Approximating Smooth Functionals with ReLU Networks

    Sep 14, 2026Shuhao JiaoNeural ApproximationsRectified Linear Unit Networks

    Department of Biostatistics, City University of Hong Kong, Hong Kong SAR, China.

  10. Nearly Tight Rademacher Bounds for Sparsely Activated Neural Networks

    Sep 8, 2026Xiaoyu Li, Zhizhou Sha, Jiaojiao Jiang +2Rectified Linear Unit NetworksConcentration Bounds

    University of New South Wales · University of Texas at Austin · University of Sydney

  11. A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizations

    Sep 2, 2026Ruiyang Hong, Hrad Ghoukasian, Anastasis KratsiosLipschitz ContinuityMetric Spaces

  12. Every Layer Counts: An Exponential L2L_2 Depth Hierarchy for ReLU Networks

    Aug 24, 2026Itay SafranRectified Linear Unit NetworksRectified Linear Unit

    Stein Faculty of Computer and Information Science, Ben-Gurion University of the Negev

  13. Input convex neural networks as surrogates in mathematical optimisation

    Aug 10, 2026Yu Liu, Jan Kronqvist, Fabricio OliveiraMixed-Integer ProgrammingNonconvex

    Department of Mathematics and Systems Analysis, School of Science, Aalto University, Espoo, Finland · Department of Mathematics, KTH Royal Institute of Technology, Stockholm, Sweden · DTU Management, Technical University of Denmark, Lyngby, Denmark

  14. Hidden Gauge Controls Feature Specialization in ReLU Networks

    Aug 7, 2026Tongxi WangFeature LearningRectified Linear Unit Networks

    School of Future Technology Southeast University Nanjing, China

  15. Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples

    Aug 7, 2026Xinliang Liu, Tong Mao, Jinchao XuNeural ApproximationsSpectral Representation Method

    School of Mathematical Sciences, Ocean University of China, Qingdao 266100, China · Shenzhen Loop Area Institute, Shenzhen 518000, P. R. China

  16. Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation

    Aug 6, 2026Chunhui Shi, Xinwen FuRectified Linear Unit NetworksFeed-Forward

  17. New Complexity-Theoretic Frontiers of Tractability for Neural Network Training

    Jul 23, 2026Cornelius Brand, Robert Ganian, Mathis RoctonRectified Linear Unit NetworksNeural Network Training

    Algorithms & Complexity Group Vienna University of Technology Favoritenstraße 9-11, 1040 Vienna, Austria

  18. Shallower ReLU Network Representations via Exact Linear Algebra

    Jul 22, 2026Kilian Rueß, Gennadiy Averkov, Florestan Brunck +7Rectified Linear Unit Networks

    University of Technology Nuremberg · Brandenburg University of Technology Cottbus–Senftenberg · University of Copenhagen +3

  19. Neural network realization of binary refinement iterates via a two-chart atlas selector

    Jul 21, 2026Tsogtgerel GantumurRectified Linear Unit NetworksIterative Refinement

    McGill University, Montr´eal, QC, Canada · National University of Mongolia, Ulaanbaatar, Mongolia · Mongolian Academy of Sciences, Institute of Mathematics and Digital Technology

  20. Effects of width-dependent model hyperparameters and ℓ2\ell_2-regularization on the loss landscape of two-layer ReLU networks

    Jul 18, 2026Haruka Eshima, Makoto YamadaRectified Linear Unit NetworksAnisotropic Loss Landscapes

    Okinawa Institute of Science and Technology, Okinawa, Japan

  21. Random Parameter Noise Does Not Make Exact ReLU Verification Easy

    Jul 15, 2026Mojtaba SoltanalianNeural Network VerificationRectified Linear Unit Networks

    Department of Electrical and Computer Engineering University of Illinois Chicago

  22. Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Width

    Jul 12, 2026Yanming Lai, Defeng Sun, Yang WangRectified Linear Unit NetworksApproximation

    Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Hong Kong, China · Department of Mathematics, The University of Hong Kong, Pokfulam, Hong Kong, China

  23. On the Principles of Deep Feedforward ReLU Networks

    Jul 8, 2026Changcun HuangRectified Linear Unit NetworksBackpropagation

    Shuitu Institute of Applied Mathematics, Chongqing 400700, P.R.C

  24. A Function-Space Dichotomy for Compositional Learning: Exponential Sub-Optimality of the Neural Tangent Kernel

    Jul 7, 2026Arkaprabha Ganguli, Emil ConstantinescuNeural Tangent KernelRectified Linear Unit Networks

    Mathematics & Computer Science Division, Argonne National Laboratory, Lemont, IL, USA

  25. Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity

    Jul 6, 2026Julia Nakhleh, Robert D. NowakRectified Linear Unit NetworksDeep Network

    Department of Computer Science · University of Wisconsin-Madison · Department of Electrical & Computer Engineering

  26. Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models

    Jul 6, 2026Stanislas Strasman, Sobihan Surendran, Sylvain Le CorffStochastic Gradient DescentGenerative Models

    SU, LPSM · Sorbonne Université and Université Paris Cité, CNRS, LPSM, F-75005 Paris, France · LOPF, Califrais’ Machine Learning Lab, Paris, France

  27. Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

    Jul 3, 2026Shuang Liang, Tom Jacobs, Guido MontúfarRectified Linear Unit NetworksStochastic Gradient Descent

    Department of Statistics & Data Science University of California, Los Angeles Los Angeles, CA 90095, USA · CISPA Helmholtz Center for Information Security · Departments of Mathematics and Statistics & Data Science University of California, Los Angeles; and Max Planck Institute for Mathematics in the Sciences

  28. Fast approximation and learning of binary classification tasks in o-minimal structures using ReLU neural networks

    Jun 29, 2026Clemens Kinn, Philipp PetersenRectified Linear Unit NetworksDecision Boundaries

    Faculty of Mathematics, University of Vienna, 1090 Vienna, Austria

  29. SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model

    Jun 29, 2026Tyler LaBonte, Vidya MuthukumarFeature LearningRectified Linear Unit Networks

    H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology · School of Electrical and Computer Engineering, Georgia Institute of Technology

  30. Convex training of Lipschitz-regularized shallow neural networks

    Jun 17, 2026Chao Yin, Antoine Lesage-LandryLipschitz ContinuityRectified Linear Unit Networks

    Polytechnique Montr´eal, GERAD & Mila, Montr´eal, QC, Canada, H3T 1J4.

  31. Conservation Laws for Modern Neural Architectures

    Jun 16, 2026Viet-Hoang Tran, Vinh Khanh Bui, Tan Lai Ngoc +3OverparameterizationGradient

    National University of Singapore · Center for AI Research, VinUniversity · Independent Researcher +1

  32. Learning Dynamics Reveal a Hierarchy of Weight-Induced Layerwise Gram Metrics

    Jun 8, 2026Claudio NordioRectified Linear Unit NetworksLearning Dynamics

  33. Layer-wise Derivative Controlled Networks Achieve Competitive Accuracy and Gradient Stability Across Data Regimes

    Jun 6, 2026Rowan MartnishnRectified Linear Unit Networks

  34. Characterizing the Discrete Geometry of ReLU Networks

    Jun 5, 2026Blake B. Gaines, Jinbo BiRectified Linear Unit NetworksIntrinsic Dimensionality

    Department of Computer Science University of Connecticut

  35. Optimal Rates for Generalization of Gradient Descent Methods with Deep Neural Networks

    Jun 4, 2026Junyu Zhou, Puyu Wang, Yunwen Lei +2Gradient DescentStochastic Gradient Descent

    Mathematical Institute for Machine Learning and Data Science, KU Eichstätt-Ingolstadt, Ingolstadt, Germany · Department of Computer Science, RPTU Kaiserslautern-Landau, Kaiserslautern, Germany · Department of Mathematics, University of Hong Kong, Hong Kong, China +1

  36. Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activations

    Jun 4, 2026Yizhe Ding, Runze Li, Jia Liu +1Rectified Linear Unit NetworksActivation Functions

    Department of Statistics, The Pennsylvania State University

  37. When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks

    Jun 3, 2026Berk Tinaz, Changzhi Xie, Mahdi SoltanolkotabiRectified Linear Unit NetworksGradient Descent

    Department of Electrical and Computer Engineering University of Southern California Los Angeles, CA, USA · Department of Computer Science University of Southern California Los Angeles, CA, USA

  38. Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networks

    May 29, 2026Yunfei Yang, Jun FanLogarithmic Sobolev InequalitiesRectified Linear Unit Networks

    School of Mathematics (Zhuhai) and Guangdong Province Key Laboratory of Computational Science, Sun Yat-sen University, Zhuhai, China. · Department of Mathematics, Hong Kong Baptist University, Kowloon, Hong Kong, China.

  39. Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Bias

    May 26, 2026James Town, Etienne Boursier, Ben Lewis +2Rectified Linear Unit NetworksStochastic Interpolants

    University of Warwick · INRIA LMO, Université Paris-Saclay

  40. Riemannian Networks over Full-Rank Correlation Matrices

    May 18, 2026Ziheng Chen, Xiaojun Wu, Bernhard Schölkopf +1Full-Rank Correlation MatricesHigh-Order Correlations

    Department of Information Engineering and Computer Science, University of Trento, Trento, Italy · Max Planck Institute for Intelligent Systems, Tübingen, Germany · School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China

  41. Shallow ReLUs^s Networks in LpL^p-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization

    May 18, 2026Weizhao Li, Fanghui Liu, Lei ShiLogarithmic Sobolev InequalitiesRectified Linear Unit Networks

    School of Mathematical Sciences Fudan University Shanghai, China · School of Mathematical Sciences Institute of Natural Sciences and MOE-LSC2026 Shanghai Jiao Tong University Shanghai, China · School of Mathematical Sciences Shanghai Key Laboratory for Contemporary Applied Mathematics25 Fudan University Shanghai, China

  42. The Symmetries of Three-Layer ReLU Networks

    May 18, 2026Johanna Marie Gegenfurtner, Moritz Grillo, Guido MontúfarRectified Linear Unit NetworksSymmetry

    Technical University of Denmark · Max Planck Institute for Mathematics in the Sciences · UCLA and MPI MiS

  43. Lower bounds for one-layer transformers that compute parity

    May 12, 2026Daniel HsuLinear AttentionRectified Linear Unit Networks

  44. Causal Explanations from the Geometric Properties of ReLU Neural Networks

    May 11, 2026Hector Woods, Philippa Ryan, Rob AlexanderRectified Linear Unit NetworksCausal

    Department of Computer Science University of York York, United Kingdom

  45. Optimizer-Induced Mode Connectivity: From AdamW to Muon

    May 11, 2026Fangzhao Zhang, Sungyoon Kim, Erica Zhang +2Rectified Linear Unit NetworksRegularization

    Stanford University

  46. Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regression

    May 8, 2026Mingsong Yan, Dongyang Li, Charles Kulick +1Kernel Ridge RegressionTransformer Attention

    Department of Mathematics, University of California, Santa Barbara, CA

  47. ConquerNet: Convolution-Smoothed Quantile ReLU Neural Networks with Minimax Guarantees

    May 7, 2026Tianpai Luo, Fangwei Wu, Weichi WuMulti-Quantile RegressionRectified Linear Unit Networks

    Department of Statistics and Data Science Tsinghua University Beijing, 100084, China

  48. Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks

    May 6, 2026Xuan Qi, Yi Wei, Fanqi Yu +3Batch NormalizationRectified Linear Unit Networks

    AI for Good Istituto Italiano di Tecnologia, Genoa, Italy Department of Computer Science University of Verona, Verona, Italy

  49. Most ReLU Networks Admit Identifiable Parameters

    May 5, 2026Moritz Grillo, Guido MontúfarRectified Linear Unit Networks

    Max Planck Institute for Mathematics in the Sciences, Leipzig · Departments of Mathematics and Statistics & Data Science, UCLA

  50. A PyTorch Library of Turing-Complete Neural Networks

    May 3, 2026Jonathan BatesPytorchNeural Network

  51. On Halting vs Converging in Recurrent Graph Neural Networks

    Apr 28, 2026Jeroen Bollen, Stijn VansummerenGraph Neural NetworksRecurrent Neural Networks

    Hasselt University

  52. Explicit integral representations and quantitative bounds for two-layer ReLU networks

    Apr 25, 2026Anthony LeeRectified Linear Unit NetworksRectified Linear Unit

    School of Mathematics, University of Bristol

  53. Relaxation-Informed Training of Neural Network Surrogate Models

    Apr 24, 2026Calvin TsaySurrogate GradientRectified Linear Unit Networks

    Department of Computing, Imperial College London, South Kensington, SW7 2AZ, United Kingdom.

  54. The Cost of Relaxation: Evaluating the Error in Convex Neural Network Verification

    Apr 20, 2026Merkouris Papamichail, Konstantinos Varsos, Giorgos Flouris +1Neural Network VerificationRectified Linear Unit Networks

    Foundation for Research and Technology – Hellas, Heraklion, Greece · University of Crete, Heraklion, Greece · Catalan Institution for Research and Advanced Studies, Barcelona, Spain +1

  55. A Complete Symmetry Classification of Shallow ReLU Networks

    Apr 15, 2026Pranavkrishnan RamakrishnanRectified Linear Unit NetworksSymmetry

  56. Complete Identification of Deep ReLU Networks through Łukasiewicz Logic

    Jan 30, 2026Yani Zhang, Helmut BölcskeiRectified Linear Unit NetworksLogic

    Chair for Mathematical Information Science, ETH Zürich

  57. Configuration-Dependent Lower Bounds for Approximation by Shallow ReLUk^k Networks on the Sphere

    Oct 5, 2025Tong Mao, Jinchao XuNeural ApproximationsRectified Linear Unit Networks

    Shenzhen Loop Area Institute, Shenzhen 518000, P. R. China · King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia.

  58. Parameterized Hardness of Zonotope Containment and Neural Network Verification

    Sep 26, 2025Vincent Froese, Moritz Grillo, Christoph Hertrich +1Rectified Linear Unit NetworksNeural Network Verification

    Technische Universität Berlin · Max Planck Institute for Mathematics in the Sciences · University of Technology Nuremberg

  59. Near-optimal estimates for the ℓp\ell^p-Lipschitz constants of deep random ReLU neural networks

    Jun 24, 2025Sjoerd Dirksen, Patrick Finke, Paul Geuchen +2Rectified Linear Unit NetworksLipschitz Continuity

  60. Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization

    May 28, 2025Etienne Boursier, Matthew Bowditch, Matthias Englert +1Anisotropic Loss LandscapesOverparameterization

    INRIA LMO, Université Paris-Saclay Orsay, France · Mathematics Institute University of Warwick Coventry, UK · Department of Computer Science University of Warwick Coventry, UK