Rectified Linear Unit Networks

Ask about this topic

Recent momentum

-50%

3 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Deep Neural Networks

Neighbouring directions

Deep Neural NetworkDeep Learning ModelsMultilayer PerceptronsDeep Learning FrameworkNeural ArchitecturesActivation FunctionsArtificial Neural NetworksNeural Model

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in Rectified Linear Unit Networks.

Period ending 2026-09-07

1 new paper

A weekly snapshot of new work published in Rectified Linear Unit Networks.

51 papers

Latest in Rectified Linear Unit Networks

Open your feed →
CardsList

  1. Nearly Tight Rademacher Bounds for Sparsely Activated Neural Networks

    Sep 8, 2026Xiaoyu Li, Zhizhou Sha, Jiaojiao Jiang +2Rectified Linear Unit NetworksConcentration Inequalities

  2. A Closed-Form Formula for Consistent Lipschitz Regression on Metric Spaces with Sparse Neural Network Realizations

    Sep 2, 2026Ruiyang Hong, Hrad Ghoukasian, Anastasis KratsiosLipschitz ConstantMetric Spaces

  3. Every Layer Counts: An Exponential L_2 Depth Hierarchy for ReLU Networks

    Aug 24, 2026Itay SafranRectified Linear Unit NetworksRectified Linear Unit

  4. Input convex neural networks as surrogates in mathematical optimisation

    Aug 10, 2026Yu Liu, Jan Kronqvist, Fabricio OliveiraMixed-Integer Linear ProgrammingSurrogate Gradient

  5. Optimal Neural Network Approximation via Empirical Least Squares with Deterministic Samples

    Aug 7, 2026Xinliang Liu, Tong Mao, Jinchao XuNeural-Network ApproximationSolution Operator

  6. Cryptanalytic Extraction of Isolated Bias-Free GLU Feed-Forward Blocks by Antipodal Separation

    Aug 6, 2026Chunhui Shi, Xinwen FuRectified Linear Unit NetworksSeparability

  7. New Complexity-Theoretic Frontiers of Tractability for Neural Network Training

    Jul 23, 2026Cornelius Brand, Robert Ganian, Mathis RoctonRectified Linear Unit NetworksActivation Functions

  8. Shallower ReLU Network Representations via Exact Linear Algebra

    Jul 22, 2026Kilian Rueß, Gennadiy Averkov, Florestan Brunck +7Rectified Linear Unit NetworksTight Bounds

  9. Neural network realization of binary refinement iterates via a two-chart atlas selector

    Jul 21, 2026Tsogtgerel GantumurRectified Linear Unit NetworksAdaptive Shape Refinement

  10. Effects of width-dependent model hyperparameters and \ell_2-regularization on the loss landscape of two-layer ReLU networks

    Jul 18, 2026Haruka Eshima, Makoto YamadaRectified Linear Unit NetworksWeight Decay

  11. Random Parameter Noise Does Not Make Exact ReLU Verification Easy

    Jul 15, 2026Mojtaba SoltanalianRectified Linear Unit NetworksRectified Linear Unit

  12. Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Width

    Jul 12, 2026Yanming Lai, Defeng Sun, Yang WangRectified Linear Unit NetworksRegularity

  13. On the Principles of Deep Feedforward ReLU Networks

    Jul 8, 2026Changcun HuangRectified Linear Unit NetworksBackpropagation

  14. A Function-Space Dichotomy for Compositional Learning: Exponential Sub-Optimality of the Neural Tangent Kernel

    Jul 7, 2026Arkaprabha Ganguli, Emil ConstantinescuNeural Tangent KernelRectified Linear Unit Networks

  15. Deep Neural Variation Spaces: A Unifying Perspective on Depth and Complexity

    Jul 6, 2026Julia Nakhleh, Robert D. NowakRectified Linear Unit NetworksSearch Depth

  16. Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models

    Jul 6, 2026Stanislas Strasman, Sobihan Surendran, Sylvain Le CorffStochastic Gradient DescentFinite Time Analysis

  17. Implicit Bias of SGD in Multivariate ReLU Networks: Effective Width Collapse

    Jul 3, 2026Shuang Liang, Tom Jacobs, Guido MontúfarRectified Linear Unit NetworksStochastic Gradient Descent

  18. Fast approximation and learning of binary classification tasks in o-minimal structures using ReLU neural networks

    Jun 29, 2026Clemens Kinn, Philipp PetersenRectified Linear Unit NetworksEmpirical Risk Minimization

  19. SGD Provably Prioritizes a Shortcut Spurious Feature in the XOR Model

    Jun 29, 2026Tyler LaBonte, Vidya MuthukumarFeature LearningRectified Linear Unit Networks

  20. Conservation Laws for Modern Neural Architectures

    Jun 16, 2026Viet-Hoang Tran, Vinh Khanh Bui, Tan Lai Ngoc +3Over-Parameterized Neural NetworksNeural Architectures

  21. Learning Dynamics Reveal a Hierarchy of Weight-Induced Layerwise Gram Metrics

    Jun 8, 2026Claudio NordioRectified Linear Unit NetworksHierarchical

  22. Layer-wise Derivative Controlled Networks Achieve Competitive Accuracy and Gradient Stability Across Data Regimes

    Jun 6, 2026Rowan MartnishnRectified Linear Unit NetworksRegularization Methods

  23. Characterizing the Discrete Geometry of ReLU Networks

    Jun 5, 2026Blake B. Gaines, Jinbo BiRectified Linear Unit NetworksPolytopes

  24. Optimal Rates for Generalization of Gradient Descent Methods with Deep Neural Networks

    Jun 4, 2026Junyu Zhou, Puyu Wang, Yunwen Lei +2Gradient DescentStochastic Gradient Descent

  25. Mitigating the Curse of Dimensionality in Uniform Convergence of Deep Neural Networks via Smooth Activations

    Jun 4, 2026Yizhe Ding, Runze Li, Jia Liu +1Rectified Linear Unit NetworksDeep Neural Network

  26. When Both Layers Learn: Training Dynamics of Representing Linear Models via ReLU Networks

    Jun 3, 2026Berk Tinaz, Changzhi Xie, Mahdi SoltanolkotabiRectified Linear Unit NetworksGradient Descent

  27. Approximation and learning of anisotropic and mixed smooth functions by deep ReLU neural networks

    May 29, 2026Yunfei Yang, Jun FanRectified Linear Unit NetworksRegularity

  28. Mildly Overparameterized ReLU Networks on Orthogonal Data: Incremental Learning and Implicit Bias

    May 26, 2026James Town, Etienne Boursier, Ben Lewis +2Rectified Linear Unit NetworksInterpolation

  29. A Geometric Analysis of Sign-Magnitude Asymmetry in a ReLU + RMSNorm Block under Ternary Quantization

    May 18, 2026Lei DongBatch NormalizationRectified Linear Unit

  30. Shallow ReLU^s Networks in L^p-Type and Sobolev Spaces: Approximation and Path-Norm Controlled Generalization

    May 18, 2026Weizhao Li, Fanghui Liu, Lei ShiRectified Linear Unit NetworksRegularity

  31. The Symmetries of Three-Layer ReLU Networks

    May 18, 2026Johanna Marie Gegenfurtner, Moritz Grillo, Guido MontúfarRectified Linear Unit NetworksSymmetry

  32. Lower bounds for one-layer transformers that compute parity

    May 12, 2026Daniel HsuMulti-Layer TransformersSoftmax Attention

  33. Causal Explanations from the Geometric Properties of ReLU Neural Networks

    May 11, 2026Hector Woods, Philippa Ryan, Rob AlexanderRectified Linear Unit NetworksExplanations

  34. Optimizer-Induced Mode Connectivity: From AdamW to Muon

    May 11, 2026Fangzhao Zhang, Sungyoon Kim, Erica Zhang +2Rectified Linear Unit NetworksRegularization Methods

  35. Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regression

    May 8, 2026Mingsong Yan, Dongyang Li, Charles Kulick +1Kernel Ridge RegressionTransformer Attention

  36. ConquerNet: Convolution-Smoothed Quantile ReLU Neural Networks with Minimax Guarantees

    May 7, 2026Tianpai Luo, Fangwei Wu, Weichi WuQuantile RegressionRectified Linear Unit Networks

  37. Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks

    May 6, 2026Xuan Qi, Yi Wei, Fanqi Yu +3Batch NormalizationRectified Linear Unit Networks

  38. Most ReLU Networks Admit Identifiable Parameters

    May 5, 2026Moritz Grillo, Guido MontúfarRectified Linear Unit NetworksReparameterization

  39. A PyTorch Library of Turing-Complete Neural Networks

    May 3, 2026Jonathan BatesPytorchArtificial Neural Networks

  40. On Halting vs Converging in Recurrent Graph Neural Networks

    Apr 28, 2026Jeroen Bollen, Stijn VansummerenGraph Neural NetworksRectified Linear Unit Networks

  41. Explicit integral representations and quantitative bounds for two-layer ReLU networks

    Apr 25, 2026Anthony LeeRectified Linear Unit NetworksRectified Linear Unit

  42. Relaxation-Informed Training of Neural Network Surrogate Models

    Apr 24, 2026Calvin TsaySurrogate GradientRectified Linear Unit Networks

  43. The Cost of Relaxation: Evaluating the Error in Convex Neural Network Verification

    Apr 20, 2026Merkouris Papamichail, Konstantinos Varsos, Giorgos Flouris +1Neural Network VerificationRectified Linear Unit Networks

  44. A Complete Symmetry Classification of Shallow ReLU Networks

    Apr 15, 2026Pranavkrishnan RamakrishnanRectified Linear Unit NetworksSymmetry

  45. Causal Mechanism Reduction: Mechanism Replacement for Neural Network Pruning and Abstraction

    Feb 27, 2026Amir AsiaeeAblationRectified Linear Unit Networks

  46. Complete Identification of Deep ReLU Networks through Łukasiewicz Logic

    Jan 30, 2026Yani Zhang, Helmut BölcskeiRectified Linear Unit NetworksLogic

  47. Configuration-Dependent Lower Bounds for Approximation by Shallow ReLU^k Networks on the Sphere

    Oct 5, 2025Tong Mao, Jinchao XuRectified Linear Unit NetworksRegularity

  48. Parameterized Hardness of Zonotope Containment and Neural Network Verification

    Sep 26, 2025Vincent Froese, Moritz Grillo, Christoph Hertrich +1Rectified Linear Unit NetworksNeural Network Verification

  49. Near-optimal estimates for the \ell^p-Lipschitz constants of deep random ReLU neural networks

    Jun 24, 2025Sjoerd Dirksen, Patrick Finke, Paul Geuchen +2Rectified Linear Unit NetworksLipschitz Constant

  50. Favorability of Loss Landscape with Weight Decay Requires Both Large Overparametrization and Initialization

    May 28, 2025Etienne Boursier, Matthew Bowditch, Matthias Englert +1Weight DecayOver-Parameterized Neural Networks

  51. Deep Network Approximation: Beyond ReLU to Diverse Activation Functions

    Jul 13, 2023Shijun Zhang, Jianfeng Lu, Hongkai ZhaoActivation FunctionsRectified Linear Unit Networks