Random Neural Network

Random neural networks (RNNs), characterized by randomly initialized weights and biases, are a burgeoning area of research aiming to understand the fundamental properties of neural networks and improve their efficiency and robustness. Current research focuses on analyzing the behavior of RNNs in various width regimes, exploring their connection to Gaussian and non-Gaussian processes, and developing novel architectures like parallel branching graph neural networks and random-coupled neural networks. These studies are significant because they provide theoretical insights into generalization, optimization, and adversarial robustness, leading to improved training algorithms and more efficient, privacy-preserving applications in diverse fields such as image processing, PDE solving, and quantum computing.

Papers

November 17, 2024

Modulating Reservoir Dynamics via Reinforcement Learning for Efficient Robot Skill Synthesis
Zahra Koulaeizadeh, Erhan Oztop
Reinforcement Learning High Efficiency Reservoir Computing Skill Learning Stochastic Dynamical System Reservoir Dynamic Random Neural Network

October 25, 2024

Residual Random Neural Networks
M. Andrecut
Neural Network Neural Network Model Feed Forward Layer Random Neural Network

October 18, 2024

On the Sparsity of the Strong Lottery Ticket Hypothesis
Emanuele Natale (COATI), Davide Ferre' (COATI), Giordano Giambartolomei, Frédéric Giroire (COATI), Frederik Mallmann-Trenn
Sparsity Increase Equivariant Network Lottery Ticket Hypothesis Random Neural Network

July 26, 2024

Robust Learning in Bayesian Parallel Branching Graph Neural Networks: The Narrow Width Limit
Zechen Zhang, Haim Sompolinsky
Graph Neural Network Robust Learning Random Neural Network Network Width

May 16, 2024

Random ReLU Neural Networks as Non-Gaussian Processes
Rahul Parhi, Pakshal Bohra, Ayoub El Biari, Mehrsa Pourya, Michael Unser
Gaussian Process Shallow Neural Network Non Gaussian Random Neural Network Deep Random Neural Network

May 8, 2024

Full error analysis of the random deep splitting method for nonlinear parabolic PDEs and PIDEs
Ariel Neufeld, Philipp Schmocker, Sizhou Wu
Error Analysis Random Neural Network Parabolic Partial Differential Equation

March 26, 2024

Random-coupled Neural Network
Haoran Liu, Mingzhe Liu, Peng Li, Jiahui Wu, Xin Jiang, Zhuo Zuo, Bingqi Liu
Neural Network Neuromorphic Computing Spike Train Random Neural Network Pulse Coupled Neural Network

March 18, 2024

Neural network representation of quantum systems
Koji Hashimoto, Yuji Hirono, Jun Maeda, Jojiro Totsuka-Yoshinaka
Gaussian Process Quantum System Field Theory Neural Network Representation Random Neural Network

February 9, 2024

How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers
Gon Buzaglo, Itamar Harel, Mor Shpigel Nacson, Alon Brutzkus, Nathan Srebro, Daniel Soudry
Neural Network Strong Generalization Stochastic Gradient Descent Neural Network Generalization Random Neural Network Typicality Effect

December 13, 2023

Universal approximation property of Banach space-valued random feature models including random neural networks
Ariel Neufeld, Philipp Schmocker
Universal Approximation Deterministic Neural Network Random Neural Network

November 2, 2023

Upper and lower bounds for the Lipschitz constant of random neural networks
Paul Geuchen, Thomas Heindl, Dominik Stöger, Felix Voigtlaender
Adversarial Example Adversarial Perturbation Lower Bound Shallow Neural Network Lipschitz Constant Random Neural Network Worst Case Robustness

June 28, 2023

Gaussian random field approximation via Stein's method with applications to wide random neural networks
Krishnakumar Balasubramanian, Larry Goldstein, Nathan Ross, Adil Salim
Financial Application RSD Difference of Gaussian Wasserstein Distance Reproducing Kernel Hilbert Space Random Neural Network Random Field Lipschitz Activation

May 25, 2023

Learning to Act through Evolution of Neural Diversity in Random Neural Networks
Joachim Winther Pedersen, Sebastian Risi
LeArning Abstract Specie Evolution Neural Computation Synaptic Plasticity Random Neural Network Neuronal Diversity

April 19, 2023

Points of non-linearity of functions generated by random neural networks
David Holmes
Neural Network Exceptional Point Simple Function ReLU Activation Random Neural Network Non Linearity Information Complexity

March 23, 2023

Associated Random Neural Networks for Collective Classification of Nodes in Botnet Attacks
Erol Gelenbe, Mert Nakıp
Random Node Random Neural Network Botnet Related Malware

January 12, 2023

Color-NeuraCrypt: Privacy-Preserving Color-Image Classification Using Extended Random Neural Networks
Zheng Qi, AprilPyone MaungMaung, Hitoshi Kiya
Deep Learning Privacy Preserving Random Neural Network Private Neural

July 25, 2022

Dimension of activity in random neural networks
David G. Clark, L. F. Abbott, Ashok Litwin-Kumar
High Dimensional Generating User Activity Box Dimension Random Network Random Neural Network Dynamical Mean Field Theory Cross Covariance

June 18, 2022

Adversarial Robustness is at Odds with Lazy Training
Yunjuan Wang, Enayat Ullah, Poorya Mianjy, Raman Arora
Neural Network Adversarial Example Adversarial Robustness Relative Probability Deep Learning Theory Random Neural Network Lazy Training

June 9, 2022

Adversarial Noises Are Linearly Separable for (Nearly) Random Neural Networks
Huishuai Zhang, Da Yu, Yiping Lu, Di He
Neural Network Adversarial Attack Adversarial Example Neural Tangent Kernel Adversarial Noise Gradient Method Random Neural Network

June 5, 2022

Demystifying the Global Convergence Puzzle of Learning Over-parameterized ReLU Nets in Very High Dimensions
Peng He
LeArning Abstract High Dimension Gradient Norm Global Convergence Rectified Linear Unit Random Neural Network Global Convergence Rate