Paper ID: 2201.04746

An Overview of Uncertainty Quantification Methods for Infinite Neural Networks

Florian Juengermann, Maxime Laasri, Marius Merkle

To better understand the theoretical behavior of large neural networks, several works have analyzed the case where a network's width tends to infinity. In this regime, the effect of random initialization and the process of training a neural network can be formally expressed with analytical tools like Gaussian processes and neural tangent kernels. In this paper, we review methods for quantifying uncertainty in such infinite-width neural networks and compare their relationship to Gaussian processes in the Bayesian inference framework. We make use of several equivalence results along the way to obtain exact closed-form solutions for predictive uncertainty.

Submitted: Jan 13, 2022