Bayesian Neural Networks

Bayesian neural networks study how to represent uncertainty in a network’s predictions by placing distributions over its weights or functions, helping distinguish uncertainty from limited data from noise inherent in the data. Common approaches include variational inference, Monte Carlo dropout, Laplace approximations, Hamiltonian Monte Carlo and stochastic-gradient MCMC, with structured priors and Bayesian last layers used to reduce computation. Researchers evaluate these methods on tasks such as UCI regression, image classification and out-of-distribution detection, language-model adaptation, and scientific or clinical forecasting, often measuring calibration as well as accuracy. A central challenge is obtaining reliable, informative uncertainty at practical cost: approximate posteriors can be under-calibrated or miss important correlations, while sampling and benchmark results can vary substantially across methods and settings.