Function Space Variational Inference

Function-space variational inference (FSVI) is a Bayesian deep learning approach that focuses on inferring the posterior distribution over functions produced by a neural network, rather than directly over its parameters. Current research emphasizes developing well-defined variational objectives and scalable algorithms, often employing Gaussian processes or implicit processes as priors and leveraging techniques like regularized KL-divergence to address challenges in optimization. This approach offers improved uncertainty quantification, particularly beneficial for safety-critical applications, and shows promise in reducing communication costs in federated learning settings by enabling efficient one-shot communication.