Stochastic Variational Inference

Stochastic Variational Inference (SVI) is a powerful approximate Bayesian inference technique used to efficiently estimate the parameters of complex probabilistic models, particularly in high-dimensional settings. Current research focuses on applying SVI to various model architectures, including Gaussian Processes (GPs), Bayesian Neural Networks (BNNs), and state-space models, often incorporating techniques like mini-batching and adaptive feature extraction to improve scalability and accuracy. This approach is proving valuable across diverse fields, enabling more robust uncertainty quantification in applications ranging from Bayesian inverse problems and time series analysis to deep learning and hybrid classical-quantum machine learning. The resulting improvements in efficiency and accuracy are driving significant advancements in various scientific domains and practical applications.

Papers

July 30, 2024

Weak neural variational inference for solving Bayesian inverse problems without forward models: applications in elastography
Vincent C. Scholz, Yaohua Zang, Phaedon-Stelios Koutsourelakis
Financial Application Variational Inference Bayesian Inverse Problem Forward Model Stochastic Variational Inference Shear Wave High Dimensional Inverse Problem Tissue Deformation

July 2, 2024

Scalable Multi-Output Gaussian Processes with Stochastic Variational Inference
Xiaoyu Jiang, Sokratia Georgaka, Magnus Rattray, Mauricio A. Alvarez
Latent Variable Stochastic Variational Inference Multi Output Gaussian Process

July 1, 2024

Adaptive RKHS Fourier Features for Compositional Gaussian Process Models
Xinxing Shi, Thomas Baldwin-McDonald, Mauricio A. Álvarez
Gaussian Process Reproducing Kernel Hilbert Space Fourier Feature Compositional Data Deep Gaussian Process Stochastic Variational Inference

June 29, 2024

Time Series Clustering with General State Space Models via Stochastic Variational Inference
Ryoichi Ishizuka, Takashi Imai, Kaoru Kawamoto
Time Series Variational Inference State Space Model Variational Bayesian Inference Time Series Clustering Stochastic Variational Inference

June 5, 2024

You Only Accept Samples Once: Fast, Self-Correcting Stochastic Variational Inference
Dominic B. Dayta
Variational Inference External Sample Hierarchical Bayesian Stochastic Variational Inference Black Box Variational Inference

April 9, 2024

Variational Stochastic Gradient Descent for Deep Neural Networks
Haotian Chen, Anna Kuzina, Babak Esmaeili, Jakub M Tomczak
Deep Neural Network Gradient Based Gradient Update Variational Learning Stochastic Variational Inference Adaptive Gradient

March 8, 2024

Variational Inference of Parameters in Opinion Dynamics Models
Jacopo Lenti, Fabrizio Silvestri, Gianmarco De Francisci Morales
Multi Agent Variational Inference Many Parameter Agent Based Model Probabilistic Generative Model Opinion Dynamic Stochastic Variational Inference

November 24, 2023

Deep Latent Force Models: ODE-based Process Convolutions for Bayesian Deep Learning
Thomas Baldwin-McDonald, Mauricio A. Álvarez
Uncertainty Quantification Bayesian Deep Learning Deep Gaussian Process Stochastic Variational Inference Convolutional Neural Process Latent Force Model

September 21, 2023

Bayesian sparsification for deep neural networks with Bayesian model reduction
Dimitrije Marković, Karl J. Friston, Stefan J. Kiebel
Deep Neural Network Generative Modeling Model Reduction Structural Prior Stochastic Variational Inference

August 15, 2023

Natural Evolution Strategies as a Black Box Estimator for Stochastic Variational Inference
Ahmad Ayaz Amin
Variational Autoencoders Bayesian Inference VAE Model Stochastic Variational Inference Low Variance Gradient Natural Evolution Strategy

November 18, 2022

Deep Gaussian Processes for Air Quality Inference
Aadesh Desai, Eshan Gujarathi, Saagar Parikh, Sachin Yadav, Zeel Patel, Nipun Batra
Fine Grained Air Quality Deep Gaussian Process Stochastic Variational Inference

May 19, 2022

Deterministic training of generative autoencoders using invertible layers
Gianluigi Silvestri, Daan Roos, Luca Ambrogioni
Supervised Autoencoder Stochastic Variational Inference Generative Autoencoder Reversible Architecture Deterministic Learner

May 11, 2022

Scalable Stochastic Parametric Verification with Stochastic Variational Smoothed Model Checking
Luca Bortolussi, Francesca Cairoli, Ginevra Carbone, Paolo Pulcini
Bayesian Inference Stochastic Variational Inference Probabilistic Model Checking Probabilistic Verification

March 15, 2022

Accelerating Stochastic Probabilistic Inference
Minta Liu, Suliang Bu
Variational Inference Variational Objective Stochastic Variational Inference

February 25, 2022

February 24, 2022

AutoIP: A United Framework to Integrate Physics into Gaussian Processes
Da Long, Zheng Wang, Aditi Krishnapriyan, Robert Kirby, Shandian Zhe, Michael Mahoney
Gaussian Process Uncertainty Quantification Theoretical Physic Data Driven Model Physical Model Stochastic Variational Inference

January 21, 2022

Training Hybrid Classical-Quantum Classifiers via Stochastic Variational Optimization
Ivana Nikoloska, Osvaldo Simeone
Quantum Machine Learning Near Term Quantum Hybrid Quantum Classical Quantum Classifier Training Time Stochastic Variational Inference Quantum Neuron

December 12, 2021

Spatial-Temporal-Fusion BNN: Variational Bayesian Feature Layer
Shiye Lei, Zhuozhuo Tu, Leszek Rutkowski, Feng Zhou, Li Shen, Fengxiang He, Dacheng Tao
Adversarial Robustness Bayesian Neural Network Fusion Network Variational Bayesian Stochastic Variational Inference