Black Box Variational Inference

Black-box variational inference (BBVI) is a technique for approximating complex posterior distributions in Bayesian models, aiming to improve efficiency and scalability compared to traditional methods. Current research focuses on enhancing BBVI's reliability and speed through improved gradient estimators (e.g., using James-Stein estimators or acceptance sampling), more efficient mixture models (like MISVAE), and refined optimization strategies (including proximal stochastic gradient descent and second-order methods). These advancements are significant because they make BBVI more robust and applicable to high-dimensional problems across diverse fields, including Bayesian phylogenetic inference and machine learning.

Papers

October 29, 2024

Batch, match, and patch: low-rank approximations for score-based variational inference
Chirag Modi, Diana Cai, Lawrence K. Saul
Variational Inference Covariance Matrix Low Rank Approximation Single Simple Patch Batch Greenkhorn Black Box Variational Inference

June 11, 2024

Efficient Mixture Learning in Black-Box Variational Inference
Alexandra Hotti, Oskar Kviman, Ricky Molén, Víctor Elvira, Jens Lagergren
Variational Autoencoder Learning Mixture Variational Distribution Black Box Variational Inference SOTA Segmentation

June 5, 2024

You Only Accept Samples Once: Fast, Self-Correcting Stochastic Variational Inference
Dominic B. Dayta
Variational Inference External Sample Hierarchical Bayesian Stochastic Variational Inference Black Box Variational Inference

May 9, 2024

Variance Control for Black Box Variational Inference Using The James-Stein Estimator
Dominic B. Dayta
Variational Inference Variance Reduction Black Box Variational Inference Rao Blackwellized

July 27, 2023

Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?
Kyurae Kim, Yian Ma, Jacob R. Gardner
Stochastic Gradient Descent Variational Inference Linear Convergence Safe Landing Variational Family Entropy Estimation Black Box Variational Inference

July 15, 2023

Variational Inference with Gaussian Score Matching
Chirag Modi, Charles Margossian, Yuling Yao, Robert Gower, David Blei, Lawrence Saul
Variational Inference Score Matching Inference Problem Variational Approximation Variational Optimization Black Box Variational Inference

May 24, 2023

On the Convergence of Black-Box Variational Inference
Kyurae Kim, Jisu Oh, Kaiwen Wu, Yi-An Ma, Jacob R. Gardner
Variational Inference Early Stage Convergence Variational Approximation Variational Family Black Box Variational Inference Smooth Density Large Scale Bayesian

April 11, 2023

Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Box
Ryan Giordano, Martin Ingram, Tamara Broderick
Variational Inference Black Box Deterministic Algorithm Posterior Approximation Variational Approximation Black Box Variational Inference Mean Field Variational Posterior Covariance

March 18, 2023

Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inference
Kyurae Kim, Kaiwen Wu, Jisu Oh, Jacob R. Gardner
Stochastic Gradient Descent Variational Inference Practical Application Log Likelihood Low Variance Gradient Gradient Variance Black Box Variational Inference Covariance Approximation

May 23, 2022

Quasi Black-Box Variational Inference with Natural Gradients for Bayesian Learning
Martin Magris, Mostafa Shabani, Alexandros Iosifidis
Natural Gradient Bayesian Learning Variational Posterior Inference Problem Black Box Variational Inference

March 29, 2022

A Framework for Improving the Reliability of Black-box Variational Inference
Manushi Welandawe, Michael Riis Andersen, Aki Vehtari, Jonathan H. Huggins
New Framework Variational Inference Approximate Bayesian Inference Variational Approximation Black Box Variational Inference

Black Box Variational Inference

Papers

Batch, match, and patch: low-rank approximations for score-based variational inference

Efficient Mixture Learning in Black-Box Variational Inference

You Only Accept Samples Once: Fast, Self-Correcting Stochastic Variational Inference

Variance Control for Black Box Variational Inference Using The James-Stein Estimator

Linear Convergence of Black-Box Variational Inference: Should We Stick the Landing?

Variational Inference with Gaussian Score Matching

On the Convergence of Black-Box Variational Inference

Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Box

Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inference

Quasi Black-Box Variational Inference with Natural Gradients for Bayesian Learning

A Framework for Improving the Reliability of Black-box Variational Inference