This monograph provides a rigorous overview of theoretical and methodological aspects of probabilistic inference and learning with Stein's method. Recipes are provided for constructing Stein discrepancies from Stein operators and Stein sets, and properties of these discrepancies such as computability, separation, convergence detection, and convergence control are discussed. Further, the connection between Stein operators and Stein variational gradient descent is set out in detail. The main definitions and results are precisely stated, and references to all proofs are provided.
Figures & tables
≜
defined as being equal
[n]
{1,…,n}
(m[n])
{σ⊆[n]:∣σ∣=m} , all size m subsets of [n]
∥⋅∥
a generic norm on Rd
IS
indicator function IS(x)=1 if x∈S , else IS(x)=0
P
the set of all probability measures on a measurable space that will be clear from context
∥⋅∥p
the norm ∥x∥p≜(∣x1∣p+⋯+∣xd∣p)1/p on Rd , p∈[1,∞)
Table 1
Figure 1: Samples generated using the unadjusted Langevin algorithm (ULA). Here the target P is a Gaussian mixture model with density p and ϵ denotes the step size parameter of ULA. Contours of logp are depicted.
Figure 2: Tuning the unadjusted Langevin algorithm (ULA). The left panel presents the mean square error associated with the empirical estimates for the means and variances of P (which cannot be computed in general), while the right panel presents the kernel Stein discrepancy ( KSD ) (which can be computed). Here we used the inverse multi-quadric kernel ( Example 2.13 ) with exponent β=1 and bandwidth ℓ=0.01 . The experiment was repeated 10 times and means and standard errors are reported.
Figure 3: Goodness-of-fit testing with kernel Stein discrepancy. Here we plot the null distribution of the kernel Stein discrepancy test statistic as approximated using the wild bootstrap (blue), the actual value of the kernel Stein discrepancy test statistic (black), and the values for which the null would be rejected (shaded). The amount of model misspecification is denoted with σ , so that when σ=0 the model is correct. The inverse multi-quadric kernel with length scale ℓ was used.
Figure 4: Goodness-of-fit testing with kernel Stein discrepancy. Here we plot the test power (i.e. the probability of rejecting the null) as a function of both the amount of model misspecification σ and the kernel length scale ℓ . Means and standard errors over 100 experiments are reported.
Figure 5: Stein points (left) and Frank–Wolfe–Stein points (right) were used to approximate the Rosenbrock target ( 70 ) (black shaded).
Figure 6: Stein variational gradient descent with fixed kernel length scale ℓ (left) and adaptively selected kernel length scale ℓ (right) were used to approximate the Rosenbrock target ( 70 ) (black shaded).
Figure 7: Stein importance sampling with positive weights (left) and unconstrained weights (right) were used to approximate the Rosenbrock target ( 70 ) (black shaded). The size of each marker is proportional to the weight that the associated state is assigned; blue circles indicate a positive weight and red triangles indicate a negative weight.
Figure 8: Images generated by SteinGAN trained on the LSUN dataset ( Yu et al., 2015 ) , consisting of nearly 3M images of bedrooms. Reproduced with permission from Liu and Wang (2017) .
Figure 9: A comparison of variational approximations to a target P (left) produced by minimizing kernel Stein discrepancy (centre) and Kullback–Leibler divergence (right). The variational approximation was based on a block neural autoregressive flow and the target P arose from a Bayesian analysis of a biochemical oxygen demand model. Reproduced with permission from Fisher et al. (2021) .
Figure 10: Training variational autoencoders on the Fashion MNIST dataset; a binary latent z of dimension d′=200 was employed and m=2 samples were used to evaluate each gradient. Reproduced with permission from Shi et al. (2022b) .
In this paper, we develop a unified framework connecting proper scoring rules and Stein's method for the estimation of unnormalized models. We construct a class of proper scoring rules that encompasses score matching, ratio matching, pseudolikelihood, and noise contrastive estimation. We show that these proper scoring rules naturally induce Stein operators. This connection yields a new construction principle for Stein operators, and enables us to study estimators based on these scoring rules through the geometry associated with the induced Stein operators. Using this geometric perspective, we provide a unified characterization of the asymptotic efficiency of the resulting estimators.
Mitsuki Nagai, Keisuke Yano
Graduate Institute for Advanced Studies, SOKENDAI · The Institute of Statistical Mathematics
Gradient estimation -- the task of computing the gradient of the expected value of a probabilistic program -- has diverse applications in scientific computing, but is notoriously difficult because of issues such as high-dimensional integration, discrete random choices, and complex stochastic dependencies. This article introduces gradient inference, a new approach to developing sound and efficient gradient estimators for probabilistic programs. Gradient inference rests on a formal reduction from a gradient estimation problem to a closely related probabilistic inference problem, whose solution can be differentiated to obtain a gradient estimator. This inference problem is obtained by applying two powerful statistical operations -- coupling and factorization -- to the input probabilistic program. Our reduction lets us leverage the rich toolkit of probabilistic inference algorithms to design novel gradient estimators that extend and improve upon existing methods. We introduce GradInf, a probabilistic programming system that facilitates the sound and automated implementation of gradient inference. GradInf is centered around programmable source-to-source transformations for coupling and factorizing higher-order probabilistic programs, whose soundness is proven in terms of a denotational semantics. Key to our development is the use of information-flow typing to allow random choices in a probabilistic program to be factored out and partially evaluated, which improves our ability to deploy sophisticated probabilistic inference algorithms. The resulting system offers practitioners a principled framework for designing gradient estimators. We apply GradInf to several challenging case studies, showing that it can express prominent gradient estimators from the literature and enables the construction of new state-of-the-art estimators that outperform the best existing baselines.
Gaurav Arya, Mathieu Huot, Moritz Schauer +2
Carnegie Mellon University, Pittsburgh, USA · Massachusetts Institute of Technology, Cambridge, USA · Chalmers University of Technology & University of Gothenburg, Gothenburg, Sweden +1
Otto's Wasserstein gradient flow of the inclusive (forward) Kullback--Leibler (KL) divergence offers a principled framework for analyzing statistical inference algorithms, yet algorithms targeting the exclusive (reverse) KL divergence are rarely studied with such tools. We establish a unified gradient-flow and PDF framework for inclusive KL inference. We show that maximum mean discrepancy minimization can be viewed as inclusive KL inference with an approximate gradient estimator, and we develop the Fisher--Rao and Wasserstein--Fisher--Rao gradient flows that directly target the inclusive KL divergence. Restricting these flows to the manifold of Gaussian distributions yields explicit gradient-flow ODEs, providing a foundation for Gaussian variational inference. Building on this viewpoint, we further introduce a local-estimator Wasserstein gradient flow whose velocity is obtained by local nonparametric regression, free of density-ratio evaluation or kernel gradients, improving the algorithmic performance over the MMD-based particle method.
Jia-Jie Zhu
Department of Mathematics, KTH Royal Institute of Technology, Lindstedtsvägen 25, 114 28, Stockholm, Sweden