Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference

Kamélia Daudel; Minh-Ngoc Tran; Cheng Zhang

stat.MLFeb 1, 2026

Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference

Authors: Kamélia Daudel, Minh-Ngoc Tran, Cheng Zhang

Organizations: ESSEC Business School · The University of Sydney Business School · Peking University

Abstract

Importance-weighted variational inference approximates intractable densities by optimizing multi-sample Monte Carlo objectives whose tightness increases with the number of importance samples $N$ . When reparameterization is unavailable, optimization relies on score-function (REINFORCE) gradient estimators. In this paper, we provide a comprehensive analysis of score-function gradient estimators for importance-weighted objectives by characterizing their asymptotic expectation and variance. We establish that standard variational inference for Monte Carlo objectives (VIMCO) gradient estimators suffer from a vanishing signal-to-noise ratio (SNR) as $N$ increases, which arises from a suboptimal control variate choice and can impair optimization. To address this issue, we derive a general mechanism for constructing VIMCO gradient estimators that avoid SNR collapse. We identify the High-Effective Sample Size optimal variance importance sampling estimator as belonging to this category of estimators and establish its $\sqrt{N}$ SNR scaling. This estimator does not, however, have zero variance at optimality and achieving its $\sqrt{N}$ SNR rate may require prohibitively large $N$ when the importance weights are degenerate. We overcome these limitations by introducing VIMCO gradient estimators that attain zero variance at optimality and support importance weight tempering. Experiments with discrete latent-variable and state-space models validate the theoretical predictions and illustrate the benefits of our estimators.

Figures & tables

Figure 1: SNR as a function of

N

for different values of

\alpha

when

\phi=1

in the Gaussian example. The solid lines correspond to the SNR estimates for different VIMCO gradient estimators, while the dashed lines correspond to the theoretical predictions. The shaded area represents the 5% to 95% percentile range across 10 independent runs.

Figure 2: Performance of VIMCO gradient estimators when $\alpha=0$ in the Gaussian example. The shaded area represents the 5% to 95% percentile range across 10 independent runs.

Figure 3: Performance of VIMCO gradient estimators in the SBN example averaged over 3 independent runs and expressed in terms of marginal log-likelihood (left) and ELBO (right).

Figure 4: SNR (left), gradient magnitude (middle) and gradient variance (right) as a function of $N$ in the SBN example at a fixed trained SBN checkpoint.

Figure 5: SNR for the VIMCO-zvAM( $\alpha$ ) and VIMCO-zv $\star$ ( $\alpha$ ) gradient estimators as a function of $N$ for various values of $\alpha$ in the SSMs example. The shaded area shows the standard deviation across replications.

Method	$S_{\text{total}}=100$		$S_{\text{total}}=500$
Method	MC Std	# iter	MC Std	# iter
ELBO-cv	0.0445	1499 (258)	0.0194	1647 (188)
VIMCO-zv $\star$ (annealed)	0.0428	782 (27)	0.0108	997 (146)
VIMCO-zvAM (annealed)	0.0536	776 (76)	0.0109	1510 (148)

Table 1: The table reports the Monte Carlo standard deviations of the variational parameter estimates, together with the number of iterations required, across 20 independent runs, and for two cases of per-iteration budget

S_{\text{total}}

Figure 6: Final marginal posterior estimates in the Variational Bayesian SSMs example.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Expectation and variance as a function of

N

for different values of

\alpha

when

\phi=0.1

in the Gaussian example. The solid lines correspond to the absolute mean and variance estimates for different VIMCO gradient estimators, while the dashed lines correspond to the theoretical predictions. The shaded area represents the 5% to 95% percentile range across 10 independent runs.

Figure 8: Performance of VIMCO gradient estimators in the SBN example averaged over 3 independent runs and expressed in terms of marginal log-likelihood (left) and ELBO (right).

Variational Inference

Variational inference approximates complex Bayesian posteriors with tractable distributions that support faster inference and uncertainty estimates.

See all 143 papers

Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space
Feb 21, 2026Kyurae Kim, Qiang Fu, Yi-An Ma +2Variational Inference
University of Pennsylvania, Philadelphia, U.S. · Yale University, New Haven, U.S. · University of California San Diego, San Diego, U.S. +1
Denoising Implicit Variational Inference
Oct 17, 2026Zitong Wang, Zekun Wu, Longlin Yu +1Variational Inference Bayesian Inference
School of Mathematical Sciences, Peking University
Transformed Samplers with Variance Reduction
Oct 7, 2026Siran Liu, Michalis Tisias, Petros DellaportasMarkov Chain Monte Carlo Variance Reduction
Department of Statistical Science, University College London, UK. · Google DeepMind, UK. · Department of Statistical Science, University College London, UK, and Department of Statistics, Athens University of Economics and Business, Greece.

Feb 21, 2026J/K move · Enter open · S save

Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space

Kyurae Kim, Qiang Fu, Yi-An Ma, Jacob R. Gardner+1

University of Pennsylvania, Philadelphia, U.S. · Yale University, New Haven, U.S. · University of California San Diego, San Diego, U.S.+1

Variational Inference

Open paper AskPDF arXiv

For approximating a target distribution given only its unnormalized log-density, stochastic gradient-based variational inference (VI) algorithms are a popular approach. For example, Wasserstein VI (WVI) and black-box VI (BBVI) perform gradient descent in measure space (Bures-Wasserstein space) and parameter space, respectively. Previously, for the Gaussian variational family, convergence guarantees for WVI have shown superiority over existing results for black-box VI with the reparametrization gradient, suggesting the measure space approach might provide some unique benefits. In this work, however, we close this gap by obtaining identical state-of-the-art iteration complexity guarantees for both. In particular, we identify that WVI's superiority stems from the specific gradient estimator it uses, which BBVI can also leverage with minor modifications. The estimator in question is usually associated with Price's theorem and utilizes second-order information (Hessians) of the target log-density. We will refer to this as Price's gradient. On the flip side, WVI can be made more widely applicable by using the reparametrization gradient, which requires only gradients of the log-density. We empirically demonstrate that the use of Price's gradient is the major source of performance improvement.

Abstract

Variational Inference

Explore similar work

Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space

Denoising Implicit Variational Inference

Transformed Samplers with Variance Reduction