Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference

Kamélia Daudel; Minh-Ngoc Tran; Cheng Zhang

stat.MLFeb 1, 2026

Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference

Authors: Kamélia Daudel, Minh-Ngoc Tran, Cheng Zhang

Organizations: ESSEC Business School · The University of Sydney Business School · Peking University

Abstract

Importance-weighted variational inference approximates intractable densities by optimizing multi-sample Monte Carlo objectives whose tightness increases with the number of importance samples $N$ . When reparameterization is unavailable, optimization relies on score-function (REINFORCE) gradient estimators. In this paper, we provide a comprehensive analysis of score-function gradient estimators for importance-weighted objectives by characterizing their asymptotic expectation and variance. We establish that standard variational inference for Monte Carlo objectives (VIMCO) gradient estimators suffer from a vanishing signal-to-noise ratio (SNR) as $N$ increases, which arises from a suboptimal control variate choice and can impair optimization. To address this issue, we derive a general mechanism for constructing VIMCO gradient estimators that avoid SNR collapse. We identify the High-Effective Sample Size optimal variance importance sampling estimator as belonging to this category of estimators and establish its $\sqrt{N}$ SNR scaling. This estimator does not, however, have zero variance at optimality and achieving its $\sqrt{N}$ SNR rate may require prohibitively large $N$ when the importance weights are degenerate. We overcome these limitations by introducing VIMCO gradient estimators that attain zero variance at optimality and support importance weight tempering. Experiments with discrete latent-variable and state-space models validate the theoretical predictions and illustrate the benefits of our estimators.

Figures & tables

Figure 1: SNR as a function of

N

for different values of

\alpha

when

\phi=1

in the Gaussian example. The solid lines correspond to the SNR estimates for different VIMCO gradient estimators, while the dashed lines correspond to the theoretical predictions. The shaded area represents the 5% to 95% percentile range across 10 independent runs.

Figure 2: Performance of VIMCO gradient estimators when $\alpha=0$ in the Gaussian example. The shaded area represents the 5% to 95% percentile range across 10 independent runs.

Figure 3: Performance of VIMCO gradient estimators in the SBN example averaged over 3 independent runs and expressed in terms of marginal log-likelihood (left) and ELBO (right).

Figure 4: SNR (left), gradient magnitude (middle) and gradient variance (right) as a function of $N$ in the SBN example at a fixed trained SBN checkpoint.

Figure 5: SNR for the VIMCO-zvAM( $\alpha$ ) and VIMCO-zv $\star$ ( $\alpha$ ) gradient estimators as a function of $N$ for various values of $\alpha$ in the SSMs example. The shaded area shows the standard deviation across replications.

Method	$S_{\text{total}}=100$		$S_{\text{total}}=500$
Method	MC Std	# iter	MC Std	# iter
ELBO-cv	0.0445	1499 (258)	0.0194	1647 (188)
VIMCO-zv $\star$ (annealed)	0.0428	782 (27)	0.0108	997 (146)
VIMCO-zvAM (annealed)	0.0536	776 (76)	0.0109	1510 (148)

Table 1: The table reports the Monte Carlo standard deviations of the variational parameter estimates, together with the number of iterations required, across 20 independent runs, and for two cases of per-iteration budget

S_{\text{total}}

Figure 6: Final marginal posterior estimates in the Variational Bayesian SSMs example.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Expectation and variance as a function of

N

for different values of

\alpha

when

\phi=0.1

in the Gaussian example. The solid lines correspond to the absolute mean and variance estimates for different VIMCO gradient estimators, while the dashed lines correspond to the theoretical predictions. The shaded area represents the 5% to 95% percentile range across 10 independent runs.

Figure 8: Performance of VIMCO gradient estimators in the SBN example averaged over 3 independent runs and expressed in terms of marginal log-likelihood (left) and ELBO (right).

Variational Inference

Variational inference approximates complex Bayesian posteriors with tractable distributions that support faster inference and uncertainty estimates.

See all 143 papers

Feb 21, 2026

Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space

For approximating a target distribution given only its unnormalized log-density, stochastic gradient-based variational inference (VI) algorithms are a popular approach. For example, Wasserstein VI (WVI) and black-box VI (BBVI) perform gradient descent in measure space (Bures-Wasserstein space) and parameter space, respectively. Previously, for the Gaussian variational family, convergence guarantees for WVI have shown superiority over existing results for black-box VI with the reparametrization gradient, suggesting the measure space approach might provide some unique benefits. In this work, however, we close this gap by obtaining identical state-of-the-art iteration complexity guarantees for both. In particular, we identify that WVI's superiority stems from the specific gradient estimator it uses, which BBVI can also leverage with minor modifications. The estimator in question is usually associated with Price's theorem and utilizes second-order information (Hessians) of the target log-density. We will refer to this as Price's gradient. On the flip side, WVI can be made more widely applicable by using the reparametrization gradient, which requires only gradients of the log-density. We empirically demonstrate that the use of Price's gradient is the major source of performance improvement.

Kyurae Kim, Qiang Fu, Yi-An Ma +2

University of Pennsylvania, Philadelphia, U.S. · Yale University, New Haven, U.S. · University of California San Diego, San Diego, U.S. +1

Variational Inference

Ask PDF

Oct 17, 2026

Denoising Implicit Variational Inference

Semi-implicit variational inference (SIVI) expands the representational capacity of variational families, but optimizing the Evidence Lower Bound (ELBO) remains challenging because the marginal score of the semi-implicit distribution is generally intractable. Unbiased Implicit Variational Inference (UIVI) and related methods address this difficulty through reparameterized ELBO gradients, but existing approaches estimate the required score indirectly through reverse-conditional sampling or approximation, which can be computationally costly and unstable. We propose Denoising Implicit Variational Inference (DIVI), which learns the marginal score directly via denoising score matching. The learned score is used as a plug-in estimator in the pathwise ELBO gradient, replacing inner reverse-conditional sampling with a score-network evaluation. We analyze DIVI as inexact stochastic gradient ascent and show, under stated assumptions, that the averaged stationarity measure is controlled by the usual optimization term and the average score-matching error. Empirically, we evaluate DIVI on both synthetic distributions and a variety of real-data Bayesian inference tasks. The results show that DIVI improves over the current UIVI baselines while reducing the cost of ELBO-gradient estimation.

Zitong Wang, Zekun Wu, Longlin Yu +1

School of Mathematical Sciences, Peking University

Variational Inference Bayesian Inference

Ask PDF

Oct 7, 2026

Transformed Samplers with Variance Reduction

Markov chain Monte Carlo (MCMC) methods are the standard tool for computing expectations under complex probability distributions. Control variates reduce the variance of the resulting estimates, but a good control variate requires solving the Poisson equation of the sampler, which rarely admits a closed-form solution. Exact solutions are available when the sampler's kernel has a known spectral decomposition on a simple reference density. In our work, we extend these solutions to general targets through a learned change of variables. A bijection, such as a normalizing flow, is trained so that the target becomes close to the reference in a latent space, and we show that Markov kernels and their Poisson solutions are transformed by any bijection. Running such samplers in the latent space then yields explicit control variates, and the estimator is consistent under mild tail conditions on the map and target. Importance sampling (IS) from the flow is the limiting case of the same construction and the control variates apply to it as well. Experiments on synthetic targets and real posteriors compare the procedure against state-of-the-art samplers and control variates.

Siran Liu, Michalis Tisias, Petros Dellaportas

Department of Statistical Science, University College London, UK. · Google DeepMind, UK. · Department of Statistical Science, University College London, UK, and Department of Statistics, Athens University of Economics and Business, Greece.

Markov Chain Monte Carlo Variance Reduction

Ask PDF

Abstract

Variational Inference

Explore similar work

Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space

Denoising Implicit Variational Inference

Transformed Samplers with Variance Reduction