stat.MENov 3, 2026

Post-stratified importance sampling

Authors: Yuling Yao, Sergei Tikhonov

Organizations: Department of Statistics and Data Science, University of Texas at Austin

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Importance sampling is a building block of modern statistical computing, but the weighted estimate is prone to high or infinite variance. Moreover, its empirical standard error is often unreliable, leaving importance sampling without practical uncertainty quantification. Motivated by multilevel regression and poststratification (MRP) in survey sampling, we propose post-stratified importance sampling (PIS) for estimating target expectations from proposal draws. PIS reformulates arbitrarily high-dimensional weighting into two one-dimensional probabilistic modeling problems: regression for the conditional mean given the importance ratio and marginal density estimation for the importance ratio. Different modeling choices yield a family of PIS estimators, encompassing ordinary importance sampling as a special case. We develop a model-based implementation that combines one-dimensional Gaussian process regression for the conditional mean and score-informed heavy-tailed density estimation for the marginal law, using both the observed weights and the available density evaluations. We study the oracle gains of PIS and establish consistency under general and model-specific conditions. Simulations and applications to cross-validation demonstrate improvements in estimation accuracy and uncertainty quantification.

Figures & tables

Figure 1: qw(w)q_{w}(w) : the marginal distribution of the scaler importance ratio under the proposal. We model the tail area (largest MM samples) by a generalized Pareto distribution, and the remaining bulk by a Dirichlet process mixture.
 (k^,τ^)←arg⁡min⁡k,τ∑i=1M[s(S−i+1)−ddwlog⁡qGPDk,τ(w)∣w=w(S−i+1)]2.(\hat{k},\hat{\tau})\leftarrow\arg\min_{k,\tau}\sum_{i=1}^{M}\left[s_{(S-i+1)}-\left.\frac{\mathrm{d}}{\mathrm{d}w}\log q_{\mathrm{GPD}}^{k,\tau}(w)\right|_{w=w_{(S-i+1)}}\right]^{2}. 
Algorithm 1 Score based marginal density estimation of ww
 μ^PIS←∑ℓ=1L−1wℓ+1⋆−wℓ⋆2(m^ℓp^ℓ+m^ℓ+1p^ℓ+1).\hat{\mu}_{\mathrm{PIS}}\leftarrow\sum_{\ell=1}^{L-1}\frac{w_{\ell+1}^{\star}-w_{\ell}^{\star}}{2}\left(\hat{m}_{\ell}\hat{p}_{\ell}+\hat{m}_{\ell+1}\hat{p}_{\ell+1}\right). 
Algorithm 2 Poststratified importance sampling
Figure 2: Comparison of PIS with regular importance sampling and PSIS on the benchmarking tasks. We report the root mean squared error (RMSE) of the estimated Ep[θ1]\mathbb{E}_{p}[\theta_{1}] (upper row), and Ep[θ12]\mathbb{E}_{p}[\theta_{1}^{2}] (lower row).
Figure 3: With both proposal and target being multivariate Gaussian, we compare PIS with self-normalized importance sampling and Pareto smoothed importance sampling, measuring performance by RMSE of the estimated Ep[θ1]\mathbb{E}_{p}[\theta_{1}] . Larger kk indicates a harder problem.
Figure 4: RMSE of the estimated Ep[θ1]\mathbb{E}_{p}[\theta_{1}] under different tail heaviness and dimensions. PIS exhibits a larger gain in higher-dimensional tasks.
Figure 5: RMSE of the estimated Ep[1{θ1>2}]\mathbb{E}_{p}[1\{\theta_{1}>2\}] under different tail regimes.
Figure 6: We use importance weighting to approximate leave-one-out metrics, including the probability integral transformation, log predictive density, coverage for a fixed interval [−2.6,2.6][-2.6,2.6] , and the interval score for (0.05,0.95)(0.05,0.95) quantiles. In this Gaussian likelihood example, PIS generally outperforms PSIS and IS.
Figure 7: Posterior-mean estimation error after calibrating mean-field variational approximations for five PosteriorDB models.
Figure 8: Empirical versus nominal coverage for intervals from three methods. Perfect calibration is the diagonal line.
Figure 9: Mean interval score for nominal 90%90\% intervals as a function of sample size at d=32d=32 .
More in this topic

Importance Sampling

Importance sampling uses weighted samples from easier or adaptive distributions to estimate quantities under a target distribution.

See all 78 papers

Explore similar work

CardsList

  1. Beyond Effective Sample Size: Effective Number of Proposals for Adaptive Importance Sampling

    Aug 15, 2026Ali Mousavi, Victor ElviraImportance Sampling

    Department of Computer Engineering, Islamic Azad University, Neyshabur Branch, Neyshabur, Iran · School of Mathematics, University of Edinburgh, United Kingdom

  2. Fixed-Size Active Statistical Inference

    Oct 19, 2026Erik Skalnes, Michael OberstVariance Reduction

    Department of Computer Science, John Hopkins University, Baltimore, MD, 21218

  3. Control Variate Design for Score-Function Gradient Estimators in Importance-Weighted Variational Inference

    Feb 1, 2026Kamélia Daudel, Minh-Ngoc Tran, Cheng ZhangVariational InferenceVariance Reduction

    ESSEC Business School · The University of Sydney Business School · Peking University