cs.LGFeb 19, 2026

Variational inference via radial transport

Authors: Luca Ghafourpour, Sinho Chewi, Alessio Figalli, Aram-Alexandre Pooladian

Organizations: ETH Zürich · Cambridge · Yale University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In variational inference (VI), the practitioner approximates a high-dimensional distribution ππ with a simple surrogate one, often a (product) Gaussian distribution. However, in many cases of practical interest, Gaussian distributions might not capture the correct radial profile of ππ, resulting in poor coverage. In this work, we approach the VI problem from the perspective of optimizing over these radial profiles. Our algorithm radVI is a cheap, effective add-on to many existing VI schemes, such as Gaussian (mean-field) VI and Laplace approximation. We provide theoretical convergence guarantees for our algorithm, owing to recent developments in optimization over the Wasserstein space--the space of probability distributions endowed with the Wasserstein distance--and new regularity properties of radial transport maps in the style of Caffarelli (2000).

Figures & tables

  Compute stochastic gradient ∇^λF(Tλ(k))\displaystyle\text{Compute stochastic gradient }\widehat{\nabla}_{\lambda}\mathcal{F}(T_{\lambda^{(k)}}) 
  λ(k+1)=ProjR+J+1,∥⋅∥Q(λ(k)−hQ−1∇^λF(Tλ(k)))\displaystyle\lambda^{(k+1)}={\rm Proj}_{\mathbb{R}^{J+1}_{+},\|\cdot\|_{Q}}\bigl(\lambda^{(k)}-hQ^{-1}\widehat{\nabla}_{\lambda}\mathcal{F}(T_{\lambda^{(k)}})\bigr) 
Algorithm 1 radVI
Figure 1: Convergence of radVI for various target distributions. See Table 1 for final-iterate comparisons between GVI and LA.
 Isotropic targets
MethodGaussianLaplaceLogisticStudent- tt
LA2.45×10−42.45\times 10^{-4}20.0020.001.6×1031.6\times 10^{3}25.8725.87
GVI7.34×10−47.34\times 10^{-4}8.248.243.963.961.991.99
radVI1.15×10−41.15\times 10^{-4}5.37×10−25.37\times 10^{-2}1.84×10−11.84\times 10^{-1}1.19×10−11.19\times 10^{-1}
Table 1: Estimated squared Wasserstein distance between various VI solutions for learning isotropic targets.
Figure 2: Comparing learned radial profiles of radVI versus other approximation methods for learning the Student- tt distribution in the isotropic ( top ) and anisotropic case (bottom) .
Figure 3: In the case where π\pi is an isotropic Gaussian with d=50d=50 , we verify that radVI is robust to the choice of α\alpha .
Figure 4: Top: Comparing whitening methods for learning the anisotropic logistic distribution, with and without radVI . Bottom: Visual comparison of true target samples, those generated by LA, and ours (LA+ radVI ).
 d=25d=25
ParameterE[z2]=4\mathbb{E}[z^{2}]=4E[x12]≈7.389\mathbb{E}[x_{1}^{2}]\approx 7.389P(∣z∣>2)≈0.317\mathbb{P}(|z|>2)\approx 0.317
GVI0.274 ±\pm 4×10−34\times 10^{-3}1.12 ±\pm 1.4×10−21.4\times 10^{-2}0
GVI+radVI2.41\bm{2.41} ±\pm 2×10−22\times 10^{-2}5.96\bm{5.96} ±1.0×10−1\pm 1.0\times 10^{-1}0.214\bm{0.214} ±\pm 3×10−33\times 10^{-3}
 d=50d=50
ParameterE[z2]=4\mathbb{E}[z^{2}]=4E[x12]≈7.389\mathbb{E}[x_{1}^{2}]\approx 7.389P(∣z∣>2)≈0.317\mathbb{P}(|z|>2)\approx 0.317
GVI0.328 ±\pm 5×10−35\times 10^{-3}1.61 ±\pm 2×10−22\times 10^{-2}0
Table 2: In d∈{25,50}d\in\{25,50\} , we compare the performance of GVI and GVI+ radVI for parameter estimation.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

 Isotropic targets
MethodGaussianLaplaceLogisticStudent- tt
LA1.48×10−41.48\times 10^{-4}43.0443.047450745068.8968.89
GVI5.07×10−45.07\times 10^{-4}18.3418.348.678.675.215.21
radVI3.71×10−43.71\times 10^{-4}7.67×10−27.67\times 10^{-2}1.96×10−11.96\times 10^{-1}1.89×10−11.89\times 10^{-1}
Appendix
Table 3: Estimated Wasserstein distance between various VI solutions for learning isotropic targets in d=100d=100 .
Figure 5: Comparing learned radial profiles of radVI versus other approximation methods for the isotropic Student- tt distribution.
Figure 6: Comparing learned radial profiles of radVI versus other approximation methods for the isotropic logistic distribution.
Figure 7: Visual comparison of true target samples, those generated by LA, and ours (LA+ radVI ), for learning the anisotropic Student- tt distribution.
Figure 8: Left: Comparing whitening methods for learning the anisotropic Laplace distribution, with and without radVI . Right: Visual comparison of true target samples, those generated by GVI, and ours (GVI+ radVI ).

Explore similar work

CardsList

  1. Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space

    Feb 21, 2026Kyurae Kim, Qiang Fu, Yi-An Ma +2Variational Inference

    University of Pennsylvania, Philadelphia, U.S. · Yale University, New Haven, U.S. · University of California San Diego, San Diego, U.S. +1

  2. Even More Guarantees for Variational Inference in the Presence of Symmetries

    Apr 23, 2026Lena Zellinger, Antonio VergariVariational Inference

    School of Informatics University of Edinburgh

  3. Variational Inference via Entropic Transport Descent

    Jun 24, 2026Vincent Pacelli, Akash Ratheesh, Evangelos TheodorouBayesian InferenceVariational Inference

    School of Aerospace Engineering Georgia Institute of Technology Atlanta, GA 30332