cs.LGMar 29, 2026

Variational Learning of Fractional Posteriors

Authors: Kian Ming A. Chai, Edwin V. Bonilla

Organizations: DSO National Laboratories, Singapore · CSIRO’s Data61, Australia

Abstract

We introduce a novel one-parameter variational objective that lower bounds the data evidence and enables the estimation of approximate fractional posteriors. We extend this framework to hierarchical construction and Bayes posteriors, offering a versatile tool for probabilistic modelling. We demonstrate two cases where gradients can be obtained analytically and a simulation study on mixture models showing that our fractional posteriors can be used to achieve better calibration compared to posteriors from the conventional variational bound. When applied to variational autoencoders (VAEs), our approach attains higher evidence bounds and enables learning of high-performing approximate Bayes posteriors jointly with fractional posteriors. We show that VAEs trained with fractional posteriors produce decoders that are better aligned for generation from the prior.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Oct 18, 2024

Predictive variational inference: Learn the predictively optimal posterior distribution

Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density such that the resulting posterior predictive distribution is as close to the true data generating process as possible, while this closeness is measured by multiple scoring rules. By optimizing the objective, the predictive variational inference is generally not the same as, or even attempting to approximate, the Bayesian posterior, even asymptotically. Rather, we interpret it as implicit hierarchical expansion. Further, the learned posterior uncertainty detects heterogeneity of parameters among the population, enabling automatic model diagnosis. This framework applies to both likelihood-exact and likelihood-free models. We demonstrate its application in real data examples.
Aug 19, 2026

Theoretical Study on the Evidential Learning-based Variational Autoencoder

A normal--inverse-gamma (NIG) latent hierarchy has four parameters, but its induced latent law does not identify all four. For σ2∼InvGamma(α,β)σ^2\sim\mathrm{InvGamma}(α,β), μ∣σ2∼N(γ,σ2/ν)μ\midσ^2\sim\mathcal{N}(γ,σ^2/ν), and z∣μ,σ2∼N(μ,σ2)z\midμ,σ^2\sim\mathcal{N}(μ,σ^2), the marginal law of zz depends on (ν,β)(ν,β) only through c=β(1+1/ν)c=β(1+1/ν). Hence the reconstruction-visible parameter space is the three-dimensional quotient (γ,α,c)(γ,α,c), with a one-dimensional fiber degree of freedom. For a fixed hierarchical variational objective, exact partial minimization of the forward KL divergence to a complete NIG prior selects a unique prior-relative representative on each fiber, yielding an exact three-coordinate reduction with the same optimum as the four-coordinate objective. Writing ρ0=2β0/ν0ρ_0=2β_0/ν_0 and T=c/{α[(γ−γ0)2+ρ0]}T=c/\{α[(γ-γ_0)^2+ρ_0]\}, we show that inverse canonical allocation 1/νcan1/ν_{\rm can} is an explicit strictly increasing function of TT. For α>1α>1, the ratio uepi/uvar=1/νcanu_{\rm epi}/u_{\rm var}=1/ν_{\rm can} is therefore determined by the quotient state and prior; for rank-only use under a common calibration, TT contains the same coordinatewise ordinal information. The residual prior gauge is characterized rather than eliminated: (γ0,ρ0)(γ_0,ρ_0) govern ordinal dependence, while (ν0,α0)(ν_0,α_0) determine numerical calibration and the analytic ceiling of 1/νcan1/ν_{\rm can}.
Sep 21, 2026

Variational objectives for amortized Bayesian inference in inverse problems: The role of posterior conditioning

Variational autoencoders (VAEs) offer an efficient approach to amortized Bayesian inference for inverse problems, but posterior accuracy can depend strongly on the choice of variational regularization, particularly when the inverse problem contains weakly identified parameter directions. This study investigates three objectives: a reverse Kullback--Leibler formulation (VAE-KL), an asymmetric Jensen--Shannon formulation (VAE-JS), and a Jensen--Shannon--Wasserstein formulation (VAE-JSWA), which replaces the reverse Kullback--Leibler regularizer with the squared 2-Wasserstein distance while retaining forward-Kullback--Leibler posterior supervision. A full-covariance Gaussian encoder and a pre-trained physics-based surrogate are used for amortized posterior inference. A local linear--Gaussian analysis in the generalized Fisher basis is developed to characterize the variance-dependent gradients of the three objectives. The formulations are first evaluated using linear--Gaussian benchmarks with known posterior solutions and subsequently tested on nonlinear physics-based inverse problems, including an inverse problem governed by a linear ODE and two PDE-constrained problems. VAE-KL performs slightly better than the other formulations in the well-conditioned benchmark, where all three approaches yield comparable posterior approximations, whereas VAE-JSWA provides substantially lower posterior errors in the strongly ill-conditioned benchmark. The nonlinear physics-based problems exhibit a similar conditioning-dependent trend, with JS-based formulations providing greater benefit as posterior ill-conditioning increases. These results indicate that posterior conditioning is an important factor in selecting variational objectives and motivate geometry-adaptive variational inference for Bayesian inverse problems.