Latent variable generative models are commonly fit using simple priors over latent variables, but draws from these priors often fail to produce realistic data. This failure is due to a mismatch between the prior and the aggregated posterior, the distribution of latent variables induced by the fitted model and the data. This mismatch is often viewed as evidence that the prior is misspecified and should be replaced. Alternatively, in modern generative models, a two-stage strategy is increasingly used where first, the model is fit, and second, the aggregated posterior is estimated (van den Oord et al.,2017; Rombach et al., 2022.). Synthetic data are then obtained by sampling from this aggregated posterior instead of the prior. To check such procedures, we introduce the aggregated posterior predictive check (APPC). Theoretically, we establish sufficient conditions under which the APPC is asymptotically calibrated. For probabilistic principal component analysis, we show that the APPC can remain calibrated under a misspecified latent prior when pervasive factors permit recovery of the signal space. Experiments with variational autoencoders show that aggregated posterior sampling improves generation for heavy-tailed and clustered data relative to Gaussian prior sampling while performing comparably to models with more flexible latent priors.
Figures & tables
pAPPC=R1r=1∑R1[d(X(r),rep)≥d(Xnew)];
Algorithm 1 Aggregated posterior predictive check
Figure 1: PPCA example. The empirical aggregated posterior predictive generates realistic samples in both well-specified and misspecified cases. The appc p -values are approximately uniform in both cases, unlike prior or conditional-replicate sampling. Top row: example draws of latent variables z (up to an invertible linear transformation). Bottom row: p -values from predictive checks over 500 trials. Kolmogorov-Smirnov tests of uniformity are in red.
Reference distribution Qm
Reference mean μmrep
Reference covariance Vmrep
P0Kθ⋆,post
B⋆μ0
Vmoracle=R⋆+B⋆Σ0B⋆
P0Kθm,post
Bmμ0
Vmlearn=Rm+BmΣ0Bm
Pagg,mKθm,post
Bmxmagg
Vmemp=Rm+BmΣagg,mBm
Table 1: PPCA reference distributions.
Figure 2: Bivariate t. Top: Draws of synthetic data Xrep . Bottom: p -values from model checks over 200 trials. G-EAP and G-DAP p -values are not uniform but have similar performance as more complex generative models. Here, Ntrain=Nagg=10000,Nnew=2000 .
Figure 3: Clustered t. Top: Draws of synthetic data Xrep . Bottom: p -values from model checks over 200 trials. G-EAP and G-DAP p -values are not uniform but show similar behavior to more complex generative models. Here, Ntrain=Nagg=10000,Nnew=2000 .
Figure 4: Financial. Top: Aggregated posterior with standard VAE (learnable variance, 1000 epochs) produces synthetic Xrep which visually capture the heavy-tailed behavior of financial data. Bottom: G-EAP p -values are approximately uniform. Here, Ntrain=Nagg=4000,Nnew=2000 .
Figure 5: MNIST. Top: G-EAP produces recognizable digits. Bottom: G-EAP p -values are approximately uniform. Here, Ntrain=Nagg=10000,Nnew=2000 .
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: PPCA example (fourth cumulant diagnostic). The appc is calibrated; the empirical aggregated posterior sampling strategy generates realistic samples in both well-specified and misspecified cases. Top row: example draws of latent variables z . Bottom row: p -values from predictive checks over 500 trials.
Figure 7: GMM. Top: Draws of data x . Aggregated posterior with standard VAE (learnable variance, 700 epochs) performs better compared to VAEs with true or mixture-based priors. Bottom: p -values from model checks over 200 trials. Here, Ntrain=Nagg=10000,Nnew=2000 .
Hyperparameter
Bivariate t2
Bivariate clustered t2
Bivariate Gaussian mixture
Financial
MNIST
Input Dimension
2
2
2
2
784
Latent Dimension
2
2
2
64
64
Hidden Dimension
128
128
128
128
400
Epochs
700
700
700
1000
350
Batch Size
256
256
256
64
256
Learning Rate
0.001
0.001
0.001
0.0001
0.0001
Appendix
Table 2: Hyperparameter settings of VAE for the datasets
Hyperparameter
Bivariate t2
Bivariate clustered t2
Bivariate Gaussian mixture
Financial
MNIST
T
100
100
100
100
100
Hidden Dimension
128
128
128
128
256
Epochs
700
500
700
300
200
Batch Size
256
256
256
128
256
Learning Rate
0.001
0.001
0.001
0.0001
0.0001
Appendix
Table 3: Hyperparameter settings of LDM for the datasets
Vanilla variational inference finds an optimal approximation to the Bayesian posterior distribution, but even the exact Bayesian posterior is often not meaningful under model misspecification. We propose predictive variational inference (PVI): a general inference framework that seeks and samples from an optimal posterior density such that the resulting posterior predictive distribution is as close to the true data generating process as possible, while this closeness is measured by multiple scoring rules. By optimizing the objective, the predictive variational inference is generally not the same as, or even attempting to approximate, the Bayesian posterior, even asymptotically. Rather, we interpret it as implicit hierarchical expansion. Further, the learned posterior uncertainty detects heterogeneity of parameters among the population, enabling automatic model diagnosis. This framework applies to both likelihood-exact and likelihood-free models. We demonstrate its application in real data examples.
Jinlin Lai, Antonio Linero, Yuling Yao
College of Information and Computer Sciences, University of Massachusetts Amherst · Department of Statistics and Data Sciences, University of Texas, Austin
Generative models are increasingly deployed as substitutes for real data in downstream scientific workflows, yet standard evaluation criteria remain focused on marginal distribution matching. We argue that this represents a fundamental gap: downstream inference is rarely a marginal operation, and a model that passes every univariate diagnostic can still produce structurally unreliable synthetic data. We introduce covariance-level dependence fidelity, measured by D_Sigma(P,Q) = ||Sigma_P - Sigma_Q||_F, as a principled, computable criterion for evaluating whether a generative model preserves the joint structure of data beyond its univariate marginals. Three results formalise this criterion. First, marginal fidelity provides no constraint on dependence structure: D_Sigma can be made arbitrarily large while all univariate marginals match exactly. Second, covariance divergence induces quantifiable downstream instability, including sign reversals in population regression coefficients. Third, bounding D_Sigma provides positive stability guarantees for dependence-sensitive procedures such as PCA via Davis-Kahan-type bounds. Empirical validation across three domains, image data (Fashion-MNIST VAE, n = 60,000), bulk RNA-seq (TCGA-BRCA, n = 1,111), and a small-sample stress test (Alzheimer's gene expression, n = 113), shows that D_Sigma/delta consistently distinguishes structure-discarding from structure-preserving generators in cases where standard marginal diagnostics show little separation, confirming that covariance-level fidelity provides information orthogonal to existing evaluation metrics across domains and sample sizes.
Modern generative modeling is dominated by transport from a noise prior to data. We propose an alternative paradigm in which generation is performed by a discrete stochastic dynamics that leaves the data distribution invariant, initialized from data-supported states rather than from noise. The framework can utilize any pretrained flow model. We develop two probability-preserving sampling mechanisms, a corrected Langevin dynamics with a Metropolis adjustment and a predictor-corrector flow, that operate directly on existing checkpoints. We validate the framework on a synthetic Swiss-roll target, ImageNet-256 and Oxford Flowers-102, where our samplers consistently improve over the original generation procedures.
Eshed Gal, Md Shahriar Rahim Siddiqui, Moshe Eliasof +1
The University of British Columbia Vancouver, Canada · University of Cambridge Cambridge, United Kingdom