Bayesian simulation-based inference (SBI) can struggle to recover increasingly concentrated posteriors as observation size grows. We consider learning sufficient summary statistics jointly with the posterior as a general principle for scalable inference in various SBI methods. A common measure taken in SBI is dimension reduction through predetermined summaries, which may partially discard posterior information. Our approach instead imposes structure on how the posterior learner depends on a learned summary (e.g., affinity), and learns the summary simultaneously with the posterior. We identify an important mechanism named two-way guidance: under a structural restriction on the summary, the posterior sufficiency of the learned summary simplifies posterior learning, whereas a suitable posterior objective guides the learned summary towards posterior sufficiency. This mechanism explains a counterintuitive finding that the structural restriction on the summary statistics improves posterior learning, rather than limit its performance. We establish theoretical guarantees on both posterior estimation error and the learned summary's departure from sufficiency, thus identifying conditions on the simulation budget for recovering increasingly concentrated posteriors as observation size grows.
Figures & tables
Figure 1: Gaussian posterior draws and contours, with the oracle posterior in the first row and increasing n across columns. All panels use common (μ,σ2) axes.
Figure 2: Gaussian posterior error: mean and SD across three seeds. Solid: summaries; dashed: raw data.
Figure 3: Brock–Hommes energy score: mean and SD across three seeds. Solid: summaries; dashed: raw data.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Brock–Hommes posterior draws and marginal KDE contours: (a) (g2,b2) and (b) (g3,b3) . Rows show the five summary-based learners followed by their raw-data counterparts; columns within each half increase n . All panels use the first observed series, training seed 0 , and 2,000 posterior draws. Solid and dashed contours enclose 50% and 90% of the full fitted marginal KDE; stars mark the generating parameter. Axes are shared within each half, and some contours extend outside the displayed range.
Figure 5: Brock–Hommes energy scores for NPE and NRE on the same first observed series: mean and sample SD across three training seeds, using 2,000 posterior draws per run. Solid lines with filled markers denote learned summaries; dashed lines with hollow markers denote raw observations. Lower scores are better, and error bars show SD rather than confidence intervals.
In this work, we study TabPFN as a training-free, modular summary network for simulation-based Bayesian inference (SBI). Tabular foundation models such as TabPFN are pretrained on broad families of synthetic tabular data-generating processes and adapt at test time through in-context learning, making them natural candidates for SBI, where posterior estimation often depends on learning informative summaries of simulated observations. We propose PFN-NPE: a general recipe that uses a pretrained TabPFN encoder as a fixed summary network for simulator outputs, then pairs the resulting summaries with a downstream inference head chosen for the problem. With normalizing flows as the default inference head, PFN-NPE matches established posterior approximation methods and sometimes outperforms them. More importantly, diagnostic probes show that the TabPFN-derived summaries often preserve useful posterior location and marginal information. These analyses also reveal a limitation in that TabPFN-derived summaries may struggle to represent the joint posterior structure even when the marginals are well recovered. Still, our experiments show that TabPFN can serve as an effective summary network across a diverse set of SBI settings, with the inference network left modular and task-dependent.
Elliot Pickens, Chiraag Gohel, Sidharth Satya
University of Pennsylvania · The George Washington University · Brown University
Empirical Bayes (EB) performs simultaneous inference across many related latent variables. Classical EB assumes that the likelihood p(x | z) is tractable. In many scientific applications, however, the likelihood is available only through a simulator. This paper develops EB for such implicit likelihoods. We introduce simulation-based empirical Bayes (SBEB), which connects nonparametric EB to simulation-based inference (SBI). SBEB computes EB estimates without an explicit density by using the observed data, simulator samples, and an amortized inference network. SBEB iteratively refines the fitted EB prior toward the population prior. With several scientific simulators and real-world data, we demonstrate that SBEB improves accuracy over SBI with a fixed prior.
Xinwei Shen, Diana Cai, Cheng Zhang +1
Department of Statistics University of Washington · Department of Computer Science Cornell University · School of Mathematical Sciences and Center for Statistical Science Peking University +1
Neural simulation-based inference (SBI) has been widely successful in inferring a relatively small number of interpretable parameters from potentially high-dimensional observations, such as images or time series. Accordingly, representation learning in SBI has focused almost exclusively on compressing the observations used to condition the posterior. More recently, however, SBI has begun to target increasingly high-dimensional parameter spaces, raising the complementary question of whether the inference target itself should be compressed. Our answer is a practical merger of SBI and latent generative modeling, which learns a low-dimensional representation of the simulator parameters, performs posterior inference directly in this latent space, and maps posterior samples back to the original parameter space. We characterize the conditions under which latent-space inference recovers the desired target posterior and systematically study its empirical trade-offs. Across four case studies and three generative families, we compare latent and standard estimators while controlling for network capacity, regularization, optimization, and training compute. At matched training compute, latent-space inference achieves accuracy and marginal calibration comparable to direct target-space inference while sampling up to more than an order of magnitude faster.
Lars Kühmichel, Stefan T. Radev, Bhanu Prasanna Koppolu +3
Department of Statistics, TU Dortmund University, Germany · Department of Cognitive Science, Rensselaer Polytechnic Institute