Simulation-Efficient Double-Flow Neural Posterior Estimation for Agent-Based Models

Agent-based models (ABMs) are widely used in economics and social sciences. They represent heterogeneous personal behaviors and their interactions by using simulations, whereas their likelihoods are often intractable for parameter estimations. Therefore, most traditional statistical inference techniques are hindered. Driven by recent advances in probabilistic machine learning, recent works have applied neural networks to achieve likelihood-free inference in large-scale ABMs. Specifically, Neural Posterior Estimation using normalizing flows provide amortized Bayesian inference from simulations, yet a pre-trained estimator may become inaccurate when the target task changes. In this work, we investigate the practicality of adapting a pre-trained conditional normalizing flow through a unified sequential neural posterior estimation test-time training (SNPE-TTT) framework. The framework includes four adaptation strategies: full-parameter adaptation, low-rank adaptation, and two update spaces identified from target-task gradients. We evaluate all four methods across economic simulation models with both tractable and intractable likelihoods. These benchmarks demonstrate that the sequential neural posterior estimation test-time training framework can adapt pre-trained posterior estimators to related inference tasks using a limited budget of target simulations. Posterior recovery and computational benefits depend on the task and adaptation strategy, with lower target-stage cost in some settings but no consistently superior posterior accuracy.

Neural Posterior Estimation (NPE) enables rapid parameter inference for complex simulators with intractable likelihoods. NPE trains an inference network to estimate a probability density over parameters given data, typically assumed to be continuous. However, many scientific models involve parameter spaces that are mixed, that is, they contain both discrete and continuous dimensions. We address this limitation by extending NPE to mixed parameter spaces through an inference network that jointly handles discrete and continuous parameters. The inference network factorizes the joint posterior into discrete and continuous components, combining an autoregressive classifier for the discrete parameters with a generative model for the continuous parameters, trained jointly under a single simulation-based objective. In addition, we propose a diagnostic tool to assess the calibration of the mixed posterior approximation. Across tractable toy examples and real-world scientific simulators, our joint inference approach yields accurate and calibrated posteriors. The inference framework is available in the sbi Python package.

Deep learning models are being used for the analysis of parametric statistical models based on simulation-only frameworks. Bayesian models using normalizing flows simulate data from a prior distribution and are composed of two deep neural networks: a summary network that learns a sufficient statistic for the parameter and a normalizing flow that conditional on the summary network can approximate the posterior distribution. Here, we explore frequentist models that are based on a single summary network. During training, input of the network is a simulated data set based on a parameter and the loss function minimizes the mean-square error between learned summary and parameter. The network thereby solves the inverse problem of parameter estimation. We propose a branched network structure that contains collapsing layers that reduce a data set to summary statistics that are further mapped through fully connected layers to approximate the parameter estimate. We motivate our choice of network structure by theoretical considerations. In simulations we demonstrate three desirable properties of parameter estimates: finite sample exactness, robustness to data contamination, and algorithm approximation. These properties are achieved offering the the network varying sample size, contaminated data, and data needing algorithmic reconstruction during the training phase. In our simulations an EM-algorithm for genetic data is automatically approximated by the network. Simulation only approaches seem to offer practical advantages in complex modeling tasks where the simpler data simulation part is left to the researcher and the more complex problem of solving the inverse problem is left to the neural network. Challenging future work includes offering pre-trained models that can be used in a wide variety of applications.