Organizations: Department of Computer Science and Engineering, University of California, Riverside · Leinweber Center for Theoretical Physics, University of Michigan · Department of Physics and Astronomy, University of California, Riverside
Markov chain Monte Carlo (MCMC) requires only the ability to evaluate the likelihood, making it a common technique for inference in complex models. However, it can have a slow mixing rate, requiring the generation of many samples to obtain good estimates and an overall high computational cost. FLARE MCMC is a multi-fidelity layered MCMC method that exploits lower-fidelity approximations of the true likelihood calculation to improve mixing and leads to overall faster performance. Such lower-fidelity likelihoods are commonly available in scientific and engineering applications where the model involves a simulation whose resolution or accuracy can be tuned. Our technique uses recursive, layered chains with simple layer tuning; it does not require the likelihood to take any form or have any particular internal mathematical structure. We demonstrate experimentally that FLARE MCMC achieves larger effective sample sizes for the same computational time across different scientific domains including hydrology and cosmology.
Figures & tables
Figure 1: One sampling step from the finest layer with two coarse fidelities and two iterations per nested chain. Refer to text in Section 3.1 .
Figure 2: Posterior of different fidelities; coarse model is the small angle approximation. See Section 4.1 .
Table 1: Mean ESS for bulk and tail distributions across 50 runs for 10 chains each, and mean value of each parameter with std error across all 500 total runs are listed. Average acceptance rates are listed per layer and the total samples are based on the average cost of likelihood evaluations per sample per method. Each chain is run for a total of 1000 seconds.
Figure 3: ESS as a function of M for two single layered ( J=1 ) methods with fixed computation time of 500 seconds each ran across 50 different chains.
Figure 5: Wasserstein distance to true distributions for the pendulum model.
Figure 7: Average (across 10 chains) absolute error in mean estimates for the four parameters, as a function of total simulation time. In the allotted 48 hours, the chains each sampled 800 samples for (plain) MCMC, 600 samples for FLARE MCMC (single), 500 samples for FLARE MCMC (double), 290 samples for MLDA (single) and 200 samples for MLDA (double)
Figure 8: Cosmology model; Trace plot for a random single run of each method
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Evolution of ωj for doubly nested layers j=1 and j=2
Figure 10: Cosmology model: Pairwise plots for all four parameters. Plotted for a random run of samples collected for 48 hours of MCMC (black) and FLARE MCMC (green).