Conditional Neural Bayes Ratio Estimation for Experimental Design Optimisation

S. A. K. Leeney; T. Gessey-Jones; W. J. Handley; E. de Lera Acedo; H. T. J. Bevins; J. L. Tutt

astro-ph.IMMar 27, 2026

Conditional Neural Bayes Ratio Estimation for Experimental Design Optimisation

Authors: S. A. K. Leeney, T. Gessey-Jones, W. J. Handley, E. de Lera Acedo, H. T. J. Bevins, J. L. Tutt

Organizations: Cavendish Astrophysics, University of Cambridge, Cambridge CB3 0HE, UK. · Kavli Institute for Cosmology, University of Cambridge, Cambridge CB3 0HA, UK. · PhysicsX, London, UK. · Institute of Astronomy, University of Cambridge, Cambridge CB3 0HA, UK.

Abstract

For frontier experiments operating at the edge of detectability, instrument design directly determines the probability of discovery. We introduce Conditional Neural Bayes Ratio Estimation (cNBRE), which extends neural Bayes ratio estimation by conditioning on design parameters, enabling a single trained network to estimate Bayes factors across a continuous design space. Applied to 21-cm radio cosmology with simulations representative of the REACH experiment, the amortised nature of cNBRE enables systematic design space exploration that would be intractable with traditional point-wise methods, while recovering established physical relationships. The analysis demonstrates a ~20 percentage point variation in detection probability with antenna orientation for a single night of observation, a design decision that would be trivial to implement if determined prior to antenna construction. This framework enables efficient, globally-informed experimental design optimisation for a wide range of scientific applications.

Figures & tables

Significance	$K$	$\log K$
$1\sigma$	$\sim 3$	$\sim 1.1$
$2\sigma$	$\sim 12$	$\sim 2.5$
$3\sigma$	$\sim 150$	$\sim 5.0$
$5\sigma$	$\sim 1.7\times 10^{6}$	$\sim 14.4$

TABLE I: Approximate correspondence between statistical significance levels and Bayes Factors.

Fig. 1: Detection probability as a function of observation time $\tau$ for three Bayes factor thresholds corresponding to $1\sigma$ ( $K\approx 3$ ), $3\sigma$ ( $K\approx 150$ ), and $5\sigma$ ( $K\approx 1.7\times 10^{6}$ ) significance.

Fig. 2: Detection probability as a function of observation time ( $\tau$ ) and signal amplitude ( $A$ ). Detection probabilities are computed on test sets with signals injected at fixed amplitudes; the network marginalizes over the full signal prior during training. The panels show $5\sigma$ , $3\sigma$ , and $1\sigma$ thresholds, confirming that detection probability improves with longer integration and larger amplitudes.

Fig. 3: Multi-panel visualisation of detection probability as a function of observation time ( $\tau$ ) and signal central frequency ( $f_{0}$ ). Detection probabilities are computed on test sets with signals injected at fixed $f_{0}$ values; the cNBRE network itself marginalizes over the full signal prior during training. The three heatmaps show $5\sigma$ , $3\sigma$ , and $1\sigma$ thresholds (left to right). Marginal plots highlight reduced sensitivity at the band edges.

Fig. 4: Polar heatmap showing the variation in detection probability (radial axis) and achievable significance (colour scale) as a function of antenna orientation angle (angular axis) for several observations over a single night. The four-fold symmetry reflects the interaction of the rectangular dipole beam with galactic foregrounds.

Fig. 5: Comparison of log-Bayes factors estimated by cNBRE (y-axis) versus nested sampling (x-axis) for 1,600 simulated data realisations (811 $M_{1}$ , 789 $M_{0}$ ). Circles denote $M_{1}$ samples (signal present); crosses denote $M_{0}$ samples (null model). The dashed line indicates perfect agreement. Both methods show broad agreement in the decision-relevant regime, with $M_{0}$ samples clustering at negative $\log K$ and $M_{1}$ samples at positive $\log K$ for both estimators. At very high Bayes factors the cNBRE estimates saturate below the nested sampling values, a consequence of classifier output saturation that is expected and practically irrelevant for detection decisions.

Bayesian Experimental Design

Bayesian experimental design uses probabilistic models to choose measurements that improve learning or support better decisions.

See all 63 papers

Cross-Fitting for Neural Posterior Estimation
Oct 19, 2026Jeffrey RegierCross-Fitting Neural Posterior Estimation
Department of Statistics, University of Michigan, Ann Arbor, MI 48109
Identifying Informative Environments for Cognition Parameter Inference via Bayesian Experimental Design
Jul 30, 2026Manisha Dubey, Rimvydas Rubavicius, N. Siddharth +1Computational Cognitive Modeling Bayesian Experimental Design
School of Informatics, University of Edinburgh, United Kingdom
Hybrid Neural Simulation-Based Inference for Robust Applications and Limited-Budget Scenarios
Sep 28, 2026Sean Benevedes, Mani Dehghan, Aishik Ghosh +1Simulation-Based Inference Cost-Aware Inference
School of Physics, Georgia Institute of Technology, Atlanta, GA 30332, USA · School of Physics, Georgia Institute of Technology, Atlanta, GA 30332, USA and Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA · Max Planck Institute for Physics, Boltzmannstr. 8, 85748 Garching, Germany

Oct 19, 2026J/K move · Enter open · S save

Cross-Fitting for Neural Posterior Estimation

Jeffrey Regier

Department of Statistics, University of Michigan, Ann Arbor, MI 48109

Cross-Fitting Neural Posterior Estimation Simulation-Based Inference Conditional Distribution Estimation

Open paper AskPDF arXiv

Neural Posterior Estimation (NPE) fits an amortized conditional density estimator to a simulated training set of complete-data realizations. NPE's training objective, log-likelihood, favors posterior approximations that assign arbitrarily high density to the training points, and typical families of amortized conditional density estimators (e.g., normalizing flows, Gaussian encoders) are flexible enough to oblige. The resulting policy of extreme overconfidence translates poorly to held-out data. Early stopping and other regularizers can prevent this degeneracy, but they are limited to selecting posterior approximations along an unfavorable optimization trajectory set by in-sample residuals, where the posterior mean is underfit while the posterior variance is overfit. To avoid this trade-off, we propose Cross-Fitting NPE (CF-NPE), which uses sample splitting to separate conditional-mean estimation from the estimation of the posterior's higher moments. Using

K

-fold cross-fitting, CF-NPE fits

K

posterior-mean regressors and then fits a conditional density estimator to the resulting out-of-fold residuals. In experiments with synthetic data and in a cosmological application, CF-NPE achieves higher held-out log-likelihood than standard NPE and typically improves calibration. CF-NPE is most useful in high-dimensional, simulation-limited settings.

Abstract

Bayesian Experimental Design

Explore similar work

Cross-Fitting for Neural Posterior Estimation

Identifying Informative Environments for Cognition Parameter Inference via Bayesian Experimental Design

Hybrid Neural Simulation-Based Inference for Robust Applications and Limited-Budget Scenarios