Combining Bayesian and Frequentist Inference for Laboratory-Specific Performance Guarantees in Copy Number Variation Detection

Austin Talbot; Alex V. Kotlar; Yue Ke

stat.MEApr 15, 2026

Combining Bayesian and Frequentist Inference for Laboratory-Specific Performance Guarantees in Copy Number Variation Detection

Authors: Austin Talbot, Alex V. Kotlar, Yue Ke

Organizations: Pillar Biosciences Inc Natick, MA, USA

Abstract

Targeted amplicon panels are widely used in oncology diagnostics, but providing per-gene performance guarantees for copy number variant (CNV) detection remains challenging due to amplification artifacts, process-mismatch heterogeneity, and limited validation sample sizes. While Bayesian CNV callers naturally quantify per-sample uncertainty, translating this into the frequentist population-level guarantees required for clinical validation, coverage rates, false-positive bounds, and minimum detectable copy-number changes, is a fundamentally different inferential problem. We show empirically that even robust Bayesian credible intervals, including coarsened posteriors and sandwich-adjusted intervals, are severely miscalibrated on panels with small amplicon counts per gene. To address this, we propose a hybrid framework that evaluates Bayesian posterior functionals on validation samples and models the resulting squared losses with a Gamma distribution, yielding tolerance intervals with valid frequentist coverage. Three components make the method practical under real-world constraints: (1) imputation that removes the influence of true CNV-positive samples without requiring known ground truth, (2) regularization to address small sample variability, and (3) evidence-based stratification on the log model evidence to accommodate non-exchangeable noise profiles arising from process mismatch. Evaluated on two targeted amplicon panels using leave-one-out cross-validation, the proposed method achieves single-digit mean absolute coverage error across all genes under both process-matched and unmatched conditions, whereas Bayesian comparators exhibit mean absolute errors exceeding 60% on clinically relevant genes such as ERBB2.

Figures & tables

Figure 1: (A) Heatmap of normalized lCNRs $X_{j,k}$ for representative samples (rows) across amplicons grouped by gene (columns). (B) Forest plot of BayesCNV posterior means $\hat{\mu}_{j}$ with $(1{-}\gamma)$ HPD credible intervals for one representative sample. (C) Conditional order-statistic imputation for one representative gene. Left: distribution of $K$ posterior means with top- $m$ suspect values (filled red markers) and threshold $t_{j}$ (dashed line). Right: distribution after replacing suspect values with order-statistic draws. (D) Gamma model fit (solid colored curve) to the empirical distribution of squared imputed posterior means $Y_{j,s}$ (grey) and pseudo observations (pink); the vertical dashed line marks the tolerance quantile $T_{j}$ , and $\sqrt{T_{j}}$ is the minimum detectable lCNR deviation. (E) Stratefied parameter estimates each sample group divided by model evidence. (F) Per-gene tolerance limits $\sqrt{T_{j}}$ in upper and lower evidence strata.

Figure 2: Interval estimation performance as a function of sample size for the four estimators. Left: posterior mean parameter estimate with

95\%

confidence intervals. Center: standard error of each estimator. Right: mean squared error. Data were generated from

\mathcal{N}(\delta,\tau^{2})

with

\delta=0.2

and

\tau^{2}=0.17

over

N\in[5,50]

Figure 3: Calibration curves of the various methods. Top: Coverages in the process-matched experiments. The far left plots the lCNR of a representative process-matched sample, with amplicons targeting the five CNV-relevant genes shown with different colors. The remaining plots show the true coverages on three of the five genes as a function of target coverages. The ideal is shown via a dotted line, while the average coverage and 95% confidence interval for each of the five considered methods is shown in a different color Bottom: These plots have identical interpretation, except when the data come from unmatched (highly noisy) samples.

	Matched					Unmatched
Method	PIK3CA	KIT	MET	FGFR1	ERBB2	PIK3CA	KIT	MET	FGFR1	ERBB2
Bayes	$33\pm 6$	$37\pm 5$	$40\pm 7$	$54\pm 2$	$66\pm 3$	$11\pm 4$	$12\pm 1$	$45\pm 7$	$25\pm 6$	$74\pm 4$
Coarsened	$6.6\pm.9$	$8.0\pm 1$	$18\pm 5$	$14\pm 3$	$42\pm 3$	$15\pm 2$	$9.5\pm 2$	$45\pm 13$	$10\pm 2$	$57\pm 7$
Sandwich	$38\pm 6$	$52\pm 6$	$47\pm 7$	$60\pm 3$	$69\pm 3$	$10\pm 2$	$7.6\pm 1.4$	$43\pm 13$	$20\pm 4$	$49\pm 8$
MSE	$4.2\pm.5$	$4.2\pm.6$	$6.4\pm.6$	$4.2\pm.7$	$5.6\pm.5$	$5.7\pm.5$	$4.4\pm.5$	$8.1\pm 2$	$7.4\pm.5$	$4.2\pm.7$
Gamma	$4.9\pm.2$	$4.4\pm.3$	$6.1\pm.6$	$5.3\pm.4$	$3.0\pm.2$	$4.4\pm.6$	$5.4\pm.1$	$5.5\pm.7$	$7.0\pm.8$	$3.2\pm.2$

Table 1: Mean absolute error

\times 100

with 95% confidence intervals for coverage evaluation on the essential panel using matched and unmatched normals. Results are shown for five methods: Ours, MSE, Bayesian, Sandwich, and Coarsened

Figure 4: The effect of stratefication in heterogeneous mixtures. (A) The distributions of log-likelihoods in the process matched and unmatched populations. (B) The tolerance intervals for MET on each of the subgroups for a stratefied fit, and all samples when group differences are ignored. (C) The associated coverages for each method.

Figure 5: The effect of imputation on MET tolerance interval estimates. (A) plots the KDEs for true, observed, and imputed MET error values. (B) plots between estimated and true 0.95 quantiles across imputation fractions, with the dashed line indicating the true positive fraction. (C) plots the true, observed, and imputed 0.95 quantiles for CCNE, EGFR, FGFR2, and MET.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 6: The bias/variance decomposition of CNV estimates as a function of noraml matching

Uncertainty Calibration

Uncertainty calibration aligns model confidence and predictive intervals with observed correctness and outcome frequencies.

See all 264 papers

Jun 16, 2026

A Bayesian Boolean Matrix Factorization with Application to Copy Number Analysis in Cancer

Binary data factorization is common, but real-valued methods ignore discreteness and yield hard-to-interpret factors. Boolean Matrix Factorization (BooMF) instead decomposes a binary matrix into two lower-rank binary matrices via logical AND and OR, expressing the data as a Boolean disjunction of interpretable patterns. In cancer genomics, BooMF can reveal coordinated feature changes that may drive tumor evolution, unlike rotational or additive decompositions. Most existing BooMF methods are heuristic, greedy, sensitive to initialization, prone to local optima, and do not support principled model selection or uncertainty quantification. We introduce Bayesian Boolean Matrix Factorization (BBMF), a fully conjugate generative model with sparsity-inducing priors. It enforces Boolean constraints, yields interpretable latent factors with coherent uncertainty quantification, and admits Gibbs sampling with closed-form full conditionals. Because cancer evolution often involves widespread, near-simultaneous chromosome-number changes (e.g., whole-genome duplication followed by instability and selection), Boolean factorizations capture these patterns more naturally than additive models. Applied to arm-level copy-number alteration data in multiple myeloma, where entries indicate presence/absence of chromosomal-arm amplifications, BBMF finds a small set of interpretable bicliques linking patient subsets to recurrently co-altered chromosomal arms, providing a compact, biologically meaningful summary of tumor heterogeneity and demonstrating BBMF's utility for uncovering discrete latent structure in complex binary data.

Adolphus Wagala, Mehmet Samur, Giovanni Parmigiani

Department of Data Science, Dana-Farber Cancer Institute · Department of Biostatistics, Harvard T.H. Chan School of Public Health

Cancer Genomics Bayesian Inference Genomics Matrix Factorization

Ask PDF

Oct 8, 2026

When Has a Bayesian Neural Network Sampled Enough? Adaptive Inference Time with Statistical Guarantees

Bayesian neural network predictions are commonly approximated using a fixed number of Monte Carlo samples per input, without controlling the resulting error that comes from this finite sample. We propose the use of confidence sequences to dynamically determine how many samples are needed while maintaining statistical guarantees. We consider several ways in which predictive probabilities are used, including identifying the most likely class, approximating the full predictive distribution, and resolving probability-threshold decisions. Sampling stops once the corresponding decision can be made with the desired guarantee. Experiments show that the method allocates the computational budget efficiently, assigning more samples to ambiguous inputs than to easy inputs while preserving reliable decisions and reducing overall latency relative to a fixed Monte Carlo budget.

Fabian Denoodt, Sibylle Hess

Eindhoven University of Technology

Adaptive Sampling Bayesian Neural Networks Efficient Neural Network Inference

Ask PDF

Aug 11, 2026

Uncertainty-Aware Deep Learning for Genomics Applications: Insights from an Empirical Study

Deep learning models have emerged as the standard computational tool for a wide range of applications in genomics. Yet, uncertainty quantification (UQ) -- and more specifically, the reliability of different uncertainty estimates in this domain -- has received little systematic attention. This work presents an empirical analysis of UQ in deep learning models, focusing on genomics applications. In a series of experiments, we contrast Deep Ensembles, Bayesian Neural Networks, and Monte Carlo-dropout methods. We assess their ability to quantify uncertainty in different scenarios, accounting for common dataset characteristics in two genomic application areas and modalities: sequence-to-activity models, and single-cell expression analysis. Our systematic comparison framework provides guidelines for the applicability and reliability of UQ methods in genomics, highlighting their strengths and limitations in different scenarios. We show that Bayesian Neural Networks are better at capturing uncertainty caused by strong class imbalance and out-of-distribution data in genomics, despite their computational disadvantages. Moreover, we show how uncertainty scores can be used to select high-quality predictions in protein-RNA interactions.

Sepideh Saran, Mahsa Ghanbari, Uwe Ohler

The Berlin Institute for Medical Systems Biology, Max Delbrück Center for Molecular Medicine, Germany · Department of Electrical Engineering and Computer Science, Technical University of Berlin, Germany · Department of Biology, Humboldt University of Berlin, Germany +1

Bayesian Neural Networks Uncertainty Quantification Deep Ensembles Genomics

Ask PDF

Abstract

Uncertainty Calibration

Explore similar work

A Bayesian Boolean Matrix Factorization with Application to Copy Number Analysis in Cancer

When Has a Bayesian Neural Network Sampled Enough? Adaptive Inference Time with Statistical Guarantees

Uncertainty-Aware Deep Learning for Genomics Applications: Insights from an Empirical Study