How Many Posterior Samples? Calibrated Stopping for Adaptive Sensing

Vincent Corlay; Andriy Enttsel

stat.MLSep 18, 2026

How Many Posterior Samples? Calibrated Stopping for Adaptive Sensing

Authors: Vincent Corlay, Andriy Enttsel

Organizations: Mitsubishi Electric R&D Centre Europe

Abstract

In classification-oriented adaptive sensing, posterior samples characterize uncertainty at the current measurement state and can serve two roles: they may guide the next sensing direction, while their class labels provide votes for the candidate classes and determine whether sensing should continue. We focus on the stopping layer that turns these votes into a declaration, without modifying the posterior sampler or sensing directions. A natural plug-in rule declares when the observed vote share exceeds a threshold. We show that this threshold is not itself a confidence guarantee: when the underlying vote mass equals the threshold, the plug-in rule declares about half the time. As alternatives, we calibrate a fixed-sample rule and a finite-horizon sequential rule to a prescribed false-declaration probability, and study exact curtailment, which stops a fixed-pool rule once its final verdict is forced. We then derive how one-round declaration probabilities determine posterior-sample cost and classification accuracy along a sensing path. On MNIST with DDRM and a fixed PCA-guided probe sequence, curtailment saves up to 62% of posterior samples. Among the evaluated rules at matched operating points, sequential stopping reduces the cost the most. At a high accuracy, that same sequential rule can trade more posterior samples for fewer measurements.

Figures & tables

Panel of Figure 1 — Figure 1: At each measurement state, posterior samples are classified into votes. The vote rule may request another posterior sample, declare a class, or return control to the outer sensing loop.

Rule	What the rule does
Plug-in	Wait for $S$ samples, then declare the empirical leader if its observed share exceeds $\tau$ .
One-look	Wait for $S$ samples, then declare only if the leader reaches a count calibrated to a chosen error probability.
Sequential	Inspect the count after every sample and declare when a calibrated sequential boundary is first crossed.
Curtailment	Apply a fixed-pool rule, but stop generating samples once every possible continuation gives the same final verdict.

Table 1: Vote-based stopping rules and their use of early stopping.

Scheme	$\tau/\tau^{\prime}$	Accuracy	Rounds	Samples
Scheme	$\tau/\tau^{\prime}$	M/P 32 /P 128	M	M/P 32 /P 128
Target measured accuracy $\approx 1.00$
Plug-in (full)	.90/.77	.98/.97/.98	12.3	406/384/399
Plug-in (curt.)	.90/.77	.98/.97/.98	12.3	154/137/144
One-look	.75/.77	.98/.97/.98	12.3	154/137/144
Sequential ( $n_{\max}{=}32$ )	.78/.78	.98/.98/.98	12.5	118 /105/110

Table 2: MNIST sensing results with all considered rules at three target accuracies.

Explore similar work

Jun 13, 2026cs.LG

False Sense of Safety in Selective Signal Classification: Auditing Bound Tightness and Exchangeability for Risk Control

Selective prediction with distribution-free risk control promises that, with confidence 1-delta over the calibration draw, the error rate of accepted inputs stays below a user budget alpha. We audit this promise on signal-domain detectors -- machine anomalous-sound detection (ASD) and AI-generated-image forensics -- for four calibration rules: uncertified empirical thresholding (NAIVE) and certified Hoeffding, Clopper-Pearson (CP), and betting (WSR) upper confidence bounds. We report three findings. (i) NAIVE thresholding, common in practice, exceeds its declared budget in 49-73% of synthetic trials (n=200 calibration points) and in up to 68% of real-data splits: a false sense of safety rather than a broken theorem, since the rule never had a certificate. (ii) Tightness matters: CP and WSR certify substantial coverage where Hoeffding certifies none, with zero observed budget overruns under exchangeable splits. (iii) Under grouped deployment (unseen machine types or generators), certified rules overrun in 9-30% of trials -- far above delta -- showing the failure lies in the broken exchangeability premise, not in the bounds; a conservative per-group threshold restores validity at a severe coverage cost.

Jingwen Zhou, Mingzhe Wang

Xidian University, Xi’an, China

Adaptive Thresholding Model Auditing Falsehood

Ask PDF

Jul 30, 2026cs.LG

Held-out evidence resolves follow-up measurement decisions in biological screens

Machine learning determines which follow-up measurements biological screens collect. In a six-rule Cell Painting battery, the highest-value rule would re-image 96.01% of the library and had a 97.14% false-activation upper bound, showing why predicted value alone cannot justify replacing a fixed plan. We developed OPAL, a held-out decision test that freezes a rule and judges unnecessary measurement, coverage and value after cost against archive-specific criteria fixed before final evaluation. A development-selected sparse Cell Painting rule had 18.2-fold lower added-well burden, but its false-discovery bound exceeded 35%, so the fixed plan remained. LINCS--LJP favored broad acquisition under point-estimate criteria set during development, not selective saving. CTRP required fallback because its frozen score missed measured opportunity. OPAL separates optimization from evidence sufficient to change an experiment.

Jia Bi, Samuel Pinilla, Chenyang Zhu

Scientific Computing Department, Science and Technology Facilities Council, Rutherford Appleton Laboratory, Didcot, UK · Diamond Light Source, Harwell Science and Innovation Campus, Didcot, UK · School of Electronics and ComputerSouthampton,Science,UK University of Southampton

Acquisition Measures

Ask PDF

Jun 10, 2026stat.ML

Conformal Bayes under Label Shift: Post-Hoc Calibration vs. In-Training Adaptation

Conformal Bayes combines Bayesian posterior predictives with conformal calibration to produce prediction sets that are both statistically valid and geometrically efficient. We study conformal Bayes under label shift from a unified perspective, identifying two complementary approaches that restore nominal target-domain coverage through importance-weighted conformal calibration but operate through independent mechanisms. \emph{Post-hoc calibration} tilts the posterior predictive toward the target domain and corrects the conformal threshold via an importance-weighted quantile, leaving the parameter posterior unchanged. \emph{In-training adaptation} tilts the parameter posterior itself to the target domain, producing a corrected predictive whose highest predictive density region serves as the highest predictive density (HPD)-based prediction set under the fitted target predictive; efficiency is model-dependent and does not imply finite-sample conditional optimality. Two controlled experiments isolate the regime-dependence of each strategy: in the low-dimensional, well-estimated regime Strategy~~A produces the narrowest valid intervals, while in the high-dimensional, underdetermined regime Strategy~~B achieves up to

43\%

width reduction at unchanged coverage, under the stated source-sampling and label-shift assumptions.

Seungjin Choi

1CROID Research and aSSIST University, Seoul, Korea.

Posterior Predictive Distribution

Ask PDF