Sequential Hypothesis Testing

Follow this topic Ask about this topic

Momentum

9 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 39

All topics

When Should an AI Workflow Release? Always-Valid Inference for Black-Box Generate-Verify Systems
May 13, 2026Young Hyun Cho, Will Wei SunAnytime-Valid Inference Sequential Hypothesis Testing
Department of Statistics, Purdue University. · Mitch Daniels School of Business, Purdue University.
Accurate Evaluation of Quickest Changepoint Detectors via Non-parametric Survival Analysis
May 11, 2026Taiki Miyagawa, Akinori F. EbiharaChange-Point Detection Sequential Hypothesis Testing
NEC Corporation, Japan
Adaptive auditing of AI systems with anytime-valid guarantees
May 7, 2026Siyu Zhou, Patrick Vossler, Venkatesh Sivaraman +2Algorithmic Auditing Anytime-Valid Inference
University of California, San Francisco · Stanford University
Optimizing Social Utility in Sequential Experiments
May 7, 2026Ander Artola Velasco, Stratis Tsirtsis, Manuel Gomez-RodriguezMechanism Design Sequential Decision Making
Max Planck Institute for Software Systems, Kaiserslautern, Germany · Hasso Plattner Institute, Potsdam, Germany
CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency
May 7, 2026Hirofumi Ota, Naoto Iwase, Yuki Ichihara +2LLM Evaluation Anytime-Valid Inference
arXiv:2605.05873v1
Finite-Sample Analysis of Elimination in Active Hypothesis Testing
May 1, 2026Ziyuan Lin, Hoang Ngoc Nguyen, Jie Xu +1Sequential Hypothesis Testing Statistical Hypothesis Testing
Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL 32611, USA
Towards E-Value Based Stopping Rules for Bayesian Deep Ensembles
Apr 20, 2026Emanuel Sommer, Rickmer Schulte, Sarah Deubner +2Markov Chain Monte Carlo Early Stopping
LMU Munich, MCML
On Cost-Aware Designs for Sequential Hypothesis Testing
Dec 22, 2025George Vershinin, Asaf Cohen, Omer GurewitzSequential Hypothesis Testing
The School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Israel
SGM: A Statistical Godel Machine for Risk-Controlled Recursive Self-Modification
Oct 11, 2025Xuening Wu, Shenqin Yin, Yanlan Kang +4Automated Machine Learning Self-Improving Agents
Fudan University · University of California, San Diego · Tongji University

May 13, 2026, stat.MLJ/K move · Enter open · S save

When Should an AI Workflow Release? Always-Valid Inference for Black-Box Generate-Verify Systems

Young Hyun Cho, Will Wei Sun

Department of Statistics, Purdue University. · Mitch Daniels School of Business, Purdue University.

Anytime-Valid Inference Sequential Hypothesis Testing LLM Agent Reliability

Open paper AskPDF arXiv

LLM-enabled AI workflows increasingly produce outputs through iterative generate-evaluate-revise loops. Each iteration can improve the candidate, but it also creates a release decision: when to stop and output the current result? This raises a statistical challenge because deployment-time evaluator scores are adaptively generated and repeatedly monitored, yet the likelihood models or exchangeability assumptions typically used for calibration are unavailable. We propose an always-valid release wrapper for existing generator-evaluator pipelines. The wrapper builds a hard-negative reference pool of high-scoring failures, calibrates deployment-time evaluator scores against this pool, and accumulates the resulting evidence with an e-process. This separates two roles: the reference pool turns black-box scores into conservative evidence, while the e-process provides validity under optional stopping. In theory, we show that a conservative reference pool yields finite-sample control of the probability of releasing on infeasible tasks, that is, tasks for which the given workflow is not capable of producing a reliable solution. We also characterize conditions under which the same conservative rule still achieves nontrivial release on feasible tasks. In an MBPP+ coding-agent case study, the wrapper reduces premature incorrect release relative to baseline stopping rules while still releasing on tasks for which the workflow repeatedly accumulates moderate supporting evidence.

Momentum

Related topics

Latest papers 39