Sequential Hypothesis Testing

Ask about this topic

Overview

Sequential hypothesis testing studies how to accept or reject claims as data arrive, so experiments can stop once evidence is sufficient instead of using a fixed sample size. Classical sequential probability ratio tests are joined by martingale and e-value methods, including testing by betting, which can preserve error guarantees under adaptive sampling and optional stopping. These tools support applications such as online change detection, fairness and model audits, clinical or A/B experiments, and evaluation of AI systems and agents. A central challenge is balancing reliable anytime-valid error control with fast detection and low data costs, especially when assumptions, data distributions, or evaluator quality may change.

Latest papers

CardsList

63 papers15 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

  1. ProCerDCL: Query-Efficient Black-Box Probabilistic Verification of Deep Learning Compiler Optimization

    Oct 27, 2026Zexin Li, Weibo Zhang, Linyi Li +2Statistical Model CheckingSequential Hypothesis Testing

    STAR Lab, George Mason University · Simon Fraser University · Nanyang Technological University

  2. Bayesian Evidence Accumulation for Graph-Based Learning in Dynamic Data-Driven Applications Systems

    Oct 26, 2026Moises Ponce, Sarah Harkins Dayton, Ioannis Schizas +1Graph ClassificationGraph Neural Networks

    University of Tennessee, Knoxville TN 37996, USA · US Army Combat Capabilities Development Command, Army Research Laboratory, Adelphi MD 20783, USA

  3. Valid Early Detection of Memorization in Diffusion Models

    Oct 25, 2026Yixiao Jing, Chia-Yu Hsu, Milad Hoseinpour +1Memorization in Generative ModelsSequential Hypothesis Testing

    Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA

  4. Lifetime-valid decisions by continually evolving AI analysts

    Oct 22, 2026Xiaojing Sun, Yuhan Zeng, Xiao WangAutomated Agent EvaluationSequential Hypothesis Testing

    Department of Statistics, Purdue University

  5. Calibration-Free Early Stopping for Chain-of-Thought Reasoning via E-processes

    Oct 18, 2026Dong-Jae Lee, Gyojin Han, Junmo KimEarly Stopping for Language Model ReasoningAdaptive Reasoning in Language Models

    KAIST

  6. The Certification Exponent: Instance-Optimal Sequential Certification of Policy Improvement from History Dependent Logs

    Oct 17, 2026Rui WangSequential Hypothesis TestingOff-Policy Evaluation

    Microsoft Vancouver, BC, Canada

  7. Monitoring for Shifts at Unknown Times: Parameter-Free Conditional Conformal Martingales

    Oct 12, 2026Tuo Liu, Drew Prinster, Francesco OrabonaDistribution Shift DetectionChange-Point Detection

    King Abdullah University of Science and Technology, Saudi Arabia · Johns Hopkins University, USA

  8. AnyRank: Reliable Anytime Policy Ranking via Sequential Testing

    Oct 10, 2026Apurva Badithela, David Snyder, Tenny Yin +6Robotic Policy EvaluationAdaptive Rank Allocation

    Princeton University · University of Pennsylvania · Toyota Research Institute

  9. Quickest Change Detection with Diffusion-Integrated Scores

    Oct 8, 2026Arman Adibi, Mohammadreza Maleki, Sanjeev Kulkarni +1Change-Point DetectionSequential Hypothesis Testing

    Augusta University · Toronto Metropolitan University · Princeton University

  10. Anytime-valid simulation-based hypothesis testing

    Oct 6, 2026Patrick Forré, Lydia BrennerSimulation-Based InferenceSequential Hypothesis Testing

    University of Amsterdam · Nikhef

  11. Learning to Outgrow a Theory: Experimental Discovery Beyond the Initial Hypothesis Space

    Oct 6, 2026SiYuan Ma, Albert Gao, Chunzheng Zhu +6Scientific DiscoverySequential Hypothesis Testing

    Nanyang Technological University · Carnegie Mellon University · Hunan University +5

  12. Valid Stopping in Adaptive Generator-Verifier Loops

    Oct 5, 2026Mahmoud Hegazy, Michael I. Jordan, Aymeric DieuleveutFDR ControlE-Values

    CMAP, École polytechnique, IP Paris, France · Inria Paris, France · EECS, UC Berkeley, USA

  13. When Is Enough Enough in Self-Evolving LLM Systems?

    Oct 3, 2026Enoch Yin, Bin Liu, Zhengling QiEarly StoppingLanguage Model Self-Improvement

    Department of Statistics, The George Washington University · Department of Statistics and Data Science, School of Management, Fudan University · Department of Decision Sciences, School of Business, The George Washington University

  14. Sequential Capacity of Quantum Processes with Finite Memory

    Oct 1, 2026Yibin WangSequential Hypothesis Testing

    Graduate School of Mathematics, Nagoya University, Nagoya, 464-8601, Aichi, Japan.

  15. Prequential E-Values for Selected-GP Near-Optimality Certificates

    Sep 30, 2026Ami Tavory, Noa CohenBlack-Box OptimizationAnytime-Valid Inference

    Meta Platforms

  16. Always-On Experimentation

    Sep 30, 2026Ricardo J. Sandoval, David Arbour, Avi Feller +1FDR ControlA/b Testing

    University of California, Berkeley · Adobe Research · Inria, Paris

  17. PEAR: Progressive Evidence-Based AutoResearch for Industrial Search Systems

    Sep 28, 2026Yifan Wang, Shipeng Zhu, Fei Xiong +6Information RetrievalSequential Hypothesis Testing

    Global E-Commerce Agentic Search Team

  18. ICMAPE: In-Context Multiagent Pure Exploration

    Sep 27, 2026Xinyi Hu, Alessio Russo, Aldo PacchianoRL ExplorationDecentralized MARL

    Boston University · Broad Institute of MIT and Harvard

  19. The Price of Peeking: Anytime-Valid Leakage Detection on ML-KEM EM Traces

    Sep 27, 2026Georgios Feretzakis, Alexandros PapaspyridisAnytime-Valid InferenceSequential Hypothesis Testing

  20. When Can Old Evaluations Certify a New Model? Label-Efficient Release Decisions under Evaluator Drift

    Sep 26, 2026Joyanta Jyoti Mondal, Mridul Banik, Md. Shifatul Ahsan Apurba +1Sequential Hypothesis TestingLLM Auditing

    Department of Computer and Information Sciences, University of Delaware, USA · Department of Biomedical Informatics and Data Science, University of Alabama at Birmingham, USA · Luddy School of Informatics, Computing, and Engineering, Indiana University Indianapolis, USA +1

  21. Human-AI-Powered Hypothesis Testing: Cost-Aware Selective AI Scoring and Sequential Human Escalation

    Sep 24, 2026Dae Woong Ham, Xuejun Zhao, Stefanus Jasin +1Cost-Aware InferenceHuman-in-the-Loop Evaluation

  22. How Many Posterior Samples? Calibrated Stopping for Adaptive Sensing

    Sep 18, 2026Vincent Corlay, Andriy EnttselActive PerceptionOptimal Stopping

    Mitsubishi Electric R&D Centre Europe

  23. Predictive Likelihood Ratios for Language Model Watermark Detection

    Sep 14, 2026Li MaBayesian InferenceLLM Watermarking

    Department of Statistics and Data Science Institute, University of Chicago

  24. Should I Use This Synthetic Dataset for Training? How to Test with Minimal Real Data

    Aug 28, 2026Zhenyu Tao, Wei Xu, Xiaohu You +2Synthetic Data EvaluationSequential Hypothesis Testing

    National Mobile Communications Research Lab, Southeast University, Nanjing 210096, China · Pervasive Communication Research Center, Purple Mountain Laboratories, Nanjing 211111, China · Department of Electronic Systems, Aalborg University, 9220 Aalborg, Denmark +2

  25. Change Detection in Probability Flow ODE: Online Testing in Diffusion Latent Spaces

    Aug 24, 2026Artem Kraevskiy, Artem ProkhorovDistribution Shift DetectionChange-Point Detection

    National Research University Higher School of Economics Moscow, Russia · The University of Sydney Business School & CIREQ & CEBDA Sydney, Australia

  26. Evidence Before Expansion: Reuse, Spawn, or Defer in Lifelong Expert Pools

    Aug 20, 2026Kentaro OdaOnline Continual LearningSequential Hypothesis Testing

    Center for Management of Information Technologies, Kagoshima University

  27. The Type-II Error of Test Supermartingales: e-Power versus the Chernoff-Stein Exponent

    Aug 19, 2026Patrick ForréStatistical Hypothesis TestingSequential Hypothesis Testing

    AI4Science Lab, Korteweg-de Vries Institute for Mathematics, University of Amsterdam

  28. Type-II Error Bounds for Test Supermartingales from Lower-Tail Hypotheses

    Aug 19, 2026Patrick ForréSequential Hypothesis TestingE-Values

    AI4Science Lab Korteweg-de Vries Institute for Mathematics University of Amsterdam

  29. Deciding When to Switch: E-Processes for Adaptive Minimax Training for Generative Adversarial Nets

    Aug 10, 2026Hyunjoo Kim, Sicheng Wu, Agastya Venkatraman +2Min-Max OptimizationSequential Hypothesis Testing

    Department of Statistics, Ewha Womans University · Department of Mathematics, Purdue University · Department of Computer Science, Purdue University

  30. From Approachability Residuals to Anytime-Valid Evidence: The Online Convex Geometry of Testing by Betting

    Aug 10, 2026Jinze ZhaoBlackwell ApproachabilitySequential Hypothesis Testing

    University of California, San Diego