stat.MLNov 11, 2026

Bet Imaginatively, not Historically in Independent-Data Sequential Testing

Authors: Nathaniel Xu, Feng Liu, Danica J. Sutherland

Organizations: UBC & Amii

Abstract

Sequential testing avoids some of the many controversies and drawbacks related to traditional batch testing with p-values. In particular, the testing by betting paradigm has developed rapidly in recent years; it frames the level of evidence against the null as the wealth of a player in a betting game. Naturally, how much a player should bet is a key consideration, and should reflect their confidence in what they know about the next observed data point. Much current work uses standard betting strategies developed for adversarial problems. In the stochastic setting, where the observed data is independent and stationary, these approaches may be suboptimal. We propose a simple betting strategy, follow the leave-one-out e-power (FLOE), that exploits the i.i.d. nature of a data stream to estimate the e-power of our current test on the underlying distribution. We average not over the historical evidence collected so far, as generic online learning-based methods do, but instead over the equally-likely possible data sequences we might have seen based on shuffling the data. With an additional principled regularization technique, this simple drop-in method yields substantial improvements in testing performance on a variety of kernel two-sample and independence testing problems.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Sequential Kernel-based Conditional Independence Testing via Adaptive Betting

    Jun 17, 2026Zheng He, Danica J. SutherlandKernel MethodsConditional Independence Testing

  2. Multi-Armed Sequential Hypothesis Testing by Betting

    Mar 18, 2026Ricardo J. Sandoval, Ian Waudby-Smith, Michael I. JordanSequential Hypothesis TestingStatistical Hypothesis Testing

  3. Semi-Supervised Hypothesis Testing by Betting on Predictions

    May 27, 2026Yaniv Tenzer, Elad Tolochinsky, Yaniv RomanoE-ValuesSemi-Supervised Learning