The Type-II Error of Test Supermartingales: e-Power versus the Chernoff-Stein Exponent

Patrick Forré

math.STAug 19, 2026

The Type-II Error of Test Supermartingales: e-Power versus the Chernoff-Stein Exponent

Authors: Patrick Forré

Organizations: AI4Science Lab, Korteweg-de Vries Institute for Mathematics, University of Amsterdam

Abstract

In safe hypothesis testing with test supermartingales, Ville's inequality provides anytime-valid type-I error guarantees for every significance level $α\in(0,1]$ , if one rejects the null hypothesis whenever the wealth process first exceeds $1/α$ . Due to an inherent asymmetry, the type-II error behaves differently. We prove two things about the latter, for a simple null and alternative. First, the mean growth rate $\mathbb{E}_{P_1}[\log E]$ , the e-power, that Kelly betting and growth-rate-optimal e-variables maximise, bounds nothing on its own. For every level $c>0$ , every $α$ and horizon $t$ we construct e-variables of conditional e-power exactly $c$ whose probability of not rejecting by $t$ is arbitrarily close to one. It forces eventual rejection, but no finite-horizon guarantee follows. Second, the quantity that does control the type-II error is the Chernoff-Stein exponent of an e-variable, $Λ(E)=\sup_{s\ge0}\{-\log \mathbb{E}_{P_1}[E^{-s}]\}$ , whose range is exactly determined: $\sup_E Λ(E)=\mathrm{KL}(P_0\|P_1)$ , the classical Chernoff-Stein exponent, and so the ceiling of its own per-e-variable form. One conditional application of Hoelder's inequality per step gives it, for every test supermartingale on an arbitrary filtered space, with no independence or product structure; the i.i.d. case adds that it is matched, and attained by nothing. The e-power has its own ceiling, $\mathrm{KL}(P_1\|P_0)$ , and that one is attained, $P_0$ -a.s. uniquely, by the likelihood ratio $R$ . The two optima are the same divergence in opposite arguments, at opposite ends of the flattened family $R^β/\mathbb{E}_{P_0}[R^β]$ : the ceiling as $β\downarrow0$ , $R$ at $β=1$ . Which $β$ is best is settled by the horizon, exactly: $R$ is optimal at $t=\log(1/α)/\mathrm{KL}(P_1\|P_0)$ alone, beaten by sharpening $(β>1)$ below it and by flattening above.

Figures & tables

Panel of Figure 1 — Figure 1: The two scalars along the flattened family, plotted on $\beta\in(0,1]$ , for $P_{0}=\mathrm{Bern}(\frac{1}{2})$ , $P_{1}=\mathrm{Bern}(0.6)$ . Solid: the tilt- $\beta$ ceiling $\lambda(\beta)=\Lambda_{(1-\beta)/\beta}(R_{\beta})$ , the largest exponent available at tilt $\beta$ ( Theorem 4.10 , Section 4.4 ), rising to $\mathrm{KL}(P_{0}\|P_{1})$ as $\beta\downarrow 0$ but never reaching it. Dotted: the e -power $c_{\beta}$ of the same design, rising in the opposite direction to its own ceiling $\mathrm{KL}(P_{1}\|P_{0})=0.02014$ at $\beta=1$ . The full exponent $\Lambda(R_{\beta})$ , optimised over $s$ , is strictly larger than $\lambda(\beta)$ ( Section 4.4 ); at $\beta=1$ the solid curve is $0$ while $\Lambda(R)=0.00508$ , the lower dashed line, which by ( 24 ) is the largest rectangle $\beta\times\lambda(\beta)$ under the solid curve — a quarter of the ceiling here.

Panel of Figure 2 — Figure 2: What the horizon-optimal tilt buys, in the Gaussian location model of Section 5.5 with $\Delta=0.8$ and $\alpha=0.05$ . Both curves are the exact type-II error, computed from Section 5.2 (a) and the law $G_{t}\sim\mathcal{N}(tD,2tD)$ . Solid: the growth-optimal design $R$ , whose exponent is a quarter of $tD$ . Dotted: the horizon-optimal member $R_{\beta^{\ast}(t)}$ of the flattened family, with $\beta^{\ast}(t)=\sqrt{\ell/(tD)}$ . The two touch only at the crossover $t=\ell/\mathrm{KL}(P_{1}\|P_{0})=9.4$ ( Theorem 5.6 (a), dashed), the dotted curve lying below by sharpening before it and by flattening after: at $t=50$ the errors are $1.1\cdot 10^{-2}$ and $6.7\cdot 10^{-4}$ , at $t=200$ they are $3.5\cdot 10^{-8}$ and $3.8\cdot 10^{-19}$ . The gap widens without bound, since the two exponents are $\frac{1}{4}tD$ and $tD(1-\beta^{\ast}(t))^{2}$ .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Main text	Replaced by	Extra hypothesis
Section 2.4 , Theorem 4.29 (a)	Section A.1	none
Section 4.1 , Section 4.4	Section A.1	none
equality case of Theorem 4.10	Section A.1	none
Section 4.4 (identity)	Section A.1	$\mathbb{P}_{0}(R_{n}>0\mid\mathcal{F}_{n-1})=1$
Section 2.4 , Section 4.4 , Theorem 4.25 , Theorem 4.29 (b)	Theorem A.11	Section A.2
Section 4.3	Section A.1	none

Appendix

Table 1: What each i.i.d.-model result becomes, and what it costs.

Statistical Hypothesis Testing

Statistical hypothesis testing develops methods for assessing evidence against a stated null while controlling false-positive and false-negative rates.

See all 101 papers

Aug 19, 2026

Type-II Error Bounds for Test Supermartingales from Lower-Tail Hypotheses

In safe hypothesis testing with test supermartingals, Ville's inequality provides anytime-valid type-I error guarantees for every significance level

α\in(0,1]

, if one rejects the null hypothesis whenever the wealth process first exceeds

\frac{1}α

. Due to an inherent asymmetry, the type-II error does not have such guarantees: a heavy concentration of the probability on the lower tail of the log-increments can lead to one catastrophic bet that undoes any amount of accumulated evidence. This paper studies how different hypotheses on those lower-tail probabilities lead to different bounds on the type-II error of the sequential test. They all reduce to one master inequality, which bounds the type-II error at level

α

, at a fixed horizon and sequentially, in terms of a one-sided Legendre transform of the (inverse-)moment generating function of the e-variables, evaluated at one number: the amount by which the lower bound of the accumulated e-powers exceeds

\log\frac{1}α

. And, the step is lossless, in the sense, that it extracts exactly a constrained information projection. Every bound presented here is a corollary, obtained by a certain majorant of the above function. The hypotheses are: a finite negative moment; an exponentially small crash probability with a moment on the winning side; a wealth floor with a conditional variance, and its Bernstein variant, which interpolates between a Gaussian regime set by the variance and an exponential one set by the scale; a sub-Gaussian or bounded-tilt lower tail; bounded log-increments; and i.i.d. increments, where the majorant is the truth. We also provide an empirical-Bernstein variant. Each hypothesis may either be read as a condition on the e-variables one has, or as the price of betting with an approximation to the likelihood ratio rather than the ratio itself, which satisfies the weakest condition for free.

Patrick Forré

AI4Science Lab Korteweg-de Vries Institute for Mathematics University of Amsterdam

Sequential Hypothesis Testing E-Values

Ask PDF

Mar 18, 2026

Multi-Armed Sequential Hypothesis Testing by Betting

We consider a variant of sequential testing by betting where, at each time step, the statistician is presented with multiple data sources (arms) and obtains data by choosing one of the arms. We consider the composite global null hypothesis

\mathscr{P}

that all arms are null in a certain sense (e.g. all dosages of a treatment are ineffective) and we are interested in rejecting

\mathscr{P}

in favor of a composite alternative

\mathscr{Q}

where at least one arm is non-null (e.g. there exists an effective treatment dosage). We posit an optimality desideratum that we describe informally as follows: even if several arms are non-null, we seek

e

-processes and sequential tests whose performance are as strong as the ones that have oracle knowledge about which arm generates the most evidence against

\mathscr{P}

. Formally, we generalize notions of log-optimality and expected rejection time optimality to more than one arm, obtaining matching lower and upper bounds for both. A key technical device in this optimality analysis is a modified upper-confidence-bound-like algorithm for unobservable but sufficiently "estimable" rewards. In the design of this algorithm, we derive nonasymptotic concentration inequalities for optimal wealth growth rates in the sense of Kelly [1956]. These may be of independent interest.

Ricardo J. Sandoval, Ian Waudby-Smith, Michael I. Jordan

University of California, Berkeley · École Normale Supérieure & Inria, Paris

Sequential Hypothesis Testing Statistical Hypothesis Testing Multi-Armed Bandits

Ask PDF

Oct 6, 2026

Anytime-valid simulation-based hypothesis testing

For a given data distribution

(X_t)_{t \in \mathbb{N}} \sim Q

i.i.d., we investigate the hypothesis testing problem:

H_0: Q = P_0

vs.

H_1: Q = P_1

, for two different model probability distributions

P_0

and

P_1

. In contrast to the standard setting, where analytic densities

p_0

and

p_1

are given, here, we consider the density-free setting, where we only have access to i.i.d. simulations

(Z^0_t)_{t \in \mathbb{N}} \sim P_0

and

(Z^1_t)_{t \in \mathbb{N}} \sim P_1

. For this simulation-based hypothesis testing setting, we construct an e-test martingale, resulting in a sequential test with anytime-valid type-I error guarantees, approximate growth optimality, geometrically decaying type-II error bounds, and asymptotic power one. Most ingredients used in our constructions are variants of well known concepts. The value of this paper lies in the compact presentation of an effective, anytime-valid solution for the density-free simulation-based sequential hypothesis testing case.

Patrick Forré, Lydia Brenner

University of Amsterdam · Nikhef

Simulation-Based Inference Sequential Hypothesis Testing

Ask PDF

Abstract

Statistical Hypothesis Testing

Explore similar work

Type-II Error Bounds for Test Supermartingales from Lower-Tail Hypotheses

Multi-Armed Sequential Hypothesis Testing by Betting

Anytime-valid simulation-based hypothesis testing