Type-II Error Bounds for Test Supermartingales from Lower-Tail Hypotheses

Patrick Forré

math.STAug 19, 2026

Type-II Error Bounds for Test Supermartingales from Lower-Tail Hypotheses

Authors: Patrick Forré

Organizations: AI4Science Lab Korteweg-de Vries Institute for Mathematics University of Amsterdam

Abstract

In safe hypothesis testing with test supermartingals, Ville's inequality provides anytime-valid type-I error guarantees for every significance level $α\in(0,1]$ , if one rejects the null hypothesis whenever the wealth process first exceeds $\frac{1}α$ . Due to an inherent asymmetry, the type-II error does not have such guarantees: a heavy concentration of the probability on the lower tail of the log-increments can lead to one catastrophic bet that undoes any amount of accumulated evidence. This paper studies how different hypotheses on those lower-tail probabilities lead to different bounds on the type-II error of the sequential test. They all reduce to one master inequality, which bounds the type-II error at level $α$ , at a fixed horizon and sequentially, in terms of a one-sided Legendre transform of the (inverse-)moment generating function of the e-variables, evaluated at one number: the amount by which the lower bound of the accumulated e-powers exceeds $\log\frac{1}α$ . And, the step is lossless, in the sense, that it extracts exactly a constrained information projection. Every bound presented here is a corollary, obtained by a certain majorant of the above function. The hypotheses are: a finite negative moment; an exponentially small crash probability with a moment on the winning side; a wealth floor with a conditional variance, and its Bernstein variant, which interpolates between a Gaussian regime set by the variance and an exponential one set by the scale; a sub-Gaussian or bounded-tilt lower tail; bounded log-increments; and i.i.d. increments, where the majorant is the truth. We also provide an empirical-Bernstein variant. Each hypothesis may either be read as a condition on the e-variables one has, or as the price of betting with an approximation to the likelihood ratio rather than the ratio itself, which satisfies the weakest condition for free.

Figures & tables

	Hypothesis	Majorant $\psi(s)$	Rate $\psi^{\star}(x)$	Tilt $s^{\ast}$
1	one negative moment	$\frac{s}{s_{0}}\kappa$ on $[0,s_{0}]$	$(s_{0}x-\kappa)_{+}$	$s_{0}$
2	crash tail, one-sided $p$ -th moment	$c_{p}M_{p}s^{p}+\frac{Ks^{2}}{a(a-s)}$	—	( 30 )
$2^{\prime}$	crash tail, $L\log L$ moment	$\frac{m_{1}T}{2}s^{2}+\frac{M}{\log T}s+\frac{Ks^{2}}{a(a-s)}$	—	$s_{\vartheta}$
$2^{\prime\prime}$	crash tail, two-sided second moment	$\frac{w_{0}s^{2}}{2}+\frac{K(s/a)^{3}}{1-s/a}$	—	( 34 )
3	floor $b$ , variance $w$	$\frac{w}{b^{2}}(e^{sb}-sb-1)$	$\frac{w}{b^{2}}h\!\left(\frac{bx}{w}\right)$	$\frac{1}{b}\log\!\left(1+\frac{bx}{w}\right)$
$3^{\prime}$	Bernstein moments, scale $\tau$	$\frac{ws^{2}}{2(1-\tau s)}$	$\frac{w}{\tau^{2}}h_{1}\!\left(\frac{\tau x}{w}\right)$	$\frac{1}{\tau}\left(1-\left(1+\frac{2\tau x}{w}\right)^{-\frac{1}{2}}\right)$

Table 1: The ladder, per observation. Here

h(z)=(1+z)\log(1+z)-z

h_{1}(z)=1+z-\sqrt{1+2z}

\mathrm{kl}(x\|y)=x\log\frac{x}{y}+(1-x)\log\frac{1-x}{1-y}

\kappa=\log\rho+s_{0}c

\bar{x}=y-\frac{x}{r}

and

q=\frac{p}{p-1}

. Rungs 2,

2^{\prime}

and

2^{\prime\prime}

share the crash-tail hypothesis ( 25 ) and differ in what they assume in addition — a one-sided

p

-th moment, an

L\log L

moment, and a two-sided second moment respectively; their transforms have no closed form, but ( 29 ) gives an explicit tilt for rung 2, and ( 35 ) and ( 36 ) give explicit exponents for rung

2^{\prime\prime}

, valid for

x\leq\frac{aw_{0}}{2}

and for

ax>K

respectively. In the moment-limited regime rung 2 gives

\frac{x^{q}}{2(4c_{p}M_{p})^{q-1}}

. Rung 5b requires

0\leq x\leq yr

. The last row is not a rung: it is the assertion that no hypothesis at all buys nothing at all, which is the no-free-lunch theorem [ For26a , Thm. 3.7] of the companion paper, and it is what makes the rest of the table necessary.

Sequential Hypothesis Testing

Sequential hypothesis testing makes decisions as evidence arrives, controlling errors while adapting how much data to collect and when to stop.

See all 63 papers

Aug 19, 2026

The Type-II Error of Test Supermartingales: e-Power versus the Chernoff-Stein Exponent

In safe hypothesis testing with test supermartingales, Ville's inequality provides anytime-valid type-I error guarantees for every significance level

α\in(0,1]

, if one rejects the null hypothesis whenever the wealth process first exceeds

1/α

. Due to an inherent asymmetry, the type-II error behaves differently. We prove two things about the latter, for a simple null and alternative. First, the mean growth rate

\mathbb{E}_{P_1}[\log E]

, the e-power, that Kelly betting and growth-rate-optimal e-variables maximise, bounds nothing on its own. For every level

c>0

, every

α

and horizon

t

we construct e-variables of conditional e-power exactly

c

whose probability of not rejecting by

t

is arbitrarily close to one. It forces eventual rejection, but no finite-horizon guarantee follows. Second, the quantity that does control the type-II error is the Chernoff-Stein exponent of an e-variable,

Λ(E)=\sup_{s\ge0}\{-\log \mathbb{E}_{P_1}[E^{-s}]\}

, whose range is exactly determined:

\sup_E Λ(E)=\mathrm{KL}(P_0\|P_1)

, the classical Chernoff-Stein exponent, and so the ceiling of its own per-e-variable form. One conditional application of Hoelder's inequality per step gives it, for every test supermartingale on an arbitrary filtered space, with no independence or product structure; the i.i.d. case adds that it is matched, and attained by nothing. The e-power has its own ceiling,

\mathrm{KL}(P_1\|P_0)

, and that one is attained,

P_0

-a.s. uniquely, by the likelihood ratio

R

. The two optima are the same divergence in opposite arguments, at opposite ends of the flattened family

R^β/\mathbb{E}_{P_0}[R^β]

: the ceiling as

β\downarrow0

R

β=1

. Which

β

is best is settled by the horizon, exactly:

R

is optimal at

t=\log(1/α)/\mathrm{KL}(P_1\|P_0)

alone, beaten by sharpening

(β>1)

below it and by flattening above.

Patrick Forré

AI4Science Lab, Korteweg-de Vries Institute for Mathematics, University of Amsterdam

Statistical Hypothesis Testing Sequential Hypothesis Testing E-Values

Ask PDF

Oct 6, 2026

Anytime-valid simulation-based hypothesis testing

For a given data distribution

(X_t)_{t \in \mathbb{N}} \sim Q

i.i.d., we investigate the hypothesis testing problem:

H_0: Q = P_0

vs.

H_1: Q = P_1

, for two different model probability distributions

P_0

and

P_1

. In contrast to the standard setting, where analytic densities

p_0

and

p_1

are given, here, we consider the density-free setting, where we only have access to i.i.d. simulations

(Z^0_t)_{t \in \mathbb{N}} \sim P_0

and

(Z^1_t)_{t \in \mathbb{N}} \sim P_1

. For this simulation-based hypothesis testing setting, we construct an e-test martingale, resulting in a sequential test with anytime-valid type-I error guarantees, approximate growth optimality, geometrically decaying type-II error bounds, and asymptotic power one. Most ingredients used in our constructions are variants of well known concepts. The value of this paper lies in the compact presentation of an effective, anytime-valid solution for the density-free simulation-based sequential hypothesis testing case.

Patrick Forré, Lydia Brenner

University of Amsterdam · Nikhef

Simulation-Based Inference Sequential Hypothesis Testing

Ask PDF

Aug 20, 2026

Information on trajectories: martingales and random times

Accounting for information flow on the path space of trajectories of a nonnegative martingale yields exact variational identities for it, even at arbitrary random times. This recovers the widely used classical concentration inequalities, from Ville to PAC-Bayes, and measures what each one discards. The tail a bound controls is itself a relative entropy, resolved by the chain rule into per-step conditional divergences. The discarded slack has an exact form in each of three geometries: a Gibbs tilt for the Azuma-Hoeffding and PAC-Bayes bounds, the crossing itself for Ville's and for pooled tests, and a dominating certificate for the

L^p

maximal bound. That certificate's optional-stopping deficit resolves per step into Bregman divergences of the running maximum. On a path-time space, the same identity gains one factor that prices anticipation: an arbitrary random time carries an e-process ``peeking penalty.'' The partition function can be read as a coalescent--a prefix-sharing probability of independent copies--and geometric mixtures of test martingales gain a pooling benefit for multi-model safe testing.

Akshay Balsubramani

Anytime-Valid Inference

Ask PDF

Abstract

Sequential Hypothesis Testing

Explore similar work

The Type-II Error of Test Supermartingales: e-Power versus the Chernoff-Stein Exponent

Anytime-valid simulation-based hypothesis testing

Information on trajectories: martingales and random times