E-Values: Betting, Calibration, and Merging
Consider a nonnegative test payoff whose expected value is at most one under the null. An unusually large payoff can then occur only rarely under that null, by Markov’s inequality. Formally, an \(e\)-variable is a statistic \(E\) satisfying \(\mathbb{E}_PE\le1\) for every null distribution \(P\). The analyses of (Shafer 2021; Vovk and Wang 2021) develop this expectation constraint into rules for calibration, combination, and sequential testing.
1. The basic guarantee
Proposition 1. For any \(e\)-variable and \(0<\alpha<1\),
Consequently \(\min(1,1/E)\) is a valid \(p\)-variable, with value one when \(E=0\).
This is Markov’s inequality. A likelihood ratio \(dQ/dP\) is an example for a simple null \(P\), provided \(Q\) is absolutely continuous with respect to \(P\). The expectation bound is the essential property; an arbitrary large score need not have it.
To see the proof without any tail approximation, use the pointwise inequality \(E\ge\alpha^{-1}\mathbf1\{E\ge\alpha^{-1}\}\). Taking null expectations gives the result. The bound can be sharp: let \(E=1/\alpha\) with probability \(\alpha\) and zero otherwise. Its expectation is one and its rejection probability is exactly \(\alpha\). Validity therefore comes from the payoff constraint, not from a normal approximation or a large sample.
For a likelihood ratio \(L=dQ/dP\), the same constraint follows from \(\mathbb{E}_PL=\int dQ=1\). A large observed ratio is evidence because an outcome that earns a large payoff under this rule must be sufficiently rare under \(P\).
2. Calibrating a \(p\)-value
For a nonnegative, nonincreasing measurable function \(f\), the condition
is necessary and sufficient for \(f(Q)\) to be an \(e\)-variable whenever \(Q\) is a valid \(p\)-variable. Necessity follows by taking \(Q\) uniform. For sufficiency, a valid \(p\)-variable is stochastically no smaller than a uniform variable; applying a decreasing function reverses the order and bounds its expectation by that integral.
For fixed \(0<\kappa<1\), \(f_\kappa(p)=\kappa p^{\kappa-1}\) integrates to one. Choosing \(\kappa\) after seeing \(p\) is different. Maximizing gives
Its integral diverges near zero, so this envelope is not a valid calibrator. A fixed mixture is valid: Tonelli’s theorem preserves the expectation bound. Uniformly mixing \(\kappa\) gives
The value at \(p=1\) is the continuous limit \(1/2\).
3. Work out the calibration calculations
Put \(a=-\log p>0\). The log of the power calibrator is \(\log\kappa+(1-\kappa)a\), whose derivative in \(\kappa\) is \(1/\kappa-a\). When \(a\ge1\), its maximum is attained at \(\kappa=1/a\); when \(a<1\), the supremum occurs as \(\kappa\) approaches one. Substitution gives the displayed envelope. Near zero its integral contains
Maximizing after observing the p-value selects the most favorable payoff separately for each outcome. The divergent integral shows that the selected payoff no longer has the required null expectation bound.
A mixture fixes its weights before seeing the p-value. For the uniform mixture, integration by parts gives
Substitute \(-\log p\) for \(a\) to obtain the earlier formula. Tonelli’s theorem averages the component expectation bounds and proves validity. A pointwise maximum has no corresponding expectation bound.
The stochastic-order argument for a general calibrator can also be made pointwise. If \(U\) is uniform and \(Q\) is super-uniform, its generalized quantile satisfies \(F_Q^{-1}(U)\ge U\). Since \(f\) is nonincreasing,
The quantile coupling covers discrete p-values as well as continuous ones, so atoms do not invalidate this sufficiency argument.
4. Combining evidence
Linearity of expectation also permits fixed convex combinations of \(E_1,\ldots,E_K\) valid for the same null, without an independence assumption:
The arithmetic mean has an additional optimality property among symmetric merging rules. If a valid symmetric rule \(F\) gives \(F(e_1,\ldots,e_K)>1\), then its output cannot exceed \(K^{-1}\sum_ke_k\). The threshold matters: constant one is also a valid merging rule and can exceed the mean when all inputs are small.
Here is the argument. Fix nonnegative finite inputs \(e_1,\ldots,e_K\), put \(m=K^{-1}\sum_ke_k\) and \(c=\max(1,m)\). Uniformly permute the inputs and, independently, retain the entire permuted vector with probability \(1/c\), replacing it by zero otherwise. Every coordinate of the resulting vector has expectation \(m/c\le1\), so these are legitimate, generally dependent e-variables.
If a symmetric rule gave \(F(e)>c\), symmetry would make its output the same on every retained permutation. Nonnegativity then implies
contradicting validity. Thus \(F(e)\le\max(1,m)\). Whenever \(F(e)>1\), this forces \(F(e)\le m\). Randomly relabeling the payoffs is what turns the symmetry of the rule into a constraint on every input vector.
Products require more. Independence is sufficient, and sequential conditional validity \(\mathbb{E}[E_t\mid\mathcal F_{t-1}]\le1\) makes \(\prod_{s\le t}E_s\) a nonnegative supermartingale. The tower property proves this one factor at a time. Ville’s inequality then controls the probability that this process ever crosses \(1/\alpha\); the martingale maximal-inequality argument explains why stopping at the first crossing preserves the bound.
Multiplication uses conditional validity or independence, whereas averaging uses only marginal expectation bounds. A collection of fixed-sample \(e\)-values therefore does not automatically justify repeated inspection and stopping.
5. A sequential betting example
Suppose \(X_t\) is binary and the null requires \(\Pr(X_t=1\mid\mathcal F_{t-1})\le p_0\). Choose a predictable \(q_t\in[p_0,1)\) and define the next payoff factor by
with \(0<p_0<1\). If the conditional success probability is \(p_t\), then
Multiplying by the already observed \(K_{t-1}\) proves the supermartingale property. To derive the crossing bound, stop at \(\tau=\inf\{t:K_t\ge1/\alpha\}\). For each finite \(N\), bounded stopping and nonnegativity give
The events increase to the event of ever crossing the threshold as \(N\) tends to infinity. Continuity of probability then gives the bound for any number of inspections.
The power interpretation of a payoff also follows from a short calculation. Suppose, for this illustration, that outcomes are i.i.d. Bernoulli\((p)\) with \(p>p_0\), and use a fixed betting probability \(q\in(p_0,1)\). The expected log increment is
where \(D(p\Vert q)=p\log(p/q)+(1-p)\log((1-p)/(1-q))\). The equality follows by adding and subtracting \(p\log p+(1-p)\log(1-p)\). The derivative with respect to \(q\) is \((p-q)/(q(1-q))\), so the expected log growth is maximized at \(q=p\).
By the law of large numbers, \(t^{-1}\log K_t\) converges to this growth rate. A well-chosen alternative therefore accumulates evidence exponentially under the alternative while retaining the null expectation bound. A badly chosen \(q\) can have negative log growth even though every factor is valid. Validity and power impose different requirements.
The alternative specifies which outcomes receive larger payoffs. A predictable choice of \(q\) can learn from the past because it is fixed before the next outcome, preserving the conditional expectation argument. Selecting it after observing that outcome does not. Fixed mixtures allocate the initial stake across alternatives while retaining the bound.
Choosing \(q_t\) from the past is permitted; choosing it after seeing \(X_t\) would invalidate the conditional expectation calculation. For example, assigning probability one to whichever outcome just occurred can create a payoff larger than one every round without providing valid evidence.