The Hot Hand: Statistics for Streaks
After several successes, is the next attempt more likely to succeed? A natural statistic divides the number of successful continuations by the number of eligible streaks. Both counts depend on the same sequence, and their ratio need not be unbiased even for independent coin tosses. These notes use (Ritzwoller and Romano 2022) for inference and (Miller and Sanjurjo 2018) for the finite-sample selection problem.
1. Define the comparison
For a binary sequence \(X_1,\ldots,X_n\) and fixed streak length \(k\), let
Over \(t=k+1,\ldots,n\), define \(\widehat P_k=\sum_tA_tX_t/\sum_tA_t\), \(\widehat Q_k=\sum_tB_tX_t/\sum_tB_t\), and \(\widehat D_k=\widehat P_k-\widehat Q_k\). A second statistic is \(\widehat P_k-\overline X\). A convention is needed when an eligible-count denominator is zero, including in permutations.
Theorem 1. For i.i.d. Bernoulli observations with fixed \(0<p<1\) and fixed \(k\),
The corresponding permutation distributions converge to these null limits. Permuting the sequence also gives a finite-sample valid test of the i.i.d. Bernoulli null, with randomization at ties if exact size is desired.
The normal limits describe first-order fluctuations, not finite-sample expectations. A random eligibility count can bias the ratio. The companion derivation calculates the leading fluctuation and then the smaller-order term responsible for that bias.
2. Why permutation removes the unknown success rate
Conditional on \(\sum_tX_t=a\), every binary sequence with \(a\) successes has the same probability under the null. Its conditional distribution is therefore uniform over permutations, independently of \(p\). Ranking the observed statistic within that distribution controls conditional size, hence unconditional size.
There is also a converse for tests with rejection probability exactly \(\alpha\) for every \(p\): completeness of the binomial family forces their average rejection probability conditional on the count to equal \(\alpha\). The unconditional requirement leaves no room to favor one success count over another.
3. A local-power calculation
Consider a stationary two-state Markov chain with probability \(1/2+\epsilon\) of repeating its previous outcome. Its marginal success probability is \(1/2\), while the population versions of \(\widehat P_1-\overline X\) and \(\widehat D_1\) are \(\epsilon\) and \(2\epsilon\).
For \(\epsilon=h/\sqrt n\), the two statistics have limits \(\mathcal{N}(h,1/4)\) and \(\mathcal{N}(2h,1)\) after multiplication by \(\sqrt n\). Thus both one-sided permutation tests have local power
When independent individuals have different success rates, permutations must stay within individuals. Under the paper’s pooled local-alternative assumptions, the fraction exhibiting dependence contributes to the noncentrality. Rejection concerns independence of the observations; a causal effect of past success requires further assumptions about state dependence.
4. A small example of the selection problem
Consider three fair coin tosses and \(k=1\). Conditional on having at least one eligible transition, the possible sequences and success-after-success ratios are
| Sequence | Number of eligible transitions | \(\widehat P_1\) |
|---|---|---|
| \(010\) | \(1\) | \(0\) |
| \(011\) | \(1\) | \(1\) |
| \(100\) | \(1\) | \(0\) |
| \(101\) | \(1\) | \(0\) |
| \(110\) | \(2\) | \(1/2\) |
| \(111\) | \(2\) | \(1\) |
Conditioning gives each of these six sequences the same probability. Averaging their ratios yields
The next toss is fair at every fixed time. The ratio nevertheless gives the same total weight to sequences with one eligible transition and those with two. Pooling transitions across many sequences would weight them differently and recover \(1/2\). The change of weighting explains the bias.
A permutation test reproduces the ratio and the convention for empty counts on every rearranged sequence. Its validity does not require an unbiased statistic: the reference distribution includes the finite-sample bias.
5. The exact conditional test, including ties
Let \(\mathcal S_a\) be the set of distinct binary sequences with \(a\) successes, and let \(S(x)\) denote the statistic with a specified empty-count convention. Under the null,
Choose \(c_a\) so that the conditional upper tail satisfies
Reject above \(c_a\) and reject at equality with probability
The conditional rejection probability is exactly \(\alpha\) whenever the boundary mass is positive. Taking its expectation over the binomial count proves unconditional exactness for every \(p\). Omitting boundary randomization gives a conservative test. This argument also covers counts \(a=0,n\), provided the statistic is defined there; no asymptotic approximation is involved.
Here the binomial completeness argument reduces to a polynomial identity. If \(r_a\) denotes conditional rejection probability and unconditional rejection probability equals \(\alpha\) for every \(0<p<1\), then
Divide by \((1-p)^n\) and put \(u=p/(1-p)\). The resulting polynomial in \(u\) vanishes for every \(u>0\), so all coefficients vanish: \(r_a=\alpha\) for every \(a\). This establishes the stated converse for exact tests, not for merely conservative ones.
6. Deriving the Markov local alternative
For the symmetric chain, the transition matrix, in the order failure then success, is
Its stationary distribution is \((1/2,1/2)\) because both columns sum to one. Therefore
while subtracting the success probability after failure gives \(2\epsilon\). For \(k=1,p=1/2\), the null standard deviations in the theorem are \(1/2\) and \(1\). Dividing the local mean shifts \(h\) and \(2h\) by those standard deviations gives the same noncentrality \(2h\).
The triangular-array CLT and permutation-limit result give the standardized limit \(\mathcal{N}(2h,1)\). Hence the limiting rejection event is \(Z+2h>z_{1-\alpha}\), yielding the stated power. The alternative keeps the marginal success probability fixed and changes how successes cluster in time.
The same chain illustrates the cost of choosing a longer streak. Its Markov property gives \(\mathbb{P}(X_t=1\mid A_t=1)=1/2+\epsilon\) for every fixed \(k\): once the most recent outcome is known, the earlier outcomes add no predictive information. Yet at \(p=1/2\), the null variances are
The standardized local shifts are \(2h/\sqrt{2^k-1}\) and \(2h/\sqrt{2^{k-1}}\). In this Markov model, a longer history adds no signal once the latest outcome is known, but it reduces the number of eligible transitions. The variance calculation therefore explains the loss of local power. Other dependence models may contain information in earlier outcomes and require a new comparison.
Conditioning on each individual’s success count gives a product of uniform permutation laws under the independent-individual null. Within-individual permutations preserve that law, so any prespecified pooled statistic can be calibrated exactly. Permuting observations between individuals would also impose equality of their baseline rates.
From a permutation cdf to a power calculation
Exact conditional validity and a limiting normal reference are different assertions. For the latter, write \(F_n^\pi\) for the conditional permutation cdf of the appropriately standardized statistic. If \(F_n^\pi(x)\to\Phi(x)\) in probability at every real \(x\), its \((1-\alpha)\) quantile \(c_n\) converges to \(z_{1-\alpha}\). Indeed, for any \(\delta>0\),
With probability tending to one the same strict inequalities hold for \(F_n^\pi\) at these two points, trapping \(c_n\) between them. If the statistic under the local alternative converges to \(Z+2h\), sandwiching its rejection event between the two fixed thresholds and then letting \(\delta\downarrow0\) gives \(1-\Phi(z_{1-\alpha}-2h)\). Continuity of the limiting cdf makes the strict/non-strict boundary immaterial asymptotically; finite-sample exactness still uses the tie rule above.
The required conditional permutation limit is an external input, not a consequence of the ordinary CLT alone: see Theorem 3.3 and Corollary 3.1 of the authors’ April 1, 2021 manuscript. The local Markov limit and its power consequence are Theorem 4.1, Remark 4.1, and Corollary 4.1 there.