The Hot Hand: Limit Theory and Proofs
The streak tests are ratios of dependent counts. To obtain their limiting laws, first replace each random denominator by its probability limit and identify the error from doing so. Following (Ritzwoller and Romano 2022), this note uses the first-order terms for variance and the second-order terms for bias.
1. The first-order expansion
Use \(A_t=\prod_{j=1}^kX_{t-j}\) and \(B_t=\prod_{j=1}^k(1-X_{t-j})\). Under i.i.d. Bernoulli sampling with \(0<p<1\), the eligible-count averages converge to \(p^k\) and \((1-p)^k\). Consequently
Boundary observations contribute only \(o_P(1)\) for fixed \(k\).
2. Overlapping streaks do not require guessing covariances
Overlapping histories do not produce covariance between these score terms under the null. The multiplier is determined by the past, whereas \(\mathbb{E}[X_t-p\mid\mathcal F_{t-1}]=0\); conditioning on that past makes each summand a martingale difference.
For the first statistic,
For the second, \(A_tB_t=0\) and hence
Multiplying by \(p(1-p)\) gives the two null variances. A bounded, finite-memory CLT completes the argument.
3. Where finite-sample bias enters
For a numerator average \(U\) and denominator average \(V\), expand around \((u,v)\):
Both terms are typically of order \(1/n\). Replacing \(V\) by its expectation drops precisely the terms responsible for the selection bias. For the success-after-success ratio, the paper obtains the leading correction \(p(1-p^{-k})/n\), which is negative. This expansion assumes fixed \(p,k\) and the specified treatment of empty streak counts; it is not a useful approximation when eligible streaks are extremely rare.
4. Beyond the i.i.d. null
For a stationary dependent sequence, collect the needed run counts into a vector \(\Gamma_t\). If a joint CLT holds with long-run covariance \(\Omega\), a ratio statistic \(h(\overline\Gamma)\) has variance
The gradient supplies the sensitivity to each count average. Contracting it with the long-run covariance gives the ratio’s variance. Finite-state irreducible, aperiodic Markov chains provide one setting in which the required count CLT holds.
Serial dependence alone does not identify a causal effect of success. For example, a persistent unobserved ability state may make both the previous and the next attempt more likely to succeed. A causal interpretation requires assumptions linking observed histories to potential outcomes under those histories.
5. Expanding the ratio explicitly
Put \(m=n-k\), \(\overline A=m^{-1}\sum_tA_t\), and \(\overline{AX}=m^{-1}\sum_tA_tX_t\). The identity
is exact whenever the denominator is positive. The numerator has mean zero and is \(O_p(m^{-1/2})\). Since \(\overline A-p^k=O_p(m^{-1/2})\), expand the reciprocal:
Multiplying shows that replacing the denominator by \(p^k\) changes the ratio by \(O_p(m^{-1})\). After subtracting the sample mean, the first influence term is therefore \((X_t-p)(A_t/p^k-1)\). The same argument applied to the failure-run denominator gives the second expansion. For fixed \(k\), replacing averages over \(m\) observations by averages over \(n\) contributes \(O_p(n^{-1})\) because the variables are bounded.
The omitted product has smaller stochastic order than the leading fluctuation, but its expectation can be nonzero. This is why a correct first-order normal approximation can coexist with finite-sample bias.
6. Checking the martingale variance carefully
Write \(H_t=A_t/p^k-1\) and \(W_t=(X_t-p)H_t\). If \(s<t\), then \(W_s\) and \(H_t\) are measurable with respect to \(\mathcal F_{t-1}\). Thus
At the same time,
Since \(A_t^2=A_t\) and \(\mathbb{E}A_t=p^k\),
For \(J_t=A_t/p^k-B_t/(1-p)^k\), the cross-product is zero: a preceding block cannot consist entirely of successes and entirely of failures. This gives the second variance. Conditioning on the history immediately before the later outcome removes all nonzero-lag covariances despite overlap between the histories.
For fixed \(k\), bounded summands satisfy Lindeberg’s condition, and a finite-memory law of large numbers gives convergence of the conditional variance average. These establish the martingale CLT. If \(k\) grows, boundedness is no longer uniform: \(p^{-k}\) grows and eligible runs may become rare.
7. The second-order calculation and its limitation
For \(h(u,v)=u/v\), the nonzero second derivatives are
The quadratic Taylor term is half the Hessian contracted with the covariance matrix of \((U,V)\). This yields
The coefficient of the bias can also be calculated. Write \(Z_t=A_t(X_t-p)\), \(q=p^k\), and average over the \(m=n-k\) eligible time indices. The exact ratio identity gives
The first term has expectation zero. To evaluate the second, consider \(\mathbb{E}[Z_tA_{t+r}]\). It vanishes for \(r\leq0\) by conditioning on the past. It also vanishes for \(r>k\), when the two blocks are independent. For \(1\leq r\leq k\), the two streak blocks together contain \(k+r\) distinct tosses, including \(X_t\). All must be successes in the product, so
There are \(m-r\) pairs at this lag. Consequently, for \(m>k\),
For fixed \(k,p\), the leading bias is therefore
A success raises today’s numerator and can create eligible transitions later. The resulting positive covariance with the denominator enters the ratio expansion with a minus sign. To turn this calculation into an expectation expansion, the final remainder still needs control.
Turning this formal expansion into an expectation expansion requires control of the remainder on events where \(V\) is small. The fact that \(V\to v>0\) in probability alone is insufficient for arbitrary unbounded ratios. Here fixed \(p,k\), bounded ratios with an explicit empty-count convention, and concentration of finite-memory run counts supply the needed control. When \(np^k\) is small, the event of no success run is not negligible and this asymptotic calculation is a poor guide.
Permutation inference recomputes the finite-sample ratio, including small denominators and empty counts. It therefore accounts for behavior that a leading normal approximation or bias correction may approximate poorly.
8. What changes under serial dependence
For example, take
At the population mean, the gradient is
If \(\sqrt n(\overline\Gamma-\mathbb{E}\Gamma_t)\Rightarrow \mathcal{N}(0,\Omega)\) and \(v,z>0\), the delta method gives \(\nabla h'\Omega\nabla h\). Here
when the covariance series is well defined. Under dependence, \(\mathbb{E}[X_t-p\mid\mathcal F_{t-1}]\) generally is not zero, so the earlier martingale simplification no longer applies. For a finite-state Markov model one can augment the state with the last \(k\) outcomes and compute the run-count moments from its transition matrix. This is the appropriate place for a long-run covariance calculation.
Consider a persistent latent state that raises success probabilities. Observing success gives information about that state and predicts another success, even if changing the previous outcome itself would have no causal effect. An independence test detects this dependence; it does not distinguish that explanation from a causal effect of success.
Completing the expectation remainder
Here is a direct justification of the bias coefficient, with any fixed bounded value assigned when \(\overline A=0\). Put \(E_m=\{|\overline A-q|\le q/2\}\). Split the run indicators into their \(k\) residue classes: indicators in one class depend on disjoint blocks of tosses and are independent. A union bound and the bounded-variable exponential inequality give \(\mathbb P(E_m^c)\le C e^{-cm}\) for fixed \(p,k\), with constants depending on them. The same splitting, or expansion of a fourth moment, gives
For the fourth-moment bound, expand the fourth power of a sum of centered bounded independent variables within a residue class. Only terms in which each index occurs at least twice survive, giving \(O(m^2)\) terms before division by \(m^4\). Combining the finitely many classes preserves the order. The second-moment bound follows from the martingale covariances already computed.
On \(E_m\), the absolute expectation of the exact remainder is at most
On \(E_m^c\), work with the original bounded ratio minus its first two Taylor terms. All three terms are uniformly bounded for fixed \(q>0\), so their expectation over that event is \(O(e^{-cm})\). This also covers the zero-denominator convention. Therefore
This argument proves the leading term; it does not claim the stronger \(O(n^{-2})\) remainder written in the archival Taylor sketch. The negative covariance sign in the mixed Hessian term is essential.