Heavy-Tailed Cluster Inference: Bootstrap Failure and Score Subsampling
Studentization divides a score sum by an estimate based on its squared scores. It produces a normal limit only when the score distribution permits that approximation. In (Chiang, Sasaki, and Wang 2024), the number of clusters grows but a few extreme scores remain influential in both numerator and denominator. This differs from the few-large-cluster sign argument, which starts from a finite collection of well-behaved score limits.
1. The main results
Let \(S_g=X_g'\varepsilon_g\) be the regression score for independent, identically distributed clusters. Suppose the average design matrix converges to a positive definite \(Q\). For a scalar contrast \(r'\theta\), the leading score is \(U_g=r'Q^{-1}S_g\).
Theorem 1. Under the paper’s sampling and stable-domain-of-attraction conditions, with tail index \(a\in(1,2]\), the cluster \(t\) statistic has a Gaussian limit if and only if \(a=2\). For \(1<a<2\), the Rademacher wild cluster bootstrap is inconsistent. Cluster score subsampling consistently estimates the continuous limiting distribution when \(b\to\infty\), \(b/G\to0\), and the number of simulated subsamples diverges.
The index \(a=2\) includes the normal domain of attraction; finite variance is sufficient but is not the exact boundary. The claim concerns the score distribution, not cluster size in isolation.
2. Why studentization does not remove the tail problem
Ignoring negligible design-estimation terms, the statistic is
For \(a<2\), choose the tail normalization \(A_G\) so that \(G\Pr(|U_g|>A_G)\) is of constant order. Both \(A_G^{-1}\sum_gU_g\) and \(A_G^{-2}\sum_gU_g^2\) have nontrivial limits driven by the large observations. The denominator remains random at the same scale as the numerator. Their ratio is a self-normalized stable limit, rather than a standard normal variable.
Centering the denominator is negligible because
and the correction is smaller by a factor of order \(G^{-1}\) relative to the squared stable normalization.
3. Why the bootstrap and subsampling differ
Conditional on the observed scores, a Rademacher bootstrap changes their signs and fixes their magnitudes. Under heavy tails, the realized extreme magnitudes continue to affect the distribution as the sample grows. The conditional bootstrap law therefore retains randomness that it would need to average out to estimate the fixed sampling law.
Subsampling uses subsets of \(b=o(G)\) clusters, each at its own smaller-sample tail scale. The studentized subset statistic has the same limiting ratio, while centering at the full-sample estimate becomes negligible on that scale. Averaging across many subsets then estimates its quantiles without estimating the tail index.
4. The centering rate in score subsampling
Let the scalar score observations be \(S_g\) with mean \(\delta\), and write \(U_g=S_g-\delta\). For a subset \(I\) of size \(b\), use
The numerator would be centered at \(\delta\) in the population \(b\)-sample statistic. Replacing it by \(\overline S_G\) changes that numerator by \(b(\overline S_G-\delta)\). If the stable normalization is \(A_m=m^{1/a}\ell(m)\), then
For a fixed distribution in the stated domain of attraction, regular-variation bounds and \(a>1\) make this tend to zero when \(b/G\to0\). The denominator has order \(A_b\) and a nondegenerate positive limit after that normalization. Thus full-sample centering is negligible. The condition \(a>1\) is doing real work: the positive power \(1-1/a\) is what offsets the centering error.
For the full statistic, \(\sum U_g=O_p(A_G)\) implies
This justifies removing the denominator’s centering term. The squared-score sum itself still has a random limit: \(\sum U_g^2/A_G^2\) need not approach a constant for \(a<2\).
5. Why many overlapping subsets still estimate a distribution
First imagine centering subsets at the true \(\delta\). For a threshold \(x\), average the bounded kernel \(\mathbf1\{T_I\leq x\}\) over all subsets of size \(b\). Its expectation is the cdf of the \(b\)-sample statistic.
A useful variance argument avoids treating the overlapping subsets as independent. Randomly permute the \(G\) observations, form \(k=\lfloor G/b\rfloor\) disjoint blocks, and average the kernel over those blocks. Unconditionally, the blocks are independent and this average has variance at most \(1/(4k)\). Conditional expectation over the random permutation gives the complete subsampling average. The law of total variance then implies
The centering calculation transfers this result to the feasible statistic at continuity points of the limit cdf. Monotonicity of cdfs and a finite grid of continuity points then give uniform-in-\(x\) convergence when the limit cdf is continuous. If only \(M\) random subsets are evaluated independently conditional on the data, their additional Monte Carlo variance is at most \(1/(4M)\).
There are two errors to reduce. Increasing the subset size \(b\) makes its statistic approach the limit; increasing the number of evaluated subsets estimates that statistic’s distribution more accurately. Enumerating every subset cannot compensate for keeping the subset size too small.
6. Uniform validity requires three uniform approximations
Let \(J_m(x,P)\) be the cdf of the correctly centered \(m\)-sample statistic. A sufficient route to uniform coverage over a class \(\mathcal P\) is
uniform consistency of the empirical subsampling cdf for \(J_b\), and uniform control of the full-sample centering error together with regularity of the relevant quantiles. These premises allow the triangle inequality
If the cdf error is at most \(\eta\), the estimated \(u\)-quantile lies between population quantiles with levels near \(u-\eta\) and \(u+\eta\). Regularity at those quantiles makes the probability between the bounds small. At a discontinuity, that last step can fail and requires different inequalities or boundary randomization.
These requirements are stronger than the pointwise theorem. Uniform centering control requires a bound on \(a-1\) and on the slowly varying ratio across the distribution class. Likewise, finite variance for each distribution does not supply a uniform Lindeberg condition; uniform integrability of standardized squared scores is one sufficient condition.
The elementary array \(x_{m,n}=\mathbf1\{n\leq m\}\) illustrates the logical problem: for every fixed \(m\), \(x_{m,n}\to0\), while \(x_{n,n}=1\). A representation theorem for pointwise weak convergence cannot remove this diagonal obstruction. Adapting to an unknown fixed tail index and obtaining uniform coverage over an unrestricted class of tail shapes are different claims.
7. Why sign resampling retains the extremes
A simple diagnostic is the largest squared-score share,
In an ordinary finite-variance regime, no single observation dominates and \(R_G\to0\) in probability. In the stable regime with \(a<2\), large order statistics remain important. Conditional Rademacher draws replace \(U_g\) by \(\xi_gU_g\) but leave every \(U_g^2\) unchanged. Their distribution consequently retains the realized configuration of extreme magnitudes.
One can see the obstruction in a conditional fourth moment. Set \(w_g=U_g/(\sum_hU_h^2)^{1/2}\) and consider the idealized signed statistic \(T^*=\sum_g\xi_gw_g\), holding the scores fixed. Independent fair signs give
Thus studentization fixes the conditional variance but leaves the conditional shape dependent on the observed concentration of squared scores. Since \(\sum_gw_g^4\geq(\max_gw_g^2)^2=R_G^2\), an influential score prevents this fourth moment from approaching the Gaussian value. For bootstrap consistency in the stable regime, the concentration statistic itself retains a random limit. The signed reference law keeps that sample-specific randomness instead of estimating a fixed sampling distribution.
If one score supplied all the squared magnitude, the reference distribution would consist of two points, \(-1\) and \(1\). If \(G\) scores had equal magnitudes, its fourth moment would be \(3-2/G\). These are limiting illustrations of the same formula. They do not replace the paper’s point-process proof, which relates extreme order statistics to the stable law. They explain why a standard error that grows with the largest cluster is not sufficient to repair the critical value.
Subsampling estimates the distribution of the ratio directly. It can therefore be useful when the squared-score sum has a random limiting scale, rather than consistently estimating a deterministic variance.
The fourth-moment calculation explains what fixing the observed magnitudes leaves unresolved; the paper’s point-process argument proves the bootstrap failure. Smaller subsets use their own tail scale and usually omit the full sample’s largest observations, allowing the subsampling distribution to estimate the self-normalized limit.