Nonparametric Inference for the Mean

For each fixed distribution with finite nonzero variance, the central limit theorem supports asymptotic inference for the mean. Uniform validity over a family of distributions is a stronger demand: the distribution may change with sample size. We begin with a triangular-array law of large numbers, then examine the tail conditions needed for uniform inference.

Theorem 1 (Weak Law of Large Numbers for Triangular Arrays). Let \(Y_{n,1}, \dots, Y_{n,n}\) be a triangular array whose entries are i.i.d. within each row. Suppose the \(n\)th row has the cumulative distribution function \(G_n\). Assume \(G_n \xrightarrow{d} G\), and \(\mathbb{E} |Y_{n,1}| \to \mathbb{E} |Y| < \infty\) as \(n\to\infty\); suppose \(Y\) has cumulative distribution function \(G\). Then,

\[\begin{aligned} \overline Y_n \triangleq \frac1n \sum_{1 \leq i \leq n} Y_{n,i} \xrightarrow{\mathop{\mathrm{\mathbf{P}}}} \mathbb{E} Y ~~ \text{as } n \to \infty ~. \end{aligned}\]

The proof separates convergence of the laws from control of their tails. Truncation handles the bounded part, while uniform integrability makes the discarded part negligible.

Lemma 2. Let \(Y_{n,1}, \dots, Y_{n,n}\) be i.i.d. random variables, with the \(n\)th row having the cumulative distribution function \(G_n\). Suppose \(\mu(G_n)\) is finite, and satisfies the asymptotic uniform integrability condition

\[\begin{aligned} \lim_{\eta \to \infty} \limsup_{n\to\infty} \mathbb{E}_{G_n} [|Y_{n,i} - \mu(G_n)| \mathbb 1 \{|Y_{n,i} - \mu(G_n)| \geq \eta\}] = 0 ~. \end{aligned}\tag{1}\]

Putting \(\overline Y_n \triangleq \frac1n \sum_{1 \leq i \leq n} Y_{n,i}\), we have, under \(G_n\), \(\overline Y_n - \mu(G_n) \to 0\) in probability (“in \(G_n\)-probability”).

Proof.

Center each row: put \(V_{n,i}=Y_{n,i}-\mu(G_n)\). For fixed \(K\), let \(V_{n,i}^{(K)}=V_{n,i}1_{\{|V_{n,i}|\leq K\}}\). Independence within the row gives

\[\operatorname{Var}\left(\frac1n\sum_iV_{n,i}^{(K)}\right)\leq K^2/n,\qquad \mathbb{E}\left|\frac1n\sum_i(V_{n,i}-V_{n,i}^{(K)})\right|\leq \mathbb{E}|V_{n,1}|1_{\{|V_{n,1}|>K\}}.\]

Since \(\mathbb{E}V_{n,1}=0\), the absolute value of \(\mathbb{E}V_{n,1}^{(K)}\) is bounded by the same tail expectation. Chebyshev handles the bounded centered average, and Markov handles the discarded tail. Taking \(n\to\infty\) and then \(K\to\infty\) proves the claim.

◻

Lemma 3. Let \(X_n\) and \(X\) be real-valued random variables such that \(X_n \xrightarrow{d} X\).

If \(X_n\) is uniformly bounded for all \(n\), then \(\mathbb{E} X_n \to \mathbb{E} X\).
Fix \(t>0\) such that \(\mathbb{P}(|X|=t)=0\). Then, \(\mathbb{E} [|X_n| \mathbb 1 \{|X_n| \leq t\}] \to \mathbb{E} [|X| \mathbb 1 \{|X| \leq t\}]\). If, furthermore, \(\mathbb{E}|X_n| \to \mathbb{E}|X|\), then \(\mathbb{E} [|X_n| \mathbb 1 \{|X_n| > t\}] \to \mathbb{E} [|X| \mathbb 1 \{|X| > t\}]\).

Proof of Lemma 3. If \(|X_n|\leq C\) almost surely for every \(n\), Portmanteau applied to the open complement of \([-C,C]\) gives \(|X|\leq C\) almost surely. The clipping function \(x\mapsto\max(-C,\min(x,C))\) is bounded and continuous and equals each of these variables on its support. Weak convergence therefore gives \(\mathbb{E}X_n\to\mathbb{E}X\).

For part (b), the bounded function \(h_t(x)=|x|1_{\{|x|\leq t\}}\) is continuous except possibly at \(x=\pm t\). These discontinuities have zero limiting probability by assumption, so the continuous mapping theorem gives \(h_t(X_n)\Rightarrow h_t(X)\). Part (a), with bound \(t\), gives convergence of their expectations. Subtracting this convergence from \(\mathbb{E}|X_n|\to\mathbb{E}|X|<\infty\) proves the tail assertion. The bounded truncated-moment assertion needs continuity only at the chosen threshold, not at every point of the distribution. ◻

Proof of Theorem 1. Write \(Y_n=Y_{n,1}\). For every \(K>0\), the bounded continuous function \(|y|\wedge K\) has convergent expectations. Thus

\[\mathbb{E}(|Y_n|-K)_+=\mathbb{E}|Y_n|-\mathbb{E}(|Y_n|\wedge K)\longrightarrow\mathbb{E}(|Y|-K)_+.\]

Since \(|y|1_{\{|y|>2K\}}\leq2(|y|-K)_+\), letting \(K\to\infty\) proves asymptotic uniform integrability of \(Y_n\). For the signed clipping function \(c_K(y)=\max(-K,\min(y,K))\), weak convergence gives \(\mathbb{E}c_K(Y_n)\to\mathbb{E}c_K(Y)\), while \(|y-c_K(y)|=(|y|-K)_+\). The preceding tail bound therefore proves \(\mu_n:=\mathbb{E}Y_n\to\mathbb{E}Y\).

\[\mathbb{E}[|Y_n-\mu_n|1_{\{|Y_n-\mu_n|>K\}}]\leq2\mathbb{E}[|Y_n|1_{\{|Y_n|>K/2\}}].\]

The centered rows satisfy Lemma 2. Thus \(\bar Y_n-\mu_n\to0\) in probability, and combining this with \(\mu_n\to\mathbb{E}Y\) proves the theorem. ◻

We now apply the array results to the studentised sample mean. Consider the \(t\)-test constructed from i.i.d. data \(X_1, \dots, X_n\) following some distribution \(F\). The classical setting is \(F = \mathcal{N}(\mu,\sigma^2)\). The test of \(H_0 : \mu = \mu_0\) against \(H_1 : \mu > \mu_0\) is based on the test-statistic

\[\begin{aligned} t_n = t_n (X_1, \dots, X_n) = \frac{\sqrt{n} (\overline X_n - \mu_0)}{s_n} ~, \end{aligned}\]

where

\[\begin{aligned} s_n^2 = \frac{\sum_{1 \leq i \leq n} (X_i - \overline X_n)^2}{n-1} ~. \end{aligned}\]

For Gaussian data with \(\sigma^2>0\), \(t_n\) has the central Student distribution with \(n-1\) degrees of freedom under \(H_0:\mu=\mu_0\). Under an alternative it has a noncentral Student distribution with noncentrality \(\sqrt n(\mu-\mu_0)/\sigma\). For a fixed, possibly nonnormal \(F\) with mean \(\mu_0\) and finite nonzero variance, the ordinary CLT, consistency of the sample variance, and Slutsky give \(t_n\Rightarrow \mathcal{N}(0,1)\). Consequently the one-sided test using \(t_{n-1,1-\alpha}\) has rejection probability

\[\begin{aligned} \alpha_n (F) = \mathop{\mathrm{\mathbf{P}}}_{F} (t_n \geq t_{n-1,1-\alpha}) \end{aligned}\]

Under these null assumptions, \(\alpha_n(F)\to\alpha\). This is pointwise asymptotic validity of the level. For a fixed alternative with \(\mu(F)>\mu_0\) and finite nonzero variance, \(t_n\to+\infty\) in probability, so power tends to one. Neither claim is asserted for arbitrary distributions without these moment and variance assumptions.

For the exact Gaussian assertion, an orthogonal change of coordinates separates the sample-mean direction from the \(n-1\) residual directions. Thus \(Z=\sqrt n(\bar X_n-\mu)/\sigma\sim\mathcal{N}(0,1)\) is independent of \(Q=(n-1)s_n^2/\sigma^2\sim\chi^2_{n-1}\), and

\[t_n=\frac{Z+\sqrt n(\mu-\mu_0)/\sigma}{\sqrt{Q/(n-1)}}.\]

This is the central or noncentral Student law according to whether the shift is zero. Since \(Q/(n-1)\to1\) in probability, the central Student laws converge to the standard normal. Strict increase and continuity of the normal distribution function then imply convergence of their fixed interior quantiles.

For a nonnormal fixed distribution, the weak law applied to \(X_i-\mu\) and \((X_i-\mu)^2\) gives

\[s_n^2=\frac n{n-1}\left(\frac1n\sum_i(X_i-\mu)^2-(\bar X_n-\mu)^2\right)\xrightarrow{P}\sigma^2.\]

This supplies the sample-variance step in the pointwise CLT, using only the finite second moment. Any convention on the event \(s_n=0\) is asymptotically irrelevant here because its probability tends to zero.

The convergence \(t_{n-1,1-\alpha} \to z_{1-\alpha}\) concerns deterministic critical values. The harder question is whether the rejection probabilities approach the nominal level uniformly over a family of distributions. A failure of uniformity allows \(F\) to vary with sample size. For a fixed distribution satisfying the pointwise theorem, the behaviour \(\alpha_n (F) \to \zeta \neq \alpha\) cannot occur. The next example shows why allowing the distribution to vary makes a difference.

Lemma 4. Fix \(n\geq2\) and test \(H_0:\mu=0\) against a positive mean using a finite upper critical value. Define the statistic at zero sample variance by \(t_n=+\infty\) if \(\bar X_n>0\), \(t_n=-\infty\) if \(\bar X_n<0\), and \(t_n=0\) otherwise. Over all null distributions with finite positive variance, the supremum rejection probability is one.

Proof.

For any \(0<c<1\) choose \(0<p<1\) with \((1-p)^n=c\). Let \(P(X=p)=1-p\) and \(P(X=-(1-p))=p\). This distribution has mean zero and variance \(p(1-p)>0\). With probability \(c\) all observations equal \(p\). On that event, \(\sqrt n\bar X_n=\sqrt n\,p>0\) and \(s_n=0\), so the test rejects by the stated convention. Its rejection probability is at least \(c\). Let \(c\uparrow1\). This is a supremum over distributions for each fixed sample size; it does not contradict the pointwise limit.

◻

The counterexample shows what can happen without uniform tail control. In the rest of this section, \(\mathscr F\) consists of distributions with finite positive variance, and we set \(\mu_0=0\) by translating the observations. Let \(\widetilde{\mathscr F}\subseteq\mathscr F\) satisfy

\[\begin{aligned} \lim\limits_{M\to\infty} \sup_{F \in \widetilde{\mathscr F}} \mathbb{E}_F \left[ \frac{|X-\mu(F)|^2}{\sigma^2(F)} \mathbb 1 \left\{ \frac{|X-\mu(F)|}{\sigma(F)} > M \right\} \right] = 0 ~. \end{aligned}\tag{2}\]

This condition controls the second-moment contribution from large standardised observations, uniformly over the family. It excludes the tail behaviour used in Lemma 4 to defeat a uniform normal approximation.

Proposition 5. Define \(\widetilde{\mathscr F}\) to be a collection of distributions \(F \in \mathscr F\) satisfying (2). Suppose \(X_{n1}, \dots, X_{nn}\) are i.i.d. random variables following \(F_n \in \widetilde{\mathscr F}\) with \(\mu(F_n) = 0\) (but finite variance). Under \(F_n\), we then have

\[\begin{aligned} \frac{\frac1n \sum_{1 \leq i \leq n} X_{ni}^2}{\sigma^2(F_n)} \xrightarrow{\mathop{\mathrm{\mathbf{P}}}} 1 ~. \end{aligned}\]

Proof.

Put \(V_{n,i}=X_{n,i}^2/\sigma^2(F_n)\). Then \(\mathbb{E}V_{n,i}=1\). Condition (2) makes these nonnegative row variables uniformly integrable, since \(V_{n,i}>K\) is equivalent to \(|X_{n,i}|/\sigma(F_n)>\sqrt K\). Their centered versions \(V_{n,i}-1\) therefore satisfy Lemma 2. It follows that \(n^{-1}\sum_iV_{n,i}\to1\) in probability. No individual \(V_{n,i}\) is claimed to converge to one.

◻

Proposition 6. Define \(\widetilde{\mathscr F}\) to be a collection of distributions \(F \in \mathscr F\) satisfying (2). Pick \(F_n \in \widetilde{\mathscr F}\). Assume

\[\begin{aligned} \frac{\sqrt n \mu(F_n)}{\sigma(F_n)} \xrightarrow{n \to \infty} \delta ~, \end{aligned}\]

where \(|\delta|=\infty\) is allowed. Suppose \(X_1, \dots, X_n\) are i.i.d. random variables following \(F_n\). Define the \(t\)-statistic as before; denote it \(t_n\). If \(|\delta| < \infty\), then \(t_n \xrightarrow{d} \mathcal{N}(\delta,1)\) under \(F_n\); otherwise, if \(\delta = \pm \infty\), then \(t_n \xrightarrow{\mathop{\mathrm{\mathbf{P}}}} \pm \infty\) under \(F_n\).

Proof.

Standardize by the population mean and variance: \(Z_{n,i}=(X_{n,i}-\mu(F_n))/\sigma(F_n)\). These variables are i.i.d. within each row, centered, with variance one. Condition (2) gives the Lindeberg bound \(\mathbb{E}[Z_{n,1}^21_{\{|Z_{n,1}|>\varepsilon\sqrt n\}}]\to0\), so \(\sqrt n\bar Z_n\Rightarrow \mathcal{N}(0,1)\). The same truncation argument as in Proposition 5 gives \(n^{-1}\sum_iZ_{n,i}^2\to1\). Consequently

\[\frac{s_n^2}{\sigma^2(F_n)}=\frac n{n-1}\left(\frac1n\sum_iZ_{n,i}^2-\bar Z_n^2\right)\xrightarrow{P}1.\]

This identity handles the dependence of the sample-centered residuals; it does not assign them the original distribution. With \(\delta_n=\sqrt n\mu(F_n)/\sigma(F_n)\),

\[t_n=\frac{\sqrt n\bar Z_n+\delta_n}{s_n/\sigma(F_n)}.\]

If \(\delta_n\to\delta\in\mathbb R\), Slutsky gives \(\mathcal{N}(\delta,1)\). If \(\delta_n\to\pm\infty\), the numerator is \(\delta_n+O_P(1)\) and the positive denominator tends to one, so \(t_n\to\pm\infty\) in probability. The zero-denominator event has probability tending to zero.

◻

Theorem 7. Under (2), let \(c_n\to z_{1-\alpha}\) be either the normal critical value or the corresponding Student critical values, with \(0<\alpha<1\). Then

\[\sup_{F\in\widetilde{\mathscr F}}\left|P_F(t_n>c_n)-\left[1-\Phi\left(z_{1-\alpha}-\frac{\sqrt n\mu(F)}{\sigma(F)}\right)\right]\right|\longrightarrow0.\tag{3}\]

In particular the test has uniformly correct asymptotic level on \(\widetilde{\mathscr F}_0=\{F\in\widetilde{\mathscr F}:\mu(F)=0\}\). Along any sequence \(F_n\in\widetilde{\mathscr F}\) with standardized mean \(\sqrt n\mu(F_n)/\sigma(F_n)\to\delta\), its power tends to \(1-\Phi(z_{1-\alpha}-\delta)\), with the natural limits zero and one at \(\delta=-\infty,+\infty\).

Proof.

If the uniform approximation failed, choose sample sizes and distributions where the absolute error stays bounded below. Pass to a subsequence on which the standardized means converge in the extended real line. Proposition 6 gives the claimed power limit on that subsequence, whether the limit is finite or infinite. This contradicts the lower bound on the error. The null statement follows by setting the standardized mean to zero.

◻

For a fixed lower bound \(d\) on the standardized mean, the same approximation yields

\[\liminf_n\inf_{\substack{F\in\widetilde{\mathscr F}:\\ \sqrt n\mu(F)/\sigma(F)\geq d}}P_F(t_n>c_n)\geq1-\Phi(z_{1-\alpha}-d),\]

provided these sets are nonempty eventually. Equality requires that the family contain a sequence whose standardized means approach \(d\); it does not follow from the tail condition alone.

The final argument concerns unrestricted families closed under mixing. A small contamination can change the mean while leaving a fixed-size sample almost unchanged. The precise conclusion concerns the supremum and infimum of rejection probabilities among distributions with a prescribed mean. These extrema are the same for every mean; individual rejection probabilities can still differ. Formally, suppose \(\mathscr F\) is now a family of distributions on \(\mathbb R\) such that

\(\mu(F) < \infty\) and exists for all \(F \in \mathscr F\);
for all \(\mu \in \mathbb R\), there exists some \(F \in \mathscr F\) such that \(\mu(F) = \mu\); and
for all \(\lambda \in [0,1]\) and \(F,G \in \mathscr F\), \(\lambda F + (1-\lambda)G \in \mathscr F\).

Suppose \(X_1, \dots, X_n \stackrel{\rm i.i.d.}{\sim} F \in \mathscr F\), and define a measurable test function taking values in \([0,1]\), \(\varphi_n = \varphi_n(X_1, \dots, X_n)\) along with \(\mathscr G_m = \{F \in \mathscr F : \mu(F) = m\}\). Then, the following quantities are independent of \(m\):

\[\begin{aligned} \inf_{F \in \mathscr G_m} \mathbb{E}_F \varphi_n ~~,~~ \sup_{F \in \mathscr G_m} \mathbb{E}_F \varphi_n ~. \end{aligned}\]

To prove the supremum assertion, fix \(m_0\), there exists some subsequence \(F_k\) such that \(\mathbb{E}_{F_k} \varphi_n \xrightarrow{k \to \infty} \sup_{F \in \mathscr G_{m_0}} \mathbb{E}_F \varphi_n \eqqcolon M\). To reach a contradiction, we take another \(m_1 \neq m_0\), and we claim

\[\begin{aligned} \sup_{F \in \mathscr G_{m_1}} \mathbb{E}_F \varphi_n = M ~. \end{aligned}\]

Choose a contamination distribution whose mean compensates for its small mixing weight. For a distribution \(H_k\) with mean \(h_k\) satisfying

\[\begin{aligned} m_1 = \left( 1- \frac1k \right) m_0 + \frac1k h_k ~, \end{aligned}\]

define

\[\begin{aligned} G_k = \left( 1- \frac1k \right) F_k + \frac1k H_k ~. \end{aligned}\]

In other words, \(G_k \in \mathscr G_{m_1}\), and drawing from \(G_k\) amounts to first identifying whether we draw from \(F_k\) or from \(H_k\) (the former happens with probability \(1-\frac1k\), and the latter happens with the remaining probability \(\frac1k\)), and then draw from \(F_k\) or \(H_k\). So, a sample of size \(n\) drawn from \(G_k\) would consist entirely of sample draws from \(F_k\) with probability \((1-\frac1k)^n\). As such,

\[\begin{aligned} \sup_{F \in \mathscr G_{m_1}} \mathbb{E}_F \varphi_n \geq \mathbb{E}_{G_k} \varphi_n \geq \left( 1-\frac1k \right)^n \mathbb{E}_{F_k} \varphi_n \xrightarrow{k\to\infty} M ~, \end{aligned}\]

implying

\[\begin{aligned} \sup_{F \in \mathscr G_{m_1}} \mathbb{E}_F \varphi_n \geq \sup_{F \in \mathscr G_{m_0}} \mathbb{E}_F \varphi_n ~. \end{aligned}\]

Clearly, exchange the roles \(m_0\) and \(m_1\) would then establish the equality. The case for the infimum follows from applying the arguments above to \(1-\varphi_n\) instead of \(\varphi_n\).

The contamination argument gives the Bahadur–Savage impossibility theorem. Its conclusion concerns tests that control size over the entire family: such control prevents power against any alternative in that family.

Theorem 8 (Bahadur-Savage). For \(\mathscr F\) satisfying the three conditions described above, the following statements are true:

Any test of \(H_0 : \mu(F)=0\) against \(H_1 : \mu(F) \neq 0\) that has size \(\alpha\) for the family \(\mathscr F\) has power less than or equal to \(\alpha\) for any alternative \(F \in \mathscr F\).
Any test of \(H_0 : \mu(F)=0\) against \(H_1 : \mu(F) \neq 0\) that has power \(\beta\) against some alternative \(F \in \mathscr F\) has size greater than equal to \(\beta\).

Proof. Let \(\alpha=\sup_{F\in\mathscr G_0}\mathbb{E}_F\varphi_n\) be the size. The contamination argument proves \(\sup_{F\in\mathscr G_m}\mathbb{E}_F\varphi_n=\alpha\) for every real \(m\). Every alternative belongs to one such class, so its rejection probability is at most \(\alpha\). Equivalently, if one alternative has rejection probability \(\beta\), then \(\alpha\geq\beta\). The argument applies at each fixed sample size and to every measurable test taking values in \([0,1]\). ◻