Asymptotic Power, Relative Efficiency, and Optimal Tests
Two consistent tests can both have power tending to one at every fixed alternative, while requiring different sample sizes to detect a small departure from the null. Local alternatives retain a nontrivial power comparison as sample size grows. We calculate these power functions, interpret their slopes through relative efficiency, and then use the local likelihood experiment to bound the power of every test.
Throughout, tests take values in \([0,1]\), allowing randomization, and \(z_u=\Phi^{-1}(u)\) is the lower-tail standard-normal quantile. Thus a level-\(\alpha\) upper-tail normal test uses \(z_{1-\alpha}\). In the iid parametric sections, the score and information come from quadratic mean differentiability (QMD), and the likelihood expansion is local asymptotic normality (LAN).
First consider simple hypotheses with densities \(p_0,p_1\) relative to \(\mu\). The sum of type-I and type-II error probabilities is
Proposition 1. A minimizing test equals one where \(p_1>p_0\), zero where \(p_1<p_0\), and can take any value on the equality set.1 Moreover,
Proof. Write \(S_{0,1}(\varphi)=1+\int\varphi(p_0-p_1)\,d\mu\), and minimize the integrand pointwise. Since \(\int(p_1-p_0)d\mu=0\), the integral of its positive part is half its absolute integral. This gives the formula and also describes all minimizers, up to null sets. ◻
Finite-sample optimal tests therefore need not be unique. For a fixed-level likelihood-ratio test, the decisions away from its boundary are prescribed, while randomization on the boundary is constrained by the size. Asymptotic optimality allows still more tests to share the same limiting performance.
1. Asymptotic Power Functions and Asymptotic Relative Efficiency
The power is \(\pi_n(\theta)=\mathbb{E}_\theta\varphi_n\). For a composite null \(\Theta_0\), asymptotic level \(\alpha\) means \(\limsup_n\sup_{\theta\in\Theta_0}\pi_n(\theta)\le\alpha\). Pointwise power limits need not exist for arbitrary tests. For two consistent tests, however, the limit is one at each fixed alternative and tells us little about their relative performance near the null.
Example 2 (Sign test). Let \(X_i\) have cdf \(F(x-\theta)\), where \(F\) is continuous at zero, \(F(0)=1/2\), and zero is its unique median. For testing \(\theta=0\) against \(\theta>0\), put
The test rejects when \(2\sqrt n(T_n-1/2)>z_{1-\alpha}\). Its null rejection probability tends to \(\alpha\). If \(\theta_n\to0\), the bounded Bernoulli triangular-array CLT gives
The approximation is valid even if the displayed threshold diverges: convergence of cdfs to the continuous normal cdf is uniform in the threshold. Suppose further that \(F\) is differentiable at zero with \(f(0)>0\). For positive \(\theta_n\to0\),
Consequently, if \(\sqrt n\theta_n\to0\), the power tends to \(\alpha\); if \(\sqrt n\theta_n\to h\in(0,\infty)\), it tends to \(1-\Phi(z_{1-\alpha}-2hf(0))\); and if \(\sqrt n\theta_n\to\infty\), it tends to one. Thus the nontrivial transition occurs at order \(n^{-1/2}\). At each fixed positive alternative, uniqueness of the median gives \(p(\theta)>1/2\), and the law of large numbers proves consistency.
Theorem 3 (Local power). Suppose, for every \(h_n\to h\), under \(\theta_n=h_n/\sqrt n\),
Assume \(\mu\) is differentiable at zero, \(\sigma\) is continuous there, and \(\sigma(0)>0\). Reject when \(\sqrt n[T_n-\mu(0)]>\sigma(0)z_{1-\alpha}+o(1)\), with deterministic cutoff error. Then
Proof. Subtract the local mean, use \(\sqrt n[\mu(\theta_n)-\mu(0)]\to h\mu'(0)\), and apply Slutsky's theorem and normal continuity. This is the sequential formulation of van der Vaart, Theorem 14.7. Asymptotic level at most \(\alpha\) alone does not force this power formula; the calibrated cutoff is part of the statement. ◻
Remark 4. The slope or efficacy is \(a=\mu'(0)/\sigma(0)\). For positive slopes, larger \(a\) gives higher power at every fixed positive local shift. The relative efficiency of the first test with respect to the second is \((a_1/a_2)^2\), with the sample-size convention established in Theorem 7.
Remark 5 (Third lemma and the two-sample median test). Let \(X_1,\ldots,X_m\) have cdf \(F\), and \(Y_1,\ldots,Y_n\) have cdf \(F(\cdot-\theta)\), independently, with \(N=m+n\) and \(m/N\to\lambda\in(0,1)\). Assume the location family is QMD, has finite positive information, and has a continuous positive density \(f\) near its unique median \(q\); assume also that \(f\) is absolutely continuous and tends to zero at negative infinity. Let \(R_{N,i}\) be the ascending pooled ranks and put
Under the null there are no ties. With \(k=\lfloor(N+1)/2\rfloor\), each rank is uniform, so \(\mathbb{E}_0T_N=nk/N^2=n/(2N)+O(N^{-1})\). The rank projection of van der Vaart, Theorem 13.5 and Example 14.14 gives
The hypotheses match this rank theorem: the underlying null distribution is continuous, the score function is the bounded indicator of the lower half of the ranks, and both sample proportions have nonzero limits. This projection can also be obtained by expanding the pooled empirical median and substituting it into the empirical cdf of the second sample. The null variance is \(\lambda(1-\lambda)/4\).
For completeness, write \(B_i=\mathbf1\{X_i\le q\}-1/2\) and \(C_j=\mathbf1\{Y_j\le q\}-1/2\). Under the null these are independent, centered, and have variance \(1/4\). Thus the variance of the displayed projection is
The rank theorem uses \(\phi(u)=\mathbf1\{u\le1/2\}\), which is continuous except at one point and satisfies \(N^{-1}\sum_{i=1}^N\phi^2(i/(N+1))\to\int_0^1\phi^2=1/2\). Its coefficients are zero in the first sample and \(1/N\) in the second; centering these coefficients gives exactly the two coefficients above. The deterministic centering error \(nk/N^2-n/(2N)=O(N^{-1})\) disappears after multiplication by \(\sqrt N\).
For \(\theta_N=h/\sqrt N\), the local likelihood score is \(-N^{-1/2}\sum_{j=1}^n f'(Y_j)/f(Y_j)\). Its covariance with the projection is
The joint CLT and third lemma therefore give local mean \(-h\lambda(1-\lambda)f(q)\). Positive shifts reduce the number of second-sample observations below the pooled median, so the correctly oriented statistic is \(U_N=n/(2N)-T_N\), rejecting for large values. Its slope is \(2\sqrt{\lambda(1-\lambda)}f(q)\). Stating the orientation prevents a sign error from reversing the intended alternative.
Proposition 6 (Consistency). If \(T_n\to\mu(\theta)\) in \(P_\theta\)-probability and the rejection cutoffs tend to \(\mu(0)\), then the tests are consistent whenever \(\mu(\theta)>\mu(0)\). Alternatively, suppose Theorem 3 holds with positive slope and each power function is nondecreasing. Then, for positive \(\theta_n\), power tends to \(\alpha\) if \(\sqrt n\theta_n\to0\), and to one if \(\sqrt n\theta_n\to\infty\). In particular the tests are consistent at every fixed positive alternative.
Proof. The first claim is the definition of convergence in probability with a strict gap. For the second, sandwich the power between its values at zero and at \(h/\sqrt n\), then let \(h\downarrow0\); for divergent local shifts, bound it below by the power at \(h/\sqrt n\) and let \(h\to\infty\). These are the arguments in van der Vaart, Lemmas 14.15–14.16. ◻
Sample-size efficiency needs a precise calibration convention. Fix \(0<\alpha<\gamma<1\), and for each procedure suppose its size is at most \(\alpha\) at every sample size, with asymptotic size \(\alpha\). Boundary randomization can be used when needed. Let \(n_i(\theta)\) be the smallest sample size at which procedure \(i\) has power at least \(\gamma\) against \(\theta>0\).
Theorem 7 (Pitman efficiency). Suppose \(\|P_{n,\theta}-P_{n,0}\|_{\rm TV}\to0\) as \(\theta\to0\) for each fixed \(n\). Suppose the two calibrated procedures satisfy Theorem 3 for every convergent sequence of local shifts, with positive slopes \(a_i\). Then, as \(\theta\downarrow0\),
This is a version of van der Vaart, Theorem 14.19, with the local sequential and finite-sample calibration assumptions made explicit.
Proof. Put \(h_i=(z_{1-\alpha}-z_{1-\gamma})/a_i>0\). At sample size \(\lceil(h_i+\epsilon)^2/\theta^2\rceil\), local power tends to a number strictly larger than \(\gamma\), giving the upper bound on the minimum. For the lower bound, suppose some sample sizes \(m\le(h_i-\epsilon)^2/\theta^2\) reach power \(\gamma\). Bounded \(m\) are impossible: total-variation continuity and size at most \(\alpha\) force their power to stay below \(\gamma\). Along an unbounded subsequence, extract \(\sqrt m\theta\to h\in[0,h_i-\epsilon]\). The sequential local-power formula gives a limit strictly below \(\gamma\), another contradiction. Let \(\epsilon\downarrow0\). This proof does not infer monotonicity in sample size from a comparison of two powers. ◻
Theorem 8 (Random critical values). In an iid family QMD at an interior point \(\theta_0\), suppose \(\mu\) has positive right derivative at \(\theta_0\), \(\sigma>0\), and for every bounded nonnegative local sequence \(h_n\), under \(\theta_n=\theta_0+h_n/\sqrt n\),
Reject when \(\sqrt n[T_n-\mu(\theta_0)]>\hat c_n\), where \(\hat c_n\to\sigma z_{1-\alpha}\) in null probability. Then the size tends to \(\alpha\), and for \(h_n\to h\ge0\),
If \(\theta_j\downarrow\theta_0\), \(n_j\to\infty\), \(\sqrt{n_j}(\theta_j-\theta_0)=O(1)\), and the powers tend to \(\beta\in(\alpha,1)\), then
Proof. LAN implies contiguity along bounded local alternatives, so convergence of the random critical value transfers from the null to these alternatives. Slutsky proves the power formula. For the final assertion, every convergent subsequence of the bounded local shifts must have limit \((z_{1-\alpha}-z_{1-\beta})\sigma/\mu'(\theta_0)\), because the limiting power is strictly increasing in the shift. Thus the entire sequence has this limit. The bounded-shift and diverging-sample-size conditions are needed here; local normality alone says nothing about arbitrary distant alternatives or bounded sample sizes. The minimum-sample-size formulation in Theorem 7 supplies those controls by a different argument. ◻
Lemma 9. Suppose the family is QMD at \(\theta_0\), and for each fixed \(h\ge0\), \(\sqrt nT_n\Rightarrow\mathcal{N}(hm,\sigma^2)\) under \(\theta_0+h/\sqrt n\), with \(m,\sigma>0\). Then these limits also hold along \(h_n\to h\), and the calibrated upper-tail tests have efficacy \(m/\sigma\).
Proof. QMD gives \(\sqrt{p_{\theta_0+t}}=\sqrt{p_{\theta_0}}+t\dot\ell\sqrt{p_{\theta_0}}/2+o_{L^2}(|t|)\). Subtract this expansion at \(t=h_n/\sqrt n\) and \(t=h/\sqrt n\). Their one-observation squared Hellinger distance is \(o(n^{-1})\), so the total-variation distance of the product laws tends to zero. The fixed-shift limit transfers to the varying shift. Subsequences then give centered normality for all bounded shifts, with \(\mu(\theta)=m(\theta-\theta_0)\). The cutoff must still be specified as in Theorem 3 or 8; a limit law alone does not define a test. ◻
Here is the distance estimate used in Lemma 9. Define \(H^2(P,Q)=\int(\sqrt p-\sqrt q)^2\) and the affinity \(\rho(P,Q)=\int\sqrt{pq}=1-H^2(P,Q)/2\). Tensorization of the affinity and \(1-(1-x)^n\le nx\), for \(0\le x\le1\), give
Cauchy–Schwarz also gives \(\|P-Q\|_{\rm TV}\le H(P,Q)\), since \(\int(\sqrt p+\sqrt q)^2\le4\). In Lemma 9 the difference of square-root densities is \((h_n-h)\dot\ell\sqrt{p_{\theta_0}}/(2\sqrt n)+o_{L^2}(n^{-1/2})\), so its squared norm is \(o(n^{-1})\). The product-law bound therefore tends to zero. It controls the difference of the expectations of every measurable test with values in \([0,1]\), including tests whose rejection sets vary with \(n\).
2. Asymptotically Most Powerful, Asymptotically Uniformly Most Powerful, and Locally Asymptotically Uniformly Most Powerful Tests
Assume now QMD at an interior scalar parameter \(\theta_0\), with \(0<I=\mathbb{E}_{\theta_0}\dot\ell^2<\infty\). Write
Under the null \(Z_n\Rightarrow\mathcal{N}(0,I)\). The third lemma gives \(Z_n\Rightarrow\mathcal{N}(hI,I)\) under \(\theta_0+h/\sqrt n\), also along \(h_n\to h\) by Lemma 9's Hellinger argument.
Lemma 10 (Likelihood-ratio power bound). Fix \(h>0\) and \(\alpha_n\to\alpha\in(0,1)\). Let \(\varphi_{n,h}\) be a Neyman–Pearson test of size exactly \(\alpha_n\) against \(\theta_0+h/\sqrt n\), rejecting above a log-likelihood cutoff \(c_{n,h}\) and randomizing on equality. Then
The same power limit holds for most-powerful tests against \(\theta_0+h_n/\sqrt n\) when \(h_n\to h\ge0\). At \(h=0\), power tends to \(\alpha\), but the vanishing-boundary-mass assertion need not hold.
Proof. Under the null, the log likelihood has limit \(\mathcal{N}(-h^2I/2,h^2I)\). Its continuous strictly increasing cdf traps the size-calibrated cutoffs at the stated quantile, and makes the boundary mass vanish. Under the alternative, the third lemma changes the mean to \(h^2I/2\), with the same variance. The upper-tail probability gives \(B_\alpha(h)\). Contiguity also transfers the vanishing probability of the boundary event.
To spell out the cutoff step, let \(W_n=\log L_{n,h}\), let \(F\) be its limiting null cdf, and let \(c=F^{-1}(1-\alpha)\). Exact size implies \(P_0(W_n>c_{n,h})\le\alpha_n\le P_0(W_n\ge c_{n,h})\). For each \(\epsilon>0\), the limiting tail at \(c+\epsilon\) is strictly below \(\alpha\), and the tail at \(c-\epsilon\) is strictly above it. These inequalities rule out cutoffs outside \((c-\epsilon,c+\epsilon)\) eventually. Once the cutoff is inside that interval,
Letting \(\epsilon\downarrow0\) proves the boundary claim. The same deterministic-cutoff sandwich under the alternative gives the tail at \(c\); subtracting its mean and dividing by its standard deviation yields \((c-h^2I/2)/(h\sqrt I)=z_{1-\alpha}-h\sqrt I\).
For varying shifts tending to a positive \(h\), LAN and the same argument apply. If \(h_n\to0\), QMD gives total-variation distance tending to zero between the alternative product law and the null product law, so every size-\(\alpha_n\) test has power tending to \(\alpha\). This treats the degenerate endpoint without using a nonexistent nondegenerate likelihood quantile. ◻
A sequence \(\varphi_n\) is asymptotically most powerful (AMP) at level \(\alpha\) against a specified alternative sequence \(\theta_n\) if it has asymptotic level at most \(\alpha\) and, for every other such sequence \(\psi_n\),
The expectations in this comparison are under the alternatives. For \(\theta_n=\theta_0+h/\sqrt n\), \(h>0\), AMP is equivalent to convergence of power to \(B_\alpha(h)\). To prove the universal upper bound, fix \(\delta>0\). Eventually \(\psi_n\) has size at most \(\alpha+\delta\), so the Neyman–Pearson lemma and Lemma 10 give \(\limsup_n\mathbb{E}_{\theta_n}\psi_n\le B_{\alpha+\delta}(h)\). Send \(\delta\downarrow0\). A test attaining the bound is therefore AMP; conversely compare an AMP test with the score test below, which attains the bound. Full power convergence is required: merely having the correct limsup would permit poor power along another subsequence.
Example 11 (Rao's score test). Define
The null CLT gives asymptotic size \(\alpha\). The third lemma gives, for every \(h_n\to h\ge0\),
Thus the score test attains the local power bound for every nonnegative finite shift. This is an asymptotic optimality assertion; the normal cutoff does not assert exact finite-sample maximization of the derivative of power.
The convergence is uniform on every compact interval \(0\le h\le c\). Otherwise a subsequence of shifts would have errors bounded away from zero. Compactness gives a further subsequence converging to some \(h\in[0,c]\), including possibly zero. Sequential convergence and continuity of \(B_\alpha\) contradict the retained error. Hence
Finally, asymptotically uniformly most powerful (AUMP) means that for every asymptotically level-\(\alpha\) competitor,
Equivalently, this comparison holds along every alternative sequence \(\theta_n>\theta_0\), with no restriction on its rate or distance from the null. A fixed local shift does not give an equivalent definition.
The locally asymptotically uniformly most powerful (LAUMP) condition restricts the comparison to bounded local shifts: for every \(c>0\),
The score test is LAUMP under the stated QMD assumptions. If a competitor exceeded it by a fixed amount somewhere in a compact local interval, extract convergent shifts \(h_n\to h\). Lemma 10 and the size-\(\alpha+\delta\) comparison bound its limiting power by \(B_\alpha(h)\), whereas the score test converges to that value, a contradiction. At \(h=0\), use the total-variation argument from Lemma 10. These local conclusions do not establish global AUMP without additional control away from bounded local alternatives.
For minimizing the sum of errors, any value on \(p_0=p_1\) is optimal. For the fixed-level Neyman–Pearson problem, randomization on \(p_1=cp_0\) is chosen to attain the required size, and need not be unique pointwise.↩︎