The Neyman–Pearson Lemma and Unbiased Testing

For a simple null and a simple alternative, each possible sample contributes a certain amount of power for a given cost in null rejection probability. Their ratio is the likelihood ratio. The Neyman–Pearson lemma proves that rejecting at the largest ratios is optimal, with randomization used to fill any remaining probability at the boundary. We then consider what changes for composite alternatives, unbiasedness, and worst-case power.

A randomized test is a measurable function \(\phi:\mathcal X\to[0,1]\), implemented by rejecting when an independent uniform variable is at most \(\phi(X)\). Its power is \(\beta_\phi(\theta)=E_\theta\phi(X)\); \(1-\beta_\phi(\theta)\) is its type-II error probability at an alternative. A level-\(\alpha\) test satisfies \(\sup_{\theta\in\Theta_0}E_\theta\phi\le\alpha\). It is most powerful against a specified alternative if it maximizes power there, and uniformly most powerful if it does so simultaneously at every alternative.

For simple hypotheses \(P_0,P_1\), the likelihood ratio \(p_1/p_0\) measures power gained per unit of null rejection probability. Randomization permits partial inclusion of a boundary atom. Without randomization, the discrete optimization is a subset-selection problem; a greedy likelihood-ratio cutoff need not solve it exactly.

Lemma 1. Let \(p_0,p_1\) be densities relative to a common measure, and \(0<\alpha<1\). There is a finite \(k\ge0\) and a test \(\phi^*\) of null size exactly \(\alpha\) satisfying

\[\phi^*(x)=1\text{ if }p_1(x)>kp_0(x),\qquad\phi^*(x)=0\text{ if }p_1(x)<kp_0(x).\]

Any such test is most powerful at level \(\alpha\). Every most powerful test agrees with this rule away from \(\{p_1=kp_0\}\), almost everywhere. A most powerful test either has size \(\alpha\) or has power one.

Existence. On \(\{p_0>0\}\), let \(R=p_1/p_0\); it is finite \(P_0\)-almost surely. Choose a \((1-\alpha)\)-quantile \(k\) so that

\[P_0(R>k)\le\alpha\le P_0(R\ge k).\]

This includes \(k=0\). If \(b=P_0(R=k)>0\), set \(\gamma=[\alpha-P_0(R>k)]/b\); otherwise the two tail probabilities agree with \(\alpha\), and take any \(\gamma\in[0,1]\). Put \(\phi^*=1\) above the threshold, \(\gamma\) on the boundary, and zero below it. On \(\{p_0=0,p_1>0\}\), always reject; this gains power at no null cost. Values where both densities vanish are immaterial. These definitions give exact size. The strict tail \(P_0(R>c)\) is nonincreasing and right-continuous; it is not a cdf.

Sufficiency. For any competitor \(\psi\) of size at most \(\alpha\), the threshold rule gives the pointwise inequality \((\phi^*-\psi)(p_1-kp_0)\ge0\), including the boundary where the second factor is zero. Hence

\[E_1\phi^*-E_1\psi=\int(\phi^*-\psi)(p_1-kp_0)\,d\mu+k(\alpha-E_0\psi)\ge0.\]

Necessity. If \(\psi\) is also most powerful, the left side is zero, so both nonnegative terms on the right vanish. A nonnegative integrand with zero integral is zero almost everywhere. On \(p_1>kp_0\) this forces \(\psi=1\), and on \(p_1<kp_0\) it forces \(\psi=0\). Finally, if \(E_0\psi<\alpha\) and \(E_1\psi<1\), the test \(\psi_t=\psi+t(1-\psi)\), for sufficiently small \(t>0\), remains of level \(\alpha\) and has strictly larger power. This contradicts optimality and proves the size-or-unit-power assertion.

Remark 2. On a positive-mass likelihood-ratio boundary, many allocations can be optimal; only their required null integral matters. A constant boundary probability is a convenient choice. Away from this boundary the rule is unique almost everywhere. If both densities vanish on a set, modifying the test there changes neither size nor power, whatever its measure under an unnecessarily large dominating measure.

Corollary 3. For \(0<\alpha<1\), the optimal power is strictly larger than \(\alpha\) whenever \(P_0\ne P_1\). If \(P_0=P_1\), it equals \(\alpha\).

Proof. Distinct probability measures have a measurable set \(A\) with \(P_1(A)>P_0(A)\). For sufficiently small \(\varepsilon>0\), \(\psi=\alpha+\varepsilon(\mathbf1_A-P_0(A))\) lies in \([0,1]\), has null size \(\alpha\), and has power \(\alpha+\varepsilon(P_1(A)-P_0(A))>\alpha\). The optimal power is at least this value.

1. Unbiased Tests

A level-\(\alpha\) test is unbiased if \(E_\theta\phi\ge\alpha\) at every alternative. The constant test \(\phi\equiv\alpha\) has power \(\alpha\) everywhere. Therefore a UMP test is unbiased, since it must dominate that competitor at each alternative. UMP-unbiased optimality instead compares only with other unbiased tests and can exist when unrestricted UMP optimality fails.

2. Efficiency

Power limits describe local sensitivity when their convergence assumptions hold. Convergence of statistical experiments supports limit-experiment comparisons, but it does not by itself supply convergence for every arbitrarily chosen sequence of statistics or rejection rules. We first give exact Gaussian comparisons and then state the local normal assumptions needed for a sample-size calculation.

Example 4. Let \(X\sim N_k(\theta,\Sigma)\), with known positive-definite \(\Sigma\) and nonzero vector \(c\). Put \(v=c^\top\Sigma c>0\), and \(z_p=\Phi^{-1}(p)\). For \(H_0:c^\top\theta=0\) against \(c^\top\theta>0\), the test \(c^\top X>z_{1-\alpha}\sqrt v\) is UMP, with power

\[1-\Phi\left(z_{1-\alpha}-\frac{c^\top\theta}{\sqrt v}\right).\]

To prove this, fix an alternative \(\theta_1\), let \(u=c^\top\theta_1>0\), and set \(\theta_0=\theta_1-(u/v)\Sigma c\). Then \(c^\top\theta_0=0\) and

\[\log\frac{p_{\theta_1}(X)}{p_{\theta_0}(X)}=\frac u v c^\top X-\frac{u^2}{2v}.\]

Neyman–Pearson gives the stated cutoff, which has level \(\alpha\) at every point of the null hyperplane and does not depend on the chosen alternative. The same rule is level \(\alpha\) for the larger null \(c^\top\theta\le0\).

For the two-sided alternative, no unrestricted UMP test can simultaneously equal the unique most powerful upper-tail and lower-tail tests. The symmetric rule \(\phi^*=\mathbf1\{|c^\top X|>z_{1-\alpha/2}\sqrt v\}\) is, however, UMP unbiased. Here is the additional argument.

Fix any point \(\theta_0\) on the null hyperplane and the submodel \(\theta=\theta_0+(u/v)\Sigma c\). The scalar \(Y=c^\top X/\sqrt v\) has mean \(u/\sqrt v\) and variance one. The residual \(W=X-(\Sigma c/v)c^\top X\) is independent of \(Y\), with a law independent of \(u\). Average any competing test over \(W\) to obtain a scalar test \(q(Y)\). Global unbiasedness of the original test implies that its submodel power is at least \(\alpha\) for \(u\ne0\), and at most \(\alpha\) at zero. Smoothness of the Gaussian power function gives

\[E_0q(Y)=\alpha,\qquad E_0[Yq(Y)]=0.\]

The second identity follows by differentiating at the minimum at zero; boundedness of \(q\) and Gaussian integrability justify differentiation.

For any nonzero scalar mean \(\delta\), the likelihood ratio is \(r_\delta(y)=e^{\delta y-\delta^2/2}\), a strictly convex function. Let \(a+by\) be the line through its values at \(\pm z_{1-\alpha/2}\). Convexity makes \(r_\delta-a-by\) negative inside the two cutoffs and positive outside. Therefore \((\phi^*-q)(r_\delta-a-by)\ge0\). Integration and the two moment constraints yield \(E_\delta\phi^*\ge E_\delta q\). Comparing with the constant test also proves unbiasedness of \(\phi^*\). Each full-model alternative lies on one of the submodels just considered, completing the proof.

A maximin test maximizes \(\inf_{\theta\in\Theta_1}E_\theta\phi\) subject to level \(\alpha\) on \(\Theta_0\). If power is continuous and alternative points approach a null point, a uniform power guarantee \(\beta>\alpha\) over all those alternatives is impossible: take the limit along that sequence. This is a statement about adjoining parameter sets, not the measure-theoretic notion of contiguity.

Proposition 5. Suppose the dominated densities \(p_\theta(x)\) are jointly measurable, and let \(\zeta\) and \(\xi\) be probability measures on the null and alternative parameter spaces. Form the mixture densities \(h=\int p_\theta\,d\zeta\) and \(g=\int p_\theta\,d\xi\). Let \(\phi^*\) be a most powerful level-\(\alpha\) test of \(h\) against \(g\), with power \(B\). If \(\phi^*\) is level \(\alpha\) at every original null and its minimum original alternative power equals \(B\), then it is maximin. These mixtures are least favorable: every other mixture pair has optimal simple-testing power at least \(B\).

Proof. Tonelli permits the mixture integrations. Every original level-\(\alpha\) competitor \(\psi\) is feasible under \(h\), so

\[\inf_{\theta\in\Theta_1}E_\theta\psi\le\int E_\theta\psi\,d\xi(\theta)=E_g\psi\le B.\]

The assumed minimum power of \(\phi^*\) attains this bound. For any other null mixture \(h'\) and alternative mixture \(g'\), \(\phi^*\) remains feasible under \(h'\), and \(E_{g'}\phi^*\ge\inf_{\Theta_1}E_\theta\phi^*=B\). Thus the optimal simple-testing power for \((h',g')\) is at least \(B\).

If a competing test is also maximin, both inequalities in the first display must be equalities; it is consequently also most powerful for \((h,g)\). Simple-test uniqueness gives uniqueness up to \((h+g)d\mu\)-null sets. To conclude equality almost surely under every original model distribution, that mixture measure must dominate the entire model; without this extra condition, mixture-null sets may matter elsewhere.

Example 6. For \(X\sim \mathcal{N}(\mu,1)\), test \(\mu=0\) against \(|\mu|\ge\varepsilon>0\). Mix the two alternatives \(\pm\varepsilon\) with equal weights. Their mixture likelihood ratio is \(e^{-\varepsilon^2/2}\cosh(\varepsilon X)\), strictly increasing in \(|X|\). Its most powerful test is therefore \(|X|>c\), \(c=z_{1-\alpha/2}\).

To verify the missing maximin condition, its power at \(\mu\ge0\) is \(b(\mu)=1-\Phi(c-\mu)+\Phi(-c-\mu)\). Its derivative is \(\varphi(c-\mu)-\varphi(c+\mu)\ge0\), since \(|c-\mu|\le c+\mu\). Power is symmetric in \(\mu\), so its minimum on \(|\mu|\ge\varepsilon\) is attained at \(\pm\varepsilon\) and equals its mixture power. Proposition 5 proves maximin optimality.

For a sequence of statistics, local power measures the shift in the limiting mean relative to its standard deviation. The following statement makes explicit both the correct centering and the range over which a sample-size conclusion is justified.

Theorem 7. Assume an i.i.d. model is differentiable in quadratic mean at \(\theta_0\), with finite Fisher information. Let \(m(\theta)\) have positive right derivative \(d=m'_+(\theta_0)>0\), and let \(\sigma>0\). Suppose that for every sequence \(\theta_n\ge\theta_0\) with \(\sqrt n(\theta_n-\theta_0)=O(1)\),

\[\sqrt n\{T_n-m(\theta_n)\}\Rightarrow \mathcal{N}(0,\sigma^2)\quad\text{under }P_{\theta_n}^{\otimes n}.\]

Reject when \(\sqrt n\{T_n-m(\theta_0)\}>\widehat c_n\), where \(\widehat c_n\to_{P_{\theta_0}}z_{1-\alpha}\sigma\). Then the null rejection probability tends to \(\alpha\), and for \(\sqrt n(\theta_n-\theta_0)\to h\ge0\), the limiting power is

\[b(h)=1-\Phi\left(z_{1-\alpha}-\frac{hd}\sigma\right).\]

For \(\alpha<\beta<1\), if \(\theta_k\downarrow\theta_0\), \(n_k\to\infty\), \(\sqrt{n_k}(\theta_k-\theta_0)=O(1)\), and power tends to \(\beta\), then

\[n_k\sim\frac{(z_{1-\alpha}-z_{1-\beta})^2\sigma^2}{d^2(\theta_k-\theta_0)^2}.\]

The notation \(\theta_k\downarrow\theta_0\) here can be replaced by any approach from above; monotonicity of that parameter sequence is unnecessary.

Proof. QMD gives the local log-likelihood expansion \(h\Delta_n-h^2I/2+o_P(1)\), with \(\Delta_n\Rightarrow \mathcal{N}(0,I)\). The limiting likelihood ratio has expectation one, so Le Cam's first lemma gives contiguity of each convergent local alternative sequence to the null. Thus the critical-value convergence transfers to that sequence. If \(I=0\), the likelihood ratio limit is one and the same transfer holds. The derivative gives \(\sqrt n[m(\theta_n)-m(\theta_0)]\to hd\). Therefore the centered normal variable exceeds a threshold tending to \(z_{1-\alpha}\sigma-hd\); continuity of the normal cdf proves the power formula, including \(h=0\).

For the sample-size statement, put \(h_k=\sqrt{n_k}(\theta_k-\theta_0)\). Every subsequence has a further subsequence with \(h_k\to h\), by boundedness. Along it, the just-proved power formula gives \(b(h)=\beta\). Since \(b\) is strictly increasing, its only solution is \(h_\beta=(z_{1-\alpha}-z_{1-\beta})\sigma/d>0\). Hence every convergent subsequence has the same limit, so \(h_k\to h_\beta\), and squaring yields the formula. The same proof works for sample sizes indexed by \(k\), by embedding any increasing subsequence of sample sizes in the sequential hypothesis above.

A useful extension removes the explicit local boundedness if, additionally, the power of \(\phi_n\) is nondecreasing in \(n\) for each fixed alternative and \(n_k\to\infty\). Choose \(h_-<h_\beta<h_+\), and sample sizes \(m_k^\pm\sim h_\pm^2/(\theta_k-\theta_0)^2\). Their limiting powers bracket \(\beta\). Monotonicity then forces \(m_k^-<n_k<m_k^+\) eventually. Let \(h_-\uparrow h_\beta\) and \(h_+\downarrow h_\beta\). This is the missing ordering assumption in the archive's sample-size comparison.

The QMD-to-LAN and contiguity steps used here are proved in the local asymptotic normality notes and the contiguity notes.

Local normality alone cannot justify the original claim for arbitrary nonlocal sample sizes. For example, in a normal location model let \(Z_n=\sqrt n\overline X\), and choose \(a_n\to\infty\). Use the ordinary normal statistic when \(|Z_n|\le a_n\); outside that event, use an independent coin of success probability \(\beta\) to place the statistic just above or just below the critical value. Every bounded local alternative still has the usual normal limit, because the modification has probability tending to zero there. But along alternatives with \(\sqrt n\theta_n\gg a_n\), the power tends to \(\beta\) regardless of the local sample-size formula. Taking \(a_n=n^{1/8}\), \(\theta_n=n^{-1/4}\), for example, gives \(n\theta_n^2\to\infty\). This resolves the gap by restricting the theorem or imposing the additional monotonicity, rather than assuming it from QMD.

Proposition 8. Suppose the model is QMD at an interior point \(\theta_0\), and for every fixed real \(h\),

\[\sqrt nT_n\Rightarrow \mathcal{N}(hm,\sigma^2)\quad\text{under }P_{\theta_0+h/\sqrt n}^{\otimes n},\qquad m>0,\ \sigma>0.\]

Then the preceding local-power result applies with \(m(\theta)=m(\theta-\theta_0)\) and a consistent normal critical value. The efficacy is \(m/\sigma\). For two such tests, the Pitman relative efficiency of the first relative to the second is \((m_1/\sigma_1)^2/(m_2/\sigma_2)^2\), under the sample-size conditions of Theorem 7.

Proof. One must first extend the fixed-\(h\) assertion to varying bounded local parameters. If \(h_n\to h\), QMD yields

\[\left\|\sqrt{p_{\theta_0+h_n/\sqrt n}}-\sqrt{p_{\theta_0+h/\sqrt n}}\right\|_2=o(n^{-1/2}).\]

Indeed subtract the two QMD expansions: their linear difference is \((h_n-h)\dot\ell_{\theta_0}\sqrt{p_{\theta_0}}/(2\sqrt n)\), and both remainders are \(o(n^{-1/2})\). The squared Hellinger distance between product measures is at most \(n\) times that between their one-observation laws, so their total variation distance tends to zero. Thus the fixed-\(h\) limit transfers to \(h_n\), and subtracting \(h_nm\) centers it. Every bounded sequence has convergent subsequences, proving the sequential centered normal assumption. A critical value \(z_{1-\alpha}\widehat\sigma_n\), with \(\widehat\sigma_n\to_{P_{\theta_0}}\sigma\), satisfies the remaining assumption. Finally the local power depends on \(hm/\sigma\); solving for sample size gives the squared ratio above. This efficacy calculation does not, by itself, establish unrestricted optimality among all tests.