Ranks discard the numerical spacing between observations while retaining their order. Under a continuous common distribution, their permutation law is known, which makes them useful for testing. To study power and large-sample behavior, we approximate a rank statistic by a sum of independent terms through its projection. The choice of score then determines which alternatives the test is sensitive to.
Let \(X_1,\ldots,X_n\) be i.i.d. A projection replaces a statistic by its first-order conditional-expectation terms; for a U-statistic, these terms are obtained from its symmetric kernel. The projection identities fix the \(L^2\) interpretation.
Let \(X_{N(1)}<\cdots<X_{N(N)}\) be the order statistics of an i.i.d. real sample from a continuous cdf \(F\). There are no ties almost surely. The rank \(R_{Ni}\) is determined by \(X_i=X_{N(R_{Ni})}\). Continuity suffices for the rank laws below; a density is needed only for the density formulas.
Definition 1. A rank statistic is any measurable function of the vector \(R_N=(R_{N1},\ldots,R_{NN})\).
A general linear rank statistic is \(\sum_i a_N(i,R_{Ni})\), for an array \(a_N\). We focus on simple linear rank statistics, whose array factors into deterministic coefficients and scores:
\[T_N=\sum_{i=1}^NC_{Ni}a_{N,R_{Ni}}.\]
Write \(\overline C_N=N^{-1}\sum_iC_{Ni}\) and \(\overline a_N=N^{-1}\sum_ka_{Nk}\).
Lemma 2. The rank vector is uniform on the \(N!\) permutations of \(1,\ldots,N\), and is independent of the ordered sample. If \(F\) has density \(f\), the ordered sample has joint density \(N!\prod_if(x_i)\) on \(x_1<\cdots<x_N\), and its \(k\)th coordinate has density
Proof. Exchangeability assigns the same probability to each ordering even after restricting the ordered values to any measurable set. The ordering events partition the sample space, so each has conditional probability \(1/N!\). This proves uniformity, independence, and the conditional identity. With a density, summing the \(N!\) permuted joint densities gives the ordered joint density. For the marginal, choose which observation is at \(x\), which \(k-1\) lie below it, and integrate the remaining coordinates above it. For uniforms, integrating powers against this beta density yields \(EU_{N(k)}=k/(N+1)\) and \(EU_{N(k)}^2=k(k+1)/[(N+1)(N+2)]\), giving the variance.
Put \(v_a=N^{-1}\sum_k(a_{Nk}-\overline a_N)^2\). A single random score has variance \(v_a\). Two scores at distinct positions have covariance \(-v_a/(N-1)\), since they are drawn without replacement, or because their total is constant. Expanding the variance of \(\sum_i(C_{Ni}-\overline C_N)a_{N,R_{Ni}}\) and using \(\sum_i(C_{Ni}-\overline C_N)=0\) gives the stated formula. ◻
Example 3. Pool independent samples \(X_1,\ldots,X_m,Y_1,\ldots,Y_n\), with \(N=m+n\). Under equality of their continuous distributions the pooled ranks have the preceding permutation law. Large values of the Wilcoxon rank sum \(W=\sum_{j=1}^nR_{N,Y_j}\) provide evidence that the second distribution is stochastically larger. Stochastic ordering does not require every second-sample rank to exceed every first-sample rank. Without ties,
Here \(U_{N(k)}\) is the \(k\)th order statistic of \(N\) independent uniforms, not a single uniform variable. Local optimality requires a particular score function determined by the alternative model; arbitrary exact scores are not automatically optimal.
Theorem 4. Suppose \(\varphi\in L^2(0,1)\) is not constant almost everywhere, and \(A_N=\sum_i(C_{Ni}-\overline C_N)^2>0\). For either score sequence \(d_{Nk}\), define
Proof for exact scores. Set \(U_i=F(X_i)\); these are independent uniforms and their ranks agree with those of \(X_i\) almost surely. The preceding conditional identity gives \(a_{N,R_{Ni}}=E[\varphi(U_i)\mid R_N]\), whence \(T_N=E[\widetilde T_N\mid R_N]\). This already proves equality of means.
The sigma-fields \(\mathcal G_N=\sigma(R_N)\) increase: deleting the last label from a rank vector recovers the earlier vector. Also,
For each fixed value \(u\) of \(U_1\), the ordinary strong law gives this limit with probability one; integrating over \(u\) justifies the assertion jointly. Thus \(U_1\), and hence \(\varphi(U_1)\), is measurable in \(\mathcal G_\infty\). The \(L^2\) martingale convergence theorem yields
The limit is the variable \(\varphi(U_1)\) itself, not its expectation. In particular \(\operatorname{Var}(a_{N,R_{N1}})\to\sigma_\varphi^2=\operatorname{Var}\varphi(U_1)>0\). The exact variance formula gives
Since \(T_N\) is a projection of \(\widetilde T_N\), its mean square error is the difference of these variances. This proves the first assertion. ◻
Proof for grid scores. The rank convergence, almost-everywhere continuity, and the assumed second moments imply \(b_{N,R_{N1}}\to\varphi(U_1)\) in \(L^2\). Explicitly, convergence in probability together with convergence of the squared expectations gives uniform integrability of the squares; then \(|b_{N,R_{N1}}-\varphi(U_1)|^2\le2b_{N,R_{N1}}^2+2\varphi(U_1)^2\) and truncation give convergence of the mean square error. Combining this with the exact-score limit gives
Apply Lemma 2 to the centered difference of the two rank statistics. Its variance is at most \(A_N(N-1)^{-1}\sum_k(b_{Nk}-a_{Nk})^2=o(A_N)\), while either statistic has variance asymptotic to \(A_N\sigma_\varphi^2\). Their centered independent approximations coincide. The triangle inequality in \(L^2\) therefore proves the assertion for grid scores, including the respective deterministic means. ◻
Remark 5. The approximation theorem alone does not give a normal limit. If additionally
then \((T_N-ET_N)/\sqrt{\operatorname{Var}T_N}\Rightarrow \mathcal{N}(0,1)\). To verify Lindeberg, put \(d_{Ni}=(C_{Ni}-\overline C_N)/\sqrt{A_N}\) and \(Z_i=(\varphi(U_i)-E\varphi(U_i))/\sigma_\varphi\). Since \(\sum_i d_{Ni}^2=1\), the Lindeberg sum is bounded by
Indeed each sampled value is bounded by the average over the interval immediately to its right. Monotone functions are continuous almost everywhere; Fatou applied to \(\varphi^2(R_{N1}/(N+1))\) gives the matching lower limit. For a nonnegative nonincreasing function, use intervals to the left. A signed monotone function splits into its positive and negative parts; apply these bounds to their squares separately. Thus any monotone square-integrable score satisfies the grid condition. Bounded-variation functions on \([0,1]\) also satisfy it, since their squares are bounded and Riemann integrable. Nonnegativity alone would not justify the displayed inequality.
Example 7. For two samples, take \(C_{Ni}=0\) on the first \(m\) observations and \(C_{Ni}=1\) on the last \(n\). Then \(\overline C_N=n/N\) and
\[A_N=m(n/N)^2+n(m/N)^2=\frac{mn}{N}.\]
If \(m/N\to\lambda\in(0,1)\), the maximal squared centered coefficient divided by \(A_N\) is \(O(N^{-1})\), proving the CLT condition.
Example 8. For \(\varphi(u)=u\), exact and grid scores coincide: \(a_{Nk}=k/(N+1)\). The normalized Wilcoxon rank sum \(W_N=\sum_{j=1}^nR_{N,Y_j}/(N+1)\), used to test equality of two distributions, has independent approximation
For example, \(\sum_k(k/(N+1)-1/2)^2=N(N-1)/[12(N+1)]\); substituting in Lemma 2 yields the first variance. Independence of the uniform transforms gives the second. This is a two-sample distribution test, not a test of independence between paired coordinates.
1. General Prescription for Constructing Rank Statistics
For fixed \(N\), suppose independent observations have densities \(f_{C_{Ni}\theta}\), with common density \(f_0\) under the null. Assume the model is differentiable in \(L^1\) at zero:
The integral of the derivative is zero, so \(E_0\ell_0=0\). Telescoping the finite product of densities gives its \(L^1\) derivative \(\prod_i f_0(x_i)\sum_i C_{Ni}\ell_0(x_i)\). Integrating this derivative on the event \(R_N=r\) yields
The remainder is uniform over the finitely many rank vectors. Thus the derivative of the power of a rank test \(\psi(R_N)\) at zero is \(E_0[\psi(R_N)S(R_N)]\), where
Among tests of a given null rejection probability, rejecting at large \(S\), with boundary randomization if needed, maximizes this linear functional: integrate \((\psi^*-\psi)(S-c)\ge0\). This is local most powerfulness for positive \(\theta\), not exact optimality at every fixed alternative. With the generalized quantile \(F_0^{-1}\), the generating function is \(\varphi=\ell_0\circ F_0^{-1}\).
Example 9. For the uncensored proportional-hazards family \(1-F_\theta=(1-F_0)^\theta\), the null parameter is \(\theta=1\). Its cumulative hazard is \(\Lambda_\theta=\theta\Lambda_0\), and
Hence the local score is \(\varphi(u)=1+\log(1-u)\). Its integral is zero and its square integral is finite. The sign in front of the logarithm matters: increasing the hazard makes lifetimes smaller.
For a derivation, \(-\log(1-U_i)\) are independent rate-one exponentials. Their first minimum has mean \(1/N\); by memorylessness each subsequent spacing has mean \(1/(N-1),1/(N-2),\ldots\). Summing the first \(k\) spacings gives the formula. The exact likelihood scores are \(1-s_{Nk}\). Adding a constant to all scores changes a linear rank statistic by a deterministic constant; negating them reverses the rejection direction. These are the complete-data scores associated with the log-rank construction; a censored-data test additionally requires the censoring model and risk sets.
2. Constructing Signed Rank Statistics
Write \(R_{Ni}^+\) for the rank of \(|X_i|\) and \(\epsilon_i=\operatorname{sgn}(X_i)\). A signed-rank statistic is \(T_N=\sum_i a_{N,R_{Ni}^+}\epsilon_i\).
Lemma 10. Under i.i.d. sampling from a continuous distribution symmetric about zero, the sign vector consists of independent uniform signs and is independent of the absolute sample and its ranks. The absolute-rank vector is uniform on the \(N!\) permutations, and
Proof. For any measurable \(B\subset(0,\infty)\), symmetry gives \(P(|X|\in B,X>0)=P(|X|\in B,X<0)=P(|X|\in B)/2\). There is no mass at zero. This proves independence of sign and magnitude; independence across observations gives the vector statement. The magnitude distribution is continuous, so Lemma 2 gives the rank permutation law. Conditional on the magnitudes, the signs are independent, centered, and have variance one. Thus the conditional variance is \(\sum_i a_{N,R_{Ni}^+}^2=\sum_ka_{Nk}^2\), a deterministic quantity. ◻
Theorem 11. Let \(F^+\) be the cdf of \(|X|\) under the continuous symmetric null, and \(\varphi\in L^2(0,1)\). For exact scores \(a_{Nk}=E\varphi(U_{N(k)})\), define
The same holds for grid scores if \(\varphi\) is continuous almost everywhere and satisfies the grid second-moment condition in Theorem 4.
Proof. The score errors are functions of the magnitudes. Conditioning on them, all cross terms involving distinct signs vanish. Exchangeability then gives
by the scalar score convergence proved in Theorem 4; its argument does not require the score to be nonconstant. The summands of \(\widetilde T_N\) are i.i.d., centered, with variance \(\int\varphi^2\). The ordinary CLT and the mean-square approximation prove the assertion. The same scalar convergence for grid scores proves their version. ◻
Example 12. For \(\varphi(u)=u\), let \(W_N=\sum_iR_{Ni}^+\epsilon_i\). Since the theorem applies to \(W_N/(N+1)\),
This centered signed version equals \(2W^+-N(N+1)/2\), where \(W^+=\sum_iR_{Ni}^+\mathbf1\{X_i>0\}\). The two conventions therefore give equivalent tests after their respective centering and scaling.
For local power, take the location model \(f(x-\theta)\), with a positive symmetric density \(f\) that is absolutely continuous and has integrable derivative. Translation is differentiable in \(L^1\): \([f(\cdot-\theta)-f]/\theta\to-f'\), which follows by integrating translates of \(f'\) and their \(L^1\) continuity. Thus the preceding derivative argument applies. Symmetry makes \(f'/f\) odd, so the full-sample score is
The independence in Lemma 10 gives the factor \(1/(2^NN!)\). These scores maximize the derivative of power among signed-rank tests of a fixed size. If one also wants the square-integrable score CLT, assume finite location Fisher information \(\int(f'/f)^2f<\infty\). This derivative calculation concerns local alternatives; it is not a full finite-alternative power formula.
Example 13. For i.i.d. paired observations \((X_i,Y_i)\) with continuous marginal distributions, let \(S_i\) and \(R_i\) be their respective ranks. Each rank vector is marginally a uniform permutation, but the two vectors need not be independent. The empirical rank means are both \((N+1)/2\), and
To check the constant, expanding the centered numerator subtracts exactly \(N(N+1)^2/4\), rather than adding the two larger terms that appear in the archival calculation.
Under independence of \(X\) and \(Y\), condition on \(S\). The remaining ranks are uniform, so Lemma 2 gives \(E\rho_N=0\) and \(\operatorname{Var}\rho_N=1/(N-1)\). Conditional on \(S\), use scores \(k/(N+1)\) and coefficients \(S_i\) in Theorem 4 and Remark 5. Their maximal centered square divided by their sum is \(O(N^{-1})\). The conditional law does not depend on the particular permutation \(S\), so the resulting CLT is also unconditional: \(\sqrt N\rho_N\Rightarrow \mathcal{N}(0,1)\).
A smooth-score approximation also works for independent observations with different continuous cdfs \(F_1,\ldots,F_N\). Define
The integrals are Stieltjes integrals against a finite signed measure. The approximation is a sum of independent single-observation functions; its expectation need not equal \(ET_N\).
Lemma 14. For \(\varphi\in C^2[0,1]\), a universal constant \(K\) satisfies
Proof. Suppress the \(N\) subscripts, and replace \(C_i\) by \(c_i=C_i-\overline C\). This changes \(T\) by a constant. It also changes \(\widetilde T\) by a constant, since continuity of \(\overline F\) gives \(\int_x^\infty\varphi'(\overline F(t))\,d\overline F(t)=\varphi(1)-\varphi(\overline F(x))\). Thus we may assume \(\sum_i c_i=0\).
Set \(\delta_i=R_i/(N+1)-\overline F(X_i)\). The rank identity and independence give
Conditional on \(X_i\), the remaining part of \((N+1)\delta_i\) is a sum of \(N-1\) independent centered variables bounded by one. Expanding its fourth power leaves only single fourth moments and paired second moments, bounded by \(N+3N^2\). Including the bounded bias therefore gives \(E\delta_i^4\le K/N^2\).
Taylor expansion yields \(T=T_0+T_1+T_2\), where \(T_0=\sum_i c_i\varphi(\overline F(X_i))\), \(T_1=\sum_i c_i\varphi'(\overline F(X_i))\delta_i\), and
Write \(d_i=\varphi'(\overline F(X_i))\). The bias part \(\sum_i c_i d_i b_i\) of \(T_1\) has second moment at most \(\|\varphi'\|_\infty^2\sum_i c_i^2/N\), by Cauchy–Schwarz. The remaining term is
Let \(A_i(x)=\int_x^\infty\varphi'(\overline F(t))\,dF_i(t)\), so \(|A_i|\le M=\|\varphi'\|_\infty\). Conditional expectation on \(X_i\) makes the corresponding summand zero; conditioning on \(X_j\) gives its other first-order projection. Hence
Subtracting \(\widehat Q\) changes the denominator and adds diagonal terms. Independence across \(j\), \(|A_i-EA_i|\le2M\), and \((\sum_i|c_i|)^2\le N\sum_i c_i^2\) bound the variance of this difference by \(KM^2\sum_i c_i^2/N^2\).
Finally, \(Q\) consists only of two-coordinate terms. Its remainder after the Hájek projection is the orthogonal sum of its canonical pair components. For the pair \(\{i,j\}\), the projection norm is at most the norm of
Combining the four bounds, using \(\operatorname{Var}(\sum_{j=1}^4Z_j)\le4\sum_j\operatorname{Var}Z_j\), proves the lemma. ◻
To use it for a CLT, the error bound must be small relative to the relevant variance, and the independent approximation must satisfy an appropriate CLT condition. Smoothness alone does not remove the need for Lindeberg or another sufficient normality condition.