Stochastic Convergence Exercises: Student Laws and Gaussian Directions

A sum of independent squared standard normals has mean equal to the number of terms and relative fluctuations that tend to zero. Dividing by this sum therefore has a simple limiting effect: Student variables approach a normal law, and fixed coordinates of a uniform spherical direction become Gaussian after rescaling. The exercises below prove these assertions and distinguish convergence in law from the stronger tail control needed for convergence of moments.

These are Exercises 1, 2, 4, 16, 17 from A. W. van der Vaart, Asymptotic Statistics (1998), Chapter 2, p. 24. The question numbers identify the original exercises. Write \(X_n\Rightarrow X\) for convergence in distribution and \(X_n\xrightarrow{P}X\) for convergence in probability.

For the underlying definitions, see Slutsky and tightness.

1. Student distributions approach the normal law

Question (Exercise 1). If \(X_n\) has a Student \(t\)-distribution with \(n\) degrees of freedom, show that \(X_n\Rightarrow \mathcal{N}(0,1)\).

Answer.

Let \(Z,Z_1,Z_2,\ldots\) be independent standard normal random variables, and put

\[ V_n=\sum_{j=1}^{n}Z_j^2. \]

The normal–chi-square representation of the \(t\)-distribution gives

\[ X_n\stackrel{d}{=}\frac{Z}{\sqrt{V_n/n}}. \]

Let \(\phi(x)=(2\pi)^{-1/2}e^{-x^2/2}\). Then \(\phi'(x)=-x\phi(x)\), and every polynomial times \(\phi(x)\) tends to zero as \(|x|\to\infty\). Integration by parts gives

\[ \begin{aligned} \mathbb{E} Z^2 &=\int_{\mathbb R}x^2\phi(x)\,dx =-\int_{\mathbb R}x\phi'(x)\,dx\\ &=-[x\phi(x)]_{-\infty}^{\infty} +\int_{\mathbb R}\phi(x)\,dx=1,\\ \mathbb{E} Z^4 &=-\int_{\mathbb R}x^3\phi'(x)\,dx\\ &=-[x^3\phi(x)]_{-\infty}^{\infty} +3\int_{\mathbb R}x^2\phi(x)\,dx=3. \end{aligned} \]

Consequently,

\[ \operatorname{Var}(Z^2) =\mathbb{E} Z^4-(\mathbb{E} Z^2)^2=3-1=2. \]

For independent \(Z_1,\ldots,Z_n\), therefore,

\[ \begin{aligned} \mathbb{E} V_n&=\sum_{j=1}^n\mathbb{E} Z_j^2=n,\\ \operatorname{Var}(V_n) &=\sum_{j=1}^n\operatorname{Var}(Z_j^2) +2\sum_{j<k}\operatorname{Cov}(Z_j^2,Z_k^2)\\ &=2n+0=2n,\\ \operatorname{Var}(V_n/n)&=n^{-2}(2n)=2/n. \end{aligned} \]

Hence, by Chebyshev’s inequality, for every \(\varepsilon>0\),

\[ \mathbb P\!\left(\left|\frac{V_n}{n}-1\right|>\varepsilon\right) \le \frac{2}{n\varepsilon^2}\longrightarrow0. \]

Thus \(V_n/n\xrightarrow{P}1\), and continuity of the square root gives

\[ \sqrt{V_n/n}\xrightarrow{P}1. \]

Slutsky’s lemma now yields

\[ {X_n\Rightarrow Z\sim \mathcal{N}(0,1).} \]

The preceding proof uses the ratio representation to reduce the limit to a law of large numbers. To verify the representation itself, compute the density of the ratio using the standard normal density \(\phi\) above. Suppose \(V_n\sim\chi_n^2\) is independent of \(Z\), and put \(T_n=Z/\sqrt{V_n/n}\). Conditional on \(V_n=v>0\), a change of variables gives

\[ f_{T_n\mid V_n=v}(t) =\sqrt{\frac vn}\,\phi\!\left(t\sqrt{\frac vn}\right). \]

Combining this with the chi-square density and integrating over \(v\),

\[ \begin{aligned} f_{T_n}(t) &=\int_0^\infty \sqrt{\frac vn}\frac{e^{-t^2v/(2n)}}{\sqrt{2\pi}} \frac{v^{n/2-1}e^{-v/2}}{2^{n/2}\Gamma(n/2)}\,dv\\ &=\frac{1}{\sqrt{2\pi n}\,2^{n/2}\Gamma(n/2)} \int_0^\infty v^{(n+1)/2-1} \exp\!\left[-\frac{1+t^2/n}{2}v\right]dv\\ &=\frac{\Gamma((n+1)/2)}{\sqrt{2\pi n}\,2^{n/2}\Gamma(n/2)} \left(\frac{2}{1+t^2/n}\right)^{(n+1)/2}\\ &=\frac{\Gamma((n+1)/2)}{\sqrt{n\pi}\,\Gamma(n/2)} \left(1+\frac{t^2}{n}\right)^{-(n+1)/2}. \end{aligned} \]

In the third line, the substitution

\[ u=\frac{1+t^2/n}{2}v \]

reduces the integral to the defining gamma integral. The resulting density is exactly the Student \(t_n\) density.

The ratio limit can also be proved without invoking Slutsky’s lemma. Restrict the numerator to a bounded interval and keep the denominator close to one; on their intersection, the quotient error has a deterministic bound. Let \(A_n=\sqrt{V_n/n}\). For \(v\ge0\),

\[ |\sqrt v-1|=\frac{|v-1|}{\sqrt v+1}\le |v-1|, \]

so \(A_n\xrightarrow{P}1\). Fix \(M>0\) and \(\varepsilon>0\), and set

\[ \delta=\min\left(\frac12,\frac{\varepsilon}{2M}\right). \]

On \(\{|Z|\le M,\ |A_n-1|\le\delta\}\),

\[ A_n\ge\frac12, \qquad \left|\frac Z{A_n}-Z\right| =|Z|\frac{|1-A_n|}{A_n} \le2M\delta\le\varepsilon. \]

Hence

\[ \mathbb P(|T_n-Z|>\varepsilon) \le\mathbb P(|Z|>M)+\mathbb P(|A_n-1|>\delta). \]

For fixed \(M\), the second probability tends to zero. Letting \(M\to\infty\) afterward proves \(T_n\xrightarrow{P}Z\) on this constructed probability space, and thus \(X_n\Rightarrow Z\), since \(X_n\) and \(T_n\) have identical laws for each \(n\).

2. Weak convergence and Student moments

Question (Exercise 2). Does Exercise 1 immediately imply \(\mathbb{E} X_n^p\to\mathbb{E} Z^p\) for every fixed \(p\in\mathbb N\), where \(Z\sim \mathcal{N}(0,1)\)? Is this moment convergence true?

Answer.

It does not follow from convergence in distribution alone. Nevertheless, for this \(t_n\) sequence,

\[ {\mathbb{E} X_n^p\longrightarrow\mathbb{E} Z^p \quad\text{for every fixed }p\in\mathbb N,} \]

with the moments understood for sufficiently large \(n\).

To separate weak convergence from moment convergence, add a displacement whose probability tends to zero but whose contribution to a moment does not. Let

\[ B_n\sim\operatorname{Bernoulli}(1/n), \qquad W_n=Z+nB_n. \]

For every \(\varepsilon>0\),

\[ \mathbb P(|W_n-Z|>\varepsilon) \le \mathbb P(B_n=1)=\frac1n\longrightarrow0. \]

Consequently \(W_n\Rightarrow Z\), but

\[ \mathbb{E} W_n=\mathbb{E} Z+n\mathbb{E} B_n=1 \not\longrightarrow 0=\mathbb{E} Z. \]

Weak convergence controls expectations of bounded continuous functions; \(x\mapsto x^p\) is unbounded.

Use \(X_n\stackrel d=Z/\sqrt{V_n/n}\), with \(V_n\sim\chi_n^2\) independent of \(Z\). Put \(\phi(x)=(2\pi)^{-1/2}e^{-x^2/2}\), so \(\phi^\prime(x)=-x\phi(x)\).

For every real \(p>0\), the substitution \(u=x^2/2\) gives

\[ \begin{aligned} \mathbb{E}|Z|^p &=\frac2{\sqrt{2\pi}}\int_0^\infty x^p e^{-x^2/2}\,dx\\ &=\frac2{\sqrt{2\pi}}\int_0^\infty (2u)^{p/2}e^{-u}(2u)^{-1/2}\,du\\ &=\frac{2^{p/2}}{\sqrt\pi} \int_0^\infty u^{(p+1)/2-1}e^{-u}\,du\\ &=\frac{2^{p/2}\Gamma((p+1)/2)}{\sqrt\pi}. \end{aligned} \]

For \(a>0\), substitute \(v=2u\) into the negative chi-square moment:

\[ \begin{aligned} \mathbb{E} V_n^{-a} &=\frac1{2^{n/2}\Gamma(n/2)} \int_0^\infty v^{n/2-a-1}e^{-v/2}\,dv\\ &=\frac{2^{n/2-a}}{2^{n/2}\Gamma(n/2)} \int_0^\infty u^{n/2-a-1}e^{-u}\,du\\ &=2^{-a}\frac{\Gamma(n/2-a)}{\Gamma(n/2)}, \qquad n>2a. \end{aligned} \]

The integrand is integrable at infinity because of the exponential factor. Near zero, its relevant power is \(v^{n/2-a-1}\), and

\[ \int_0^1v^{n/2-a-1}\,dv<\infty \quad\Longleftrightarrow\quad \frac n2-a>0. \]

For \(n\le2a\), the same integral is infinite; on \((0,1)\), the exponential factor is bounded below by \(e^{-1/2}\), so it cannot remove this divergence.

Independence and Tonelli’s theorem apply to the nonnegative product even when its expectation is infinite. Thus, for \(p>0\),

\[ \mathbb{E}|X_n|^p =n^{p/2}\mathbb{E}|Z|^p\,\mathbb{E} V_n^{-p/2}. \]

For \(n>p\), the complete formula is

\[ \begin{aligned} \mathbb{E}|X_n|^p &=n^{p/2} \frac{2^{p/2}\Gamma((p+1)/2)}{\sqrt\pi} 2^{-p/2}\frac{\Gamma((n-p)/2)}{\Gamma(n/2)}\\ &=\frac{n^{p/2}\Gamma((p+1)/2)\Gamma((n-p)/2)} {\sqrt\pi\,\Gamma(n/2)}. \end{aligned} \]

For \(n\le p\), the absolute moment is infinite. In particular, an odd moment in this range must not be assigned the value zero by symmetry: both its positive and negative parts have infinite expectation.

For an integer \(k\ge1\), integration by parts yields

\[ \begin{aligned} \mathbb{E} Z^{2k} &=-\int_{\mathbb R}x^{2k-1}\phi'(x)\,dx\\ &=-[x^{2k-1}\phi(x)]_{-\infty}^{\infty} +(2k-1)\int_{\mathbb R}x^{2k-2}\phi(x)\,dx\\ &=(2k-1)\mathbb{E} Z^{2k-2}\\ &=(2k-1)(2k-3)\cdots3\cdot1\cdot\mathbb{E} Z^0\\ &=(2k-1)!!. \end{aligned} \]

Repeated use of \(\Gamma(z)=(z-1)\Gamma(z-1)\) gives

\[ \begin{aligned} \Gamma(n/2) &=\left(\frac n2-1\right)\left(\frac n2-2\right) \cdots\left(\frac n2-k\right)\Gamma(n/2-k)\\ &=2^{-k}\prod_{j=1}^k(n-2j)\,\Gamma(n/2-k). \end{aligned} \]

Consequently,

\[ \begin{aligned} 2^{-k}\frac{\Gamma(n/2-k)}{\Gamma(n/2)} &=\frac1{\prod_{j=1}^k(n-2j)},\\ \mathbb{E} X_n^{2k} &=(2k-1)!!\prod_{j=1}^k\frac n{n-2j} \longrightarrow(2k-1)!!. \end{aligned} \]

The number of factors \(k\) is fixed while \(n\to\infty\). This is a fixed-order moment statement. For odd \(p<n\), absolute integrability and the equality in law \(X_n\stackrel d=-X_n\) imply

\[ \mathbb{E} X_n^p=\mathbb{E}(-X_n)^p=-\mathbb{E} X_n^p, \qquad \mathbb{E} X_n^p=0. \]

3. What the chi-square approximation says

Question (Exercise 4). In what sense is a chi-square distribution with \(n\) degrees of freedom approximately normal?

Answer.

If \(Q_n\sim\chi_n^2\), then

\[ {\frac{Q_n-n}{\sqrt{2n}}\Rightarrow \mathcal{N}(0,1).} \]

Indeed, write

\[ Q_n\stackrel{d}{=}\sum_{j=1}^{n}Z_j^2, \]

where the \(Z_j\) are independent standard normals. The variables

\[ U_j=\frac{Z_j^2-1}{\sqrt2} \]

are independent and identically distributed, with

\[ \mathbb{E} U_j=0,\qquad \operatorname{Var}(U_j)=1. \]

The central limit theorem therefore gives

\[ \frac{Q_n-n}{\sqrt{2n}} \stackrel{d}{=} \frac1{\sqrt n}\sum_{j=1}^{n}U_j \Rightarrow \mathcal{N}(0,1). \]

Thus, for every \(t\in\mathbb R\),

\[ \mathbb P(Q_n\le n+\sqrt{2n}\,t)\longrightarrow\Phi(t). \]

Pointwise convergence of distribution functions becomes uniform because the limit is continuous. A finite grid of normal quantiles controls the whole real line by monotonicity. Let

\[ H_n(t)=\mathbb P\left(\frac{Q_n-n}{\sqrt{2n}}\le t\right). \]

Fix an integer \(k\ge2\), let \(t_j=\Phi^{-1}(j/k)\) for \(1\le j\le k-1\), and set \(t_0=-\infty\), \(t_k=\infty\). Define distribution functions at these endpoints by their limits, and put

\[ \Delta_{n,k}=\max_{0\le j\le k}|H_n(t_j)-\Phi(t_j)|. \]

For fixed \(k\), all interior grid points are continuity points and the endpoint differences are zero, so \(\Delta_{n,k}\to0\). If \(t_j\le t\le t_{j+1}\), monotonicity gives

\[ \begin{aligned} H_n(t)-\Phi(t) &\le H_n(t_{j+1})-\Phi(t_j)\\ &=[H_n(t_{j+1})-\Phi(t_{j+1})] +[\Phi(t_{j+1})-\Phi(t_j)]\\ &\le\Delta_{n,k}+1/k,\\ H_n(t)-\Phi(t) &\ge H_n(t_j)-\Phi(t_{j+1})\\ &=[H_n(t_j)-\Phi(t_j)] -[\Phi(t_{j+1})-\Phi(t_j)]\\ &\ge-\Delta_{n,k}-1/k. \end{aligned} \]

It follows that

\[ \limsup_{n\to\infty}\sup_{t\in\mathbb R}|H_n(t)-\Phi(t)|\le1/k. \]

Let \(k\to\infty\). Finally, \(x\mapsto(x-n)/\sqrt{2n}\) is a bijection of \(\mathbb R\), so

\[ \sup_x\left|\mathbb P(Q_n\le x)-\Phi\left(\frac{x-n}{\sqrt{2n}}\right)\right| =\sup_t|H_n(t)-\Phi(t)|\longrightarrow0. \]

This is a precise meaning of the approximation \(\chi_n^2\approx \mathcal{N}(n,2n)\). The finite-grid proof is the argument behind Lemma 2.11 (p. 12). The characteristic-function calculation for the CLT explains where the normal limit comes from.

4. A Gaussian vector has a uniform direction

Question (Exercise 16). Let \(X_1,\ldots,X_n\) be i.i.d. standard normal, and set \(N^2=\sum_{i=1}^nX_i^2\). Show that \(U=(X_1,\ldots,X_n)/N\) is uniform on \(S^{n-1}\): for every orthogonal transformation \(O\), \(OU\) and \(U\) have the same law.

Answer.

Let

\[ G=(X_1,\ldots,X_n)^\top,\qquad N=\|G\|=\left(\sum_{i=1}^{n}X_i^2\right)^{1/2}, \qquad U=\frac GN. \]

Since \(\mathbb P(N=0)=0\), this defines \(U\) almost surely, and \(\|U\|=1\).

Normalizing removes the radius. To identify the remaining direction, first check that rotating the Gaussian vector leaves its law unchanged. The density of \(G\) is

\[ f_G(x)=(2\pi)^{-n/2}\exp\!\left(-\frac{\|x\|^2}{2}\right). \]

Let \(O\) be orthogonal. Then

\[ O^\top O=I_n,\qquad |\det O|=1,\qquad \|O^\top x\|=\|x\|. \]

By change of variables, the density of \(OG\) is

\[ f_{OG}(x) = f_G(O^\top x)|\det O^\top| = (2\pi)^{-n/2}e^{-\|x\|^2/2} =f_G(x). \]

Hence \(OG\stackrel d=G\). Also \(\|OG\|=\|G\|\), so

\[ OU = \frac{OG}{\|G\|} = \frac{OG}{\|OG\|} \stackrel d= \frac G{\|G\|} =U. \]

Thus

\[ {OU\stackrel d=U \quad\text{for every orthogonal }O,} \]

which is the required meaning of uniformity.

For \(n\ge2\), let \(S\) denote surface area on \(S^{n-1}\), and let \(\omega_{n-1}=S(S^{n-1})\). The radial integral in polar coordinates is

\[ \begin{aligned} I_n &=\int_0^\infty r^{n-1}e^{-r^2/2}\,dr\\ &=2^{n/2-1}\int_0^\infty u^{n/2-1}e^{-u}\,du =2^{n/2-1}\Gamma(n/2), \end{aligned} \]

where \(u=r^2/2\). Since the Gaussian density integrates to one,

\[ 1=(2\pi)^{-n/2}\omega_{n-1}I_n, \qquad \omega_{n-1}=\frac{(2\pi)^{n/2}}{2^{n/2-1}\Gamma(n/2)} =\frac{2\pi^{n/2}}{\Gamma(n/2)}. \]

For Borel sets \(A\subseteq S^{n-1}\) and \(C\subseteq(0,\infty)\),

\[ \begin{aligned} \mathbb P(U\in A,N\in C) &=(2\pi)^{-n/2}\int_A\int_C e^{-r^2/2}r^{n-1}\,dr\,dS(u)\\ &=\frac{S(A)}{\omega_{n-1}} \int_C\frac{r^{n-1}e^{-r^2/2}}{2^{n/2-1}\Gamma(n/2)}\,dr. \end{aligned} \]

The second factor is a probability distribution on \((0,\infty)\). Taking \(C=(0,\infty)\) yields normalized surface measure for \(U\); the factorization also proves that \(U\) and \(N\) are independent.

For \(n=1\), \(U\) equals \(1\) or \(-1\), each with probability \(1/2\).

Orthogonal invariance characterizes the uniform law, but this uniqueness also needs justification. Adding a small independent Gaussian gives a density to which polar coordinates apply. Let \(V\) be any sphere-valued random vector with \(OV\stackrel d=V\) for every orthogonal \(O\). Let \(G\sim \mathcal{N}_n(0,I_n)\) be independent of \(V\). For \(\epsilon>0\), the vector \(W_\epsilon=V+\epsilon G\) has density

\[ f_\epsilon(x) =(2\pi\epsilon^2)^{-n/2} \mathbb{E}\exp\left[-\frac{\|x-V\|^2}{2\epsilon^2}\right]. \]

For every orthogonal \(O\),

\[ \begin{aligned} f_\epsilon(Ox) &=(2\pi\epsilon^2)^{-n/2} \mathbb{E}\exp\left[-\frac{\|x-O^\top V\|^2}{2\epsilon^2}\right]\\ &=f_\epsilon(x). \end{aligned} \]

Any two vectors of equal norm are related by an orthogonal transformation, so this density depends only on \(\|x\|\). Polar coordinates then show that \(W_\epsilon/\|W_\epsilon\|\) has normalized surface law. The denominator is nonzero almost surely because \(W_\epsilon\) has a density. On the fixed joint probability space,

\[ V+\epsilon G\longrightarrow V, \qquad \|V+\epsilon G\|\longrightarrow\|V\|=1, \qquad \frac{V+\epsilon G}{\|V+\epsilon G\|}\longrightarrow V \]

almost surely as \(\epsilon\downarrow0\); the quotient is well-defined for all sufficiently small \(\epsilon\) on every such sample path. For every bounded continuous \(h\) on the sphere, dominated convergence gives

\[ \mathbb{E} h(V) =\lim_{\epsilon\downarrow0} \mathbb{E} h\left(\frac{W_\epsilon}{\|W_\epsilon\|}\right) =\frac1{\omega_{n-1}}\int_{S^{n-1}}h(u)\,dS(u). \]

This identifies the law of \(V\) with normalized surface measure. For \(n=1\), invariance under multiplication by \(-1\) directly forces equal masses on \(\{-1,1\}\). Thus the Gaussian representation used in Exercise 17 applies to every uniform spherical vector in the stated sense.

5. Two coordinates on a high-dimensional sphere

Question (Exercise 17). For \(U_n\) uniform on the unit sphere \(S^{n-1}\subset\mathbb R^n\), show that \(\sqrt n(U_{n,1},U_{n,2})\) converges in distribution to two independent standard normals.

Answer.

Let \(Z_1,Z_2,\ldots\) be independent standard normal variables. By Exercise 16, for \(n\ge2\),

\[ U_n\stackrel d= \frac{(Z_1,\ldots,Z_n)} {\left(\sum_{j=1}^{n}Z_j^2\right)^{1/2}}. \]

Therefore

\[ \sqrt n\,(U_{n,1},U_{n,2}) \stackrel d= \frac{(Z_1,Z_2)} {\left(n^{-1}\sum_{j=1}^{n}Z_j^2\right)^{1/2}}. \]

Put

\[ S_n=\frac1n\sum_{j=1}^{n}Z_j^2. \]

Since

\[ \mathbb{E} S_n=1,\qquad \operatorname{Var}(S_n)=\frac2n, \]

Chebyshev’s inequality gives \(S_n\xrightarrow{P}1\), and hence

\[ S_n^{-1/2}\xrightarrow{P}1. \]

The numerator has the \(\mathcal{N}_2(0,I_2)\) distribution for every \(n\). Slutsky’s lemma yields

\[ { \sqrt n\,(U_{n,1},U_{n,2}) \Rightarrow(Z_1,Z_2)\sim \mathcal{N}_2(0,I_2). } \]

The limiting coordinates are independent by the construction of \(Z_1,Z_2\). Slutsky’s lemma does not require the numerator to be independent of \(S_n\).

The denominator contains the same coordinates as the numerator, so their dependence must be allowed in the error estimate. A union bound suffices. Write \(R_n=S_n^{-1/2}\) and \(Z^{(2)}=(Z_1,Z_2)\). On \(\{|S_n-1|\le1/2\}\),

\[ \begin{aligned} |R_n-1| &=\left|\frac1{\sqrt{S_n}}-1\right|\\ &=\frac{|1-\sqrt{S_n}|}{\sqrt{S_n}}\\ &=\frac{|1-S_n|}{\sqrt{S_n}(1+\sqrt{S_n})}\\ &\le\sqrt2\,|1-S_n|. \end{aligned} \]

For any \(\epsilon>0\), set \(\delta=\min(1/2,\epsilon/\sqrt2)\). Then

\[ \mathbb P(|R_n-1|>\epsilon) \le\mathbb P(|S_n-1|>\delta) \le\frac2{n\delta^2}\longrightarrow0. \]

Although \(R_n\) depends on \(Z_1,Z_2\), for every \(M>0\) and \(\varepsilon>0\),

\[ \begin{aligned} \mathbb P(\|R_nZ^{(2)}-Z^{(2)}\|_2>\varepsilon) &=\mathbb P(|R_n-1|\,\|Z^{(2)}\|_2>\varepsilon)\\ &\le\mathbb P(\|Z^{(2)}\|_2>M) +\mathbb P(|R_n-1|>\varepsilon/M)\\ &\le\frac{\mathbb{E}(Z_1^2+Z_2^2)}{M^2} +\mathbb P(|R_n-1|>\varepsilon/M)\\ &=\frac2{M^2}+\mathbb P(|R_n-1|>\varepsilon/M). \end{aligned} \]

Let \(n\to\infty\) with \(M\) fixed and then let \(M\to\infty\). This proves convergence in probability of the constructed scaled vectors to \(Z^{(2)}\), and hence the required convergence in law for the original \(U_n\).

The joint characteristic function of the limit is

\[ \mathbb{E} e^{i(sZ_1+tZ_2)} =\mathbb{E} e^{isZ_1}\,\mathbb{E} e^{itZ_2} =e^{-s^2/2}e^{-t^2/2} =e^{-(s^2+t^2)/2}. \]

This factorization identifies both the standard normal marginals and their independence.

Source: A. W. van der Vaart, Asymptotic Statistics, Chapter 2, “Stochastic Convergence”, pp. 5–24. The questions above are stated in the notation used in these solutions.