Weak Convergence, Tightness, and Metric Spaces

For a stochastic process, convergence of every fixed finite collection of coordinates does not by itself control the whole path. We also need to rule out oscillations that escape those coordinates. The partition criterion below supplies this control and constructs a tight limiting random element. The convergence conventions distinguish almost-sure, probability, and distributional convergence.

A stochastic process is a collection of time-indexed random variables \(X_t : \Omega \to \mathop{\mathrm{\mathbb{R}}}\), denoted \(X = \left\{ X_t : \: t \in T \right\}\). These are all defined on an ambient probability space \((\Omega, \mathop{\mathrm{\mathfrak{F}}}, \mathop{\mathrm{\mathds{P}}})\).

When its sample paths are bounded, \(X\) can be viewed as a map into the space of bounded functions, i.e.

\[X: \Omega \to \ell^{\infty} (T)\]

and the weak convergence of maps with values in \(\mathop{\mathrm{\mathscr{C}}}[a,b]\) relative to \(\mathop{\mathrm{\mathscr{C}}}[a,b]\) can also be thought of being relative to \(D[a,b]\) or \(\ell^{\infty}[a,b]\) (if we endow the sup norm on all of these functional spaces).

We assume each coordinate \(X_{n,t}\) is measurable, but the map into \(\ell^\infty(T)\) need not be Borel measurable. For a bounded function \(Z\) on the underlying probability space, \(\mathbb E^*Z\) is the infimum of expectations of measurable majorants, and \(\mathbb E_*Z=-\mathbb E^*(-Z)\); outer and inner probabilities use indicator functions. Here \(X_n\Rightarrow X\) means \(\mathbb E^*f(X_n)\to\mathbb Ef(X)\) for every bounded continuous real \(f\), where \(X\) is a Borel measurable random element with a tight law. Applying this definition also to \(-f\) gives the matching inner-expectation convergence. If the maps are Borel measurable, these are the usual weak-convergence conventions.

Theorem 1. \({X_n : \Omega_n \to \ell^{\infty} (T)}\xrightarrow{\mathrm{d}}\text{tight random element}\) if and only if the following are true:

for all \(t_1,\dots,t_k\in T\),

\[{\left( X_{n,t_1}, \dots, X_{n,t_k} \right)}\xrightarrow{\mathrm{d}}Y\]

where \(Y \in \mathop{\mathrm{\mathbb{R}}}^k\).
for all \(\varepsilon, \eta > 0\), there exists a partition of \(T\) into \(T_1,\dots,T_k\) such that

\[\limsup_{n\to\infty} \mathbb{P}^* \left( \sup_{i} \sup_{s,t \in T_i} \abs{X_{n,s} - X_{n,t}} \geqslant\varepsilon\right) \leqslant\eta\]

Proof. \((\Leftarrow)\) The finite partitions will serve two purposes: they give finite-dimensional approximations and define a metric in which limiting paths are uniformly continuous. Set \(\varepsilon_m=\eta_m=2^{-m}\), so that the error probabilities are summable. Choose the finite partitions in (ii) successively, replacing each new partition by its common refinement with the earlier ones. Refining a partition can only reduce the oscillation within its cells. Write the resulting nonempty cells as \(T_1^m,\ldots,T_{k_m}^m\).

Fix \(m\). Then, define the semimetric (pseudometric)

\[\rho_m(s,t)= \begin{cases} 0,&\text{if }s,t\in T_j^m\text{ for some }1\le j\le k_m,\\ 1,&\text{otherwise}. \end{cases}\]

The zero case means that the two points lie in the same cell, whichever cell that is. In particular, \(\rho_m(t,t)=0\). If \(s\) and \(t\) are in different cells, any third point must be in a different cell from at least one of them, which proves the triangle inequality. Every \(\rho_m\)-ball of radius in \((0,1)\) is a partition cell, and every cell has \(\rho_m\)-diameter zero.

Since the partitions are nested, \(\rho_m\le\rho_{m+1}\). Define

\[\rho(s,t)=\sum_{m=1}^{\infty}2^{-m}\rho_m(s,t).\]

For \(s,t\in T_j^m\), the first \(m\) terms vanish, so

\[\rho(s,t)\le\sum_{r>m}2^{-r}=2^{-m}.\]

Thus \((T,\rho)\) is totally bounded. Pick one representative \(t_j^m\) from every cell and put \(T_0=\{t_j^m:m\ge1,\ 1\le j\le k_m\}\). This is a countable dense set. In the other direction,

\[\rho(s,t)<2^{-m} \quad\Longrightarrow\quad 2^{-m}\rho_m(s,t)<2^{-m} \quad\Longrightarrow\quad \rho_m(s,t)=0.\]

The finite-dimensional limits in (i) are consistent, because taking a coordinate subvector is continuous. Kolmogorov’s extension theorem therefore gives a process \(X^\circ\) on \(T_0\) with those limiting distributions. The extension theorem supplies the coordinate laws, but does not yet give the path regularity needed for a random element of the function space. We next show that its paths are uniformly \(\rho\)-continuous.

For a finite set \(S\subset T_0\), define

\[w_{m,S}(x)= \max_{1\le j\le k_m}\ \max_{s,t\in S\cap T_j^m}|x_s-x_t|,\]

where an empty maximum is zero. This is a continuous function of finitely many coordinates. Portmanteau applied to the open event \(\{w_{m,S}>2^{-m}\}\) gives

\[\begin{aligned} \mathbb P\bigl(w_{m,S}(X^\circ)>2^{-m}\bigr) &\le\liminf_{n\to\infty} \mathbb P\bigl(w_{m,S}(X_n)>2^{-m}\bigr)\\ &\le\limsup_{n\to\infty} \mathbb P^*\left( \max_j\sup_{s,t\in T_j^m}|X_{n,s}-X_{n,t}|\ge2^{-m} \right)\\ &\le2^{-m}. \end{aligned}\]

Exhaust \(T_0\) by increasing finite sets \(S\). Continuity of probability for increasing events yields the same bound for

\[w_m(X^\circ)=\max_j\sup_{s,t\in T_0\cap T_j^m} |X^\circ_s-X^\circ_t|.\]

By Borel–Cantelli, almost surely \(w_m(X^\circ)\le2^{-m}\) for all sufficiently large \(m\). Together with the implication above, this proves uniform \(\rho\)-continuity on \(T_0\). The paths are also bounded: for one such \(m\), their absolute values are bounded by the largest absolute value at the finitely many representatives plus \(2^{-m}\).

Identify points at \(\rho\)-distance zero and complete the resulting metric space to a compact space \(K\). Each uniformly continuous path on \(T_0\) extends uniquely to \(K\); restrict that extension to \(T\) and call it \(X\). On the exceptional null event, take the zero path.

Define \(\pi_m(t)=t_j^m\) when \(t\in T_j^m\), and put \(Y_m=X\circ\pi_m\). Uniform continuity and \(\rho(t,\pi_m(t))\le2^{-m}\) imply

\[\|Y_m-X\|_T\longrightarrow0 \qquad\text{almost surely}.\]

Each \(Y_m\) is a measurable finite-dimensional random element of \(\ell^\infty(T)\). Its paths are uniformly \(\rho\)-continuous, since \(\rho(s,t)<2^{-m}\) forces \(\pi_m(s)=\pi_m(t)\). Thus \(X\) is a measurable random element of the space of uniformly \(\rho\)-continuous functions, which is isometric to the separable Banach space \(C(K)\).

Here tightness can be checked directly. Fix \(\eta>0\). Since \(\|Y_m-X\|_T\to0\) almost surely, choose increasing \(m_j\) with \(\mathbb P(\|Y_{m_j}-X\|_T>2^{-j})<\eta2^{-j-2}\). The range of \(Y_{m_j}\) is finite dimensional; restricting its finitely many coordinate values to a sufficiently large closed box produces a compact set \(K_j\) with \(\mathbb P(Y_{m_j}\notin K_j)<\eta2^{-j-2}\). The closed set

\[A=\bigcap_{j\geq1}\{x\in\ell^\infty(T):\operatorname{dist}(x,K_j)\leq2^{-j}\}\]

is totally bounded: for any radius, choose large \(j\) and a finite net of \(K_j\). It is closed in the complete space \(\ell^\infty(T)\), hence compact. A union bound gives \(\mathbb P(X\in A)\geq1-\eta\). This proves tightness without assuming that arbitrary laws on a nonseparable function space are tight.

For fixed \(m\), condition (i) and the continuous map from the vector of cell values to its piecewise constant path give

\[X_n\circ\pi_m\Rightarrow Y_m.\]

Also,

\[\|X_n-X_n\circ\pi_m\|_T \le\max_j\sup_{s,t\in T_j^m}|X_{n,s}-X_{n,t}|,\]

so the approximation error is controlled by (ii). For a \(1\)-Lipschitz function \(f:\ell^\infty(T)\to[0,1]\),

\[\begin{aligned} |\mathbb{E}^*f(X_n)-\mathbb{E}f(X)| &\le2^{-m} +\mathbb P^*(\|X_n-X_n\circ\pi_m\|_T>2^{-m})\\ &\quad+|\mathbb{E}f(X_n\circ\pi_m)-\mathbb{E}f(Y_m)|\\ &\quad+\mathbb{E}\min(1,\|Y_m-X\|_T). \end{aligned}\]

The order of limits matters: the finite-dimensional approximation is fixed while the original sequence converges. First let \(n\to\infty\) with \(m\) fixed. The middle expectation difference vanishes, and the first two terms have limit superior at most \(2^{1-m}\). Then let \(m\to\infty\) and apply dominated convergence to the last term. The bounded-Lipschitz argument detailed below proves \(X_n\Rightarrow X\).

\((\Rightarrow)\) Finite-dimensional convergence follows from the continuity of coordinate evaluation. For (ii), fix \(\varepsilon,\eta>0\) and choose a compact set \(A\subset\ell^\infty(T)\) with \(\mathbb P(X\in A)>1-\eta\). Cover \(A\) by finitely many sup-norm balls of radius \(\varepsilon/8\), centered at bounded functions \(f_1,\ldots,f_r\).

Partition the ranges of these finitely many functions into intervals of length \(\varepsilon/4\), and partition \(T\) by the resulting vectors of interval indices. There are only finitely many cells. Within each cell, every \(f_j\) has oscillation at most \(\varepsilon/4\), so every \(x\in A\) has oscillation strictly less than \(\varepsilon/2\).

The open neighborhood \(G=\{x:\operatorname{dist}(x,A)<\varepsilon/4\}\) therefore consists of functions whose oscillation in every cell is strictly less than \(\varepsilon\). Portmanteau, with inner probability \(\mathbb P_*\), gives

\[\liminf_n\mathbb P_*(X_n\in G)\ge\mathbb P(X\in G)>1-\eta.\]

The event of cell oscillation at least \(\varepsilon\) is contained in \(\{X_n\notin G\}\). Taking complements proves the outer-probability bound in (ii). ◻

Why bounded Lipschitz tests suffice here. The preceding estimate applies to any bounded Lipschitz test by rescaling. Let \(0\leq f\leq1\) be continuous and let the limit law put mass at least \(1-\eta\) on a compact \(K\). Given \(\varepsilon>0\), choose \(L<\infty\) so that the Lipschitz function

\[g(x)=\min\left(1,\inf_{y\in K}\bigl(f(y)+L\|x-y\|_T\bigr)\right)\]

satisfies \(|g-f|\leq\varepsilon\) on \(K\). To see this, uniform continuity of \(f|_K\) controls nearby \(y\), while choosing \(L\delta\geq1\) makes distant \(y\) irrelevant to the infimum. Continuity of \(f-g\) and compactness give some \(\delta>0\) such that \(|f-g|<2\varepsilon\) whenever \(\operatorname{dist}(x,K)<\delta\). The bounded Lipschitz function \(h(x)=\min(1,\operatorname{dist}(x,K)/\delta)\) then yields

\[\limsup_n\mathbb P^*(\operatorname{dist}(X_n,K)\geq\delta) \leq\lim_n\mathbb E^*h(X_n)=\mathbb Eh(X)\leq\eta.\]

Using measurable majorants for the exceptional event, monotonicity and subadditivity of outer expectation give

\[\limsup_n|\mathbb E^*f(X_n)-\mathbb Ef(X)|\leq4\varepsilon+2\eta.\]

In this bound the expectation difference for \(g\) tends to zero by the bounded Lipschitz result, and the two errors for \(f-g\) are bounded by \(2\varepsilon\) plus the corresponding exceptional probability. Let \(\varepsilon,\eta\downarrow0\), and rescale an arbitrary bounded continuous function. This supplies the weak-convergence step used above, including for nonmeasurable maps. ◻

1. Ancillary Facts

Proposition 2. Let \(T_n\) be a sequence of random variables distributed following \(t\)-distribution with \(n\) degrees of freedom. Then, \({T_n}\xrightarrow{\mathrm{d}}\mathcal{N}(0,1)\) as \(n \to \infty\).

Proof. Method 1. Represent a Student variable as a normal variable divided by an independent estimate of its scale. The law of large numbers makes the denominator approach one. Let \(S,S_1,S_2,\ldots\) be independent standard normal variables, and write

\[Y_n=\sum_{j=1}^nS_j^2\sim\chi_n^2, \qquad T_n\stackrel d=\frac{S}{\sqrt{Y_n/n}}.\]

Since \(\mathbb{E}S_j^2=1\) and \(\operatorname{Var}(S_j^2)=2\),

\[\mathbb{E}(Y_n/n)=1,\qquad \operatorname{Var}(Y_n/n)=2/n.\]

Chebyshev’s inequality gives \(Y_n/n\to_P1\). The square root is continuous at \(1\), and Slutsky’s lemma gives \(T_n\Rightarrow \mathcal{N}(0,1)\).

Method 2. We can also compare densities directly. Pointwise convergence alone is insufficient, so after obtaining it we use the fact that both densities integrate to one. The density is

\[f_n(t)=\frac{\Gamma((n+1)/2)}{\sqrt{n\pi}\,\Gamma(n/2)} \left(1+\frac{t^2}{n}\right)^{-(n+1)/2}.\]

Stirling’s formula implies \(\Gamma(z+1/2)/\Gamma(z)\sim z^{1/2}\). With \(z=n/2\), the leading factor tends to \(1/\sqrt{2\pi}\). For fixed \(t\),

\[-\frac{n+1}{2}\log\left(1+\frac{t^2}{n}\right) =-\frac{n+1}{2}\left(\frac{t^2}{n}+O(n^{-2})\right) \longrightarrow-\frac{t^2}{2}.\]

Thus \(f_n(t)\to\phi(t)\), the standard normal density. To pass from densities to distributions, use \(0\le\min(f_n,\phi)\le\phi\) and dominated convergence:

\[\int_{\mathbb R}|f_n-\phi| =2-2\int_{\mathbb R}\min(f_n,\phi)\longrightarrow0.\]

This proves convergence in total variation, hence weak convergence. ◻

Remark 3. Weak convergence by itself does not imply convergence of moments. For the Student variables above, however, the absolute moment is

\[\mathbb{E}|T_n|^p =\frac{n^{p/2}\Gamma((p+1)/2)\Gamma((n-p)/2)} {\sqrt{\pi}\,\Gamma(n/2)},\qquad 0<p<n.\]

This follows by writing \(T_n=S/\sqrt{Y_n/n}\) and evaluating the normal absolute moment and negative chi-square moment. It is positive for both even and odd \(p\); the signed odd moments, when finite, vanish by symmetry. For \(p\ge n\), the absolute moment is infinite.

For completeness, independence in the normal/chi-square representation gives \(\mathbb E|T_n|^p=n^{p/2}\mathbb E|S|^p\mathbb EY_n^{-p/2}\), by Tonelli, with an infinite value allowed. The substitutions \(u=s^2/2\) in the normal integral and \(v=y/2\) in the chi-square integral yield

\[\mathbb E|S|^p=\frac{2^{p/2}\Gamma((p+1)/2)}{\sqrt\pi},\qquad \mathbb EY_n^{-p/2}=\frac{2^{-p/2}\Gamma((n-p)/2)}{\Gamma(n/2)}\quad(n>p).\]

Near zero, the latter integrand is a positive constant times \(y^{(n-p)/2-1}\), so it is not integrable when \(p\geq n\). This proves both the moment formula and its exact range of finiteness.

For each fixed real \(q>0\), the gamma-ratio asymptotic gives

\[\mathbb{E}|T_n|^q \longrightarrow \frac{2^{q/2}\Gamma((q+1)/2)}{\sqrt{\pi}} =\mathbb{E}|S|^q.\]

To verify asymptotic uniform integrability of \(|T_n|^p\), fix \(q>p\). The \(q\)-th moments are bounded for all sufficiently large \(n\), and

\[\begin{aligned} \mathbb{E}\left[|T_n|^p\mathbf1_{\{|T_n|^p>M\}}\right] &\le M^{-(q-p)/p}\mathbb{E}|T_n|^q,\\ \lim_{M\to\infty}\limsup_{n\to\infty} \mathbb{E}\left[|T_n|^p\mathbf1_{\{|T_n|^p>M\}}\right] &=0. \end{aligned}\]

For fixed integer \(p\), this justifies \(\mathbb{E}T_n^p\to\mathbb{E}S^p\). The tail expectation is not equal to the tail probability; the higher-moment bound is what makes this argument work.

Proposition 4. \(X_n\), \(X\) discrete and supported on \(\mathop{\mathrm{\mathbb{N}}}\). Then, \({X_n}\xrightarrow{\mathrm{d}}X\) if and only if \(\mathop{\mathrm{\mathds{P}}}(X_n = x) \xrightarrow{n\to\infty} \mathop{\mathrm{\mathds{P}}}(X=x)\) for all \(x \in \mathop{\mathrm{\mathbb{N}}}\).

Proof. If \(X_n\Rightarrow X\), the interval \((k-1/2,k+1/2)\) is a continuity set for every \(k\in\mathbb N\), and its probability equals \(\mathbb P(X_n=k)\). This proves convergence of the point masses. Conversely, suppose all point masses converge. For any \(\varepsilon>0\), choose a finite \(A\subset\mathbb N\) with \(\mathbb P(X\notin A)<\varepsilon\). Convergence of its finitely many point masses gives \(\mathbb P(X_n\notin A)\to\mathbb P(X\notin A)\). Therefore

\[\limsup_n\sum_{k\in\mathbb N}|\mathbb P(X_n=k)-\mathbb P(X=k)| \leq\limsup_n\left(\sum_{k\in A}|\mathbb P(X_n=k)-\mathbb P(X=k)|+\mathbb P(X_n\notin A)+\mathbb P(X\notin A)\right) \leq2\varepsilon.\]

Let \(\varepsilon\downarrow0\). This proves total variation convergence, and therefore weak convergence. Requiring the limiting masses to sum to one prevents loss of mass at infinity. ◻

Remark 5 (Demonstrating Statements in Fact 10). Consider 10: if \(X_n=o_P(1)\) and \(Z_n=O_P(1)\), then \(X_n+Z_n=O_P(1)\). The two vectors need not be independent.

Fix \(\eta>0\). Choose \(M\) so that \(\mathbb P(\|Z_n\|>M)<\eta/2\) for all sufficiently large \(n\). Since \(X_n\to_P0\), we also have \(\mathbb P(\|X_n\|>1)<\eta/2\) eventually. The triangle inequality gives

\[\begin{aligned} \mathbb P(\|X_n+Z_n\|>M+1) &\le\mathbb P(\|X_n\|>1)+\mathbb P(\|Z_n\|>M)\\ &<\eta \end{aligned}\]

for all sufficiently large \(n\). This proves asymptotic boundedness in probability. If we use the convention with a supremum over all \(n\), enlarge the threshold to control the finitely many remaining random vectors; each has a tail probability tending to zero.

Proposition 6. If, for all \(\varepsilon> 0\), \(\sum_{n \in \mathop{\mathrm{\mathbb{N}}}} \mathop{\mathrm{\mathds{P}}}(\abs{X_n} > \varepsilon) < \infty\), then \({X_n}\xrightarrow{\mathrm{a.s.}}0\).

Proof. For each positive integer \(m\), the first Borel–Cantelli lemma gives

\[\mathbb P\bigl(|X_n|>1/m\text{ infinitely often}\bigr)=0.\]

Take the intersection of these probability-one events over \(m\). On that event, for every \(m\) there is an \(N_m\) such that \(|X_n|\le1/m\) whenever \(n\ge N_m\). Given any \(\varepsilon>0\), choose \(m\) with \(1/m<\varepsilon\). It follows that \(X_n\to0\) on the same probability-one event.

The conclusion is convergence to zero, not eventual equality to zero. ◻

Proposition 7. If \({\mathbb{E}X_n}\xrightarrow{n \to \infty}\mu\) and \({\mathop{\mathrm{\mathds{V}}}X_n}\xrightarrow{n \to \infty}0\), then \({X_n}\xrightarrow{\mathds{P}}\mu\).

Proof. The variance assumption means that each \(X_n\) has a finite second moment. The bias–variance identity gives

\[\mathbb E(X_n-\mu)^2=\operatorname{Var}(X_n)+(\mathbb EX_n-\mu)^2\longrightarrow0.\]

For every \(\varepsilon>0\), Markov's inequality applied to the nonnegative squared error gives \(\mathbb P(|X_n-\mu|>\varepsilon)\leq\varepsilon^{-2}\mathbb E(X_n-\mu)^2\to0\). Thus there is even convergence in mean square. ◻

Proposition 8. If \({\sqrt{n} (T_n - \theta)}\xrightarrow{\mathrm{d}}D\), then \({T_n}\xrightarrow{\mathds{P}}\theta\).

Proof. Put \(W_n=\sqrt n(T_n-\theta)\). Since \(W_n\Rightarrow D\), where \(D\) is a real-valued random variable, Portmanteau applied to the closed set \(\{x:|x|\ge M\}\) gives

\[\limsup_n\mathbb P(|W_n|\ge M)\le\mathbb P(|D|\ge M).\]

Fix \(\varepsilon,\eta>0\) and choose \(M\) large enough that the right-hand side is smaller than \(\eta\). For all sufficiently large \(n\), \(\varepsilon\sqrt n\ge M\), so

\[\mathbb P(|T_n-\theta|>\varepsilon) =\mathbb P(|W_n|>\varepsilon\sqrt n) \le\mathbb P(|W_n|\ge M).\]

Thus the limit superior is at most \(\eta\). Let \(\eta\downarrow0\). Both tails are controlled by the same bound. ◻

Lemma 9 (Basic Facts About Compact Sets). A compact set \(K\) in a metric space is

Totally bounded
Separable

Proof. Fix \(\varepsilon>0\). The balls \(\{B_\varepsilon(x):x\in K\}\) form an open cover of \(K\). Compactness gives a finite subcover, so \(K\) is totally bounded. The radius is fixed before extracting the subcover.

For each \(m\ge1\), choose a finite \(2^{-m}\)-net \(F_m\subset K\). Then

\[F=\bigcup_{m=1}^{\infty}F_m\]

is countable. Given \(x\in K\) and \(\varepsilon>0\), choose \(m\) with \(2^{-m}<\varepsilon\) and a point \(y\in F_m\) with \(d(x,y)<2^{-m}\). Thus \(F\) is dense in \(K\), proving separability. If \(K\) is empty, both conclusions hold directly. ◻

Definition 10. The Skorokhod space, denoted \(D[a,b]\), is the space of all càdlàg functions on \([a,b]\).

Lemma 11. If \(a<b\), then \(D[a,b]\) is not separable under the sup norm.

Proof. The sup norm keeps two step functions a fixed distance apart even when their jump times are arbitrarily close. To turn this into an obstruction to separability, for each \(\alpha\in(a,b)\), let

\[f_\alpha(x)=\mathbf1_{\{x\ge\alpha\}},\qquad x\in[a,b].\]

This is a right-continuous step function with a left limit at every point of \((a,b]\), so \(f_\alpha\in D[a,b]\). If \(\alpha<\beta\), then \(f_\alpha(\alpha)=1\) and \(f_\beta(\alpha)=0\), while both functions take values in \(\{0,1\}\). Hence

\[\|f_\alpha-f_\beta\|_\infty=1.\]

Suppose \(D[a,b]\) had a countable dense set \(\{g_1,g_2,\ldots\}\). For every \(\alpha\), choose the first \(g_j\) within sup-norm distance \(1/3\) of \(f_\alpha\). Two distinct step functions cannot choose the same \(g_j\), since the triangle inequality would make their distance less than \(2/3\). This would inject the uncountable interval \((a,b)\) into \(\mathbb N\), a contradiction.

The nondegenerate-interval assumption matters: on \([a,a]\), this function space is isometric to \(\mathbb R\) and is separable. ◻