Convergence and Large-Sample Probability
Different limit theorems answer different questions. Almost-sure convergence follows individual sample paths; convergence in probability controls the chance of a large error; convergence in distribution compares laws. We first develop the tools for moving between these descriptions, then apply moment bounds and Borel–Cantelli to averages of independent observations.
Definition 1. A sequence of random vectors, denoted \(\{X_n\}\) is said to converge in probability to \(X\) if
This is denoted \({X_n} \xrightarrow{\mathop{\mathrm{\mathbf{P}}}} {X}\).
Definition 2. A sequence of random vectors, denoted \(\{X_n\}\) is said to converge in distribution to \(X\) if
at all continuity points \(x\) of the map \(x \mapsto \mathop{\mathrm{\mathbf{P}}}(X \leq x)\). This is denoted \({X_n} \xrightarrow{d} {X}\).
Definition 3. A sequence of random vectors, denoted \(\{X_n\}\) is said to converge almost surely to \(X\) if
This is denoted \({X_n} \xrightarrow{\rm a.s.} {X}\).
Theorem 4 (Portmanteau). The following are equivalent for random vectors \(X_n\) and random vector \(X\):
\(\mathop{\mathrm{\mathbf{P}}}(X_n \leq x) \to \mathop{\mathrm{\mathbf{P}}}(X \leq x)\) at all continuity points of \(x \mapsto \mathop{\mathrm{\mathbf{P}}}(X \leq x)\)
\(\mathbb{E} f(X_n) \to \mathbb{E} f(X)\) for all bounded, continuous functions \(f\)
\(\mathbb{E} f(X_n) \to \mathbb{E} f(X)\) for all bounded, Lipschitz functions \(f\)
\(\liminf \mathbb{E} f(X_n) \geq \mathbb{E} f(X)\) for all nonnegative, continuous functions \(f\)
\(\liminf \mathop{\mathrm{\mathbf{P}}}(X_n \in G) \geq \mathop{\mathrm{\mathbf{P}}}(X \in G)\) for all open sets \(G\)
\(\limsup \mathop{\mathrm{\mathbf{P}}}(X_n \in H) \leq \mathop{\mathrm{\mathbf{P}}}(X \in H)\) for all closed sets \(H\)
\(\mathop{\mathrm{\mathbf{P}}}(X_n \in B) \to \mathop{\mathrm{\mathbf{P}}}(X \in B)\) for all Borel sets \(B\) with \(\mathop{\mathrm{\mathbf{P}}}(X \in \partial B) = 0\).
Proof. Number the assertions (i)–(vii) in their displayed order. Write \(F_n(x)=\mathbb P(X_n\leq x)\) and \(F(x)=\mathbb P(X\leq x)\), with inequalities between vectors understood coordinatewise.
(i) ⇒ (ii). Let \(f\) be bounded and continuous. By rescaling, it suffices to consider \(|f|\leq1\). Fix \(\varepsilon>0\) and choose a large half-open rectangle \(I=\prod_{r=1}^d(a_r,b_r]\) with \(\mathbb P(X\notin I)<\varepsilon\). Choose every endpoint outside the atoms of the corresponding marginal of \(X\). This is possible because each marginal has at most countably many atoms.
At every vertex of such a rectangle, \(F\) is continuous: a small change in the coordinates can change the lower-orthant event only when a coordinate of \(X\) lies in a small interval around the corresponding vertex coordinate. Those probabilities tend to zero because the coordinates are not atoms. For any rectangle \(J=\prod_{r=1}^d(u_r,v_r]\) with the same endpoint property, inclusion–exclusion gives
where \((c_e)_r=u_r\) if \(e_r=0\) and \((c_e)_r=v_r\) if \(e_r=1\). In particular, \(\mathbb P(X_n\notin I)<\varepsilon\) for all sufficiently large \(n\).
The function \(f\) is uniformly continuous on the compact rectangle \(\overline I\). Partition \(I\) into finitely many disjoint half-open rectangles \(I_1,\ldots,I_m\), using cut points that also avoid marginal atoms, so that the oscillation of \(f\) on each cell is at most \(\varepsilon\). Pick \(x_j\in I_j\) and set
Then \(|f-f_\varepsilon|\leq\varepsilon\) on \(I\), while \(f_\varepsilon=0\) outside \(I\). Consequently, for all sufficiently large \(n\),
For the step function itself, there are only finitely many cell probabilities to compare:
Taking \(n\) large enough to make this last difference smaller than \(\varepsilon\), the triangle inequality yields
Since \(\varepsilon\) is arbitrary, (ii) follows. Choosing endpoints outside marginal atoms makes this argument valid even when \(F\) has discontinuities elsewhere.
(ii) ⇒ (iii). Every bounded Lipschitz function is bounded and continuous.
(iii) ⇒ (v). Let \(G\) be open; the cases \(G=\varnothing\) and \(G=\mathbb R^d\) are immediate. Define
Distance to a nonempty set is \(1\)-Lipschitz, so \(f_m\) is bounded and Lipschitz. Moreover, \(0\leq f_m\leq\mathbf1_G\) and \(f_m\uparrow\mathbf1_G\): every point of the open set has positive distance from its complement. For each fixed \(m\), (iii) gives
Letting \(m\to\infty\) and applying monotone convergence proves (v).
(v) ⇔ (vi). If \(H\) is closed, then \(H^c\) is open. Taking complements gives
The same calculation starting with an open set and its closed complement proves the converse.
(v), (vi) ⇒ (vii). For a Borel set \(B\), use \(B^\circ\subseteq B\subseteq\overline B\) to obtain
If \(\mathbb P(X\in\partial B)=0\), the two outside probabilities both equal \(\mathbb P(X\in B)\), since \(\partial B=\overline B\setminus B^\circ\). The desired convergence follows by squeezing.
(vii) ⇒ (i). Fix a continuity point \(x\) of \(F\). Choose vectors \(a^{(m)}\uparrow x\) and \(b^{(m)}\downarrow x\), strictly below and above \(x\) in every coordinate, with all coordinates avoiding marginal atoms. The lower orthants with these corners are continuity sets: their boundaries are contained in the union of the coordinate hyperplanes through their corners, each of which has probability zero. Thus (vii) and monotonicity give, for each fixed \(m\),
Let \(m\to\infty\). Continuity of \(F\) at \(x\) makes both outside terms tend to \(F(x)\), proving (i).
(ii) ⇒ (iv). Let \(f\) be nonnegative and continuous, and fix \(M>0\). The truncation \(h_M=\min(f,M)\) is bounded and continuous, and \(h_M\leq f\). Hence
Now let \(M\to\infty\). Since \(h_M(X)\uparrow f(X)\), monotone convergence gives \(\mathbb E h_M(X)\uparrow\mathbb E f(X)\), including when this expectation is infinite. This proves (iv); the limit in \(n\) is taken before the truncation level increases.
(iv) ⇒ (ii). For bounded continuous \(f\), choose \(M\geq\|f\|_\infty\). Both \(M+f\) and \(M-f\) are nonnegative and continuous. Applying (iv) separately to them gives
After subtracting \(M\) and rearranging,
Thus the expectations converge. Together, these implications prove all seven assertions equivalent. ◻
Theorem 5 (Continuous mapping). Let \(g:\mathbb R^k\to\mathbb R^\ell\) be Borel measurable and continuous at every point of a Borel set \(C\) with \(P(X\in C)=1\). Then:
\({X_n} \xrightarrow{\rm a.s.} {X} \implies {g(X_n)} \xrightarrow{\rm a.s.} {g(X)}\).
\({X_n} \xrightarrow{d} {X} \implies {g(X_n)} \xrightarrow{d} {g(X)}\).
\({X_n} \xrightarrow{\mathop{\mathrm{\mathbf{P}}}} {X} \implies {g(X_n)} \xrightarrow{\mathop{\mathrm{\mathbf{P}}}} {g(X)}\).
Proof. Borel measurability of \(g\) ensures that \(g(X_n)\) and \(g(X)\) are random vectors. In the almost sure and in-probability assertions, the variables are defined on a common probability space.
(i) Almost sure convergence. Let \(A=\{\omega:X_n(\omega)\to X(\omega)\}\cap\{\omega:X(\omega)\in C\}\). Both events have probability one, so \(\mathbb P(A)=1\). Fix \(\omega\in A\) and \(\varepsilon>0\). Continuity of \(g\) at \(X(\omega)\) provides a \(\delta>0\) such that
Since \(X_n(\omega)\to X(\omega)\), the left inequality holds with \(y=X_n(\omega)\) for every sufficiently large \(n\). Thus \(g(X_n(\omega))\to g(X(\omega))\) on \(A\), proving almost sure convergence. The choices of \(\delta\) and the starting index may depend on \(\omega\); uniform continuity is not required.
(ii) Convergence in distribution. Let \(H\subseteq\mathbb R^\ell\) be closed and put \(B=g^{-1}(H)\). The set \(B\) is Borel, but it need not be closed because \(g\) need not be continuous everywhere. We claim that
Indeed, take \(x\in\overline B\cap C\). There is a sequence \(x_j\in B\) with \(x_j\to x\). Continuity of \(g\) at \(x\) gives \(g(x_j)\to g(x)\), and \(g(x_j)\in H\) for every \(j\). Since \(H\) is closed, \(g(x)\in H\), so \(x\in B\). This proves the inclusion. Consequently, using \(\mathbb P(X\in C^c)=0\),
Now apply the closed-set form of Portmanteau to the closed set \(\overline B\subseteq\mathbb R^k\):
This holds for every closed \(H\). The converse direction of Portmanteau, now in \(\mathbb R^\ell\), therefore gives \(g(X_n)\Rightarrow g(X)\).
(iii) Convergence in probability. We first extract an almost surely convergent further subsequence from any subsequence. Let \((X_{n_j})\) be arbitrary. Since \(X_{n_j}\to X\) in probability, we can choose increasing indices \(j_m\) such that
The sum of these probabilities is finite. By the first Borel–Cantelli lemma, with probability one only finitely many of the events occur. Hence \(\|X_{n_{j_m}}-X\|\leq2^{-m}\) eventually, and
For every fixed \(\varepsilon>0\), the indicators of \(\{\|g(X_{n_{j_m}})-g(X)\|>\varepsilon\}\) then tend to zero almost surely and are bounded by one. Dominated convergence gives
To conclude for the entire sequence, suppose instead that \(g(X_n)\) failed to converge to \(g(X)\) in probability. Then there would be \(\varepsilon,\eta>0\) and a subsequence \((n_j)\) such that
The construction above supplies a further subsequence along which these same probabilities tend to zero, a contradiction. Therefore \(g(X_n)\to g(X)\) in probability. ◻
Lemma 6. (Helly selection). Every sequence of probability measures \(\mu_n\) on \(\mathbb R^d\) has a subsequence and a finite measure \(\mu\) of total mass at most one such that \(\int f\,d\mu_{n_j}\to\int f\,d\mu\) for every continuous compactly supported \(f\). This is vague convergence; it need not be convergence in distribution to a probability law.
Proof.
Map each coordinate by \(x\mapsto\frac12+\pi^{-1}\arctan x\) into \((0,1)\) and regard the image laws as measures on the compact cube \([0,1]^d\). On a compact cube, take nested finite dyadic partitions. A diagonal subsequence makes the mass of every cell converge. The limiting masses are nonnegative, sum to one at each level, and add consistently from children to parents. They define a probability on nested cell choices: divide the unit interval into intervals of these masses, and subdivide each interval according to its children. A uniform point selects a nested sequence, outside a countable set of endpoints.
The closures of the selected cells are nested compact sets whose diameters tend to zero, so they determine a unique point \(Z\). Choose a representative \(z_A\) in every cell \(A\) of the level-\(m\) partition \(\mathcal P_m\). The representative selected by the uniform point is a measurable, finite-valued random vector \(Z_m\); since \(Z_m\to Z\), the limit \(Z\) is measurable. Write \(p_A\) for the limiting mass of cell \(A\), and \(\nu_n\) for the image laws on the cube. For continuous \(f\) on the cube, uniform continuity supplies a modulus \(\omega_f(\delta)\to0\). If \(\delta_m\) is the largest cell diameter, then
For fixed \(m\), the finite sums converge by construction. Hence the limit superior of \(|\int f\,d\nu_{n_j}-\mathbb E f(Z)|\) is at most \(2\omega_f(\delta_m)\); now let \(m\to\infty\). This proves compact-cube weak selection, even if the limiting point lies on a cell boundary.
Restrict the law of \(Z\) to the open cube and pull it back through the inverse coordinate map. The resulting measure \(\mu\) has mass at most one. If \(f\in C_c(\mathbb R^d)\), its transformed function is zero near the boundary of the cube, so extension by zero is continuous on the whole compact cube. Applying the convergence just proved gives \(\int f\,d\mu_{n_j}\to\int f\,d\mu\). Boundary mass is the mass lost at infinity.
◻
Theorem 7 (Prokhorov).
\({X_n} \xrightarrow{d} {X} \implies (X_n)\) is uniformly tight.
\(\{X_n\}\) is uniformly tight implies there exists \(X\) and \(\{n_j\}\) such that \({(X_{n_j})} \xrightarrow{d} {X}\).
Proof. Uniform tightness means that for every \(\varepsilon>0\) some compact \(K\subset\mathbb R^d\) satisfies \(\sup_n\mathbb P(X_n\notin K)<\varepsilon\). In Euclidean space it is enough to use closed balls.
Suppose first that \(X_n\Rightarrow X\). Fix \(\varepsilon>0\) and choose \(R\) with \(\mathbb P(\|X\|\geq R)<\varepsilon\). This is possible because \(X\) is finite almost surely. The set \(\{x:\|x\|\geq R\}\) is closed, so
There is consequently \(N\) such that the left-hand tail probability is below \(\varepsilon\) for all \(n\geq N\). Each of the finitely many variables \(X_1,\ldots,X_{N-1}\) also has tails tending to zero. Increase \(R\) to make their tail probabilities smaller than \(\varepsilon\) as well. The resulting closed ball proves uniform tightness.
Conversely, write \(\mu_n\) for the law of \(X_n\). Helly selection, proved above without using this theorem, gives a subsequence converging vaguely to a measure \(\mu\) with total mass at most one. Fix \(\varepsilon>0\), choose a compact \(K\) with \(\mu_n(K)>1-\varepsilon\) for every \(n\), and take \(h\in C_c(\mathbb R^d)\) with \(0\leq h\leq1\) and \(h=1\) on \(K\). For example, a sufficiently large ball contains \(K\), and a continuous radial cutoff is one on that ball and zero outside a larger ball. Vague convergence gives
Thus \(1\geq\mu(\mathbb R^d)\geq1-\varepsilon\) for every \(\varepsilon>0\), and \(\mu\) is a probability measure. For the same cutoff, both \(\int(1-h)\,d\mu_{n_j}\leq\varepsilon\) and \(\int(1-h)\,d\mu\leq\varepsilon\). If \(f\) is bounded and continuous, then \(fh\) is continuous with compact support and
Let \(j\to\infty\), then \(\varepsilon\downarrow0\). Portmanteau proves weak convergence to \(\mu\). A random vector \(X\) with law \(\mu\) may be realized as the identity map on \((\mathbb R^d,\mathcal B(\mathbb R^d),\mu)\). ◻
1. Large Sample Theory
1.1. Law of Large Numbers
Theorem 8 (Strong LLN). Let \(X_1,X_2,\dots\) be i.i.d. random variables on some probability space \((\Omega,\mathscr F,\mathop{\mathrm{\mathbf{P}}})\). Let \(S_n \triangleq X_1 + \dots + X_n\). If \(\mathbb{E} X_1 = \mu\) and \(\mathbb{E}|X_1|^4 < \infty\), then
or, equivalently, \(\mathop{\mathrm{\mathbf{P}}}(\lim\limits_{n \to \infty} \frac{S_n}{n} = \mu) = 1\).
Proof.
Subtract the mean and write \(Y_i=X_i-\mu\). Independence and centering eliminate all terms in the fourth-power expansion except those in which each index appears at least twice:
Thus for each \(\varepsilon>0\), \(\sum_nP(|S_n/n-\mu|>\varepsilon)\leq C\varepsilon^{-4}\sum_nn^{-2}<\infty\). Borel–Cantelli, applied to \(\varepsilon=1,1/2,1/3,\ldots\), proves almost sure convergence.
There are \(n\) terms with all four indices equal. The surviving terms with two distinct indices each repeated twice number \(6\binom n2=3n(n-1)\); independence makes their expectations \((\mathbb EY_1^2)^2\). Every other term has an index occurring exactly once and hence vanishes by centering. This accounts for every term in the fourth-power expansion. The countable intersection of the Borel–Cantelli probability-one events ensures convergence for every positive tolerance on the same set of paths.
◻
Theorem 9 (Weak LLN). Let \(X_1,X_2,\dots\) be i.i.d. random variables on some probability space \((\Omega,\mathscr F,\mathop{\mathrm{\mathbf{P}}})\). Let \(S_n \triangleq X_1 + \dots + X_n\). If \(\mathbb{E} X_1 = \mu\), then
Proof.
Only \(\mathbb{E}|X_1|<\infty\) is needed. Put \(X_i^{(K)}=X_i1_{\{|X_i|\leq K\}}\). For fixed \(K\), the truncated variables are i.i.d. and bounded, so Chebyshev gives \(n^{-1}\sum_iX_i^{(K)}-\mathbb{E}X_1^{(K)}\to0\) in probability. Moreover,
The common tail bound tends to zero as \(K\to\infty\). Markov’s inequality and the triangle inequality complete the proof by taking \(n\to\infty\) first and then \(K\to\infty\).
More explicitly, put \(r_K=\mathbb E(|X_1|\mathbf1_{\{|X_1|>K\}})\). For \(r_K<\varepsilon/3\), the triangle inequality followed by Chebyshev and Markov yields
The first term vanishes for fixed \(K\), and dominated convergence gives \(r_K\downarrow0\). This proves the result under the finite first-moment assumption, without assuming the fourth moment required in the preceding strong-law proof.
◻
Remark 10. Note that for a sequence of events \(A_1,A_2,\dots\), we can put \(N_n (\omega) = \mathbb 1_{A_1} (\omega) + \dots + \mathbb 1_{A_n} (\omega)\) to count the occurrence of \(\omega\) in the events \(A_1,\dots,A_n\). This leads us to reformulate \(\{A_n \text{ i.o.}\} = \{\omega: \sup_n N_n(\omega) = \infty\}\), so we can reduce the problem of characterizing the condition for Borel-Cantelli lemma to that of studying the behaviour of \(N_n\).
The event-count formulation also suggests a variance proof. If \(\{A_1,A_2,\dots\}\) are independent events, then we can derive explicit probabilistic bounds on \(N_n\) given some “threshold value” \(x\). To start, note that \(p_k = \mathop{\mathrm{\mathbf{P}}}(A_k) = \mathbb{E} \mathbb 1_{A_k}\), and \(\mathop{\mathrm{Var}}(\mathbb 1_{A_k}) = p_k (1-p_k) \leq p_k\). Therefore, \(\mathbb{E} N_n = p_1 + \dots p_n = m_n\), and \(\mathop{\mathrm{Var}}(N_n) \leq m_n\). For \(n\) large enough that \(m_n>x\), Chebyshev gives
Note that independence is used to obtain the estimate after the third inequality. Furthermore, if \(\sum_{n \in \mathbb N} p_n\) diverges, then the limit of the leftmost term above is zero for each \(x\). From there follows the fact that, because \(\mathop{\mathrm{\mathbf{P}}}(\sup_k N_k \leq x) \leq \mathop{\mathrm{\mathbf{P}}}(N_n \leq x)\) for each \(n\), \(\mathop{\mathrm{\mathbf{P}}}(\sup_k N_k \leq x) = 0\) and \(\mathop{\mathrm{\mathbf{P}}}(\sup_k N_k < \infty) = 0\) (from taking unions over all \(x\)). So, \(\mathop{\mathrm{\mathbf{P}}}(A_n \text{ i.o.}) = \mathop{\mathrm{\mathbf{P}}}(\sup_n N_n = \infty) = 1\).
Proposition 11 (Borel-Cantelli, Second Lemma: Reformulation). Suppose \(\sum_{n \in \mathbb N} \mathop{\mathrm{\mathbf{P}}}(A_n)\) diverges, and
Then, \(\mathop{\mathrm{\mathbf{P}}}(A_n \text{ i.o.}) = 1\).
Proof. Let \(N_n=\sum_{k=1}^n\mathbf1_{A_k}\) and \(m_n=\mathbb EN_n\to\infty\). Expanding the square, without an independence assumption, gives
The hypothesis therefore supplies a subsequence \(n_r\) along which the last expectation tends to zero. For any fixed integer \(M\geq0\), eventually \(m_{n_r}>M\), and
Since \(\{\sup_nN_n\leq M\}\subseteq\{N_{n_r}\leq M\}\), the former event has probability zero. Take its union over the nonnegative integers \(M\): the total number of occurrences is infinite almost surely, which is the desired conclusion. The use of a subsequence is exactly what permits the stated limit inferior assumption. ◻
1.2. Convergence in Distribution
Distribution functions, \(F_n\), are said to converge weakly to the distribution function \(F\) if, for every continuity point \(x\), \(\lim\limits_{n\to\infty} F_n(x) = F(x)\). This is sometimes written \(F_n \Rightarrow F\). For stochastic convergence, let \(X_n\) and \(X\) be random variables, and \(F_n\) and \(F\) be their respective distribution functions. Then, \(X_n\) is said to converge in distribution, or converge in law, to \(X\) if \(F_n \Rightarrow F\). This is usually written as \(X_n \xrightarrow{d} X\).
On the real line, weak convergence can also be represented by almost-sure convergence after choosing a new common probability space. The following theorem preserves each individual law; it does not preserve an arbitrary original coupling.
Theorem 12. Let \(\mu_n\) and \(\mu\) be probability measures on \((\mathbb R, \mathscr B_{\mathbb R})\), and \(\mu_n \Rightarrow \mu\). Then, there exists random variables \(Y_n\) and \(Y\) on a probability space \((\Omega, \mathscr F, \mathop{\mathrm{\mathbf{P}}})\) such that \(Y_n\) has law \(\mu_n\), \(Y\) has law \(\mu\), and \(Y_n (\omega) \to Y(\omega)\) for all \(\omega \in \Omega\).
Proof. Let \(F_n,F\) be the distribution functions, and define their generalized inverses on \((0,1)\) by
These are finite, nondecreasing, Borel functions. Right-continuity of a distribution function gives \(q_n(u)\leq x\) if and only if \(u\leq F_n(x)\), and the same equivalence for \(q\). Thus for a uniform \(U\) on \((0,1)\), \(q_n(U)\) has distribution function \(F_n\) and \(q(U)\) has distribution function \(F\).
We claim \(q_n(u)\to q(u)\) at every continuity point \(u\) of \(q\). Fix such a point and small \(\delta>0\) with \(u\pm\delta\in(0,1)\). Choose continuity points \(x_-<q(u-\delta)\) and \(x_+>q(u+\delta)\) of \(F\), as close to those respective quantiles as desired. Such points exist because a distribution function has at most countably many discontinuities. The inverse relation gives
Weak convergence gives \(F_n(x_-)<u<F_n(x_+)\) for all sufficiently large \(n\), so \(x_-<q_n(u)\leq x_+\). Let the chosen endpoints approach the two quantiles, then let \(\delta\downarrow0\). Continuity of \(q\) at \(u\) squeezes the lower and upper limits to \(q(u)\).
The discontinuities of the monotone function \(q\) form a countable set \(D\). Take \(\Omega=(0,1)\setminus D\) with the trace Borel sigma-field and restricted Lebesgue probability, and define \(Y_n(u)=q_n(u)\), \(Y(u)=q(u)\). Removing this null set changes none of the laws, and convergence now holds for every \(u\in\Omega\), as stated. This construction provides new jointly defined variables; it imposes no claim about the joint distribution of any original versions. ◻
Theorem 13. Let \(h : \mathbb R \to \mathbb R\) be a measurable function, and the set of discontinuities, \(D_h\), is measurable. If \(\mu_n \Rightarrow \mu\) and \(\mu(D_h) = 0\), then \(\mu_n h^{-1} \Rightarrow \mu h^{-1}\).
Proof. Use Theorem 12 to realize the measures as the laws of \(Y_n,Y\) with \(Y_n\to Y\) almost surely. Since \(P(Y\in D_h)=\mu(D_h)=0\), continuity at the limiting point gives \(h(Y_n)\to h(Y)\) almost surely. For a bounded continuous real \(f\), dominated convergence gives
The bounded-continuous form of Portmanteau proves the assertion. Measurability of \(h\) ensures that the image laws in this display are defined. ◻
To state weak convergence directly for probability measures, use sets whose boundaries carry no limiting mass. We say the boundary of a set \(A\), denoted \(\partial A\), consists of limit points of sequences in \(A\) and \(A^c\). A set \(A\) is called a \(\mu\)-continuity set if \(\mu(\partial A) = 0\) and it is a Borel set.
Proposition 14. The following conditions are equivalent:
\(\mu_n \Rightarrow \mu\);
\(\int f \dif \mu_n \to \int f \dif \mu\) for every bounded, continuous, real-valued function \(f\);
\(\mu_n(A) \to \mu(A)\) for every \(\mu\)-continuity set \(A\).
Proof. Realize each \(\mu_n\) and \(\mu\) as a random variable law, for example by the quantile construction above on a uniform probability space; no convergence assumption is needed for this realization. Then \(\int f\,d\mu_n=\mathbb Ef(X_n)\) and \(\mu_n(A)=P(X_n\in A)\). Assertions (i), (ii), and (vii) of Theorem 4 are exactly the three statements here, so their already proved equivalence applies. ◻