Martingales

Suppose observations arrive one at a time and we repeatedly update the conditional expectation of a fixed integrable quantity. The resulting sequence is a martingale: given the present information, its next conditional mean equals its present value. This example motivates the definition and the role of the filtration.

Definition 1 (Martingale). Let \(X_1,X_2,\dots\) be a sequence of random variables on a probability space \((\Omega, \mathscr F, \mathop{\mathrm{\mathbf{P}}})\). Let \(\mathscr F_1, \mathscr F_2, \dots\) be a sequence of \(\sigma\)-fields in \(\mathscr F\). Then, the (double) sequence \(\{(X_n, \mathscr F_n)\}_{n \in \mathbb N}\) is called a martingale if the following conditions are satisfied for all \(n \in \mathbb N\):

(\(\{\mathscr F_n\}\) form a filtration) \(\mathscr F_n \subset \mathscr F_{n+1}\);
(\(\{X_n\}\) is adapted to the filtration) \(X_n\) is \(\mathscr F_n\)-measurable;
\(\mathbb{E}|X_n| < \infty\); and
\(\mathbb{E}[X_{n+1} \mid \mathscr F_n] = X_n\) with probability one.

The filtration is part of the definition: a sequence is called a martingale if it is a martingale relative to some sequence of \(\sigma\)-fields. Therefore, the sequence of \(\sigma\)-fields \(\mathscr H_n = \sigma(X_1,\dots,X_n)\) also works for a process already known to be a martingale. This follows by conditioning again on the smaller filtration. This is referred to as the smallest \(\sigma\)-fields with respect to which \(X_n\) are a martingale.

Remark 2. Note that \(\mathbb{E}[X_{n+1} \mid \mathscr F_n] = X_n\) with probability one gives rise to the following relation for all \(n\):

\[\int_A X_{n+1} \dif \mathop{\mathrm{\mathbf{P}}}= \int_A X_n \dif \mathop{\mathrm{\mathbf{P}}}\quad \text{for all } A \in \mathscr F_n ~.\]

This requirement follows because \(X_n\) is \(\mathscr F_n\)-measurable. Since \(\mathscr F_n\) is a nested sequence of \(\sigma\)-fields, it follows that \(\int_A X_n \dif \mathop{\mathrm{\mathbf{P}}}= \int_A X_{n+1} \dif \mathop{\mathrm{\mathbf{P}}}= \dots = \int_A X_{n+k} \dif \mathop{\mathrm{\mathbf{P}}}\) for any \(k \geq 1\). So, \(X_n\) is a version of \(\mathbb{E}[X_{n+k} \mid \mathscr F_n]\) for any \(k \geq 1\), with probability one.

Remark 3. It is equivalent to give the defining conditions of a martingale using differences; to do so, let \(\Delta_n = X_n - X_{n-1}\), with \(\Delta_1 = X_1\). Then, by linearity, the condition \(\mathbb{E}[X_{n+1} \mid \mathscr F_n] = X_n\) is equivalent to \(\mathbb{E}[\Delta_{n+1} \mid \mathscr F_n] = 0\), by linearity. Therefore, the sets \(X_1, \dots, X_n\) and \(\Delta_1, \dots, \Delta_n\) generate the same \(\sigma\)-field.

Example 4. Let \(\Omega = (0,1]\), \(\mathscr F\) be the collection of Borel sets of \(\Omega\), and \(\mathop{\mathrm{\mathbf{P}}}\) be the Lebesgue measure over \(\mathscr F\). Then, let \(\mathscr F_n\) be the (finite!) \(\sigma\)-field generated by the partition of \(\Omega\) into dyadic intervals, \((k2^{-n}, (k+1)2^{-n}]\) for \(k \in [0,2^n)\). Let \(\nu\) be any finite positive Borel measure on \((0,1]\). Its restriction to each finite sigma-field has the following Radon–Nikodym derivative with respect to Lebesgue measure:

\[X_n(\omega) = \frac{\nu((k2^{-n}, (k+1)2^{-n}])}{2^{-n}} \quad \text{for } \omega \in (k2^{-n}, (k+1)2^{-n}] ~.\]

Clearly, \(X_n\) is a martingale with respect to \(\mathscr F_n\). Note that \(A \in \mathscr F_n\) implies \(A \in \mathscr F_{n+1}\), and \(\int_A X_{n+1} \dif \mathop{\mathrm{\mathbf{P}}}= \nu(A)\) for any \(A \in \mathscr F_n\).

Example 5. Let \(X\) be an integrable random variable defined on \((\Omega, \mathscr F, \mathop{\mathrm{\mathbf{P}}})\). Let \(\mathscr F_n\) be nondecreasing \(\sigma\)-fields in \(\mathscr F\). If we let \(X_n = \mathbb{E}[X \mid \mathscr F_n]\), the tower property gives \(\mathbb{E}[X_{n+1} \mid \mathscr F_n] = X_n\). Therefore, \(X_n\), as defined, is a martingale relative to \(\mathscr F_n\).

Example 6 (Branching process). Let \(\{N_{n,k}:n,k\geq1\}\) be i.i.d. nonnegative integer-valued random variables with \(0<m=\mathbb{E} N_{1,1}<\infty\). Set \(Z_0=1\) and \(Z_n=\sum_{k=1}^{Z_{n-1}}N_{n,k}\), interpreting an empty sum as zero. With \(\mathscr F_n=\sigma(N_{j,k}:1\leq j\leq n,\ k\geq1)\), independence of the next row gives \(\mathbb{E}[Z_{n+1}\mid\mathscr F_n]=mZ_n\). Induction gives \(\mathbb{E} Z_n=m^n\), so \(X_n=Z_n/m^n\) is an integrable martingale. It is also a martingale for the smaller natural filtration \(\sigma(Z_0,\ldots,Z_n)\), by the tower property.

To justify the conditional mean of a random sum, write

\[Z_{n+1}=\sum_{k\geq1}1_{\{Z_n\geq k\}}N_{n+1,k}.\]

Each indicator is \(\mathscr F_n\)-measurable and the next-row variable has conditional mean \(m\). Conditional monotone convergence gives \(\mathbb{E}[Z_{n+1}\mid\mathscr F_n]=m\sum_{k\geq1}1_{\{Z_n\geq k\}}=mZ_n\). Induction from \(Z_0=1\) proves integrability at every time, so normalization by \(m^n\) is legitimate.

Definition 7 (Submartingale). Let \(X_1,X_2,\dots\) be a sequence of random variables on a probability space \((\Omega, \mathscr F, \mathop{\mathrm{\mathbf{P}}})\). Let \(\mathscr F_1, \mathscr F_2, \dots\) be a sequence of \(\sigma\)-fields in \(\mathscr F\). Then, the (double) sequence \(\{(X_n, \mathscr F_n)\}_{n \in \mathbb N}\) is called a submartingale if the following conditions are satisfied for all \(n \in \mathbb N\):

(\(\{\mathscr F_n\}\) form a filtration) \(\mathscr F_n \subset \mathscr F_{n+1}\);
(\(\{X_n\}\) is adapted to the filtration) \(X_n\) is \(\mathscr F_n\)-measurable;
\(\mathbb{E}|X_n| < \infty\); and
\(\mathbb{E}[X_{n+1} \mid \mathscr F_n] \geq X_n\) with probability one.

By way of nomenclature, a sequence is called a supermartingale if \(\mathbb{E}[X_{n+1} \mid \mathscr F_n] \leq X_n\) with probability one.

Proposition 8.

Let \(X_1,X_2,\dots\) be a martingale relative to \(\mathscr F_1, \mathscr F_2, \dots\). If \(\varphi\) is a convex function, and \(\varphi(X_n)\) are integrable, then \(\varphi(X_1),\varphi(X_2),\dots\) form a submartingale relative to \(\mathscr F_1, \mathscr F_2, \dots\).
Let \(X_1,X_2,\dots\) be a submartingale relative to \(\mathscr F_1, \mathscr F_2, \dots\). If \(\varphi\) is a nondecreasing, convex function, and \(\varphi(X_n)\) are integrable, then \(\varphi(X_1),\varphi(X_2),\dots\) form a submartingale relative to \(\mathscr F_1, \mathscr F_2, \dots\).

Proof. Conditional Jensen gives

\[\mathbb{E}[\varphi(X_{n+1})\mid\mathscr F_n]\geq\varphi(\mathbb{E}[X_{n+1}\mid\mathscr F_n]).\]

For a martingale the right side is \(\varphi(X_n)\). For a submartingale, monotonicity of \(\varphi\) gives the same lower bound. Adaptedness and the assumed integrability complete both assertions. Monotonicity in the second assertion is necessary: the deterministic submartingale \(X_n=n\) becomes a decreasing sequence under the convex function \(\varphi(x)=-x\). ◻

A stopping time permits the observation time to depend on the path, provided the decision to stop uses only information already available. Let \(\tau\) be a random variable taking values in the positive integers together with infinity. We say \(\tau\) is a stopping time with respect to \(\{\mathscr F_n\}\) if \(\{\omega : \tau(\omega) = k\} \in \mathscr F_k\) for every finite \(k\). Equivalently, we can rewrite the condition as \(\{\omega : \tau(\omega) \leq k\} \in \mathscr F_k\) for every finite \(k\).

Proposition 9 (Optional Sampling). Let \(X_1,\dots,X_n\) be a submartingale with respect to \(\mathscr F_1, \dots, \mathscr F_n\). Let \(\tau_1,\tau_2\) be stopping times satisfying \(1 \leq \tau_1 \leq \tau_2 \leq n\). Then, \(X_{\tau_1}, X_{\tau_2}\) is a submartingale with respect to \(\mathscr F_{\tau_1}, \mathscr F_{\tau_2}\).

Proof. For a bounded stopping time \(\tau\), define

\[\mathscr F_\tau=\{A\in\mathscr F_n:A\cap\{\tau\leq k\}\in\mathscr F_k\text{ for every }k\leq n\}.\]

The inclusion \(\mathscr F_{\tau_1}\subset\mathscr F_{\tau_2}\) follows by intersecting with \(\{\tau_2\leq k\}\subset\{\tau_1\leq k\}\). Also \(X_\tau\) is \(\mathscr F_\tau\)-measurable, since its restriction to \(\{\tau=j\}\) is \(X_j\), and it is integrable because \(|X_\tau|\leq\sum_{j=1}^n|X_j|\).

Fix \(A\in\mathscr F_{\tau_1}\). The event \(A\cap\{\tau_1\leq k<\tau_2\}\) belongs to \(\mathscr F_k\). Telescoping along each stopped path gives

\[\mathbb{E}[1_A(X_{\tau_2}-X_{\tau_1})]=\sum_{k=1}^{n-1}\mathbb{E}\left[1_{A\cap\{\tau_1\leq k<\tau_2\}}\mathbb{E}[X_{k+1}-X_k\mid\mathscr F_k]\right]\geq0.\]

The defining integral characterization of conditional expectation therefore gives \(\mathbb{E}[X_{\tau_2}\mid\mathscr F_{\tau_1}]\geq X_{\tau_1}\) almost surely. For a martingale the displayed expectations are zero, giving equality. ◻

Theorem 10 (Kolmogorov). Let \(X_1,\dots,X_n\) be a submartingale. Then, for any \(\varepsilon > 0\),

\[\mathop{\mathrm{\mathbf{P}}}\left( \max_{1 \leq k \leq n} X_k \geq \varepsilon \right) \leq \frac{\mathbb{E} |X_n|}{\varepsilon} ~.\]

Proof. Partition the crossing event into the disjoint events

\[A_k=\{X_1<\varepsilon,\ldots,X_{k-1}<\varepsilon,\ X_k\geq\varepsilon\}\in\mathscr F_k.\]

Repeated conditioning gives \(\mathbb{E}[X_n\mid\mathscr F_k]\geq X_k\), so

\[\varepsilon\sum_{k=1}^n\mathbb{P}(A_k)\leq\sum_{k=1}^n\mathbb{E}[X_k1_{A_k}]\leq\sum_{k=1}^n\mathbb{E}[X_n1_{A_k}]=\mathbb{E}[X_n1_{\cup_kA_k}]\leq\mathbb{E}X_n^+\leq\mathbb{E}|X_n|.\]

Dividing by \(\varepsilon\) proves the claim. ◻

Theorem 11 (Doob; Martingale Convergence). Let \(X_1, X_2, \dots\) be a submartingale. Suppose \(K = \sup_{n \in \mathbb N} \mathbb{E} |X_n| < \infty\). Then, \(X_n \xrightarrow{\rm a.s.} X\), where \(X\) is a random variable such that \(\mathbb{E}|X| \leq K\).

Proof, using the upcrossing inequality below. For each rational pair \(a<b\), monotone convergence and Lemma 12 give

\[\mathbb{E}\mathscr U_\infty(a,b)=\lim_n\mathbb{E}\mathscr U_n(a,b)\leq\frac{K+|a|}{b-a}<\infty.\]

Thus \(\mathscr U_\infty(a,b)<\infty\) almost surely. Remove the union of the exceptional sets over the countably many rational pairs. If a remaining path had \(\liminf X_n<\limsup X_n\), a rational interval strictly between these two extended-real limits would be crossed infinitely often. Therefore \(X_n\) has an extended-real limit on this event. Fatou gives \(\mathbb{E}\liminf_n|X_n|\leq K\), which rules out infinite limits with positive probability. Defining the limit arbitrarily on the exceptional null set yields an integrable random variable \(X\) with \(\mathbb{E}|X|\leq K\). ◻

The proof of Doob’s martingale convergence theorem controls repeated crossings between two levels. Fix real numbers \(a<b\). Define successive entrance times by

\[\tau_1=\inf\{j\geq1:X_j\leq a\},\qquad \tau_{2r}=\inf\{j>\tau_{2r-1}:X_j\geq b\},\qquad \tau_{2r+1}=\inf\{j>\tau_{2r}:X_j\leq a\}.\]

The infimum of an empty set is infinity, and every later entrance time is then infinity. These are stopping times: each decision at time \(j\) depends only on the observations through \(j\). The number of completed upcrossings by time \(n\) is \(\mathscr U_n=\sum_{r\geq1}\mathbb1_{\{\tau_{2r}\leq n\}}\). It increases with \(n\). The interval \([a,b]\) consists of values of the process, not observation times.

For an equivalent predictable description, put \(U_1=0\) and, for \(k\geq2\), let \(U_k\) indicate whether the increment from \(k-1\) to \(k\) is taken during an unfinished crossing:

\[U_k=\begin{cases} 0,&U_{k-1}=0,\ X_{k-1}>a,\quad\text{or }U_{k-1}=1,\ X_{k-1}\geq b,\\ 1,&U_{k-1}=0,\ X_{k-1}\leq a,\quad\text{or }U_{k-1}=1,\ X_{k-1}<b. \end{cases}\]

Lemma 12 (Upcrossing Inequality). The number of upcrossings of a submartingale \(\{X_n, \mathscr F_n\}_{n \in \mathbb N}\), denoted \(\mathscr U_n\) as above, satisfies

\[\mathbb{E} \mathscr U_n \leq \frac{\mathbb{E} |X_n| + |a|}{b-a} ~.\]

Proof. Let \(Y_k=(X_k-a)^+\) and \(c=b-a\). Convexity and monotonicity show that \(Y\) is a nonnegative submartingale; this is the second part of Proposition 8. The same entrance times describe crossings of \(Y\) from zero to at least \(c\). The gain

\[G_n=\sum_{k=2}^n U_k(Y_k-Y_{k-1})\]

is at least \(c\mathscr U_n\) on every path. Each completed holding interval begins at zero and ends at a value at least \(c\); any unfinished interval also has nonnegative gain. Moreover, \(U_k\) is \(\mathscr F_{k-1}\)-measurable, so the submartingale property gives

\[\mathbb{E}\sum_{k=2}^n(1-U_k)(Y_k-Y_{k-1})\geq0. \qquad \mathbb{E} G_n\leq\mathbb{E}(Y_n-Y_1)\leq\mathbb{E} Y_n.\]

Consequently \((b-a)\mathbb{E}\mathscr U_n\leq\mathbb{E}(X_n-a)^+\leq\mathbb{E}|X_n|+|a|\), which proves the bound. ◻

Lemma 13. Let \(Z\) be an integrable random variable, and \(\mathscr F_n\) be arbitrary \(\sigma\)-fields. Then, the random variables \(\mathbb{E}[Z \mid \mathscr F_n]\) are uniformly integrable.

Proof. Put \(W_n=\mathbb{E}[Z\mid\mathscr F_n]\), \(A_{n,K}=\{|W_n|>K\}\), and \(M=\mathbb{E}|Z|\). Conditional Jensen and Markov give \(\mathbb{E}|W_n|\leq M\) and \(\mathbb{P}(A_{n,K})\leq M/K\). Since \(A_{n,K}\in\mathscr F_n\), for every \(R>0\),

\[\mathbb{E}[|W_n|1_{A_{n,K}}]\leq\mathbb{E}[|Z|1_{A_{n,K}}]\leq\mathbb{E}[|Z|1_{\{|Z|>R\}}]+\frac{RM}{K}.\]

The bound is uniform in \(n\). First let \(K\to\infty\), then \(R\to\infty\); integrability of \(Z\) makes the last tail expectation tend to zero. This proves uniform integrability without any nesting assumption on the sigma-fields. ◻

For martingales formed from conditional expectations of one integrable variable, uniform integrability identifies the limiting value. Let \(\{\mathscr F_n\}_{n \in \mathbb N}\) be \(\sigma\)-fields such that \(\mathscr F_1 \subset \mathscr F_2 \subset \cdots\). We write \(\mathscr F_n \uparrow \mathscr F_\infty\) if \(\bigcup_{n \in \mathbb N} \mathscr F_n\) generates the \(\sigma\)-field \(\mathscr F_\infty\).

Proposition 14. Suppose \(\mathscr F_n \uparrow \mathscr F_\infty\) and \(Z\) is an integrable random variable. Then, \(\mathbb{E}[Z \mid \mathscr F_n] \to \mathbb{E}[Z \mid \mathscr F_\infty]\) with probability one.

Proof. The sequence \(W_n=\mathbb{E}[Z\mid\mathscr F_n]\) is a martingale with \(\sup_n\mathbb{E}|W_n|\leq\mathbb{E}|Z|\). Theorem 11 gives an almost-sure limit \(W\), chosen \(\mathscr F_\infty\)-measurable. Lemma 13 and almost-sure convergence also give \(\mathbb{E}|W_n-W|\to0\): clip both variables to \([-K,K]\), apply dominated convergence to their clipped difference, and control both clipping errors by uniform integrability and Fatou.

For \(A\in\mathscr F_m\) and every \(n\geq m\), \(\mathbb{E}[W_n1_A]=\mathbb{E}[Z1_A]\). Passing to the limit in \(L^1\) proves this identity with \(W\) in place of \(W_n\). The collection of events satisfying \(\mathbb{E}[W1_A]=\mathbb{E}[Z1_A]\) is a Dynkin class, and contains the algebra \(\bigcup_m\mathscr F_m\). The pi-lambda theorem extends the identity to all of \(\mathscr F_\infty\). This is precisely the definition of \(W=\mathbb{E}[Z\mid\mathscr F_\infty]\). The convergence holds both almost surely and in \(L^1\). ◻