Mixing Conditions and a Nonmixing AR(1) Process
How much independence does a limit theorem need? For a stationary sequence, one possible answer is to require that events far apart become nearly independent. We first quantify this requirement and relate it to the variance of a sample mean. The final example shows why a small covariance and a small mixing coefficient are different conditions.
Consider a stochastic process indexed by some potentially uncountable set \(I\), which we write as \(\{X_i\}_{i \in I}\). The observations are uncorrelated when they satisfy \(\mathop{\mathrm{Cov}}(X_i,X_j) = 0\). Independence implies this equality when the covariances exist, but the converse fails. To recover laws of large numbers and central limit theorems for dependent observations, we need conditions on their joint distributions.
The conditions below measure how much dependence remains between observations separated in the index set \(I\). For example, \(X_i\) is independent of \(X_j\) for some \(j\) sufficiently “far away” from \(i\) (if \(I\) is a subset of \(\mathbb R\)), but this pairwise condition alone does not control the joint distributions needed for limit theorems. The coefficients below compare entire collections of observations. Under mixing, dependence may persist at every finite separation, provided that it tends to zero.
The coefficients must compare whole collections of observations. Pairwise covariances alone would miss dependence visible only in larger collections. We therefore begin with the information generated by an interval of indices.
Let \(\mathcal F_k^n = \sigma(\{X_i : k \leq i \leq n\})\) be the \(\sigma\)-field generated by the random variables \(\{X_i : k \leq i \leq n\}\) (again the index set need not be discrete or countable). \(k\) and \(n\) can take the value \(\mp \infty\), respectively.
We establish long-range independence by examining sets belonging to “far enough away” \(\sigma\)-fields. Recall measurable sets (events) \(A\) and \(B\) are independent if \(\mathop{\mathrm{\mathbb{P}}}(A \cap B) = \mathop{\mathrm{\mathbb{P}}}(A) \mathop{\mathrm{\mathbb{P}}}(B)\) for a probability measure defined on the common measurable space \((\Omega,\mathcal F)\), where \(\mathcal F\) is the smallest \(\sigma\)-field containing \(\{\mathcal F_k^n\}_{k \leq n}\).
Definition 1. Let \(\{X_i:i\in I\}\) be a stochastic process. The \(\alpha\)-mixing coefficient is
A stochastic process \(X\) is strong-mixing if \(\alpha_X(k) \to 0\) as \(k \to \infty\). \(X\) is \(m\)-dependent if \(\alpha_X(k)=0\) for all \(k > m\). \(X\) is stationary if for all \(c,k,j_1,\dots,j_k \in \mathbb N\), the joint distributions of \((X_{j_1},\dots,X_{j_k})\) and \((X_{j_1+c},\dots,X_{j_k+c})\) are identical.
Definition 2. For the remainder of the post the process is indexed by \(\mathbb Z\). For sigma-fields \(\mathcal A,\mathcal B\), define
The last supremum runs over finite measurable partitions of \(\Omega\), with \(A_i\in\mathcal A\) and \(B_j\in\mathcal B\). If no nonconstant square-integrable variables exist, set \(\rho=0\). For each coefficient \(c\in\{\alpha,\phi,\psi,\rho,\beta\}\), its process version is \(c_X(k)=\sup_{t\in\mathbb Z}c(\mathcal F_{-\infty}^t,\mathcal F_{t+k}^{\infty})\). The time supremum is part of every definition.
Lemma 3 ((Doukhan 1994, chap. 1, Proposition 1)). For any \(\sigma\)-fields \(\mathscr A, \mathscr B \in \mathcal F\),
The coefficient comparisons are recorded in Bradley (2005), §1, inequalities (1.10)–(1.13). They imply that convergence to zero of any of the other listed coefficients implies strong mixing.
Some of the comparisons in Lemma 3 are immediate from their finite partitions. For \(A\in\mathcal A\) and \(B\in\mathcal B\), the two-by-two partitions \((A,A^c)\) and \((B,B^c)\) have four absolute covariance entries of the same size, giving \(\beta\ge2\alpha\). For arbitrary partitions, each conditional row has total absolute deviation at most \(2\phi\); averaging the rows gives \(\beta\le\phi\). Also
by applying the definition to both \(B\) and \(B^c\). Finally, the correlation of the indicators of \(A,B\) is their covariance divided by a standard-deviation product at most \(1/4\), giving \(\rho\ge4\alpha\). The two upper bounds for \(\rho\) are the quoted coefficient-comparison results.
The next estimate turns a bound on event probabilities into a bound on covariances. Its successive forms replace boundedness by moment assumptions, at the cost of a smaller power of the mixing coefficient.
Lemma 4 ((Hall and Heyde 1980, Theorem A.5, Corollary A.1, Corollary A.2)). Let \(X,Y\) be random variables defined on \((\Omega,\mathcal F,\mathop{\mathrm{\mathbb{P}}})\). Suppose \(X\) and \(Y\) are \(\mathcal G\)- and \(\mathcal H\)-measurable, respectively, and \(|X| \leq C_1\) and \(|Y| \leq C_2\). Then,
Furthermore, under an additional requirement that \(\mathbb{E}|X|^p < \infty\) for some \(p > 1\), we have
Finally, suppose additionally that \(\mathbb{E}|Y|^q < \infty\) for some \(q > 1\) such that \(\frac1p + \frac1q < 1\), then
Proof.
Write \(\alpha=\alpha(\mathcal G,\mathcal H)\). For bounded variables, shift \(X,Y\) by their lower bounds. The layer-cake representation gives
The integrand is bounded in absolute value by \(\alpha\), proving (1).
For (2), only \(Y\) remains bounded. If \(\alpha=0\), the sigma-fields are independent and the covariance is zero. If \(\|X\|_p=0\), the result is also immediate. Otherwise set \(A=\|X\|_p\alpha^{-1/p}\) and decompose \(X=X_0+X_1\) with \(X_0=X1_{\{|X|\leq A\}}\). Then
For (3), neither variable need be bounded. Let \(r=p/(p-1)<q\) and \(B=\|Y\|_q\alpha^{-1/q}\). Split \(Y=Y_0+Y_1\) at \(B\). The preceding bound and Hölder give
Since \(\|Y_1\|_r\leq B^{1-q/r}\|Y\|_q^{q/r}=\|Y\|_q\alpha^{1-1/p-1/q}\), the two terms add to the asserted constant eight. The zero-norm and independent cases are handled as before.
◻
Lemma 5. For \(X\in L^p(\mathcal G)\), \(Y\in L^q(\mathcal H)\), \(1<p<\infty\), and \(1/p+1/q=1\),
At \(p=1,q=\infty\) the bound is \(2\phi(\mathcal G,\mathcal H)\|X\|_1\|Y\|_\infty\).
Proof.
First take simple \(X,Y\), with level-set partitions \((A_i)\) and \((B_j)\). Write \(a_i=P(A_i)>0\), \(b_j=P(B_j)\), and \(t_{ij}=P(B_j\mid A_i)-b_j\). The total variation bound on each row gives \(\sum_j|t_{ij}|\leq2\phi\): sum the positive entries and use the definition of \(\phi\) on the union of their \(B_j\). Also \(\sum_i a_i|t_{ij}|\leq2b_j\). Weighted Hölder therefore gives, for \(1<q<\infty\),
Pairing \(\sum_jt_{ij}y_j\) with the values of \(X\) and applying Hölder once more proves the stated covariance bound for simple variables. Approximate in \(L^p\) and \(L^q\) to get the general case. For \(p=1,q=\infty\), the row-sum bound directly gives \(|\sum_jt_{ij}y_j|\leq2\phi\|Y\|_\infty\); approximate bounded \(Y\) uniformly and \(X\) in \(L^1\). If \(\phi=0\), all these covariances are zero.
◻
1. Population Mean
For covariance-stationary data, write \(\bar X_n=n^{-1}\sum_{i=1}^nX_i\) and \(R(k)=\operatorname{Cov}(X_0,X_k)\) for \(k\geq0\). Counting pairs at each lag gives
If \(\sum_{k\geq1}|R(k)|<\infty\), dominated convergence yields the finite long-run variance \(\sigma_\infty^2=R(0)+2\sum_{k\geq1}R(k)\). This is a sufficient condition, not a necessary characterization of bounded variances.
For a strictly stationary process bounded by one, Lemma 4 gives \(|R(k)|\leq4\alpha_X(k)\). Thus summability of \(\alpha_X\) suffices. More generally, if \(\mathbb{E}|X_0|^{2+\delta}<\infty\), apply (3) with both moment exponents equal to \(2+\delta\) to obtain
Consequently \(\sum_{k\geq1}\alpha_X(k)^{\delta/(2+\delta)}<\infty\) gives absolute covariance summability. These variance calculations alone do not prove a CLT; the next theorem uses the stronger finite-dependence assumption.
Theorem 6. Suppose \(\{X_i\}_{i \in \mathbb N}\) is a stationary, \(m\)-dependent sequence with mean \(\mu\) and \(\mathop{\mathrm{Var}}(X_1) < \infty\). Then,
where \(\sigma_\infty^2 = \mathop{\mathrm{Var}}(X_1) + 2 \sum_{k \in \mathbb N} R(k)\).
Proof of Theorem 6. Center the process, writing \(Y_i=X_i-\mu\). Because covariances vanish beyond lag \(m\), the preceding variance formula gives \(n^{-1}\operatorname{Var}(\sum_{i=1}^nY_i)\to\sigma_\infty^2\ge0\). If this limit is zero, Chebyshev proves convergence to the degenerate normal law. Suppose first that \(|Y_i|\le C\) and \(\sigma_\infty^2>0\).
Choose \(b_n=\lfloor n^{1/3}\rfloor\). Partition the observations into consecutive retained blocks of length \(b_n\), separated by discarded gaps of length \(m\), and discard the final incomplete block. The retained block sums \(B_{n,j}\) are independent: each block is separated by more than \(m\) from the union of all preceding blocks, so independence follows successively from the sigma-field definition. For any subset \(A\) of indices,
because every summand has at most \(2m+1\) potentially nonzero covariances and each covariance is bounded by \(EY_1^2\). The number of discarded observations is \(O(nm/b_n+b_n+m)=o(n)\). Their sum divided by \(\sqrt n\) therefore tends to zero in \(L^2\). The triangle inequality for \(L^2\) norms now gives
Each \(|B_{n,j}|\le Cb_n=o(\sqrt n)\), so for every fixed \(\varepsilon>0\) the Lindeberg indicators \(\mathbf1\{|B_{n,j}|>\varepsilon s_n\}\) eventually vanish. The independent triangular-array CLT gives \(s_n^{-1}\sum_j B_{n,j}\Rightarrow \mathcal{N}(0,1)\). Restoring the negligible gaps proves the bounded case.
For general \(EY_1^2<\infty\), set \(Y_i^{(K)}=Y_i\mathbf1\{|Y_i|\le K\}-E[Y_i\mathbf1\{|Y_i|\le K\}]\). This is still stationary and \(m\)-dependent, and its CLT follows from the bounded case (including zero limiting variance). The centered tail \(D_i^{(K)}=Y_i-Y_i^{(K)}\) satisfies
Also \(Y_i^{(K)}\to Y_i\) in \(L^2\), so each of the finitely many covariance terms in its long-run variance converges to the corresponding original term. Thus \(\sigma_K^2\to\sigma_\infty^2\). For a bounded Lipschitz test function, the difference between expectations for the original and truncated normalized sums is bounded by its Lipschitz constant times the square root of the preceding tail bound. First let \(n\to\infty\), then \(K\to\infty\), to obtain the claimed normal limit. ◻
A number of other central limit theorem results are in (Hall and Heyde 1980, chap. 5.4).
2. Mixingales
3. Important Counterexamples
3.1. AR(1) process is not strong-mixing (Andrews 1984)
Decay of linear coefficients does not by itself imply strong mixing. Let \((\varepsilon_t)_{t\in\mathbb Z}\) be i.i.d. Bernoulli \(p\) variables with \(0<p<1\), and set
The series converges absolutely, defines a strictly stationary process, and satisfies \(X_t=\rho X_{t-1}+\varepsilon_t\). Nevertheless one future value retains the earlier innovation digits.
Lemma 7. If \(A\in\mathcal F_{-\infty}^t\) has \(P(A)>0\) and there are \(B_s\in\mathcal F_{t+s}^{\infty}\) with \(P(B_s\mid A)=1\) and \(P(B_s)\leq\ell<1\) for every \(s\), then \(\alpha_X(s)\geq P(A)(1-\ell)>0\).
Proof. Use \(P(A\cap B_s)=P(A)\) in the definition of \(\alpha_X(s)\). The uniform strict bound \(\ell<1\) is necessary.
◻
Lemma 8. For each fixed \(s\), the innovation \(\varepsilon_t\) is a Borel function of \(X_{t+s}\) almost surely.
Proof.
For \(0<\rho<1/2\), digit strings are unique. If two strings first differ at position \(j\), the difference of their series has magnitude at least
More constructively, the leading digit is zero if \(x<1\) and one if \(x\geq1\), since a string starting with zero has value at most \(\rho/(1-\rho)<1\). Subtract that digit and divide by \(\rho\) to recover the next digit; iterating gives Borel digit functions.
At \(\rho=1/2\), this is ordinary binary expansion of \(X/2\). Nonuniqueness occurs only at the countably many dyadic values, whose expansions are eventually constant. An i.i.d. Bernoulli sequence with \(0<p<1\) is eventually all zeros or all ones with probability zero. Choose either Borel convention at the exceptional values. The \(s\)th recovered digit of \(X_{t+s}\) is \(\varepsilon_t\) almost surely.
◻
Lemma 9. For this process, \(\alpha_X(s)\geq p(1-p)\) for every positive integer \(s\). In particular it is not strong mixing.
Proof. The event \(A=\{\varepsilon_t=1\}\) belongs to the past sigma-field because \(\varepsilon_t=X_t-\rho X_{t-1}\). Let \(B_s\) be the event that the \(s\)th digit recovered from \(X_{t+s}\) is one. Then \(B_s\) belongs to the future sigma-field and equals \(A\) up to a null set. Thus \(P(A\cap B_s)-P(A)P(B_s)=p-p^2\).
◻