Gaussian Process
A Gaussian process is determined by its mean and covariance, and the variance of an increment defines a natural distance between indices. This distance connects probability with geometry: separated indices can force a large supremum, while covers can bound it from above. We begin with independent Gaussian variables, then control the supremum of a separable process by finite approximations.
A stochastic process \(\{X_t\}_{t \in T}\) (with an infinite index set \(T\)) is said to be a centred Gaussian process if \((X_{t_1}, \dots, X_{t_n})\) is centred and jointly Gaussian for all \(n \geq 1\) and \(t_1, \dots, t_n \in T\). Note that a Gaussian process satisfies the equality \(\mathbb{E} e^{\lambda(X_t - X_s)} = e^{\frac{\lambda^2}{2} \mathbb{E}[(X_t-X_s)^2]}\), which gives the canonical metric \(d(s,t) = (\mathbb{E}[(X_t-X_s)^2])^{\frac12}\). Gaussian processes will serve two purposes here: one, it helps us develop lower bounds for chaining; two, comparison arguments transfer some of the resulting bounds to other processes.
To start, we look at the instance when the index set \(T\) is finite. This reduces to studying the case of having finitely many Gaussian random variables. We begin with the following bounds on the supremum of Gaussian random variables.
Lemma 1. Let \(X_1, \dots, X_n\) be i.i.d. Gaussian random variables with mean zero and variance \(\sigma^2\). Then,
Proof. The upper bound follows from Proposition 3. For the lower bound, scale to \(\sigma=1\) and put \(M_n=\max_{i\leq n}X_i\). For \(\eta\geq0\),
Writing \(x=\eta+u\) and using \((\eta+u)^2\leq2\eta^2+2u^2\) gives
For all sufficiently large \(n\), take \(\eta_n=\sqrt{\log(c_0n)}\). Then \(n\Pr(X_1>\eta_n)\geq1\), so independence gives \(\Pr(M_n>\eta_n)\geq1-e^{-1}\). Therefore,
for sufficiently large \(n\). For each of the finitely many remaining \(n\geq2\), \(\mathbb{E} M_n\geq\mathbb{E}\max(X_1,X_2)=1/\sqrt\pi>0\), so reduce the universal constant if necessary. The case \(n=1\) is immediate. Multiplying by \(\sigma\) proves the result. ◻
We next pass from finite collections to infinite index sets \(T\). Separability lets us approximate the supremum by maxima over increasing finite sets. A bound uniform over those sets can then pass to the limit.
Theorem 2. Let \(\{X_t\}_{t \in T}\) be a separable, centred Gaussian process such that \(P (\sup_{t \in T} |X_t| < \infty) > 0\). Let \(f:\mathbb R\to\mathbb R\) be even and convex (hence nondecreasing on \([0,\infty)\)), and let \(g\sim\mathcal{N}(0,1)\). Then,
\(\sigma = \sigma(X) \triangleq \sup_{t \in T} \sqrt{\mathbb{E} X_t^2} < \infty\) and \(\mathbb{E} \sup_{t \in T} |X_t| < \infty\).
The following inequalities hold true:
\[\begin{aligned} \mathbb{E} f \left( \sup_{t \in T} |X_t| - \mathbb{E} \sup_{t \in T} |X_t| \right) &\leq \mathbb{E} f \left( \frac{\pi\sigma}{2} g \right) ~, \\ \mathbf P\,\,\!\!\left( \left| \sup_{t \in T} |X_t| - \mathbb{E} \sup_{t \in T} |X_t| \right| > t \right) &\leq 2 e^{-\frac{t^2}{2\pi^2 \sigma^2}} ~. \end{aligned}\tag{1}\]
Proof. By separability, choose \(t_1,t_2,\ldots\) so that \(S=\sup_{t\in T}|X_t|=\sup_j|X_{t_j}|\) almost surely. The assumption gives finite \(M\) and \(\delta>0\) with \(\Pr(S\leq M)\geq\delta\). Since \(\Pr(|X_t|\leq M)\geq\delta\) for every \(t\), the normal distribution immediately implies \(\sigma=\sup_t(\mathbb{E} X_t^2)^{1/2}<\infty\). If \(\sigma=0\), all claims are immediate.
Put \(S_n=\max_{j\leq n}|X_{t_j}|\). A finite Gaussian vector has a representation \((X_{t_1},\ldots,X_{t_n})=AG\) with \(G\) standard normal. The function \(F(G)=\max_j|(AG)_j|\) is \(\sigma\)-Lipschitz, because every row of \(A\) has Euclidean norm at most \(\sigma\). Gaussian concentration, obtained from the log-Sobolev inequality in Theorem 10, gives
Choose \(r\) so that the right-hand side is less than \(\delta\). Since \(\Pr(S_n\leq M)\geq\delta\), this forces \(\mathbb{E} S_n\leq M+r\). Monotone convergence now gives \(\mathbb{E} S<\infty\), and hence \(S<\infty\) almost surely. In particular, \(S_n\to S\) in \(L^1\). Passing to the limit in the finite-dimensional concentration bound proves the stated tail bound (in fact, it gives the stronger denominator \(2\sigma^2\)).
To compare convex functions of the fluctuation, introduce an independent copy and apply conditional Jensen. Let \(G'\) be an independent standard normal vector. Conditional Jensen gives
Along the Gaussian rotation \(G_u=G\cos u+G'\sin u\), \(0\leq u\leq\pi/2\), write \(H_u=-G\sin u+G'\cos u\). Then \(G_u\) and \(H_u\) are independent standard normal vectors and
The identity holds for Lipschitz \(F\) by approximation, or by absolute continuity along almost every path. Jensen over the interval, evenness of \(f\), and \(\|\nabla F\|\leq\sigma\) give
Conditionally on \(G_u\), the inner product is normal with standard deviation at most \(\sigma\). Apply this to \(S_n\) and then use \(L^1\) convergence and Fatou’s lemma to obtain the comparison for \(S\). ◻
For completeness, the finite-dimensional concentration step can be obtained without assuming differentiability of the maximum. If \(V:\mathbb R^m\to\mathbb R\) is \(\sigma\)-Lipschitz, smooth it by convolution; the smoothed functions remain \(\sigma\)-Lipschitz and converge pointwise. Their at-most-linear growth gives integrability of every exponential under the Gaussian law. For a smooth approximation put \(\Lambda(\lambda)=\log\mathbb Ee^{\lambda V(G)}\). Applying the Gaussian log-Sobolev inequality to \(e^{\lambda V/2}\) gives
Integrate \((\Lambda(\lambda)/\lambda)'\le\sigma^2/2\), starting from \(\Lambda(\lambda)/\lambda\to\mathbb EV\) as \(\lambda\downarrow0\). Thus \(\mathbb Ee^{\lambda(V-\mathbb EV)}\le e^{\lambda^2\sigma^2/2}\). Apply this to \(V\) and \(-V\), optimize the exponential Markov bound, and pass through the smoothing by domination. This proves the two-sided concentration bound used above.
The rotation comparison also passes through smoothing: the gradient bound is preserved, and the finite convex function \(f\) is continuous and bounded below by \(f(0)\). Conditional normality and even convex monotonicity bound the expectation at every smoothing level by the same \(\mathbb Ef((\pi\sigma/2)g)\), possibly infinite. Fatou applied to \(f-f(0)\) passes first to the nonsmooth maximum, and then to the increasing finite maxima \(S_n\), whose means converge by the preceding argument. These passages do not assume in advance that the infinite supremum is integrable.
Proposition 3. Let \(\{X_t\}_{t \in T}\) be a separable, centred Gaussian process such that \(P (\sup_{t \in T} |X_t| < \infty) > 0\). Then, for all \(p \geq 1\),
Proof. Put \(S=\sup_{t\in T}|X_t|\). The preceding convex comparison with \(f(x)=|x|^p\) gives
For a standard normal \(g\), direct integration gives
where the second bound follows from Stirling’s formula, with a fixed constant covering bounded \(p\). Also,
Minkowski’s inequality therefore yields
◻
For lower bounds, a separated set of indices supplies a finite collection of process values with increment variances bounded away from zero. The next step is to compare this collection with independent Gaussian variables. Heuristically, if \(D\) is a maximal \(\varepsilon\)-packing of \(T\) and suppose \(\{X_t\}_{t\in D}\) is set of independent random variables, then, we should expect to see \(\mathbb{E} \sup_{t \in T} X_t \geq \mathbb{E} \sup_{t \in D} X_t \gtrsim \sqrt{\log |D|}\). Here, note a feature about the lower bounds: we require a subset of \(T\) that is well-separated, so as to apply the insights from the development of maximal inequalities. These packings naturally obtained in terms of a packing of \(T\). Note that here, we avoid measurability issues by studying stochastic processes through their finite-dimensional marginals, i.e. \(\mathbb{E} \sup_{t \in T} \: X_t\) ought to be understood formally as \(\sup_{T_0 \subset T} \mathbb{E} \sup_{t \in T_0} X_t\), where the outer supremum is taken over all finite subsets \(T_0\) of \(T\). Let us first develop some tools that will allow us to pursue progress in this direction.