Contiguity, Change of Measure, and M-Estimation
Contiguity becomes useful when the data-generating law changes with the sample size. A calculation made under one law can then remain valid under nearby alternatives, even though the limiting distribution of the statistic changes. We will keep the two tasks separate: transferring a negligible error and calculating a new limiting law.
We will start with a calibration problem, prove the three Le Cam lemmas, and then prove the argmax theorem. The last theorem answers a different question: how convergence of an entire objective function determines the limit of its maximizer. Together, these results explain how to derive an estimator’s law under local alternatives without repeating its analysis from the beginning.
1. A calibration error that consistency cannot see
Suppose a scale is used \(n\) times to weigh an object whose true weight has already been subtracted. The readings are independent with variance one. Under correct calibration their mean is zero; under a small calibration error their mean is \(h/\sqrt n\). Write the laws of the whole sample as
Under both laws, \(\overline X_n\to0\) in probability. Consistency therefore gives the same answer in the two experiments. But the uncertainty in the average is also of order \(n^{-1/2}\). On that scale,
For example, the level-\(\alpha\) test that rejects when \(Z_n>z_{1-\alpha}\) has power
Here \(\Phi\) is the standard normal cdf and \(z_{1-\alpha}=\Phi^{-1}(1-\alpha)\). The calibration error goes to zero, but its effect on rejection probability does not.
The normal calculation identifies the scale at which a vanishing calibration error still affects power. In a more complicated problem, we may have proved an expansion only under \(P_n\),
and want the law of \(T_n\) under \(Q_{n,h}\). There are two separate tasks. We must show that the remainder still vanishes under the alternative, and then find the alternative law of the leading term. Contiguity handles the first task; likelihood-ratio tilting handles the second.
1.1. The sample likelihood ratio locates the relevant scale
For a general shift \(\delta_n\), put \(a_n=\sqrt n\,\delta_n\). Subtracting the two normal log densities gives
For \(a_n=h\), \(\mathbb{E}_{P_n}L_n^2=e^{h^2}\). Cauchy–Schwarz consequently gives, for every sample event \(A_n\),
The reverse likelihood ratio has the same second moment under \(Q_{n,h}\), so the implication works in both directions. Any event becoming negligible under one experiment becomes negligible under the other. This remains true for bounded sequences \(a_n\), even if they do not converge.
To see why this scale is special, compare the total variation distance at smaller and larger displacements. With total variation defined as \(\sup_A|Q_n(A)-P_n(A)|\), the likelihood-ratio rejection region gives
For \(a_n>0\), the maximizing event is \(\{Z_n>a_n/2\}\): subtract its probabilities under \(\mathcal{N}(a_n,1)\) and \(\mathcal{N}(0,1)\). Symmetry gives the same formula when \(a_n<0\).
If \(a_n\to0\), the two experiments become close even in total variation. If \(a_n\to h\ne0\), their distance stays positive, but their negligible events agree. If \(|a_n|\to\infty\), the event \(\{\operatorname{sgn}(a_n)Z_n>|a_n|/2\}\) has probability tending to zero under \(P_n\) and to one under \(Q_n\). Thus an alternative can be too close to detect, locally distinguishable, or consistently distinguishable, depending on its size relative to sampling error.
2. General experiments and the direction of contiguity
Let \(P_n\) and \(Q_n\) be probability measures on \((\Omega_n,\mathcal A_n)\). The sample spaces may vary with \(n\); no independence, density model, or Euclidean parameter is required here.
Definition 1. We write \(Q_n\lhd P_n\) if, for every sequence \(A_n\in\mathcal A_n\),
The sequences are mutually contiguous when both \(Q_n\lhd P_n\) and \(P_n\lhd Q_n\) hold.
Keep the direction explicit: a statement proved negligible under \(P_n\) transfers to \(Q_n\) when \(Q_n\lhd P_n\). The definition does not assert that probabilities of ordinary events, or limiting distributions, are equal.
An equivalent form is asymptotic uniform absolute continuity: for each \(\varepsilon>0\) there are \(\delta>0\) and \(n_0\) such that
To prove this from contiguity, suppose it fails. Choose increasing \(n_k\) and events \(A_{n_k}\) with \(P_{n_k}(A_{n_k})<1/k\) but \(Q_{n_k}(A_{n_k})\ge\varepsilon\), and use the empty event at other indices. This contradicts the definition. The converse follows by applying the displayed implication to any sequence whose \(P_n\)-probabilities vanish.
This also explains why absolute continuity separately at each \(n\) is insufficient. The allowed \(\delta\) could deteriorate with \(n\). In the calibration example, normal sample laws are mutually absolutely continuous even when \(|a_n|\to\infty\) and the experiments separate completely.
Conversely, contiguity does not require finite-sample absolute continuity. On \(\{0,1\}\), take \(P_n=\delta_0\) and \(Q_n=(1-1/n)\delta_0+n^{-1}\delta_1\). Their total variation distance is \(1/n\), so they are mutually contiguous, although \(Q_n\not\ll P_n\) for any finite \(n\).
2.1. Likelihood ratios when singular parts are present
Recall that \(Q\ll P\) means \(P(A)=0\) implies \(Q(A)=0\) for every measurable \(A\). Two measures are mutually singular if there is a measurable set carrying all of one measure and none of the other. A pair of measures need be neither absolutely continuous in a given direction nor mutually singular; this is why we keep both parts of the decomposition below.
Set \(\mu_n=P_n+Q_n\), \(p_n=dP_n/d\mu_n\), and \(q_n=dQ_n/d\mu_n\). Define
where the displayed ratios are set to zero when their denominators vanish. These are derivatives of the absolutely continuous parts. In particular, \(R_n\) need not equal \(1/L_n\) on the singular pieces.
For any measurable \(A\),
Indeed, the first term is \(\int_{A\cap\{p_n>0\}}q_n\,d\mu_n\); the second accounts for the rest. The first measure is absolutely continuous with respect to \(P_n\), and the second is concentrated on a \(P_n\)-null set, giving the Lebesgue decomposition. More generally, for bounded measurable \(f\),
The final term has absolute value at most \(\|f\|_\infty s_n\). We will retain it in the proofs. Dropping it without an absolute-continuity assumption loses part of the alternative law. The measure-theoretic background is developed in the Radon–Nikodym note.
3. Le Cam’s first lemma: when likelihood limits retain all the mass
Lemma 2. For the experiments and ratios above, the following statements are equivalent.
\(Q_n\lhd P_n\).
\(s_n\to0\) and the likelihood ratios are uniformly integrable under \(P_n\):
\[ \lim_{K\to\infty}\sup_n \mathbb{E}_{P_n}[L_n\mathbf1_{\{L_n>K\}}]=0. \]Every subsequential distributional limit \(L\) of \(L_n\) under \(P_n\) satisfies \(\mathbb{E}L=1\).
Every subsequential distributional limit \(R\) of \(R_n\) under \(Q_n\) satisfies \(\Pr(R>0)=1\).
For every sequence of measurable real statistics \(T_n\), \(T_n=o_{P_n}(1)\) implies \(T_n=o_{Q_n}(1)\).
The quantifiers in 3 and 4 concern every convergent subsequence.
The likelihood ratio tells us how much alternative probability is attached to a sample that is rare under the reference law. There are two ways probability can be lost from a proposed likelihood limit: a nonvanishing singular part, or increasingly large ratios on increasingly rare events. Statement 2 rules out both.
When \(P_n\) and \(Q_n\) are mutually absolutely continuous at every \(n\), apply statement 2 in each direction: mutual contiguity is equivalent to uniform integrability of \(L_n\) under \(P_n\) and \(L_n^{-1}\) under \(Q_n\).
3.1. Proof: contiguity and uniform integrability
Assume 1. Since \(P_n(p_n=0)=0\), contiguity gives \(s_n\to0\). If uniform integrability failed, there would be an \(\varepsilon>0\), increasing indices \(n_k\), and thresholds \(K_k\to\infty\) such that
The indices can be chosen increasing because each individual \(L_n\) is integrable. Markov’s inequality gives \(P_{n_k}(L_{n_k}>K_k)\le1/K_k\), whereas change of measure gives \(Q_{n_k}(L_{n_k}>K_k)\ge\varepsilon\). Taking empty events off this subsequence contradicts contiguity. Thus 1 implies 2.
Conversely, split a sample event according to whether \(L_n\le K\):
If \(P_n(A_n)\to0\), first let \(n\to\infty\) with \(K\) fixed, then let \(K\to\infty\). Statement 2 makes the right side vanish, proving 1.
3.2. Proof: why the limiting expectation must be one
The ratios are tight under \(P_n\) because \(P_n(L_n>K)\le1/K\). Suppose along some subsequence \(L_n\Rightarrow L\). For each fixed \(K\), the bounded continuous truncation gives
Under 2, the omitted tails vanish uniformly and \(\mathbb{E}_{P_n}L_n=1-s_n\to1\). Letting \(K\to\infty\) therefore yields \(\mathbb{E}L=1\). This proves 2 implies 3.
For the converse, first show \(s_n\to0\). Otherwise some subsequence has \(s_n\ge\varepsilon\). Tightness supplies a further weakly convergent subsequence, and truncation implies
contrary to 3. Thus all the means tend to one. Along any weakly convergent subsequence, 3 and the truncation identity now give
Since \(x\mathbf1_{\{x>2K\}}\le2(x-K)_+\) for \(x\ge0\), this implies uniform integrability along that subsequence; its finitely many initial terms cause no problem. If the entire sequence were not uniformly integrable, we could choose offending indices and thresholds as in the preceding proof and then extract a weakly convergent subsequence. It would still have the offending tails, contradicting what we just proved. Hence 3 implies 2.
For a concrete failure, let \(P_n(\{1\})=1/n\) on \(\{0,1\}\) and let \(Q_n=\delta_1\). Then \(L_n=n\mathbf1_{\{1\}}\) has expectation one for every \(n\), but converges to zero in \(P_n\)-probability. All the likelihood mass sits on an event whose reference probability vanishes. Expectations equal to one before taking limits are therefore not enough.
3.3. Proof: the reverse ratio and negligible statistics
Assume 1 and let \(R_n\Rightarrow R\) under \(Q_n\) along a subsequence. Given \(\varepsilon>0\), use asymptotic uniform absolute continuity to choose \(\delta>0\). For \(0<t<\delta\), the event
has \(P_n\)-probability at most \(t\), because \(p_n<tq_n\) there. Thus \(Q_n(R_n<t)=Q_n(B_n(t))<\varepsilon\) eventually. The open-set part of the portmanteau theorem gives \(\Pr(R<t)\le\varepsilon\), so \(\Pr(R=0)=0\). This proves 4.
Conversely, suppose 4 holds and \(P_n(A_n)\to0\). If \(Q_n(A_n)\) failed to vanish, choose a subsequence bounded away from zero. Since \(\mathbb{E}_{Q_n}R_n\le1\), extract a further subsequence with \(R_n\Rightarrow R\) under \(Q_n\). For every \(t>0\),
The closed-set inequality bounds the limit superior by \(\Pr(R\le t)\). Letting \(t\downarrow0\) makes this zero by 4, a contradiction.
Finally, 1 implies 5 by applying contiguity to \(\{|T_n|>\varepsilon\}\) for each \(\varepsilon>0\). Taking \(T_n=\mathbf1_{A_n}\) proves the converse. The same argument applies to norms of vector-valued remainders. This completes all the equivalences.
3.4. The lognormal criterion
If, under \(P_n\),
then \(L_n\Rightarrow e^Z\), and
A Gaussian log likelihood has the required exponential moment. The mean-one characterization gives \(Q_n\lhd P_n\). Applying the positivity characterization with the roles of \(P_n\) and \(Q_n\) interchanged gives \(P_n\lhd Q_n\). This includes \(\sigma^2=0\), where the limit is the constant one. The negative half-variance term is precisely what normalizes the limiting likelihood ratio.
4. Le Cam’s second lemma: from square-root likelihoods to log likelihoods
The next issue is how to obtain that lognormal likelihood limit. Ordinary Taylor expansion of \(\log p_\theta\) can demand derivatives or domination conditions that are unnecessary. Square roots of densities already belong to an \(L^2\) space, which gives a more flexible expansion.
We prove a Lindeberg form of the second lemma for independent, non-identically distributed observations. Its assumptions explicitly control the small increments and any singular mass. The general uniformly asymptotically negligible formulation and its truncation proof are given in Wellner’s Chapter 3, Lemma 3.3. The version below includes the quadratic-mean differentiable i.i.d. application, which we then derive.
Lemma 3. Let \(P_n=\bigotimes_{i=1}^{k_n}P_{ni}\) and \(Q_n=\bigotimes_{i=1}^{k_n}Q_{ni}\), with coordinate densities \(p_{ni},q_{ni}\) relative to a common dominating measure for each coordinate. Under \(P_n\), define
Suppose, for some finite \(\sigma^2\ge0\),
and, for every \(\varepsilon>0\),
Then, writing \(W_n=\sum_iT_{ni}\) and \(L_n=dQ_n^a/dP_n\),
Consequently \(P_n\) and \(Q_n\) are mutually contiguous.
The product likelihood and its singular part can be identified explicitly. The set \(A_n=\bigcap_i\{p_{ni}>0\}\) has full \(P_n\)-probability. On it the derivative of the absolutely continuous part of \(Q_n\) is \(\prod_i(q_{ni}/p_{ni})\); the remaining part is supported on the \(P_n\)-null complement. Independence under \(Q_n\) gives
This proves that coordinate support changes cannot leave a nonvanishing singular part in the product experiment. It also justifies the product formula for \(L_n\) used in the logarithm calculation.
4.1. Proof: normalization determines the first drift term
All expectations in this proof are under \(P_n\). The definition gives
Thus \(\sum_i\mathbb{E}T_{ni}\to-\sigma^2/4\). We also need the variance of the centered sum. For any \(\varepsilon>0\),
Taking the limit superior and then \(\varepsilon\downarrow0\) shows that the largest second moment vanishes. Since \(\max_i s_{ni}\le\sum_i s_{ni}\to0\), the largest absolute mean vanishes as well. Moreover,
The last sum is bounded by the normalization identity. Therefore \(\sum_i\operatorname{Var}(T_{ni})\to\sigma^2\). The centered variables satisfy Lindeberg’s condition: eventually their mean shifts are smaller than \(\varepsilon/2\), and
The triangular-array CLT now gives the stated limit of \(W_n\). When \(\sigma^2=0\), the same conclusion follows directly from Chebyshev’s inequality.
4.2. Proof: the quadratic sum is stable and the largest increment vanishes
For fixed \(\varepsilon>0\), truncate the squares at \(\varepsilon\). Independence gives
The expected discarded sum tends to zero by Lindeberg’s condition. Thus, for any \(\eta>0\), Chebyshev and Markov bound the limit superior of \(\Pr(|\sum_iT_{ni}^2-\sigma^2|>\eta)\) by a constant times \(\varepsilon^2\sigma^2/\eta^2\). Letting \(\varepsilon\downarrow0\) proves
Also, by a union bound,
Both estimates are needed for the logarithm expansion: the sum of squares supplies the quadratic drift, while the maximum bound puts every increment in a common Taylor neighborhood.
4.3. Proof: the logarithm supplies the second drift term
For \(|t|\le1\), Taylor’s formula gives
With probability tending to one all the \(T_{ni}\) lie in this range. On that event,
In particular, zeros of the likelihood ratio cause no difficulty on this event. The quadratic term contributes another \(-\sigma^2/4\), giving the total drift \(-\sigma^2/2\). The first lemma’s lognormal criterion completes the proof.
5. How quadratic-mean differentiability produces LAN
Let \(\theta_0\) be an interior point of a parameter set in \(\mathbb R^d\), and let \(p_\theta\) be densities relative to a fixed measure \(\mu\). The model is differentiable in quadratic mean at \(\theta_0\) if there is a score \(\ell\in L^2(P_{\theta_0})^d\) such that
The derivative is taken in an integrated square-root metric. Pointwise derivatives of the density are one way to verify it, but are not part of the definition. See the companion notes on quadratic-mean differentiability and local asymptotic normality.
Theorem 4. Under this assumption, \(\mathbb{E}_{\theta_0}\ell=0\). Put \(I=\mathbb{E}_{\theta_0}[\ell\ell^\top]\). For each fixed \(h\in\mathbb R^d\), with observations i.i.d. under \(P_{\theta_0}\),
The two product laws are mutually contiguous. Positive definiteness of \(I\) is not needed for this conclusion; it is needed later when we invert \(I\).
5.1. Proof: support changes, means, and the \(L^2\) remainder
Write \(p_0=p_{\theta_0}\), \(p_t=p_{\theta_0+t}\), and
On \(\{p_0=0\}\) the entire QMD integrand is \(p_t\). On its complement, divide the square-root expansion by \(\sqrt{p_0}/2\). Consequently,
The normalization identity from the second lemma gives \(\mathbb{E}_0T_t=-\mathbb{E}_0T_t^2/4-s_t=O(\|t\|^2)\). On the other hand, Cauchy–Schwarz applied to the QMD remainder gives \(\mathbb{E}_0T_t=t^\top\mathbb{E}_0\ell+o(\|t\|)\). Let \(t\) approach zero along each coordinate direction. Comparing the two equations proves \(\mathbb{E}_0\ell=0\).
Now fix \(h\) and take \(t=h/\sqrt n\). With \(V_n=\sqrt n\,T_{h/\sqrt n}\) and \(V=h^\top\ell\) as functions of one observation, we have \(V_n\to V\) in \(L^2(P_{\theta_0})\). In particular the squares are uniformly integrable. One can see the required tail estimate directly. For fixed \(A\),
In the second term split into \(\{|V|>A/2\}\) and \(\{|V_n-V|>A/2\}\). The first part has arbitrarily small expectation for large \(A\); for fixed \(A\) the second vanishes as \(n\to\infty\), by \(L^2\) convergence and integrability of \(V^2\). Finitely many initial \(n\) have integrable tails as well. Hence, for each \(\varepsilon>0\),
Also \(ns_{h/\sqrt n}\to0\) and \(n\mathbb{E}_0T_{h/\sqrt n}^2\to h^\top I h\). These verify every assumption of the second lemma.
To identify the random linear term, let
Independence and QMD imply
It follows that \(W_n=h^\top\Delta_n-h^\top I h/4+o_P(1)\). The second lemma subtracts the other quarter, and the multivariate CLT gives the law of \(\Delta_n\). This proves the expansion and contiguity.
The result explains why smooth experiments have a Gaussian local approximation. For a Gaussian observation \(\Delta\sim\mathcal{N}_d(0,I)\), tilting by \(\exp(h^\top\Delta-h^\top I h/2)\) changes its mean to \(Ih\). The likelihood of the original sample has precisely this form to first order. The statement proved here is for fixed \(h\); it does not by itself give a remainder uniform over a compact collection of \(h\)’s.
6. Le Cam’s third lemma: transfer the distribution by tilting
Lemma 5. Suppose \(Q_n\lhd P_n\), and let \(X_n\) take values in a metric space \(S\). Assume ordinary weak convergence of measurable random elements,
Then
defines a probability measure on \(S\), and \(X_n\xRightarrow[Q_n]{}\nu\).
The first lemma gives \(L\ge0\) and \(\mathbb{E}L=1\). Thus the formula is a probability measure: countable additivity follows from monotone convergence. It weights a limiting sample by its limiting likelihood ratio.
6.1. Proof with the truncation step included
Take bounded continuous \(f:S\to\mathbb R\). For fixed \(K\), the function \((x,l)\mapsto f(x)(l\wedge K)\) is bounded and continuous on \(S\times[0,\infty)\), so
The exact change-of-measure formula gives the bound
By the first lemma, \(s_n\to0\) and the tail expectation is uniformly small for large \(K\). The corresponding limiting error is at most \(\|f\|_\infty\mathbb{E}(L-K)_+\), which also vanishes. Let \(n\to\infty\), then \(K\to\infty\), to obtain
This is the bounded-continuous-function characterization of weak convergence, completing the proof. Joint convergence is essential: the marginal limit of \(X_n\) alone says nothing about which values receive the larger likelihood weights.
6.2. Calculate the Gaussian mean shift
The most frequently used form has a jointly normal statistic and log likelihood. Suppose
The first lemma supplies mutual contiguity. To calculate the tilted law of \(T\), evaluate its characteristic function:
One can evaluate that transform even with singular covariance without a density or an inverse matrix. Represent \(T=m+AG\) and \(Z=-\sigma^2/2+b^\top G\), with \(G\) a vector of independent standard normals, \(AA^\top=\Sigma\), \(Ab=c\), and \(b^\top b=\sigma^2\). For a real pair \(u,v\), direct Gaussian integration gives \(\mathbb E e^{(u+iv)G_1}=\exp((u+iv)^2/2)\). Multiplying these one-dimensional transforms gives
which simplifies to the displayed characteristic function. The exponential is integrable because its modulus is \(e^Z\), whose expectation is one.
The first equality follows by writing the jointly normal vector as an affine transformation of independent standard normals and using their exponential transform; no invertibility of its covariance matrix is needed. Therefore
The shift is the covariance with the log likelihood, not its standard deviation. If a statistic is asymptotically uncorrelated with the score in a particular direction, its Gaussian limit cannot detect that local direction at first order.
For instance, suppose the earlier expansion has \(\mathbb{E}_0\psi=0\) and \(\mathbb{E}_0\|\psi\|^2<\infty\). Under the QMD alternatives, the joint CLT for \(\psi\) and \(\ell\) gives
Here contiguity transfers the \(o_P(1)\) remainder, and the third lemma shifts the leading term. In the scale-calibration example, \(\psi(x)=\ell(x)=x\), so \(C=V=1\) and the shift is exactly \(h\). The same calculation underlies local power and the definition of regular estimators.
7. Why maximizers need a separate convergence theorem
Suppose an estimator is defined by maximizing \(M_n(h)\). Knowing the limit of \(M_n(h)\) at each fixed \(h\) does not control an optimization over all \(h\): the maximizer can seek out a location where the approximation is poor. The argmax theorem combines uniform control on compact sets with a condition preventing the maximizer from escaping those sets.
First consider a deterministic compact metric space \(K\). Let \(M\) be upper semicontinuous and have a unique maximizer \(h^*\). Upper semicontinuity means that \(h_j\to h\) implies \(\limsup_jM(h_j)\le M(h)\). For \(\varepsilon>0\), the closed set
is compact. If nonempty, \(M\) attains its supremum there: take a maximizing sequence, extract a convergent subsequence, and use upper semicontinuity. Uniqueness therefore gives the strictly positive gap
If \(\|M_n-M\|_K\le\delta\) and \(M_n(\widehat h_n)\ge\sup_K M_n-r_n\), then
Thus \(2\delta+r_n<\gamma_\varepsilon\) forces \(d(\widehat h_n,h^*)<\varepsilon\). This gap argument explains the theorem. A random limiting objective requires a distributional version of it, and a noncompact space requires localization.
8. The argmax theorem on a metric space
Theorem 6. Let \(H\) be a metric space and \(M_n,M\) real-valued stochastic processes on \(H\). Assume the restrictions to compact sets are measurable random elements of \(\ell^\infty(K)\) and
Suppose that almost every path of \(M\) is upper semicontinuous and has a unique global maximizer \(h^*\). Assume \(h^*\) is measurable and has a tight law. Let \(\widehat h_n\) be measurable \(H\)-valued random elements satisfying
Assume also compact containment: for each \(\eta>0\) there is a compact \(K_\eta\subset H\) such that
Then \(\widehat h_n\Rightarrow h^*\) in \(H\).
Here \(\ell^\infty(K)\) consists of bounded functions on \(K\) with norm \(\|f\|_K=\sup_{h\in K}|f(h)|\). Convergence in this space is stronger than convergence of finite-dimensional distributions. We state the measurable version so that every probability in the proof is an ordinary probability. The outer-probability extension used for nonmeasurable empirical processes requires the corresponding weak-convergence framework; the measurability hypotheses must be replaced by the corresponding outer-probability assumptions.
8.1. Proof, step 1: localize both maximizers
Fix a closed set \(F\subset H\) and \(\eta>0\). Compact containment gives a compact set controlling \(\widehat h_n\); tightness gives another controlling \(h^*\). Their union is compact, so there is a single nonempty compact \(K\) such that
It is enough to control \(\Pr(\widehat h_n\in F\cap K)\), because
If \(F\cap K\) is empty, its probability is zero and the desired bound follows immediately. In the rest of the proof assume it is nonempty.
8.2. Proof, step 2: compare suprema through a continuous map
For any nonempty subset \(A\subset K\) and bounded functions \(f,g\),
Indeed \(f(h)\le g(h)+\|f-g\|_K\) for every \(h\in A\); take suprema, then interchange \(f\) and \(g\). Therefore
is a \(2\)-Lipschitz real-valued function on \(\ell^\infty(K)\). The continuous mapping theorem yields
Only the two supremum functionals enter this step; continuity of the argmax map or of the paths of \(M_n\) is not required. We apply the continuous mapping theorem to those suprema.
8.3. Proof, step 3: use approximate maximization with a random error
On \(\{\widehat h_n\in F\cap K\}\),
Thus \(D_n\ge-r_n\) on that event. For every fixed \(a>0\),
The last line uses the portmanteau inequality for the closed interval \([-a,\infty)\) and \(r_n\to0\) in probability. Now let \(a\downarrow0\). Continuity of probabilities along decreasing events gives the upper bound \(\Pr(D\ge0)\). No assumption about an atom at zero is needed.
8.4. Proof, step 4: uniqueness makes the limiting gap strict
On the event \(\{h^*\in K\setminus F\}\), \(\sup_K M=M(h^*)\). Since \(F\cap K\) is compact and the limiting path is upper semicontinuous, it attains its maximum on \(F\cap K\). That maximum is strictly below \(M(h^*)\) by uniqueness. Hence \(D<0\) on this event, and
Combining all four steps,
Let \(\eta\downarrow0\). The inequality holds for every closed \(F\), so the portmanteau theorem proves \(\widehat h_n\Rightarrow h^*\). This proves the theorem, including the localization step.
8.5. What breaks if the assumptions are omitted?
Pointwise convergence can miss a moving spike even on a compact space. On \(H=[0,1]\), let \(M(h)=-h^2\), \(c_n=1/2+1/(n+2)\), and
For every fixed \(h\), \(M_n(h)\to M(h)\), since the distinct spike locations visit \(h\) at most once. Yet the maximizers \(c_n\) tend to \(1/2\), while the unique maximizer of \(M\) is zero. The supremum-norm error is always two.
Uniform convergence on each compact also does not prevent escape. On \(H=\mathbb R\), set
Every fixed compact eventually sees exactly \(-h^2\), but the global maximizer is \(n\). Compact containment is therefore a separate assertion to prove, usually through a rate bound or a bound on the objective outside a large ball.
9. Two applications: consistency and a local Gaussian law
For consistency, let \(M\) be deterministic with a well-separated maximizer \(\theta_0\):
If \(\|M_n-M\|_H=o_P(1)\) and \(M_n(\widehat\theta_n)\ge M_n(\theta_0)-r_n\) with \(r_n=o_P(1)\) nonnegative, the deterministic gap calculation gives
This proves consistency in full. Global separation replaces compactness in this formulation. Alternatively, local uniform convergence, upper semicontinuity, uniqueness, and compact containment give consistency through the general argmax theorem. A constant distributional limit is also a probability limit: apply portmanteau to the closed set \(\{\theta:d(\theta,\theta_0)\ge\varepsilon\}\).
For an empirical criterion \(M_n(\theta)=n^{-1}\sum_i m_\theta(X_i)\), the stochastic part of this argument is a uniform law of large numbers. Identification supplies the separation. These are separate pieces of an M-estimator consistency proof.
9.1. Complete the square before taking the argmax
For a limiting distribution, rescale around the true parameter. Suppose on \(H=\mathbb R^d\) the local criterion satisfies, uniformly on each compact set,
with \(I\) positive definite. For log likelihood, \(\mathcal M_n(h)\) is the log likelihood at \(\theta_0+h/\sqrt n\) minus its value at \(\theta_0\). Assume the rescaled estimator \(\widehat h_n=\sqrt n(\widehat\theta_n-\theta_0)\) is an approximate maximizer of this criterion and satisfies compact containment. These assumptions include the needed local process convergence and rate control; fixed-\(h\) LAN alone does not supply them.
The limiting criterion can be written exactly as
It is continuous, tends to \(-\infty\) as \(\|h\|\to\infty\), and has the unique maximizer \(h^*=I^{-1}\Delta\). The argmax theorem therefore gives
We can also recover the stronger linear representation needed for change of measure. Let \(h_n^0=I^{-1}\Delta_n\), which is tight. On a compact set containing both \(h_n^0\) and \(\widehat h_n\), write \(e_n\) for the supremum error in the quadratic approximation. Approximate maximization and the completed square give
The completed square turns the loss in objective value into a squared distance from the quadratic maximizer. Both approximation errors vanish on every fixed compact. Tightness makes the probability of leaving a sufficiently large compact arbitrarily small. Thus \(\widehat h_n=I^{-1}\Delta_n+o_P(1)\).
Under a fixed local alternative \(\theta_0+h_0/\sqrt n\), the first lemma transfers this remainder, and the third lemma changes the score limit to \(\mathcal{N}_d(Ih_0,I)\). Consequently,
This returns to the calibration problem in a general smooth model. The alternative shifts the center by an amount comparable to estimation error. The estimator’s centered local law is recovered by combining the score expansion, contiguity, likelihood tilting, and the geometry of the objective.
10. Sources and further reading
The first and third lemmas and their likelihood-ratio interpretation are treated in (A. W. van der Vaart 1998, vol. 3, chap. 6); the QMD-to-LAN result is in Chapter 7. The metric-space argmax theorem is Theorem 3.2.2 of (Aad W. van der Vaart and Wellner 2000). The proofs here spell out the truncation, singular-mass, and compact-localization steps in the conventions used throughout these notes.
Online author sources are Wellner’s lower-bounds notes, Section 3 and his Delft empirical-process notes, corrected June 1, 2005, Chapter 2, Section 2. The latter gives the compact-set argument behind the argmax theorem. The weak-convergence note supplies the portmanteau and continuous-mapping results used above.