Quadratic Mean Differentiability
In a smooth model, nearby parameter values produce a likelihood ratio with a linear random term and a quadratic drift. Ordinary Taylor expansion suggests this form but may not control the summed remainder. Quadratic-mean differentiability replaces pointwise differentiability of the log density by an integrated expansion of its square root. We begin with normal and exponential-family calculations to identify what this condition must accomplish.
Quadratic-mean differentiability controls an integrated expansion of square-root densities. The contiguity note proves how this condition produces local asymptotic normality, including the score mean, support changes, and the likelihood remainder. Its third-lemma calculation then gives the distribution of a statistic under local alternatives.
For the ensuing discussion, let \(\Theta \subset \mathbb R^d\) be an open subset. Consider a family of parametric models \(\mathcal P = \{P_\theta : \theta \in \Theta\}\), where \(P_\theta\) is defined on a (measurable) sample space \((\mathcal X, \mathscr A)\). Assume \(P_\theta\) is absolutely continuous with respect to a \(\sigma\)-finite measure \(\mu\), i.e. we define \(p_\theta (x) = \frac{\dif P_\theta (x)}{\dif \mu(x)}\). We want to specifically look at smooth parametric family.
For testing a simple null \(\theta_0\) against \(\theta_1\), write \(\mathcal L_N(\theta)=\prod_{i=1}^Np_\theta(X_i)\) and \(\Lambda_N=\log\{\mathcal L_N(\theta_1)/\mathcal L_N(\theta_0)\}\). The Neyman–Pearson test rejects for large \(\Lambda_N\), with boundary randomization when needed to attain its level.
In the normal location model with known \(\sigma^2>0\), the constants in the density cancel, and summing the individual squared differences gives
For a fixed \(\theta_1\ne\theta_0\), the LLN under \(\theta_0\) gives \(\Lambda_N/N\to_P-(\theta_1-\theta_0)^2/(2\sigma^2)\); thus \(\Lambda_N\to_P-\infty\). Fixed alternatives become distinguishable, whereas alternatives at distance \(N^{-1/2}\) produce a nondegenerate local problem.
Substitute \(\theta_1=\theta_0+h/\sqrt N\). Expanding \(\theta_1^2-\theta_0^2=2\theta_0h/\sqrt N+h^2/N\) gives the exact identity
Hence \(\Lambda_N\sim \mathcal{N}(-h^2/(2\sigma^2),h^2/\sigma^2)\), including the degenerate law at zero when \(h=0\). The random term is linear in the local displacement; the deterministic drift is negative and quadratic.
For a natural one-parameter exponential family, take \(p_\theta(x)=\exp\{\theta T(x)-A(\theta)\}\) relative to its fixed base measure, with \(\theta_0\) in the interior of the natural parameter domain. Differentiating the normalizing integral there gives \(E_{\theta_0}T=A'(\theta_0)\) and \(\operatorname{Var}_{\theta_0}(T)=A''(\theta_0)\). The log likelihood ratio is
Taylor expansion of the single deterministic function \(A\) yields
Subtracting cancels the mean of the sum, so
Interior natural parameters provide finite moments locally, justifying both derivatives and the CLT. This repeats the normal calculation with \(A''(\theta_0)\) in place of \(\sigma^{-2}\).
The examples suggest a general question: which integrated smoothness condition guarantees a second-order likelihood expansion? Formally, we seek
It suffices to control square-root densities in the quadratic norm. Squaring the approximation recovers information about density normalization, while the quadratic norm controls the remainder. To that end, define
Here, \(\eta(\cdot, \theta)\) is understood as a \(d\)-dimensional vector of real-valued functions. The remainder measures the error in approximating the square-root density by its linear part. The broad heuristic is it is enough to look at the \(L^2\) norm of this “remainder”, and require that it be “small” as we scale the alternative hypothesis \(h\).
By the heuristic above, a parametric family \(\mathcal P = \{ P_\theta : \theta \in \Theta \}\) is said to be quadratic mean differentiable (qmd) at \(\theta_0\) if there exists a vector of real-valued functions, \(\eta(\cdot, \theta) = (\eta_1(\cdot, \theta), \dots, \eta_d(\cdot,\theta))^\top\), such that
for the fixed dominating measure \(\mu\), as \(|h| \to 0\). This definition is particularly useful because of the following result.
Lemma 1. If the model is QMD at the interior point \(\theta_0\), then \(\eta_j\in L^2(\mu)\), \(\eta_j=0\) almost everywhere on \(\{p_{\theta_0}=0\}\), and the score
has mean zero and finite second moment under \(P_{\theta_0}\).
Proof. Put \(s=\sqrt{p_{\theta_0}}\), and for a fixed direction \(u\), \(s_t=\sqrt{p_{\theta_0+tu}}\). QMD says
Each difference quotient belongs to \(L^2\), so its limit does also. Taking coordinate directions gives \(\eta_j\in L^2\). On \(\{s=0\}\), the quotients for \(t>0\) are nonnegative and those for \(t<0\) are nonpositive. Their common \(L^2\) limit is therefore zero there. Both signs of \(t\) are available because \(\theta_0\) is interior.
Normalization gives \(\|s_t\|_2=\|s\|_2=1\), hence
Cauchy–Schwarz justifies the limit because the first factor converges in \(L^2\) and the second converges to \(2s\) there. Thus \(E_{\theta_0}(u^\top\dot\ell_{\theta_0})=0\). Finally \(E\|\dot\ell_{\theta_0}\|^2=4\int\|\eta\|^2\,d\mu<\infty\). ◻
The information matrix is therefore
Lemma 2. For every \(h\in\mathbb R^d\),
Proof. Expand the square as \(\sum_{i=1}^d\sum_{j=1}^d h_ih_j\eta_i\eta_j\). Each product is integrable by Cauchy–Schwarz, so finite summation and integration commute. The \((i,j)\)-entry of the integral is \(I_{ij}/4\), giving the identity. In particular the information matrix is symmetric and nonnegative definite; invertibility, when used later, makes it positive definite. ◻
Here is a sufficient condition that works in every finite parameter dimension. The differentiability assumption refers to square-root densities, including points where a density vanishes.
Theorem 3. Suppose, outside one fixed \(\mu\)-null set, \(\theta\mapsto s_\theta(x)=\sqrt{p_\theta(x)}\) is continuously differentiable on a neighborhood of \(\theta_0\), with gradient \(\eta(x,\theta)\). Suppose \(I(\theta)=4\int\eta(x,\theta)\eta(x,\theta)^\top\,d\mu\) is finite nearby and continuous at \(\theta_0\). Then the model is QMD at \(\theta_0\), with derivative \(\eta(\cdot,\theta_0)\).
At points with positive density, \(\eta=\nabla p_\theta/(2\sqrt{p_\theta})\); at points with zero density, \(\eta=0\), since the differentiable nonnegative function \(s_\theta(x)\) has a local minimum.
Proof. First the gradients are continuous in \(L^2(\mu;\mathbb R^d)\). For any \(\theta_n\to\theta_0\), pointwise continuity gives \(\eta_n\to\eta_0\), and continuity of the trace of \(I\) gives \(\int\|\eta_n\|^2\to\int\|\eta_0\|^2\). Scheffé's argument applied to these nonnegative squared norms gives their \(L^1\) convergence: write the absolute difference as their sum minus twice their minimum, and use dominated convergence for the minimum. Now \(\|\eta_n-\eta_0\|^2\to0\) pointwise and is bounded by \(2\|\eta_n\|^2+2\|\eta_0\|^2\), which converges in \(L^1\). The generalized dominated-convergence argument gives \(\|\eta_n-\eta_0\|_2\to0\). To see that last argument directly, subtract the integrand from its nonnegative majorant and apply Fatou; the convergent integrals of the majorants force the limsup of the smaller integrals to be zero.
Choose \(h\) so the entire segment \(\theta_0+th\), \(0\le t\le1\), stays in the neighborhood. The fundamental theorem of calculus gives
Jensen, Cauchy–Schwarz, and Tonelli imply
This is exactly QMD. The result is the sufficient condition of van der Vaart, Lemma 7.6; the argument above explicitly supplies the multivariate segment and integrated convergence. ◻
Proposition 4. Let \(X_1,\ldots,X_n\) be iid from \(P_{\theta_0}\), where \(\theta_0\) is interior, the model is QMD there, and \(I=I(\theta_0)\) is nonsingular. Assume on a fixed neighborhood \(U\) that the log densities are finite \(P_{\theta_0}\)-almost surely and
Let \(\hat\theta_n\to_P\theta_0\) be measurable approximate likelihood maximizers satisfying
Likelihood contrasts may be used if individual log-density integrals do not exist. Then
Proof. Write \(P=P_{\theta_0}\), \(m_\theta=\log p_\theta-\log p_{\theta_0}\), \(\Delta_n=\mathbb G_n\dot\ell_{\theta_0}\), and \(\mathbb G_n=\sqrt n(P_n-P)\). QMD and the scalar logarithm expansion show, for every deterministic \(h_n\to h\),
Indeed the square-root ratio satisfies \(\sqrt n\,2(\sqrt{p_{\theta_0+h_n/\sqrt n}/p_{\theta_0}}-1)\to h^\top\dot\ell_{\theta_0}\) in \(L^2(P)\), and \(2\log(1+w/2)=w+O(w^2)\) near zero. The Lipschitz bound gives \(|a_n|\le M\sup_n\|h_n\|\). This square-integrable envelope upgrades convergence in probability to \(L^2(P)\). Therefore
The LAN expansion, proved in the linked contiguity note, and the decomposition \(nP_nm_{\theta_0+h_n/\sqrt n}=\mathbb G_na_n+nPm_{\theta_0+h_n/\sqrt n}\), then give
These deterministic sequential limits yield the second-order population expansion \(Pm_{\theta_0+v}=-v^\top Iv/2+o(\|v\|^2)\): for any sequence \(v\to0\), take \(n\) of order \(\|v\|^{-2}\) and extract a convergent subsequence of its normalized directions. The preceding limit applies to each such subsequence.
Two empirical-process steps are needed before substituting the random fitted direction: root-\(n\) boundedness of a consistent approximate maximizer, and stochastic equicontinuity of the local remainder under the square-integrable Lipschitz envelope. These are the explicit inputs of van der Vaart, Theorem 5.23 (including its differentiability-in-probability footnote), used in the proof of Theorem 5.39. They give \(\hat h_n=\sqrt n(\hat\theta_n-\theta_0)=O_P(1)\) and, for every \(H_n=O_P(1)\),
We retain those two rate/equicontinuity results as cited theorem inputs. The final optimization step is elementary: the quadratic on the right is maximized at \(h_n^*=I^{-1}\Delta_n=O_P(1)\). Both \(\hat\theta_n\) and \(\theta_0+h_n^*/\sqrt n\) lie in \(U\) with probability tending to one. Approximate likelihood maximization and completion of the square give
Positive definiteness forces \(\hat h_n-h_n^*\to_P0\). Lemma 1 and the multivariate CLT give \(\Delta_n\Rightarrow N_d(0,I)\), yielding the assertion. Consistency alone would not suffice: in the normal mean model, \(\bar X_n+n^{-1/4}\) is consistent but has a diverging root-\(n\) error. ◻
Source for the explicit theorem inputs: van der Vaart, Asymptotic Statistics, Theorems 5.23 and 5.39, pp. 53–54 and 65–66; the square-root sufficient condition is Lemma 7.6, pp. 95–96.