When Pre-Testing Yields Valid Inference
A difference-in-differences estimate is often reported only after a pre-trends test fails to reject. The reported confidence interval is therefore evaluated in a sample selected by the pre-test. Its coverage depends on the joint distribution of the treatment-effect estimate and the estimated pre-trends.
The clever trick in de Chaisemartin and D’Haultfœuille (2024) is how they express coverage and passing the pre-tests as two symmetric convex events in the same centered Gaussian distribution. They apply the Gaussian correlation inequality to bound the joint probability from below, then divide by the probability of passing the pre-tests to establish conditional coverage. Their argument allows arbitrary correlation between the estimation error and the specification-test estimates.
Their result applies under the maintained assumptions that justify the original estimator. In the DID application, these include the parallel-trends restriction identifying the post-treatment effect. The analysis below follows the July 4, 2024 version of the paper.
1. Reporting after a specification test
They consider inference on a parameter \(\beta_0\in\mathbb R^p\) based on an estimator \(\widehat\beta_n\). Write \(\mathcal P_0\) for the set of data distributions under which the maintained model and the stated asymptotic approximation hold. Let \(\mathcal P_1\) be the set of distributions satisfying the tested restrictions. They assume
For example, parallel trends across all dates implies zero pre-treatment differences in trends. The specification tests examine those pre-treatment restrictions. Identification of the treatment effect uses the restriction on the untreated post-treatment outcome as well.
For a fixed number \(J\) of specification tests, let \(T_{j,n}\) be the \(j\)th statistic and \(q_{j,n}\) its critical value. Define the event on which the researcher reports inference by
If \(\mathrm{CR}_n\) is a confidence region, the coverage probability of interest is
Their null analysis holds \(P\in\mathcal P_0\) fixed; the notation below suppresses the subscript \(P\). The procedure, the parameter, and the collection of specification tests are fixed before observing the sample. The statements concern the limit as \(n\to\infty\) at this distribution.
2. Two convex sets in a joint Gaussian distribution
They use the Gaussian correlation inequality of Royen (2014), which states that, for a centered Gaussian probability measure \(\mu\) on a finite-dimensional Euclidean space and symmetric convex Borel sets \(A,B\),
Here \(-A\triangleq\{-a:a\in A\}\) and \(-B\triangleq\{-b:b\in B\}\) are the reflections of the two sets through the origin: every coordinate of every point changes sign. Symmetry means \(A=-A\) and \(B=-B\), so a point belongs to either set exactly when its negative does. For example, reflecting \([1,2]\) gives \([-2,-1]\), while reflecting \([-c,c]\) gives the same interval.
When \(A\) represents an event for a random vector \(X\), its probability is \(\mathbb P(X\in A)\). Reflection gives the event \(\{X\in -A\}=\{-X\in A\}\), with the same interpretation for \(B\). The complement of the event is \(\{X\notin A\}\), written \(\{X\in A^c\}\).
The inequality allows arbitrary correlation among the Gaussian random variables. They apply this result to the joint Gaussian limit of the estimation error and the specification-test estimates, as follows.
Proposition 1. (Conditional Gaussian coverage.) Let \((Z,W)\in\mathbb R^p\times\mathbb R^q\) be jointly centered Gaussian. Let \(C\subseteq\mathbb R^p\) and \(K\subseteq\mathbb R^q\) be symmetric convex Borel sets, with \(\mathbb P(W\in K)>0\). Then
Proof. Let \(\mu\) be the joint law of \((Z,W)\) and define
The two sets are convex and symmetric about zero. Their probabilities satisfy
Substituting into the Gaussian correlation inequality and dividing by \(\mathbb P(W\in K)\) proves the result. \(\square\)
For a standard normal scalar \(Z\), the interval \(C=[-z_{1-\alpha/2},z_{1-\alpha/2}]\) has probability \(1-\alpha\). For \(Z\sim\mathcal{N}(0,I_p)\), the ball
has probability \(1-\alpha\). Here \(z_u\) and \(\chi^2_{p,u}\) denote the \(u\) quantiles of the standard normal and chi-squared distributions. Proposition 1 gives conditional coverage at least \(1-\alpha\) in either case. If \(Z\) and \(W\) are independent, the coverage probability equals \(1-\alpha\).
2.1. Acceptance regions for specification tests
A quadratic statistic \(T(w)=w'Aw\), with \(A\) positive semidefinite, has acceptance region \(\{w:w'Aw\le q\}\). Convexity follows from
for \(0\le\lambda\le1\). The equality \(T(-w)=T(w)\) gives symmetry.
Separate two-sided tests of coordinates produce the box \(\{w:|w_k|\le a_k,\ k=1,\ldots,q\}\). More generally, a test of a linear contrast has an acceptance region \(|r'w|\le a\), which is a strip between two parallel hyperplanes.
Suppose each statistic is a finite-valued convex function \(T_j:\mathbb R^q\to[0,\infty)\) satisfying \(T_j(-w)=T_j(w)\) and \(T_j(0)=0\). Passing every test means
Each sublevel set is convex and symmetric, so their intersection has both properties. This intersection is the object to which Proposition 1 applies. The critical values determine its size and the probability of reporting inference.
3. One pre-test and one estimation error
Let \(Z,W\) have standard normal marginals and correlation \(\rho\), and retain the experiment when \(|W|\le a\), where \(a>0\). For \(|\rho|<1\), write
Let \(\phi\) and \(\Phi\) be the standard normal density and distribution function, and put \(D_a=2\Phi(a)-1\).
Proposition 2. (Conditional mean and variance.) For \(-1\le\rho\le1\),
Proof. The truncated density of \(W\) is symmetric, so its conditional mean is zero. Since \(\phi'(w)=-w\phi(w)\), integration by parts gives
Thus \(\operatorname{Var}(W\mid |W|\le a)=1-2a\phi(a)/D_a\). Independence leaves \(\varepsilon\) standard normal after conditioning on \(|W|\le a\). Applying the decomposition in (6) yields
When \(|\rho|=1\), \(Z=\rho W\) almost surely and the same formulas follow directly. \(\square\)
The term \(\rho W\) is the component of the estimation error shared with the pre-test. Selection restricts this component, reducing the conditional variance by an amount proportional to \(\rho^2\). Coverage can also be calculated directly.
Proposition 3. (Central intervals.) For every \(a,c>0\),
For \(|\rho|<1\), this probability equals
Proof. Conditional on \(W=w\), the variable \(Z\) has law \(\mathcal{N}(\rho w,s^2)\). Its probability of falling in \([-c,c]\) is the expression in square brackets, denoted \(g(w)\). Integration over the truncated density of \(W\) gives the formula.
The function \(g\) is even. Put \(r=|\rho|\). For \(w\ge0\),
because \(|c+rw|\ge|c-rw|\) and \(\phi\) decreases with the absolute value of its argument. Hence
The unconditional expectation is a weighted average of these two quantities. Therefore
If \(|\rho|=1\), the conditional coverage is
It equals one when \(a\le c\) and is at least \(2\Phi(c)-1\) when \(a>c\). \(\square\)
4. Conditional limits and Wald inference
They obtain their asymptotic coverage result by applying the Gaussian argument to the joint limiting distribution. To transfer it to the sample, the joint and marginal probabilities in (2) must converge, and the limiting denominator must be positive. Positive critical values supply both properties in the finite-dimensional setting.
Lemma 4. (Acceptance probability and boundary.) Let \(W\) be a centered Gaussian vector, and let \(T_1,\ldots,T_J\) be finite-valued, convex, even functions on \(\mathbb R^q\) with \(T_j(0)=0\). For \(q_j>0\), define \(K\) as in (5). Then
Proof. Let \(L\) be the linear support of \(W\). On \(L\), a nonconstant Gaussian vector has a strictly positive density with respect to Lebesgue measure of dimension \(\dim L\). Finite convex functions on a Euclidean space are continuous. Since \(T_j(0)=0<q_j\) for every \(j\), a neighborhood of zero in \(L\) lies in the strict acceptance region. Its Gaussian probability is positive.
For any \(j\), a point \(w\in L\) with \(T_j(w)=q_j\) belongs to the boundary in \(L\) of \(K_j=\{w\in L:T_j(w)\le q_j\}\). Indeed, for \(t>1\), convexity and \(T_j(0)=0\) imply
Points outside \(K_j\) therefore approach \(w\) along its ray from zero. The boundary of a convex set with nonempty interior in \(L\) has Lebesgue measure zero in \(L\), and hence Gaussian probability zero. The event in the second conclusion is contained in the union of these \(J\) boundaries. If \(L=\{0\}\), all the statistics equal zero almost surely and the conclusions follow from \(q_j>0\). \(\square\)
Assumption 5. (Joint approximation.) At the maintained distribution, suppose
The displayed covariance matrix is positive semidefinite. For \(j=1,\ldots,J\), assume
where the functions \(T_j\) satisfy the conditions of Lemma 4. All random quantities are measurable.
Theorem 6. (Conditional asymptotic coverage.) Under Assumption 5, for any closed symmetric convex set \(C\subseteq\mathbb R^p\),
Proof. Define
Continuity of the \(T_j\), joint convergence, and Slutsky’s theorem give \((Z_n,M_n)\Rightarrow(Z,M)\). The boundary of \(C\times(-\infty,0]\) is contained in
A convex set’s boundary has Lebesgue measure zero in \(\mathbb R^p\). Since \(Z\sim\mathcal{N}(0,I_p)\), \(\mathbb P(Z\in\partial C)=0\). Lemma 4 gives \(\mathbb P(M=0)=0\) and \(\mathbb P(M\le0)>0\). Convergence on continuity sets therefore yields
Taking the ratio proves the equality. Proposition 1 proves the inequality. \(\square\)
4.1. Estimated covariance matrices
They also allow the covariance matrix to be consistently estimated. Let \(a_n\to\infty\) be a deterministic rate and suppose the joint approximation includes
Assume \(\widehat V_n\) is symmetric and positive definite with probability tending to one. Set
In what follows, quantities involving \(\widehat V_n^{-1}\) or \(\widehat V_n^{-1/2}\) are defined on the event \(\{\widehat V_n\not\succ0\}\), whose probability tends to zero. This allows Corollary 7 to be stated conditional only on \(S_n\).
Corollary 7. (Wald confidence regions and tests.) Suppose (12) and the specification-test conditions of Assumption 5 hold, with \((U,W)\) jointly centered Gaussian. Then the region
has limiting conditional coverage at least \(1-\alpha\). Under \(H_0:\beta_0=b_0\), the test that rejects when the displayed quadratic at \(b_0\) exceeds \(\chi^2_{p,1-\alpha}\) has limiting conditional rejection probability at most \(\alpha\).
Proof. The symmetric inverse square root is continuous at \(V>0\), so Slutsky’s theorem gives a standard normal marginal limit for \(Z_n\) and preserves its joint convergence with \(W_n\). On the samples where \(\widehat V_n\) is positive definite, the coverage event is
Apply Theorem 6 to the centered ball \(C=\{z:z'z\le\chi^2_{p,1-\alpha}\}\). The exceptional samples have conditional probability bounded by their unconditional probability divided by \(\mathbb P(S_n)\), which tends to a positive number. Their contribution vanishes. Under the null, rejection is the complement of the coverage event. \(\square\)
For \(a_n=\sqrt n\), the normalization is \(a_n^2=n\). In the scalar case, the interval is
A normalization by estimated standard errors can also be imposed directly through a joint limit for the studentized estimates. For local-polynomial RD estimators, this requires a joint Gaussian approximation with smoothing biases negligible after normalization or removed by a justified bias correction. Covariate-continuity and density-continuity estimates can enter \(W_n\) at their respective rates.