Testing Treatment Effect Heterogeneity
Posted on
This note is for Hypothesis Testing for Treatment Effect Heterogeneity via an Adaptively Augmented Debiased Estimator
Introduction
\[H_0:\tau(x) \text{ is constant}\quad\text{v.s.} \quad H_a:\tau(x) \text{ is not constant}\]- Shen and He (2015) and Shen et al. (2017): reduce the testing for the existence of heterogeneity to testing the number of components in a logistic-normal mixture model and propose an EM-type test with a tractable limiting distribution.
- Fan, Song and Lu (2017): consider the sup-score test with a semiparameteric change-plane model
- Qian et al. (2021): propose marginal screening test with a linear specification for $\tau(x)$. However, the size of the test could be distorted or the power could be compromised if the assumed (semi-)parameteric model is misspecified
- Lee et al. (2017) and Fan et al. (2922): propose doubly-robust-type nonparameteric estimators for the CATE function based on local linear estimation. They then construct a uniform confidence band, which can be used to test the hypothesis.
- Crump et al. (2008): estimate $E[Y^1\mid X=x]$ and $E[Y^0\mid X=x]$ separately using a series expansion and test the hypothesis by examining the difference in these quantities.
- estimate $E[Y^1\mid X=x]$ and $E[Y^0\mid X=x]$ separately using a series expansion and test the hypothesis by examining the difference in these quantities.
- may suffer from the curse of dimensionality in $X$. They do not cover cases in which the outcome variable is right-censored.
- Sant’Anna (2021): propose test statistics based on Kaplan-Meier integrals; however, under a stringent condition on dependency between the outcome variable of interest and the censoring variable.
- A bootstrap-based test proposed by Sun et al. (2022) also accommodates right-censored outcomes with possibly high-dimensional covariates, but it requires an a priori partition of the covariate space and is designed for independent censoring, which is stronger than the current paper’s assumption on censoring.
Reformulate the hypothesis:
\[H_0:\Var(\tau(X)) = 0\quad \text{v.s.} \quad H_a: \Var(\tau(X)) > 0\]construct a Wald-type test statistics based on an estimator of $\Var(\tau(X))$
advantages of this approach over the aformentioned nonparameteric tests:
- the test statistics of the existing methods are directly based on a functional estimate $\hat\tau(\cdot)$ so that $\hat\tau(\cdot)$ itself needs to have tractable distributional properties asymptotically.
- some machine learning methods that are empirically known to have good prediction accuracy but without known distributional properties are not readily applicable for this purpose.
- in contrast, the proposed test treats $\tau(\cdot)$ as the nuisance parameter for estimating the target parameter $\Var(\tau(X))$.
- most existing methods are tailored only for complete data. In contrast, the proposed method easily adapts to a right-censored outcome with some modifications.
as is well recognized in the literature (Chernozhukov et al., 2018), flexible nonparameteric estimation of the nuisance parameter may cause non-negligible bias in the target parameter estimates
- to eliminate such bias, employ an EIF-based correction combined with a cross-fitting procedure, advocated by Hines et al. (2022) and Kennedy (2022)
- using this technique, it can be shown that the asymptotic null distribution of the resultng debiased estimator is a normal distribution with mean zero and variance equal to that of EIF.
- however, this debiasing produces an interesting problem in the specific context, that is, the EIF for $\Var(\tau(X))$ is degenerate to zero when the null hypothesis holds. This degeneracy invalidates the standard chi-square approximation of the Wald-type test statistics.
- to address the degeneracy problem, propose adaptively augmenting the Wald-type test statistic such that the test is performed by a modification locally around the null. More specifically, the proposed method controls the degree of augmentation by weighting so that the larger the true $\Var(\tau(X))$ is, the less the debiased estimator is augmented, which alleviates the power loss caused by augmentation.
- a recent preprint by Sanchez Becerra (2023): confronts the same degeneracy problem for testing $\Var(\tau(X)) = 0$, but with a different approach based on an adaptive confidence interval that relies on higher-order terms in the limiting distribution when degeneracy occurs.
- while their approach avoids any power loss due to augmentation, it does not generalize easily to censored outcomes.
- the proposed test needs a propensity score, known or estimated, and the approach given in the present paper offers a convenient and effective solution, especially in the cases of censored outcomes.
Methodology
- all observed variables $W = (Y, A, X)$
- propensity score: $e_a(X) = P(A=a\mid X)$
- conditional expectation of the observed outcome: $m(X, A) = E[Y\mid X, A]$
write the target parameter $\Var(\tau(X))$ as the functional of the distribution $P$ of the observed data $W$, i.e., $\psi(P) = \Var(\tau(X))$
standard assumptions in the causal inference literature
the CATE function:
\[\tau(X) = m(X, 1) - m(X, 0)\]and the average treatement effect (ATE)
\[\bar \tau = \bbE[Y\mid A = 1] - \bbE[Y\mid A = 0]\]the plug-in estimator fo $\psi(P)$
\[\psi(\hat P) = \frac 1N \sum_{i=1}^N(\hat\tau(X_i) - \hat{\bar\tau})^2\]where $\hat\tau(\cdot)$ and $\hat{\bar\tau}$ are some first-step estimates of $\tau(\cdot)$ and $\bar\tau$
the distributional property of $\psi(\hat P)$ is generally intractable unless we put stringent conditions on $\hat\tau(\cdot)$
the efficient influence function (EIF)-based correction is a remedy for such problem, the sample average of the EIF approximates the first-order bias of $\psi(\hat P)$ caused by the first-step nuisance estimation of $\tau(\cdot)$ and $\bar\tau$
the EIF for $\psi(P)$ as
\[\phi(W; P)\]the asymptotic properties of the targeted maximum likelihood estimator (TMLE) of $\psi(P)$
the debiased estimator is denoted as $\hat \psi^\star(P)$ (see Algorithm 1)
intuitively, the EIF captures the first-order sensitivity of $\psi(P)$ to a small perturbation of the distribution of $W$. $\Var(\tau(X))$ is on a quadratic scale in the amount of treatment-effect heterogeneity.
Thus, under a perturbation of size $\varepsilon$, its change is heuristically of the form $(c+\varepsilon)^2 - c^2$, where $c$ denotes the underlying magnitude of heterogeneity.
- when $c> 0$, the leading term is linear in $\epsilon$, so the first-order sensitivity is nonzero.
- however, under the null, $c = 0$, and the leading change is instead of order $\varepsilon^2$.
- therefore, the first-order term vanishes, which is reflected in the degeneracy of the EIF.
2.2 Proposed Method
augment the original test statistic, $N\hat\psi^\star(P)^2/\hat\sigma^2$
let $T_N$ and $t_\alpha$ denote some auxiliary non-negative test statistic and its auxiliary critical value at the significance level $\alpha$
let $w_N$ be a weight function depending only on observation
(a) $T_N\rightarrow_d \cT$ for some continuous distribution $\cT$ under the null hypothesis: $\psi(P) = 0$
(b) $1-F_\cT(t_\alpha)=\alpha$, where $F_\cT$ is the distribution function for $\cT$
(c) $w_N\rightarrow_p 1$ under the null hypothesis and $w_N\rightarrow_p 0$ under the alternative hypothesis.
the augmented test statistic
\[T_N^\star = (1-w_N) N\hat\psi^\star(P)^2 + w_NT_N\\ c_{\alpha, N} = (1-w_N) \hat\sigma^2q_{1-\alpha}(\chi_1^2) + w_Nt_\alpha\]one must select $(T_N, t_\alpha)$ such that the auxiliary test has the correct size
**a formal analysis under local alternatives, where $\psi(P)$ approaches 0 and $N$ increases, is technically challenging. This is because the asymptotic **
the choice of the auxiliary test statistic should play a significant role when $\psi(P)$ is small because $(T_N, t_\alpha)$ approximates $(T_N^\star, c_{\alpha, N})$ with the mixing proportion $w_N$ close to one in this situation.
3 Extensions to Right-Censored Data
introduce a censoring time variables $C_i$ such that one only observe $\min(Y_i, C_i)$ instead of $Y_i$
specify treatment heterogeneity as
\[\tau(X) = \bbE[\cY(Y^1) - \cY(Y^0)\mid X]\]for some deterministic monotone transformation $\cY$
A notable example is $\cY(t) = t\cap h$, measuring the treatment mean survival time (RMST)
4 Monte Carlo Simulation
4.1 The Uncensored Case
first assess the validity of the test
two treatment assignment mechanisms are considered
- random assignment
- non-random assignment
then check the power
5 Application to AIDS Trial
AIDS Clinical Trials Group Protocol 175 (ACTG 175)
2139 patients into four treatment groups