Edgeworth Expansion

Posted on Jan 24, 2024

Tags: Edgeworth, Asymptotic

This note is based on Shao, J. (2003). Mathematical statistics (2nd ed). Springer. and Hwang, J. (2019). Note on Edgeworth Expansions and Asymptotic Refinements of Percentile t-Bootstrap Methods. Bootstrap Methods.

If $F_n\rightarrow_w F$ and $F$ is continuous on $R^k$, then

\[\lim_{n\rightarrow \infty} \sup_{x\in R^k}\vert F_n(x) - F(x)\vert = 0\]

Let $Y_n$ be a sequence of random variables, ${\mu_n}$ and ${\sigma_n}$ be sequences of real numbers such that $\sigma_n > 0$ for all $n$, and $(Y_n-\mu_n)/\sigma_n\rightarrow_d N(0,1)$. Then by Polya’s Theorem,

\[\lim_{n\rightarrow\infty} \sup_x\vert F_{(Y_n-\mu_n)/\sigma_n}(x) - \Phi(x)\vert = 0\,. \lim_{n\rightarrow\infty} \sup_x\vert F_{(Y_n-\mu_n)/\sigma_n}(x) - \Phi(x)\vert = 0\,.\]

Let $W_n = (Y_n-\mu_n)/\sigma_n$. The convergence speed of the above can be used to assess whether $\Phi$ provides a good approximation to the cdf $F_{W_n}$.

Also, sometimes we would like to find an approximation to $F_{W_n}$ that is better than $\Phi$ in terms of convergence speed. The edgeworth expansion is a useful tool for these purposes.

Let $W_n = n^{-1/2}\sum_{i=1}^n (X_i-\mu)/\sigma$ where $X_1,X_2,\ldots,$ are i.i.d. random variables with $EX_1=\mu$ and $\Var(X_1)=\sigma^2$.

Assume that the mgf of $Z = (X_1-\mu)/\sigma$ is finite and positive in a neighborhood of 0. The cumulant generating function of $Z$ has the expansion

\[\kappa(t) = \sum_{j=1}^\infty \frac{\kappa_j}{j!}t^j\,,\]

and the mgf of $W_n$ is equal to

\[\psi_n(t) = [\exp(\kappa(t/\sqrt n))]^2 = \exp\left(\frac{t^2}{2}+\sum_{j=3}^\infty \frac{\kappa_jt^j}{j!n^{(j-2)/2}} \right)\]

Using the series expansion, we obtain that

\[\psi_n(t) = e^{t^2/2} (1 + n^{-1/2}r_1(t) + \cdots + n^{-j/2}r_j(t) + \cdots)\]

where $r_j$ is a polynomial of degree $3j$ depending on $\kappa_3,\ldots, \kappa_{j+2}$ but not on $n, j=1,2,\ldots$.

confused, how to derive?

Particularly,

\[r_1(t) = \frac{1}{6}\kappa_3 t^3, \quad r_2(t) = \frac{1}{24}\kappa_4t^4 + \frac{1}{72}\kappa_3^2t^6\]

Using a second-order expansion of exponential function, i.e.,

\[\exp(x + o(n^{-1})) = 1+x+\frac{x^2}{2} + o(n^{-1})\]

yields

\[\exp\left(\frac{\kappa_3}{\sqrt n}\frac{t^3}{3!} + \frac{\kappa_4}{n}\frac{t^4}{4!}+o(n^{-1})\right) = 1+\frac{\kappa_3}{\sqrt n}\frac{t^3}{6} + \frac{\kappa_4}{n}\frac{t^4}{24} + \frac{\kappa_3^2}{n}\frac{t^6}{72} + o(n^{-1})\]

Since $\psi_n(t) = \int e^{tx}dF_{W_n}(x)$ and $e^{t^2/2} = \int e^{tx}d\Phi(x)$, the expansion suggests the inverse expansion

\[F_{W_n}(x) = \Phi(x) + n^{-1/2}R_1(x) + \cdots + n^{-j/2} R_j(x) + \cdots\]

where $R_j(x)$ is a function satisfying $\int e^{tx}dR_j(x) = r_j(t)e^{t^2/2}, j=1,2,\ldots,$

$R_j$’s can be obtained once $r_j$’s are derived, it follows that

\[R_1(x) = -\frac 16\kappa_3(x^2 - 1)\Phi'(x)\]

and

\[R_2(x) = -\left[\frac{1}{24}\kappa_4 x(x^2-3) + \frac{1}{72}\kappa_3^2 x(x^4-10x^2+15)\right]\Phi'(x)\]

Let $\phi(x)$ be the cdf and pdf of standard normal random variable, respectively, and

\[H_j(x) = (-1)^j \frac{\phi^{(j)}(x)}{\phi(x)}\]

In particular, \(H_2(x) = x^2 - 1,\; H_3(x) = x^3 - 3x; H_5(x) = x^5 - 10x^3 + 15x\)

An Edgeworth expansion is a series representation for $G_n(x)$ expressed as powers of $n^{-1/2}$, i.e.,

\[G_n(x) = P(Z_n\le x) = \Phi(x) - \sqrt{1}{\sqrt n}\frac{\kappa_3}{6}H_2(x)\phi(x) -\frac 1n \left(\frac{\kappa_4}{24}H_3(x)+\frac{\kappa_3^2}{72}H_5(x)\right)\phi(x) + o(1/n)\,.\]