Review on Random Matrix Theory
Posted on
This note is for Paul, D., & Aue, A. (2014). Random matrix theory in statistics: A review. Journal of Statistical Planning and Inference, 150, 1–29.
Two kinds of random matrices that have been central to most of the developments in RMT
- the sample covariance matrix: often referred to as the Wishart matrix
- the Wigner matrix:
both being symmetric or Hermitian matrices
The classical RMT model for Wishart matrices requires specifying two sequences, the sample size $n$, and the dimension $p=p(n)$, so that $p\rightarrow \infty$ as $n\rightarrow\infty$, and
\[\lim_{n\rightarrow \infty}\frac pn \rightarrow \gamma \in (0, \infty)\]Empirical spectral distribution (ESD) of $X$: the empirical distribution of the eigenvalues
The first question is to ask about the ESD is whether this random distribution converges to a probability distribution in an appropriate sense as the dimension of the matrix grows
Semicircle law: Wigner (1958) showed that the expected ESD of an $n\times n$ Wigner matrix with Gaussian entries, multiplied by $1/\sqrt n$, converges in distribution to the semicircle law that has pdf
\[f(x) = \frac{1}{2\pi}\sqrt{4-x^2}1_{[-2,2]}(x)\]mp law: an analogous result for the sample covariance matrix
Stieltjes transform
The Stieltjes transform plays nearly as useful a role in RMT as the Fourier transform in classical probability theory. The Stieltjes transform of a measure $\mu$ on the real line in defined as the function
\[S_\mu(z) = \int \frac{1}{x-z}\mu(dx), z\in C^+\]Suppose that ${P_n}$ is a sequence of Borel probability measures on the real line with Stieltjes transforms ${s_n}$. If $\lim_{n\rightarrow \infty}s_n(z)=s(z)$ for all $z\in C^+$, then there exists a Borel probability measure $P$ with Stieltjes transform $S_P=s$ iff
\[\lim_{v\rightarrow \infty}iv s(iv) = -1\]in which cases $P_n$ converges $P$ converges to $P$ in distribution.
Suppose that for each $N\ge 1$, $W_N$ is an $N\times N$ Hermitian random matrix, so that its eigenvalues are all real, with ESD $F^{W_N}$. Then, the Stieltjes transform of $F^{W_N}$, say $s_N$, is given by $s_N(z)=N^{-1}\tr((W_N-zI_N)^{-1})$.
In view of Lemma 3.4, in order to prove that the sequence of ESDs $F^{W_N}$ converges to a dimension $F$, say (in probability or almost surely), one needs to check that ${s_N}$ satisfies the conditions of the lemma (in probability or almost surely).
As an illustration, suppose that $W_n=X_n/\sqrt n$,
\[s_n(z) = \frac 1n\tr((W_n-zI_n)^{-1})=\frac 1n\sum_{k=1}^n\left(\frac{X_{kk}}{\sqrt n}-z-\alpha_{n,k}^\star(W_{n,k}-zI_{n-1})^{-1}\alpha_{n,k}\right)^{-1}\]Approximately,
\[s_n(z)\approx \frac{1}{-z-s_n(z)}\]for large enough $n$. Then it is expected that for each $z\in C^+$, $s_n(z)$ converges a.s. to $s(z)$ which satisfies the identity $s(z)(z+s(z))=-1$.
For Wishart matrix, approximately,
\[s_n(z) \approx \frac{1}{1-\gamma-\gamma zs_n(z)-z}, z\in C^+\]