Isotropic vs. Anisotropic
I came across isotropic and anisotropic covariance functions in kjytay’s blog, and then I found more materials, chapter 4 from the book Gaussian Processes for Machine Learning, via the reference in StackExchange: What is an isotropic (spherical) covariance matrix?.
Firstly of all, a covariance function must be symmetric and positive semi-definite.
A stationary covariance function is a function of $\x - \x’$. Thus it is invariant to translations in the input space.
For a stochastic process,
- weakly stationary: a process has constant mean and whose covariance function is invariant to translations
- strictly stationary: all of its finite dimensional distributions are invariant to translations.
If further the covariance function is a function only of $\vert \x -\x’\vert$ then it is called isotropic. These are also known as radial basis functions (RBFs).
A covariance is said to be anisotropic if it is not isotropic, such as setting\[r^2(\x,\x') = (\x - \x')^TM(\x - \x')\]
for some positive semidefinite $M$.