WeiYa's Work Yard

A dog, who fell into the ocean of statistics, tries to write down his ideas and notes to save himself.

Canonical Variate Analysis

Posted on
Tags: Canonical

This note is based on Campbell, N. A. (1979). CANONICAL VARIATE ANALYSIS: SOME PRACTICAL ASPECTS. 243.

Consider $g$ groups of data, with $v$ variables measured on each of $n_k$ individuals for the $k$-th group.

  • $x_{km}$: the vector of obs. on the $m$-th individual for the $k$-th group

Define the sums of squares and products (SSQPR) matrix for the $k$-th group as


and write

for the within-groups SSQPR matrix on

degrees of freedom.

Define the between-groups SSQPR matrix as



The simplest formulation of canonical variate analysis is the distribution-free one of finding that linear combination of the original variables which maximizes the variation between groups, relative to the variation within groups.

That is, find the canonical vector $c_1$ which maximizes the ration $c_1^TBc_1/c_1^TWc_1$; the vector is usually scaled so that $c_1^TWc_1=n_w$. The maximized ratio gives the first canonical root $f_1$.

Use of Lagrange multipliers leads directly to the eigenanalysis

Let $h=\min(v, g-1)$,

  • $C=[c_1,\ldots,c_h]$
  • $F=[f_1,\ldots,f_h]$




the canonical variates are uncorrelated both within and between groups, and have unit variance within groups.


then an equivalent formulation is to maximize the ratio $c_1^TBc_1/c_1^TTc_1$, leading to the eigenanalysis

The ration $r_1^2$ is the square of the first sample canonical correlation coefficient. The vector $c_1$ is scaled so that $c_1^Tc_1=n_w(1-r_1^2)^{-1}=n_w(1+f_1)$, so that again $c_1^TBc_1=n_wr_1^2(1-r_1^2)^{-1}=n_wf$ and $c_1^Wc_1=n_w$.

Now assume that $x_{km}\sim N_v(\mu_k,\Sigma)$. The maximized likelihood when the $\mu_k$ are unrestricted is

with $v(v+1)/2+gv$ estimated parameters. why $e^{-nv/2}$

The maximized likelihood for the hypothesis specifying equality of the $\mu_k$ is

with $v(v+1)/2 + v$ estimated parameters. This leads to the well-known likelihood ratio statistic given by $\vert W\vert/\vert W+B\vert$, and commonly referred as Wilks $\Lambda$. The statistic $\Lambda$ may be written as

(by the Cayley-Hamilton Theorem)

Published in categories Note