KARHUNEN LOUVE TRANSFORM

The Karhunen and Louve transform is the projection on the principal components (eigenvectors) of a set of vectors. In other words the KL tranforms finds the basis on which the components of the given set of vectors are statistically independent.

We assume to have N vectors, v(1) ... v(N) of dimension M, which we can arrange in a MxN matrix,

Uik = [ v(1) ... v(N) ] = v(k),i

We further assume that the vectors have zero mean, or the mean vector has been subtracted. The covariance matrix of the vectors (a MxM matrix) is

Cij = ∑k v(k),i v(k),j = ∑k Uik Ujk = U Ut

When the number of vectors is much smaller than their dimensionality (N < M) the covariance matrix is highly singular, and its rank is at most N. We then consider the NxN matrix

C'kh = ∑i v(k),i v(h),i = ∑i Uik Uih = Ut U

and suppose that this matrix can be diagonalized,

C'kh = E'kl Ll E'hl = E' L E't

where L is the diagonal matrix of the eigenvalues of C', and E' is formed by the eigenvectors of C',

E'hl = [ e'(1) ... e'(N) ] = e'(l),h

The eigenvectors are orthonormal,

E'hlt E'lk = E'lh E'lk = e'h e'k = dh,k

Therefore

C'kh E'hn = E'kl Ll E'hl E'hn = E'kn Ln

and it can readily verified that the matrix UE' (of size NxM) is formed by M eigenvectors of C, with the same eigenvalues of C',

Cij Ujk E'hl = Uih Ujh Ujk E'hl = Uih C'hk E'kl = Uih E'hl Ll

Therefore the matrix of the non-null N eigenvectors of C is

Ejl = Ujk E'kl = [ e(1) ... e(N) ] = e(l),h

Cij e(l)j = Ll e(l)i

Marco Corvi - Page hosted by geocities.com.