KLT Completely Decorrelates the Signal

Now we show that among all possible orthogonal transforms, KLT is optimal in the following sense:

• KLT completely decorrelates the signal
• KLT maximally compacts the energy (information) contained in the signal.

The first property is simply due to the definition of KLT, and the second property is due to the fact that KLT redistributes the energy among the $N$ components in such a way that most of the energy is contained in a small number of components of ${\bf y}={\bf\Phi}^T {\bf x}$, as we will show later.

To see the first property, consider the mean vector ${\bf m}_y$ and covariance matrix ${\bf\Sigma}_y$ of ${\bf y}={\bf\Phi}^T {\bf x}$:

$${\bf m}_y = E({\bf y})=E({\bf\Phi}^T {\bf x})={\bf\Phi}^T E({\bf x}) ={\bf\Phi}^T {\bf m}_x$$

$${\bf\Sigma}_y = E({\bf yy}^{T})-{\bf m}_y {\bf m}_y^T =E[({\bf\Phi}^{T}{\bf x})({\bf\Phi}^{T}{\bf x})^{T}] -({\bf\Phi}^T {\bf m}_x) ({\bf\Phi}^T {\bf m}_x)^T$$

$$= E[{\bf\Phi}^{T}({\bf xx}^{T}){\bf\Phi}]-{\bf\Phi}^T {\bf m}_x {\b... ...x^T {\bf\Phi} = {\bf\Phi}^T [ E({\bf xx}^{T})-{\bf m}_x {\bf m}_x^T ] {\bf\Phi}$$

$$= {\bf\Phi}^{T}{\bf\Sigma}_x{\bf\Phi}={\bf\Lambda}=diag[\lambda_1, \lambda_2, \cdots,\lambda_{N}]$$

The above can also be written in matrix form:

$${\bf\Sigma}_y =\left[ \begin{array}{ccc} \cdots & \cdots &... ... & \vdots \\ 0 & \cdots & \sigma^2_{{N}} \end{array} \right]$$

We can make two observations:

• After KLT, the covariance matrix of the signal ${\bf y}={\bf\Phi}^T {\bf x}$ is diagonalized, i.e., the covariance $\sigma_{ij}=0$ between any two components $y_i$ and $y_j$ is always zero. In other words, the signal is completely decorrelated.
• The variance of $y_i$ is the same as the ith eigenvalue of the covariance matrix of ${\bf x}$, i.e., $\sigma^2_i=\lambda_i$.