Preprocessing for ICA

To simplify the ICA algorithms, the following preprocessing steps are usually taken:

• Centering

  Subtract the mean $E\{{\mathbf x}\}$ from the observed variable ${\mathbf x=As}$ so it has zero mean. By doing so, the sources ${\mathbf s}$ also become zero mean because $E\{{\mathbf x}\}={\mathbf A}\;E\{{\mathbf s}\}=0$. When the mixing matrix ${\mathbf A}$ is available, $E\{{\mathbf s}\}$ can be estimated to be ${\mathbf A}^{-1}E\{{\mathbf x}\}$.

• Whitening

  Transform observed variables $X$ so that they are uncorrelated and have unit variance. We first obtain the eigenvalues $\lambda_i$ and their corresponding eigenvectors ${\mathbf \phi}_i$ of the covariance matrix $E\{{\mathbf xx}^T\}$, and form the diagonal eigenvalue matrix ${\mathbf \Lambda}=diag(\lambda_1, \cdots,\lambda_m)$ and orthogonal eigenvector matrix ${\mathbf \Phi}= [{\mathbf \phi}_1,\cdots,{\mathbf \phi}_m]$ ( ${\mathbf \Phi}^{-1}={\mathbf \Phi}^T$). We have
  

  $$E\{{\mathbf xx}^T\}{\mathbf \Phi}={\mathbf \Phi\Lambda}, \;\... ...\{{\mathbf xx}^T\} {\mathbf \Phi \Lambda^{-1/2}}={\mathbf I}$$
  
  
  If we carry out a linear transform ${\mathbf T}={\mathbf \Lambda}^{-1/2} {\mathbf \Phi}^T$ so that ${\mathbf x'=TX}=({\mathbf \Lambda}^{-1/2}{\mathbf \Phi}^T) X$, the covariance matrix of ${\mathbf x}'$ becomes
  

  $$E\{{\mathbf x'x'}^T\} = E\{({\mathbf \Lambda}^{-1/2}{\mathbf \Phi}^T {\mathbf x}) ({\math... ... \Lambda}^{-1/2}{\mathbf \Phi}^T {\mathbf xx}^T {\mathbf \Phi \Lambda}^{-1/2}\}$$

  $$= {\mathbf \Lambda}^{-1/2}{\mathbf \Phi}^T E\{{\mathbf xx}^T\} {\mathbf \Phi \Lambda}^{-1/2}={\mathbf I }$$

  
  
  After the transform ${\mathbf T}={\mathbf \Lambda}^{-1/2} {\mathbf \Phi}^T$, the mixing process becomes
  
  $${\mathbf x'=Tx=TAs=A's}$$
  
  
  Here the new mixing matrix ${\mathbf A'=TA}$ is orthogonal, as it satisfies:
  
  $$E\{{\mathbf x'x'}^T\}={\mathbf I}={\mathbf A}'E\{{\mathbf ss}^T\} {\mathbf A'}^T={\mathbf A'A'}^T$$
  
  
  (recall that we assume $E\{{\mathbf ss}^T\}={\mathbf I}$). Similarly we also know that the matrix ${\mathbf W}$ we seek is also orthogonal, as
  
  $$E\{{\mathbf ss}^T\}={\mathbf I}={\mathbf W}E\{{\mathbf x'x'}^T\} {\mathbf W}^T={\mathbf WW}^T$$
  
  

  The whitening process reduces the independent variables, the $n^2$ components of the mixing matrix $A'$ to half ($n(n-1)/2$) due to the constraint that ${\mathbf A}'$ is orthogonal. Moreover, the whitening can also reduce the dimensionality of the problem by ignoring the components corresponding to very small eigenvalues (PCA).