Hermitian and unitary matrices

A square matrix ${\bf A}$ is a Hermitian matrix iff it is equal to its complex conjugate transpose ${\bf A}^*=\overline{\bf A}^T={\bf A}$. If a Hermitian matrix $\overline{\bf A}={\bf A}$ is real, it is a symmetric matrix, ${\bf A}^T={\bf A}$.

${\bf A}$ is a unitary matrix iff ${\bf A}^*\,{\bf A}={\bf I}$, i.e., its conjugate transpose is equal to its inverse ${\bf A}^*={\bf A}^{-1}$. When a unitary matrix $\overline{\bf A}={\bf A}$ is real, it becomes an orthogonal matrix, ${\bf A}^T={\bf A}^{-1}$.

The column (or row) vectors ${\bf a}_1,\cdots,{\bf a}_n$ of a unitary matrix ${\bf A}=[{\bf a}_1,\cdots,{\bf a}_N]$ are orthonormal, i.e. they are both orthogonal and normalized, i.e.,

$$<{\bf a}_i, {\bf a}_j>={\bf a}_i^T\overline{\bf a}_j=\sum_k ... ...{ \begin{array}{ll} 1 & i=j \\ 0 & i\neq j \end{array} \right.$$

As we will see later, any Hermitian matrix ${\bf A}$ can be converted to a diagonal matrix ${\bf\Lambda}$ (or diagonalized) by a particular unitary matrix ${\bf\Phi}$:

$${\bf\Phi}^*{\bf A}{\bf\Phi}={\bf\Lambda}$$

where ${\bf\Lambda}$ is a diagonal matrix, i.e., all its off diagonal elements are 0.

Given any unitary matrix ${\bf A}$, we can define a unitary transform of a vector ${\bf x}=[x_1,\cdots,x_n]^T$:

$$\left\{ \begin{array}{l} {\bf y} =\left[ \begin{array}{c} y... ..._i \;\;\;\;\;\;\mbox{(inverse transform)} \end{array} \right.$$

When ${\bf A}=\overline{\bf A}$ is real, ${\bf A}^{-1}={\bf A}^T$ is an orthogonal matrix and the corresponding transform is an orthogonal transform.

The first equation above is the forward transform and can be written in component form as:

$$y_i=\overline{\bf a}_i^T{\bf x}=<{\bf x},{\bf a}_i> =\sum_{j=1}^n x_j\overline{a}_{ij},\;\;\;\;\;\;(i=1,\cdots,n)$$

The transform coefficient is an inner product $y_i=<{\bf x},{\bf a}_i>$, representing the projection of vector ${\bf x}$ onto the ith column vector ${\bf a}_i$ of the transform matrix ${\bf A}$. The second equation is the inverse transform and can also be written in component form as:

$$x_j=\sum_{i=1}^n a_{ji} \; y_i,\;\;\;\;\;\;(j=1,\cdots,n)$$

By this transform, vector ${\bf x}$ is represented as a linear combination (weighted sum) of the $n$ column vectors ${\bf a}_i,{\bf a}_2, \cdots, {\bf a}_n$ of matrix ${\bf A}$. Geometrically, ${\bf x}$ is a point in the n-dimensional space spanned by these $n$ orthonormal basis vectors. Each coefficient (coordinate) $y_i$ is the projection of ${\bf x}$ onto the corresponding basis vector ${\bf a}_i$.

As the n-dimensional space can be spanned by the column vectors of any n by n unitary (orthogonal) matrix, a vector ${\bf x}$ in the space can be represented by any of such matrices, each defining a different transform.

Examples:

• When ${\bf A}={\bf I}=[\cdots,{\bf e}_i,\cdots]$ is an identity matrix, we have
  
  $${\bf x}=\sum_{i=1}^n y_i {\bf a}_i=\sum_{i=1}^n x_i {\bf e}_i$$
  
  
  where ${\bf e}_i=[0,\cdots,0,1,0,\cdots,0]^T$ is the ith column of ${\bf I}$ with the ith element equal 1 and all other 0.

• When $a_{m,n}=w[m,n]=e^{-j2\pi mn/N}$, the corresponding transform is discrete Fourier transform. The nth column vector ${\bf w}_n$ of the transform matrix ${\bf W}=[{\bf w}_0,\cdots,{\bf w}_{N-1}]$ represents a sinusoid of a frequency $nf_0$, and the corresponding complex coordinate $y_n=({\bf x},{\bf w}_n)$ represents the magnitude $\vert y_n\vert$ and phase $\angle y_n$ of this nth frequency component. The Fourier transform ${\bf y}={\bf W}{\bf x}$ represents a rotation of the coordinate system.

A unitary (orthogonal) transform ${\bf y}={\bf A}{\bf x}$ can be interpreted geometrically as the rotation of the vector $X$ about the origin, or equivalently, the representation of the same vector in a rotated coordinate system. A unitary (orthogonal) transform ${\bf y}={\bf A}^*{\bf x}$ does not change the vector's norm (length)

$$\vert\vert{\bf y}\vert\vert^2= {\bf y}^T\overline{\bf y} =({... ...f x} ={\bf x}^T\overline{\bf x}=\vert\vert{\bf x}\vert\vert^2$$

as $\overline{\bf A}{\bf A}^T={\bf I}$. This is the Parseval's identity that indicates that the norm or length of a vector is conserved under any unitary transform. If ${\bf X}$ is interpreted as a signal, then its length $\vert\vert{\bf x}\vert\vert^2=\vert\vert{\bf y}\vert\vert^2$ represents the total energy or information contained in the signal, which is conserved during any unitary transform. However, some other features of the signal may change, e.g., the signal may be decorrelated and its total energy redistributed among its components after the transform, which may be desirable in many applications.

If ${\bf x}$ is a random vector with mean vector ${\bf m}_x$ and covariance matrix ${\bf\Sigma}_x$:

$${\bf m}_x=E( {\bf x} ),\;\;\;\;\; {\bf\Sigma}_x=E({\bf x}{\bf x}^*)-{\bf m}_x{\bf m}_x^*$$

then its transform ${\bf y}={\bf A}^*{\bf x}$ has the following mean vector and covariance matrix:

$${\bf m}_y = E({\bf y})=E({\bf A}^* {\bf x})={\bf A}^* E({\bf x}) ={\bf A}^* {\bf m}_x$$

$${\bf\Sigma}_y = E({\bf yy}^*)-{\bf m}_y {\bf m}_y^* =E[({\bf A}^*{\bf x})({\bf A}^*{\bf x})^*] -({\bf A}^* {\bf m}_x) ({\bf A}^* {\bf m}_x)^*$$

$$= E[{\bf A}^*({\bf xx}^*){\bf A}]-{\bf A}^* {\bf m}_x {\bf m}_x^* {\bf A} = {\bf A}^* [ E({\bf xx}^*)-{\bf m}_x {\bf m}_x^* ] {\bf A}$$

$$= {\bf A}^*{\bf\Sigma}_x{\bf A}$$

In general the unitary transform of any square matrix ${\bf A}$ by a unitary matrix ${\bf R}$ is

$${\bf B}={\bf R}^*{\bf A}{\bf R}$$