Conservation of Degrees of Freedom

We now show that the degrees of freedom (d.o.f., the number of independent variables in the signal) are conserved in the SVD transform (same as any other transforms). In spatial domain the d.o.f. of the image matrix ${\bf A}_{N\times N}$ (assuming $M=N=R$ for simplicity) is $N^2$. Now in the transform domain, both ${\bf U}$ and ${\bf V}$ have the same d.o.f. $(N^2-N)/2$ for the following reason. The first column vector has $N$ elements subject to normalization, i.e., $N-1$ d.o.f.; the second vector is the same except it also has to be orthogonal to the first one, and therefore has $N-2$ d.o.f.; the third vector has to be orthogonal to the first two vectors and therefore has $N-3$ d.o.f.; etc. Now the total d.o.f. of all $N$ vectors is

$$(N-1)+(N-2)+\cdots+1=\sum_{i=1}^{N-1} i=(N-1)N/2=(N^2-N)/2$$

Together with the $N$ d.o.f. in $\Lambda$, the over all d.o.f. is

$$2\;(N^2-N)/2+N=N^2$$

This indicates that the signal, in either the original spatial domain ($A$) or the transform domain ($\Lambda$, $U$ and $V$), always has the same degrees of freedom.