Basic Definitions

• Feature Space

  The image objects of interest are represented by a set of $N$ features extracted from the image. They form an N-dimensional space called the feature space, with each dimension representing one of the features.

• Patterns

  Each image object to be recognized is represented as an N-dimensional vector ${\bf x}=[x_1,\cdots,x_N]^T$, called a pattern, or a sample, which is a point in the feature space: ${\bf x} \in R^N$,

• Pattern Classes (categories)

  A set of $c$ classes $\Omega=\{\omega_1, \cdots, \omega_c \}$.

• Pattern Classification

  Any given pattern vector ${\bf x} \in R^N$ is to be classified into one of the $c$ classes $\omega \in \Omega$. Classification process can be carried out as a maximization process:
  

  $${\bf x} \in \omega_k\;\;\;\mbox{iff}\;\;\; D_k({\bf x}) = max\{D_i,\;i=1,\cdots,C \}$$
  
  
  where $D_i({\bf x})$ is the discriminant function of class $\omega_i$ which can be obtained according to the specific method used. Classification can be considered as a partitioning of the feature space into $c$ regions each corresponding to a class, by the boundaries described by:
  
  $$D_i({\bf x})=D_j({\bf x}),\;\;\;\;\mbox{for all}\;i\neq j$$
  
  

  

  Example: Iris flower data set

• Feature Selection

  The number of features actually used in classification can be reduced from $N$ to $M$ by feature selection to (1) use only those features most relevant to a specific application, and (2) reduce the computational cost.

  Feature selection may be achieved in two different ways:

  • Choose $M < N$ features directly from the $N$ available features ( $C_N^M=M!/N!(N-M)!$ ways to do so.)

  • Use $M$ linear combinations of the $N$ features as new features:
    
    $${\bf y}=\left[ \begin{array}{c} y_1 \vdots y_M \end{a... ...a}_1^T \vdots {\bf a}_M^T \end{array} \right] {\bf x}$$
    
    
    where ${\bf A}$ is an $N\times M$ transform matrix ${\bf A}=[{\bf a}_1,\cdots,{\bf a}_M]$, of which ${\bf a}_m^T=[a_{1m},\cdots,a_{Nm}]$ is the mth N-D column vector, and ${\bf y}=[y_1,\cdots,y_M]^T$ is the resulting M-D vector containing $M$ new features. The mth element of ${\bf y}$, one of the $M$ new features, is a linear combination of all $N$ original features $x_1,\cdots,x_N$:
    
    $$y_m={\bf a}_m^T {\bf x}=\sum_{n=1}^N a_{nm} x_n$$
    
    

  Aome criterion is needed to guide the selection to decide which $M$ out of the $N$ features to choose, or how to find the matrix ${\bf A}$.