Nearest Neighbor Classifier

• Training

  This is a supervised classification method as a set of training samples (patterns of known classes) is assumed to be available for all c classes:
  

  $$\{{\bf x}_1^{(k)},\cdots,{\bf x}_{K_k}^{(k)}\} \;\;\;\;(k=1,\cdots,C)$$
  
  

  The classification is directly based on these training samples with no further training needed.

• Classification

  First define the nearest neighbor distance between an input pattern ${\bf x}$ to be classified and a class represented by its training samples as
  

  $$d({\bf x},\omega_k)\stackrel{\triangle}{=} min \{d_p({\bf x},{\bf x}_i^{(k)}),\;\;{\bf x}_i\in\omega_i\}$$
  
  
  Then a pattern ${\bf x}$ of unknown class is classified to its nearest neighbor's class:
  
  $${\bf x}\in \omega_k\;\;\;\mbox{iff}\;\;\;d({\bf x},\omega_k) =min \{d({\bf x},\omega_j),\;\;j=1,\cdots,C \}$$
  
  

Since this method depends highly on individual training samples, it is sensitive to noise.