Paradigms of Learning

1. Pattern Associator

   Training (supervised):

   A set of pairs of patterns $\{ ({\bf x}_k, {\bf y}_k), i=1,2, \cdots, K\}$ is repeatedly presented to the NN which then learns to establish the relationship (association) between two sets of patterns:

   $$f: {\bf x} \in R^n \rightarrow {\bf y} \in R^m$$
   i.e., the associative relationship between the two patterns of the pairs is stored in the NN.

   Testing:

   When one pattern in a pair is presented, the NN will produce the other.

2. Auto-associator

   Training:

   A set of patterns $\{{\bf x}_k, k=1,2, \cdots, K\}$ is repeatedly presented to the NN which learns and remembers them, i.e., the patterns are stored in the NN.

   Testing:

   When part of a pattern, or a similar pattern (pattern with noise) is presented to the NN, the complete original pattern will be retrieved through pattern completion by the NN.

3. Pattern Classifier

   This is a variation of the first type of NN. The input patterns are classified by the NN into a set of classes (represented by names or any other symbols). This can be considered as a special type of mapping:

   $$f: {\bf x} \in R^n \rightarrow \omega_i \in \{\omega_1, \cdots, \omega_m\}$$

4. Regularity Detector

   The NN discovers the regularity in the inputs so that patterns of various types can be automatically detected and classified into a set of classes. This is an unsupervised learning process.

Examples of association:

• ${\bf x}_1$: Evolution $\rightarrow$ ${\bf y}_1$: Darwin
• ${\bf x}_2$: Einstein $\rightarrow$ ${\bf y}_2$: $E=mc^2$
• ${\bf x}_3$: sounding bell $\rightarrow$ ${\bf y}_3$: food