Learning Vector Quantization (LVQ)

LVQ is a supervised learning algorithm based on a set of training vectors with known classes (labeled). The learning rule for unsupervised competitive algorithm is modified to take advantage of the class information. Specifically, if the winning node belongs to the same class of the input vector, its similarity to the input vector is positively reinforced by moving its weight vector closer to the input vector:

$${\bf w}^{new}={\bf w}^{old}+\triangle {\bf w} ={\bf w}^{old}+\eta\;({\bf x}-{\bf w}^{old})$$

However, if the winning node does not belong to the class of the input vector, its weight vector is moved away from the input vector:

$${\bf w}^{new}={\bf w}^{old}-\triangle {\bf w} ={\bf w}^{old}-\eta\;({\bf x}-{\bf w}^{old})$$

The weight vectors of all other nodes are unchanged.

This LVQ algorithm can be improved if two nodes closest to the input vector are considered. If the winning node does not belong to the class of the input but the second closest node does, and if the input vector is close enough to the boundary between the two nodes, then both of the weight vectors of both of these nodes are updated, so that the closest node is moved away from the input vector while the second closest node is moved closer to the it.