Appendix: Kernel

Generally speaking, a kernel is a continuous function $k(x,y)$ that takes two arguments $x$ and $y$ (real numbers, functions, vectors, etc.) and maps them to a real value independent of the order of the arguments, i.e., $k(x,y)=k(y,x) \in {\cal R}$.

Examples:

• ${\bf x}$ and ${\bf y}$ can be two n-dimensional vectors and
  
  $$k({\bf x},{\bf y})={\bf x}^T {\bf y}=\sum_n x_n y_n=k({\bf y},{\bf x})$$
  
  
  Or the kernel can be defined as
  
  $$k({\bf x},{\bf y})=exp(-\vert\vert{\bf x}-{\bf y}\vert\vert^2/2\sigma^2)$$
  
  

• $x(t)$ and $y(t)$ can be two functions and
  
  $$k(x(t),y(t))=\int x(t) y(t) dt=k(y(t),x(t))$$
  
  

• $x(t)$ and $y(t)$ can be two functions and their convolution is the kernel:
  
  $$k(x(t),y(t))=x(t)*y(t)=\int_{-\infty}^{\infty} x(\tau) y(t-\tau) d\tau =y(t)*x(t)=k(y(t),x(t))$$
  
  
  where $t$ is a parameter.

such that K(x, s) = K(s, x).