# Kernels

## Kernel Functions

A kernel is a function \kappa that for all \mathbf{t}, \mathbf{x} \in X satisfies \kappa(\mathbf{t}, \mathbf{x}) = \langle\Phi(\mathbf{t}),\Phi(\mathbf{x})\rangle, where \Phi is a mapping from X to an (inner product) feature space F, \Phi : \mathbf{t} \longmapsto \Phi(\mathbf{t}) \in F.

The following functions take two array-like objects `t` (M, P) and `x` (N, P) and compute the (M, N) matrix \mathbf{K^t} with entries

> \mathbf{K^t}_{ij} = \kappa(\mathbf{t}_i, \mathbf{x}_j).

## Kernel Classes

## Functions

mlpy.kernel_linear(t, x)

  Linear kernel, t_i’ x_j.

mlpy.kernel_polynomial(t, x, gamma=1.0, b=1.0, d=2.0)

  Polynomial kernel, (gamma t_i’ x_j + b)^d.

mlpy.kernel_gaussian(t, x, sigma=1.0)

  Gaussian kernel, exp(-||t_i - x_j||^2 / 2 * sigma^2).

mlpy.kernel_exponential(t, x, sigma=1.0)

  Exponential kernel, exp(-||t_i - x_j|| / 2 * sigma^2).

mlpy.kernel_sigmoid(t, x, gamma=1.0, b=1.0)

  Sigmoid kernel, tanh(gamma t_i’ x_j + b).

Example:

    >>> import mlpy
    >>> x = [[5, 1, 3, 1], [7, 1, 11, 4], [0, 4, 2, 9]] # three training points
    >>> K = mlpy.kernel_gaussian(x, x, sigma=10) # compute the kernel matrix K_ij = k(x_i, x_j)
    >>> K
    array([[ 1.        ,  0.68045064,  0.60957091],
           [ 0.68045064,  1.        ,  0.44043165],
           [ 0.60957091,  0.44043165,  1.        ]])
    >>> t = [[8, 1, 5, 1], [7, 1, 11, 4]] # two test points
    >>> Kt = mlpy.kernel_gaussian(t, x, sigma=10) # compute the test kernel matrix Kt_ij = <Phi(t_i), Phi(x_j)> = k(t_i, x_j)
    >>> Kt
    array([[ 0.93706746,  0.7945336 ,  0.48190899],
           [ 0.68045064,  1.        ,  0.44043165]])

## Centering in Feature Space

The centered kernel matrix \mathbf{\tilde{K}^t} is computed by:

> \mathbf{\tilde{K}^t}_{ij} = \left\langle \Phi(\mathbf{t}_i) - \frac{1}{N} \sum_{m=1}^N{\Phi(\mathbf{x}_m)}, \Phi(\mathbf{x}_j) - \frac{1}{N} \sum_{n=1}^N{\Phi(\mathbf{x}_n)} \right\rangle.

We can express \mathbf{\tilde{K}^t} in terms of \mathbf{K^t} and \mathbf{K}:

> \mathbf{\tilde{K}^t}_{ij} = \mathbf{K^t} - \mathbf{1}^T_N \mathbf{K} - \mathbf{K^t} \mathbf{1}_N + \mathbf{1}^T_N \mathbf{K} \mathbf{1}_N

where \mathbf{1}_N is the N \times M matrix with all entries equal to 1/N and \mathbf{K} is \mathbf{K}_{ij} = \kappa(\mathbf{x}_i, \mathbf{x}_j).

mlpy.kernel_center(Kt, K)

  Centers the testing kernel matrix Kt respect the training kernel matrix K. If Kt = K (kernel_center(K, K), where K = k(x_i, x_j)), the function centers the kernel matrix K.

  Parameters:
    Kt2d array_like object (M, N)

      test kernel matrix Kt_ij = k(t_i, x_j). If Kt = K the function centers the kernel matrix K

    K2d array_like object (N, N)

      training kernel matrix K_ij = k(x_i, x_j)

  Returns:
    Ktcentered2d numpy array (M, N)

      centered version of Kt

Example:

    >>> Kcentered = mlpy.kernel_center(K, K) # center K
    >>> Kcentered
    array([[ 0.19119746, -0.07197215, -0.11922531],
           [-0.07197215,  0.30395696, -0.23198481],
           [-0.11922531, -0.23198481,  0.35121011]])
    >>> Ktcentered = mlpy.kernel_center(Kt, K) # center the test kernel matrix Kt respect to K
    >>> Ktcentered
    array([[ 0.15376875,  0.06761464, -0.22138339],
           [-0.07197215,  0.30395696, -0.23198481]])

## Make a Custom Kernel

TODO

[Scolkopf98]

Bernhard Scholkopf, Alexander Smola, and Klaus-Robert Muller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10(5):1299–1319, July 1998.