### Navigation

- index
- modules |
- next |
- previous |
- mlpy 2.2.0 documentation »

# Supervised Classification

Every classifier must be initialized with a specific set of parameters. Two distinct methods are deployed for the training (`compute()`) and the testing (`predict()`) phases. Whenever possible, the real valued prediction is stored in the realpred variable.

## Support Vector Machines (SVMs)

class `mlpy.``Svm`(kernel='linear', kp=0.1, C=1.0, tol=0.001, eps=0.001, maxloops=1000, cost=0.0, alpha_tversky=1.0, beta_tversky=1.0, opt_offset=True)

  Support Vector Machines (SVM).

  Example : |

      >>> import numpy as np
      >>> import mlpy
      >>> xtr = np.array([[1.0, 2.0, 3.0, 1.0],  # first sample
      ...                 [1.0, 2.0, 3.0, 2.0],  # second sample
      ...                 [1.0, 2.0, 3.0, 1.0]]) # third sample
      >>> ytr = np.array([1, -1, 1])             # classes
      >>> mysvm = mlpy.Svm()                     # initialize Svm class
      >>> mysvm.compute(xtr, ytr)                # compute SVM
      1
      >>> mysvm.predict(xtr)                     # predict SVM model on training data
      array([ 1, -1,  1])
      >>> xts = np.array([4.0, 5.0, 6.0, 7.0])   # test point
      >>> mysvm.predict(xts)                     # predict SVM model on test point
      -1
      >>> mysvm.realpred                         # real-valued prediction
      -5.5
      >>> mysvm.weights(xtr, ytr)                # compute weights on training data
      array([ 0.,  0.,  0.,  1.])

  Initialize the Svm class

  Parameters :

  kernel : string [‘linear’, ‘gaussian’, ‘polynomial’, ‘tr’, ‘tversky’]

    kernel

  kp : float

    kernel parameter (two sigma squared) for gaussian and polynomial kernel

  C : float

    regularization parameter

  tol : float

    tolerance for testing KKT conditions

  eps : float

    convergence parameter

  maxloops : integer

    maximum number of optimization loops

  cost : float [-1.0, ..., 1.0]

    for cost-sensitive classification

  alpha_tversky : float

    positive multiplicative parameter for the norm of the first vector

  beta_tversky : float

    positive multiplicative parameter for the norm of the second vector

  opt_offset : bool

    compute the optimal offset

  `compute`(x, y)

    Compute SVM model

    Parameters :

    x : 2d ndarray float (samples x feats)

      training data

    y : 1d ndarray integer (-1 or 1)

      classes

    Returns :

    conv : integer

      svm convergence (0: false, 1: true)

  `predict`(p)

    Predict svm model on a test point(s)

    Parameters :

    p : 1d or 2d ndarray float (samples x feats)

      test point(s)training dataInput

    Returns :

    cl : integer or 1d ndarray integer

      class(es) predicted

    Attributes :

    Svm.realpred : float or 1d ndarray float

      real valued prediction

  `weights`(x, y)

    Return feature weights

    Parameters :

    x : 2d ndarray float (samples x feats)

      training data

    y : 1d ndarray integer (-1 or 1)

      classes

    Returns :

    fw : 1d ndarray float

      feature weights

Note

For tr kernel (Terminated Ramp Kernel) see [Merler06].

## K Nearest Neighbor (KNN)

class `mlpy.``Knn`(k, dist='se')

  k-Nearest Neighbor (KNN).

  Example:

      >>> import numpy as np
      >>> import mlpy
      >>> xtr = np.array([[1.0, 2.0, 3.1, 1.0],  # first sample
      ...                 [1.0, 2.0, 3.0, 2.0],  # second sample
      ...                 [1.0, 2.0, 3.1, 1.0]]) # third sample
      >>> ytr = np.array([1, -1, 1])             # classes
      >>> myknn = mlpy.Knn(k = 1)                # initialize knn class
      >>> myknn.compute(xtr, ytr)              # compute knn
      1
      >>> myknn.predict(xtr)                   # predict knn model on training data
      array([ 1, -1,  1])
      >>> xts = np.array([4.0, 5.0, 6.0, 7.0]) # test point
      >>> myknn.predict(xts)                   # predict knn model on test point
      -1
      >>> myknn.realpred                       # real-valued prediction
      0.0

  Initialize the Knn class.

  Parameters :

  k : int (odd > = 1)

    number of NN

  dist : string (‘se’ = SQUARED EUCLIDEAN, ‘e’ = EUCLIDEAN)

    adopted distance

  `compute`(x, y)

    Store x and y data.

    Parameters :

    x : 2d ndarray float (samples x feats)

      training data

    y : 1d ndarray integer (-1 or 1 for binary classification)

      : 1d ndarray integer (1, ..., nclasses for multiclass classificatio) classes

    Returns : | 1

    Raises :

    ValueError

      if not (1 <= k <= #samples)

    ValueError

      if there aren’e at least 2 classes

    ValueError

      if, in case of 2-classes problems, the lables are not 1 and -1

    ValueError

      if, in case of n-classes problems, the lables are not int from 1 to n

  `predict`(p)

    Predict knn model on a test point(s).

    Parameters :

    p : 1d or 2d ndarray float (sample(s) x feats)

      test sample(s)

    Returns : | the predicted value(s) on success: integer or 1d numpy array integer (-1 or 1) for binary classification integer or 1d numpy array integer (1, ..., nclasses) for multiclass classification 0 on succes with non unique classification -2 otherwise

    Raises :

    StandardError

      if no Knn method computed

## Fisher Discriminant Analysis (FDA)

Described in [Mika01].

class `mlpy.``Fda`(C=1)

  Fisher Discriminant Analysis.

  Example:

      >>> import numpy as np
      >>> import mlpy
      >>> xtr = np.array([[1.0, 2.0, 3.1, 1.0],  # first sample
      ...                 [1.0, 2.0, 3.0, 2.0],  # second sample
      ...                 [1.0, 2.0, 3.1, 1.0]]) # third sample
      >>> ytr = np.array([1, -1, 1])             # classes
      >>> myfda = mlpy.Fda()                   # initialize fda class
      >>> myfda.compute(xtr, ytr)              # compute fda
      1
      >>> myfda.predict(xtr)                   # predict fda model on training data
      array([ 1, -1,  1])
      >>> xts = np.array([4.0, 5.0, 6.0, 7.0]) # test point
      >>> myfda.predict(xts)                   # predict fda model on test point
      -1
      >>> myfda.realpred                       # real-valued prediction
      -42.51475717037367
      >>> myfda.weights(xtr, ytr)              # compute weights on training data
      array([  9.60629896,   9.77148463,   9.82027615,  11.58765243])

  Initialize Fda class.

  Parameters :

  C : float

    regularization parameter

  `compute`(x, y)

    Compute fda model.

    Parameters :

    x : 2d numpy array float (sample x feature)

      training data

    y : 1d numpy array integer (two classes, 1 or -1)

      classes

    Returns : | 1

  `predict`(p)

    Predict fda model on test point(s).

    Parameters :

    p : 1d or 2d ndarray float (sample(s) x feats)

      test sample(s)

    Returns :

    cl : integer or 1d numpy array integer

      class(es) predicted

    Attributes :

    self.realpred : float or 1d numpy array float

      real valued prediction

  `weights`(x, y)

    Return feature weights.

    Parameters :

    x : 2d ndarray float (samples x feats)

      training data

    y : 1d ndarray integer (-1 or 1)

      classes

    Returns :

    fw : 1d ndarray float

      feature weights

## Spectral Regression Discriminant Analysis (SRDA)

Described in [Cai08].

class `mlpy.``Srda`(alpha=1.0)

  Spectral Regression Discriminant Analysis (SRDA).

  Example:

      >>> import numpy as np
      >>> import mlpy
      >>> xtr = np.array([[1.0, 2.0, 3.1, 1.0],  # first sample
      ...                 [1.0, 2.0, 3.0, 2.0],  # second sample
      ...                 [1.0, 2.0, 3.1, 1.0]]) # third sample
      >>> ytr = np.array([1, -1, 1])             # classes
      >>> mysrda = mlpy.Srda()                 # initialize srda class
      >>> mysrda.compute(xtr, ytr)             # compute srda
      1
      >>> mysrda.predict(xtr)                  # predict srda model on training data
      array([ 1, -1,  1])
      >>> xts = np.array([4.0, 5.0, 6.0, 7.0]) # test point
      >>> mysrda.predict(xts)                  # predict srda model on test point
      -1
      >>> mysrda.realpred                      # real-valued prediction
      -6.8283034257748758
      >>> mysrda.weights(xtr, ytr)             # compute weights on training data
      array([ 0.10766721,  0.21533442,  0.51386623,  1.69331158])

  Initialize the Srda class.

  Parameters :

  alpha : float(>=0.0)

    regularization parameter

  `compute`(x, y)
    Compute Srda model.
      Initialize array of alphas a.

    Parameters :

    x : 2d ndarray float (samples x feats)

      training data

    y : 1d ndarray integer (-1 or 1)

      classes

    Returns : | 1

    Raises :

    LinAlgError

      if x is singular matrix in __PenRegrModel

  `predict`(p)

    Predict Srda model on test point(s).

    Parameters :

    p : 1d or 2d ndarray float (sample(s) x feats)

      test sample(s)

    Returns :

    cl : integer or 1d numpy array integer

      class(es) predicted

    Attributes :

    self.realpred : float or 1d numpy array float

      real valued prediction

  `weights`(x, y)

    Return feature weights.

    Parameters :

    x : 2d ndarray float (samples x feats)

      training data

    y : 1d ndarray integer (-1 or 1)

      classes

    Returns :

    fw : 1d ndarray float

      feature weights

## Penalized Discriminant Analysis (PDA)

Described in [Ghosh03].

class `mlpy.``Pda`(Nreg=3)

  Penalized Discriminant Analysis (PDA).

  Example:

      >>> import numpy as np
      >>> import mlpy
      >>> xtr = np.array([[1.0, 2.0, 3.1, 1.0],  # first sample
      ...                 [1.0, 2.0, 3.0, 2.0],  # second sample
      ...                 [1.0, 2.0, 3.1, 1.0]]) # third sample
      >>> ytr = np.array([1, -1, 1])             # classes
      >>> mypda = mlpy.Pda()                   # initialize pda class
      >>> mypda.compute(xtr, ytr)              # compute pda
      1
      >>> mypda.predict(xtr)                   # predict pda model on training data
      array([ 1, -1,  1])
      >>> xts = np.array([4.0, 5.0, 6.0, 7.0]) # test point
      >>> mypda.predict(xts)                   # predict pda model on test point
      -1
      >>> mypda.realpred                       # real-valued prediction
      -7.6106885609535624
      >>> mypda.weights(xtr, ytr)              # compute weights on training data
      array([  4.0468174 ,   8.0936348 ,  18.79228266,  58.42466988])

  Initialize Pda class.

  Parameters :

  Nreg : int

    number of regressions

  `compute`(x, y)

    Compute Pda model.

    Parameters :

    x : 2d ndarray float (samples x feats)

      training data

    y : 1d ndarray integer (-1 or 1)

      classes

    Returns : | 1

    Raises :

    LinAlgError

      if x is singular matrix in __PenRegrModel

  `predict`(p)

    Predict Pda model on test point(s).

    Parameters :

    p : 1d or 2d ndarray float (sample(s) x feats)

      test sample(s)

    Returns :

    cl : integer or 1d numpy array integer

      class(es) predicted

    Attributes :

    self.realpred : float or 1d numpy array float

      real valued prediction

  `weights`(x, y)

    Compute feature weights.

    Parameters :

    x : 2d ndarray float (samples x feats)

      training data

    y : 1d ndarray integer (-1 or 1)

      classes

    Returns :

    fw : 1d ndarray float

      feature weights

## Diagonal Linear Discriminant Analysis (DLDA)

class `mlpy.``Dlda`(nf=0, tol=10, overview=False, bal=False)

  Diagonal Linear Discriminant Analysis.

  Example:

      >>> from numpy import *
      >>> from mlpy import *
      >>> xtr = array([[1.1, 2.4, 3.1, 1.0],  # first sample
      ...              [1.2, 2.3, 3.0, 2.0],  # second sample
      ...              [1.3, 2.2, 3.5, 1.0],  # third sample
      ...              [1.4, 2.1, 3.2, 2.0]]) # fourth sample
      >>> ytr = array([1, -1, 1, -1])         # classes
      >>> mydlda = Dlda(nf = 2)                     # initialize dlda class
      >>> mydlda.compute(xtr, ytr)        # compute dlda
      1
      >>> mydlda.predict(xtr)             # predict dlda model on training data
      array([ 1, -1,  1, -1])
      >>> xts = array([4.0, 5.0, 6.0, 7.0])   # test point
      >>> mydlda.predict(xts)                 # predict dlda model on test point
      -1
      >>> mydlda.realpred                     # real-valued prediction
      -21.999999999999954
      >>> mydlda.weights(xtr, ytr)        # compute weights on training data
      array([  2.13162821e-14,   0.00000000e+00,   0.00000000e+00,   4.00000000e+00])

  Initialize Dlda class.

  Parameters :

  nf : int (1 <= nf >= #features)

    the number of the best features that you want to use in the model. If nf = 0 the system stops at a number of features corresponding to a peak of accuracy

  tol : int

    in case of nf = 0 it’s the number of steps of classification to be calculated after the peak to avoid a local maximum

  overview : bool

    set True to print informations about the accuracy of the classifier at every step of the compute

  bal : bool

    set True if it’s reasonable to consider the unbalancement of the test set similar to the one of the training set

  `compute`(x, y, mf=0)

    Compute Dlda model.

    Parameters :

    x : 2d ndarray float (samples x feats)

      training data

    y : 1d ndarray integer (-1 or 1)

      classes

    mf : int

      number of classification steps to be calculated more on a model already computed

    Returns : | 1

    Raises :

    LinAlgError

      if x is singular matrix

  `predict`(p)

    Predict Dlda model on test point(s).

    Parameters :

    p : 1d or 2d ndarray float (sample(s) x feats)

      test sample(s)

    Returns :

    cl : integer or 1d numpy array integer

      class(es) predicted

    Attributes :

    self.realpred : float or 1d numpy array float

      real valued prediction

  `weights`(x, y)

    Return feature weights.

    Parameters :

    x : 2d ndarray float (samples x feats)

      training data

    y : 1d ndarray integer (-1 or 1)

      classes

    Returns :

    fw : 1d ndarray float

      feature weights, they are going to be > 0 for the features chosen for the classification and = 0 for all the others

[Vapnik95] | V Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, 1995.

[Cristianini] | N Cristianini and J Shawe-Taylor. An introduction to support vector machines. Cambridge University Press.

[Merler06] | S Merler and G Jurman. Terminated Ramp - Support Vector Machine: a nonparametric data dependent kernel. Neural Network, 19:1597-1611, 2006.

[Nasr09]

18. Nasr, S. Swamidass, and P. Baldi. Large scale study of multiplemolecule queries. Journal of Cheminformatics, vol. 1, no. 1, p. 7, 2009.

[Mika01] | S Mika and A Smola and B Scholkopf. An improved training algorithm for kernel fisher discriminants. Proceedings AISTATS 2001, 2001.

[Cristianini02] | N Cristianini, J Shawe-Taylor and A Elisseeff. On Kernel-Target Alignment. Advances in Neural Information Processing Systems, Volume 14, 2002.

[Cai08] | D Cai, X He, J Han. SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis. Knowledge and Data Engineering, IEEE Transactions on Volume 20, Issue 1, Jan. 2008 Page(s):1 - 12.

[Ghosh03] | D Ghosh. Penalized discriminant methods for the classification of tumors from gene expression data. Biometrics on Volume 59, Dec. 2003 Page(s):992 - 1000(9).

### Navigation

- index
- modules |
- next |
- previous |
- mlpy 2.2.0 documentation »