### Navigation

- index
- modules |
- next |
- previous |
- mlpy 2.2.0 documentation »

# Regression

## Ordinary Least Squares and Ridge Regression

class `mlpy.``RidgeRegression`(alpha=0.0)

  Ridge Regression and Ordinary Least Squares (OLS).

  Initialization.

  Parameters :

  alpha : float (>= 0.0)

    regularization (0.0: OLS)

  New in version 2.2.0.

  `beta`()

    Return b_1, ..., b_p.

  `beta0`()

    Return b_0.

  `learn`(x, y)

    Compute the regression coefficients.

    Parameters :

    x : numpy 2d array (n x p)

      matrix of regressors

    y : numpy 1d array (n)

      response

  `pred`(x)

    Compute the predicted response.

    Parameters :

    x : numpy 2d array (nxp)

      matrix of regressors

    Returns :

    yp : 1d ndarray

      predicted response

  `selected`()

    Returns the regressors ranking.

Note

The predicted response is computed as:

\hat{y} = \beta_0 + X \boldsymbol\beta

Example (requires matplotlib module):

    >>> import numpy as np
    >>> import mlpy
    >>> import matplotlib.pyplot as plt
    >>> x = np.array([[1], [2], [3], [4], [5], [6]]) # p = 1
    >>> y = np.array([0.13, 0.19, 0.31, 0.38, 0.49, 0.64])
    >>> rr = mlpy.RidgeRegression(alpha=0.0) # OLS
    >>> rr.learn(x, y)
    >>> y_hat = rr.pred(x)
    >>> plt.figure(1)
    >>> plt.plot(x[:, 0], y, 'o') # show y
    >>> plt.plot(x[:, 0], y_hat) # show y_hat
    >>> plt.show()

    >>> rr.beta0()
    0.0046666666666667078
    >>> rr.beta()
    array([ 0.10057143])

## Kernel Ridge Regression

class `mlpy.``KernelRidgeRegression`(kernel, alpha)

  Ridge Regression and Ordinary Least Squares (OLS).

  Initialization.

  Parameters : | alpha : float (> 0.0)

  New in version 2.2.0.

  `learn`(x, y)

    Compute the regression coefficients.

    Parameters :

    x : numpy 2d array (n x p)

      matrix of regressors

    y : numpy 1d array (n)

      response

  `pred`(x)

    Compute the predicted response.

    Parameters :

    x : numpy 2d array (n x p)

      matrix of regressors

    Returns :

    yp : 1d ndarray

      predicted response

Example (requires matplotlib module):

    >>> import numpy as np
    >>> import mlpy
    >>> import matplotlib.pyplot as plt
    >>> x = np.array([[1], [2], [3], [4], [5], [6]]) # p = 1
    >>> y = np.array([0.13, 0.19, 0.31, 0.38, 0.49, 0.64])
    >>> kernel = mlpy.KernelGaussian(sigma=0.01)
    >>> krr = mlpy.KernelRidgeRegression(kernel=kernel, alpha=0.01)
    >>> krr.learn(x,y)
    >>> y_hat = krr.pred(x)
    >>> plt.figure(1)
    >>> plt.plot(x[:, 0], y, 'o') # show y
    >>> plt.plot(x[:, 0], y_hat) # show y_hat
    >>> plt.show()

## Least Angle Regression (LAR)

Least Angle Regression is described in [Efron04].

Covariates should be standardized to have mean 0 and unit length, and the response should have mean 0:

\sum_{i=1}^n{x_{ij}} = 0, \hspace{1cm} \sum_{i=1}^n{x_{ij}^2} = 1, \hspace{1cm} \sum_{i=1}^n{y_i} = 0 \hspace{1cm} \mathrm{for} \hspace{0.2cm} j = 1, 2, \dots, p.

class `mlpy.``Lar`(m=None)

  LAR.

  Initialization.

  Parameters :

  m : int (> 0)

    max number of steps (= number of features selected). If m=None -> m=x.shape[1] in .learn(x, y)

  New in version 2.2.0.

  `beta`()

    Return b_1, ..., b_p.

  `learn`(x, y)

    Compute the regression coefficients.

    Parameters :

    x : numpy 2d array (nxp)

      matrix of regressors

    y : numpy 1d array (n)

      response

  `pred`(x)

    Compute the predicted response.

    Parameters :

    x : numpy 2d array (nxp)

      matrix of regressors

    Returns :

    yp : 1d ndarray

      predicted response

  `selected`()

    Returns the regressors ranking.

  `steps`()

    Return the number of steps really performed.

## LASSO (LARS implementation)

It implements simple modifications of the LARS algorithm that produces Lasso estimates. See [Efron04] and [Tibshirani96].

Covariates should be standardized to have mean 0 and unit length, and the response should have mean 0:

\sum_{i=1}^n{x_{ij}} = 0, \hspace{1cm} \sum_{i=1}^n{x_{ij}^2} = 1, \hspace{1cm} \sum_{i=1}^n{y_i} = 0 \hspace{1cm} \mathrm{for} \hspace{0.2cm} j = 1, 2, \dots, p.

class `mlpy.``Lasso`(m)

  LASSO computed with LARS algoritm.

  Initialization.

  Parameters :

  m : int (> 0)

    max number of steps.

  New in version 2.2.0.

  `beta`()

    Return b_1, ..., b_p.

  `learn`(x, y)

    Compute the regression coefficients.

    Parameters :

    x : numpy 2d array (nxp)

      matrix of regressors

    y : numpy 1d array (n)

      response

  `pred`(x)

    Compute the predicted response.

    Parameters :

    x : numpy 2d array (nxp)

      matrix of regressors

    Returns :

    yp : 1d ndarray

      predicted response

  `selected`()

    Returns the regressors ranking.

  `steps`()

    Return the number of steps really performed.

## Gradient Descent

class `mlpy.``GradientDescent`(kernel, t, stepsize)

  Gradient Descent Method

  Initialization.

  Parameters :

  kernel: kernel object

    kernel

  t : int (> 0)

    number of iterations

  stepsize: float

    step size

  New in version 2.2.0.

  `learn`(x, y)

    Compute the regression coefficients.

    Parameters :

    x : numpy 2d array (n x p)

      matrix of regressors

    y : numpy 1d array (n)

      response

  `pred`(x)

    Compute the predicted response.

    Parameters :

    x : numpy 2d array (n x p)

      matrix of regressors

    Returns :

    yp : 1d ndarray

      predicted response

[Efron04] | (1, 2) Bradley Efron, Trevor Hastie, Iain Johnstone and Robert Tibshirani. Least Angle Regression. Annals of Statistics, 2004, volume 32, pages 407-499.

[Tibshirani96] | Robert Tibshirani. Regression shrinkage and selection via the lasso. J. Royal. Statist. Soc B., 1996, volume 58, number 1, pages 267-288.

### Navigation

- index
- modules |
- next |
- previous |
- mlpy 2.2.0 documentation »