| Multivariate Pattern Analysis in Python

### Navigation

- index
- next |
- previous |
- ExpPsy Project |
- NeuroDebian Project |
- PyMVPA 0.4.x Home |
- PyMVPA Home |
- Documentation »
- Module Reference »

This content refers to the previous stable release of PyMVPA. Please visit www.pymvpa.org for the most recent version of PyMVPA and its documentation.

# clfs.transerror

## Module: `clfs.transerror`

Inheritance diagram for `mvpa.clfs.transerror`:

Utility class to compute the transfer error of classifiers.

## Classes

### `ClassifierError`

class `mvpa.clfs.transerror.``ClassifierError`(clf, labels=None, train=True, **kwargs)

  Bases: `mvpa.misc.state.ClassWithCollections`

  Compute (or return) some error of a (trained) classifier on a dataset.

  See also

  Please refer to the documentation of the base class for more information:

  `ClassWithCollections`

  Note

  Available state variables:

  - confusion: State variable
  - training_confusion: Proxy training_confusion from underlying classifier.

  (States enabled by default are listed with +)

  See also

  Please refer to the documentation of the base class for more information:

  `ClassWithCollections`

  Initialization.

  Parameters:

  - clf (Classifier) – Either trained or untrained classifier
  - labels (list) – if provided, should be a set of labels to add on top of the ones present in testdata
  - train (bool) – unless train=False, classifier gets trained if trainingdata provided to __call__
  - enable_states (None or list of basestring) – Names of the state variables which should be enabled additionally to default ones
  - disable_states (None or list of basestring) – Names of the state variables which should be disabled

  `clf`
  `confusion` = None

    TODO Think that labels might be also symbolic thus can’t directly be indicies of the array

  `labels`
  `untrain`()

    Untrain the *Error which relies on the classifier

### `ConfusionBasedError`

class `mvpa.clfs.transerror.``ConfusionBasedError`(clf, labels=None, confusion_state='training_confusion', **kwargs)

  Bases: `mvpa.clfs.transerror.ClassifierError`

  For a given classifier report an error based on internally computed error measure (given by some ConfusionMatrix stored in some state variable of Classifier).

  This way we can perform feature selection taking as the error criterion either learning error, or transfer to splits error in the case of SplitClassifier

  See also

  Please refer to the documentation of the base class for more information:

  `ClassifierError`

  Note

  Available state variables:

  - confusion: State variable
  - training_confusion: Proxy training_confusion from underlying classifier.

  (States enabled by default are listed with +)

  See also

  Please refer to the documentation of the base class for more information:

  `ClassifierError`

  Initialization.

  Parameters:

  - clf (Classifier) – Either trained or untrained classifier
  - confusion_state – Id of the state variable which stores ConfusionMatrix
  - labels (list) – if provided, should be a set of labels to add on top of the ones present in testdata
  - train (bool) – unless train=False, classifier gets trained if trainingdata provided to __call__
  - enable_states (None or list of basestring) – Names of the state variables which should be enabled additionally to default ones
  - disable_states (None or list of basestring) – Names of the state variables which should be disabled
  - enable_states – Names of the state variables which should be enabled additionally to default ones
  - disable_states – Names of the state variables which should be disabled

### `ConfusionMatrix`

class `mvpa.clfs.transerror.``ConfusionMatrix`(labels=None, labels_map=None, **kwargs)

  Bases: `mvpa.clfs.transerror.SummaryStatistics`

  Class to contain information and display confusion matrix.

  Implementation of the SummaryStatistics in the case of classification problem. Actual computation of confusion matrix is delayed until all data is acquired (to figure out complete set of labels). If testing data doesn’t have a complete set of labels, but you like to include all labels, provide them as a parameter to the constructor.

  Confusion matrix provides a set of performance statistics (use asstring(description=True) for the description of abbreviations), as well ROC curve (http://en.wikipedia.org/wiki/ROC_curve) plotting and analysis (AUC) in the limited set of problems: binary, multiclass 1-vs-all.

  Initialize ConfusionMatrix with optional list of labels

  Parameters:

  - labels (list) – Optional set of labels to include in the matrix
  - labels_map (None or dict) – Dictionary from original dataset to show mapping into numerical labels
  - targets – Optional set of targets
  - predictions – Optional set of predictions

  `asstring`(short=False, header=True, summary=True, description=False)

    ‘Pretty print’ the matrix

    Parameters:

    - short (bool) – if True, ignores the rest of the parameters and provides consise 1 line summary
    - header (bool) – print header of the table
    - summary (bool) – print summary (accuracy)
    - description (bool) – print verbose description of presented statistics

  `error`
  `getLabels_map`()
  `labels`
  `labels_map`
  `matrices`

    Return a list of separate confusion matrix per each stored set

  `matrix`
  `percentCorrect`
  `plot`(labels=None, numbers=False, origin='upper', numbers_alpha=None, xlabels_vertical=True, numbers_kwargs={}, **kwargs)

    Provide presentation of confusion matrix in image

    Parameters:

    - labels (list of int or basestring) – Optionally provided labels guarantee the order of presentation. Also value of None places empty column/row, thus provides visual groupping of labels (Thanks Ingo)
    - numbers (bool) – Place values inside of confusion matrix elements
    - numbers_alpha (None or float) – Controls textual output of numbers. If None – all numbers are plotted in the same intensity. If some float – it controls alpha level – higher value would give higher contrast. (good value is 2)
    - origin (basestring) – Which left corner diagonal should start
    - xlabels_vertical (bool) – Either to plot xlabels vertical (benefitial if number of labels is large)
    - numbers_kwargs (dict) – Additional keyword parameters to be added to numbers (if numbers is True)
    - **kwargs – Additional arguments given to imshow (eg me cmap)

    Return type: | (fig, im, cb) – figure, imshow, colorbar

  `setLabels_map`(val)

### `ROCCurve`

class `mvpa.clfs.transerror.``ROCCurve`(labels, sets=None)

  Bases: `object`

  Generic class for ROC curve computation and plotting

  Parameters:

  - labels (list) – labels which were used (in order of values if multiclass, or 1 per class for binary problems (e.g. in SMLR))
  - sets (list of tuples) – list of sets for the analysis

  `ROCs`
  `aucs`

    Compute and return set of AUC values 1 per label

  `plot`(label_index=0)
    TODO: make it friendly to labels given by values?
      should we also treat labels_map?

### `RegressionStatistics`

class `mvpa.clfs.transerror.``RegressionStatistics`(**kwargs)

  Bases: `mvpa.clfs.transerror.SummaryStatistics`

  Class to contain information and display on regression results.

  Initialize RegressionStatistics

  Parameters:

  - targets – Optional set of targets
  - predictions – Optional set of predictions

  `asstring`(short=False, header=True, summary=True, description=False)

    ‘Pretty print’ the statistics

  `error`
  `plot`(plot=True, plot_stats=True, splot=True)

    Provide presentation of regression performance in image

    Parameters:

    - plot (bool) – Plot regular plot of values (targets/predictions)
    - plot_stats (bool) – Print basic statistics in the title
    - splot (bool) – Plot scatter plot

    Return type: | (fig, im, cb) – figure, imshow, colorbar

### `SummaryStatistics`

class `mvpa.clfs.transerror.``SummaryStatistics`(targets=None, predictions=None, values=None, sets=None)

  Bases: `object`

  Basic class to collect targets/predictions and report summary statistics

  It takes care about collecting the sets, which are just tuples (targets, predictions, values). While ‘computing’ the matrix, all sets are considered together. Children of the class are responsible for computation and display.

  Initialize SummaryStatistics

  targets or predictions cannot be provided alone (ie targets without predictions)

  Parameters:

  - targets – Optional set of targets
  - predictions – Optional set of predictions
  - values – Optional set of values (which served for prediction)
  - sets – Optional list of sets

  `add`(targets, predictions, values=None)

    Add new results to the set of known results

  `asstring`(short=False, header=True, summary=True, description=False)

    ‘Pretty print’ the matrix

    Parameters:

    - short (bool) – if True, ignores the rest of the parameters and provides consise 1 line summary
    - header (bool) – print header of the table
    - summary (bool) – print summary (accuracy)
    - description (bool) – print verbose description of presented statistics

  `compute`()

    Actually compute the confusion matrix based on all the sets

  `error`
  `reset`()

    Cleans summary – all data/sets are wiped out

  `sets`
  `stats`
  `summaries`

    Return a list of separate summaries per each stored set

### `TransferError`

class `mvpa.clfs.transerror.``TransferError`(clf, errorfx=MeanMismatchErrorFx(), labels=None, null_dist=None, **kwargs)

  Bases: `mvpa.clfs.transerror.ClassifierError`

  Compute the transfer error of a (trained) classifier on a dataset.

  The actual error value is computed using a customizable error function. Optionally the classifier can be trained by passing an additional training dataset to the __call__() method.

  See also

  Please refer to the documentation of the base class for more information:

  `ClassifierError`

  Note

  Available state variables:

  - confusion: State variable
  - null_prob+: Stores the probability of an error result under the NULL hypothesis
  - samples_error: Per sample errors computed by invoking the error function for each sample individually. Errors are available in a dictionary with each samples origid as key.
  - training_confusion: Proxy training_confusion from underlying classifier.

  (States enabled by default are listed with +)

  See also

  Please refer to the documentation of the base class for more information:

  `ClassifierError`

  Initialization.

  Parameters:

  - clf (Classifier) – Either trained or untrained classifier
  - errorfx – Functor that computes a scalar error value from the vectors of desired and predicted values (e.g. subclass of ErrorFunction)
  - labels (list) – if provided, should be a set of labels to add on top of the ones present in testdata
  - null_dist (instance of distribution estimator) –
  - train (bool) – unless train=False, classifier gets trained if trainingdata provided to __call__
  - enable_states (None or list of basestring) – Names of the state variables which should be enabled additionally to default ones
  - disable_states (None or list of basestring) – Names of the state variables which should be disabled
  - enable_states – Names of the state variables which should be enabled additionally to default ones
  - disable_states – Names of the state variables which should be disabled

  `errorfx`
  `null_dist`

### Navigation

- index
- next |
- previous |
- ExpPsy Project |
- NeuroDebian Project |
- PyMVPA 0.4.x Home |
- PyMVPA Home |
- Documentation »
- Module Reference »