# algorithms.clustering.imm

## Module: `algorithms.clustering.imm`

Inheritance diagram for `nipy.algorithms.clustering.imm`:

Infinite mixture model : A generalization of Bayesian mixture models with an unspecified number of classes

## Classes

### `IMM`

class nipy.algorithms.clustering.imm.IMM(alpha=0.5, dim=1)

  Bases: `BGMM`

  The class implements Infinite Gaussian Mixture model or Dirichlet Process Mixture model. This is simply a generalization of Bayesian Gaussian Mixture Models with an unknown number of classes.

  __init__(alpha=0.5, dim=1)
    Parameters:
      alpha: float, optional,

        the parameter for cluster creation

      dim: int, optional,

        the dimension of the the data

      Note: use the function set_priors() to set adapted priors
  average_log_like(x, tiny=1e-15)

    returns the averaged log-likelihood of the mode for the dataset x

    Parameters:
      x: array of shape (n_samples,self.dim)

        the data used in the estimation process

      tiny = 1.e-15: a small constant to avoid numerical singularities
  bayes_factor(x, z, nperm=0, verbose=0)

    Evaluate the Bayes Factor of the current model using Chib’s method

    Parameters:
      x: array of shape (nb_samples,dim)

        the data from which bic is computed

      z: array of shape (nb_samples), type = np.int_

        the corresponding classification

      nperm=0: int

        the number of permutations to sample to model the label switching issue in the computation of the Bayes Factor By default, exhaustive permutations are used

      verbose=0: verbosity mode
    Returns:
      bf (float) the computed evidence (Bayes factor)

    Notes

    See: Marginal Likelihood from the Gibbs Output Journal article by Siddhartha Chib; Journal of the American Statistical Association, Vol. 90, 1995

  bic(like, tiny=1e-15)

    Computation of bic approximation of evidence

    Parameters:
      like, array of shape (n_samples, self.k)

        component-wise likelihood

      tiny=1.e-15, a small constant to avoid numerical singularities
    Returns:
      the bic value, float
  check()

    Checking the shape of sifferent matrices involved in the model

  check_x(x)

    essentially check that x.shape[1]==self.dim

    x is returned with possibly reshaping

  conditional_posterior_proba(x, z, perm=None)

    Compute the probability of the current parameters of self given x and z

    Parameters:
      x: array of shape (nb_samples, dim),

        the data from which bic is computed

      z: array of shape (nb_samples), type = np.int_,

        the corresponding classification

      perm: array ok shape(nperm, self.k),typ=np.int_, optional

        all permutation of z under which things will be recomputed By default, no permutation is performed

  cross_validated_update(x, z, plike, kfold=10)

    This is a step in the sampling procedure that uses internal corss_validation

    Parameters:
      x: array of shape(n_samples, dim),

        the input data

      z: array of shape(n_samples),

        the associated membership variables

      plike: array of shape(n_samples),

        the likelihood under the prior

      kfold: int, or array of shape(n_samples), optional,

        folds in the cross-validation loop

    Returns:
      like: array od shape(n_samples),

        the (cross-validated) likelihood of the data

  estimate(x, niter=100, delta=0.0001, verbose=0)

    Estimation of the model given a dataset x

    Parameters:
      x array of shape (n_samples,dim)

        the data from which the model is estimated

      niter=100: maximal number of iterations in the estimation process
      delta = 1.e-4: increment of data likelihood at which

        convergence is declared

      verbose=0: verbosity mode
    Returns:
      bic an asymptotic approximation of model evidence
  evidence(x, z, nperm=0, verbose=0)

    See bayes_factor(self, x, z, nperm=0, verbose=0)

  guess_priors(x, nocheck=0)

    Set the priors in order of having them weakly uninformative this is from Fraley and raftery; Journal of Classification 24:155-181 (2007)

    Parameters:
      x, array of shape (nb_samples,self.dim)

        the data used in the estimation process

      nocheck: boolean, optional,

        if nocheck==True, check is skipped

  guess_regularizing(x, bcheck=1)

    Set the regularizing priors as weakly informative according to Fraley and raftery; Journal of Classification 24:155-181 (2007)

    Parameters:
      x array of shape (n_samples,dim)

        the data used in the estimation process

  initialize(x)

    initialize z using a k-means algorithm, then update the parameters

    Parameters:
      x: array of shape (nb_samples,self.dim)

        the data used in the estimation process

  initialize_and_estimate(x, z=None, niter=100, delta=0.0001, ninit=1, verbose=0)

    Estimation of self given x

    Parameters:
      x array of shape (n_samples,dim)

        the data from which the model is estimated

      z = None: array of shape (n_samples)

        a prior labelling of the data to initialize the computation

      niter=100: maximal number of iterations in the estimation process
      delta = 1.e-4: increment of data likelihood at which

        convergence is declared

      ninit=1: number of initialization performed

        to reach a good solution

      verbose=0: verbosity mode
    Returns:
      the best model is returned
  likelihood(x, plike=None)

    return the likelihood of the model for the data x the values are weighted by the components weights

    Parameters:
      x: array of shape (n_samples, self.dim),

        the data used in the estimation process

      plike: array of shape (n_samples), optional,

        the density of each point under the prior

    Returns:
      like, array of shape (nbitem, self.k)
      component-wise likelihood
  likelihood_under_the_prior(x)

    Computes the likelihood of x under the prior

    Parameters:
      x, array of shape (self.n_samples,self.dim)
    Returns:
      w, the likelihood of x under the prior model (unweighted)
  map_label(x, like=None)

    return the MAP labelling of x

    Parameters:
      x array of shape (n_samples,dim)

        the data under study

      like=None array of shape(n_samples,self.k)

        component-wise likelihood if like==None, it is recomputed

    Returns:
      z: array of shape(n_samples): the resulting MAP labelling

        of the rows of x

  mixture_likelihood(x)

    Returns the likelihood of the mixture for x

    Parameters:
      x: array of shape (n_samples,self.dim)

        the data used in the estimation process

  plugin(means, precisions, weights)

    Set manually the weights, means and precision of the model

    Parameters:
      means: array of shape (self.k,self.dim)
      precisions: array of shape (self.k,self.dim,self.dim)

        or (self.k, self.dim)

      weights: array of shape (self.k)
  pop(z)

    compute the population, i.e. the statistics of allocation

    Parameters:
      z array of shape (nb_samples), type = np.int_

        the allocation variable

    Returns:
      hist array shape (self.k) count variable
  probability_under_prior()

    Compute the probability of the current parameters of self given the priors

  reduce(z)

    Reduce the assignments by removing empty clusters and update self.k

    Parameters:
      z: array of shape(n),

        a vector of membership variables changed in place

    Returns:
      z: the remapped values
  sample(x, niter=1, sampling_points=None, init=False, kfold=None, verbose=0)

    sample the indicator and parameters

    Parameters:
      x: array of shape (n_samples, self.dim)

        the data used in the estimation process

      niter: int,

        the number of iterations to perform

      sampling_points: array of shape(nbpoints, self.dim), optional

        points where the likelihood will be sampled this defaults to x

      kfold: int or array, optional,

        parameter of cross-validation control by default, no cross-validation is used the procedure is faster but less accurate

      verbose=0: verbosity mode
    Returns:
      likelihood: array of shape(nbpoints)

        total likelihood of the model

  sample_and_average(x, niter=1, verbose=0)

    sample the indicator and parameters the average values for weights,means, precisions are returned

    Parameters:
      x = array of shape (nb_samples,dim)

        the data from which bic is computed

      niter=1: number of iterations
    Returns:
      weights: array of shape (self.k)
      means: array of shape (self.k,self.dim)
      precisions: array of shape (self.k,self.dim,self.dim)

        or (self.k, self.dim) these are the average parameters across samplings

    Notes

    All this makes sense only if no label switching as occurred so this is wrong in general (asymptotically).

    fix: implement a permutation procedure for components identification

  sample_indicator(like)

    Sample the indicator from the likelihood

    Parameters:
      like: array of shape (nbitem,self.k)

        component-wise likelihood

    Returns:
      z: array of shape(nbitem): a draw of the membership variable

    Notes

    The behaviour is different from standard bgmm in that z can take arbitrary values

  set_constant_densities(prior_dens=None)

    Set the null and prior densities as constant (assuming a compact domain)

    Parameters:
      prior_dens: float, optional

        constant for the prior density

  set_priors(x)

    Set the priors in order of having them weakly uninformative this is from Fraley and raftery; Journal of Classification 24:155-181 (2007)

    Parameters:
      x, array of shape (n_samples,self.dim)

        the data used in the estimation process

  show(x, gd, density=None, axes=None)

    Function to plot a GMM, still in progress Currently, works only in 1D and 2D

    Parameters:
      x: array of shape(n_samples, dim)

        the data under study

      gd: GridDescriptor instance
      density: array os shape(prod(gd.n_bins))

        density of the model one the discrete grid implied by gd by default, this is recomputed

  show_components(x, gd, density=None, mpaxes=None)

    Function to plot a GMM – Currently, works only in 1D

    Parameters:
      x: array of shape(n_samples, dim)

        the data under study

      gd: GridDescriptor instance
      density: array os shape(prod(gd.n_bins))

        density of the model one the discrete grid implied by gd by default, this is recomputed

      mpaxes: axes handle to make the figure, optional,

        if None, a new figure is created

  simple_update(x, z, plike)

    > This is a step in the sampling procedure

    that uses internal corss_validation

    Parameters:
      x: array of shape(n_samples, dim),

        the input data

      z: array of shape(n_samples),

        the associated membership variables

      plike: array of shape(n_samples),

        the likelihood under the prior

    Returns:
      like: array od shape(n_samples),

        the likelihood of the data

  test(x, tiny=1e-15)

    Returns the log-likelihood of the mixture for x

    Parameters:
      x array of shape (n_samples,self.dim)

        the data used in the estimation process

    Returns:
      ll: array of shape(n_samples)

        the log-likelihood of the rows of x

  train(x, z=None, niter=100, delta=0.0001, ninit=1, verbose=0)

    Idem initialize_and_estimate

  unweighted_likelihood(x)

    return the likelihood of each data for each component the values are not weighted by the component weights

    Parameters:
      x: array of shape (n_samples,self.dim)

        the data used in the estimation process

    Returns:
      like, array of shape(n_samples,self.k)

        unweighted component-wise likelihood

    Notes

    Hopefully faster

  unweighted_likelihood_(x)

    return the likelihood of each data for each component the values are not weighted by the component weights

    Parameters:
      x: array of shape (n_samples,self.dim)

        the data used in the estimation process

    Returns:
      like, array of shape(n_samples,self.k)

        unweighted component-wise likelihood

  update(x, z)

    Update function (draw a sample of the IMM parameters)

    Parameters:
      x array of shape (n_samples,self.dim)

        the data used in the estimation process

      z array of shape (n_samples), type = np.int_

        the corresponding classification

  update_means(x, z)

    Given the allocation vector z, and the corresponding data x, resample the mean

    Parameters:
      x: array of shape (nb_samples,self.dim)

        the data used in the estimation process

      z: array of shape (nb_samples), type = np.int_

        the corresponding classification

  update_precisions(x, z)

    Given the allocation vector z, and the corresponding data x, resample the precisions

    Parameters:
      x array of shape (nb_samples,self.dim)

        the data used in the estimation process

      z array of shape (nb_samples), type = np.int_

        the corresponding classification

  update_weights(z)

    Given the allocation vector z, resmaple the weights parameter

    Parameters:
      z array of shape (n_samples), type = np.int_

        the allocation variable

### `MixedIMM`

class nipy.algorithms.clustering.imm.MixedIMM(alpha=0.5, dim=1)

  Bases: `IMM`

  Particular IMM with an additional null class. The data is supplied together with a sample-related probability of being under the null.

  __init__(alpha=0.5, dim=1)
    Parameters:
      alpha: float, optional,

        the parameter for cluster creation

      dim: int, optional,

        the dimension of the the data

      Note: use the function set_priors() to set adapted priors
  average_log_like(x, tiny=1e-15)

    returns the averaged log-likelihood of the mode for the dataset x

    Parameters:
      x: array of shape (n_samples,self.dim)

        the data used in the estimation process

      tiny = 1.e-15: a small constant to avoid numerical singularities
  bayes_factor(x, z, nperm=0, verbose=0)

    Evaluate the Bayes Factor of the current model using Chib’s method

    Parameters:
      x: array of shape (nb_samples,dim)

        the data from which bic is computed

      z: array of shape (nb_samples), type = np.int_

        the corresponding classification

      nperm=0: int

        the number of permutations to sample to model the label switching issue in the computation of the Bayes Factor By default, exhaustive permutations are used

      verbose=0: verbosity mode
    Returns:
      bf (float) the computed evidence (Bayes factor)

    Notes

    See: Marginal Likelihood from the Gibbs Output Journal article by Siddhartha Chib; Journal of the American Statistical Association, Vol. 90, 1995

  bic(like, tiny=1e-15)

    Computation of bic approximation of evidence

    Parameters:
      like, array of shape (n_samples, self.k)

        component-wise likelihood

      tiny=1.e-15, a small constant to avoid numerical singularities
    Returns:
      the bic value, float
  check()

    Checking the shape of sifferent matrices involved in the model

  check_x(x)

    essentially check that x.shape[1]==self.dim

    x is returned with possibly reshaping

  conditional_posterior_proba(x, z, perm=None)

    Compute the probability of the current parameters of self given x and z

    Parameters:
      x: array of shape (nb_samples, dim),

        the data from which bic is computed

      z: array of shape (nb_samples), type = np.int_,

        the corresponding classification

      perm: array ok shape(nperm, self.k),typ=np.int_, optional

        all permutation of z under which things will be recomputed By default, no permutation is performed

  cross_validated_update(x, z, plike, null_class_proba, kfold=10)

    This is a step in the sampling procedure that uses internal corss_validation

    Parameters:
      x: array of shape(n_samples, dim),

        the input data

      z: array of shape(n_samples),

        the associated membership variables

      plike: array of shape(n_samples),

        the likelihood under the prior

      kfold: int, optional, or array

        number of folds in cross-validation loop or set of indexes for the cross-validation procedure

      null_class_proba: array of shape(n_samples),

        prior probability to be under the null

    Returns:
      like: array od shape(n_samples),

        the (cross-validated) likelihood of the data

      z: array of shape(n_samples),

        the associated membership variables

    Notes

    When kfold is an array, there is an internal reshuffling to randomize the order of updates

  estimate(x, niter=100, delta=0.0001, verbose=0)

    Estimation of the model given a dataset x

    Parameters:
      x array of shape (n_samples,dim)

        the data from which the model is estimated

      niter=100: maximal number of iterations in the estimation process
      delta = 1.e-4: increment of data likelihood at which

        convergence is declared

      verbose=0: verbosity mode
    Returns:
      bic an asymptotic approximation of model evidence
  evidence(x, z, nperm=0, verbose=0)

    See bayes_factor(self, x, z, nperm=0, verbose=0)

  guess_priors(x, nocheck=0)

    Set the priors in order of having them weakly uninformative this is from Fraley and raftery; Journal of Classification 24:155-181 (2007)

    Parameters:
      x, array of shape (nb_samples,self.dim)

        the data used in the estimation process

      nocheck: boolean, optional,

        if nocheck==True, check is skipped

  guess_regularizing(x, bcheck=1)

    Set the regularizing priors as weakly informative according to Fraley and raftery; Journal of Classification 24:155-181 (2007)

    Parameters:
      x array of shape (n_samples,dim)

        the data used in the estimation process

  initialize(x)

    initialize z using a k-means algorithm, then update the parameters

    Parameters:
      x: array of shape (nb_samples,self.dim)

        the data used in the estimation process

  initialize_and_estimate(x, z=None, niter=100, delta=0.0001, ninit=1, verbose=0)

    Estimation of self given x

    Parameters:
      x array of shape (n_samples,dim)

        the data from which the model is estimated

      z = None: array of shape (n_samples)

        a prior labelling of the data to initialize the computation

      niter=100: maximal number of iterations in the estimation process
      delta = 1.e-4: increment of data likelihood at which

        convergence is declared

      ninit=1: number of initialization performed

        to reach a good solution

      verbose=0: verbosity mode
    Returns:
      the best model is returned
  likelihood(x, plike=None)

    return the likelihood of the model for the data x the values are weighted by the components weights

    Parameters:
      x: array of shape (n_samples, self.dim),

        the data used in the estimation process

      plike: array of shape (n_samples), optional,

        the density of each point under the prior

    Returns:
      like, array of shape (nbitem, self.k)
      component-wise likelihood
  likelihood_under_the_prior(x)

    Computes the likelihood of x under the prior

    Parameters:
      x, array of shape (self.n_samples,self.dim)
    Returns:
      w, the likelihood of x under the prior model (unweighted)
  map_label(x, like=None)

    return the MAP labelling of x

    Parameters:
      x array of shape (n_samples,dim)

        the data under study

      like=None array of shape(n_samples,self.k)

        component-wise likelihood if like==None, it is recomputed

    Returns:
      z: array of shape(n_samples): the resulting MAP labelling

        of the rows of x

  mixture_likelihood(x)

    Returns the likelihood of the mixture for x

    Parameters:
      x: array of shape (n_samples,self.dim)

        the data used in the estimation process

  plugin(means, precisions, weights)

    Set manually the weights, means and precision of the model

    Parameters:
      means: array of shape (self.k,self.dim)
      precisions: array of shape (self.k,self.dim,self.dim)

        or (self.k, self.dim)

      weights: array of shape (self.k)
  pop(z)

    compute the population, i.e. the statistics of allocation

    Parameters:
      z array of shape (nb_samples), type = np.int_

        the allocation variable

    Returns:
      hist array shape (self.k) count variable
  probability_under_prior()

    Compute the probability of the current parameters of self given the priors

  reduce(z)

    Reduce the assignments by removing empty clusters and update self.k

    Parameters:
      z: array of shape(n),

        a vector of membership variables changed in place

    Returns:
      z: the remapped values
  sample(x, null_class_proba, niter=1, sampling_points=None, init=False, kfold=None, co_clustering=False, verbose=0)

    sample the indicator and parameters

    Parameters:
      x: array of shape (n_samples, self.dim),

        the data used in the estimation process

      null_class_proba: array of shape(n_samples),

        the probability to be under the null

      niter: int,

        the number of iterations to perform

      sampling_points: array of shape(nbpoints, self.dim), optional

        points where the likelihood will be sampled this defaults to x

      kfold: int, optional,

        parameter of cross-validation control by default, no cross-validation is used the procedure is faster but less accurate

      co_clustering: bool, optional

        if True, return a model of data co-labelling across iterations

      verbose=0: verbosity mode
    Returns:
      likelihood: array of shape(nbpoints)

        total likelihood of the model

      pproba: array of shape(n_samples),

        the posterior of being in the null (the posterior of null_class_proba)

      coclust: only if co_clustering==True,

        sparse_matrix of shape (n_samples, n_samples), frequency of co-labelling of each sample pairs across iterations

  sample_and_average(x, niter=1, verbose=0)

    sample the indicator and parameters the average values for weights,means, precisions are returned

    Parameters:
      x = array of shape (nb_samples,dim)

        the data from which bic is computed

      niter=1: number of iterations
    Returns:
      weights: array of shape (self.k)
      means: array of shape (self.k,self.dim)
      precisions: array of shape (self.k,self.dim,self.dim)

        or (self.k, self.dim) these are the average parameters across samplings

    Notes

    All this makes sense only if no label switching as occurred so this is wrong in general (asymptotically).

    fix: implement a permutation procedure for components identification

  sample_indicator(like, null_class_proba)

    sample the indicator from the likelihood

    Parameters:
      like: array of shape (nbitem,self.k)

        component-wise likelihood

      null_class_proba: array of shape(n_samples),

        prior probability to be under the null

    Returns:
      z: array of shape(nbitem): a draw of the membership variable

    Notes

    Here z=-1 encodes for the null class

  set_constant_densities(null_dens=None, prior_dens=None)

    Set the null and prior densities as constant (over a supposedly compact domain)

    Parameters:
      null_dens: float, optional

        constant for the null density

      prior_dens: float, optional

        constant for the prior density

  set_priors(x)

    Set the priors in order of having them weakly uninformative this is from Fraley and raftery; Journal of Classification 24:155-181 (2007)

    Parameters:
      x, array of shape (n_samples,self.dim)

        the data used in the estimation process

  show(x, gd, density=None, axes=None)

    Function to plot a GMM, still in progress Currently, works only in 1D and 2D

    Parameters:
      x: array of shape(n_samples, dim)

        the data under study

      gd: GridDescriptor instance
      density: array os shape(prod(gd.n_bins))

        density of the model one the discrete grid implied by gd by default, this is recomputed

  show_components(x, gd, density=None, mpaxes=None)

    Function to plot a GMM – Currently, works only in 1D

    Parameters:
      x: array of shape(n_samples, dim)

        the data under study

      gd: GridDescriptor instance
      density: array os shape(prod(gd.n_bins))

        density of the model one the discrete grid implied by gd by default, this is recomputed

      mpaxes: axes handle to make the figure, optional,

        if None, a new figure is created

  simple_update(x, z, plike, null_class_proba)

    One step in the sampling procedure (one data sweep)

    Parameters:
      x: array of shape(n_samples, dim),

        the input data

      z: array of shape(n_samples),

        the associated membership variables

      plike: array of shape(n_samples),

        the likelihood under the prior

      null_class_proba: array of shape(n_samples),

        prior probability to be under the null

    Returns:
      like: array od shape(n_samples),

        the likelihood of the data under the H1 hypothesis

  test(x, tiny=1e-15)

    Returns the log-likelihood of the mixture for x

    Parameters:
      x array of shape (n_samples,self.dim)

        the data used in the estimation process

    Returns:
      ll: array of shape(n_samples)

        the log-likelihood of the rows of x

  train(x, z=None, niter=100, delta=0.0001, ninit=1, verbose=0)

    Idem initialize_and_estimate

  unweighted_likelihood(x)

    return the likelihood of each data for each component the values are not weighted by the component weights

    Parameters:
      x: array of shape (n_samples,self.dim)

        the data used in the estimation process

    Returns:
      like, array of shape(n_samples,self.k)

        unweighted component-wise likelihood

    Notes

    Hopefully faster

  unweighted_likelihood_(x)

    return the likelihood of each data for each component the values are not weighted by the component weights

    Parameters:
      x: array of shape (n_samples,self.dim)

        the data used in the estimation process

    Returns:
      like, array of shape(n_samples,self.k)

        unweighted component-wise likelihood

  update(x, z)

    Update function (draw a sample of the IMM parameters)

    Parameters:
      x array of shape (n_samples,self.dim)

        the data used in the estimation process

      z array of shape (n_samples), type = np.int_

        the corresponding classification

  update_means(x, z)

    Given the allocation vector z, and the corresponding data x, resample the mean

    Parameters:
      x: array of shape (nb_samples,self.dim)

        the data used in the estimation process

      z: array of shape (nb_samples), type = np.int_

        the corresponding classification

  update_precisions(x, z)

    Given the allocation vector z, and the corresponding data x, resample the precisions

    Parameters:
      x array of shape (nb_samples,self.dim)

        the data used in the estimation process

      z array of shape (nb_samples), type = np.int_

        the corresponding classification

  update_weights(z)

    Given the allocation vector z, resmaple the weights parameter

    Parameters:
      z array of shape (n_samples), type = np.int_

        the allocation variable

## Functions

nipy.algorithms.clustering.imm.co_labelling(z, kmax=None, kmin=None)

  return a sparse co-labelling matrix given the label vector z

  Parameters:
    z: array of shape(n_samples),

      the input labels

    kmax: int, optional,

      considers only the labels in the range [0, kmax[

  Returns:
    colabel: a sparse coo_matrix,

      yields the co labelling of the data i.e. c[i,j]= 1 if z[i]==z[j], 0 otherwise

nipy.algorithms.clustering.imm.main()

  Illustrative example of the behaviour of imm