# labs.utils.reproducibility_measures

## Module: `labs.utils.reproducibility_measures`

Functions for computing reproducibility measures.

General procedure is:

  - dataset is subject to jacknife subampling (‘splitting’),

  - each subsample being analysed independently,

  - a reproducibility measure is then derived;

It is used to produce the work described in Analysis of a large fMRI cohort:

Statistical and methodological issues for group analyses. Thirion B, Pinel P, Meriaux S, Roche A, Dehaene S, Poline JB. Neuroimage. 2007 Mar;35(1):105-20.

Bertrand Thirion, 2009-2010

## Functions

nipy.labs.utils.reproducibility_measures.bootstrap_group(nsubj, ngroups)

  Split the proposed group into redundant subgroups by bootstrap

  Parameters:
    nsubj (int) the number of subjects in the population
    ngroups(int) Number of subbgroups to be drawn
  Returns:
    samples: a list of ngroups arrays containing

      the indexes of the subjects in each subgroup

nipy.labs.utils.reproducibility_measures.cluster_reproducibility(data, vardata, domain, ngroups, sigma, method='crfx', swap=False, verbose=0, **kwargs)

  Returns a measure of cluster-level reproducibility of activation patterns (i.e. how far clusters are from each other)

  Parameters:
    data: array of shape (nvox,nsubj)

      the input data from which everything is computed

    vardata: array of shape (nvox,nsubj)

      the variance of the data that is also available

    domain: referential- and domain- defining image instance
    ngroups (int),

      Number of subbgroups to be drawn

    sigma (float): parameter that encodes how far far is
    threshold (float):

      binarization threshold

    method=’crfx’, string to be chosen among ‘crfx’, ‘cmfx’ or ‘cffx’

      inference method under study

    swap = False: if True, a random sign swap of the data is performed

      This is used to simulate a null hypothesis on the data.

    verbose=0 verbosity mode
  Returns:
    score (float): the desired cluster-level reproducibility index

nipy.labs.utils.reproducibility_measures.cluster_threshold(stat_map, domain, th, csize)

  Perform a thresholding of a map at the cluster-level

  Parameters:
    stat_map: array of shape(nbvox)

      the input data

    domain: Nifti1Image instance,

      referential- and domain-defining image

    th (float): cluster-forming threshold
    csize (int>0): cluster size threshold
  Returns:
    binary array of shape (nvox): the binarized thresholded map

  Notes

  Should be replaced by a more standard function in the future

nipy.labs.utils.reproducibility_measures.conjunction(x, vx, k)

  Returns a conjunction statistic as the sum of the k lowest t-values

  Parameters:
    x: array of shape(nrows, ncols),

      effect matrix

    vx: array of shape(nrows, ncols),

      variance matrix

    k: int,

      number of subjects in the conjunction

  Returns:
    t array of shape(nrows): conjunction statistic

nipy.labs.utils.reproducibility_measures.coord_bsa(domain, betas, theta=3.0, dmax=5.0, ths=0, thq=0.5, smin=0, afname=None)

  main function for performing bsa on a dataset where bsa = nipy.labs.spatial_models.bayesian_structural_analysis

  Parameters:
    domain: image instance,

      referential- and domain-defining image

    betas: array of shape (nbnodes, subjects),

      the multi-subject statistical maps

    theta: float, optional

      first level threshold

    dmax: float>0, optional

      expected cluster std in the common space in units of coord

    ths: int, >=0), optional

      representatitivity threshold

    thq: float, optional,

      posterior significance threshold should be in [0,1]

    smin: int, optional,

      minimal size of the regions to validate them

    afname: string, optional

      path where intermediate results cam be pickled

  Returns:
    afcoord array of shape(number_of_regions,3):

      coordinate of the found landmark regions

nipy.labs.utils.reproducibility_measures.draw_samples(nsubj, ngroups, split_method='default')

  Draw randomly ngroups sets of samples from [0..nsubj-1]

  Parameters:
    nsubj, int, the total number of items
    ngroups, int, the number of desired groups
    split_method: string, optional,

      to be chosen among ‘default’, ‘bootstrap’, ‘jacknife’ if ‘bootstrap’, then each group will be nsubj

      > drawn with repetitions among nsubj

      if ‘jacknife’ the population is divided into

        ngroups disjoint equally-sized subgroups

      if ‘default’, ‘bootstrap’ is used when nsubj < 10 * ngroups

        otherwise jacknife is used

  Returns:
    samples, a list of ngroups array that represent the subsets.
    fixme this should allow variable bootstrap,
    i.e. draw ngroups of groupsize among nsubj

nipy.labs.utils.reproducibility_measures.fttest(x, vx)

  Assuming that x and vx represent a effect and variance estimates, returns a cumulated (‘fixed effects’) t-test of the data over each row

  Parameters:
    x: array of shape(nrows, ncols): effect matrix
    vx: array of shape(nrows, ncols): variance matrix
  Returns:
    t array of shape(nrows): fixed effect statistics array

nipy.labs.utils.reproducibility_measures.get_cluster_position_from_thresholded_map(stat_map, domain, thr=3.0, csize=10)

  the clusters above thr of size greater than csize in 18-connectivity are computed

  Parameters:
    stat_map array of shape (nbvox),

      map to threshold

    mask: Nifti1Image instance,

      referential- and domain-defining image

    thr: float, optional,

      cluster-forming threshold

    cisze=10: int

      cluster size threshold

  Returns:
    positions array of shape(k,anat_dim):

      the cluster positions in physical coordinates where k= number of clusters if no such cluster exists, None is returned

nipy.labs.utils.reproducibility_measures.get_peak_position_from_thresholded_map(stat_map, domain, threshold)

  The peaks above thr in 18-connectivity are computed

  Parameters:
    stat_map: array of shape (nbvox): map to threshold
    deomain: referential- and domain-defining image
    thr, float: cluster-forming threshold
  Returns:
    positions array of shape(k,anat_dim):

      the cluster positions in physical coordinates where k= number of clusters if no such cluster exists, None is returned

nipy.labs.utils.reproducibility_measures.group_reproducibility_metrics(mask_images, contrast_images, variance_images, thresholds, ngroups, method, cluster_threshold=10, number_of_samples=10, sigma=6.0, do_clusters=True, do_voxels=True, do_peaks=True, swap=False)

  Main function to perform reproducibility analysis, including nifti1 io

  Parameters:
    threshold: list or 1-d array,

      the thresholds to be tested

  Returns:
    cluster_rep_results: dictionary,

      results of cluster-level reproducibility analysis

    voxel_rep_results: dictionary,

      results of voxel-level reproducibility analysis

    peak_rep_results: dictionary,

      results of peak-level reproducibility analysis

nipy.labs.utils.reproducibility_measures.histo_repro(h)

  Given the histogram h, compute a standardized reproducibility measure

  Parameters:
    h array of shape(xmax+1), the histogram values
  Returns:
    hr, float: the measure

nipy.labs.utils.reproducibility_measures.map_reproducibility(data, vardata, domain, ngroups, method='crfx', swap=False, verbose=0, **kwargs)

  Return a reproducibility map for the given method

  Parameters:
    data: array of shape (nvox,nsubj)

      the input data from which everything is computed

    vardata: array of the same size

      the corresponding variance information

    domain: referential- and domain-defining image
    ngroups (int): the size of each subrgoup to be studied
    threshold (float): binarization threshold

      (makes sense only if method==rfx)

    method=’crfx’, string to be chosen among ‘crfx’, ‘cmfx’, ‘cffx’

      inference method under study

    verbose=0 verbosity mode
  Returns:
    rmap: array of shape(nvox)

      the reproducibility map

nipy.labs.utils.reproducibility_measures.mfx_ttest(x, vx)

  Idem fttest, but returns a mixed-effects statistic

  Parameters:
    x: array of shape(nrows, ncols): effect matrix
    vx: array of shape(nrows, ncols): variance matrix
  Returns:
    t array of shape(nrows): mixed effect statistics array

nipy.labs.utils.reproducibility_measures.peak_reproducibility(data, vardata, domain, ngroups, sigma, method='crfx', swap=False, verbose=0, **kwargs)

  Return a measure of cluster-level reproducibility of activation patterns (i.e. how far clusters are from each other)

  Parameters:
    data: array of shape (nvox,nsubj)

      the input data from which everything is computed

    vardata: array of shape (nvox,nsubj)

      the variance of the data that is also available

    domain: referential- and domain-defining image
    ngroups (int),

      Number of subbgroups to be drawn

    sigma: float, parameter that encodes how far far is
    threshold: float, binarization threshold
    method: string to be chosen among ‘crfx’, ‘cmfx’ or ‘cffx’,

      inference method under study

    swap = False: if True, a random sign swap of the data is performed

      This is used to simulate a null hypothesis on the data.

    verbose=0 verbosity mode
  Returns:
    score (float): the desired cluster-level reproducibility index

nipy.labs.utils.reproducibility_measures.split_group(nsubj, ngroups)

  Split the proposed group into random disjoint subgroups

  Parameters:
    nsubj (int) the number of subjects to be split
    ngroups(int) Number of subbgroups to be drawn
  Returns:
    samples: a list of ngroups arrays containing

      the indexes of the subjects in each subgroup

nipy.labs.utils.reproducibility_measures.statistics_from_position(target, data, sigma=1.0)

  Return a number characterizing how close data is from target using a kernel-based statistic

  Parameters:
    target: array of shape(nt,anat_dim) or None

      the target positions

    data: array of shape(nd,anat_dim) or None

      the data position

    sigma=1.0 (float), kernel parameter

      or a distance that say how good good is

  Returns:
    sensitivity (float): how well the targets are fitted

      by the data in [0,1] interval 1 is good 0 is bad

nipy.labs.utils.reproducibility_measures.ttest(x)

  Returns the t-test for each row of the data x

nipy.labs.utils.reproducibility_measures.voxel_reproducibility(data, vardata, domain, ngroups, method='crfx', swap=False, verbose=0, **kwargs)

  return a measure of voxel-level reproducibility of activation patterns

  Parameters:
    data: array of shape (nvox,nsubj)

      the input data from which everything is computed

    vardata: array of shape (nvox,nsubj)

      the corresponding variance information ngroups (int): Number of subbgroups to be drawn

    domain: referential- and domain-defining image
    ngourps: int,

      number of groups to be used in the resampling procedure

    method: string, to be chosen among ‘crfx’, ‘cmfx’, ‘cffx’

      inference method under study

    verbose: bool, verbosity mode
  Returns:
    kappa (float): the desired reproducibility index

nipy.labs.utils.reproducibility_measures.voxel_thresholded_ttest(x, threshold)

  Returns a binary map of the ttest>threshold