# 4. Batch compare models on different compute engines

## `sasmodels.compare_many`

Program to compare results from many random parameter sets for a given model.

The result is a comma separated value (CSV) table that can be redirected from standard output into a file and loaded into a spreadsheet.

The models are compared for each parameter set and if the difference is greater than expected for that precision, the parameter set is labeled as bad and written to the output, along with the random seed used to generate that parameter value. This seed can be used with `compare` to reload and display the details of the model.

sasmodels.compare_many.calc_stats(target: np.ndarray, value: np.ndarray, index: Any) → Tuple[float, float, float, float]

  Calculate statistics between the target value and the computed value.

  target and value are the vectors being compared, with the difference normalized by target to get relative error. Only the elements listed in index are used, though index may be and empty slice defined by slice(None, None).

  Returns:

  > maxrel the maximum relative difference

  > rel95 the relative difference with the 5% biggest differences ignored

  > maxabs the maximum absolute difference for the 5% biggest differences

  > maxval the maximum value for the 5% biggest differences

sasmodels.compare_many.compare_instance(name: str, data: Data, index: Any, N: int = 1, mono: bool = True, cutoff: float = 1e-05, base: str = 'single', comp: str = 'double') → None

  Compare the model under different calculation engines.

  name is the name of the model.

  data is the data object giving \(q, \Delta q\) calculation points.

  index is the active set of points.

  N is the number of comparisons to make.

  cutoff is the polydispersity weight cutoff to make the calculation a little bit faster.

  base and comp are the names of the calculation engines to compare.

sasmodels.compare_many.main(argv: List[str]) → None

  Main program.

sasmodels.compare_many.print_column_headers(pars: Dict[str, float], parts: List[str]) → None

  Generate column headers for the differences and for the parameters, and print them to standard output.

sasmodels.compare_many.print_help() → None

  Print usage string, the option description and the list of available models.

sasmodels.compare_many.print_usage() → None

  Print the command usage string.