Loading...
Searching...
No Matches
mpi_collective_benchmark.cc File Reference
`#include <chrono>`
`#include <functional>`
`#include <iostream>`
`#include <iomanip>`
`#include <string>`
`#include <thread>`
`#include <tuple>`
`#include <vector>`
`#include <dune/common/exceptions.hh>`
`#include <dune/common/timer.hh>`
`#include <dune/common/parametertree.hh>`
`#include <dune/common/parametertreeparser.hh>`
`#include <dune/common/parallel/future.hh>`
`#include <dune/common/parallel/mpihelper.hh>`

## Functions

template<class CC>
void | communicate (CC &cc)
template<class CC>
Dune::Future< void > | startCommunication (CC &cc)
template<class CC>
double | runBlocking (CC &cc)
template<class CC>
double | runNonblockingWait (CC &cc)
std::tuple< double, double > | runNonblockingSleep (decltype(Dune::MPIHelper::getCommunication())&cc, std::chrono::duration< double > wait_time)
std::tuple< double, double > | runNonblockingActive (decltype(Dune::MPIHelper::getCommunication())&cc, std::chrono::duration< double > wait_time)
std::tuple< double, double > | determineOverlap (std::function< std::tuple< double, double >(std::chrono::duration< double >)> fun)
void | printHeader ()
void | run (int s)
int | main (int argc, char **argv)

## Variables

Dune::ParameterTree | options
| Benchmark for measure the possible overlap of computation and communication at MPI collective communications.
std::vector< std::string > | all_methods

## Function Documentation

## communicate()

template<class CC>

void communicate | ( | CC & | cc | ) |

## determineOverlap()

std::tuple< double, double > determineOverlap | ( | std::function< std::tuple< double, double >(std::chrono::duration< double >)> | fun | ) |

## main()

int main | ( | int | argc,
|  | char ** | argv )

## printHeader()

void printHeader | ( |  | ) |

## run()

void run | ( | int | s | ) |

## runBlocking()

template<class CC>

double runBlocking | ( | CC & | cc | ) |

## runNonblockingActive()

std::tuple< double, double > runNonblockingActive | ( | decltype(Dune::MPIHelper::getCommunication())& | cc,
|  | std::chrono::duration< double > | wait_time )

## runNonblockingSleep()

std::tuple< double, double > runNonblockingSleep | ( | decltype(Dune::MPIHelper::getCommunication())& | cc,
|  | std::chrono::duration< double > | wait_time )

## runNonblockingWait()

template<class CC>

double runNonblockingWait | ( | CC & | cc | ) |

## startCommunication()

template<class CC>

Dune::Future< void > startCommunication | ( | CC & | cc | ) |

## Variable Documentation

## all_methods

std::vector<std::string> all_methods

Initial value:
= {"allreduce",
"barrier",
"broadcast",
"gather",
"allgather",
"scatter"}

## options

Dune::ParameterTree options

Benchmark for measure the possible overlap of computation and communication at MPI collective communications.

This benchmark is inspired by the sandia micro benchmark: W. Lawry, C. Wilson, A. Maccabe, R. Brightwell. COMB: A Portable Benchmark Suite for Assessing MPI Overlap. In Proceedings of the IEEE International Conference on Cluster Computing (CLUSTER 2002), p. 472, 2002. http://www.cs.sandia.gov/smb/overhead.html

The following communication times are measured: Blocking: Blocking call. E.g. MPI_Allreduce Nonblocking_wait (NB_Wait): Nonblocking (e.g. MPI_Iallreduce) call directly followed by MPI_Wait. Nonblocking_sleep (NB_Sleep): Nonblocking call followed by a busy wait until the work time has passed. Then MPI_Wait. Nonblocking_active (NB_active): Nonblocking call followed by a basy wait where in every iteration MPI_Test is called until the work time has passed. The MPI_wait.

The overhead is computed as the time for the Nonblocking call plus the time for MPI_Wait. The iteration time is the time for the whole communication. The available part of the communication time(avail(%)) is computed as 1-(overhead/base_t), where base_t is the time for calling the method with wait time = 0. The overhead is determined by increasing the work time successive until it is the dominant factor in the iteration time. Then the overhead is computed as iter_t-work_t.

Usage: mpirun ./mpi_collective_benchmark [options]

options: -method: default: allreduce. possible methods: allreduce, barrier, broadcast, gather, allgather, scatter -iterations: default: 10000. Number of iterations for measure the time for one communication -allMethods: default:0. If 1 iterates over all available methods -startSize: default: n, where n is the size of MPI_COMM_WORLD. runs the benchmark for different communicator sizes, starting with startSize. After every run the size is doubled. Finally one run is made for the whole communicator. -verbose: default: 0. If 1 prints intermediate information while determining the overhead. -threshold: default: 2. The threshold when the work time is the dominant factor in the iteration time. (Similar to the threshold in the sandia benchmark) -nohdr: default: 0. Suppress output of the header.

options can be set either in the options.ini file or can be pass at the command-line (-key value).

To get a good 'available' value for the NB_sleep communication, some MPI implementation need to spawn an extra thread. With MPICH you can activate this by setting the environment variable MPI_ASYNC_PROGRESS to 1, with IntelMPI the variable is called I_MPI_ASYNC_PROGRESS. (https://software.intel.com/en-us/mpi-developer-reference-linux-asynchronous-progress-control)