Loading...
Searching...
No Matches
dolfinx::MPI Namespace Reference

MPI support functionality. More...

## Classes

class | Comm
| A duplicate MPI communicator and manage lifetime of the communicator. More...
struct | dependent_false
struct | mpi_type_mapping
| MPI Type. More...

## Enumerations

enum class | tag : std::uint16_t { consensus_pcx = 1200 , consensus_pex = 1201 , consensus_nbx = 1202 }
| MPI communication tags.

## Functions

int | rank (MPI_Comm comm)
| Return process rank for the communicator.
int | size (MPI_Comm comm)
void | check_error (MPI_Comm comm, int code)
| Check MPI error code. If the error code is not equal to MPI_SUCCESS, then std::abort is called.
constexpr std::array< std::int64_t, 2 > | local_range (int rank, std::int64_t N, int size)
| Return local range for the calling process, partitioning the global [0, N - 1] range across all ranks into partitions of almost equal size.
constexpr int | index_owner (int size, std::size_t index, std::size_t N)
| Return which rank owns index in global range [0, N - 1] (inverse of MPI::local_range).
std::vector< int > | compute_graph_edges_pcx (MPI_Comm comm, std::span< const int > edges)
| Determine incoming graph edges using the PCX consensus algorithm.
std::vector< int > | compute_graph_edges_nbx (MPI_Comm comm, std::span< const int > edges, int tag=static_cast< int >(tag::consensus_nbx))
| Determine incoming graph edges using the NBX consensus algorithm.
template<typename U>
std::pair< std::vector< std::int32_t >, std::vector< typename std::remove_reference_t< typename U::value_type > > > | distribute_to_postoffice (MPI_Comm comm, const U &x, std::array< std::int64_t, 2 > shape, std::int64_t rank_offset)
| Distribute row data to 'post office' ranks.
template<typename U>
std::vector< typename std::remove_reference_t< typename U::value_type > > | distribute_from_postoffice (MPI_Comm comm, std::span< const std::int64_t > indices, const U &x, std::array< std::int64_t, 2 > shape, std::int64_t rank_offset)
| Distribute rows of a rectangular data array from post office ranks to ranks where they are required.
template<typename U>
std::vector< typename std::remove_reference_t< typename U::value_type > > | distribute_data (MPI_Comm comm0, std::span< const std::int64_t > indices, MPI_Comm comm1, const U &x, int shape1)
| Distribute rows of a rectangular data array to ranks where they are required (scalable version).

## Variables

template<typename T>
MPI_Datatype | mpi_t = mpi_type_mapping<T>::type
| Retrieves the MPI data type associated to the provided type.

## Detailed Description

MPI support functionality.

## Function Documentation

## check_error()

void check_error | ( | MPI_Comm | comm,
|  | int | code )

Check MPI error code. If the error code is not equal to MPI_SUCCESS, then std::abort is called.

Parameters

  [in] | comm | MPI communicator.
  [in] | code | Error code returned by an MPI function call.

## compute_graph_edges_nbx()

std::vector< int > compute_graph_edges_nbx | ( | MPI_Comm | comm,
|  | std::span< const int > | edges,
|  | int | tag = static_cast<int>(tag::consensus_nbx) )

Determine incoming graph edges using the NBX consensus algorithm.

Given a list of outgoing edges (destination ranks) from this rank, this function returns the incoming edges (source ranks) to this rank.

Note
  This function is for sparse communication patterns, i.e. where the number of ranks that communicate with each other is relatively small. It is scalable, i.e. no arrays the size of the communicator are constructed and the communication pattern is sparse. It implements the NBX algorithm presented in https://dx.doi.org/10.1145/1837853.1693476.
  The order of the returned ranks is not deterministic.
  Collective.

Parameters

  [in] | comm | MPI communicator
  [in] | edges | Edges (ranks) from this rank (the caller).
  [in] | tag | Tag used in non-blocking MPI calls. A tag can be required when this function is called a second time on some ranks before a previous call has completed on all other ranks.

Returns
  Ranks that have defined edges from them to this rank.

Note
  An alternative to passing a tag is to ensure that there is an implicit or explicit barrier before and after the call to this function.

## compute_graph_edges_pcx()

std::vector< int > compute_graph_edges_pcx | ( | MPI_Comm | comm,
|  | std::span< const int > | edges )

Determine incoming graph edges using the PCX consensus algorithm.

Given a list of outgoing edges (destination ranks) from this rank, this function returns the incoming edges (source ranks) to this rank.

Note
  This function is for sparse communication patterns, i.e. where the number of ranks that communicate with each other is relatively small. It is not scalable as arrays the size of the communicator are allocated. It implements the PCX algorithm described in https://dx.doi.org/10.1145/1837853.1693476.
  For sparse graphs, this function has \(O(p)\) cost, where \(p\)is the number of MPI ranks. It is suitable for modest MPI rank counts.
  The order of the returned ranks is not deterministic.
  Collective

Parameters

  [in] | comm | MPI communicator
  [in] | edges | Edges (ranks) from this rank (the caller).

Returns
  Ranks that have defined edges from them to this rank.

## distribute_data()

template<typename U>

std::vector< typename std::remove_reference_t< typename U::value_type > > distribute_data | ( | MPI_Comm | comm0,
|  | std::span< const std::int64_t > | indices,
|  | MPI_Comm | comm1,
|  | const U & | x,
|  | int | shape1 )

Distribute rows of a rectangular data array to ranks where they are required (scalable version).

This function determines local neighborhoods for communication, and then uses MPI neighbourhood collectives to exchange data. It is scalable if the neighborhoods are relatively small, i.e. each process communicated with a modest number of other processes.

Parameters

  [in] | comm0 | Communicator to distribute data across.
  [in] | indices | Global indices of the data (row indices) required by the calling process.
  [in] | comm1 | Communicator across which x is distributed. Can be MPI_COMM_NULL on ranks where x is empty.
  [in] | x | Data (2D array, row-major) on calling process to be distributed (by row). The global index for the [0, ..., n) local rows is assumed to be the local index plus the offset for this rank on comm1.
  [in] | shape1 | The number of columns of the data array x.

Returns
  The data for each index in indices (row-major storage).

Precondition
  shape1 > 0

## distribute_from_postoffice()

template<typename U>

std::vector< typename std::remove_reference_t< typename U::value_type > > distribute_from_postoffice | ( | MPI_Comm | comm,
|  | std::span< const std::int64_t > | indices,
|  | const U & | x,
|  | std::array< std::int64_t, 2 > | shape,
|  | std::int64_t | rank_offset )

Distribute rows of a rectangular data array from post office ranks to ranks where they are required.

This function determines local neighborhoods for communication, and then using MPI neighbourhood collectives to exchange data. It is scalable if the neighborhoods are relatively small, i.e. each process communicated with a modest number of other processes.

Parameters

  [in] | comm | The MPI communicator.
  [in] | indices | Global indices of the data (row indices) required by calling process.
  [in] | x | Data (2D array, row-major) on calling process which may be distributed (by row). The global index for the [0, ..., n) local rows is assumed to be the local index plus the offset for this rank.
  [in] | shape | The global shape of x.
  [in] | rank_offset | The rank offset such that global index of local row i in x is rank_offset + i. It is usually computed using MPI_Exscan on comm1 from MPI::distribute_data.

Returns
  The data for each index in indices (row-major storage).

Precondition
  shape1 > 0.

## distribute_to_postoffice()

template<typename U>

std::pair< std::vector< std::int32_t >, std::vector< typename std::remove_reference_t< typename U::value_type > > > distribute_to_postoffice | ( | MPI_Comm | comm,
|  | const U & | x,
|  | std::array< std::int64_t, 2 > | shape,
|  | std::int64_t | rank_offset )

Distribute row data to 'post office' ranks.

This function takes row-wise data that is distributed across processes. Data is not duplicated across ranks. The global index of a row is its local row position plus the offset for the calling process. The post office rank for a row is determined by applying dolfinx::MPI::index_owner to the global index, and the row is then sent to the post office rank. The function returns that row data for which the caller is the post office.

Parameters

  [in] | comm | MPI communicator.
  [in] | x | Data to distribute (2D, row-major layout).
  [in] | shape | The global shape of x.
  [in] | rank_offset | The rank offset such that global index of local row i in x is rank_offset + i. It is usually computed using MPI_Exscan.

Returns
  (0) local indices of my post office data and (1) the data (row-major). It does not include rows that are in x, i.e. rows for which the calling process is the post office.

## index_owner()

int index_owner | ( | int | size,
|  | std::size_t | index,
|  | std::size_t | N )

constexpr

Return which rank owns index in global range [0, N - 1] (inverse of MPI::local_range).

Parameters

  [in] | size | Number of MPI ranks.
  [in] | index | The index to determine the owning rank of.
  [in] | N | Total number of indices.

Returns
  Rank of the owning process.

## local_range()

std::array< std::int64_t, 2 > local_range | ( | int | rank,
|  | std::int64_t | N,
|  | int | size )

constexpr

Return local range for the calling process, partitioning the global [0, N - 1] range across all ranks into partitions of almost equal size.

Parameters

  [in] | rank | MPI rank of the caller
  [in] | N | The value to partition.
  [in] | size | Number of MPI ranks across which to partition N.

## size()

int size | ( | MPI_Comm | comm | ) |

Return size of the group (number of processes) associated with the communicator.

## Variable Documentation

## mpi_t

template<typename T>

MPI_Datatype mpi_t = mpi_type_mapping<T>::type

Retrieves the MPI data type associated to the provided type.

Template Parameters

  T | cpp type to map