# mpi4py.util.pool

Added in version 4.0.0.

See also

This module intends to be a drop-in replacement for the `multiprocessing.pool` interface from the Python standard library. The `Pool` class exposed here is implemented as a thin wrapper around `MPIPoolExecutor`.

Note

The `mpi4py.futures` package offers a higher level interface for asynchronously pushing tasks to MPI worker process, allowing for a clear separation between submitting tasks and waiting for the results.

class mpi4py.util.pool.Pool

  Pool using MPI processes as workers.

  __init__(processes=None, initializer=None, initargs=(), **kwargs)

    Initialize a new Pool instance.

    Parameters:

      - processes (int | None) – Number of worker processes.

      - initializer (Callable[[...], object] | None) – An callable used to initialize workers processes.

      - initargs (Iterable[Any]) – A tuple of arguments to pass to the initializer.

      - kwargs (Any)

    Return type:

      None

    Note

    Additional keyword arguments are passed down to the `MPIPoolExecutor` constructor.

    Warning

    The maxtasksperchild and context arguments of `multiprocessing.pool.Pool` are not supported. Specifying maxtasksperchild or context with a value other than `None` will issue a warning of category `UserWarning`.

  apply(func, args=(), kwds={})

    Call func with arguments args and keyword arguments kwds.

    Equivalent to `func(*args, **kwds)`.

    Parameters:

      - func (Callable[[...], T])

      - args (Iterable[Any])

      - kwds (Mapping[str, Any])

    Return type:

      T

  apply_async(func, args=(), kwds={}, callback=None, error_callback=None)

    Asynchronous version of `apply()` returning `ApplyResult`.

    Parameters:

      - func (Callable[..., T])

      - args (Iterable[Any])

      - kwds (Mapping[str, Any])

      - callback (Callable[[T], object] | None)

      - error_callback (Callable[[BaseException], object] | None)

    Return type:

      AsyncResult[T]

  map(func, iterable, chunksize=None)

    Apply func to each element in iterable.

    Equivalent to `list(map(func, iterable))`.

    Block until all results are ready and return them in a `list`.

    The iterable is choped into a number of chunks which are submitted as separate tasks. The (approximate) size of these chunks can be specified by setting chunksize to a positive integer.

    Consider using `imap()` or `imap_unordered()` with explicit chunksize for better efficiency.

    Parameters:

      - func (Callable[[S], T])

      - iterable (Iterable[S])

      - chunksize (int | None)

    Return type:

      list[T]

  map_async(func, iterable, chunksize=None, callback=None, error_callback=None)

    Asynchronous version of `map()` returning `MapResult`.

    Parameters:

      - func (Callable[[S], T])

      - iterable (Iterable[S])

      - chunksize (int | None)

      - callback (Callable[[T], None] | None)

      - error_callback (Callable[[BaseException], None] | None)

    Return type:

      MapResult[T]

  imap(func, iterable, chunksize=1)

    Like `map()` but return an `iterator`.

    Equivalent to `map(func, iterable)`.

    Parameters:

      - func (Callable[[S], T])

      - iterable (Iterable[S])

      - chunksize (int)

    Return type:

      Iterator[T]

  imap_unordered(func, iterable, chunksize=1)

    Like `imap()` but ordering of results is arbitrary.

    Parameters:

      - func (Callable[[S], T])

      - iterable (Iterable[S])

      - chunksize (int)

    Return type:

      Iterator[T]

  starmap(func, iterable, chunksize=None)

    Apply func to each argument tuple in iterable.

    Equivalent to `list(itertools.starmap(func, iterable))`.

    Block until all results are ready and return them in a `list`.

    The iterable is choped into a number of chunks which are submitted as separate tasks. The (approximate) size of these chunks can be specified by setting chunksize to a positive integer.

    Consider using `istarmap()` or `istarmap_unordered()` with explicit chunksize for better efficiency.

    Parameters:

      - func (Callable[[...], T])

      - iterable (Iterable[Iterable[Any]])

      - chunksize (int | None)

    Return type:

      list[T]

  starmap_async(func, iterable, chunksize=None, callback=None, error_callback=None)

    Asynchronous version of `starmap()` returning `MapResult`.

    Parameters:

      - func (Callable[..., T])

      - iterable (Iterable[Iterable[Any]])

      - chunksize (int | None)

      - callback (Callable[[T], None] | None)

      - error_callback (Callable[[BaseException], None] | None)

    Return type:

      MapResult[T]

  istarmap(func, iterable, chunksize=1)

    Like `starmap()` but return an `iterator`.

    Equivalent to `itertools.starmap(func, iterable)`.

    Parameters:

      - func (Callable[[...], T])

      - iterable (Iterable[Iterable[Any]])

      - chunksize (int)

    Return type:

      Iterator[T]

  istarmap_unordered(func, iterable, chunksize=1)

    Like `istarmap()` but ordering of results is arbitrary.

    Parameters:

      - func (Callable[[...], T])

      - iterable (Iterable[Iterable[Any]])

      - chunksize (int)

    Return type:

      Iterator[T]

  close()

    Prevent any more tasks from being submitted to the pool.

    Return type:

      None

  terminate()

    Stop the worker processes without completing pending tasks.

    Return type:

      None

  join()

    Wait for the worker processes to exit.

    Return type:

      None

class mpi4py.util.pool.ThreadPool

  Bases: `Pool`

  Pool using threads as workers.

class mpi4py.util.pool.AsyncResult

  Asynchronous result.

  get(timeout=None)

    Return the result when it arrives.

    If timeout is not `None` and the result does not arrive within timeout seconds then raise `TimeoutError`.

    If the remote call raised an exception then that exception will be reraised.

    Parameters:

      timeout (float | None)

    Return type:

      T

  wait(timeout=None)

    Wait until the result is available or timeout seconds pass.

    Parameters:

      timeout (float | None)

    Return type:

      None

  ready()

    Return whether the call has completed.

    Return type:

      bool

  successful()

    Return whether the call completed without raising an exception.

    If the result is not ready then raise `ValueError`.

    Return type:

      bool

class mpi4py.util.pool.ApplyResult

  Bases: `AsyncResult`

  Result type of `apply_async()`.

class mpi4py.util.pool.MapResult

  Bases: `AsyncResult`

  Result type of `map_async()` and `starmap_async()`.