# Welcome to the MPIRE documentation!

MPIRE, short for MultiProcessing Is Really Easy, is a Python package for multiprocessing. MPIRE is faster in most scenarios, packs more features, and is generally more user-friendly than the default multiprocessing package. It combines the convenient map like functions of `multiprocessing.Pool` with the benefits of using copy-on-write shared objects of `multiprocessing.Process`, together with easy-to-use worker state, worker insights, worker init and exit functions, timeouts, and progress bar functionality.

## Features

- Faster execution than other multiprocessing libraries. See benchmarks.

- Intuitive, Pythonic syntax

- Multiprocessing with `map`/`map_unordered`/`imap`/`imap_unordered`/`apply`/`apply_async` functions

- Easy use of copy-on-write shared objects with a pool of workers (copy-on-write is only available for start method `fork`, so it’s not supported on Windows)

- Each worker can have its own state and with convenient worker init and exit functionality this state can be easily manipulated (e.g., to load a memory-intensive model only once for each worker without the need of sending it through a queue)

- Progress bar support using tqdm (`rich` and notebook widgets are supported)

- Progress dashboard support

- Worker insights to provide insight into your multiprocessing efficiency

- Graceful and user-friendly exception handling

- Timeouts, including for worker init and exit functions

- Automatic task chunking for all available map functions to speed up processing of small task queues (including numpy arrays)

- Adjustable maximum number of active tasks to avoid memory problems

- Automatic restarting of workers after a specified number of tasks to reduce memory footprint

- Nested pool of workers are allowed when setting the `daemon` option

- Child processes can be pinned to specific or a range of CPUs

- Optionally utilizes dill as serialization backend through multiprocess, enabling parallelizing more exotic objects, lambdas, and functions in iPython and Jupyter notebooks.

MPIRE has been tested on Linux, macOS, and Windows. There are a few minor known caveats for Windows and macOS users, which can be found at Windows.

## Contents

- Installation
- Getting started
- Usage
  - WorkerPool
    - Starting a WorkerPool
    - Process start method
    - CPU pinning
    - Accessing the worker ID
    - Shared objects
    - Worker state
    - Keep alive
    - Worker insights
    - Dill
    - Order tasks
  - Map family
    - map family of functions
    - Progress bar
    - Worker init and exit
    - Task chunking
    - Maximum number of active tasks
    - Worker lifespan
    - Timeouts
    - Numpy arrays
  - Apply family
  - Dashboard
- Troubleshooting
- API Reference
- Contribution guidelines
- Changelog