Skip to content
Back to top

# `gevent.monkey` – Make the standard library cooperative

Make the standard library cooperative.

The primary purpose of this module is to carefully patch, in place, portions of the standard library with gevent-friendly functions that behave in the same way as the original (at least as closely as possible).

The primary interface to this is the `patch_all()` function, which performs all the available patches. It accepts arguments to limit the patching to certain modules, but most programs should use the default values as they receive the most wide-spread testing, and some monkey patches have dependencies on others.

Patching should be done as early as possible in the lifecycle of the program. For example, the main module (the one that tests against `__main__` or is otherwise the first imported) should begin with this code, ideally before any other imports:

    from gevent import monkey
    monkey.patch_all()

A corollary of the above is that patching should be done on the main thread and should be done while the program is single-threaded.

Tip

Some frameworks, such as gunicorn, handle monkey-patching for you. Check their documentation to be sure.

Warning

Patching too late can lead to unreliable behaviour (for example, some modules may still use blocking sockets) or even errors.

Tip

Be sure to read the documentation for each patch function to check for known incompatibilities.

## Querying

Sometimes it is helpful to know if objects have been monkey-patched, and in advanced cases even to have access to the original standard library functions. This module provides functions for that purpose.

- `is_module_patched()`

- `is_object_patched()`

- `get_original()`

## Plugins and Events

Beginning in gevent 1.3, events are emitted during the monkey patching process. These events are delivered first to `gevent.events` subscribers, and then to setuptools entry points.

The following events are defined. They are listed in (roughly) the order that a call to `patch_all()` will emit them.

- `gevent.events.GeventWillPatchAllEvent`

- `gevent.events.GeventWillPatchModuleEvent`

- `gevent.events.GeventDidPatchModuleEvent`

- `gevent.events.GeventDidPatchBuiltinModulesEvent`

- `gevent.events.GeventDidPatchAllEvent`

Each event class documents the corresponding setuptools entry point name. The entry points will be called with a single argument, the same instance of the class that was sent to the subscribers.

You can subscribe to the events to monitor the monkey-patching process and to manipulate it, for example by raising `gevent.events.DoNotPatch`.

You can also subscribe to the events to provide additional patching beyond what gevent distributes, either for additional standard library modules, or for third-party packages. The suggested time to do this patching is in the subscriber for `gevent.events.GeventDidPatchBuiltinModulesEvent`. For example, to automatically patch psycopg2 using psycogreen when the call to `patch_all()` is made, you could write code like this:

    # mypackage.py
    def patch_psycopg(event):
        from psycogreen.gevent import patch_psycopg
        patch_psycopg()

In your `setup.py` you would register it like this:

    from setuptools import setup
    setup(
        ...
        entry_points={
            'gevent.plugins.monkey.did_patch_builtins': [
                'psycopg2 = mypackage:patch_psycopg',
            ],
        },
        ...
    )

For more complex patching, gevent provides a helper method that you can call to replace attributes of modules with attributes of your own modules. This function also takes care of emitting the appropriate events.

- `patch_module()`

## Use as a module

Sometimes it is useful to run existing python scripts or modules that were not built to be gevent aware under gevent. To do so, this module can be run as the main module, passing the script and its arguments. For details, see the `main()` function.

Changed in version 1.3b1: Added support for plugins and began emitting will/did patch events.

exception MonkeyPatchWarning

  Bases: `RuntimeWarning`

  The type of warnings we issue.

  Added in version 1.3a2.

get_original(mod_name, item_name)

  Retrieve the original object from a module.

  If the object has not been patched, then that object will still be retrieved.

  Parameters:

    - mod_name (str|sequence) – The name of the standard library module, e.g., `'socket'`. Can also be a sequence of standard library modules giving alternate names to try, e.g., `('thread', '_thread')`; the first importable module will supply all item_name items.

    - item_name (str|sequence) – A string or sequence of strings naming the attribute(s) on the module `mod_name` to return.

  Returns:

    The original value if a string was given for `item_name` or a sequence of original values if a sequence was passed.

is_module_patched(mod_name)

  Check if a module has been replaced with a cooperative version.

  Parameters:

    mod_name (str) – The name of the standard library module, e.g., `'socket'`.

is_object_patched(mod_name, item_name)

  Check if an object in a module has been replaced with a cooperative version.

  Parameters:

    - mod_name (str) – The name of the standard library module, e.g., `'socket'`.

    - item_name (str) – The name of the attribute in the module, e.g., `'create_connection'`.

patch_all(socket=True, dns=True, time=True, select=True, thread=True, os=True, ssl=True, subprocess=True, sys=False, aggressive=True, Event=True, builtins=True, signal=True, queue=True, contextvars=True, **kwargs)

  Do all of the default monkey patching (calls every other applicable function in this module).

  Returns:

    A true value if patching all modules wasn’t cancelled, a false value if it was.

  Changed in version 1.1: Issue a `warning` if this function is called multiple times with different arguments. The second and subsequent calls will only add more patches, they can never remove existing patches by setting an argument to `False`.

  Changed in version 1.1: Issue a `warning` if this function is called with `os=False` and `signal=True`. This will cause SIGCHLD handlers to not be called. This may be an error in the future.

  Changed in version 1.3a2: `Event` defaults to True.

  Changed in version 1.3b1: Defined the return values.

  Changed in version 1.3b1: Add `**kwargs` for the benefit of event subscribers. CAUTION: gevent may add and interpret additional arguments in the future, so it is suggested to use prefixes for kwarg values to be interpreted by plugins, for example, `patch_all(mylib_futures=True)`.

  Changed in version 1.3.5: Add queue, defaulting to True, for Python 3.7.

  Changed in version 1.5: Remove the `httplib` argument. Previously, setting it raised a `ValueError`.

  Changed in version 1.5a3: Add the `contextvars` argument.

  Changed in version 1.5: Better handling of patching more than once.

patch_builtins()

  Make the builtin `__import__()` function greenlet safe under Python 2.

  Note

  This does nothing under Python 3 as it is not necessary. Python 3 features improved import locks that are per-module, not global.

  Deprecated since version 23.7.0: Does nothing on any supported platform.

patch_dns()

  Replace DNS functions in `socket` with cooperative versions.

  This is only useful if `patch_socket()` has been called and is done automatically by that method if requested.

patch_module(target_module, source_module, items=None)

  Replace attributes in target_module with the attributes of the same name in source_module.

  The source_module can provide some attributes to customize the process:

  - `__implements__` is a list of attribute names to copy; if not present, the items keyword argument is mandatory. `__implements__` must only have names from the standard library module in it.

  - `_gevent_will_monkey_patch(target_module, items, warn, **kwargs)`

  - `_gevent_did_monkey_patch(target_module, items, warn, **kwargs)` These two functions in the source_module are called if they exist, before and after copying attributes, respectively. The “will” function may modify items. The value of warn is a function that should be called with a single string argument to issue a warning to the user. If the “will” function raises `gevent.events.DoNotPatch`, no patching will be done. These functions are called before any event subscribers or plugins.

  Parameters:

    items (list) – A list of attribute names to replace. If not given, this will be taken from the source_module `__implements__` attribute.

  Returns:

    A true value if patching was done, a false value if patching was canceled.

  Added in version 1.3b1.

patch_os()

  Replace `os.fork()` with `gevent.fork()`, and, on POSIX, `os.waitpid()` with `gevent.os.waitpid()` (if the environment variable `GEVENT_NOWAITPID` is not defined). Does nothing if fork is not available.

  Caution

  This method must be used with `patch_signal()` to have proper `SIGCHLD` handling and thus correct results from `waitpid`. `patch_all()` calls both by default.

  Caution

  For `SIGCHLD` handling to work correctly, the event loop must run. The easiest way to help ensure this is to use `patch_all()`.

patch_queue()

  Patch objects in `queue`.

  This replaces `SimpleQueue`, `PriorityQueue`, `Queue` and `LifoQueue` with their gevent equivalents.

  Added in version 1.3.5.

  Changed in version 25.4.1: In addition to `SimpleQueue`, now also patches `Queue`, `PriorityQueue` and `LifoQueue`.`

  Note that only documented attributes are the same between gevent and the standard library. Internal implementation details are very different.

patch_select(aggressive=True)

  Replace `select.select()` with `gevent.select.select()` and `select.poll()` with `gevent.select.poll` (where available).

  If `aggressive` is true (the default), also remove other blocking functions from `select` .

  - `select.epoll()`

  - `select.kqueue()`

  - `select.kevent()`

  - `select.devpoll()` (Python 3.5+)

patch_signal()

  Make the `signal.signal()` function work with a `monkey-patched os`.

  Caution

  This method must be used with `patch_os()` to have proper `SIGCHLD` handling. `patch_all()` calls both by default.

  Caution

  For proper `SIGCHLD` handling, you must yield to the event loop. Using `patch_all()` is the easiest way to ensure this.

  See also

  `gevent.signal`

patch_socket(dns=True, aggressive=True)

  Replace the standard socket object with gevent’s cooperative sockets.

  Parameters:

    dns (bool) – When true (the default), also patch address resolution functions in `socket`. See Name Resolution (DNS) for details.

patch_ssl() → None

  Replace `ssl.SSLSocket` object and socket wrapping functions in `ssl` with cooperative versions.

  This is only useful if `patch_socket()` has been called.

  It is important to call this function before `ssl` has been imported. For more information, see `gevent.ssl`.

patch_subprocess()

  Replace `subprocess.call()`, `subprocess.check_call()`, `subprocess.check_output()` and `subprocess.Popen` with `cooperative versions`.

  Note

  On Windows under Python 3, the API support may not completely match the standard library.

  Note

  On macOS, this changes the `multiprocessing` start method to ‘fork’. It defaults to ‘spawn’.

  Note

  On Python 3.14+ and platforms other than macOS and Windows, this changes the `multiprocessing` start method to ‘fork’. It defaults to ‘forkserver’.

patch_sys(stdin=True, stdout=True, stderr=True)

  Patch sys.std[in,out,err] to use a cooperative IO via a threadpool.

  This is relatively dangerous and can have unintended consequences such as hanging the process or misinterpreting control keys when `input()` and `raw_input()` are used. `patch_all()` does not call this function by default.

  This method does nothing on Python 3. The Python 3 interpreter wants to flush the TextIOWrapper objects that make up stderr/stdout at shutdown time, but using a threadpool at that time leads to a hang.

  Deprecated since version 23.7.0: Does nothing on any supported version.

patch_thread(threading=True, _threading_local=True, Event=True, logging=True, existing_locks=True) → None

  Replace the standard `thread` module to make it greenlet-based.

  Parameters:

    - threading (bool) – When True (the default), also patch `threading`.

    - _threading_local (bool) – When True (the default), also patch `_threading_local.local`.

    - logging (bool) – When True (the default), also patch locks taken if the logging module has been configured.

    - existing_locks (bool) – When True (the default), and the process is still single threaded, make sure that any `threading.RLock` (and, under Python 3, `importlib._bootstrap._ModuleLock`) instances that are currently locked can be properly unlocked. Important: This is a best-effort attempt and, on certain implementations, may not detect all locks. It is important to monkey-patch extremely early in the startup process. Setting this to False is not recommended, especially on Python 2.

  Caution

  Monkey-patching `thread` and using `multiprocessing.Queue` or `concurrent.futures.ProcessPoolExecutor` (which uses a `Queue`) will hang the process.

  Monkey-patching with this function and using sub-interpreters (and advanced C-level API) and threads may be unstable on certain platforms.

  Changed in version 1.1b1: Add logging and existing_locks params.

  Changed in version 1.3a2: `Event` defaults to True.

patch_time()

  Replace `time.sleep()` with `gevent.sleep()`.