# Notes on `sys.monitoring`

Note

This documentation was written at the advent of Python 3.12. Future versions of Python may behave differently. It is however hoped that most of the concepts herein will remain relevant.

Python 3.12 introduced a new monitoring system under `sys.monitoring`. This system lets users monitor a selection of events that may be interesting for e.g. performance profiling or debugging purposes. Event monitoring is set “per tool”, so that multiple tools can be running at the same time. For each tool the events can be monitored globally per thread or locally per code object (or a mixture of both). For each tool-event combination a callback can be registered that will be called on the occurrence of the event. The callbacks are just regular functions and can do most of the things supported by Python, they also have the ability to return a special value to tell the monitoring system to disable triggering future events for the current code location.

## What does this mean for Numba?

When the interpreter “encounters” a monitoring event (it actually issues them) it triggers any callbacks that are associated with that event across all tools that have registered monitoring for said event. In the case of Numba there are problems…

Numba has made it so that there’s no Python interpreter involved in the execution of a function, the function is compiled and its execution path exists only in machine code. To get to the machine code from the interpreter the Numba dispatcher is invoked, this is the last place in the stack where (in `nopython` mode) the Python interpreter is readily available. The dispatcher is also in some way part of the execution of the function, without the dispatcher the call to the machine code cannot easily happen from user space. As a result of this, the monitoring types and event types that Numba can support are somewhat limited as there’s such limited interpreter involvement in execution!

Looking at monitoring types in turn. Local monitoring is requested by setting monitoring on a code object. In practice this instructs the interpreter to augment the bytecode at runtime by switching certain opcodes for “instrumented” opcodes. These instrumented opcodes go via a special path in the interpreter loop whereby they will issue an “event” in association with a particular instruction at a particular offset. For example, a `RETURN` opcode might be replaced by an `INSTRUMENTED_RETURN` and a `PY_RETURN` event would by issued when the instrumented instruction is interpreted. This event and the offset at which it occurred being forwarded to the monitoring system. Unfortunately this presents an issue for Numba, there is no interpreter involved with execution and so events will not be emitted. It does seem like it would be possible to handle a few types of event, such as `PY_START` and `PY_RETURN` by analysing the code object at dispatch time. However, it’s possible for a user to de-instrument the code object and/or dynamically disable monitoring at a particular code location whilst executing, and as a result emulating the semantics of this would be prohibitively challenging and would likely require constant interaction with the interpreter. As a result, Numba does not support local event monitoring, the compiled function will still execute correctly if it has been set, it just has no effect on `sys.monitoring`.

Considering per-thread global monitoring, this manifests as the user setting some global state on the interpreter for a given thread. This state can be accessed via the `sys.monitoring` Python API, it’s also accessible via CPython internals. This kind of monitoring is a little more amenable to working with Numba as there’s no code object involved and state mutation during execution can only occur via object mode calls.

## What does Numba do in practice?

As there’s no Python or C API to issue events (the concept is heavily linked to the interpreter itself), Numba has to look for tool-event combinations at appropriate locations in the dispatch sequence and then manually call the associated callbacks (essentially doing what the interpreter does when it issues an event). In the case of the Numba dispatcher, only a few events are relevant and only four are supported, namely

- `sys.monitoring.events.PY_START` (Python function starting).

- `sys.monitoring.events.PY_RETURN` (Python function returning).

- `sys.monitoring.events.RAISE` (Python function raised an exception).

- `sys.monitoring.events.PY_UNWIND` (Python function exiting during exception unwinding).

These events don’t really exist in the machine code, but would exist had the interpreter interpreted the equivalent bytecode. The dispatcher therefore checks for monitoring of `PY_START` just before control is transferred to the machine code and calls any associated callbacks. The same is done for `PY_RETURN` just after control is transferred back to the dispatcher from the machine code. This behaviour essentially emulates the interpreter executing bytecode and lets tools such as `cProfile` be able to “see” the Numba compiled function as part of the standard interpreted execution. In the case of an exception being raised in the machine code, the associated error state is handled just after control is transferred back to the dispatcher, at this point `RAISE` and `PY_UNWIND` event monitoring is checked and registered callbacks are invoked.

A note on offsets. The callback functions often take an “offset” argument which is the bytecode offset at which the event triggering the callback was encountered. In the case of `PY_START` this seems to be associated with the offset of the `RESUME` bytecode. In the case of `PY_RETURN` this is associated with the offset of one of the `RETURN` bytecodes, most generally this would only be known at runtime as there could be multiple return paths. As a result, Numba elects to just set all offsets to zero. It may eventually be possible to do some analysis and transfer the appropriate runtime information to the dispatcher from the machine code, however, at the present time the effort to do this vastly outweighs the gain.