Hide navigation sidebar
Hide table of contents sidebar
Skip to content
Back to top
View this page
Toggle table of contents sidebar

# Python allocators

Python has a layer of abstraction between code that allocates memory and the system allocator (malloc, free, realloc, etc). Although the system allocator is quite fast and efficient, it is still generic and is not tuned to the specific allocation patterns of the Python interpreter, especially those regarding small Python objects. To improve performance and reduce fragmentation, Python has a specialized allocator that handles allocation of small objects and defers to the system allocator for large ones. This allocator is called `pymalloc`.

## How `pymalloc` works

Requests greater than 512 bytes are routed to the system’s allocator. This means that even if `pymalloc` is active, it will only affect requests for 512 bytes or less. For those small requests, the `pymalloc` allocator will allocate big chunks of memory from the system allocator and then subdivide those big chunks.

`pymalloc` works with 3 hierarchical data structures:

- Arenas: These are chunks of memory that `pymalloc` directly requests from the system allocator using `mmap`. Arenas are always a multiple of 4 kilobytes. Arenas are subdivided into pools of different types.

- Pools: Pools contain fixed size blocks of memory. Each pool only contains blocks of a single consistent size, though different pools have blocks of different sizes. Pools are used to easily find, allocate, and free memory blocks of a given size.

- Blocks: These are the fundamental units of storage. Small allocation requests to `pymalloc` are always satisfied by returning a pointer to a specific block. This block can be bigger than what the allocation requested.

When the user code is done with a block, it is marked as unused. When all blocks in a pool have been marked as unused, the pool is marked as unused. Only when all pools in an arena have been marked as unused can the arena be returned to the system allocator. This rarely happens.

The following table shows the relationship between the allocation request and the actual block size that will be returned:

Allocation request to block size
Request in bytes | Size of allocated block | Size class index
--- | --- | ---
1-8 | 8 | 0
9-16 | 16 | 1
17-24 | 24 | 2
25-32 | 32 | 3
33-40 | 40 | 4
41-48 | 48 | 5
49-56 | 56 | 6
57-64 | 64 | 7
65-72 | 72 | 8
… | … | …
497-504 | 504 | 62
505-512 | 512 | 63

As indicated before, any request for more than 512 bytes will be made directly using the system allocator.

## How does this affect memory profiling?

Memory profiling with `pymalloc` active can yield some surprising results. This is because when a request for memory comes, `pymalloc` will try to reuse existing memory that has previously been used for other Python objects. This has two main consequences:

- Requests for small amounts of memory that can be satisfied from an existing arena won’t result in a call to the system allocator and therefore won’t appear in the profiler reports at all.

- Requests for small amounts of memory that can’t be satisfied from an existing arena will result in a new arena being allocated from the system allocator, which will be shown as an `mmap` request whose size is a multiple of 4 kilobytes.

This may look confusing, and you may wish to deactivate `pymalloc` when profiling memory, but that is not necessarily a good idea. When your application runs, it normally runs with `pymalloc` activated, which means that any allocation patterns that you see when profiling with `pymalloc` activated correspond exactly to what your application will be doing.

Note

If you are interested in why your application is using a lot of memory, you usually want to profile it exactly how it is normally executed, so you usually want `pymalloc` to be active.

There are some cases when `pymalloc` gets in the way, though. For instance, if you are hunting memory leaks using the `--leaks` option of some of the `memray` reporters, then having `pymalloc` active during profiling makes this task more difficult.

Note

When hunting for memory leaks using the `--leaks` option, you want to see every allocation and deallocation independently, so you usually want `pymalloc` deactivated.

Because `pymalloc` may not release all memory to the system even if you have finished using it, the profiler will incorrectly report it as leaked. You’ll also be misled about the size of the leak, since the size of the leaked allocation will be reported as the size of the arena (some multiple of 4 kilobytes), rather than the size of the leaked object (never more than 512 bytes). Also, if just one block owned by an arena is leaked, that will prevent the arena from being returned to the system allocator, but you won’t be able to tell which object was leaked, since a single arena can be used to satisfy many allocation requests from many places, and only the place where the arena was created by requesting more memory from the system will have been recorded.

Caution

Note that running without `pymalloc` will result in slower executions and much bigger report files, as many extra allocations will be seen.

## How can I deactivate `pymalloc`

To deactivate `pymalloc` you can set the `PYTHONMALLOC=malloc` environment variable or execute Python with `-Xdev`. Or, you can pass the `--trace-python-allocators` flag to `memray run` so that, even though `pymalloc` is still used, Memray sees every call to the `pymalloc` allocator instead of only the ones where it needs to make a request to the system allocator.