# A Map of the Numba Repository

The Numba repository is quite large, and due to age has functionality spread around many locations. To help orient developers, this document will try to summarize where different categories of functionality can be found.

## Support Files

### Build and Packaging

- setup.py - Standard Python distutils/setuptools script

- MANIFEST.in - Distutils packaging instructions

- requirements.txt - Pip package requirements, not used by conda

- versioneer.py - Handles automatic setting of version in installed package from git tags

- .flake8 - Preferences for code formatting. Files should be fixed and removed from the exception list as time allows.

- .readthedocs.yml - Configuration file for Read the Docs.

- buildscripts/condarecipe.local - Conda build recipe

### Continuous Integration

- azure-pipelines.yml - Azure Pipelines CI config (active: Win/Mac/Linux)

- buildscripts/azure/ - Azure Pipeline configuration for specific platforms

- buildscripts/incremental/ - Generic scripts for building Numba on various CI systems

- codecov.yml - Codecov.io coverage reporting

### Documentation

- LICENSE - License for Numba

- LICENSES.third-party - License for third party code vendored into Numba

- README.rst - README for repo, also uploaded to PyPI

- CONTRIBUTING.md - Documentation on how to contribute to project (out of date, should be updated to point to Sphinx docs)

- CHANGE_LOG - History of Numba releases, also directly embedded into Sphinx documentation

- docs/ - Documentation source

- docs/_templates/ - Directory for templates (to override defaults with Sphinx theme)

- docs/Makefile - Used to build Sphinx docs with `make`

- docs/source - ReST source for Numba documentation

- docs/_static/ - Static CSS and image assets for Numba docs

- docs/make.bat - Not used (remove?)

- numba/scripts/generate_lower_listing.py - Dump all registered implementations decorated with `@lower*` for reference documentation. Currently misses implementations from the higher level extension API.

## Numba Source Code

Numba ships with both the source code and tests in one package.

- numba/ - all of the source code and tests

### Public API

These define aspects of the public Numba interface.

- numba/core/decorators.py - User-facing decorators for compiling regular functions on the CPU

- numba/core/extending.py - Public decorators for extending Numba (`overload`, `intrinsic`, etc) - numba/experimental/structref.py - Public API for defining a mutable struct

- numba/core/ccallback.py - `@cfunc` decorator for compiling functions to a fixed C signature. Used to make callbacks.

- numba/np/ufunc/decorators.py - ufunc/gufunc compilation decorators

- numba/core/config.py - Numba global config options and environment variable handling

- numba/core/annotations - Gathering and printing type annotations of Numba IR

- numba/core/annotations/pretty_annotate.py - Code highlighting of Numba functions and types (both ANSI terminal and HTML)

- numba/core/event.py - A simple event system for applications to listen to specific compiler events.

### Dispatching

- numba/core/dispatcher.py - Dispatcher objects are compiled functions produced by `@jit`. A dispatcher has different implementations for different type signatures.

- numba/_dispatcher.cpp - C++ dispatcher implementation (for speed on common data types)

### Compiler Pipeline

- numba/core/compiler.py - Compiler pipelines and flags

- numba/core/errors.py - Numba exception and warning classes

- numba/core/ir.py - Numba IR data structure objects

- numba/core/bytecode.py - Bytecode parsing and function identity (??)

- numba/core/interpreter.py - Translate Python interpreter bytecode to Numba IR

- numba/core/analysis.py - Utility functions to analyze Numba IR (variable lifetime, prune branches, etc)

- numba/core/controlflow.py - Control flow analysis of Numba IR and Python bytecode

- numba/core/typeinfer.py - Type inference algorithm

- numba/core/transforms.py - Numba IR transformations

- numba/core/rewrites - Rewrite passes used by compiler

- numba/core/rewrites/__init__.py - Loads all rewrite passes so they are put into the registry

- numba/core/rewrites/registry.py - Registry object for collecting rewrite passes

- numba/core/rewrites/ir_print.py - Write print() calls into special print nodes in the IR

- numba/core/rewrites/static_raise.py - Converts exceptions with static arguments into a special form that can be lowered

- numba/core/rewrites/static_getitem.py - Rewrites getitem and setitem with constant arguments to allow type inference

- numba/core/rewrites/static_binop.py - Rewrites binary operations (specifically `**`) with constant arguments so faster code can be generated

- numba/core/inline_closurecall.py - Inlines body of closure functions to call site. Support for array comprehensions, reduction inlining, and stencil inlining.

- numba/core/postproc.py - Postprocessor for Numba IR that computes variable lifetime, inserts del operations, and handles generators

- numba/core/lowering.py - General implementation of lowering Numba IR to LLVM numba/core/environment.py - Runtime environment object

- numba/core/withcontexts.py - General scaffolding for implementing context managers in nopython mode, and the objectmode context manager

- numba/core/pylowering.py - Lowering of Numba IR in object mode

- numba/core/pythonapi.py - LLVM IR code generation to interface with CPython API

- numba/core/targetconfig.py - Utils for target configurations such as compiler flags.

### Type Management

- numba/core/typeconv/ - Implementation of type casting and type signature matching in both C++ and Python

- numba/core/types/ - definition of the Numba type hierarchy, used everywhere in compiler to select implementations

- numba/core/consts.py - Constant inference (used to make constant values available during codegen when possible)

- numba/core/datamodel - LLVM IR representations of data types in different contexts

- numba/core/datamodel/models.py - Models for most standard types

- numba/core/datamodel/registry.py - Decorator to register new data models

- numba/core/datamodel/packer.py - Pack typed values into a data structure

- numba/core/datamodel/testing.py - Data model tests (this should move??)

- numba/core/datamodel/manager.py - Map types to data models

### Compiled Extensions

Numba uses a small amount of compiled C/C++ code for core functionality, like dispatching and type matching where performance matters, and it is more convenient to encapsulate direct interaction with CPython APIs.

- numba/_arraystruct.h - Struct for holding NumPy array attributes. Used in helperlib and the Numba Runtime.

- numba/_helperlib.c - C functions required by Numba compiled code at runtime. Linked into ahead-of-time compiled modules

- numba/_helpermod.c - Python extension module with pointers to functions from `_helperlib.c`

- numba/_dynfuncmod.c - Python extension module exporting _dynfunc.c functionality

- numba/_dynfunc.c - C level Environment and Closure objects (keep in sync with numba/target/base.py)

- numba/mathnames.h - Macros for defining names of math functions

- numba/_pymodule.h - C macros for Python 2/3 portable naming of C API functions

- numba/mviewbuf.c - Handles Python memoryviews

- numba/_typeof.h | numba/_typeof.cpp - C++ implementation of type fingerprinting, used by dispatcher

- numba/_numba_common.h - Portable C macro for marking symbols that can be shared between object files, but not outside the library.

### Misc Support

- numba/_version.py - Updated by versioneer

- numba/core/runtime - Language runtime. Currently manages reference-counted memory allocated on the heap by Numba-compiled functions

- numba/core/ir_utils.py - Utility functions for working with Numba IR data structures

- numba/core/cgutils.py - Utility functions for generating common code patterns in LLVM IR

- numba/core/utils.py - Python 2 backports of Python 3 functionality (also imports local copy of `six`)

- numba/misc/appdirs.py - Vendored package for determining application config directories on every platform

- numba/misc/POST.py - A power-on-self-test script Numba uses in CI to make sure the test runner and compilation is working.

- numba/core/compiler_lock.py - Global compiler lock because Numba’s usage of LLVM is not thread-safe

- numba/misc/special.py - Python stub implementations of special Numba functions (prange, gdb*)

- numba/core/itanium_mangler.py - Python implementation of Itanium C++ name mangling

- numba/misc/findlib.py - Helper function for locating shared libraries on all platforms

- numba/core/debuginfo.py - Helper functions to construct LLVM IR debug info

- numba/core/unsafe/refcount.py - Read reference count of object

- numba/core/unsafe/eh.py - Exception handling helpers

- numba/core/unsafe/nrt.py - Numba runtime (NRT) helpers

- numba/cpython/unsafe/tuple.py - Replace a value in a tuple slot

- numba/np/unsafe/ndarray.py - NumPy array helpers

- numba/core/unsafe/bytes.py - Copying and dereferencing data from void pointers

- numba/core/callwrapper.py - Handles argument unboxing and releasing the GIL when moving from Python to nopython mode

- numba/np/numpy_support.py - Helper functions for working with NumPy and translating Numba types to and from NumPy dtypes.

- numba/core/tracing.py - Decorator for tracing Python calls and emitting log messages

- numba/core/funcdesc.py - Classes for describing function metadata (used in the compiler)

- numba/core/sigutils.py - Helper functions for parsing and normalizing Numba type signatures

- numba/core/serialize.py - Support for pickling compiled functions

- numba/core/caching.py - Disk cache for compiled functions

- numba/np/npdatetime.py - Helper functions for implementing NumPy datetime64 support

- numba/misc/llvm_pass_timings.py - Helper to record timings of LLVM passes.

- numba/cloudpickle - Vendored cloudpickle subpackage

### Core Python Data Types

- numba/_hashtable.h | numba/_hashtable.cpp - Adaptation of the Python 3.7 hash table implementation

- numba/cext/dictobject.h | numba/cext/dictobject.c - C level implementation of typed dictionary

- numba/typed/dictobject.py - Nopython mode wrapper for typed dictionary

- numba/cext/listobject.h | numba/cext/listobject.c - C level implementation of typed list

- numba/typed/listobject.py - Nopython mode wrapper for typed list

- numba/typed/typedobjectutils.py - Common utilities for typed dictionary and list

- numba/cpython/unicode.py - Unicode strings (Python 3.5 and later)

- numba/typed - Python interfaces to statically typed containers

- numba/typed/typeddict.py - Python interface to typed dictionary

- numba/typed/typedlist.py - Python interface to typed list

- numba/experimental/jitclass - Implementation of experimental JIT compilation of Python classes

- numba/core/generators.py - Support for lowering Python generators

### Math

- numba/_random.c - Reimplementation of NumPy / CPython random number generator

- numba/_lapack.c - Wrappers for calling BLAS and LAPACK functions (requires SciPy)

### ParallelAccelerator

Code transformation passes that extract parallelizable code from a function and convert it into multithreaded gufunc calls.

- numba/parfors/parfor.py - General ParallelAccelerator

- numba/parfors/parfor_lowering.py - gufunc lowering for ParallelAccelerator

- numba/parfors/array_analysis.py - Array analysis passes used in ParallelAccelerator

### Stencil

Implementation of `@stencil`:

- numba/stencils/stencil.py - Stencil function decorator (implemented without ParallelAccelerator)

- numba/stencils/stencilparfor.py - ParallelAccelerator implementation of stencil

### Debugging Support

- numba/misc/gdb_hook.py - Hooks to jump into GDB from nopython mode

- numba/misc/cmdlang.gdb - Commands to setup GDB for setting explicit breakpoints from Python

### Type Signatures (CPU)

Some (usually older) Numba supported functionality separates the declaration of allowed type signatures from the definition of implementations. This package contains registries of type signatures that must be matched during type inference.

- numba/core/typing - Type signature module

- numba/core/typing/templates.py - Base classes for type signature templates

- numba/core/typing/cmathdecl.py - Python complex math (`cmath`) module

- numba/core/typing/bufproto.py - Interpreting objects supporting the buffer protocol

- numba/core/typing/mathdecl.py - Python `math` module

- numba/core/typing/listdecl.py - Python lists

- numba/core/typing/builtins.py - Python builtin global functions and operators

- numba/core/typing/setdecl.py - Python sets

- numba/core/typing/npydecl.py - NumPy ndarray (and operators), NumPy functions

- numba/core/typing/arraydecl.py - Python `array` module

- numba/core/typing/context.py - Implementation of typing context (class that collects methods used in type inference)

- numba/core/typing/collections.py - Generic container operations and namedtuples

- numba/core/typing/ctypes_utils.py - Typing ctypes-wrapped function pointers

- numba/core/typing/enumdecl.py - Enum types

- numba/core/typing/cffi_utils.py - Typing of CFFI objects

- numba/core/typing/typeof.py - Implementation of typeof operations (maps Python object to Numba type)

- numba/core/typing/asnumbatype.py - Implementation of `as_numba_type` operations (maps Python types to Numba type)

- numba/core/typing/npdatetime.py - Datetime dtype support for NumPy arrays

### Target Implementations (CPU)

Implementations of Python / NumPy functions and some data models. These modules are responsible for generating LLVM IR during lowering. Note that some of these modules do not have counterparts in the typing package because newer Numba extension APIs (like overload) allow typing and implementation to be specified together.

- numba/core/cpu.py - Context for code gen on CPU

- numba/core/base.py - Base class for all target contexts

- numba/core/codegen.py - Driver for code generation

- numba/core/boxing.py - Boxing and unboxing for most data types

- numba/core/intrinsics.py - Utilities for converting LLVM intrinsics to other math calls

- numba/core/callconv.py - Implements different calling conventions for Numba-compiled functions

- numba/core/options.py - Container for options that control lowering

- numba/core/optional.py - Special type representing value or `None`

- numba/core/registry.py - Registry object for collecting implementations for a specific target

- numba/core/imputils.py - Helper functions for lowering

- numba/core/externals.py - Registers external C functions needed to link generated code

- numba/core/fastmathpass.py - Rewrite pass to add fastmath attributes to function call sites and binary operations

- numba/core/removerefctpass.py - Rewrite pass to remove unnecessary incref/decref pairs

- numba/core/descriptors.py - empty base class for all target descriptors (is this needed?)

- numba/cpython/builtins.py - Python builtin functions and operators

- numba/cpython/cmathimpl.py - Python complex math module

- numba/cpython/enumimpl.py - Enum objects

- numba/cpython/hashing.py - Hashing algorithms

- numba/cpython/heapq.py - Python `heapq` module

- numba/cpython/iterators.py - Iterable data types and iterators

- numba/cpython/listobj.py - Python lists

- numba/cpython/mathimpl.py - Python `math` module

- numba/cpython/numbers.py - Numeric values (int, float, etc)

- numba/cpython/printimpl.py - Print function

- numba/cpython/randomimpl.py - Python and NumPy `random` modules

- numba/cpython/rangeobj.py - Python range objects

- numba/cpython/slicing.py - Slice objects, and index calculations used in slicing

- numba/cpython/setobj.py - Python set type

- numba/cpython/tupleobj.py - Tuples (statically typed as immutable struct)

- numba/misc/cffiimpl.py - CFFI functions

- numba/misc/quicksort.py - Quicksort implementation used with list and array objects

- numba/misc/mergesort.py - Mergesort implementation used with array objects

- numba/np/arraymath.py - Math operations on arrays (both Python and NumPy)

- numba/np/arrayobj.py - Array operations (both NumPy and buffer protocol)

- numba/np/linalg.py - NumPy linear algebra operations

- numba/np/npdatetime.py - NumPy datetime operations

- numba/np/npyfuncs.py - Kernels used in generating some NumPy ufuncs

- numba/np/npyimpl.py - Implementations of most NumPy ufuncs

- numba/np/polynomial/polynomial_functions.py - Implementations of NumPy `Polynomial` functions

- numba/np/polynomial/polynomial_core.py - Implementations of NumPy `Polynomial` class

- numba/np/ufunc_db.py - Big table mapping types to ufunc implementations

### Ufunc Compiler and Runtime

- numba/np/ufunc - ufunc compiler implementation

- numba/np/ufunc/_internal.h | numba/np/ufunc/_internal.c - Python extension module with helper functions that use CPython & NumPy C API

- numba/np/ufunc/_ufunc.c - Used by _internal.c

- numba/np/ufunc/gufunc_scheduler.h | numba/np/ufunc/gufunc_scheduler.cpp - Schedule work chunks to threads

- numba/np/ufunc/dufunc.py - Special ufunc that can compile new implementations at call time

- numba/np/ufunc/ufuncbuilder.py - Top-level orchestration of ufunc/gufunc compiler pipeline

- numba/np/ufunc/sigparse.py - Parser for generalized ufunc indexing signatures

- numba/np/ufunc/parallel.py - Codegen for `parallel` target

- numba/np/ufunc/array_exprs.py - Rewrite pass for turning array expressions in regular functions into ufuncs

- numba/np/ufunc/wrappers.py - Wrap scalar function kernel with loops

- numba/np/ufunc/workqueue.h | numba/np/ufunc/workqueue.c - Threading backend based on pthreads/Windows threads and queues

- numba/np/ufunc/omppool.cpp - Threading backend based on OpenMP

- numba/np/ufunc/tbbpool.cpp - Threading backend based on TBB

### Unit Tests (CPU)

CPU unit tests (GPU target unit tests listed in later sections

- runtests.py - Convenience script that launches test runner and turns on full compiler tracebacks

- .coveragerc - Coverage.py configuration

- numba/runtests.py - Entry point to unittest runner

- numba/testing/_runtests.py - Implementation of custom test runner command line interface

- numba/tests/test_* - Test cases

- numba/tests/*_usecases.py - Python functions compiled by some unit tests

- numba/tests/support.py - Helper functions for testing and special TestCase implementation

- numba/tests/dummy_module.py - Module used in `test_dispatcher.py`

- numba/tests/npyufunc - ufunc / gufunc compiler tests

- numba/testing - Support code for testing

- numba/testing/loader.py - Find tests on disk

- numba/testing/notebook.py - Support for testing notebooks

- numba/testing/main.py - Numba test runner

### Command Line Utilities

- bin/numba - Command line stub, delegates to main in `numba_entry.py`

- numba/misc/numba_entry.py - Main function for `numba` command line tool

- numba/pycc - Ahead of time compilation of functions to shared library extension

- numba/pycc/__init__.py - Main function for `pycc` command line tool

- numba/pycc/cc.py - User-facing API for tagging functions to compile ahead of time

- numba/pycc/compiler.py - Compiler pipeline for creating standalone Python extension modules

- numba/pycc/llvm_types.py - Aliases to LLVM data types used by `compiler.py`

- numba/pycc/modulemixin.c - C file compiled into every compiled extension. Pulls in C source from Numba core that is needed to make extension standalone

- numba/pycc/platform.py - Portable interface to platform-specific compiler toolchains

- numba/pycc/decorators.py - Deprecated decorators for tagging functions to compile. Use `cc.py` instead.

### CUDA GPU Target

Note that the CUDA target does reuse some parts of the CPU target.

- numba/cuda/ - The implementation of the CUDA (NVIDIA GPU) target and associated unit tests

- numba/cuda/decorators.py - Compiler decorators for CUDA kernels and device functions

- numba/cuda/deviceufunc.py - Custom ufunc dispatch for CUDA

- numba/cuda/dispatcher.py - Dispatcher for CUDA JIT functions

- numba/cuda/printimpl.py - Special implementation of device printing

- numba/cuda/libdevice.py - Registers libdevice functions

- numba/cuda/kernels/ - Custom kernels for reduction and transpose

- numba/cuda/device_init.py - Initializes the CUDA target when imported

- numba/cuda/compiler.py - Compiler pipeline for CUDA target

- numba/cuda/intrinsic_wrapper.py - CUDA device intrinsics (shuffle, ballot, etc)

- numba/cuda/initialize.py - Deferred initialization of the CUDA device and subsystem. Called only when user imports `numba.cuda`

- numba/cuda/simulator_init.py - Initializes the CUDA simulator subsystem (only when user requests it with env var)

- numba/cuda/random.py - Implementation of random number generator

- numba/cuda/api.py - User facing APIs imported into `numba.cuda.*`

- numba/cuda/stubs.py - Python placeholders for functions that only can be used in GPU device code

- numba/cuda/simulator/ - Simulate execution of CUDA kernels in Python interpreter

- numba/cuda/vectorizers.py - Subclasses of ufunc/gufunc compilers for CUDA

- numba/cuda/args.py - Management of kernel arguments, including host<->device transfers

- numba/cuda/target.py - Typing and target contexts for GPU

- numba/cuda/cudamath.py - Type signatures for math functions in CUDA Python

- numba/cuda/errors.py - Validation of kernel launch configuration

- numba/cuda/nvvmutils.py - Helper functions for generating NVVM-specific IR

- numba/cuda/testing.py - Support code for creating CUDA unit tests and capturing standard out

- numba/cuda/cudadecl.py - Type signatures of CUDA API (threadIdx, blockIdx, atomics) in Python on GPU

- numba/cuda/cudaimpl.py - Implementations of CUDA API functions on GPU

- numba/cuda/codegen.py - Code generator object for CUDA target

- numba/cuda/cudadrv/ - Wrapper around CUDA driver API

- numba/cuda/cudadrv/dummyarray.py - Used to hold array information on the host, but not the data.

- numba/cuda/tests/ - CUDA unit tests, skipped when CUDA is not detected

- numba/cuda/tests/cudasim/ - Tests of CUDA simulator

- numba/cuda/tests/nocuda/ - Tests for NVVM functionality when CUDA not present

- numba/cuda/tests/cudapy/ - Tests of compiling Python functions for GPU

- numba/cuda/tests/cudadrv/ - Tests of Python wrapper around CUDA API