# Developer Guide

Do not hang around in Pythran code base without your developer guide! It is the compass that will guide you in the code jungle!

## Disclaimer

This document is a never ending work-in-progress draft. Please contribute!

## Configuration

Pythran can be configured with a rc file. An example is found in pythran/pythran.cfg. Look at it! To customize it:

    $> cp pythran/pythran.cfg ~/.pythranrc

In particular, you may want to add `-g -O0` to the `cxxflags`.

## Coding Style

All Python code must be conform to the PEP 8, and the `flake8` command must not yield any message when run on our database. Additionally, avoid backslashes, and try to make your code as concise as possible.

> $> flake8 pythran/.py pythran//.py –exclude=”pythran/tests/test.py,__init__.py”

C++ code use spaces (no tabs) and a tab width of 4.

## File Hierarchy

Listing the top level directory yields the following entries:

setup.py

  The files that describes what gets installed, that holds `PyPI` entries and such.

docs/

  If you’re reading this document, you know what it’s all about! `MANUAL` is the user documentation and `DEVGUIDE` is the developer documentation.

  Use `make` from this directory to produce the static website.

LICENSE

  Boring but important stuff.

MANIFEST.in

  Describe additional stuff to package there.

README.rst

  Quick introduction and description of _pythran_.

pythran/

  The source of all things.

pythran/tests/

  The source of all issues.

pythran/pythonic/

  Where C++ back-end lies.

## Validation

`pythran` uses the `unittest` module and the pytest package to manage test cases. The whole validation suite is run through the command:

    $> python -m pytest pythran/tests

To run it faster we use the `pytest` extension xdist, the test suite will run using all available cores. Otherwise it might run very slowly, something like four hours on a decent laptop :’(.

Note that it is possible to use the `pytest` module to pass a subset of the test suite:

    $> pytest -n 8 pythran/tests/test_list.py

runs all the tests found in `pythran/tests/test_list.py`.

Only compiler tests can be check using test filtering:

    $> pytest -n 8 pythran/tests -m "not module"

There are two kinds of tests in `pythran`:

1. unit tests that test a specific feature of the implementation. Such tests are listed as method of a class deriving from `test_env.TestEnv` and must call the `run_test(function_to_translate, *effective_parameters, **name_to_signature)` method 1. It translates `function_to_translate` into a native function using the type annotations given in the `name_to_signature` dictionary, runs both the python and the native version with `effective_parameters` as arguments and asserts the results are the same.

   1

     See examples in `pythran/tests/test_base.py` for more details.

2. test cases that are just plain python modules to be converted in native module by `pythran`. It is used to test complex situations, codes or benchmarks found on the web etc. They are just translated, not run. These test cases lie in `pythran/tests/cases/` and are listed in `pythran/tests/test_cases.py`.

## C++ runtime

The C++ code generated by `pythran` relies on a specific back-end, `pythonic`. It is a set of headers that mimics Python’s intrinsics and collections behavior in C++. It lies in `pythran/pythonic/`. There is one directory per module, e.g. `pythran/pythonic/numpy` for the `numpy` module, and one file per function, e.g. `pythran/pythonic/numpy/ones.hpp` for the `numpy.ones` function. Type definitions are stored in the seperate `pythran/pythonic/types` directory, one header per type. Each function header must be `#includ`-able independently, i.e. it itself includes all the type and function definition it needs. This helps keeping compilation time low.

All Pythran functions and types live in the `pythonic` namespace. Each extra module defines a new namespace, like `pythonic::math` or `pythonic::random`, and each type is defined in the `pythonic::types` namespace. The `DECLARE_FUNCTOR` and `DEFINE_FUNCTOR` macros from `pythonic/utils/functor.hpp` is commonly used to convert functions into functors and put them into the mandatory `functor` namespace.

The pythonic runtime can be used without Python support, so it is important to protect all Python-specific stuff inside `ENABLE_PYTHON_MODULE` guard.

All methods are represented by functions in Pythran. The associated pseudo-modules are prefixed and suffixed by a double underscore `__`, as in `pythran/pythonic/__list__`.

## Benchmarking and Testing

Stand-alone algorithms are put into `pythran/tests/cases`. They must be valid Pythran input (including spec annotations). To be taken into account by the validation suite, they must be listed in `pythran/tests/test_cases.py`. To be taken into account by the benchmarking suite, they must have a line starting with the `#runas` directive. Check `pythran/tests/matmul.py` for a complete example.

To run the benchmark suite, one can rely on:

    $> python setup.py bench --mode=<mode>

where <mode> is one among:

python

  Uses the interpreter used to run `setup.py`.

pythran

  Uses the Pythran compiler.

pythran+omp

  Uses the Pythran compiler in OpenMP mode.

All measurements are made using the `timeit` module. The number of iterations is customizable through the `--nb-iter` switch.

## How to

Add support for a new module
  1. Provide its C++ implementation in `pythran/pythonic++/<mymodule>`.

       `pythran/pythonic++/math/*.hpp` and `pythran/pythonic++/__list__/*.hpp` are good example to referer to.

  2. Provide its description in `pythran/tables.py`. Each function, method

       or variable must be listed there with the appropriate description.

  3. Provide its test suite in `pythran/tests/` under the name

       `test_my_module.py`. One test case per function, method or variable is great.

Add a new analysis

  1. Subclass one of `ModuleAnalysis`, `FunctionAnalysis` or `NodeAnalysis`.

  2. List analysis required by yours in the parent constructor, they will be built automatically and stored in the attribute with the corresponding uncameled name.

  3. Write your analysis as a regular `ast.NodeVisitor`. The analysis result must be stored in `self.result`.

  4. Use it either from another pass’s constructor, or through the `passmanager.gather` function.

Push changes into the holy trunk

  1. Use the `github` interface and the pull/push requests features

  2. Make your dev available on the web and asks for a merge on the IRC

       channel `#pythran` on `irc.oftc.net` (via your browser: https://webchat.oftc.net)