1Developer Guide 2############### 3 4Do not hang around in Pythran code base without your developer guide! It is 5the compass that will guide you in the code jungle! 6 7Disclaimer 8---------- 9 10This document is a never ending work-in-progress draft. Please contribute! 11 12Configuration 13------------- 14 15Pythran can be configured with a rc file. An example is found in `pythran/pythran.cfg`. 16Look at it! To customize it:: 17 18 $> cp pythran/pythran.cfg ~/.pythranrc 19 20In particular, you may want to add ``-g -O0`` to the ``cxxflags``. 21 22Coding Style 23------------ 24 25All Python code must be conform to the PEP 8, and the ``flake8`` command must not 26yield any message when run on our database. Additionally, avoid backslashes, 27and try to make your code as concise as possible. 28 29 $> flake8 pythran/*.py pythran/*/*.py --exclude="pythran/tests/test*.py,__init__.py" 30 31C++ code use spaces (no tabs) and a tab width of 4. 32 33File Hierarchy 34-------------- 35 36Listing the top level directory yields the following entries: 37 38setup.py 39 The files that describes what gets installed, that holds ``PyPI`` entries 40 and such. 41 42docs/ 43 If you're reading this document, you know what it's all about! ``MANUAL`` 44 is the user documentation and ``DEVGUIDE`` is the developer documentation. 45 46 Use ``make`` from this directory to produce the static website. 47 48LICENSE 49 Boring but important stuff. 50 51MANIFEST.in 52 Describe additional stuff to package there. 53 54README.rst 55 Quick introduction and description of _pythran_. 56 57pythran/ 58 The source of all things. 59 60pythran/tests/ 61 The source of all issues. 62 63pythran/pythonic/ 64 Where C++ back-end lies. 65 66 67Validation 68---------- 69 70``pythran`` uses the ``unittest`` module and the `pytest 71<http://pytest.org/latest/>`_ package to manage test cases. The whole 72validation suite is run through the command:: 73 74 $> python -m pytest pythran/tests 75 76To run it faster we use the ``pytest`` extension `xdist 77<https://pypi.org/project/pytest-xdist/>`_, the test suite will run using all 78available cores. Otherwise it might run **very** slowly, something like four 79hours on a decent laptop :'(. 80 81Note that it is possible to use the ``pytest`` module to pass a subset of the 82test suite:: 83 84 $> pytest -n 8 pythran/tests/test_list.py 85 86runs all the tests found in ``pythran/tests/test_list.py``. 87 88Only compiler tests can be check using test filtering:: 89 90 $> pytest -n 8 pythran/tests -m "not module" 91 92There are two kinds of tests in ``pythran``: 93 941. unit tests that test a specific feature of the implementation. Such tests 95 are listed as method of a class deriving from ``test_env.TestEnv`` and must 96 call the ``run_test(function_to_translate, *effective_parameters, 97 **name_to_signature)`` method [1]_. It translates ``function_to_translate`` 98 into a native function using the type annotations given in the 99 ``name_to_signature`` dictionary, runs both the python and the native 100 version with ``effective_parameters`` as arguments and asserts the results 101 are the same. 102 103 .. [1] See examples in ``pythran/tests/test_base.py`` for more details. 104 1052. test cases that are just plain python modules to be converted in native 106 module by ``pythran``. It is used to test complex situations, codes or 107 benchmarks found on the web etc. They are just translated, not run. These 108 test cases lie in ``pythran/tests/cases/`` and are listed in 109 ``pythran/tests/test_cases.py``. 110 111 112C++ runtime 113----------- 114 115The C++ code generated by ``pythran`` relies on a specific back-end, 116``pythonic``. It is a set of headers that mimics Python's intrinsics and 117collections behavior in C++. It lies in ``pythran/pythonic/``. There is one 118directory per module, e.g. ``pythran/pythonic/numpy`` for the ``numpy`` module, 119and one file per function, e.g. ``pythran/pythonic/numpy/ones.hpp`` for the 120``numpy.ones`` function. Type definitions are stored in the seperate 121``pythran/pythonic/types`` directory, one header per type. Each function header 122must be ``#includ``-able independently, i.e. it itself includes all the type 123and function definition it needs. This helps keeping compilation time low. 124 125All Pythran functions and types live in the ``pythonic`` namespace. Each extra 126module defines a new namespace, like ``pythonic::math`` or 127``pythonic::random``, and each type is defined in the ``pythonic::types`` 128namespace. The ``DECLARE_FUNCTOR`` and ``DEFINE_FUNCTOR`` macros from 129``pythonic/utils/functor.hpp`` is commonly used to convert functions into 130functors and put them into the mandatory ``functor`` namespace. 131 132The pythonic runtime can be used without Python support, so it is important to 133protect all Python-specific stuff inside ``ENABLE_PYTHON_MODULE`` guard. 134 135All methods are represented by functions in Pythran. The associated 136pseudo-modules are prefixed and suffixed by a double underscore ``__``, as in 137``pythran/pythonic/__list__``. 138 139 140Benchmarking and Testing 141------------------------ 142 143Stand-alone algorithms are put into ``pythran/tests/cases``. They must be valid 144Pythran input (including spec annotations). To be taken into account by the 145validation suite, they must be listed in ``pythran/tests/test_cases.py``. To be 146taken into account by the benchmarking suite, they must have a line starting 147with the ``#runas`` directive. Check ``pythran/tests/matmul.py`` for a complete 148example. 149 150To run the benchmark suite, one can rely on:: 151 152 $> python setup.py bench --mode=<mode> 153 154where *<mode>* is one among: 155 156python 157 Uses the interpreter used to run ``setup.py``. 158 159pythran 160 Uses the Pythran compiler. 161 162pythran+omp 163 Uses the Pythran compiler in OpenMP mode. 164 165All measurements are made using the ``timeit`` module. The number of iterations 166is customizable through the ``--nb-iter`` switch. 167 168How to 169------ 170 171:Add support for a new module: 172 1. Provide its C++ implementation in ``pythran/pythonic++/<mymodule>``. 173 ``pythran/pythonic++/math/*.hpp`` and 174 ``pythran/pythonic++/__list__/*.hpp`` are good example to referer to. 175 2. Provide its description in ``pythran/tables.py``. Each function, method 176 or variable must be listed there with the appropriate description. 177 3. Provide its test suite in ``pythran/tests/`` under the name 178 ``test_my_module.py``. One test case per function, method or variable 179 is great. 180 181:Add a new analysis: 182 1. Subclass one of ``ModuleAnalysis``, ``FunctionAnalysis`` or ``NodeAnalysis``. 183 2. List analysis required by yours in the parent constructor, they will be built automatically and stored in the attribute with the corresponding uncameled name. 184 3. Write your analysis as a regular ``ast.NodeVisitor``. The analysis result must be stored in ``self.result``. 185 4. Use it either from another pass's constructor, or through the ``passmanager.gather`` function. 186 187 188:Push changes into the holy trunk: 189 1. Use the ``github`` interface and the pull/push requests features 190 2. Make your dev available on the web and asks for a merge on the IRC 191 channel ``#pythran`` on ``irc.oftc.net`` 192 (via your browser: https://webchat.oftc.net) 193