1 2Contributing to Numba 3===================== 4 5We welcome people who want to make contributions to Numba, big or small! 6Even simple documentation improvements are encouraged. If you have 7questions, don't hesitate to ask them (see below). 8 9 10Communication 11------------- 12 13Real-time Chat 14'''''''''''''' 15 16Numba uses Gitter for public real-time chat. To help improve the 17signal-to-noise ratio, we have two channels: 18 19* `numba/numba <https://gitter.im/numba/numba>`_: General Numba discussion, 20 questions, and debugging help. 21* `numba/numba-dev <https://gitter.im/numba/numba-dev>`_: Discussion of PRs, 22 planning, release coordination, etc. 23 24Both channels are public, but we may ask that discussions on numba-dev move to 25the numba channel. This is simply to ensure that numba-dev is easy for core 26developers to keep up with. 27 28Note that the Github issue tracker is the best place to report bugs. Bug 29reports in chat are difficult to track and likely to be lost. 30 31Forum 32..... 33 34Numba uses Discourse as a forum for longer running threads such as design 35discussions and roadmap planning. There are various categories available and it 36can be reached at: `numba.discourse.group <https://numba.discourse.group/>`_. 37 38Mailing-list 39'''''''''''' 40 41We have a public mailing-list that you can e-mail at numba-users@anaconda.com. 42You can subscribe and read the archives on 43`Google Groups <https://groups.google.com/a/continuum.io/forum/#!forum/numba-users>`_. 44 45Weekly Meetings 46''''''''''''''' 47 48The core Numba developers have a weekly video conference to discuss roadmap, 49feature planning, and outstanding issues. These meetings are invite only, but 50minutes will be taken and will be posted to the 51`Numba wiki <https://github.com/numba/numba/wiki/Meeting-Minutes>`_. 52 53.. _report-numba-bugs: 54 55Bug tracker 56'''''''''''' 57 58We use the `Github issue tracker <https://github.com/numba/numba/issues>`_ 59to track both bug reports and feature requests. If you report an issue, 60please include specifics: 61 62* what you are trying to do; 63* which operating system you have and which version of Numba you are running; 64* how Numba is misbehaving, e.g. the full error traceback, or the unexpected 65 results you are getting; 66* as far as possible, a code snippet that allows full reproduction of your 67 problem. 68 69Getting set up 70-------------- 71 72If you want to contribute, we recommend you fork our `Github repository 73<https://github.com/numba/numba>`_, then create a branch representing 74your work. When your work is ready, you should submit it as a pull 75request from the Github interface. 76 77If you want, you can submit a pull request even when you haven't finished 78working. This can be useful to gather feedback, or to stress your changes 79against the :ref:`continuous integration <travis_ci>` platform. In this 80case, please prepend ``[WIP]`` to your pull request's title. 81 82.. _buildenv: 83 84Build environment 85''''''''''''''''' 86 87Numba has a number of dependencies (mostly `NumPy <http://www.numpy.org/>`_ 88and `llvmlite <https://github.com/numba/llvmlite>`_) with non-trivial build 89instructions. Unless you want to build those dependencies yourself, we 90recommend you use `conda <http://conda.pydata.org/miniconda.html>`_ to 91create a dedicated development environment and install precompiled versions 92of those dependencies there. 93 94First add the Anaconda Cloud ``numba`` channel so as to get development builds 95of the llvmlite library:: 96 97 $ conda config --add channels numba 98 99Then create an environment with the right dependencies:: 100 101 $ conda create -n numbaenv python=3.6 llvmlite numpy scipy jinja2 cffi 102 103.. note:: 104 This installs an environment based on Python 3.6, but you can of course 105 choose another version supported by Numba. To test additional features, 106 you may also need to install ``tbb`` and/or ``llvm-openmp`` and 107 ``intel-openmp``. 108 109To activate the environment for the current shell session:: 110 111 $ conda activate numbaenv 112 113.. note:: 114 These instructions are for a standard Linux shell. You may need to 115 adapt them for other platforms. 116 117Once the environment is activated, you have a dedicated Python with the 118required dependencies:: 119 120 $ python 121 Python 3.6.6 |Anaconda, Inc.| (default, Jun 28 2018, 11:07:29) 122 [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin 123 Type "help", "copyright", "credits" or "license" for more information. 124 >>> import llvmlite 125 >>> llvmlite.__version__ 126 '0.24.0' 127 128 129Building Numba 130'''''''''''''' 131 132For a convenient development workflow, we recommend you build Numba inside 133its source checkout:: 134 135 $ git clone git://github.com/numba/numba.git 136 $ cd numba 137 $ python setup.py build_ext --inplace 138 139This assumes you have a working C compiler and runtime on your development 140system. You will have to run this command again whenever you modify 141C files inside the Numba source tree. 142 143Running tests 144''''''''''''' 145 146Numba is validated using a test suite comprised of various kind of tests 147(unit tests, functional tests). The test suite is written using the 148standard :py:mod:`unittest` framework. 149 150The tests can be executed via ``python -m numba.runtests``. If you are 151running Numba from a source checkout, you can type ``./runtests.py`` 152as a shortcut. Various options are supported to influence test running 153and reporting. Pass ``-h`` or ``--help`` to get a glimpse at those options. 154Examples: 155 156* to list all available tests:: 157 158 $ python -m numba.runtests -l 159 160* to list tests from a specific (sub-)suite:: 161 162 $ python -m numba.runtests -l numba.tests.test_usecases 163 164* to run those tests:: 165 166 $ python -m numba.runtests numba.tests.test_usecases 167 168* to run all tests in parallel, using multiple sub-processes:: 169 170 $ python -m numba.runtests -m 171 172* For a detailed list of all options:: 173 174 $ python -m numba.runtests -h 175 176The numba test suite can take a long time to complete. When you want to avoid 177the long wait, it is useful to focus on the failing tests first with the 178following test runner options: 179 180* The ``--failed-first`` option is added to capture the list of failed tests 181 and to re-execute them first:: 182 183 $ python -m numba.runtests --failed-first -m -v -b 184 185* The ``--last-failed`` option is used with ``--failed-first`` to execute 186 the previously failed tests only:: 187 188 $ python -m numba.runtests --last-failed -m -v -b 189 190When debugging, it is useful to turn on logging. Numba logs using the 191standard ``logging`` module. One can use the standard ways (i.e. 192``logging.basicConfig``) to configure the logging behavior. To enable logging 193in the test runner, there is a ``--log`` flag for convenience:: 194 195 $ python -m numba.runtests --log 196 197 198Development rules 199----------------- 200 201Code reviews 202'''''''''''' 203 204Any non-trivial change should go through a code review by one or several of 205the core developers. The recommended process is to submit a pull request 206on github. 207 208A code review should try to assess the following criteria: 209 210* general design and correctness 211* code structure and maintainability 212* coding conventions 213* docstrings, comments 214* test coverage 215 216Coding conventions 217'''''''''''''''''' 218 219All Python code should follow :pep:`8`. Our C code doesn't have a 220well-defined coding style (would it be nice to follow :pep:`7`?). 221Code and documentation should generally fit within 80 columns, for 222maximum readability with all existing tools (such as code review UIs). 223 224Numba uses `Flake8 <http://flake8.pycqa.org/en/latest/>`_ to ensure a consistent 225Python code format throughout the project. ``flake8`` can be installed 226with ``pip`` or ``conda`` and then run from the root of the Numba repository:: 227 228 flake8 numba 229 230Optionally, you may wish to setup `pre-commit hooks <https://pre-commit.com/>`_ 231to automatically run ``flake8`` when you make a git commit. This can be 232done by installing ``pre-commit``:: 233 234 pip install pre-commit 235 236and then running:: 237 238 pre-commit install 239 240from the root of the Numba repository. Now ``flake8`` will be run each time 241you commit changes. You can skip this check with ``git commit --no-verify``. 242 243Numba has started the process of using `type hints <https://www.python.org/dev/peps/pep-0484/>`_ in its code base. This 244will be a gradual process of extending the number of files that use type hints, as well as going from voluntary to 245mandatory type hints for new features. `Mypy <http://mypy-lang.org/>`_ is used for automated static checking. 246 247At the moment, only certain files are checked by mypy. The list can be found in ``mypy.ini``. When making changes to 248those files, it is necessary to add the required type hints such that mypy tests will pass. Only in exceptional 249circumstances should ``type: ignore`` comments be used. 250 251If you are contributing a new feature, we encourage you to use type hints, even if the file is not currently in the 252checklist. If you want to contribute type hints to enable a new file to be in the checklist, please add the file to the 253``files`` variable in ``mypy.ini``, and decide what level of compliance you are targetting. Level 3 is basic static 254checks, while levels 2 and 1 represent stricter checking. The levels are described in details in ``mypy.ini``. 255 256There is potential for confusion between the Numba module ``typing`` and Python built-in module ``typing`` used for type 257hints, as well as between Numba types---such as ``Dict`` or ``Literal``---and ``typing`` types of the same name. 258To mitigate the risk of confusion we use a naming convention by which objects of the built-in ``typing`` module are 259imported with an ``pt`` prefix. For example, ``typing.Dict`` is imported as ``from typing import Dict as ptDict``. 260 261Stability 262''''''''' 263 264The repository's ``master`` branch is expected to be stable at all times. 265This translates into the fact that the test suite passes without errors 266on all supported platforms (see below). This also means that a pull request 267also needs to pass the test suite before it is merged in. 268 269.. _travis_ci: 270 271Platform support 272'''''''''''''''' 273 274Every commit to the master branch is automatically tested on all of the 275platforms Numba supports. This includes ARMv7, ARMv8, POWER8, as well as both 276AMD and NVIDIA GPUs. The build system however is internal to Anaconda, so we 277also use `Travis CI <https://travis-ci.org/numba/numba>`_ and 278`Azure <https://dev.azure.com/numba/numba/_build>`_ to provide public continuous 279integration information for as many combinations as can be supported by the 280service. Travis CI automatically tests all pull requests on OS X and Linux, as 281well as a sampling of different Python and NumPy versions, Azure does the same 282but also includes Windows. If you see problems on platforms you are unfamiliar 283with, feel free to ask for help in your pull request. The Numba core developers 284can help diagnose cross-platform compatibility issues. 285 286 287Documentation 288------------- 289 290The Numba documentation is split over two repositories: 291 292* This documentation is in the ``docs`` directory inside the 293 `Numba repository <https://github.com/numba/numba>`_. 294 295* The `Numba homepage <http://numba.pydata.org>`_ has its sources in a 296 separate repository at https://github.com/numba/numba-webpage 297 298 299Main documentation 300'''''''''''''''''' 301 302This documentation is under the ``docs`` directory of the `Numba repository`_. 303It is built with `Sphinx <http://sphinx-doc.org/>`_ and 304`numpydoc <https://numpydoc.readthedocs.io/>`_, which are available using 305conda or pip; i.e. ``conda install sphinx numpydoc``. 306 307To build the documentation, you need the bootstrap theme:: 308 309 $ pip install sphinx_bootstrap_theme 310 311You can edit the source files under ``docs/source/``, after which you can 312build and check the documentation:: 313 314 $ make html 315 $ open _build/html/index.html 316 317Core developers can upload this documentation to the Numba website 318at http://numba.pydata.org by using the ``gh-pages.py`` script under ``docs``:: 319 320 $ python gh-pages.py version # version can be 'dev' or '0.16' etc 321 322then verify the repository under the ``gh-pages`` directory and use 323``git push``. 324 325Web site homepage 326''''''''''''''''' 327 328The Numba homepage on http://numba.pydata.org can be fetched from here: 329https://github.com/numba/numba-webpage 330 331After pushing documentation to a new version, core developers will want to 332update the website. Some notable files: 333 334* ``index.rst`` # Update main page 335* ``_templates/sidebar_versions.html`` # Update sidebar links 336* ``doc.rst`` # Update after adding a new version for numba docs 337* ``download.rst`` # Updata after uploading new numba version to pypi 338 339After updating run:: 340 341 $ make html 342 343and check out ``_build/html/index.html``. To push updates to the Web site:: 344 345 $ python _scripts/gh-pages.py 346 347then verify the repository under the ``gh-pages`` directory. Make sure the 348``CNAME`` file is present and contains a single line for ``numba.pydata.org``. 349Finally, use ``git push`` to update the website. 350