1.. highlight:: python 2.. _`goodpractices`: 3 4Good Integration Practices 5================================================= 6 7Install package with pip 8------------------------------------------------- 9 10For development, we recommend you use venv_ for virtual environments and 11pip_ for installing your application and any dependencies, 12as well as the ``pytest`` package itself. 13This ensures your code and dependencies are isolated from your system Python installation. 14 15Next, place a ``setup.py`` file in the root of your package with the following minimum content: 16 17.. code-block:: python 18 19 from setuptools import setup, find_packages 20 21 setup(name="PACKAGENAME", packages=find_packages()) 22 23Where ``PACKAGENAME`` is the name of your package. You can then install your package in "editable" mode by running from the same directory: 24 25.. code-block:: bash 26 27 pip install -e . 28 29which lets you change your source code (both tests and application) and rerun tests at will. 30This is similar to running ``python setup.py develop`` or ``conda develop`` in that it installs 31your package using a symlink to your development code. 32 33.. _`test discovery`: 34.. _`Python test discovery`: 35 36Conventions for Python test discovery 37------------------------------------------------- 38 39``pytest`` implements the following standard test discovery: 40 41* If no arguments are specified then collection starts from :confval:`testpaths` 42 (if configured) or the current directory. Alternatively, command line arguments 43 can be used in any combination of directories, file names or node ids. 44* Recurse into directories, unless they match :confval:`norecursedirs`. 45* In those directories, search for ``test_*.py`` or ``*_test.py`` files, imported by their `test package name`_. 46* From those files, collect test items: 47 48 * ``test`` prefixed test functions or methods outside of class 49 * ``test`` prefixed test functions or methods inside ``Test`` prefixed test classes (without an ``__init__`` method) 50 51For examples of how to customize your test discovery :doc:`example/pythoncollection`. 52 53Within Python modules, ``pytest`` also discovers tests using the standard 54:ref:`unittest.TestCase <unittest.TestCase>` subclassing technique. 55 56 57Choosing a test layout / import rules 58------------------------------------- 59 60``pytest`` supports two common test layouts: 61 62Tests outside application code 63^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 64 65Putting tests into an extra directory outside your actual application code 66might be useful if you have many functional tests or for other reasons want 67to keep tests separate from actual application code (often a good idea): 68 69.. code-block:: text 70 71 setup.py 72 mypkg/ 73 __init__.py 74 app.py 75 view.py 76 tests/ 77 test_app.py 78 test_view.py 79 ... 80 81This has the following benefits: 82 83* Your tests can run against an installed version after executing ``pip install .``. 84* Your tests can run against the local copy with an editable install after executing ``pip install --editable .``. 85* If you don't have a ``setup.py`` file and are relying on the fact that Python by default puts the current 86 directory in ``sys.path`` to import your package, you can execute ``python -m pytest`` to execute the tests against the 87 local copy directly, without using ``pip``. 88 89.. note:: 90 91 See :ref:`pytest vs python -m pytest` for more information about the difference between calling ``pytest`` and 92 ``python -m pytest``. 93 94Note that this scheme has a drawback if you are using ``prepend`` :ref:`import mode <import-modes>` 95(which is the default): your test files must have **unique names**, because 96``pytest`` will import them as *top-level* modules since there are no packages 97to derive a full package name from. In other words, the test files in the example above will 98be imported as ``test_app`` and ``test_view`` top-level modules by adding ``tests/`` to 99``sys.path``. 100 101If you need to have test modules with the same name, you might add ``__init__.py`` files to your 102``tests`` folder and subfolders, changing them to packages: 103 104.. code-block:: text 105 106 setup.py 107 mypkg/ 108 ... 109 tests/ 110 __init__.py 111 foo/ 112 __init__.py 113 test_view.py 114 bar/ 115 __init__.py 116 test_view.py 117 118Now pytest will load the modules as ``tests.foo.test_view`` and ``tests.bar.test_view``, allowing 119you to have modules with the same name. But now this introduces a subtle problem: in order to load 120the test modules from the ``tests`` directory, pytest prepends the root of the repository to 121``sys.path``, which adds the side-effect that now ``mypkg`` is also importable. 122 123This is problematic if you are using a tool like `tox`_ to test your package in a virtual environment, 124because you want to test the *installed* version of your package, not the local code from the repository. 125 126.. _`src-layout`: 127 128In this situation, it is **strongly** suggested to use a ``src`` layout where application root package resides in a 129sub-directory of your root: 130 131.. code-block:: text 132 133 setup.py 134 src/ 135 mypkg/ 136 __init__.py 137 app.py 138 view.py 139 tests/ 140 __init__.py 141 foo/ 142 __init__.py 143 test_view.py 144 bar/ 145 __init__.py 146 test_view.py 147 148 149This layout prevents a lot of common pitfalls and has many benefits, which are better explained in this excellent 150`blog post by Ionel Cristian Mărieș <https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure>`_. 151 152.. note:: 153 The new ``--import-mode=importlib`` (see :ref:`import-modes`) doesn't have 154 any of the drawbacks above because ``sys.path`` and ``sys.modules`` are not changed when importing 155 test modules, so users that run 156 into this issue are strongly encouraged to try it and report if the new option works well for them. 157 158 The ``src`` directory layout is still strongly recommended however. 159 160 161Tests as part of application code 162^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 163 164Inlining test directories into your application package 165is useful if you have direct relation between tests and application modules and 166want to distribute them along with your application: 167 168.. code-block:: text 169 170 setup.py 171 mypkg/ 172 __init__.py 173 app.py 174 view.py 175 test/ 176 __init__.py 177 test_app.py 178 test_view.py 179 ... 180 181In this scheme, it is easy to run your tests using the ``--pyargs`` option: 182 183.. code-block:: bash 184 185 pytest --pyargs mypkg 186 187``pytest`` will discover where ``mypkg`` is installed and collect tests from there. 188 189Note that this layout also works in conjunction with the ``src`` layout mentioned in the previous section. 190 191 192.. note:: 193 194 You can use Python3 namespace packages (PEP420) for your application 195 but pytest will still perform `test package name`_ discovery based on the 196 presence of ``__init__.py`` files. If you use one of the 197 two recommended file system layouts above but leave away the ``__init__.py`` 198 files from your directories it should just work on Python3.3 and above. From 199 "inlined tests", however, you will need to use absolute imports for 200 getting at your application code. 201 202.. _`test package name`: 203 204.. note:: 205 206 In ``prepend`` and ``append`` import-modes, if pytest finds a ``"a/b/test_module.py"`` 207 test file while recursing into the filesystem it determines the import name 208 as follows: 209 210 * determine ``basedir``: this is the first "upward" (towards the root) 211 directory not containing an ``__init__.py``. If e.g. both ``a`` 212 and ``b`` contain an ``__init__.py`` file then the parent directory 213 of ``a`` will become the ``basedir``. 214 215 * perform ``sys.path.insert(0, basedir)`` to make the test module 216 importable under the fully qualified import name. 217 218 * ``import a.b.test_module`` where the path is determined 219 by converting path separators ``/`` into "." characters. This means 220 you must follow the convention of having directory and file 221 names map directly to the import names. 222 223 The reason for this somewhat evolved importing technique is 224 that in larger projects multiple test modules might import 225 from each other and thus deriving a canonical import name helps 226 to avoid surprises such as a test module getting imported twice. 227 228 With ``--import-mode=importlib`` things are less convoluted because 229 pytest doesn't need to change ``sys.path`` or ``sys.modules``, making things 230 much less surprising. 231 232 233.. _`virtualenv`: https://pypi.org/project/virtualenv/ 234.. _`buildout`: http://www.buildout.org/ 235.. _pip: https://pypi.org/project/pip/ 236 237.. _`use tox`: 238 239tox 240------ 241 242Once you are done with your work and want to make sure that your actual 243package passes all tests you may want to look into `tox`_, the 244virtualenv test automation tool and its `pytest support 245<https://tox.readthedocs.io/en/latest/example/pytest.html>`_. 246tox helps you to setup virtualenv environments with pre-defined 247dependencies and then executing a pre-configured test command with 248options. It will run tests against the installed package and not 249against your source code checkout, helping to detect packaging 250glitches. 251 252.. _`venv`: https://docs.python.org/3/library/venv.html 253