1.. highlight:: python
2.. _`goodpractices`:
3
4Good Integration Practices
5=================================================
6
7Install package with pip
8-------------------------------------------------
9
10For development, we recommend you use venv_ for virtual environments and
11pip_ for installing your application and any dependencies,
12as well as the ``pytest`` package itself.
13This ensures your code and dependencies are isolated from your system Python installation.
14
15Next, place a ``setup.py`` file in the root of your package with the following minimum content:
16
17.. code-block:: python
18
19    from setuptools import setup, find_packages
20
21    setup(name="PACKAGENAME", packages=find_packages())
22
23Where ``PACKAGENAME`` is the name of your package. You can then install your package in "editable" mode by running from the same directory:
24
25.. code-block:: bash
26
27     pip install -e .
28
29which lets you change your source code (both tests and application) and rerun tests at will.
30This is similar to running ``python setup.py develop`` or ``conda develop`` in that it installs
31your package using a symlink to your development code.
32
33.. _`test discovery`:
34.. _`Python test discovery`:
35
36Conventions for Python test discovery
37-------------------------------------------------
38
39``pytest`` implements the following standard test discovery:
40
41* If no arguments are specified then collection starts from :confval:`testpaths`
42  (if configured) or the current directory. Alternatively, command line arguments
43  can be used in any combination of directories, file names or node ids.
44* Recurse into directories, unless they match :confval:`norecursedirs`.
45* In those directories, search for ``test_*.py`` or ``*_test.py`` files, imported by their `test package name`_.
46* From those files, collect test items:
47
48  * ``test`` prefixed test functions or methods outside of class
49  * ``test`` prefixed test functions or methods inside ``Test`` prefixed test classes (without an ``__init__`` method)
50
51For examples of how to customize your test discovery :doc:`example/pythoncollection`.
52
53Within Python modules, ``pytest`` also discovers tests using the standard
54:ref:`unittest.TestCase <unittest.TestCase>` subclassing technique.
55
56
57Choosing a test layout / import rules
58-------------------------------------
59
60``pytest`` supports two common test layouts:
61
62Tests outside application code
63^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
64
65Putting tests into an extra directory outside your actual application code
66might be useful if you have many functional tests or for other reasons want
67to keep tests separate from actual application code (often a good idea):
68
69.. code-block:: text
70
71    setup.py
72    mypkg/
73        __init__.py
74        app.py
75        view.py
76    tests/
77        test_app.py
78        test_view.py
79        ...
80
81This has the following benefits:
82
83* Your tests can run against an installed version after executing ``pip install .``.
84* Your tests can run against the local copy with an editable install after executing ``pip install --editable .``.
85* If you don't have a ``setup.py`` file and are relying on the fact that Python by default puts the current
86  directory in ``sys.path`` to import your package, you can execute ``python -m pytest`` to execute the tests against the
87  local copy directly, without using ``pip``.
88
89.. note::
90
91    See :ref:`pytest vs python -m pytest` for more information about the difference between calling ``pytest`` and
92    ``python -m pytest``.
93
94Note that this scheme has a drawback if you are using ``prepend`` :ref:`import mode <import-modes>`
95(which is the default): your test files must have **unique names**, because
96``pytest`` will import them as *top-level* modules since there are no packages
97to derive a full package name from. In other words, the test files in the example above will
98be imported as ``test_app`` and ``test_view`` top-level modules by adding ``tests/`` to
99``sys.path``.
100
101If you need to have test modules with the same name, you might add ``__init__.py`` files to your
102``tests`` folder and subfolders, changing them to packages:
103
104.. code-block:: text
105
106    setup.py
107    mypkg/
108        ...
109    tests/
110        __init__.py
111        foo/
112            __init__.py
113            test_view.py
114        bar/
115            __init__.py
116            test_view.py
117
118Now pytest will load the modules as ``tests.foo.test_view`` and ``tests.bar.test_view``, allowing
119you to have modules with the same name. But now this introduces a subtle problem: in order to load
120the test modules from the ``tests`` directory, pytest prepends the root of the repository to
121``sys.path``, which adds the side-effect that now ``mypkg`` is also importable.
122
123This is problematic if you are using a tool like `tox`_ to test your package in a virtual environment,
124because you want to test the *installed* version of your package, not the local code from the repository.
125
126.. _`src-layout`:
127
128In this situation, it is **strongly** suggested to use a ``src`` layout where application root package resides in a
129sub-directory of your root:
130
131.. code-block:: text
132
133    setup.py
134    src/
135        mypkg/
136            __init__.py
137            app.py
138            view.py
139    tests/
140        __init__.py
141        foo/
142            __init__.py
143            test_view.py
144        bar/
145            __init__.py
146            test_view.py
147
148
149This layout prevents a lot of common pitfalls and has many benefits, which are better explained in this excellent
150`blog post by Ionel Cristian Mărieș <https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure>`_.
151
152.. note::
153    The new ``--import-mode=importlib`` (see :ref:`import-modes`) doesn't have
154    any of the drawbacks above because ``sys.path`` and ``sys.modules`` are not changed when importing
155    test modules, so users that run
156    into this issue are strongly encouraged to try it and report if the new option works well for them.
157
158    The ``src`` directory layout is still strongly recommended however.
159
160
161Tests as part of application code
162^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
163
164Inlining test directories into your application package
165is useful if you have direct relation between tests and application modules and
166want to distribute them along with your application:
167
168.. code-block:: text
169
170    setup.py
171    mypkg/
172        __init__.py
173        app.py
174        view.py
175        test/
176            __init__.py
177            test_app.py
178            test_view.py
179            ...
180
181In this scheme, it is easy to run your tests using the ``--pyargs`` option:
182
183.. code-block:: bash
184
185    pytest --pyargs mypkg
186
187``pytest`` will discover where ``mypkg`` is installed and collect tests from there.
188
189Note that this layout also works in conjunction with the ``src`` layout mentioned in the previous section.
190
191
192.. note::
193
194    You can use Python3 namespace packages (PEP420) for your application
195    but pytest will still perform `test package name`_ discovery based on the
196    presence of ``__init__.py`` files.  If you use one of the
197    two recommended file system layouts above but leave away the ``__init__.py``
198    files from your directories it should just work on Python3.3 and above.  From
199    "inlined tests", however, you will need to use absolute imports for
200    getting at your application code.
201
202.. _`test package name`:
203
204.. note::
205
206    In ``prepend`` and ``append`` import-modes, if pytest finds a ``"a/b/test_module.py"``
207    test file while recursing into the filesystem it determines the import name
208    as follows:
209
210    * determine ``basedir``: this is the first "upward" (towards the root)
211      directory not containing an ``__init__.py``.  If e.g. both ``a``
212      and ``b`` contain an ``__init__.py`` file then the parent directory
213      of ``a`` will become the ``basedir``.
214
215    * perform ``sys.path.insert(0, basedir)`` to make the test module
216      importable under the fully qualified import name.
217
218    * ``import a.b.test_module`` where the path is determined
219      by converting path separators ``/`` into "." characters.  This means
220      you must follow the convention of having directory and file
221      names map directly to the import names.
222
223    The reason for this somewhat evolved importing technique is
224    that in larger projects multiple test modules might import
225    from each other and thus deriving a canonical import name helps
226    to avoid surprises such as a test module getting imported twice.
227
228    With ``--import-mode=importlib`` things are less convoluted because
229    pytest doesn't need to change ``sys.path`` or ``sys.modules``, making things
230    much less surprising.
231
232
233.. _`virtualenv`: https://pypi.org/project/virtualenv/
234.. _`buildout`: http://www.buildout.org/
235.. _pip: https://pypi.org/project/pip/
236
237.. _`use tox`:
238
239tox
240------
241
242Once you are done with your work and want to make sure that your actual
243package passes all tests you may want to look into `tox`_, the
244virtualenv test automation tool and its `pytest support
245<https://tox.readthedocs.io/en/latest/example/pytest.html>`_.
246tox helps you to setup virtualenv environments with pre-defined
247dependencies and then executing a pre-configured test command with
248options.  It will run tests against the installed package and not
249against your source code checkout, helping to detect packaging
250glitches.
251
252.. _`venv`: https://docs.python.org/3/library/venv.html
253