1.. be in -*- rst -*- mode! 2 3General tips and tricks 4================================ 5 6Interactively passing positional arguments 7----------------------------------------------- 8 9If you invoke ``tox`` like this: 10 11.. code-block:: shell 12 13 tox -- -x tests/test_something.py 14 15the arguments after the ``--`` will be substituted 16everywhere where you specify ``{posargs}`` in your 17test commands, for example using ``pytest``: 18 19.. code-block:: ini 20 21 [testenv] 22 # Could also be in a specific ``[testenv:<NAME>]`` section 23 commands = pytest {posargs} 24 25or using ``nosetests``: 26 27.. code-block:: ini 28 29 [testenv] 30 commands = nosetests {posargs} 31 32the above ``tox`` invocation will trigger the test runners to 33stop after the first failure and to only run a particular test file. 34 35You can specify defaults for the positional arguments using this 36syntax: 37 38.. code-block:: ini 39 40 [testenv] 41 commands = nosetests {posargs:--with-coverage} 42 43.. _recreate: 44 45Dependency changes and tracking 46------------------------------- 47 48Creating virtual environments and installing dependencies is a expensive operation. 49Therefore tox tries to avoid it whenever possible, meaning it will never perform this 50unless it detects with absolute certainty that it needs to perform an update. A tox 51environment creation is made up of: 52 53- create the virtual environment 54- install dependencies specified inside deps 55- if it's a library project (has build package phase), install library dependencies 56 (with potential extras) 57 58These three steps are only performed once (given they all succeeded). Subsequent calls 59that don't detect changes to the traits of that step will not alter the virtual 60environment in any way. When a change is detected for any of the steps, the entire 61virtual environment is removed and the operation starts from scratch (this is 62because it's very hard to determine what would the delta changes would be needed - 63e.g. a dependency could migrate from one dependency to another, and in this case 64we would need to install the new while removing the old one). 65 66Here's what traits we track at the moment for each steps: 67 68- virtual environment trait is tied to the python path the :conf:`basepython` 69 resolves too (if this config changes, the virtual environment will be recreated), 70- :conf:`deps` sections changes (meaning any string-level change for the entries, note 71 requirement file content changes are not tracked), 72- library dependencies are tracked at :conf:`extras` level (because there's no 73 Python API to enquire about the actual dependencies in a non-tool specific way, 74 e.g. setuptools has one way, flit something else, and poetry another). 75 76Whenever you change traits that are not tracked we recommend you to manually trigger a 77rebuild of the tox environment by passing the ``-r`` flag for the tox invocation. For 78instance, for a setuptools project whenever you modify the ``install_requires`` keyword 79at the next run force the recreation of the tox environment by passing the recreate cli 80tox flag. 81 82.. _`TOXENV`: 83 84Selecting one or more environments to run tests against 85-------------------------------------------------------- 86 87Using the ``-e ENV[,ENV36,...]`` option you explicitly list 88the environments where you want to run tests against. For 89example, given the previous sphinx example you may call: 90 91.. code-block:: shell 92 93 tox -e docs 94 95which will make ``tox`` only manage the ``docs`` environment 96and call its test commands. You may specify more than 97one environment like this: 98 99.. code-block:: shell 100 101 tox -e py27,py36 102 103which would run the commands of the ``py27`` and ``py36`` testenvironments 104respectively. The special value ``ALL`` selects all environments. 105 106You can also specify an environment list in your ``tox.ini``: 107 108.. code-block:: ini 109 110 [tox] 111 envlist = py27,py36 112 113or override it from the command line or from the environment variable 114``TOXENV``: 115 116.. code-block:: shell 117 118 export TOXENV=py27,py36 # in bash style shells 119 120.. _artifacts: 121 122Access package artifacts between multiple tox-runs 123-------------------------------------------------------- 124 125If you have multiple projects using tox you can make use of 126a ``distshare`` directory where ``tox`` will copy in sdist-packages so 127that another tox run can find the "latest" dependency. This feature 128allows to test a package against an unreleased development version 129or even an uncommitted version on your own machine. 130 131By default, ``{homedir}/.tox/distshare`` will be used for 132copying in and copying out artifacts (i.e. Python packages). 133 134For project ``two`` to depend on the ``one`` package you use 135the following entry: 136 137.. code-block:: ini 138 139 # example two/tox.ini 140 [testenv] 141 # install latest package from "one" project 142 deps = {distshare}/one-*.zip 143 144That's all. tox running on project ``one`` will copy the sdist-package 145into the ``distshare`` directory after which a ``tox`` run on project 146``two`` will grab it because ``deps`` contain an entry with the 147``one-*.zip`` pattern. If there is more than one matching package the 148highest version will be taken. ``tox`` uses verlib_ to compare version 149strings which must be compliant with :pep:`386`. 150 151If you want to use this with Jenkins_, also checkout the :ref:`jenkins artifact example`. 152 153.. _verlib: https://bitbucket.org/tarek/distutilsversion/ 154 155basepython defaults, overriding 156+++++++++++++++++++++++++++++++ 157 158For any ``pyXY`` test environment name the underlying ``pythonX.Y`` executable 159will be searched in your system ``PATH``. Similarly, for ``jython`` and 160``pypy`` the respective ``jython`` and ``pypy-c`` names will be looked for. 161The executable must exist in order to successfully create *virtualenv* 162environments. On Windows a ``pythonX.Y`` named executable will be searched in 163typical default locations using the ``C:\PythonX.Y\python.exe`` pattern. 164 165All other targets will use the system ``python`` instead. You can override any 166of the default settings by defining the :conf:`basepython` variable in a 167specific test environment section, for example: 168 169.. code-block:: ini 170 171 [testenv:docs] 172 basepython = python2.7 173 174Avoiding expensive sdist 175------------------------ 176 177Some projects are large enough that running an sdist, followed by 178an install every time can be prohibitively costly. To solve this, 179there are two different options you can add to the ``tox`` section. First, 180you can simply ask tox to please not make an sdist: 181 182.. code-block:: ini 183 184 [tox] 185 skipsdist=True 186 187If you do this, your local software package will not be installed into 188the virtualenv. You should probably be okay with that, or take steps 189to deal with it in your commands section: 190 191.. code-block:: ini 192 193 [testenv] 194 commands = python setup.py develop 195 pytest 196 197Running ``setup.py develop`` is a common enough model that it has its own 198option: 199 200.. code-block:: ini 201 202 [testenv] 203 usedevelop=True 204 205And a corresponding command line option ``--develop``, which will set 206``skipsdist`` to True and then perform the ``setup.py develop`` step at the 207place where ``tox`` normally performs the installation of the sdist. 208Specifically, it actually runs ``pip install -e .`` behind the scenes, which 209itself calls ``setup.py develop``. 210 211There is an optimization coded in to not bother re-running the command if 212``$projectname.egg-info`` is newer than ``setup.py`` or ``setup.cfg``. 213 214.. include:: ../links.rst 215 216 217Understanding ``InvocationError`` exit codes 218-------------------------------------------- 219 220When a command (defined by ``commands =`` in ``tox.ini``) fails, 221it has a non-zero exit code, 222and an ``InvocationError`` exception is raised by ``tox``: 223 224.. code-block:: shell 225 226 ERROR: InvocationError for command 227 '<command defined in tox.ini>' (exited with code 1) 228 229If the command starts with ``pytest`` or ``python setup.py test`` for instance, 230then the `pytest exit codes`_ are relevant. 231 232On unix systems, there are some rather `common exit codes`_. 233This is why for exit codes larger than 128, 234if a signal with number equal to ``<exit code> - 128`` is found 235in the :py:mod:`signal` module, an additional hint is given: 236 237.. code-block:: shell 238 239 ERROR: InvocationError for command 240 '<command>' (exited with code 139) 241 Note: this might indicate a fatal error signal (139 - 128 = 11: SIGSEGV) 242 243where ``<command>`` is the command defined in ``tox.ini``, with quotes removed. 244 245The signal numbers (e.g. 11 for a segmentation fault) can be found in the 246"Standard signals" section of the `signal man page`_. 247Their meaning is described in `POSIX signals`_. 248 249Beware that programs may issue custom exit codes with any value, 250so their documentation should be consulted. 251 252 253Sometimes, no exit code is given at all. 254An example may be found in `pytest-qt issue #170`_, 255where Qt was calling ``abort()`` instead of ``exit()``. 256 257.. seealso:: :ref:`ignoring exit code`. 258 259.. _`pytest exit codes`: https://docs.pytest.org/en/latest/usage.html#possible-exit-codes 260.. _`common exit codes`: http://www.faqs.org/docs/abs/HTML/exitcodes.html 261.. _`abort()``: http://www.unix.org/version2/sample/abort.html 262.. _`pytest-qt issue #170`: https://github.com/pytest-dev/pytest-qt/issues/170 263.. _`signal man page`: http://man7.org/linux/man-pages/man7/signal.7.html 264.. _`POSIX signals`: https://en.wikipedia.org/wiki/Signal_(IPC)#POSIX_signals 265