1# Why Test Extensions
2
3Previously, Inkscape extensions were not tested for quality or correctness. But since 1.0, the extensions repository is far more strict about requiring tests and requiring tests to pass before changes can be merged in.
4
5You may find yourself being frustrated by the tests, especially if at first it doesn't make sense why they are failing. But these tests are important and I ask that everyone be as kind as they can to make sure the quality of the repository is maintained.
6
7# Running Tests
8
9You must install the program `pytest` in order to run these tests. Both Pytest and Pytest-Coverage are required to run tests.
10
11Usually the best way to install it is:
12
13```shell
14$ pip3 install pytest pytest-cov
15```
16
17You may run all tests by omitting any other parameters or select tests by adding the test filename that you want to run.
18
19```shell
20$ pytest
21$ pytest tests/test_my_extension.py
22```
23
24You can also run tests until the first time they fail, and ask pytest to run the previously failed tests first. This can cut down how long pytest takes to run before hitting a failure.
25
26```shell
27$ pytest -x --ff
28```
29
30More info here: https://docs.pytest.org/en/latest/getting-started.html
31
32# Test Files
33
34Each extension should have its own test file in the tests directory. This test may be a series of function tests or "comparison" tests. The comparison tests will fail whenever the output of an extension changes, so often they will need to be updated to reflect your changes.
35
36Usually the test file will be named `tests/test_{name_of_extension}.py` using the same name as the extension file itself. For tests covering inkex and other modules you may find test files have the format `tests/test_{package}_{module}.py` or similar.
37
38Each test can be run independently as shown in the previous section.
39
40# Test Data
41
42As well as python test files, each test will normally depend on additional data. From source svg files, to output comparison tests and other such things.
43
44This data is always held in `tests/data`, when writing tests, please make sure your data goes into the right directory. If you are updating the comparison test, usually you just need to rename the `export` file generated and remove the `.export` suffix to enable it.
45
46See tests/data/README.md for further information.
47
48# Writing or Updating tests
49
50You need to read the documentation available inside the tester module to learn how to write tests, or what the test code means. From a python3 terminal type:
51
52```python
53from inkex import tester
54help(tester)
55```
56
57# Coverage
58
59Coverage reports tell us how much of an extension is being exercised when tests are run.
60
61The latest coverage report for master branch can be found at
62https://inkscape.gitlab.io/extensions/coverage/.
63
64To run a complete coverage report, you can specify the `--cov=.` option like so:
65
66```shell
67$ pytest --cov=. --cov-report term
68```
69
70For a single extension coverage report, you can limit it further with:
71
72```shell
73$ pytest --cov=my_extension.py --cov-report term
74```
75
76## Testing Options
77
78Tests can be run with these options that are provided as environment variables:
79
80    FAIL_ON_DEPRECATION=1 - Will instantly fail any use of deprecated APIs
81    EXPORT_COMPARE=1 - Generate output files from comparisons. This is useful for manually checking the output as well as updating the comparison data.
82    NO_MOCK_COMMANDS=1 - Instead of using the mock data, actually call commands. This will also generate the msg files similar to export compare.
83    INKSCAPE_COMMAND=/other/inkscape - Use a different Inkscape (for example development version) while running commands. Works outside of tests too.
84    XML_DIFF=1 - Attempt to output an XML diff file, this can be useful for debugging to see differences in context.
85    DEBUG_KEY=1 - Export mock file keys for debugging. This is a highly specialised option for debugging key generation.
86
87For example:
88
89```shell
90$ EXPORT_COMPARE=1 pytest
91```
92
93or
94
95```shell
96export EXPORT_COMPARE=1
97pytest
98```
99
100# Testing custom extensions
101
102The same testing framework can be used in your own extension repositories by requiring the inkex module and using the inkex.tester module set. It should be available with Inkscape or can be installed via pypi.
103
104This is a great way of ensuring you have access to the same tools Inkscape uses to test, and makes it easier for your external extension to make its way to the core repository without resistance.
105