1ignores
2=======
3
4Sanity tests for individual files can be skipped, and specific errors can be ignored.
5
6When to Ignore Errors
7---------------------
8
9Sanity tests are designed to improve code quality and identify common issues with content.
10When issues are identified during development, those issues should be corrected.
11
12As development of Ansible continues, sanity tests are expanded to detect issues that previous releases could not.
13To allow time for existing content to be updated to pass newer tests, ignore entries can be added.
14New content should not use ignores for existing sanity tests.
15
16When code is fixed to resolve sanity test errors, any relevant ignores must also be removed.
17If the ignores are not removed, this will be reported as an unnecessary ignore error.
18This is intended to prevent future regressions due to the same error recurring after being fixed.
19
20When to Skip Tests
21------------------
22
23Although rare, there are reasons for skipping a sanity test instead of ignoring the errors it reports.
24
25If a sanity test results in a traceback when processing content, that error cannot be ignored.
26If this occurs, open a new `bug report <https://github.com/ansible/ansible/issues/new?template=bug_report.md>`_ for the issue so it can be fixed.
27If the traceback occurs due to an issue with the content, that issue should be fixed.
28If the content is correct, the test will need to be skipped until the bug in the sanity test is fixed.
29
30    Caution should be used when skipping sanity tests instead of ignoring them.
31    Since the test is skipped entirely, resolution of the issue will not be automatically detected.
32    This will prevent prevent regression detection from working once the issue has been resolved.
33    For this reason it is a good idea to periodically review skipped entries manually to verify they are required.
34
35Ignore File Location
36--------------------
37
38The location of the ignore file depends on the type of content being tested.
39
40Ansible Collections
41~~~~~~~~~~~~~~~~~~~
42
43Since sanity tests change between Ansible releases, a separate ignore file is needed for each Ansible major release.
44
45The filename is ``tests/sanity/ignore-X.Y.txt`` where ``X.Y`` is the Ansible release being used to test the collection.
46
47Maintaining a separate file for each Ansible release allows a collection to pass tests for multiple versions of Ansible.
48
49Ansible
50~~~~~~~
51
52When testing Ansible, all ignores are placed in the ``test/sanity/ignore.txt`` file.
53
54Only a single file is needed because ``ansible-test`` is developed and released as a part of Ansible itself.
55
56Ignore File Format
57------------------
58
59The ignore file contains one entry per line.
60Each line consists of two columns, separated by a single space.
61Comments may be added at the end of an entry, started with a hash (``#``) character, which can be proceeded by zero or more spaces.
62Blank and comment only lines are not allowed.
63
64The first column specifies the file path that the entry applies to.
65File paths must be relative to the root of the content being tested.
66This is either the Ansible source or an Ansible collection.
67File paths cannot contain a space or the hash (``#``) character.
68
69The second column specifies the sanity test that the entry applies to.
70This will be the name of the sanity test.
71If the sanity test is specific to a version of Python, the name will include a dash (``-``) and the relevant Python version.
72If the named test uses error codes then the error code to ignore must be appended to the name of the test, separated by a colon (``:``).
73
74Below are some example ignore entries for an Ansible collection::
75
76    roles/my_role/files/my_script.sh shellcheck:SC2154 # ignore undefined variable
77    plugins/modules/my_module.py validate-modules:E105 # ignore license check
78    plugins/modules/my_module.py import-3.8 # needs update to support collections.abc on Python 3.8+
79
80It is also possible to skip a sanity test for a specific file.
81This is done by adding ``!skip`` after the sanity test name in the second column.
82When this is done, no error code is included, even if the sanity test uses error codes.
83
84Below are some example skip entries for an Ansible collection::
85
86    plugins/module_utils/my_util.py validate-modules!skip # waiting for bug fix in module validator
87    plugins/lookup/my_plugin.py compile-2.6!skip # Python 2.6 is not supported on the controller
88
89Ignore File Errors
90------------------
91
92There are various errors that can be reported for the ignore file itself:
93
94- syntax errors parsing the ignore file
95- references a file path that does not exist
96- references to a sanity test that does not exist
97- ignoring an error that does not occur
98- ignoring a file which is skipped
99- duplicate entries
100