1When reviewing a test, make sure the test follows the 2[format][format] and [style][style] guidelines. 3 4In addition, the test should be checked for the following: 5 6## All tests 7<input type="checkbox"> 8The test passes when it's supposed to pass 9 10<input type="checkbox"> 11The test fails when it's supposed to fail 12 13<input type="checkbox"> 14The test is testing what it thinks it's testing 15 16<input type="checkbox"> 17The spec backs up the expected behavior in the test. 18 19<input type="checkbox"> 20The test is automated as either [reftest][reftest] or a 21[script test][scripttest] unless there's a very good reason why the 22test must be manual. 23 24<input type="checkbox"> 25The test does not use external resources. 26 27<input type="checkbox"> 28The test does not use proprietary features (vendor-prefixed or otherwise). 29 30 31## Reftests Only 32<input type="checkbox"> 33The test has a [self-describing][selftest] statement 34 35<input type="checkbox"> 36The self-describing statement is accurate, precise, simple, and 37self-explanatory. Your mother/husband/roommate/brother/bus driver 38should be able to say whether the test passed or failed within a few 39seconds, and not need to spend several minutes thinking or asking 40questions. 41 42<input type="checkbox"> 43The reference file is accurate and will render pixel-perfect 44identically to the test on all platforms. 45 46<input type="checkbox"> 47The reference file uses a different technique that won't fail in 48the same way as the test. 49 50<input type="checkbox"> 51The title is descriptive but not too wordy. 52 53<input type="checkbox"> 54The test is as cross-platform as reasonably possible, working 55across different devices, screen resolutions, paper sizes, etc. If 56there are limitations (e.g. the test will only work on 96dpi 57devices, or screens wider than 200 pixels) that these are documented 58in the instructions. 59 60 61## Script Tests Only 62 63<input type="checkbox"> 64The test uses the most specific asserts possible (e.g. doesn't use 65`assert_true` for everything). 66 67<input type="checkbox"> 68The number of tests in each file and the test names are consistent 69across runs and browsers. It is best to avoid the pattern where there is 70a test that asserts that the feature is supported and bails out without 71running the rest of the tests in the file if it isn't. 72 73<input type="checkbox"> 74The test avoids patterns that make it less likely to be stable. 75In particular, tests should avoid setting internal timeouts, since the 76time taken to run it may vary on different devices; events should be used 77instead (if at all possible). 78 79<input type="checkbox"> 80The test uses `idlharness.js` if it covers the use case. 81 82<input type="checkbox"> 83Tests in a single file are separated by one empty line. 84 85 86## In-depth Checklist 87 88<input type="checkbox"> 89A test does not use self-closing start tag ("/" (U+002F)) when using the 90HTML syntax. 91 92<input type="checkbox"> 93The test does not use the Unicode byte order mark (BOM U+FEFF). The test 94uses Unix line endings (LF, no CR). The executable bit is not set 95unnecessarily. 96 97<input type="checkbox"> 98For indentation, spaces are preferred over tabs. 99 100<input type="checkbox"> 101The test does not contain trailing whitespace (whitespace at the end of 102lines). 103 104<input type="checkbox"> 105The test does not contain commented-out code. 106 107<input type="checkbox"> 108The test does not use `console.*` methods for anything. The 109[script test][scripttest] harness never relies on `console.*` methods in 110any way, and so use of `console.*` methods in tests is usually just the 111equivalent of extra `printf`s in production code; i.e., leftover debugging 112that isn't actually useful to the next person running the test. It also 113introduces useless overhead when running tests in automation. 114 115<input type="checkbox"> 116The test is placed in the relevant directory, based on the /TR latest 117version link if available. 118 119<input type="checkbox"> 120If the test needs code running on the server side, the server code must 121be written in python, and the python code must be reviewed carefully to 122ensure it isn't doing anything dangerous. 123 124[format]: ./test-format-guidelines.html 125[style]: ./test-style-guidelines.html 126[reftest]: ./reftests.html 127[scripttest]: ./testharness-documentation.html 128[selftest]: ./test-style-guidelines.html#self-describing 129