1A reftest is a test that compares the visual output of one file (the 2test case) with the output of one or more other files (the 3references). The test and the reference must be carefully written so 4that when the test passes they have identical rendering, but different 5rendering when the test fails. 6 7## How to Run Reftests 8 9Reftests can be run manually simply by opening the test and the 10reference file in multiple windows or tabs and either placing them 11side-by side or flipping between the two. In automation the comparison 12is done in an automated fashion, which can lead to differences too 13small for the human eye to notice causing tests to fail. 14 15## Components of a Reftest 16 17In the simplest case, a reftest consists of a pair of files called the 18*test* and the *reference*. 19 20The *test* file is the one that makes use of the technology being 21tested. It also contains a `link` element with `rel="match"` or 22`rel="mismatch"` and `href` attribute pointing to the *reference* file 23e.g. `<link rel=match href=references/green-box-ref.html>`. 24 25The *reference* file is typically written to be as simple as possible, 26and does not use the technology under test. It is desirable that the 27reference be rendered correctly even in UAs with relatively poor 28support for CSS and no support for the technology under test. 29 30When the `<link>` element in the *test* has `rel="match"`, the test 31only passes if the *test* and *reference* have pixel-perfect identical 32rendering. `rel="mismatch"` inverts this so the test only passes when 33the renderings differ. 34 35In general the files used in a reftest should follow the 36[format][format] and [style][style] guidelines. The *test* should also 37be [self-describing][selfdesc], to allow a human to determine whether 38the the rendering is as expected. 39 40Note that references can be shared between tests; this is strongly 41encouraged since it permits optimizations when running tests. 42 43## Controlling When Comparison Occurs 44 45By default reftest screenshots are taken in response to the `load` 46event firing. In some cases it is necessary to delay the screenshot 47later than this, for example becase some DOM manipulation is 48required to set up the desired test conditions. To enable this, the 49test may have a `class="reftest-wait"` attribute specified on the root 50element. This will cause the screenshot to be delayed until the `load` 51event has fired and the `reftest-wait` class has been removed from the 52root element (technical note: the implementation in wptrunner uses 53mutation observers so the screenshot will be triggered in the 54microtask checkpoint after the class is removed. Because the harness 55isn't synchronized with the browser event loop it is dangerous to rely 56on precise timing here). 57 58## Matching Multiple References 59 60Sometimes it is desirable for a file to match multiple references or, 61in rare cases, to allow it to match more than one possible 62reference. Note: *this is not currently supported by test runners and 63so best avoided if possible until that support improves*. 64 65Multiple references linked from a single file are interpreted as 66multiple possible renderings for that file. `<link rel=[mis]match>` 67elements in a reference create further conditions that must be met in 68order for the test to pass. For example, consider a situation where 69`a.html` has `<link rel=match href=b.html>` and `<link rel=match 70href=c.html>`, `b.html` has `<link rel=match href=b1.html>` and `c.html` 71has `<link rel=mismatch href=c1.html>`. In this case, to pass we must 72either have `a.html`, `b.html` and `b1.html` all rendering identically, or 73`a.html` and `c.html` rendering identically, but `c.html` rendering 74differently from `c1.html`. 75 76## Fuzzy Matching 77 78In some situations a test may have subtle differences in rendering 79compared to the reference due to e.g. antialiasing. This may cause the 80test to pass on some platforms but fail on others. In this case some 81affordance for subtle discrepancies is desirable. However no mechanism 82to allow this has yet been standardized. 83 84## Limitations 85 86In some cases, a test cannot be a reftest. For example, there is no 87way to create a reference for underlining, since the position and 88thickness of the underline depends on the UA, the font, and/or the 89platform. However, once it's established that underlining an inline 90element works, it's possible to construct a reftest for underlining 91a block element, by constructing a reference using underlines on a 92```<span>``` that wraps all the content inside the block. 93 94## Example Reftests 95 96These examples are all [self-describing][selfdesc] tests as they 97each have a simple statement on the page describing how it should 98render to pass the tests. 99 100### HTML example 101 102### Test File 103 104This test verifies that a right-to-left rendering of **SAW** within a 105```<bdo>``` element displays as **WAS**. 106 107([view page rendering][html-reftest-example]) 108 109```html 110<!DOCTYPE html> 111<meta charset="utf-8"> 112<title>BDO element dir=rtl</title> 113<link rel="help" href="http://www.whatwg.org/specs/web-apps/current-work/#the-bdo-element"> 114<meta name="assert" content="BDO element's DIR content attribute renders corrently given value of 'rtl'."> 115<link rel="match" href="test-bdo-001.html"> 116<p>Pass if you see WAS displayed below.</p> 117<bdo dir="rtl">SAW</bdo> 118``` 119 120### Reference File 121 122The reference file must look exactly like the test file, 123except that the code behind it is different. 124 125* All metadata is removed. 126* The ```title``` need not match. 127* The markup that created the actual test data is 128 different: here, the same effect is created with 129 very mundane, dependable technology. 130 131([view page rendering][html-reffile-example]) 132 133```html 134<!DOCTYPE html> 135<meta charset="utf-8"> 136<title>HTML Reference File</title> 137<p>Pass if you see WAS displayed below.</p> 138<p>WAS</p> 139``` 140 141[testharness]: ./testharness-documentation.html 142[format]: ./test-format-guidelines.html 143[style]: ./test-style-guidelines.html 144[selfdesc]: ./test-style-guidelines.html#self-describing-tests 145[reference-links]: ./test-templates.html#reference-links 146[html-reftest-example]: ./html-reftest-example.html 147[html-reffile-example]: ./html-reffile-example.html 148[css-reftest-example]: http://test.csswg.org/source/css21/borders/border-bottom-applies-to-009.xht 149[css-reffile-example]: http://test.csswg.org/source/css21/borders/border-bottom-applies-to-001-ref.xht 150[svg-reftest-example]: http://test.csswg.org/source/css-transforms-1/translate/svg-translate-001.html 151[svg-reffile-example]: http://test.csswg.org/source/css-transforms-1/translate/reference/svg-translate-ref.html 152[indicating-failure]: ./test-style-guidelines.html#failure 153