1This file contains a list of notes for those writing testcases and those
2writing expect scripts.  At present, they're in random order.
3
4Verbosity Levels
5
6- each level adds verbosity so level 2 prints all level 1 and level 2 stuff.
7
81) Print a one-liner indicating the testcase being run (and maybe special
9   compiler options).
10
112) Print compiler and program invocation including arguments and their output.
12   Proc's gcc_load and gcc_start handle the latter two.
13
143) Print detailed testcase analysis like "Looking for pattern ...", etc.
15
164) Maximum verbosity.  Print anything else of interest.
17
18send_log conventions
19
20Various messages are stored in gcc.log by the testing framework and we
21try to augment them with some of our own.  The format of the framework
22messages are:
23
24PASS: blah blah ...
25FAIL: blah blah ...
26
27so we use
28
29XXXX: blah blah ...
30
31Current messages are:
32
33EXEC: program being executed (so compiler path and args are recorded)
34STAT: intermediate pass/fail statistics
35
36Test scripts must ignore the compiler messages "path prefix never used"
37and "linker input file unused".  Don't let their appearance cause a testcase
38to fail.  See lib/dg.exp for the exact regsub to use.
39
40If you're unclear about which directory a testcase should be installed in,
41ask gcc-local.
42
43Have the text of a fail message be the same as that for pass.
44IE: have
45
46        if ...success...
47                pass "pr 1234"
48        else
49                fail "pr 1234"
50
51not
52
53        if ...success...
54                pass "pr 1234 passed"
55        else
56                fail "pr 1234 failed"
57
58
59This lets test-tool (which drives the nightly tests) do a better job
60at tracking which tests have digressed or been fixed.
61
62DO NOT PUT NON-PORTABLE TESTCASES IN gcc.c-torture.
63
64ANY TARGET SPECIFIC TESTCASE MUST HAVE APPROPRIATE CODE TO PREVENT IT FROM
65CAUSING A `FAILURE' ON UNSUPPORTED PLATFORMS.
66
67The "torture" tests are meant to be generic tests that can run on any
68target.  So you have to be careful about endianness, assumptions about
69sizes of datatypes, etc etc.
70
71For tests that merely need to compile, put them in the "compile" directory.
72
73For tests which should give an error, put them in the "noncompile" directory
74and update noncompile.exp appropriately (see examples in noncompile.exp).
75
76For IEEE FP specific tests, put them in execute/ieee.
77
78For execution tests, put them in execute.
79
80If a test does not fit into the torture framework, use the dg framework.
81
82
83Copyright (C) 1997-2019 Free Software Foundation, Inc.
84
85Copying and distribution of this file, with or without modification,
86are permitted in any medium without royalty provided the copyright
87notice and this notice are preserved.
88