1Running the tests and understanding the results 2----------------------------------------------- 3 4To run all the tests just do 5$ ./test.sh 6 7For each test the output begins with the line: 8making test: dir/file options... 9 10dir is the dir where the test is made, file is the texinfo file processed and 11the optional options are the options passed to texi2html when doing the test. 12Then the test results are outputted. There are 5 type of results outputed: 13 14status -> always 15stderr line count -> always 16diffs -> optionnal 17test with tidy -> optionnal 18test validity -> optionnal 19 20For each of these, the type is outputted indented. The results appear on 21lines below, more indented. When the test fails there is !!! on the line. 22For example: 23 24making test: formatting/not_closed_in_menu.texi 25 status: 26 passed 27 stderr line count: 28 !!! bad line count: 7 != 0 29 30I now turn the the individual result type: 31 32* status 33The exit status of the command is tested for success or failure. 34 35* stderr line count 36The number of stderr lines is compared against a reference. 37 38* diffs 39out files are diffed against reference files present in dir_res. A '.' 40means a successfull comparison. If there are more than one test in that 41dir, there might be files differing or files not found if the files 42haven't allready been regenerated. 43 44* test with tidy 45tidy is run on resulting html files. 46 47* test validity 48validate is run on resulting html files. 49 50 51Adding a test 52------------- 53 54To add a test, you have to add a line in test.sh, near the end. 55Here is the syntax: 56 57test_texi dir file "option1 option2" stderr_lines suffix basename ignore_tags validate test_tidy fail 58 59dir is the only mandatory argument. There are defaults for the other 60arguments when not given. However if you want to specify, say, basename 61and use the defaults for the preceding ones, you have to set them 62explicitely. 63 64dir: the directory where the files are. 65file: the texinfo file. If not present dir.texi is used 66"option1 option2": options passed to texi2html. Default is no option 67stderr_lines: number of lines of stderr. Default is 0 68suffix: suffix of the file, used to find the basename. 'texi' is assumed 69basename: basename of the generated files. The basename of file is the default 70ignore_tags: if set to 'ignore_tags' or 'yes' rcs tags are ignored 71validate: if unset or set to 'validate' validation is tried 72test_tidy: if unset or set to 'tidy' or 'test_tidy', tidy is tried 73fail: if unset or set to 'success' the exit code should be 0 74 75* Simple example: 76test_texi macros 77 78test macros.texi in the macros dir, without options, assuming generated 79files beginning with macros. 80 81* Complex example: 82test_texi sectionning no_node.texi "-prefix chapter_split_no_node -split chapter" 0 texi chapter_split_no_node 83 84Test no_node.texi in the sectionning dir, with the options: 85-prefix chapter_split_no_node -split chapter 86no lines on stderr, suffix is texi and basename for generated files is 87chapter_split_no_node (which matches the option -prefix chapter_split_no_node). 88 89Remarks: 90The stderr output is in basename.2, the result of the first pass in 91basename.passtexi, of the second pass in basename.passfirst. 92the -test option is always passed. 93To run a single test you can run test.sh with the arguments you would have 94passed to test_texi. For example 95 96$./test.sh viper viper.texi "-split chapter" 97 98will run a test for viper.texi in the viper directory with option 99-split chapter 100 101 102Regenerating tests 103------------------ 104 105make regenerate copy all the *.html, *.2, *.passtexi and *.passfirst from 106the test directories to the result directories (test directory with _res 107appended). 108This is usefull when the new result is the good result and you want to 109avoid manually copying the good files to the result directory. 110 111To add a new test directory add it to the SUBDIR_TESTS variable in 112Makefile.am (and rerun automake, autoconf...). 113