Running the tests and understanding the results
-----------------------------------------------

To run all the tests just do
$ ./test.sh

For each test the output begins with the line:
making test: dir/file options...

dir is the dir where the test is made, file is the texinfo file processed and
the optional options are the options passed to texi2html when doing the test.
Then the test results are outputted. There are 5 type of results outputed:

status                               -> always
stderr line count                    -> always
diffs                                -> optionnal
test with tidy                       -> optionnal
test validity                        -> optionnal

For each of these, the type is outputted indented. The results appear on
lines below, more indented. When the test fails there is !!! on the line.
For example:

making test: formatting/not_closed_in_menu.texi
  status:
    passed
  stderr line count:
    !!! bad line count:       7 != 0

I now turn the the individual result type:

* status
The exit status of the command is tested for success or failure.

* stderr line count
The number of stderr lines is compared against a reference.

* diffs
out files are diffed against reference files present in dir_res. A '.'
means a successfull comparison. If there are more than one test in that
dir, there might be files differing or files not found if the files
haven't allready been regenerated.

* test with tidy
tidy is run on resulting html files.

* test validity
validate is run on resulting html files.


Adding a test
-------------

To add a test, you have to add a line in test.sh, near the end.
Here is the syntax:

test_texi dir file "option1 option2" stderr_lines suffix basename ignore_tags validate test_tidy fail

dir is the only mandatory argument. There are defaults for the other
arguments when not given. However if you want to specify, say, basename
and use the defaults for the preceding ones, you have to set them
explicitely.

dir: the directory where the files are.
file: the texinfo file. If not present dir.texi is used
"option1 option2": options passed to texi2html. Default is no option
stderr_lines: number of lines of stderr. Default is 0
suffix: suffix of the file, used to find the basename. 'texi' is assumed
basename: basename of the generated files. The basename of file is the default
ignore_tags: if set to 'ignore_tags' or 'yes' rcs tags are ignored
validate: if unset or set to 'validate' validation is tried
test_tidy: if unset or set to 'tidy' or 'test_tidy', tidy is tried
fail: if unset or set to 'success' the exit code should be 0

* Simple example:
test_texi macros

test macros.texi in the macros dir, without options, assuming generated
files beginning with macros.

* Complex example:
test_texi sectionning no_node.texi "-prefix chapter_split_no_node -split chapter" 0 texi chapter_split_no_node

Test no_node.texi in the sectionning dir, with the options:
-prefix chapter_split_no_node -split chapter
no lines on stderr, suffix is texi and basename for generated files is
chapter_split_no_node (which matches the option -prefix chapter_split_no_node).

Remarks:
The stderr output is in basename.2, the result of the first pass in
basename.passtexi, of the second pass in basename.passfirst.
the -test option is always passed.
To run a single test you can run test.sh with the arguments you would have
passed to test_texi. For example

$./test.sh viper viper.texi "-split chapter"

will run a test for viper.texi in the viper directory with option
-split chapter


Regenerating tests
------------------

make regenerate copy all the *.html, *.2, *.passtexi and *.passfirst from
the test directories to the result directories (test directory with _res
appended).
This is usefull when the new result is the good result and you want to
avoid manually copying the good files to the result directory.

To add a new test directory add it to the SUBDIR_TESTS variable in
Makefile.am (and rerun automake, autoconf...).