1FORM Test Suite
2===============
3
4This directory contains a collection of test cases that can be used for
5verifying the behaviour of FORM. It also has a script to run the test cases and
6check the results.
7
8Prerequisites
9-------------
10
11The test runner script is written in [Ruby](https://www.ruby-lang.org/)
12and requires Ruby 1.8 or later. The script uses the so-called `test/unit`
13library. In some Linux distributions the library is installed together with
14Ruby, while some distributions may have the library as an optional package,
15or one may need to manually install
16[test-unit](http://test-unit.github.io/test-unit/en/) via the `gem` command.
17Currently, the script runs only on Unix-like systems.
18
19Usage
20-----
21
22### From the build system
23
24To use the test suite from the automatic build system
25(see also the [INSTALL](../INSTALL) file),
26run
27
28```
29# in the root build directory
30make check
31```
32
33which tests the executables (release versions) compiled by the build system.
34
35### Testing in the standalone mode
36
37Alternatively, one can run the test runner script directly:
38
39```
40# in the "check" directory
41./check.rb
42```
43
44By default, it tests `form` found in $PATH.
45To check another executable, give the path as a command line option:
46
47```
48./check.rb /path/to/form
49```
50
51One can specify a TFORM (or ParFORM) executable in this way.
52TFORM and ParFORM will be run with 4 CPUs (can be changed by the `--cpu N`
53option).
54
55By default, all test cases in all FORM files (`*.frm`) found in the `check`
56directory (not in subdirectories) are used. To select test cases or FORM files
57to be run, give their names as command line options, for example,
58
59```
60./check.rb examples.frm
61./check.rb Issue8
62```
63
64For more advanced options, see the help message shown by the `--help` option.
65
66Writing tests
67-------------
68
69### Where to add test cases?
70
71Currently, the standard test set (run by default) consists of 3 files:
72
73- `examples.frm`: Examples found in the manual.
74- `features.frm`: Test cases for newly added features.
75- `fixes.frm`: Test cases for bug fixes.
76
77Each test case in these files should finish in a short time: the timeout is set
78to 10 seconds. Bigger tests that take more time are put in subdirectories
79(e.g., forcer) and should be specified by command-line options when the test
80suite is invoked.
81
82### Structure of a test case
83
84A test case is given as a fold in a FORM file. A simple example is:
85
86```
87*--#[ Test1 :
88S x;
89L F = (1+x)^2;
90P;
91.end
92assert succeeded?
93assert result("F") =~ expr("1 + 2*x + x^2")
94*--#] Test1 :
95```
96
97The fold name `Test1` gives the name of the test case, which should be unique.
98The part before `.end` is a normal FORM program. After `.end`, one can write
99a Ruby program to check the results. In this example, `assert` method (which is
100provided by some unit test class) is used for checking whether its argument is
101`true`. The first assertion checks `succeeded?`, which gives `true` if the FORM
102successfully finishes. The second assertion checks the printed result of the
103expression `F` by a regular expression matching (`=~`). In the left-hand side,
104`result("F")` returns the (lastly) printed output for the expression `F` as
105a string. In the right-hand side, `expr("...")` makes a regular expression with
106removing white spaces in its argument. Since `expr()` removes all white spaces,
107one can also put new lines, for example,
108
109```
110*--#[ Test2 :
111S x;
112L F = (1+x)^2;
113P +s;
114.end
115assert succeeded?
116assert result("F") =~ expr("
117       + 1
118       + 2*x
119       + x^2
120")
121*--#] Test2 :
122```
123
124which is convenient to copy and paste a long output from a terminal.
125
126### Tips
127
128- To verify that FORM finishes with a certain error, one can use
129  `assert compile_error?` or `assert runtime_error?`.
130- Two or more FORM programs, separated by `.end`, can be put in a test case.
131  Then the part after the last `.end` is for Ruby.
132- To skip a test case for some condition, one can specify it by `#pend_if`.
133  (See the result of grepping `pend_if` in the existing files.)
134- When a test case requires other text files, one can use `#prepare write`.
135  (See the result of grepping `prepare` in the existing files.)
136