1Background: 2 3This is to perform regression testing, which is to say, make sure the 4software works the same way that it used to. This is not to test new 5features (see "modify an existing tree" for that). 6 7When code has been changed for a certain program that is involved in a 8regression testing tree, go to that tree, run the test, and make sure 9that the results have not changed (in any bad way). This is meant to 10be fast and easy. 11 12The test consists of command scripts that were written before. The 13scripts are executed, and the output is compared with the output of 14an older execution. The should ususally be identical. 15 16--------------------------------------------------------------------------- 17 18To perform regression testing, one needs the data (via 'make regress_data'). 19Given that, only the @test and @show_diffs are needed (sample commands are 20shown with a preceeding prompt, '> '). 21 22 1. Run the regression testing script. It goes through the steps: 23 24 a. Create a "results" directory, including a timestamp. 25 b. Copy all "data" files into it. 26 c. 'cd' into the "results" directory. 27 d. For each command script in the commands directory, execute it, 28 storing all output text into a file called e."command name". 29 30 > ./@test 31 32 33 2. To compare against an older execution, run the @show_diffs script. 34 It goes through the steps: 35 36 a. Make a list of the "results" directories. 37 b. Create a new "diffs" directory. 38 c. Compare files between the newest and oldest "results" dirs: 39 - cd into newest "results" directory 40 - foreach file (*) 41 diff $file ../other.result/$file >& ../diffs/d.$file 42 43 > ./@show_diffs 44 45 46 3. Look at the sizes of the diff files, or the contents (only 47 expected differences should exist). 48 49 > ls -l diffs 50 > cat diffs/* 51