README
1
2This directory contains regression tests for make(1).
3
4To invoke the tests, please refer to tests(7).
5
6----------------------------------------------------------------------------
7
8The rest of this file is intended for developers.
9
10The tests are invoked via the test.sh script or prove(1) from p5-Test-Harness.
11Tests are normally executed in a special test directory that is built under
12/tmp. The reason for this is, that make tests are generally influenced by
13all file in a directory, by files in one of make's obscure object directories
14as well as in other directories make happens to look into. Therefor the
15test scripts build a clean environment in the temp directory and the
16tests are executed by cd-ing into that directory and invoking make. The
17output of the make run (standard output, standard error and the exit status)
18are written into files that are created in another directory. So the layout
19for the shell/builtin test looks like:
20
21 ./shell/builtin/ - directory with test stuff
22 /tmp/make.${USER}/shell/builtin - actual test directory
23 /tmp/make.${USER}/shell/builtin.OUTPUT - output files
24
25So a full test consists of the following steps:
26
27 setup - Set up the test environment by creating the test directory
28 and populating it with the needed files. If the test
29 directory already exists an error is printed.
30
31 run - Run the test and produce the output into the output
32 directory.
33
34 show - Show the result files on the screen.
35
36 compare - Compare the results in the output directory with those
37 in the test source directory. This just prints whether
38 the test was ok or not in the format used by prove(1).
39
40 diff - Diff the output files and the expected output files.
41
42 reset - Reset the test to its initial state.
43
44 clean - Remove both the test directory and the output directory.
45
46Each of these steps can independently be invoked with the test script
47contained in each directory. These test scripts are called test.t.
48Additionally the scripts understand the following commands:
49
50 test - Run setup, run and compare.
51
52 prove - Run setup, run, compare and clean. This is identically
53 to invoking the script without an argument.
54
55 desc - Print a short test description.
56
57 update - Update the expected results from the actual results.
58
59The test script has the following syntax:
60
61 % test.t [-v] [-m path_to_make_binary] command
62
63To invoke it via prove(1) use:
64
65 % [MAKE_PROG=path_to_make_binary] prove [options] [files/directories]
66
67Example:
68 % sh test.t -m `pwd`/../obj/make run
69 % MAKE_PROG=/usr/obj/usr/src/usr.bin/make/make prove -r
70
71The test scripts use the following environment variables that can be set
72by the user in the test script's environment:
73
74 WORK_BASE
75 - Base directory for working files. If not set
76 /tmp/make.${USER} is used.
77
78 MAKE_PROG
79 - Path to the make program to test. If not set
80 /usr/bin/make is used.
81
82The following variables are available to test scripts:
83
84 SRC_BASE
85 - test source base directory. This is determined by
86 repeatedly doing cd .. and checking for common.sh.
87 Therefor this can fail if a test source directory is
88 actually a symbolic link and is physically not located
89 below the directory containing common.sh.
90
91 SUBDIR
92 - subdirectory below WORK_BASE and SRC_BASE for current test
93
94 WORK_DIR
95 - ${WORK_BASE}/${SUBDIR}
96
97 SRC_DIR
98 - ${SRC_BASE}/${SUBDIR}
99
100The following variables and functions may be defined by the test script.
101This also lists special filenames.
102
103 DESC
104 A one-line description of the test.
105
106 TEST_MAKE_DIRS
107 A list of pairs of directory names and modes. These
108 directories are created during setup and reset. When
109 the directory already exists (during reset) only the
110 mode change is applied.
111
112 TEST_MAKE_DIRS="subdir 775 subdir/sub 555"
113
114 TEST_COPY_FILES
115 A list of pairs of file names and modes. These files
116 are copied from the source to the working directory
117 during setup and reset. When the file already exists
118 (during reset) only the mode change is applied. Files
119 may be copied from/to sub-directories. The sub-directory
120 in the working directory must already exists (see
121 TEST_MAKE_DIRS).
122
123 TEST_COPY_FILES="libtest.a 444 subdir/libfoo.a 444"
124
125 TEST_TOUCH
126 List of pairs of file names and arguments to touch(1).
127 During setup and reset for each list element touch(1)
128 is executed.
129
130 TEST_TOUCH="file1 '-t 200501011257'"
131
132 TEST_LINK
133 List of pairs of filenames. Each pair is passed to ln(1).
134 All names are prefixed with the working directory.
135
136 Makefile
137 If a file with this name exists in the source directory
138 it is automatically copied to the working directory.
139
140 setup_test()
141 If this function exists it is executed at the end of the
142 setup.
143
144 reset_test()
145 If this function exists it is executed at the end of the
146 reset.
147
148 TEST_CLEAN_FILES
149 A list of file to be deleted when resetting.
150
151 TEST_N
152 Number of tests in this script. If not set this is assumed
153 to be 1.
154
155 TEST_<number>
156 Arguments to make for test number <number>. If not set
157 the default argument of test<number> is used. To run a test
158 without argument to make, set TEST_<number> to the empty string.
159
160 TEST_<number>_SKIP
161 To skip a test (for whatever reason) this should be set
162 to a string explaining the reason for skipping the test.
163
164 TEST_<number>_TODO
165 For a test that should fail this is a short string describing
166 what the problem in make(1) is that should be fixed.
167
168 run_test()
169 Function to run a test. This function gets a single argument
170 which is the number of the test to executed. The default
171 function evaluates the variable TEST_<number> and calls
172 make with the arguments in this variable.
173
174