README
1#
2# @(#)README 1.2 16/06/05 Copyright 2016 J. Schilling
3#
4
5 This is the test suite for the Schily Bourne Shell
6
7The directory "common" contains the linkage to the test suite, that calls the
8test framework in "$SRCROOT/tests", the directory "shelltests" contains the
9shell test suite itself.
10
11To run the tests, simply call "make" to test the just compiled Schily Bourne
12Shell at "$SRCROOT/sh/OBJ/<arch-dir>/sh". If a single test fails and is not
13expected to fail, all further tests are stopped. Tests that are expected to
14fail are e.g. tests that check whether it works to retrieve all 32 bits from
15the exit code of a process. Such a POSIX-specific test is expected to fail
16on non-POSIX compliant platforms like Linux and cause the test suite to
17continue with other tests.
18
19Failed tests leave debugging files in the subdirectory where the test did
20take place, e.g. "$SRCROOT/sh/tests/shelltests/exit". If the output of the
21test suite contains the word "FAIL", an unexpected failure was seen. If the
22output of the test suite contains the word "XFAIL", an expected failure was
23seen.
24
25To run all tests even in case of a single unexpected failure, call
26"EXPECT_FAIL=true make".
27
28To run the tests with other shells, you need to specify the absolute path
29to the shell in question. Call e.g.: "shell=/usr/bin/ksh93 make" or
30"EXPECT_FAIL=true shell=/usr/bin/ksh93 make". With other shells, more
31tests are defined to be tests that are expected to fail. ksh93 e.g. does
32not handle octal numbers in arithmetic expressions the way POSIX requires.
33
34Note that the test suite contains tests for "bosh" specific features.
35
36Many tests have been taken from the "mksh" test suite (see notes in the
37related test scripts) and some of these tests are currently disabled as
38they fail with "bosh" but it has not yet been verified whether this is
39an expected failure or whether "bosh" fails because of a non-compliance.
40
41The following tests are currently disabled:
42
43$SRCROOT/sh/tests/shelltests/here/here02.sh:
44
45here07: POSIX introduced a non-compliance we do not yet support
46
47$SRCROOT/sh/tests/shelltests/glob/glob.sh:
48
49glob05: The results depend on whether we use gmatch() from
50 AT&Ts libgen or our own implementation from gmatch.c
51
52$SRCROOT/sh/tests/shelltests/parameter/parameter02.sh:
53
54parameter-2-00:#4
55parameter-2-00:#6
56parameter-2-00:#7
57parameter-2-00:#9
58parameter-2-00:#14
59parameter-2-00:#15
60parameter-2-00:#25
61parameter-2-00:#28
62parameter-2-00:#29
63parameter-2-00:#30
64parameter-2-00:#31
65parameter-2-00:#32
66parameter-2-00:#35
67parameter-2-00:#39
68
69parameter-2-01:#7
70parameter-2-01:#8
71parameter-2-01:#9
72
73parameter-2-08
74