1README -- Tcl test suite design document.
2
3Contents:
4---------
5
6    1. Introduction
7    2. Running tests
8    3. Adding tests
9    4. Incompatibilities with prior Tcl versions
10
111. Introduction:
12----------------
13
14This directory contains a set of validation tests for the Tcl commands
15and C Library procedures for Tcl.  Each of the files whose name ends
16in ".test" is intended to fully exercise the functions in the C source
17file that corresponds to the file prefix.  The C functions and/or Tcl
18commands tested by a given file are listed in the first line of the
19file.
20
212. Running tests:
22-----------------
23
24We recommend that you use the "test" target of Tcl's Makefile to run
25the test suite.  From the directory in which you build Tcl, simply
26type "make test".  This will create a special executable named
27tcltest in which the testing scripts will be evaluated.  To create
28the tcltest executable without running the test suite, simple type
29"make tcltest".
30
31All the configuration options of the tcltest package are available
32during a "make test" by defining the TESTFLAGS environment variable.
33For example,if you wish to run only those tests in the file append.test,
34you can type:
35
36	make test TESTFLAGS="-file append.test"
37
38For interactive testing, the Tcl Makefile provides the "runtest" target.
39Type "make runtest" in your build directory, and the tcltest executable
40will be created, if necessary, then it will run interactively.  At the
41command prompt, you may type any Tcl commands.  If you type
42"source ../tests/all.tcl", the test suite will run.  You may use the
43tcltest::configure command to configure the test suite run as an
44alternative to command line options via TESTFLAGS.  You might also
45wish to use the tcltest::testConstraint command to select the constraints
46that govern which tests are run.  See the documentation for the tcltest
47package for details.
48
493. Adding tests:
50----------------
51
52Please see the tcltest man page for more information regarding how to
53write and run tests.
54
55Please note that the all.tcl file will source your new test file if
56the filename matches the tests/*.test pattern (as it should).  The
57names of test files that contain regression (or glass-box) tests
58should correspond to the Tcl or C code file that they are testing.
59For example, the test file for the C file "tclCmdAH.c" is
60"cmdAH.test".  Test files that contain black-box tests may not
61correspond to any Tcl or C code file so they should match the pattern
62"*_bb.test".
63
64Be sure your new test file can be run from any working directory.
65
66Be sure no temporary files are left behind by your test file.
67Use [tcltest::makeFile], [tcltest::removeFile], and [tcltest::cleanupTests]
68properly to be sure of this.
69
70Be sure your tests can run cross-platform in both a build environment
71as well as an installation environment.  If your test file contains
72tests that should not be run in one or more of those cases, please use
73the constraints mechanism to skip those tests.
74
754. Incompatibilities of package tcltest 2.1 with
76   testing machinery of very old versions of Tcl:
77------------------------------------------------
78
791) Global variables such as VERBOSE, TESTS, and testConfig of the
80   old machinery correspond to the [configure -verbose],
81   [configure -match], and [testConstraint] commands of tcltest 2.1,
82   respectively.
83
842) VERBOSE values were longer numeric.  [configure -verbose] values
85   are lists of keywords.
86
873) When you run "make test", the working dir for the test suite is now
88   the one from which you called "make test", rather than the "tests"
89   directory.  This change allows for both unix and windows test
90   suites to be run simultaneously without interference with each
91   other or with existing files.  All tests must now run independently
92   of their working directory.
93
944) The "all" file is now called "all.tcl"
95
965) The "defs" and "defs.tcl" files no longer exist.
97
986) Instead of creating a doAllTests file in the tests directory, to
99   run all nonPortable tests, just use the "-constraints nonPortable"
100   command line flag.  If you are running interactively, you can run
101   [tcltest::testConstraint nonPortable 1] (after loading the tcltest
102   package).
103
1047) Direct evaluation of the *.test files by the "source" command is no
105   longer recommended.  Instead, "source all.tcl" and use the "-file" and
106   "-notfile" options of tcltest::configure to control which *.test files
107   are evaluated.
108