1README -- Tcl test suite design document. 2 3Contents: 4--------- 5 6 1. Introduction 7 2. Running tests 8 3. Adding tests 9 4. Incompatibilities with prior Tcl versions 10 111. Introduction: 12---------------- 13 14This directory contains a set of validation tests for the Tcl commands 15and C Library procedures for Tcl. Each of the files whose name ends 16in ".test" is intended to fully exercise the functions in the C source 17file that corresponds to the file prefix. The C functions and/or Tcl 18commands tested by a given file are listed in the first line of the 19file. 20 212. Running tests: 22----------------- 23 24We recommend that you use the "test" target of Tcl's Makefile to run 25the test suite. From the directory in which you build Tcl, simply 26type "make test". This will create a special executable named 27tcltest in which the testing scripts will be evaluated. To create 28the tcltest executable without running the test suite, simple type 29"make tcltest". 30 31All the configuration options of the tcltest package are available 32during a "make test" by defining the TESTFLAGS environment variable. 33For example,if you wish to run only those tests in the file append.test, 34you can type: 35 36 make test TESTFLAGS="-file append.test" 37 38For interactive testing, the Tcl Makefile provides the "runtest" target. 39Type "make runtest" in your build directory, and the tcltest executable 40will be created, if necessary, then it will run interactively. At the 41command prompt, you may type any Tcl commands. If you type 42"source ../tests/all.tcl", the test suite will run. You may use the 43tcltest::configure command to configure the test suite run as an 44alternative to command line options via TESTFLAGS. You might also 45wish to use the tcltest::testConstraint command to select the constraints 46that govern which tests are run. See the documentation for the tcltest 47package for details. 48 493. Adding tests: 50---------------- 51 52Please see the tcltest man page for more information regarding how to 53write and run tests. 54 55Please note that the all.tcl file will source your new test file if 56the filename matches the tests/*.test pattern (as it should). The 57names of test files that contain regression (or glass-box) tests 58should correspond to the Tcl or C code file that they are testing. 59For example, the test file for the C file "tclCmdAH.c" is 60"cmdAH.test". Test files that contain black-box tests may not 61correspond to any Tcl or C code file so they should match the pattern 62"*_bb.test". 63 64Be sure your new test file can be run from any working directory. 65 66Be sure no temporary files are left behind by your test file. 67Use [tcltest::makeFile], [tcltest::removeFile], and [tcltest::cleanupTests] 68properly to be sure of this. 69 70Be sure your tests can run cross-platform in both a build environment 71as well as an installation environment. If your test file contains 72tests that should not be run in one or more of those cases, please use 73the constraints mechanism to skip those tests. 74 754. Incompatibilities of package tcltest 2.1 with 76 testing machinery of very old versions of Tcl: 77------------------------------------------------ 78 791) Global variables such as VERBOSE, TESTS, and testConfig of the 80 old machinery correspond to the [configure -verbose], 81 [configure -match], and [testConstraint] commands of tcltest 2.1, 82 respectively. 83 842) VERBOSE values were longer numeric. [configure -verbose] values 85 are lists of keywords. 86 873) When you run "make test", the working dir for the test suite is now 88 the one from which you called "make test", rather than the "tests" 89 directory. This change allows for both unix and windows test 90 suites to be run simultaneously without interference with each 91 other or with existing files. All tests must now run independently 92 of their working directory. 93 944) The "all" file is now called "all.tcl" 95 965) The "defs" and "defs.tcl" files no longer exist. 97 986) Instead of creating a doAllTests file in the tests directory, to 99 run all nonPortable tests, just use the "-constraints nonPortable" 100 command line flag. If you are running interactively, you can run 101 [tcltest::testConstraint nonPortable 1] (after loading the tcltest 102 package). 103 1047) Direct evaluation of the *.test files by the "source" command is no 105 longer recommended. Instead, "source all.tcl" and use the "-file" and 106 "-notfile" options of tcltest::configure to control which *.test files 107 are evaluated. 108