1How to add recipes 2================== 3 4For any test that you want to perform, you write a script located in 5test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and 6{name} is a unique name of your choice. 7 8Please note that if a test involves a new testing executable, you will need to 9do some additions in test/Makefile. More on this later. 10 11 12Naming conventions 13================= 14 15A test executable is named test/{name}test.c 16 17A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two 18digit number and {name} is a unique name of your choice. 19 20The number {nn} is (somewhat loosely) grouped as follows: 21 2200-04 sanity, internal and essential API tests 2305-09 individual symmetric cipher algorithms 2410-14 math (bignum) 2515-19 individual asymmetric cipher algorithms 2620-24 openssl commands (some otherwise not tested) 2725-29 certificate forms, generation and verification 2830-35 engine and evp 2960-79 APIs 30 70 PACKET layer 3180-89 "larger" protocols (CA, CMS, OCSP, SSL, TSA) 3290-98 misc 3399 most time consuming tests [such as test_fuzz] 34 35 36A recipe that just runs a test executable 37========================================= 38 39A script that just runs a program looks like this: 40 41 #! /usr/bin/perl 42 43 use OpenSSL::Test::Simple; 44 45 simple_test("test_{name}", "{name}test", "{name}"); 46 47{name} is the unique name you have chosen for your test. 48 49The second argument to `simple_test' is the test executable, and `simple_test' 50expects it to be located in test/ 51 52For documentation on OpenSSL::Test::Simple, do 53`perldoc util/perl/OpenSSL/Test/Simple.pm'. 54 55 56A recipe that runs a more complex test 57====================================== 58 59For more complex tests, you will need to read up on Test::More and 60OpenSSL::Test. Test::More is normally preinstalled, do `man Test::More' for 61documentation. For OpenSSL::Test, do `perldoc util/perl/OpenSSL/Test.pm'. 62 63A script to start from could be this: 64 65 #! /usr/bin/perl 66 67 use strict; 68 use warnings; 69 use OpenSSL::Test; 70 71 setup("test_{name}"); 72 73 plan tests => 2; # The number of tests being performed 74 75 ok(test1, "test1"); 76 ok(test2, "test1"); 77 78 sub test1 79 { 80 # test feature 1 81 } 82 83 sub test2 84 { 85 # test feature 2 86 } 87 88 89Changes to test/build.info 90========================== 91 92Whenever a new test involves a new test executable you need to do the 93following (at all times, replace {NAME} and {name} with the name of your 94test): 95 96* add {name} to the list of programs under PROGRAMS_NO_INST 97 98* create a three line description of how to build the test, you will have 99to modify the include paths and source files if you don't want to use the 100basic test framework: 101 102 SOURCE[{name}]={name}.c 103 INCLUDE[{name}]=.. ../include 104 DEPEND[{name}]=../libcrypto libtestutil.a 105 106Generic form of C test executables 107================================== 108 109 #include "testutil.h" 110 111 static int my_test(void) 112 { 113 int testresult = 0; /* Assume the test will fail */ 114 int observed; 115 116 observed = function(); /* Call the code under test */ 117 if (!TEST_int_eq(observed, 2)) /* Check the result is correct */ 118 goto end; /* Exit on failure - optional */ 119 120 testresult = 1; /* Mark the test case a success */ 121 end: 122 cleanup(); /* Any cleanup you require */ 123 return testresult; 124 } 125 126 int setup_tests(void) 127 { 128 ADD_TEST(my_test); /* Add each test separately */ 129 return 1; /* Indicate success */ 130 } 131 132You should use the TEST_xxx macros provided by testutil.h to test all failure 133conditions. These macros produce an error message in a standard format if the 134condition is not met (and nothing if the condition is met). Additional 135information can be presented with the TEST_info macro that takes a printf 136format string and arguments. TEST_error is useful for complicated conditions, 137it also takes a printf format string and argument. In all cases the TEST_xxx 138macros are guaranteed to evaluate their arguments exactly once. This means 139that expressions with side effects are allowed as parameters. Thus, 140 141 if (!TEST_ptr(ptr = OPENSSL_malloc(..))) 142 143works fine and can be used in place of: 144 145 ptr = OPENSSL_malloc(..); 146 if (!TEST_ptr(ptr)) 147 148The former produces a more meaningful message on failure than the latter. 149 150