1[/ 2 / Copyright (c) 2003 Boost.Test team 3 / 4 / Distributed under the Boost Software License, Version 1.0. (See accompanying 5 / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) 6 /] 7 8 9[section:test_output Controlling outputs] 10The output produced by a test module is one of the major assets the __UTF__ brings to users. In comparison with any 11kind of manual/assert based solution the __UTF__ provide following services: 12 13[/TOFIX into proper list] 14[itemized_list 15[All test errors are reported uniformly[br] 16 The test execution monitor along with standardized output from all included 17 [link boost_test.testing_tools testing tools] provides uniform reporting for all errors 18 including fatal errors, like memory assess violation and uncaught exceptions.] 19[Detailed information on the source of an error[br] 20 The __UTF__ test tool's based assertion provides as much information as possible about cause of error, 21 usually allowing you to deduce what is wrong without entering the debugger or core analysis.] 22[Separation of the test errors description (test log) from the results report summary (test results report) 23 The information produced during test execution, including all error, warning and info messages from the test 24 tools, executed test units notification constitute the test log. By default all entries in the test log are 25 directed to the standard output. Once testing is completed the __UTF__ may produce a summary test report with 26 different levels of detail. The test report is by default directed to the standard error output.] 27[Flexibility in what is shown in the output[br] 28The __UTF__ provides the ability to configure what is shown in both the test log and the test report. The 29 configuration is supported both at runtime, during test module invocation and at compile time from within a 30 test module.] 31[Flexibility in how output is formatted[br] 32The __UTF__ provides the ability to configure the format of the test module output. At the moment only 2 formats 33 are supported by the __UTF__ itself, the well defined public interface allows you to customize an output for 34 your purposes almost any way you want.] 35] 36 37[tip More details about the class implementing the formatting of the logs can be found in the following reference sections. 38 39* [classref boost::unit_test::unit_test_log_formatter] defines the interface for all loggers. Built-in HRF and XML 40 interfaces derive from this class 41* [classref boost::unit_test::test_results] defines the information carried by the tests to provide reports and logs. 42] 43 44[section Test log output] 45 46The test log is produced during the test execution. All entries in the test log are assigned a particular log 47level. Only the entries with level that exceeds the ['active log level threshold] actually 48appear in the test log output. Log levels are arranged by the 'importance' of the log entries. Here is 49the list of all levels in order of increasing 'importance': 50 51 52[table:id_messages Messages 53 [ 54 [Notifications] 55 [Meaning] 56 ] 57 [ 58 [Success] 59 [This category includes messages that provide information on successfully passed assertions] 60 ] 61 [ 62 [Test tree traversal] 63 [This category includes messages that are produced by the __UTF__ core and indicate which test suites/cases are currently being executed or skipped] 64 ] 65 [ 66 [General information] 67 [This category includes general information messages produced in most cases by a test module author using the 68 macro __BOOST_TEST_MESSAGE__] 69 ] 70 [ 71 [Warning] 72 [This category includes messages produced by failed warning level assertions] 73 ] 74 [ 75 [Non fatal error] 76 [This category includes messages produced by failed check level assertions] 77 ] 78 79 [ 80 [Uncaught C++ exceptions] 81 [This category includes messages that are produced by the __UTF__ and provide detailed information on the C++ 82 exceptions uncaught by the test case body. 83 ] 84 ] 85 [ 86 [Non-fatal system error] 87 [This category includes messages that are produced by the __UTF__ itself and provides information about caught 88 non-fatal system error. For example it includes messages produced in the case of test case timeout or if 89 floating point values calculation errors are caught. 90 ] 91 ] 92 [ 93 [Fatal system error] 94 [This category includes messages produced by failed require level assertions and by the __UTF__ itself in case of 95 abnormal test case termination.] 96 ] 97] 98 99[note 100 The active log level works namely as threshold, not as selector. For the given active log level threshold, all 101 test log entries with ['importance] higher than threshold are enabled and all test log entries with 102 ['importance] below threshold are disabled. 103] 104 105In addition to the levels described above the test log defines two special log levels. The current log level can 106be set to: 107 108* All messages[br] 109 If active log level threshold is set to this value, all test log entries appear in the output. In practice 110 this is equivalent to setting the active log level threshold to ['success information messages] 111* Nothing[br] 112 If the active log level threshold is set to this value, none of test log entries appear in the output. This log level 113 is used to execute a ['silent] test that doesn't produce any test log and only generates a result code indicating 114 whether test failed or passed. 115 116 117By default the active log level threshold is set to "non fatal error messages" and the test log output 118is generated in the human readable format. The active log level threshold and the output format can be configured 119at runtime during a test module invocation and at compile time from within a test module using the test log 120public interfaces. For example, for automated test module output processing it might be more convenient to use 121the XML based format. 122 123In most cases, the __UTF__ can't provide an exact location, where system error occurs or uncaught C++ exception 124is thrown from. To be able to pinpoint it as close as possible the __UTF__ keeps track of checkpoints - the 125location a test module passed through. A test case entrance and exit points, a test tool invocation point the 126__UTF__ tracks automatically. Any other checkpoints should be entered by you manually. The test log provides two 127macros for this purpose: 128 129* __BOOST_TEST_CHECKPOINT__ to specify a ['named] checkpoint and 130* __BOOST_TEST_PASSPOINT__ to specify an ['unnamed] checkpoint. 131 132 133 134[endsect] [/ test log output] 135 136 137 138[/ -------------------------------------------------------------------------------------------------- ] 139[section:log_runtime_config Verbosity of the logs] 140 141The active log level threshold can be configured at runtime using the parameter __param_log_level__. 142The test log output format can be selected using either parameter __param_log_format__ or the 143parameter __param_output_format__. 144 145[endsect] [/section:log_runtime_config] 146 147 148 149 150 151[/ -------------------------------------------------------------------------------------------------- ] 152[section:testing_tool_output_disable Disabling automatic printing for specific types] 153 154Most of the [link boost_test.testing_tools testing tools] print values of their 155arguments to the output stream in some form of log statement. If arguments type does not support 156`` 157 operator<<(std::ostream&, ArgumentType const&); 158`` 159interface you will get a compilation error. You can either implement above interface or prohibit 160the [link boost_test.testing_tools testing tools] from logging argument values for 161specified type. To do so, use following statement on file level before first test case that 162includes statement failing to compile: 163 164`` 165 BOOST_TEST_DONT_PRINT_LOG_VALUE(ArgumentType) 166`` 167 168[bt_example example32..BOOST_TEST_DONT_PRINT_LOG_VALUE usage..run-fail] 169 170[endsect] [/section:testing_tool_output_disable] 171 172 173 174 175 176[/ -------------------------------------------------------------------------------------------------- ] 177[section:test_output_macro_message Custom messages] 178The macro __BOOST_TEST_MESSAGE__ is intended to be used for the purpose of injecting an additional message into the 179__UTF__ test log. These messages are not intended to indicate any error or warning conditions, but rather as 180information/status notifications. The macro signature is as follows: 181 182 __BOOST_TEST_MESSAGE__(test_message); 183 184The test_message argument can be as simple as C string literal or any custom expression that you can produce with in a 185manner similar to standard `std::iostream` operation. 186 187[important Messages generated by this tool do not appear in test log output with default value of the active log level 188 threshold. For these messages to appear the active log level threshold has to be set to a value below or equal 189 to "message". 190] 191 192[bt_example example21..__BOOST_TEST_MESSAGE__ usage..run] 193 194[endsect] [/section:test_output_macro_message] 195 196 197[/ -------------------------------------------------------------------------------------------------- ] 198 199[include checkpoints.qbk] 200[include contexts.qbk] 201[include logging_floating_point.qbk] 202[include log_format.qbk] 203[include compilation_options.qbk] 204 205 206[#ref_tests_report][section Test reports] 207[warning Section empty] 208[endsect] 209 210[include progress_display.qbk] 211[include testout_summary.qbk] 212 213[endsect] [/test_output] 214 215