1[/ 2 / Copyright (c) 2003 Boost.Test team 3 / 4 / Distributed under the Boost Software License, Version 1.0. (See accompanying 5 / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) 6 /] 7 8 9[section:testing_tool_ref Reference API for writing tests] 10 11 12 13[/ ###############################################################################################] 14[section:assertion_boost_test_universal_macro `BOOST_TEST`] 15 16 17`` 18 BOOST_TEST(statement); 19 BOOST_TEST_<level>(statement); 20 21 // replacement failure message, requires variadic macros 22 BOOST_TEST(statement, "failure message"); 23 24 // Floating point comparison, requires variadic macros, auto and decltype 25 BOOST_TEST(statement, floating_point_comparison_manipulation); 26 27 // bitwise comparison, requires variadic macros, auto and decltype 28 BOOST_TEST(statement, boost::test_tools::bitwise() ); 29 30 // element-wise comparison, for containers 31 BOOST_TEST(statement, boost::test_tools::per_element() ); 32 33 // lexicographic comparison, for containers 34 BOOST_TEST(statement, boost::test_tools::lexicographic() ); 35`` 36 37The full documentation of this macro is located [link boost_test.testing_tools.boost_test_universal_macro here]. 38 39The macro is available in three variants, corresponding to different [link boost_test.testing_tools.tools_assertion_severity_level assertion severity levels]: 40 41`` 42 BOOST_TEST // or BOOST_TEST_CHECK 43 BOOST_TEST_REQUIRE 44 BOOST_TEST_WARN 45`` 46 47 48* `"failure message"` is a C-string printed in case of failure in place of the default message. 49 See [link boost_test.testing_tools.reports this section] for 50 more details. 51* `floating_point_comparison_manipulation` is one of the floating point comparison manipulators. 52 See [link boost_test.testing_tools.boost_test_universal_macro this section] 53 for more details. 54* [classref boost::test_tools::bitwise] is a manipulator indicating that the comparison should be performed bitwise. See 55 [link boost_test.testing_tools.extended_comparison.bitwise this section] for more details 56* [classref boost::test_tools::per_element] is a manipulator indicating that the comparison should be performed on each element, in sequence, rather 57 than on containers. See 58 [link boost_test_coll_perelement this section] for more details 59* [classref boost::test_tools::lexicographic] is a manipulator indicating that the comparison should be performed with the lexicographic order. See 60 [link boost_test_coll_default_lex this section] for more details 61 62[h3 Limitations and workaround] 63There are some restrictions on the statements that are supported by this tool. Those are explained in details in 64[link boost_test_statement_limitations this] section. 65 66[endsect] 67 68 69 70[/ DECORATORS ###############################################################################################] 71[/-----------------------------------------------------------------] 72 73[section:decorator_expected_failures expected_failures (decorator)] 74 75`` 76expected_failures(counter_t number); 77`` 78 79Indicates the expected failures for a test unit. 80See [link boost_test.testing_tools.expected_failures here] for more details. 81 82[endsect] [/ section expected_failures] 83 84 85[/-----------------------------------------------------------------] 86[section:decorator_timeout timeout (decorator)] 87 88`` 89timeout(unsigned seconds); 90`` 91 92Specifies a time-out for a *test-case*, above which the test-case is forced to stop and reported as failing. 93See [link boost_test.testing_tools.timeout here] for more details. 94 95[endsect] [/ section timeout] 96 97 98[/-----------------------------------------------------------------] 99[section:decorator_tolerance tolerance (decorator)] 100 101`` 102template <typename FPT> 103 tolerance(FPT eps); 104 105template <typename FPT> 106 tolerance(test_tools::fpc::percent_tolerance_t<FPT> eps) 107`` 108 109Decorator `tolerance` specifies the default comparison tolerance for floating point type `FTP` in the decorated test 110unit. The default tolerance only applies to a particular type, so it makes sense to provide more than one `tolerance` 111decorator if we are comparing different floating point types. For more details see __floating_points_testing_impl__. 112The variant with `percent_tolerance` uses value `eps / 100` as tolerance. 113 114[bt_example decorator_13..decorator tolerance..run-fail] 115 116In the above example, in `test1`, checks on `double`s fail because they differ by more what tolerance for `double`s 117specifies. In `test2` the tolerance for `double`s is greater and therefore the checks succeed. In `test3`, we specify 118only tolerance for type `float`, and since the checks use type `double` the specified tolerance does not apply. Tolerance 119in `test4` is equivalent to that in `test1`, therefore its checks also fail. Tolerance in `test5` is equivalent to 120that in `test2`, therefore its checks also succeed. 121[endsect] [/ section decorator_tolerance] 122 123 124 125 126 127[/ DEPRECATED API] 128[/ ###############################################################################################] 129[#ref_BOOST_level][section:assertion_boost_level `BOOST_<level>`] 130 131 132`` 133 BOOST_WARN(predicate); 134 BOOST_CHECK(predicate); 135 BOOST_REQUIRE(predicate); 136`` 137 138These tools are used to validate the predicate value. The only parameter for these tools is a boolean predicate 139value that gets validated. It could be any expression that could be evaluated and converted to boolean value. The 140expression gets evaluated only once, so it's safe to pass complex expression for validation. 141 142[bt_example example34..BOOST_<level> usage..run-fail] 143 144See also: 145 146* __BOOST_LEVEL_MESSAGE__ 147 148[endsect] 149 150 151[/ ###############################################################################################] 152[section:assertion_boost_level_bitwise_eq `BOOST_<level>_BITWISE_EQUAL`] 153 154 155`` 156 BOOST_WARN_BITWISE_EQUAL(left, right); 157 BOOST_CHECK_BITWISE_EQUAL(left, right); 158 BOOST_REQUIRE_BITWISE_EQUAL(left, right); 159`` 160 161These tools are used to perform bitwise comparison of two values. The check shows all positions where left and 162right value's bits mismatch. 163 164The first parameter is the left compared value. The second parameter is the right compared value. Parameters are 165not required to be of the same type, but warning is issued if their type's size does not coincide. 166 167[bt_example example33..BOOST_<level>_BITWISE_EQUAL usage..run-fail] 168 169See also: 170 171* __BOOST_LEVEL_EQUAL__ 172 173[endsect] 174 175[/ ###############################################################################################] 176[section:assertion_boost_level_eq `BOOST_<level>_EQUAL`] 177 178`` 179 BOOST_WARN_EQUAL(left, right); 180 BOOST_CHECK_EQUAL(left, right); 181 BOOST_REQUIRE_EQUAL(left, right); 182`` 183 184Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__(left == right)`. 185The difference is that the mismatched values are reported as well. 186 187[note It is bad idea to use these tools to compare floating point values. Use __BOOST_LEVEL_CLOSE__ or 188 __BOOST_LEVEL_CLOSE_FRACTION__ tools instead. 189] 190 191[bt_example example35..BOOST_<level>_EQUAL usage..run-fail] 192 193See also: 194 195* __BOOST_LEVEL__ 196* __BOOST_LEVEL_CLOSE__ 197* __BOOST_LEVEL_NE__ 198* __BOOST_LEVEL_EQUAL_COLLECTIONS__ 199 200[endsect] 201 202[/ ###############################################################################################] 203[section:assertion_boost_level_eq_collections `BOOST_<level>_EQUAL_COLLECTIONS`] 204 205`` 206 BOOST_WARN_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end); 207 BOOST_CHECK_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end); 208 BOOST_REQUIRE_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end); 209`` 210 211These tools are used to perform an element by element comparison of two collections. They print all mismatched 212positions, collection elements at these positions and check that the collections have the same size. The first two 213parameters designate begin and end of the first collection. The two last parameters designate begin and end of the 214second collection. 215 216[bt_example example36..BOOST_<level>_EQUAL_COLLECTIONS usage..run-fail] 217 218See also: 219 220* __BOOST_LEVEL_EQUAL__ 221 222[endsect] 223 224[/ ###############################################################################################] 225[section:assertion_boost_level_close `BOOST_<level>_CLOSE`] 226 227`` 228 BOOST_WARN_CLOSE(left, right, tolerance); 229 BOOST_CHECK_CLOSE(left, right, tolerance); 230 BOOST_REQUIRE_CLOSE(left, right, tolerance); 231`` 232 233These tools are used to check on closeness using strong relationship defined by the predicate 234``check_is_close( left, right, tolerance )`` 235 236To check for the weak relationship use 237__BOOST_LEVEL_PREDICATE__ family of tools with explicit `check_is_close` invocation. 238 239 240The first parameter is the ['left] compared value. The second parameter is the 241['right] compared value. Last third parameter defines the tolerance for the comparison in 242[link boost_test.testing_tools.extended_comparison.floating_point [*percentage units]]. 243 244[note It is required for left and right parameters to be of the same floating point type. You will need to explicitly 245 resolve any type mismatch to select which type to use for comparison. 246] 247 248[note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`. 249] 250 251[bt_example example42..BOOST_<level>_CLOSE usage with small values..run-fail] 252[bt_example example43..BOOST_<level>_CLOSE usage with big values..run] 253 254See also: 255 256* __BOOST_LEVEL_CLOSE_FRACTION__ 257* __BOOST_LEVEL_SMALL__ 258* __BOOST_LEVEL_EQUAL__ 259* __floating_points_testing_tools__ 260 261[endsect] 262 263[/ ###############################################################################################] 264[section:assertion_boost_level_close_fraction `BOOST_<level>_CLOSE_FRACTION`] 265 266`` 267 BOOST_WARN_CLOSE_FRACTION(left, right, tolerance); 268 BOOST_CHECK_CLOSE_FRACTION(left, right, tolerance); 269 BOOST_REQUIRE_CLOSE_FRACTION(left, right, tolerance); 270`` 271 272These tools are used to check on closeness using strong relationship defined by the predicate 273``check_is_close(left, right, tolerance)`` 274 275To check for the weak relationship use __BOOST_LEVEL_PREDICATE__ family of tools with explicit `check_is_close` invocation. 276 277The first parameter is the ['left] compared value. The second parameter is the 278['right] compared value. Last third parameter defines the tolerance for the comparison as 279[link boost_test.testing_tools.extended_comparison.floating_point [*fraction of absolute values being compared]]. 280 281[note It is required for left and right parameters to be of the same floating point type. You will need to explicitly 282 resolve any type mismatch to select which type to use for comparison.] 283 284[note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.] 285 286[bt_example example44..BOOST_<level>_CLOSE_FRACTION usage..run-fail] 287 288See also: 289 290* __BOOST_LEVEL_CLOSE__ 291* __BOOST_LEVEL_SMALL__ 292* __BOOST_LEVEL_EQUAL__ 293* __floating_points_testing_tools__ 294 295[endsect] 296 297[/ ###############################################################################################] 298[section:assertion_boost_level_exception `BOOST_<level>_EXCEPTION`] 299 300`` 301 BOOST_WARN_EXCEPTION(expression, exception, predicate); 302 BOOST_CHECK_EXCEPTION(expression, exception, predicate); 303 BOOST_REQUIRE_EXCEPTION(expression, exception, predicate); 304`` 305 306These tools are used to perform an exception detection and validation check. Tools execute the supplied expression 307and validate that it throws an exception of supplied class (or the one derived from it) that complies with the 308supplied predicate. If the expression throws any other unrelated exception, doesn't throw at all or 309predicate evaluates to false, check fails. In comparison with __BOOST_LEVEL_THROW__ tools these 310allow performing more fine-grained checks. For example: make sure that an expected exception has specific 311error message. 312 313[bt_example example37..BOOST_<level>_EXCEPTION usage..run-fail] 314 315See also: 316 317* __BOOST_LEVEL_THROW__ 318 319[endsect] 320 321[/ ###############################################################################################] 322[section:assertion_boost_level_ge `BOOST_<level>_GE`] 323 324`` 325 BOOST_WARN_GE(left, right); 326 BOOST_CHECK_GE(left, right); 327 BOOST_REQUIRE_GE(left, right); 328`` 329 330Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left >= right )`. 331The difference is that the argument values are reported as well. 332 333[bt_example example57..BOOST_<level>_GE usage..run-fail] 334 335See also: 336 337* __BOOST_LEVEL_LE__ 338* __BOOST_LEVEL_LT__ 339* __BOOST_LEVEL_GT__ 340 341[endsect] 342 343 344[/ ###############################################################################################] 345[section:assertion_boost_level_gt `BOOST_<level>_GT`] 346 347 348`` 349 BOOST_WARN_GT(left, right); 350 BOOST_CHECK_GT(left, right); 351 BOOST_REQUIRE_GT(left, right); 352`` 353 354Check performed by these tools is the same as the one performed by __BOOST_LEVEL__`( left > right )`. 355The difference is that the argument values are reported as well. 356 357[bt_example example58..BOOST_<level>_GT usage..run-fail] 358 359See also: 360 361* __BOOST_LEVEL_LE__ 362* __BOOST_LEVEL_LT__ 363* __BOOST_LEVEL_GE__ 364 365[endsect] 366 367[/ ###############################################################################################] 368[section:assertion_boost_level_le `BOOST_<level>_LE`] 369 370`` 371 BOOST_WARN_LE(left, right); 372 BOOST_CHECK_LE(left, right); 373 BOOST_REQUIRE_LE(left, right); 374`` 375 376Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left <= right )`. 377The difference is that the argument values are reported as well. 378 379[bt_example example55..BOOST_<level>_LE usage..run-fail] 380 381See also: 382 383* __BOOST_LEVEL_LE__ 384* __BOOST_LEVEL_GE__ 385* __BOOST_LEVEL_GT__ 386 387[endsect] 388 389[/ ###############################################################################################] 390[section:assertion_boost_level_lt `BOOST_<level>_LT`] 391 392`` 393 BOOST_WARN_LT(left, right); 394 BOOST_CHECK_LT(left, right); 395 BOOST_REQUIRE_LT(left, right); 396`` 397 398Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left < right )`. 399The difference is that the argument values are reported as well. 400 401[bt_example example56..BOOST_<level>_LT usage..run-fail] 402 403See also: 404 405* __BOOST_LEVEL_LE__ 406* __BOOST_LEVEL_GE__ 407* __BOOST_LEVEL_GT__ 408 409[endsect] 410 411[/ ###############################################################################################] 412[section:assertion_boost_level_message `BOOST_<level>_MESSAGE`] 413 414`` 415 BOOST_WARN_MESSAGE(predicate, message); 416 BOOST_CHECK_MESSAGE(predicate, message); 417 BOOST_REQUIRE_MESSAGE(predicate, message); 418`` 419 420These tools perform exactly the same check as __BOOST_LEVEL__ tools. The only difference is that 421instead of generating an error/confirm message these use the supplied one. 422 423The first parameter is the boolean expression. The second parameter is the message reported in case of check 424failure. The message argument can be constructed of components of any type supporting the 425`std::ostream& operator<<(std::ostream&)`. 426 427[bt_example example38..BOOST_<level>_MESSAGE usage..run] 428 429See also: 430 431* __BOOST_LEVEL__ 432 433[endsect] 434 435 436[/ ###############################################################################################] 437[section:assertion_boost_level_ne `BOOST_<level>_NE`] 438 439 440`` 441 BOOST_WARN_NE(left, right); 442 BOOST_CHECK_NE(left, right); 443 BOOST_REQUIRE_NE(left, right); 444`` 445 446Check performed by these tools is the same as the one performed by `__BOOST_<level>__( left != right )`. 447The difference is that the matched values are reported as well. 448 449[bt_example example54..BOOST_<level>_NE usage..run-fail] 450 451See also: 452 453* __BOOST_LEVEL_EQUAL__ 454 455[endsect] 456 457[/ ###############################################################################################] 458[section:assertion_boost_level_no_throw `BOOST_<level>_NO_THROW`] 459 460 461`` 462 BOOST_WARN_NO_THROW(expression); 463 BOOST_CHECK_NO_THROW(expression); 464 BOOST_REQUIRE_NO_THROW(expression); 465`` 466 467These tools are used to perform a "no throw" check. Tools execute the supplied expression and validate that it does 468not throw any exceptions. Error would be reported by the framework even if the statement appear directly in test 469case body and throw any exception. But these tools allow proceeding further with test case in case of failure. 470 471If check is successful, tools may produce a confirmation message, in other case they produce an error message in 472a form ``error in <test-case-name>;exception was thrown by <expression>`` 473 474The only parameter is an expression to execute. You can use `do {} while(0)` block if you want to execute more than one 475statement. 476 477[bt_example example39..BOOST_<level>_NO_THROW usage..run-fail] 478 479See also: 480 481* __BOOST_LEVEL_THROW__ 482 483[endsect] 484 485[/ ###############################################################################################] 486[section:assertion_boost_level_predicate `BOOST_<level>_PREDICATE`] 487 488 489`` 490 BOOST_WARN_PREDICATE(predicate, arguments_list); 491 BOOST_CHECK_PREDICATE(predicate, arguments_list); 492 BOOST_REQUIRE_PREDICATE(predicate, arguments_list); 493`` 494 495These are generic tools used to validate an arbitrary supplied predicate functor (there is a compile time limit on 496predicate arity defined by the configurable macro `BOOST_TEST_MAX_PREDICATE_ARITY`). To 497validate zero arity predicate use __BOOST_<level>__ tools. In other cases prefer theses tools. The 498advantage of these tools is that they show arguments values in case of predicate failure. 499 500The first parameter is the predicate itself. The second parameter is the list of predicate arguments each wrapped 501in round brackets (`BOOST_PP` sequence format). 502 503[bt_example example40..BOOST_<level>_PREDICATE usage..run] 504 505[note Note difference in error log from __BOOST_<level>__] 506 507See also: 508 509* __BOOST_LEVEL__ 510 511[endsect] 512 513[/ ###############################################################################################] 514[section:assertion_boost_level_small `BOOST_<level>_SMALL`] 515 516`` 517 BOOST_WARN_SMALL(value, tolerance); 518 BOOST_CHECK_SMALL(value, tolerance); 519 BOOST_REQUIRE_SMALL(value, tolerance); 520`` 521 522These tools are used to check that supplied value is small enough. The "smallness" is defined by absolute value 523of the tolerance supplied as a second argument. Use these tools with caution. To compare to values on closeness 524it's preferable to use __BOOST_LEVEL_CLOSE__ tools instead. 525 526The first parameter is the value to check. The second parameter is the tolerance. 527 528[note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.] 529 530[bt_example example41..BOOST_<level>_SMALL usage..run-fail] 531 532See also: 533 534* __BOOST_LEVEL_CLOSE__ 535* __BOOST_LEVEL_CLOSE_FRACTION__ 536* __floating_points_testing_tools__ 537 538[endsect] 539 540[/ ###############################################################################################] 541[section:assertion_boost_level_throw `BOOST_<level>_THROW`] 542 543 544`` 545 BOOST_WARN_THROW(expression, exception); 546 BOOST_CHECK_THROW(expression, exception); 547 BOOST_REQUIRE_THROW(expression, exception); 548`` 549 550These tools are used to perform an exception detection check. Tools execute the supplied expression and validate 551that it throws an exception of supplied class (or the one derived from it) or it's child. If the statement 552throws any other unrelated exception or doesn't throw at all, check fails. 553 554If check is successful, the tool produces a confirmation message, in other case it produces an error message in a 555form 556`` 557error in <test-case-name>: exception <exception> expected 558`` 559 560The first parameter is the expression to execute. Use `do{} while(0)` block if you want to execute more than one 561statement. The second parameter is an expected exception. 562 563[bt_example example45..BOOST_<level>_THROW usage..run-fail] 564 565See also: 566 567* __BOOST_LEVEL_NO_THROW__ 568 569[endsect] 570 571 572[/ ###############################################################################################] 573[section:test_org_boost_test_case_expected_failure `BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES`] 574Indicates the number of failures for a test case. 575 576See [link boost_test.testing_tools.expected_failures here] for more details. 577[endsect] [/ expected failures] 578 579[/ ###############################################################################################] 580[section:assertion_boost_error `BOOST_ERROR`] 581 582`` 583 BOOST_ERROR(message); 584`` 585 586__BOOST_ERROR__ tool behave the same way as `__BOOST_TEST__(false, message)`. This tool is used for 587an unconditional error counter increasing and message logging. 588 589The tool's only parameter is an error message to log. 590 591[bt_example example46..BOOST_ERROR usage..run-fail] 592 593See also: 594 595* __BOOST_TEST__ 596 597[endsect] 598 599 600[/ ###############################################################################################] 601[section:assertion_boost_fail `BOOST_FAIL`] 602 603`` 604 BOOST_FAIL(message); 605`` 606 607`__BOOST_FAIL__(message)` behave the same way as `__BOOST_TEST_REQUIRE__(false, message)`. This tool is used for an 608unconditional error counter increasing, message logging and the current test case aborting. 609 610The tool's only parameter is an error message to log. 611 612[bt_example example47..BOOST_FAIL usage..run-fail] 613 614See also: 615 616* __BOOST_TEST__ 617* 618 619[endsect] 620 621 622[/ ###############################################################################################] 623[section:assertion_boost_is_defined `BOOST_IS_DEFINED`] 624 625`` 626 BOOST_IS_DEFINED(symbol); 627`` 628 629Unlike the rest of the tools in the toolbox this tool does not perform the logging itself. Its only purpose 630is to check at runtime whether or not the supplied preprocessor symbol is defined. Use it in combination with 631__BOOST_<level>__ to perform and log validation. Macros of any arity could be checked. To check the 632macro definition with non-zero arity specify dummy arguments for it. See below for example. 633 634The only tool's parameter is a preprocessor symbol that gets validated. 635 636[bt_example example48..BOOST_IS_DEFINED usage..run-fail] 637 638See also: 639 640* __BOOST_LEVEL__ 641 642[endsect] 643 644[/ ###############################################################################################] 645[section:assertion_control_under_debugger `BOOST_TEST_TOOLS_UNDER_DEBUGGER`] 646When defined, assertions evaluate their expression eagerly, as described [link boost_test.testing_tools.debugging here]. 647[endsect] [/ assertion_control_under_debugger] 648 649[/ ###############################################################################################] 650[section:assertion_control_under_debuggable `BOOST_TEST_TOOLS_DEBUGGABLE`] 651When defined, test assertions are compiled in two modes (debugger-friendly and full-featured) and the version is selected at run-time, as described [link boost_test.testing_tools.debugging here]. 652[endsect] [/ assertion_control_under_debuggable] 653 654[endsect] [/ testing_tool_ref] 655