1[/
2 / Copyright (c) 2003 Boost.Test contributors
3 /
4 / Distributed under the Boost Software License, Version 1.0. (See accompanying
5 / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
6 /]
7
8
9[section:testing_tool_ref Reference API for writing tests]
10
11
12
13[/ ###############################################################################################]
14[section:assertion_boost_test_universal_macro `BOOST_TEST`]
15
16
17``
18  BOOST_TEST(statement);
19  BOOST_TEST_<level>(statement);
20
21  // replacement failure message, requires variadic macros
22  BOOST_TEST(statement, "failure message");
23
24  // Floating point comparison, requires variadic macros, auto and decltype
25  BOOST_TEST(statement, floating_point_comparison_manipulation);
26
27  // bitwise comparison, requires variadic macros, auto and decltype
28  BOOST_TEST(statement, boost::test_tools::bitwise() );
29
30  // element-wise comparison, for containers
31  BOOST_TEST(statement, boost::test_tools::per_element() );
32
33  // lexicographic comparison, for containers
34  BOOST_TEST(statement, boost::test_tools::lexicographic() );
35``
36
37The full documentation of this macro is located [link boost_test.testing_tools.boost_test_universal_macro here].
38
39The macro is available in three variants, corresponding to different [link boost_test.testing_tools.tools_assertion_severity_level assertion severity levels]:
40
41``
42  BOOST_TEST // or BOOST_TEST_CHECK
43  BOOST_TEST_REQUIRE
44  BOOST_TEST_WARN
45``
46
47
48* `"failure message"` is a C-string printed in case of failure in place of the default message.
49  See [link boost_test.testing_tools.reports this section] for
50  more details.
51* `floating_point_comparison_manipulation` is one of the floating point comparison manipulators.
52  See [link boost_test.testing_tools.boost_test_universal_macro this section]
53   for more details.
54* [classref boost::test_tools::bitwise] is a manipulator indicating that the comparison should be performed bitwise. See
55  [link boost_test.testing_tools.extended_comparison.bitwise this section] for more details
56* [classref boost::test_tools::per_element] is a manipulator indicating that the comparison should be performed on each element, in sequence, rather
57  than on containers. See
58  [link boost_test_coll_perelement this section] for more details
59* [classref boost::test_tools::lexicographic] is a manipulator indicating that the comparison should be performed with the lexicographic order. See
60  [link boost_test_coll_default_lex this section] for more details
61
62[h3 Limitations and workaround]
63There are some restrictions on the statements that are supported by this tool. Those are explained in details in
64[link boost_test_statement_limitations this] section.
65
66[endsect]
67
68
69
70[/ DECORATORS ###############################################################################################]
71[/-----------------------------------------------------------------]
72
73[section:decorator_expected_failures expected_failures (decorator)]
74
75``
76expected_failures(counter_t number);
77``
78
79Indicates the expected failures for a test unit.
80See [link boost_test.testing_tools.expected_failures here] for more details.
81
82[endsect] [/ section expected_failures]
83
84
85[/-----------------------------------------------------------------]
86[section:decorator_timeout timeout (decorator)]
87
88``
89timeout(unsigned int seconds);
90``
91
92Specifies a time-out for a *test-case* or a *test-suite*, in wall-clock time.
93
94If a test-case lasts longer than the timeout, the test is flagged as failed. On some systems (see below),
95the test-case is forced to stop.
96
97For test-suites, the mechanism is similar: every test-unit under the test-suite is allocated a maximum
98duration time that is the remainder of the timeout after the previous tests have been executed. If a timeout occurs
99during the execution of the suite, the suite is flagged as timed-out and the remaining test-units are skipped.
100
101See [link boost_test.testing_tools.timeout here] for more details.
102
103[note The macro
104  `BOOST_SIGACTION_BASED_SIGNAL_HANDLING` is defined
105  if Boost.Test is able to force the test-case to stop.]
106
107[note The support for test suites has been added in [link ref_CHANGE_LOG_3_10 Boost 1.70 / __UTF__ v3.10]]
108
109[endsect] [/ section timeout]
110
111
112[/-----------------------------------------------------------------]
113[section:decorator_tolerance tolerance (decorator)]
114
115``
116template <typename FPT>
117  tolerance(FPT eps);
118
119template <typename FPT>
120  tolerance(test_tools::fpc::percent_tolerance_t<FPT> eps)
121``
122
123Decorator `tolerance` specifies the default comparison tolerance for floating point type `FTP` in the decorated test
124unit. The default tolerance only applies to a particular type, so it makes sense to provide more than one `tolerance`
125decorator if we are comparing different floating point types.
126The variant with `percent_tolerance` uses value ``eps / 100`` as tolerance.
127
128[note For more details see the
129  [link boost_test.testing_tools.extended_comparison.floating_point floating points comparison] section.
130]
131
132[bt_example decorator_13..decorator tolerance..run-fail]
133
134In the above example,  in `test1`, checks on `double`s fail because they differ by more what tolerance for `double`s
135specifies. In `test2` the tolerance for `double`s is greater and therefore the checks succeed. In `test3`, we specify
136only tolerance for type `float`, and since the checks use type `double` the specified tolerance does not apply. Tolerance
137in `test4` is equivalent to that in `test1`, therefore its checks also fail. Tolerance in `test5` is equivalent to
138that in `test2`, therefore its checks also succeed.
139[endsect] [/ section decorator_tolerance]
140
141
142
143
144
145[/ DEPRECATED API]
146[/ ###############################################################################################]
147[#ref_BOOST_level][section:assertion_boost_level `BOOST_<level>`]
148
149
150``
151  BOOST_WARN(predicate);
152  BOOST_CHECK(predicate);
153  BOOST_REQUIRE(predicate);
154``
155
156These tools are used to validate the predicate value. The only parameter for these tools is a boolean predicate
157value that gets validated. It could be any expression that could be evaluated and converted to boolean value. The
158expression gets evaluated only once, so it's safe to pass complex expression for validation.
159
160[bt_example example34..BOOST_<level> usage..run-fail]
161
162See also:
163
164* __BOOST_LEVEL_MESSAGE__
165
166[endsect]
167
168
169[/ ###############################################################################################]
170[section:assertion_boost_level_bitwise_eq `BOOST_<level>_BITWISE_EQUAL`]
171
172
173``
174  BOOST_WARN_BITWISE_EQUAL(left, right);
175  BOOST_CHECK_BITWISE_EQUAL(left, right);
176  BOOST_REQUIRE_BITWISE_EQUAL(left, right);
177``
178
179These tools are used to perform bitwise comparison of two values. The check shows all positions where left and
180right value's bits mismatch.
181
182The first parameter is the left compared value. The second parameter is the right compared value. Parameters are
183not required to be of the same type, but warning is issued if their type's size does not coincide.
184
185[bt_example example33..BOOST_<level>_BITWISE_EQUAL usage..run-fail]
186
187See also:
188
189* __BOOST_LEVEL_EQUAL__
190
191[endsect]
192
193[/ ###############################################################################################]
194[section:assertion_boost_level_eq `BOOST_<level>_EQUAL`]
195
196``
197  BOOST_WARN_EQUAL(left, right);
198  BOOST_CHECK_EQUAL(left, right);
199  BOOST_REQUIRE_EQUAL(left, right);
200``
201
202Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__(left == right)`.
203The difference is that the mismatched values are reported as well.
204
205[note It is bad idea to use these tools to compare floating point values. Use __BOOST_LEVEL_CLOSE__ or
206      __BOOST_LEVEL_CLOSE_FRACTION__ tools instead.
207]
208
209[bt_example example35..BOOST_<level>_EQUAL usage..run-fail]
210
211See also:
212
213* __BOOST_LEVEL__
214* __BOOST_LEVEL_CLOSE__
215* __BOOST_LEVEL_NE__
216* __BOOST_LEVEL_EQUAL_COLLECTIONS__
217
218[endsect]
219
220[/ ###############################################################################################]
221[section:assertion_boost_level_eq_collections `BOOST_<level>_EQUAL_COLLECTIONS`]
222
223``
224  BOOST_WARN_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
225  BOOST_CHECK_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
226  BOOST_REQUIRE_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
227``
228
229These tools are used to perform an element by element comparison of two collections. They print all mismatched
230positions, collection elements at these positions and check that the collections have the same size. The first two
231parameters designate begin and end of the first collection. The two last parameters designate begin and end of the
232second collection.
233
234[bt_example example36..BOOST_<level>_EQUAL_COLLECTIONS usage..run-fail]
235
236See also:
237
238* __BOOST_LEVEL_EQUAL__
239
240[endsect]
241
242[/ ###############################################################################################]
243[section:assertion_boost_level_close `BOOST_<level>_CLOSE`]
244
245``
246  BOOST_WARN_CLOSE(left, right, tolerance);
247  BOOST_CHECK_CLOSE(left, right, tolerance);
248  BOOST_REQUIRE_CLOSE(left, right, tolerance);
249``
250
251These tools are used to check on closeness using strong relationship defined by the predicate
252``check_is_close( left, right, tolerance )``
253
254To check for the weak relationship use
255__BOOST_LEVEL_PREDICATE__ family of tools with explicit `check_is_close` invocation.
256
257
258The first parameter is the ['left] compared value. The second parameter is the
259['right] compared value. Last third parameter defines the tolerance for the comparison in
260[link boost_test.testing_tools.extended_comparison.floating_point [*percentage units]].
261
262[note It is required for left and right parameters to be of the same floating point type. You will need to explicitly
263      resolve any type mismatch to select which type to use for comparison.
264]
265
266[note The floating point comparison tools are automatically added if the __UTF__
267  is included as indicated in the previous sections. The tools are implemented is in the header
268  [headerref boost/test/tools/floating_point_comparison.hpp `boost/test/tools/floating_point_comparison.hpp`].]
269
270[bt_example example42..BOOST_<level>_CLOSE usage with small values..run-fail]
271[bt_example example43..BOOST_<level>_CLOSE usage with big values..run]
272
273See also:
274
275* __BOOST_LEVEL_CLOSE_FRACTION__
276* __BOOST_LEVEL_SMALL__
277* __BOOST_LEVEL_EQUAL__
278* __floating_points_testing_tools__
279
280[endsect]
281
282[/ ###############################################################################################]
283[section:assertion_boost_level_close_fraction `BOOST_<level>_CLOSE_FRACTION`]
284
285``
286  BOOST_WARN_CLOSE_FRACTION(left, right, tolerance);
287  BOOST_CHECK_CLOSE_FRACTION(left, right, tolerance);
288  BOOST_REQUIRE_CLOSE_FRACTION(left, right, tolerance);
289``
290
291These tools are used to check on closeness using strong relationship defined by the predicate
292``check_is_close(left, right, tolerance)``
293
294To check for the weak relationship use __BOOST_LEVEL_PREDICATE__ family of tools with explicit `check_is_close` invocation.
295
296The first parameter is the ['left] compared value. The second parameter is the
297['right] compared value. Last third parameter defines the tolerance for the comparison as
298[link boost_test.testing_tools.extended_comparison.floating_point [*fraction of absolute values being compared]].
299
300[note It is required for left and right parameters to be of the same floating point type. You will need to explicitly
301      resolve any type mismatch to select which type to use for comparison.]
302
303[note The floating point comparison tools are automatically added if the __UTF__
304  is included as indicated in the previous sections. The tools are implemented is in the header
305  [headerref boost/test/tools/floating_point_comparison.hpp `boost/test/tools/floating_point_comparison.hpp`].]
306
307
308[bt_example example44..BOOST_<level>_CLOSE_FRACTION usage..run-fail]
309
310See also:
311
312* __BOOST_LEVEL_CLOSE__
313* __BOOST_LEVEL_SMALL__
314* __BOOST_LEVEL_EQUAL__
315* __floating_points_testing_tools__
316
317[endsect]
318
319[/ ###############################################################################################]
320[section:assertion_boost_level_ge `BOOST_<level>_GE`]
321
322``
323  BOOST_WARN_GE(left, right);
324  BOOST_CHECK_GE(left, right);
325  BOOST_REQUIRE_GE(left, right);
326``
327
328Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left >= right )`.
329The difference is that the argument values are reported as well.
330
331[bt_example example57..BOOST_<level>_GE usage..run-fail]
332
333See also:
334
335* __BOOST_LEVEL_LE__
336* __BOOST_LEVEL_LT__
337* __BOOST_LEVEL_GT__
338
339[endsect]
340
341
342[/ ###############################################################################################]
343[section:assertion_boost_level_gt `BOOST_<level>_GT`]
344
345
346``
347  BOOST_WARN_GT(left, right);
348  BOOST_CHECK_GT(left, right);
349  BOOST_REQUIRE_GT(left, right);
350``
351
352Check performed by these tools is the same as the one performed by __BOOST_LEVEL__`( left > right )`.
353The difference is that the argument values are reported as well.
354
355[bt_example example58..BOOST_<level>_GT usage..run-fail]
356
357See also:
358
359* __BOOST_LEVEL_LE__
360* __BOOST_LEVEL_LT__
361* __BOOST_LEVEL_GE__
362
363[endsect]
364
365[/ ###############################################################################################]
366[section:assertion_boost_level_le `BOOST_<level>_LE`]
367
368``
369  BOOST_WARN_LE(left, right);
370  BOOST_CHECK_LE(left, right);
371  BOOST_REQUIRE_LE(left, right);
372``
373
374Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left <= right )`.
375The difference is that the argument values are reported as well.
376
377[bt_example example55..BOOST_<level>_LE usage..run-fail]
378
379See also:
380
381* __BOOST_LEVEL_LE__
382* __BOOST_LEVEL_GE__
383* __BOOST_LEVEL_GT__
384
385[endsect]
386
387[/ ###############################################################################################]
388[section:assertion_boost_level_lt `BOOST_<level>_LT`]
389
390``
391  BOOST_WARN_LT(left, right);
392  BOOST_CHECK_LT(left, right);
393  BOOST_REQUIRE_LT(left, right);
394``
395
396Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left < right )`.
397The difference is that the argument values are reported as well.
398
399[bt_example example56..BOOST_<level>_LT usage..run-fail]
400
401See also:
402
403* __BOOST_LEVEL_LE__
404* __BOOST_LEVEL_GE__
405* __BOOST_LEVEL_GT__
406
407[endsect]
408
409[/ ###############################################################################################]
410[section:assertion_boost_level_message `BOOST_<level>_MESSAGE`]
411
412``
413  BOOST_WARN_MESSAGE(predicate, message);
414  BOOST_CHECK_MESSAGE(predicate, message);
415  BOOST_REQUIRE_MESSAGE(predicate, message);
416``
417
418These tools perform exactly the same check as __BOOST_LEVEL__ tools. The only difference is that
419instead of generating an error/confirm message these use the supplied one.
420
421The first parameter is the boolean expression. The second parameter is the message reported in case of check
422failure. The message argument can be constructed of components of any type supporting the
423`std::ostream& operator<<(std::ostream&)`.
424
425[bt_example example38..BOOST_<level>_MESSAGE usage..run]
426
427See also:
428
429* __BOOST_LEVEL__
430
431[endsect]
432
433
434[/ ###############################################################################################]
435[section:assertion_boost_level_ne `BOOST_<level>_NE`]
436
437
438``
439  BOOST_WARN_NE(left, right);
440  BOOST_CHECK_NE(left, right);
441  BOOST_REQUIRE_NE(left, right);
442``
443
444Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left != right )`.
445The difference is that the matched values are reported as well.
446
447[bt_example example54..BOOST_<level>_NE usage..run-fail]
448
449See also:
450
451* __BOOST_LEVEL_EQUAL__
452
453[endsect]
454
455[/ ###############################################################################################]
456[section:assertion_boost_level_predicate `BOOST_<level>_PREDICATE`]
457
458
459``
460  BOOST_WARN_PREDICATE(predicate, arguments_list);
461  BOOST_CHECK_PREDICATE(predicate, arguments_list);
462  BOOST_REQUIRE_PREDICATE(predicate, arguments_list);
463``
464
465These are generic tools used to validate an arbitrary supplied predicate functor (there is a compile time limit on
466predicate arity defined by the configurable macro `BOOST_TEST_MAX_PREDICATE_ARITY`). To
467validate zero arity predicate use __BOOST_LEVEL__ tools. In other cases prefer theses tools. The
468advantage of these tools is that they show arguments values in case of predicate failure.
469
470The first parameter is the predicate itself. The second parameter is the list of predicate arguments each wrapped
471in round brackets (`BOOST_PP` sequence format).
472
473[bt_example example40..BOOST_<level>_PREDICATE usage..run]
474
475[note Note difference in error log from __BOOST_LEVEL__]
476
477See also:
478
479* __BOOST_LEVEL__
480
481[endsect]
482
483[/ ###############################################################################################]
484[section:assertion_boost_level_no_throw `BOOST_<level>_NO_THROW`]
485
486``
487  BOOST_WARN_NO_THROW(expression);
488  BOOST_CHECK_NO_THROW(expression);
489  BOOST_REQUIRE_NO_THROW(expression);
490``
491
492These assertions validate that the execution of `expression` does not throw any exception.
493To that extent, all possible exception are caught by assertion itself and no exception is propagated to
494the test body.
495
496[tip
497It is possible to test for complex expressions with the use of constructs such as `do { /* ... */} while(0)` block.
498]
499
500[bt_example exception_nothrow..BOOST_<level>_NO_THROW usage..run-fail]
501
502See also:
503
504* __BOOST_LEVEL_THROW__
505* [link boost_test.testing_tools.exception_correctness Exception correctness] section
506
507[endsect]
508
509[/ ###############################################################################################]
510[section:assertion_boost_level_throw `BOOST_<level>_THROW`]
511
512``
513  BOOST_WARN_THROW(expression, exception_type);
514  BOOST_CHECK_THROW(expression, exception_type);
515  BOOST_REQUIRE_THROW(expression, exception_type);
516``
517
518These assertions validate that the execution of `expression` raises an /expected/ exception, which means an exception of
519the supplied `exception_type` type or of any child type.
520
521* If `expression` raises an unexpected exception, this exception is not caught by `BOOST_<level>_THROW` assertion and
522  might propagate to the test body. If not caught at all, the framework will catch it and terminate the test case
523  with the status /failed/.
524* If `expression` does not raise any exception, the the assertion fails.
525
526[warning the assertion catches only the expected exceptions.]
527
528[tip
529It is possible to test for complex expressions with the use of constructs such as `do { /* ... */} while(0)` block.
530]
531
532[bt_example exception_check..BOOST_<level>_THROW usage..run-fail]
533
534See also:
535
536* __BOOST_LEVEL_NO_THROW__
537* [link boost_test.testing_tools.exception_correctness Exception correctness] section
538
539
540[endsect]
541
542[/ ###############################################################################################]
543[section:assertion_boost_level_exception `BOOST_<level>_EXCEPTION`]
544
545``
546  BOOST_WARN_EXCEPTION(expression, exception_type, predicate);
547  BOOST_CHECK_EXCEPTION(expression, exception_type, predicate);
548  BOOST_REQUIRE_EXCEPTION(expression, exception_type, predicate);
549``
550
551As for __BOOST_LEVEL_THROW__, these assertions validate that `expression` raises an exception of the
552type specified by `exception_type` or any of its child type, with additional checks on the exception instance.
553
554* If an expected exception is raised by `expression`, the instance of the exception is passed to `predicate`
555  for further validation.
556* It behaves like __BOOST_LEVEL_THROW__ if `expression` does not raise any exception, or an unrelated exception is raised.
557
558`predicate` should be a unary function accepting an instance of `exception_type` or any of its child, and that should return
559a boolean indicating a success (`true`) or a failure (`false`).
560
561[warning the assertion catches only the expected exceptions.]
562
563[tip
564It is possible to test for complex expressions with the use of constructs such as `do { /* ... */} while(0)` block.
565]
566
567
568The example below checks that the exception carries the proper error code.
569
570[bt_example exception_check_predicate..BOOST_<level>_EXCEPTION usage..run-fail]
571
572See also:
573
574* __BOOST_LEVEL_THROW__
575* [link boost_test.testing_tools.exception_correctness Exception correctness] section
576
577[endsect]
578
579[/ ###############################################################################################]
580[section:assertion_boost_level_small `BOOST_<level>_SMALL`]
581
582``
583  BOOST_WARN_SMALL(value, tolerance);
584  BOOST_CHECK_SMALL(value, tolerance);
585  BOOST_REQUIRE_SMALL(value, tolerance);
586``
587
588These tools are used to check that supplied value is small enough. The "smallness" is defined by absolute value
589of the tolerance supplied as a second argument. Use these tools with caution. To compare to values on closeness
590it's preferable to use __BOOST_LEVEL_CLOSE__ tools instead.
591
592The first parameter is the value to check. The second parameter is the tolerance.
593
594[note The floating point comparison tools are automatically added if the __UTF__
595  is included as indicated in the previous sections. The tools are implemented is in the header
596  [headerref boost/test/tools/floating_point_comparison.hpp `boost/test/tools/floating_point_comparison.hpp`].]
597
598[bt_example example41..BOOST_<level>_SMALL usage..run-fail]
599
600See also:
601
602* __BOOST_LEVEL_CLOSE__
603* __BOOST_LEVEL_CLOSE_FRACTION__
604* __floating_points_testing_tools__
605
606[endsect]
607
608
609
610[/ ###############################################################################################]
611[section:test_org_boost_test_case_expected_failure `BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES`]
612Indicates the number of failures for a test case.
613
614See [link boost_test.testing_tools.expected_failures here] for more details.
615[endsect] [/ expected failures]
616
617[/ ###############################################################################################]
618[section:assertion_boost_error `BOOST_ERROR`]
619
620``
621  BOOST_ERROR(message);
622``
623
624__BOOST_ERROR__ tool behave the same way as `__BOOST_TEST__(false, message)`. This tool is used for
625an unconditional error counter increasing and message logging.
626
627The tool's only parameter is an error message to log.
628
629[bt_example example46..BOOST_ERROR usage..run-fail]
630
631See also:
632
633* __BOOST_TEST__
634
635[endsect]
636
637
638[/ ###############################################################################################]
639[section:assertion_boost_fail `BOOST_FAIL`]
640
641``
642  BOOST_FAIL(message);
643``
644
645`__BOOST_FAIL__(message)` behave the same way as `__BOOST_TEST_REQUIRE__(false, message)`. This tool is used for an
646unconditional error counter increasing, message logging and the current test case aborting.
647
648The tool's only parameter is an error message to log.
649
650[bt_example example47..BOOST_FAIL usage..run-fail]
651
652See also:
653
654* __BOOST_TEST__
655*
656
657[endsect]
658
659
660[/ ###############################################################################################]
661[section:assertion_boost_is_defined `BOOST_IS_DEFINED`]
662
663``
664  BOOST_IS_DEFINED(symbol);
665``
666
667Unlike the rest of the tools in the toolbox this tool does not perform the logging itself. Its only purpose
668is to check at runtime whether or not the supplied preprocessor symbol is defined. Use it in combination with
669__BOOST_LEVEL__ to perform and log validation. Macros of any arity could be checked. To check the
670macro definition with non-zero arity specify dummy arguments for it. See below for example.
671
672The only tool's parameter is a preprocessor symbol that gets validated.
673
674[bt_example example48..BOOST_IS_DEFINED usage..run-fail]
675
676See also:
677
678* __BOOST_LEVEL__
679
680[endsect]
681
682[/ ###############################################################################################]
683[section:assertion_control_under_debugger `BOOST_TEST_TOOLS_UNDER_DEBUGGER`]
684When defined, assertions evaluate their expression eagerly, as described [link boost_test.testing_tools.debugging here].
685[endsect] [/ assertion_control_under_debugger]
686
687[/ ###############################################################################################]
688[section:assertion_control_under_debuggable `BOOST_TEST_TOOLS_DEBUGGABLE`]
689When defined, test assertions are compiled in two modes (debugger-friendly and full-featured) and the version is selected at run-time, as described [link boost_test.testing_tools.debugging here].
690[endsect] [/ assertion_control_under_debuggable]
691
692[endsect] [/ testing_tool_ref]
693