1[/
2 / Copyright (c) 2003 Boost.Test team
3 /
4 / Distributed under the Boost Software License, Version 1.0. (See accompanying
5 / file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
6 /]
7
8
9[section:testing_tool_ref Reference API for writing tests]
10
11
12
13[/ ###############################################################################################]
14[section:assertion_boost_test_universal_macro `BOOST_TEST`]
15
16
17``
18  BOOST_TEST(statement);
19  BOOST_TEST_<level>(statement);
20
21  // replacement failure message, requires variadic macros
22  BOOST_TEST(statement, "failure message");
23
24  // Floating point comparison, requires variadic macros, auto and decltype
25  BOOST_TEST(statement, floating_point_comparison_manipulation);
26
27  // bitwise comparison, requires variadic macros, auto and decltype
28  BOOST_TEST(statement, boost::test_tools::bitwise() );
29
30  // element-wise comparison, for containers
31  BOOST_TEST(statement, boost::test_tools::per_element() );
32
33  // lexicographic comparison, for containers
34  BOOST_TEST(statement, boost::test_tools::lexicographic() );
35``
36
37The full documentation of this macro is located [link boost_test.testing_tools.boost_test_universal_macro here].
38
39The macro is available in three variants, corresponding to different [link boost_test.testing_tools.tools_assertion_severity_level assertion severity levels]:
40
41``
42  BOOST_TEST // or BOOST_TEST_CHECK
43  BOOST_TEST_REQUIRE
44  BOOST_TEST_WARN
45``
46
47
48* `"failure message"` is a C-string printed in case of failure in place of the default message.
49  See [link boost_test.testing_tools.reports this section] for
50  more details.
51* `floating_point_comparison_manipulation` is one of the floating point comparison manipulators.
52  See [link boost_test.testing_tools.boost_test_universal_macro this section]
53   for more details.
54* [classref boost::test_tools::bitwise] is a manipulator indicating that the comparison should be performed bitwise. See
55  [link boost_test.testing_tools.extended_comparison.bitwise this section] for more details
56* [classref boost::test_tools::per_element] is a manipulator indicating that the comparison should be performed on each element, in sequence, rather
57  than on containers. See
58  [link boost_test_coll_perelement this section] for more details
59* [classref boost::test_tools::lexicographic] is a manipulator indicating that the comparison should be performed with the lexicographic order. See
60  [link boost_test_coll_default_lex this section] for more details
61
62[h3 Limitations and workaround]
63There are some restrictions on the statements that are supported by this tool. Those are explained in details in
64[link boost_test_statement_limitations this] section.
65
66[endsect]
67
68
69
70[/ DECORATORS ###############################################################################################]
71[/-----------------------------------------------------------------]
72
73[section:decorator_expected_failures expected_failures (decorator)]
74
75``
76expected_failures(counter_t number);
77``
78
79Indicates the expected failures for a test unit.
80See [link boost_test.testing_tools.expected_failures here] for more details.
81
82[endsect] [/ section expected_failures]
83
84
85[/-----------------------------------------------------------------]
86[section:decorator_timeout timeout (decorator)]
87
88``
89timeout(unsigned seconds);
90``
91
92Specifies a time-out for a *test-case*, above which the test-case is forced to stop and reported as failing.
93See [link boost_test.testing_tools.timeout here] for more details.
94
95[endsect] [/ section timeout]
96
97
98[/-----------------------------------------------------------------]
99[section:decorator_tolerance tolerance (decorator)]
100
101``
102template <typename FPT>
103  tolerance(FPT eps);
104
105template <typename FPT>
106  tolerance(test_tools::fpc::percent_tolerance_t<FPT> eps)
107``
108
109Decorator `tolerance` specifies the default comparison tolerance for floating point type `FTP` in the decorated test
110unit. The default tolerance only applies to a particular type, so it makes sense to provide more than one `tolerance`
111decorator if we are comparing different floating point types. For more details see __floating_points_testing_impl__.
112The variant with `percent_tolerance` uses value `eps / 100` as tolerance.
113
114[bt_example decorator_13..decorator tolerance..run-fail]
115
116In the above example,  in `test1`, checks on `double`s fail because they differ by more what tolerance for `double`s
117specifies. In `test2` the tolerance for `double`s is greater and therefore the checks succeed. In `test3`, we specify
118only tolerance for type `float`, and since the checks use type `double` the specified tolerance does not apply. Tolerance
119in `test4` is equivalent to that in `test1`, therefore its checks also fail. Tolerance in `test5` is equivalent to
120that in `test2`, therefore its checks also succeed.
121[endsect] [/ section decorator_tolerance]
122
123
124
125
126
127[/ DEPRECATED API]
128[/ ###############################################################################################]
129[#ref_BOOST_level][section:assertion_boost_level `BOOST_<level>`]
130
131
132``
133  BOOST_WARN(predicate);
134  BOOST_CHECK(predicate);
135  BOOST_REQUIRE(predicate);
136``
137
138These tools are used to validate the predicate value. The only parameter for these tools is a boolean predicate
139value that gets validated. It could be any expression that could be evaluated and converted to boolean value. The
140expression gets evaluated only once, so it's safe to pass complex expression for validation.
141
142[bt_example example34..BOOST_<level> usage..run-fail]
143
144See also:
145
146* __BOOST_LEVEL_MESSAGE__
147
148[endsect]
149
150
151[/ ###############################################################################################]
152[section:assertion_boost_level_bitwise_eq `BOOST_<level>_BITWISE_EQUAL`]
153
154
155``
156  BOOST_WARN_BITWISE_EQUAL(left, right);
157  BOOST_CHECK_BITWISE_EQUAL(left, right);
158  BOOST_REQUIRE_BITWISE_EQUAL(left, right);
159``
160
161These tools are used to perform bitwise comparison of two values. The check shows all positions where left and
162right value's bits mismatch.
163
164The first parameter is the left compared value. The second parameter is the right compared value. Parameters are
165not required to be of the same type, but warning is issued if their type's size does not coincide.
166
167[bt_example example33..BOOST_<level>_BITWISE_EQUAL usage..run-fail]
168
169See also:
170
171* __BOOST_LEVEL_EQUAL__
172
173[endsect]
174
175[/ ###############################################################################################]
176[section:assertion_boost_level_eq `BOOST_<level>_EQUAL`]
177
178``
179  BOOST_WARN_EQUAL(left, right);
180  BOOST_CHECK_EQUAL(left, right);
181  BOOST_REQUIRE_EQUAL(left, right);
182``
183
184Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__(left == right)`.
185The difference is that the mismatched values are reported as well.
186
187[note It is bad idea to use these tools to compare floating point values. Use __BOOST_LEVEL_CLOSE__ or
188      __BOOST_LEVEL_CLOSE_FRACTION__ tools instead.
189]
190
191[bt_example example35..BOOST_<level>_EQUAL usage..run-fail]
192
193See also:
194
195* __BOOST_LEVEL__
196* __BOOST_LEVEL_CLOSE__
197* __BOOST_LEVEL_NE__
198* __BOOST_LEVEL_EQUAL_COLLECTIONS__
199
200[endsect]
201
202[/ ###############################################################################################]
203[section:assertion_boost_level_eq_collections `BOOST_<level>_EQUAL_COLLECTIONS`]
204
205``
206  BOOST_WARN_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
207  BOOST_CHECK_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
208  BOOST_REQUIRE_EQUAL_COLLECTIONS(left_begin, left_end, right_begin, right_end);
209``
210
211These tools are used to perform an element by element comparison of two collections. They print all mismatched
212positions, collection elements at these positions and check that the collections have the same size. The first two
213parameters designate begin and end of the first collection. The two last parameters designate begin and end of the
214second collection.
215
216[bt_example example36..BOOST_<level>_EQUAL_COLLECTIONS usage..run-fail]
217
218See also:
219
220* __BOOST_LEVEL_EQUAL__
221
222[endsect]
223
224[/ ###############################################################################################]
225[section:assertion_boost_level_close `BOOST_<level>_CLOSE`]
226
227``
228  BOOST_WARN_CLOSE(left, right, tolerance);
229  BOOST_CHECK_CLOSE(left, right, tolerance);
230  BOOST_REQUIRE_CLOSE(left, right, tolerance);
231``
232
233These tools are used to check on closeness using strong relationship defined by the predicate
234``check_is_close( left, right, tolerance )``
235
236To check for the weak relationship use
237__BOOST_LEVEL_PREDICATE__ family of tools with explicit `check_is_close` invocation.
238
239
240The first parameter is the ['left] compared value. The second parameter is the
241['right] compared value. Last third parameter defines the tolerance for the comparison in
242[link boost_test.testing_tools.extended_comparison.floating_point [*percentage units]].
243
244[note It is required for left and right parameters to be of the same floating point type. You will need to explicitly
245      resolve any type mismatch to select which type to use for comparison.
246]
247
248[note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.
249]
250
251[bt_example example42..BOOST_<level>_CLOSE usage with small values..run-fail]
252[bt_example example43..BOOST_<level>_CLOSE usage with big values..run]
253
254See also:
255
256* __BOOST_LEVEL_CLOSE_FRACTION__
257* __BOOST_LEVEL_SMALL__
258* __BOOST_LEVEL_EQUAL__
259* __floating_points_testing_tools__
260
261[endsect]
262
263[/ ###############################################################################################]
264[section:assertion_boost_level_close_fraction `BOOST_<level>_CLOSE_FRACTION`]
265
266``
267  BOOST_WARN_CLOSE_FRACTION(left, right, tolerance);
268  BOOST_CHECK_CLOSE_FRACTION(left, right, tolerance);
269  BOOST_REQUIRE_CLOSE_FRACTION(left, right, tolerance);
270``
271
272These tools are used to check on closeness using strong relationship defined by the predicate
273``check_is_close(left, right, tolerance)``
274
275To check for the weak relationship use __BOOST_LEVEL_PREDICATE__ family of tools with explicit `check_is_close` invocation.
276
277The first parameter is the ['left] compared value. The second parameter is the
278['right] compared value. Last third parameter defines the tolerance for the comparison as
279[link boost_test.testing_tools.extended_comparison.floating_point [*fraction of absolute values being compared]].
280
281[note It is required for left and right parameters to be of the same floating point type. You will need to explicitly
282      resolve any type mismatch to select which type to use for comparison.]
283
284[note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.]
285
286[bt_example example44..BOOST_<level>_CLOSE_FRACTION usage..run-fail]
287
288See also:
289
290* __BOOST_LEVEL_CLOSE__
291* __BOOST_LEVEL_SMALL__
292* __BOOST_LEVEL_EQUAL__
293* __floating_points_testing_tools__
294
295[endsect]
296
297[/ ###############################################################################################]
298[section:assertion_boost_level_exception `BOOST_<level>_EXCEPTION`]
299
300``
301  BOOST_WARN_EXCEPTION(expression, exception, predicate);
302  BOOST_CHECK_EXCEPTION(expression, exception, predicate);
303  BOOST_REQUIRE_EXCEPTION(expression, exception, predicate);
304``
305
306These tools are used to perform an exception detection and validation check. Tools execute the supplied expression
307and validate that it throws an exception of supplied class (or the one derived from it) that complies with the
308supplied predicate. If the expression throws any other unrelated exception, doesn't throw at all or
309predicate evaluates to false, check fails. In comparison with __BOOST_LEVEL_THROW__ tools these
310allow performing more fine-grained checks. For example: make sure that an expected exception has specific
311error message.
312
313[bt_example example37..BOOST_<level>_EXCEPTION usage..run-fail]
314
315See also:
316
317* __BOOST_LEVEL_THROW__
318
319[endsect]
320
321[/ ###############################################################################################]
322[section:assertion_boost_level_ge `BOOST_<level>_GE`]
323
324``
325  BOOST_WARN_GE(left, right);
326  BOOST_CHECK_GE(left, right);
327  BOOST_REQUIRE_GE(left, right);
328``
329
330Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left >= right )`.
331The difference is that the argument values are reported as well.
332
333[bt_example example57..BOOST_<level>_GE usage..run-fail]
334
335See also:
336
337* __BOOST_LEVEL_LE__
338* __BOOST_LEVEL_LT__
339* __BOOST_LEVEL_GT__
340
341[endsect]
342
343
344[/ ###############################################################################################]
345[section:assertion_boost_level_gt `BOOST_<level>_GT`]
346
347
348``
349  BOOST_WARN_GT(left, right);
350  BOOST_CHECK_GT(left, right);
351  BOOST_REQUIRE_GT(left, right);
352``
353
354Check performed by these tools is the same as the one performed by __BOOST_LEVEL__`( left > right )`.
355The difference is that the argument values are reported as well.
356
357[bt_example example58..BOOST_<level>_GT usage..run-fail]
358
359See also:
360
361* __BOOST_LEVEL_LE__
362* __BOOST_LEVEL_LT__
363* __BOOST_LEVEL_GE__
364
365[endsect]
366
367[/ ###############################################################################################]
368[section:assertion_boost_level_le `BOOST_<level>_LE`]
369
370``
371  BOOST_WARN_LE(left, right);
372  BOOST_CHECK_LE(left, right);
373  BOOST_REQUIRE_LE(left, right);
374``
375
376Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left <= right )`.
377The difference is that the argument values are reported as well.
378
379[bt_example example55..BOOST_<level>_LE usage..run-fail]
380
381See also:
382
383* __BOOST_LEVEL_LE__
384* __BOOST_LEVEL_GE__
385* __BOOST_LEVEL_GT__
386
387[endsect]
388
389[/ ###############################################################################################]
390[section:assertion_boost_level_lt `BOOST_<level>_LT`]
391
392``
393  BOOST_WARN_LT(left, right);
394  BOOST_CHECK_LT(left, right);
395  BOOST_REQUIRE_LT(left, right);
396``
397
398Check performed by these tools is the same as the one performed by `__BOOST_LEVEL__( left < right )`.
399The difference is that the argument values are reported as well.
400
401[bt_example example56..BOOST_<level>_LT usage..run-fail]
402
403See also:
404
405* __BOOST_LEVEL_LE__
406* __BOOST_LEVEL_GE__
407* __BOOST_LEVEL_GT__
408
409[endsect]
410
411[/ ###############################################################################################]
412[section:assertion_boost_level_message `BOOST_<level>_MESSAGE`]
413
414``
415  BOOST_WARN_MESSAGE(predicate, message);
416  BOOST_CHECK_MESSAGE(predicate, message);
417  BOOST_REQUIRE_MESSAGE(predicate, message);
418``
419
420These tools perform exactly the same check as __BOOST_LEVEL__ tools. The only difference is that
421instead of generating an error/confirm message these use the supplied one.
422
423The first parameter is the boolean expression. The second parameter is the message reported in case of check
424failure. The message argument can be constructed of components of any type supporting the
425`std::ostream& operator<<(std::ostream&)`.
426
427[bt_example example38..BOOST_<level>_MESSAGE usage..run]
428
429See also:
430
431* __BOOST_LEVEL__
432
433[endsect]
434
435
436[/ ###############################################################################################]
437[section:assertion_boost_level_ne `BOOST_<level>_NE`]
438
439
440``
441  BOOST_WARN_NE(left, right);
442  BOOST_CHECK_NE(left, right);
443  BOOST_REQUIRE_NE(left, right);
444``
445
446Check performed by these tools is the same as the one performed by `__BOOST_<level>__( left != right )`.
447The difference is that the matched values are reported as well.
448
449[bt_example example54..BOOST_<level>_NE usage..run-fail]
450
451See also:
452
453* __BOOST_LEVEL_EQUAL__
454
455[endsect]
456
457[/ ###############################################################################################]
458[section:assertion_boost_level_no_throw `BOOST_<level>_NO_THROW`]
459
460
461``
462  BOOST_WARN_NO_THROW(expression);
463  BOOST_CHECK_NO_THROW(expression);
464  BOOST_REQUIRE_NO_THROW(expression);
465``
466
467These tools are used to perform a "no throw" check. Tools execute the supplied expression and validate that it does
468not throw any exceptions. Error would be reported by the framework even if the statement appear directly in test
469case body and throw any exception. But these tools allow proceeding further with test case in case of failure.
470
471If check is successful, tools may produce a confirmation message, in other case they produce an error message in
472a form ``error in <test-case-name>;exception was thrown by <expression>``
473
474The only parameter is an expression to execute. You can use `do {} while(0)` block if you want to execute more than one
475statement.
476
477[bt_example example39..BOOST_<level>_NO_THROW usage..run-fail]
478
479See also:
480
481* __BOOST_LEVEL_THROW__
482
483[endsect]
484
485[/ ###############################################################################################]
486[section:assertion_boost_level_predicate `BOOST_<level>_PREDICATE`]
487
488
489``
490  BOOST_WARN_PREDICATE(predicate, arguments_list);
491  BOOST_CHECK_PREDICATE(predicate, arguments_list);
492  BOOST_REQUIRE_PREDICATE(predicate, arguments_list);
493``
494
495These are generic tools used to validate an arbitrary supplied predicate functor (there is a compile time limit on
496predicate arity defined by the configurable macro `BOOST_TEST_MAX_PREDICATE_ARITY`). To
497validate zero arity predicate use __BOOST_<level>__ tools. In other cases prefer theses tools. The
498advantage of these tools is that they show arguments values in case of predicate failure.
499
500The first parameter is the predicate itself. The second parameter is the list of predicate arguments each wrapped
501in round brackets (`BOOST_PP` sequence format).
502
503[bt_example example40..BOOST_<level>_PREDICATE usage..run]
504
505[note Note difference in error log from __BOOST_<level>__]
506
507See also:
508
509* __BOOST_LEVEL__
510
511[endsect]
512
513[/ ###############################################################################################]
514[section:assertion_boost_level_small `BOOST_<level>_SMALL`]
515
516``
517  BOOST_WARN_SMALL(value, tolerance);
518  BOOST_CHECK_SMALL(value, tolerance);
519  BOOST_REQUIRE_SMALL(value, tolerance);
520``
521
522These tools are used to check that supplied value is small enough. The "smallness" is defined by absolute value
523of the tolerance supplied as a second argument. Use these tools with caution. To compare to values on closeness
524it's preferable to use __BOOST_LEVEL_CLOSE__ tools instead.
525
526The first parameter is the value to check. The second parameter is the tolerance.
527
528[note Note that to use these tools you need to include additional header `floating_point_comparison.hpp`.]
529
530[bt_example example41..BOOST_<level>_SMALL usage..run-fail]
531
532See also:
533
534* __BOOST_LEVEL_CLOSE__
535* __BOOST_LEVEL_CLOSE_FRACTION__
536* __floating_points_testing_tools__
537
538[endsect]
539
540[/ ###############################################################################################]
541[section:assertion_boost_level_throw `BOOST_<level>_THROW`]
542
543
544``
545  BOOST_WARN_THROW(expression, exception);
546  BOOST_CHECK_THROW(expression, exception);
547  BOOST_REQUIRE_THROW(expression, exception);
548``
549
550These tools are used to perform an exception detection check. Tools execute the supplied expression and validate
551that it throws an exception of supplied class (or the one derived from it) or it's child. If the statement
552throws any other unrelated exception or doesn't throw at all, check fails.
553
554If check is successful, the tool produces a confirmation message, in other case it produces an error message in a
555form
556``
557error in <test-case-name>: exception <exception> expected
558``
559
560The first parameter is the expression to execute. Use `do{} while(0)` block if you want to execute more than one
561statement. The second parameter is an expected exception.
562
563[bt_example example45..BOOST_<level>_THROW usage..run-fail]
564
565See also:
566
567* __BOOST_LEVEL_NO_THROW__
568
569[endsect]
570
571
572[/ ###############################################################################################]
573[section:test_org_boost_test_case_expected_failure `BOOST_AUTO_TEST_CASE_EXPECTED_FAILURES`]
574Indicates the number of failures for a test case.
575
576See [link boost_test.testing_tools.expected_failures here] for more details.
577[endsect] [/ expected failures]
578
579[/ ###############################################################################################]
580[section:assertion_boost_error `BOOST_ERROR`]
581
582``
583  BOOST_ERROR(message);
584``
585
586__BOOST_ERROR__ tool behave the same way as `__BOOST_TEST__(false, message)`. This tool is used for
587an unconditional error counter increasing and message logging.
588
589The tool's only parameter is an error message to log.
590
591[bt_example example46..BOOST_ERROR usage..run-fail]
592
593See also:
594
595* __BOOST_TEST__
596
597[endsect]
598
599
600[/ ###############################################################################################]
601[section:assertion_boost_fail `BOOST_FAIL`]
602
603``
604  BOOST_FAIL(message);
605``
606
607`__BOOST_FAIL__(message)` behave the same way as `__BOOST_TEST_REQUIRE__(false, message)`. This tool is used for an
608unconditional error counter increasing, message logging and the current test case aborting.
609
610The tool's only parameter is an error message to log.
611
612[bt_example example47..BOOST_FAIL usage..run-fail]
613
614See also:
615
616* __BOOST_TEST__
617*
618
619[endsect]
620
621
622[/ ###############################################################################################]
623[section:assertion_boost_is_defined `BOOST_IS_DEFINED`]
624
625``
626  BOOST_IS_DEFINED(symbol);
627``
628
629Unlike the rest of the tools in the toolbox this tool does not perform the logging itself. Its only purpose
630is to check at runtime whether or not the supplied preprocessor symbol is defined. Use it in combination with
631__BOOST_<level>__ to perform and log validation. Macros of any arity could be checked. To check the
632macro definition with non-zero arity specify dummy arguments for it. See below for example.
633
634The only tool's parameter is a preprocessor symbol that gets validated.
635
636[bt_example example48..BOOST_IS_DEFINED usage..run-fail]
637
638See also:
639
640* __BOOST_LEVEL__
641
642[endsect]
643
644[/ ###############################################################################################]
645[section:assertion_control_under_debugger `BOOST_TEST_TOOLS_UNDER_DEBUGGER`]
646When defined, assertions evaluate their expression eagerly, as described [link boost_test.testing_tools.debugging here].
647[endsect] [/ assertion_control_under_debugger]
648
649[/ ###############################################################################################]
650[section:assertion_control_under_debuggable `BOOST_TEST_TOOLS_DEBUGGABLE`]
651When defined, test assertions are compiled in two modes (debugger-friendly and full-featured) and the version is selected at run-time, as described [link boost_test.testing_tools.debugging here].
652[endsect] [/ assertion_control_under_debuggable]
653
654[endsect] [/ testing_tool_ref]
655