1 2 -*- html -*- 3 4 EUnit overview page 5 6@title EUnit - a Lightweight Unit Testing Framework for Erlang 7 8@author Richard Carlsson <carlsson.richard@gmail.com> 9@author Mickaël Rémond <mickael.remond@process-one.net> 10 [http://www.process-one.net/] 11@copyright 2004-2007 Mickaël Rémond, Richard Carlsson 12@version {@version}, {@date} {@time} 13 14@doc EUnit is a unit testing framework for Erlang. It is very powerful 15and flexible, is easy to use, and has small syntactical overhead. 16 17<ul> 18<li>{@section Unit testing}</li> 19<li>{@section Terminology}</li> 20<li>{@section Getting started}</li> 21<li>{@section EUnit macros}</li> 22<li>{@section EUnit test representation}</li> 23</ul> 24 25EUnit builds on ideas from the family of unit testing frameworks for 26Object Oriented languages that originated with JUnit by Beck and Gamma 27(and Beck's previous framework SUnit for Smalltalk). However, EUnit uses 28techniques more adapted to functional and concurrent programming, and is 29typically less verbose than its relatives. 30 31Although EUnit uses many preprocessor macros, they have been designed to 32be as nonintrusive as possible, and should not cause conflicts with 33existing code. Adding EUnit tests to a module should thus not normally 34require changing existing code. Furthermore, tests that only exercise 35the exported functions of a module can always be placed in a completely 36separate module, avoiding any conflicts entirely. 37 38== Unit testing == 39 40Unit Testing is testing of individual program "units" in relative 41isolation. There is no particular size requirement: a unit can be a 42function, a module, a process, or even a whole application, but the most 43typical testing units are individual functions or modules. In order to 44test a unit, you specify a set of individual tests, set up the smallest 45necessary environment for being able to run those tests (often, you 46don't need to do any setup at all), you run the tests and collect the 47results, and finally you do any necessary cleanup so that the test can 48be run again later. A Unit Testing Framework tries to help you in each 49stage of this process, so that it is easy to write tests, easy to run 50them, and easy to see which tests failed (so you can fix the bugs). 51 52=== Advantages of unit testing === 53 54<dl> 55 <dt>Reduces the risks of changing the program</dt> 56 <dd>Most programs will be modified during their lifetime: bugs will be 57 fixed, features will be added, optimizations may become necessary, or 58 the code will need to be refactored or cleaned up in other ways to 59 make it easier to work with. But every change to a working program is 60 a risk of introducing new bugs - or reintroducing bugs that had 61 previously been fixed. Having a set of unit tests that you can run 62 with very little effort makes it easy to know that the code still 63 works as it should (this use is called <em>regression testing</em>; 64 see {@section Terminology}). This goes a long way to reduce the 65 resistance to changing and refactoring code.</dd> 66 <dt>Helps guide and speed up the development process</dt> 67 <dd>By focusing on getting the code to pass the tests, the programmer 68 can become more productive, not overspecify or get lost in premature 69 optimizations, and create code that is correct from the very beginning 70 (so-called <em>test-driven development</em>; see {@section 71 Terminology}).</dd> 72 <dt>Helps separate interface from implementation</dt> 73 <dd>When writing tests, the programmer may discover dependencies 74 (in order to get the tests to run) that ought not to be there, and 75 which need to be abstracted away to get a cleaner design. This helps 76 eliminate bad dependencies before they spread throughout the 77 code.</dd> 78 <dt>Makes component integration easier</dt> 79 <dd>By testing in a bottom-up fashion, beginning with the smallest 80 program units and creating a confidence in that they work as they 81 should, it becomes easier to test that a higher-level component, 82 consisting of several such units, also behaves according to 83 specification (known as <em>integration testing</em>; see {@section 84 Terminology}).</dd> 85 <dt>Is self-documenting</dt> 86 <dd>The tests can be read as documentation, typically showing both 87 examples of correct and incorrect usage, along with the expected 88 consequences.</dd> 89</dl> 90 91== Terminology == 92 93<dl> 94 <dt>Unit testing</dt> 95 <dd>Testing that a program unit behaves as it is supposed to do (in 96 itself), according to its specifications. Unit tests have an important 97 function as regression tests, when the program later is modified for 98 some reason, since they check that the program still behaves according 99 to specification.</dd> 100 <dt>Regression testing</dt> 101 <dd>Running a set of tests after making changes to a program, to check 102 that the program behaves as it did before the changes (except, of 103 course, for any intentional changes in behaviour). Unit tests are 104 important as regression tests, but regression testing can involve more 105 than just unit testing, and may also test behaviour that might not be 106 part of the normal specification (such as bug-for-bug-compatibility). 107 </dd> 108 <dt>Integration testing</dt> 109 <dd>Testing that a number of individually developed program units 110 (assumed to already have been separately unit tested) work together as 111 expected. Depending on the system being developed, integration testing 112 may be as simple as "just another level of unit testing", but might 113 also involve other kinds of tests (compare <em>system testing</em>). 114</dd> 115 <dt>System testing</dt> 116 <dd>Testing that a complete system behaves according to its 117 specification. Specifically, system testing should not require knowing 118 any details about the implementation. It typically involves testing 119 many different aspects of the system behaviour apart from the basic 120 functionality, such as performance, usability, and reliability.</dd> 121 <dt>Test-driven development</dt> 122 <dd>A program development technique where you continuously write tests 123 <em>before</em> you implement the code that is supposed to pass those 124 tests. This can help you focus on solving the right problems, and not 125 make a more complicated implementation than necessary, by letting the 126 unit tests determine when a program is "done": if it fulfils its 127 specifications, there is no need to keep adding functionality.</dd> 128 <dt>Mock object</dt> 129 <dd>Sometimes, testing some unit `A' (e.g., a function) requires that 130 it collaborates somehow with some other unit `B' (perhaps being passed 131 as an argument, or by reference) - but `B' has not been implemented 132 yet. A "mock object" - an object which, for the purposes of testing 133 `A', looks and behaves like a real `B' - might then be used instead. 134 (This is of course only useful if it would be significantly more work 135 to implement a real `B' than to create a mock object.)</dd> 136 <dt>Test case</dt> 137 <dd>A single, well-defined test, that somehow can be uniquely 138 identified. When executed, the test case either <em>passes</em> or 139 <em>fails</em>; the test report should identify exactly which test 140 cases failed.</dd> 141 <dt>Test suite</dt> 142 <dd>A collection of test cases, generally with a specific, common 143 target for testing, such as a single function, module, or subsystem. A 144 test suite may also be recursively composed by smaller test 145 suites.</dd> 146</dl> 147 148== Getting started == 149<ul> 150 <li>{@section Including the EUnit header file}</li> 151 <li>{@section Writing simple test functions}</li> 152 <li>{@section Running EUnit}</li> 153 <li>{@section Writing test generating functions}</li> 154 <li>{@section An example}</li> 155 <li>{@section Disabling testing}</li> 156 <li>{@section Avoiding compile-time dependency on EUnit}</li> 157</ul> 158 159=== Including the EUnit header file === 160 161The simplest way to use EUnit in an Erlang module is to add the 162following line at the beginning of the module (after the `-module' 163declaration, but before any function definitions): 164```-include_lib("eunit/include/eunit.hrl").''' 165 166This will have the following effect: 167<ul> 168 <li>Creates an exported function `test()' (unless testing is turned 169 off, and the module does not already contain a test() function), that 170 can be used to run all the unit tests defined in the module</li> 171 <li>Causes all functions whose names match `..._test()' or `..._test_()' 172 to be automatically exported from the module (unless testing is 173 turned off, or the `EUNIT_NOAUTO' macro is defined)</li> 174 <li>Makes all the preprocessor macros of EUnit available, to help 175 writing tests</li> 176</ul> 177 178<strong>Note:</strong> For `-include_lib(...)' to work, the Erlang 179module search path <em>must</em> contain a directory whose name ends in 180`eunit/ebin' (pointing to the `ebin' subdirectory of the EUnit 181installation directory). If EUnit is installed as `lib/eunit' under your 182Erlang/OTP system directory, its `ebin' subdirectory will be 183automatically added to the search path when Erlang starts. Otherwise, 184you need to add the directory explicitly, by passing a `-pa' flag to the 185`erl' or `erlc' command. For example, a Makefile could contain the 186following action for compiling `.erl' files: 187```erlc -pa "path/to/eunit/ebin" $(ERL_COMPILE_FLAGS) -o$(EBIN) $<''' 188or if you want Eunit to always be available when you run Erlang 189interactively, you can add a line like the following to your 190`$HOME/.erlang' file: 191```code:add_path("/path/to/eunit/ebin").''' 192 193=== Writing simple test functions === 194 195The EUnit framework makes it extremely easy to write unit tests in 196Erlang. There are a few different ways of writing them, though, so we 197start with the simplest: 198 199A function with a name ending in `..._test()' is recognized by EUnit as 200a simple test function - it takes no arguments, and its execution either 201succeeds (returning some arbitrary value that EUnit will throw away), or 202fails by throwing an exception of some kind (or by not terminating, in 203which case it will be aborted after a while). 204 205An example of a simple test function could be the following: 206```reverse_test() -> lists:reverse([1,2,3]).''' 207This just tests that the function `lists:reverse(List)' does not crash 208when `List' is `[1,2,3]'. It is not a great test, but many people write 209simple functions like this one to test the basic functionality of their 210code, and those tests can be used directly by EUnit, without changes, 211as long as their function names match. 212 213==== Use exceptions to signal failure ==== 214 215To write more interesting tests, we need to make them crash (throw an 216exception) when they don't get the result they expect. A simple way of 217doing this is to use pattern matching with `=', as in the following 218examples: 219```reverse_nil_test() -> [] = lists:reverse([]). 220 reverse_one_test() -> [1] = lists:reverse([1]). 221 reverse_two_test() -> [2,1] = lists:reverse([1,2]). 222''' 223If there was some bug in `lists:reverse/1' that made it return something 224other than `[2,1]' when it got `[1,2]' as input, then the last test 225above would throw a `badmatch' error. The first two (we assume they do 226not get a `badmatch') would simply return `[]' and `[1]', respectively, 227so both succeed. (Note that EUnit is not psychic: if you write a test 228that returns a value, even if it is the wrong value, EUnit will consider 229it a success. You must make sure that the test is written so that it 230causes a crash if the result is not what it should be.) 231 232==== Using assert macros ==== 233 234If you want to use Boolean operators for your tests, the `assert' 235macro comes in handy (see {@section EUnit macros} for details): 236```length_test() -> ?assert(length([1,2,3]) =:= 3).''' 237The `?assert(Expression)' macro will evaluate `Expression', and if that 238does not evaluate to `true', it will throw an exception; otherwise it 239just returns `ok'. In the above example, the test will thus fail if the 240call to `length' does not return 3. 241 242=== Running EUnit === 243 244If you have added the declaration 245`-include_lib("eunit/include/eunit.hrl")' to your module, as described 246above, you only need to compile the module, and run the automatically 247exported function `test()'. For example, if your module was named `m', 248then calling `m:test()' will run EUnit on all the tests defined in the 249module. You do not need to write `-export' declarations for the test 250functions. This is all done by magic. 251 252You can also use the function {@link eunit:test/1} to run arbitrary 253tests, for example to try out some more advanced test descriptors (see 254{@section EUnit test representation}). For example, running 255``eunit:test(m)'' does the same thing as the auto-generated function 256``m:test()'', while ``eunit:test({inparallel, m})'' runs the same test 257cases but executes them all in parallel. 258 259==== Putting tests in separate modules ==== 260 261If you want to separate your test code from your normal code (at least 262for testing the exported functions), you can simply write the test 263functions in a module named `m_tests' (note: not `m_test'), if your 264module is named `m'. Then, whenever you ask EUnit to test the module 265`m', it will also look for the module `m_tests' and run those tests as 266well. See `ModuleName' in the section {@section Primitives} for details. 267 268==== EUnit captures standard output ==== 269 270If your test code writes to the standard output, you may be surprised to 271see that the text does not appear on the console when the tests are 272running. This is because EUnit captures all standard output from test 273functions (this also includes setup and cleanup functions, but not 274generator functions), so that it can be included in the test report if 275errors occur. To bypass EUnit and print text directly to the console 276while testing, you can write to the `user' output stream, as in 277`io:format(user, "~w", [Term])'. The recommended way of doing this is to 278use the EUnit {@section Debugging macros}, which make it much simpler. 279 280=== Writing test generating functions === 281 282A drawback of simple test functions is that you must write a separate 283function (with a separate name) for each test case. A more compact way 284of writing tests (and much more flexible, as we shall see), is to write 285functions that <em>return</em> tests, instead of <em>being</em> tests. 286 287A function with a name ending in `..._test_()' (note the final 288underscore) is recognized by EUnit as a <em>test generator</em> 289function. Test generators return a <em>representation</em> of a <em>set 290of tests</em> to be executed by EUnit. 291 292==== Representing a test as data ==== 293 294The most basic representation of a test is a single fun-expression that 295takes no arguments. For example, the following test generator: 296```basic_test_() -> 297 fun () -> ?assert(1 + 1 =:= 2) end.''' 298will have the same effect as the following simple test: 299```simple_test() -> 300 ?assert(1 + 1 =:= 2).''' 301(in fact, EUnit will handle all simple tests just like it handles 302fun-expressions: it will put them in a list, and run them one by one). 303 304==== Using macros to write tests ==== 305 306To make tests more compact and readable, as well as automatically add 307information about the line number in the source code where a test 308occurred (and reduce the number of characters you have to type), you can 309use the `_test' macro (note the initial underscore character), like 310this: 311```basic_test_() -> 312 ?_test(?assert(1 + 1 =:= 2)).''' 313The `_test' macro takes any expression (the "body") as argument, and 314places it within a fun-expression (along with some extra information). 315The body can be any kind of test expression, just like the body of a 316simple test function. 317 318==== Underscore-prefixed macros create test objects ==== 319 320But this example can be made even shorter! Most test macros, such as the 321family of `assert' macros, have a corresponding form with an initial 322underscore character, which automatically adds a `?_test(...)' wrapper. 323The above example can then simply be written: 324```basic_test_() -> 325 ?_assert(1 + 1 =:= 2).''' 326which has exactly the same meaning (note the `_assert' instead of 327`assert'). You can think of the initial underscore as signalling 328<em>test object</em>. 329 330=== An example === 331 332Sometimes, an example says more than a thousand words. The following 333small Erlang module shows how EUnit can be used in practice. 334```-module(fib). 335 -export([fib/1]). 336 -include_lib("eunit/include/eunit.hrl"). 337 338 fib(0) -> 1; 339 fib(1) -> 1; 340 fib(N) when N > 1 -> fib(N-1) + fib(N-2). 341 342 fib_test_() -> 343 [?_assert(fib(0) =:= 1), 344 ?_assert(fib(1) =:= 1), 345 ?_assert(fib(2) =:= 2), 346 ?_assert(fib(3) =:= 3), 347 ?_assert(fib(4) =:= 5), 348 ?_assert(fib(5) =:= 8), 349 ?_assertException(error, function_clause, fib(-1)), 350 ?_assert(fib(31) =:= 2178309) 351 ].''' 352 353(Author's note: When I first wrote this example, I happened to write a 354`*' instead of `+' in the `fib' function. Of course, this showed up 355immediately when I ran the tests.) 356 357See {@section EUnit test representation} for a full list of all the ways 358you can specify test sets in EUnit. 359 360=== Disabling testing === 361 362Testing can be turned off by defining the `NOTEST' macro when compiling, 363for example as an option to `erlc', as in: 364```erlc -DNOTEST my_module.erl''' 365or by adding a macro definition to the code, <em>before the EUnit header 366file is included</em>: 367```-define(NOTEST, 1).''' 368(the value is not important, but should typically be 1 or `true'). 369Note that unless the `EUNIT_NOAUTO' macro is defined, disabling testing 370will also automatically strip all test functions from the code, except 371for any that are explicitly declared as exported. 372 373For instance, to use EUnit in your application, but with testing turned 374off by default, put the following lines in a header file: 375```-define(NOTEST, true). 376 -include_lib("eunit/include/eunit.hrl").''' 377and then make sure that every module of your application includes that 378header file. This means that you have a only a single place to modify in 379order to change the default setting for testing. To override the `NOTEST' 380setting without modifying the code, you can define `TEST' in a compiler 381option, like this: 382```erlc -DTEST my_module.erl''' 383 384See {@section Compilation control macros} for details about these 385macros. 386 387=== Avoiding compile-time dependency on EUnit === 388 389If you are distributing the source code for your application for other 390people to compile and run, you probably want to ensure that the code 391compiles even if EUnit is not available. Like the example in the 392previous section, you can put the following lines in a common header 393file: 394```-ifdef(TEST). 395 -include_lib("eunit/include/eunit.hrl"). 396 -endif.''' 397and, of course, also make sure that you place all test code that uses 398EUnit macros within `-ifdef(TEST)' or `-ifdef(EUNIT)' sections. 399 400 401== EUnit macros == 402 403Although all the functionality of EUnit is available even without the 404use of preprocessor macros, the EUnit header file defines a number of 405such macros in order to make it as easy as possible to write unit tests 406as compactly as possible and without getting too many details in the 407way. 408 409Except where explicitly stated, using EUnit macros will never introduce 410run-time dependencies on the EUnit library code, regardless of whether 411your code is compiled with testing enabled or disabled. 412 413<ul> 414<li>{@section Basic macros}</li> 415<li>{@section Compilation control macros}</li> 416<li>{@section Utility macros}</li> 417<li>{@section Assert macros}</li> 418<li>{@section Macros for running external commands}</li> 419<li>{@section Debugging macros}</li> 420</ul> 421 422=== Basic macros === 423 424<dl> 425<dt>`_test(Expr)'</dt> 426<dd>Turns `Expr' into a "test object", by wrapping it in a 427fun-expression and a source line number. Technically, this is the same 428as `{?LINE, fun () -> (Expr) end}'. 429</dd> 430</dl> 431 432=== Compilation control macros === 433 434<dl> 435<dt>`EUNIT'</dt> 436<dd>This macro is always defined to `true' whenever EUnit is enabled at 437compile time. This is typically used to place testing code within 438conditional compilation, as in: 439```-ifdef(EUNIT). 440 % test code here 441 ... 442 -endif.''' 443e.g., to ensure that the code can be compiled without including the 444EUnit header file, when testing is disabled. See also the macros `TEST' 445and `NOTEST'. 446</dd> 447 448<dt>`EUNIT_NOAUTO'</dt> 449<dd>If this macro is defined, the automatic exporting or stripping of 450test functions will be disabled. 451</dd> 452 453<dt>`TEST'</dt> 454<dd>This macro is always defined (to `true', unless previously defined 455by the user to have another value) whenever EUnit is enabled at compile 456time. This can be used to place testing code within conditional 457compilation; see also the macros `NOTEST' and `EUNIT'. 458 459For testing code that is strictly dependent on EUnit, it may be 460preferable to use the `EUNIT' macro for this purpose, while for code 461that uses more generic testing conventions, using the `TEST' macro may 462be preferred. 463 464The `TEST' macro can also be used to override the `NOTEST' macro. If 465`TEST' is defined <em>before</em> the EUnit header file is 466included (even if `NOTEST' is also defined), then the code will be 467compiled with EUnit enabled. 468</dd> 469 470<dt>`NOTEST'</dt> 471<dd>This macro is always defined (to `true', unless previously defined 472by the user to have another value) whenever EUnit is <em>disabled</em> 473at compile time. (Compare the `TEST' macro.) 474 475This macro can also be used for conditional compilation, but is more 476typically used to disable testing: If `NOTEST' is defined 477<em>before</em> the EUnit header file is included, and `TEST' 478is <em>not</em> defined, then the code will be compiled with EUnit 479disabled. See also {@section Disabling testing}. 480</dd> 481 482<dt>`NOASSERT'</dt> 483<dd>If this macro is defined, the assert macros will have no effect, 484when testing is also disabled. See {@section Assert macros}. When 485testing is enabled, the assert macros are always enabled automatically 486and cannot be disabled. 487</dd> 488 489<dt>`ASSERT'</dt> 490<dd>If this macro is defined, it overrides the NOASSERT macro, forcing 491the assert macros to always be enabled regardless of other settings. 492</dd> 493 494<dt>`NODEBUG'</dt> 495<dd>If this macro is defined, the debugging macros will have no effect. 496See {@section Debugging macros}. `NODEBUG' also implies `NOASSERT', 497unless testing is enabled. 498</dd> 499 500<dt>`DEBUG'</dt> 501<dd>If this macro is defined, it overrides the NODEBUG macro, forcing 502the debugging macros to be enabled. 503</dd> 504</dl> 505 506=== Utility macros === 507 508The following macros can make tests more compact and readable: 509 510<dl> 511<dt>`LET(Var,Arg,Expr)'</dt> 512<dd>Creates a local binding `Var = Arg' in `Expr'. (This is the same as 513`(fun(Var)->(Expr)end)(Arg)'.) Note that the binding is not exported 514outside of `Expr', and that within `Expr', this binding of `Var' will 515shadow any binding of `Var' in the surrounding scope. 516</dd> 517<dt>`IF(Cond,TrueCase,FalseCase)'</dt> 518<dd>Evaluates `TrueCase' if `Cond' evaluates to `true', or otherwise 519evaluates `FalseCase' if `Cond' evaluates to `false'. (This is the same 520as `(case (Cond) of true->(TrueCase); false->(FalseCase) end)'.) Note 521that it is an error if `Cond' does not yield a boolean value. 522</dd> 523</dl> 524 525=== Assert macros === 526 527(Note that these macros also have corresponding forms which start with 528an "`_'" (underscore) character, as in `?_assert(BoolExpr)', that create 529a "test object" instead of performing the test immediately. This is 530equivalent to writing `?_test(assert(BoolExpr))', etc.) 531 532If the macro `NOASSERT' is defined before the EUnit header file is 533included, these macros have no effect when testing is also disabled; see 534{@section Compilation control macros} for details. 535 536<dl> 537<dt>`assert(BoolExpr)'</dt> 538<dd>Evaluates the expression `BoolExpr', if testing is enabled. Unless 539the result is `true', an informative exception will be generated. If 540there is no exception, the result of the macro expression is the atom 541`ok', and the value of `BoolExpr' is discarded. If testing is disabled, 542the macro will not generate any code except the atom `ok', and 543`BoolExpr' will not be evaluated. 544 545Typical usage: 546```?assert(f(X, Y) =:= [])''' 547 548The `assert' macro can be used anywhere in a program, not just in unit 549tests, to check pre/postconditions and invariants. For example: 550```some_recursive_function(X, Y, Z) -> 551 ?assert(X + Y > Z), 552 ...''' 553</dd> 554<dt>`assertNot(BoolExpr)'</dt> 555<dd>Equivalent to `assert(not (BoolExpr))'. 556</dd> 557<dt>`assertMatch(GuardedPattern, Expr)'</dt> 558<dd>Evaluates `Expr' and matches the result against `GuardedPattern', if 559testing is enabled. If the match fails, an informative exception will be 560generated; see the `assert' macro for further details. `GuardedPattern' 561can be anything that you can write on the left hand side of the `->' 562symbol in a case-clause, except that it cannot contain comma-separated 563guard tests. 564 565The main reason for using `assertMatch' also for simple matches, instead 566of matching with `=', is that it produces more detailed error messages. 567 568Examples: 569```?assertMatch({found, {fred, _}}, lookup(bloggs, Table))''' 570```?assertMatch([X|_] when X > 0, binary_to_list(B))''' 571</dd> 572<dt>`assertNotMatch(GuardedPattern, Expr)'</dt> 573<dd>The inverse case of assertMatch, for convenience. 574</dd> 575<dt>`assertEqual(Expect, Expr)'</dt> 576<dd>Evaluates the expressions `Expect' and `Expr' and compares the 577results for equality, if testing is enabled. If the values are not 578equal, an informative exception will be generated; see the `assert' 579macro for further details. 580 581`assertEqual' is more suitable than `assertMatch' when the 582left-hand side is a computed value rather than a simple pattern, and 583gives more details than `?assert(Expect =:= Expr)'. 584 585Examples: 586```?assertEqual("b" ++ "a", lists:reverse("ab"))''' 587```?assertEqual(foo(X), bar(Y))''' 588</dd> 589<dt>`assertNotEqual(Unexpected, Expr)'</dt> 590<dd>The inverse case of assertEqual, for convenience. 591</dd> 592<dt>`assertException(ClassPattern, TermPattern, Expr)'</dt> 593<dt>`assertError(TermPattern, Expr)'</dt> 594<dt>`assertExit(TermPattern, Expr)'</dt> 595<dt>`assertThrow(TermPattern, Expr)'</dt> 596<dd>Evaluates `Expr', catching any exception and testing that it matches 597the expected `ClassPattern:TermPattern'. If the match fails, or if no 598exception is thrown by `Expr', an informative exception will be 599generated; see the `assert' macro for further details. The 600`assertError', `assertExit', and `assertThrow' macros, are equivalent to 601using `assertException' with a `ClassPattern' of `error', `exit', or 602`throw', respectively. 603 604Examples: 605```?assertError(badarith, X/0)''' 606```?assertExit(normal, exit(normal))''' 607```?assertException(throw, {not_found,_}, throw({not_found,42}))''' 608</dd> 609</dl> 610 611=== Macros for running external commands === 612 613Keep in mind that external commands are highly dependent on the 614operating system. You can use the standard library function `os:type()' 615in test generator functions, to produce different sets of tests 616depending on the current operating system. 617 618Note: these macros introduce a run-time dependency on the EUnit library 619code, if compiled with testing enabled. 620 621<dl> 622<dt>`assertCmd(CommandString)'</dt> 623<dd>Runs `CommandString' as an external command, if testing is enabled. 624Unless the returned status value is 0, an informative exception will be 625generated. If there is no exception, the result of the macro expression 626is the atom `ok'. If testing is disabled, the macro will not generate 627any code except the atom `ok', and the command will not be executed. 628 629Typical usage: 630```?assertCmd("mkdir foo")''' 631</dd> 632<dt>`assertCmdStatus(N, CommandString)'</dt> 633<dd>Like the `assertCmd(CommandString)' macro, but generates an 634exception unless the returned status value is `N'. 635</dd> 636<dt>`assertCmdOutput(Text, CommandString)'</dt> 637<dd>Runs `CommandString' as an external command, if testing is enabled. 638Unless the output produced by the command exactly matches the specified 639string `Text', an informative exception will be generated. (Note that 640the output is normalized to use a single LF character as line break on 641all platforms.) If there is no exception, the result of the macro 642expression is the atom `ok'. If testing is disabled, the macro will not 643generate any code except the atom `ok', and the command will not be 644executed. 645</dd> 646<dt>`cmd(CommandString)'</dt> 647<dd>Runs `CommandString' as an external command. Unless the returned 648status value is 0 (indicating success), an informative exception will be 649generated; otherwise, the result of the macro expression is the output 650produced by the command, as a flat string. The output is normalized to 651use a single LF character as line break on all platforms. 652 653This macro is useful in the setup and cleanup sections of fixtures, 654e.g., for creating and deleting files or perform similar operating 655system specific tasks, to make sure that the test system is informed of 656any failures. 657 658A Unix-specific example: 659```{setup, 660 fun () -> ?cmd("mktemp") end, 661 fun (FileName) -> ?cmd("rm " ++ FileName) end, 662 ...}''' 663</dd> 664</dl> 665 666=== Debugging macros === 667 668To help with debugging, EUnit defines several useful macros for printing 669messages directly to the console (rather than to the standard output). 670Furthermore, these macros all use the same basic format, which includes 671the file and line number where they occur, making it possible in some 672development environments (e.g., when running Erlang in an Emacs buffer) 673to simply click on the message and jump directly to the corresponding 674line in the code. 675 676If the macro `NODEBUG' is defined before the EUnit header file is 677included, these macros have no effect; see 678{@section Compilation control macros} for details. 679 680<dl> 681<dt>`debugHere'</dt> 682<dd>Just prints a marker showing the current file and line number. Note 683that this is an argument-less macro. The result is always `ok'.</dd> 684<dt>`debugMsg(Text)'</dt> 685<dd>Outputs the message `Text' (which can be a plain string, an IO-list, 686or just an atom). The result is always `ok'.</dd> 687<dt>`debugFmt(FmtString, Args)'</dt> 688<dd>This formats the text like `io:format(FmtString, Args)' and outputs 689it like `debugMsg'. The result is always `ok'.</dd> 690<dt>`debugVal(Expr)'</dt> 691<dd>Prints both the source code for `Expr' and its current value. E.g., 692`?debugVal(f(X))' might be displayed as "`f(X) = 42'". (Large terms are 693truncated to the depth given by the macro `EUNIT_DEBUG_VAL_DEPTH', which 694defaults to 15 but can be overridden by the user.) The result is always the 695value of `Expr', so this macro can be wrapped around any expression to 696display its value when the code is compiled with debugging enabled.</dd> 697<dt>`debugVal(Expr, Depth)'</dt> 698<dd>Like `debugVal(Expr)', but prints terms truncated to the given depth.</dd> 699<dt>`debugTime(Text,Expr)'</dt> 700<dd>Prints `Text' and the wall clock time for evaluation of `Expr'. The 701result is always the value of `Expr', so this macro can be wrapped 702around any expression to show its run time when the code is compiled 703with debugging enabled. For example, `List1 = ?debugTime("sorting", 704lists:sort(List))' might show as "`sorting: 0.015 s'".</dd> 705 706</dl> 707 708 709== EUnit test representation == 710 711The way EUnit represents tests and test sets as data is flexible, 712powerful, and concise. This section describes the representation in 713detail. 714 715<ul> 716<li>{@section Simple test objects}</li> 717<li>{@section Test sets and deep lists}</li> 718<li>{@section Titles}</li> 719<li>{@section Primitives}</li> 720<li>{@section Control}</li> 721<li>{@section Fixtures}</li> 722<li>{@section Lazy generators}</li> 723</ul> 724 725=== Simple test objects === 726 727A <em>simple test object</em> is one of the following: 728<ul> 729 <li>A nullary functional value (i.e., a fun that takes zero 730 arguments). Examples: 731```fun () -> ... end''' 732```fun some_function/0''' 733```fun some_module:some_function/0''' 734 </li> 735 <li>A tuple `{test, ModuleName, FunctionName}', where `ModuleName' and 736 `FunctionName' are atoms, referring to the function 737 `ModuleName:FunctionName/0'</li> 738 <li>(Obsolete) A pair of atoms `{ModuleName, FunctionName}', equivalent to 739 `{test, ModuleName, FunctionName}' if nothing else matches first. This 740 might be removed in a future version.</li> 741 <li>A pair `{LineNumber, SimpleTest}', where `LineNumber' is a 742 nonnegative integer and `SimpleTest' is another simple test 743 object. `LineNumber' should indicate the source line of the test. 744 Pairs like this are usually only created via `?_test(...)' macros; 745 see {@section Basic macros}.</li> 746</ul> 747In brief, a simple test object consists of a single function that takes 748no arguments (possibly annotated with some additional metadata, i.e., a 749line number). Evaluation of the function either <em>succeeds</em>, by 750returning some value (which is ignored), or <em>fails</em>, by throwing 751an exception. 752 753=== Test sets and deep lists === 754 755A test set can be easily created by placing a sequence of test objects 756in a list. If `T_1', ..., `T_N' are individual test objects, then `[T_1, 757..., T_N]' is a test set consisting of those objects (in that order). 758 759Test sets can be joined in the same way: if `S_1', ..., `S_K' are test 760sets, then `[S_1, ..., S_K]' is also a test set, where the tests of 761`S_i' are ordered before those of `S_(i+1)', for each subset `S_i'. 762 763Thus, the main representation of test sets is <em>deep lists</em>, and 764a simple test object can be viewed as a test set containing only a 765single test; there is no difference between `T' and `[T]'. 766 767A module can also be used to represent a test set; see `ModuleName' 768under {@section Primitives} below. 769 770=== Titles === 771 772Any test or test set `T' can be annotated with a title, by wrapping it 773in a pair `{Title, T}', where `Title' is a string. For convenience, any 774test which is normally represented using a tuple can simply be given a 775title string as the first element, i.e., writing `{"The Title", ...}' 776instead of adding an extra tuple wrapper as in `{"The Title", {...}}'. 777 778 779=== Primitives === 780 781The following are primitives, which do not contain other test sets as 782arguments: 783<dl> 784<dt>`ModuleName::atom()' 785</dt> 786<dd>A single atom represents a module name, and is equivalent to 787`{module, ModuleName}'. This is often used as in the call 788`eunit:test(some_module)'. 789</dd> 790<dt>`{module, ModuleName::atom()}' 791</dt> 792<dd>This composes a test set from the exported test functions of the 793named module, i.e., those functions with arity zero whose names end 794with `_test' or `_test_'. Basically, the `..._test()' functions become 795simple tests, while the `..._test_()' functions become generators. 796 797In addition, EUnit will also look for another module whose name is 798`ModuleName' plus the suffix `_tests', and if it exists, all the tests 799from that module will also be added. (If `ModuleName' already contains 800the suffix `_tests', this is not done.) E.g., the specification 801`{module, mymodule}' will run all tests in the modules `mymodule' and 802`mymodule_tests'. Typically, the `_tests' module should only contain 803test cases that use the public interface of the main module (and no 804other code). 805</dd> 806<dt>`{application, AppName::atom(), Info::list()}' 807</dt> 808<dd>This is a normal Erlang/OTP application descriptor, as found in an 809 `.app' file. The resulting test set consists of the modules listed in 810 the `modules' entry in `Info'. 811</dd> 812<dt>`{application, AppName::atom()}' 813</dt> 814<dd>This creates a test set from all the modules belonging to the 815specified application, by consulting the application's `.app' file 816(see `{file, FileName}'), or if no such file exists, by testing all 817object files in the application's <tt>ebin</tt>-directory (see `{dir, 818Path}'); if that does not exist, the `code:lib_dir(AppName)' directory 819is used. 820</dd> 821<dt>`Path::string()' 822</dt> 823<dd>A single string represents the path of a file or directory, and is 824equivalent to `{file, Path}', or `{dir, Path}', respectively, depending 825on what `Path' refers to in the file system. 826</dd> 827<dt>`{file, FileName::string()}' 828</dt> 829<dd>If `FileName' has a suffix that indicates an object file (`.beam'), 830EUnit will try to reload the module from the specified file and test it. 831Otherwise, the file is assumed to be a text file containing test 832specifications, which will be read using the standard library function 833`file:path_consult/2'. 834 835Unless the file name is absolute, the file is first searched for 836relative to the current directory, and then using the normal search path 837(`code:get_path()'). This means that the names of typical "app" files 838can be used directly, without a path, e.g., `"mnesia.app"'. 839</dd> 840<dt>`{dir, Path::string()}' 841</dt> 842<dd>This tests all object files in the specified directory, as if they 843had been individually specified using `{file, FileName}'. 844</dd> 845<dt>`{generator, GenFun::(() -> Tests)}' 846</dt> 847<dd>The generator function `GenFun' is called to produce a test 848set. 849</dd> 850<dt>`{generator, ModuleName::atom(), FunctionName::atom()}' 851</dt> 852<dd>The function `ModuleName:FunctionName()' is called to produce a test 853set. 854</dd> 855<dt>`{with, X::any(), [AbstractTestFun::((any()) -> any())]}' 856</dt> 857<dd>Distributes the value `X' over the unary functions in the list, 858turning them into nullary test functions. An `AbstractTestFun' is like 859an ordinary test fun, but takes one argument instead of zero - it's 860basically missing some information before it can be a proper test. In 861practice, `{with, X, [F_1, ..., F_N]}' is equivalent to `[fun () -> 862F_1(X) end, ..., fun () -> F_N(X) end]'. This is particularly useful if 863your abstract test functions are already implemented as proper 864functions: `{with, FD, [fun filetest_a/1, fun filetest_b/1, fun 865filetest_c/1]}' is equivalent to `[fun () -> filetest_a(FD) end, fun () 866-> filetest_b(FD) end, fun () -> filetest_c(FD) end]', but much more 867compact. See also {@section Fixtures}, below. 868</dd> 869</dl> 870 871=== Control === 872 873The following representations control how and where tests are executed: 874<dl> 875<dt>`{spawn, Tests}'</dt> 876<dd>Runs the specified tests in a separate subprocess, while the current 877test process waits for it to finish. This is useful for tests that need 878a fresh, isolated process state. (Note that EUnit always starts at least 879one such a subprocess automatically; tests are never executed by the 880caller's own process.)</dd> 881<dt>`{spawn, Node::atom(), Tests}'</dt> 882<dd>Like `{spawn, Tests}', but runs the specified tests on the given 883Erlang node.</dd> 884<dt>`{timeout, Time::number(), Tests}'</dt> 885<dd>Runs the specified tests under the given timeout. Time is in 886seconds; e.g., 60 means one minute and 0.1 means 1/10th of a second. If 887the timeout is exceeded, the unfinished tests will be forced to 888terminate. Note that if a timeout is set around a fixture, it includes 889the time for setup and cleanup, and if the timeout is triggered, the 890entire fixture is abruptly terminated (without running the 891cleanup). The default timeout for an individual test is 5 seconds.</dd> 892<dt>`{inorder, Tests}'</dt> 893<dd>Runs the specified tests in strict order. Also see `{inparallel, 894Tests}'. By default, tests are neither marked as `inorder' or 895`inparallel', but may be executed as the test framework chooses.</dd> 896<dt>`{inparallel, Tests}'</dt> 897<dd>Runs the specified tests in parallel (if possible). Also see 898`{inorder, Tests}'.</dd> 899<dt>`{inparallel, N::integer(), Tests}'</dt> 900<dd>Like `{inparallel, Tests}', but running no more than `N' subtests 901simultaneously.</dd> 902</dl> 903 904=== Fixtures === 905 906A "fixture" is some state that is necessary for a particular set of 907tests to run. EUnit's support for fixtures makes it easy to set up such 908state locally for a test set, and automatically tear it down again when 909the test set is finished, regardless of the outcome (success, failures, 910timeouts, etc.). 911 912To make the descriptions simpler, we first list some definitions: 913<table border="0" cellspacing="4"> 914<tr> 915<td>`Setup'</td><td>`() -> (R::any())'</td> 916</tr> 917<tr> 918<td>`SetupX'</td><td>`(X::any()) -> (R::any())'</td> 919</tr> 920<tr> 921<td>`Cleanup'</td><td>`(R::any()) -> any()'</td> 922</tr> 923<tr> 924<td>`CleanupX'</td><td>`(X::any(), R::any()) -> any()'</td> 925</tr> 926<tr> 927<td>`Instantiator'</td><td>`((R::any()) -> Tests) | {with, [AbstractTestFun::((any()) -> any())]}'</td> 928</tr> 929<tr> 930<td>`Where'</td><td>`local | spawn | {spawn, Node::atom()}'</td> 931</tr> 932</table> 933(these are explained in more detail further below.) 934 935The following representations specify fixture handling for test sets: 936<dl> 937<dt>`{setup, Setup, Tests | Instantiator}'</dt> 938<dt>`{setup, Setup, Cleanup, Tests | Instantiator}'</dt> 939<dt>`{setup, Where, Setup, Tests | Instantiator}'</dt> 940<dt>`{setup, Where, Setup, Cleanup, Tests | Instantiator}'</dt> 941<dd>`setup' sets up a single fixture for running all of the specified 942tests, with optional teardown afterwards. The arguments are described in 943detail below. 944</dd> 945<dt>`{node, Node::atom(), Tests | Instantiator}'</dt> 946<dt>`{node, Node::atom(), Args::string(), Tests | Instantiator}'</dt> 947<dd>`node' is like `setup', but with a built-in behaviour: it starts a 948slave node for the duration of the tests. The atom `Node' should have 949the format `nodename@full.machine.name', and `Args' are the optional 950arguments to the new node; see `slave:start_link/3' for details. 951</dd> 952<dt>`{foreach, Where, Setup, Cleanup, [Tests | Instantiator]}'</dt> 953<dt>`{foreach, Setup, Cleanup, [Tests | Instantiator]}'</dt> 954<dt>`{foreach, Where, Setup, [Tests | Instantiator]}'</dt> 955<dt>`{foreach, Setup, [Tests | Instantiator]}'</dt> 956<dd>`foreach' is used to set up a fixture and optionally tear it down 957afterwards, repeated for each single one of the specified test sets. 958</dd> 959<dt>`{foreachx, Where, SetupX, CleanupX, 960 Pairs::[{X::any(), ((X::any(), R::any()) -> Tests)}]}'</dt> 961<dt>`{foreachx, SetupX, CleanupX, Pairs}'</dt> 962<dt>`{foreachx, Where, SetupX, Pairs}'</dt> 963<dt>`{foreachx, SetupX, Pairs}'</dt> 964<dd>`foreachx' is like `foreach', but uses a list of pairs, each 965containing an extra argument `X' and an extended instantiator function. 966</dd> 967</dl> 968 969A `Setup' function is executed just before any of the specified tests 970are run, and a `Cleanup' function is executed when no more of the 971specified tests will be run, regardless of the reason. A `Setup' 972function takes no argument, and returns some value which will be passed 973as it is to the `Cleanup' function. A `Cleanup' function should do 974whatever necessary and return some arbitrary value, such as the atom 975`ok'. (`SetupX' and `CleanupX' functions are similar, but receive one 976additional argument: some value `X', which depends on the context.) When 977no `Cleanup' function is specified, a dummy function is used which has 978no effect. 979 980An `Instantiator' function receives the same value as the `Cleanup' 981function, i.e., the value returned by the `Setup' function. It should 982then behave much like a generator (see {@section Primitives}), and 983return a test set whose tests have been <em>instantiated</em> with the 984given value. A special case is the syntax `{with, [AbstractTestFun]}' 985which represents an instantiator function that distributes the value 986over a list of unary functions; see {@section Primitives}: `{with, X, 987[...]}' for more details. 988 989A `Where' term controls how the specified tests are executed. The 990default is `spawn', which means that the current process handles the 991setup and teardown, while the tests are executed in a subprocess. 992`{spawn, Node}' is like `spawn', but runs the subprocess on the 993specified node. `local' means that the current process will handle both 994setup/teardown and running the tests - the drawback is that if a test 995times out so that the process is killed, the <em>cleanup will not be 996performed</em>; hence, avoid this for persistent fixtures such as file 997operations. In general, `local' should only be used when: 998<ul> 999 <li>the setup/teardown needs to be executed by the process that will 1000 run the tests;</li> 1001 <li>no further teardown needs to be done if the process is killed 1002 (i.e., no state outside the process was affected by the setup)</li> 1003</ul> 1004 1005=== Lazy generators === 1006 1007Sometimes, it can be convenient not to produce the whole set of test 1008descriptions before the testing begins; for example, if you want to 1009generate a huge amount of tests that would take up too much space to 1010keep in memory all at once. 1011 1012It is fairly easy to write a generator which, each time it is called, 1013either produces an empty list if it is done, or otherwise produces a 1014list containing a single test case plus a new generator which will 1015produce the rest of the tests. This demonstrates the basic pattern: 1016 1017```lazy_test_() -> 1018 lazy_gen(10000). 1019 1020 lazy_gen(N) -> 1021 {generator, 1022 fun () -> 1023 if N > 0 -> 1024 [?_test(...) 1025 | lazy_gen(N-1)]; 1026 true -> 1027 [] 1028 end 1029 end}.''' 1030 1031When EUnit traverses the test representation in order to run the tests, 1032the new generator will not be called to produce the next test until the 1033previous test has been executed. 1034 1035Note that it is easiest to write this kind of recursive generator using 1036a help function, like the `lazy_gen/1' function above. It can also be 1037written using a recursive fun, if you prefer to not clutter your 1038function namespace and are comfortable with writing that kind of code. 1039