1# Googletest Primer 2 3 4## Introduction: Why googletest? 5 6*googletest* helps you write better C++ tests. 7 8googletest is a testing framework developed by the Testing 9Technology team with Google's specific 10requirements and constraints in mind. No matter whether you work on Linux, 11Windows, or a Mac, if you write C++ code, googletest can help you. And it 12supports *any* kind of tests, not just unit tests. 13 14So what makes a good test, and how does googletest fit in? We believe: 15 161. Tests should be *independent* and *repeatable*. It's a pain to debug a test 17 that succeeds or fails as a result of other tests. googletest isolates the 18 tests by running each of them on a different object. When a test fails, 19 googletest allows you to run it in isolation for quick debugging. 201. Tests should be well *organized* and reflect the structure of the tested 21 code. googletest groups related tests into test cases that can share data 22 and subroutines. This common pattern is easy to recognize and makes tests 23 easy to maintain. Such consistency is especially helpful when people switch 24 projects and start to work on a new code base. 251. Tests should be *portable* and *reusable*. Google has a lot of code that is 26 platform-neutral, its tests should also be platform-neutral. googletest 27 works on different OSes, with different compilers (gcc, icc, and MSVC), with 28 or without exceptions, so googletest tests can easily work with a variety of 29 configurations. 301. When tests fail, they should provide as much *information* about the problem 31 as possible. googletest doesn't stop at the first test failure. Instead, it 32 only stops the current test and continues with the next. You can also set up 33 tests that report non-fatal failures after which the current test continues. 34 Thus, you can detect and fix multiple bugs in a single run-edit-compile 35 cycle. 361. The testing framework should liberate test writers from housekeeping chores 37 and let them focus on the test *content*. googletest automatically keeps 38 track of all tests defined, and doesn't require the user to enumerate them 39 in order to run them. 401. Tests should be *fast*. With googletest, you can reuse shared resources 41 across tests and pay for the set-up/tear-down only once, without making 42 tests depend on each other. 43 44Since googletest is based on the popular xUnit architecture, you'll feel right 45at home if you've used JUnit or PyUnit before. If not, it will take you about 10 46minutes to learn the basics and get started. So let's go! 47 48## Beware of the nomenclature 49 50_Note:_ There might be some confusion of idea due to different 51definitions of the terms _Test_, _Test Case_ and _Test Suite_, so beware 52of misunderstanding these. 53 54Historically, googletest started to use the term _Test Case_ for grouping 55related tests, whereas current publications including the International Software 56Testing Qualifications Board ([ISTQB](http://www.istqb.org/)) and various 57textbooks on Software Quality use the term _[Test 58Suite](http://glossary.istqb.org/search/test%20suite)_ for this. 59 60The related term _Test_, as it is used in the googletest, is corresponding to 61the term _[Test Case](http://glossary.istqb.org/search/test%20case)_ of ISTQB 62and others. 63 64The term _Test_ is commonly of broad enough sense, including ISTQB's 65definition of _Test Case_, so it's not much of a problem here. But the 66term _Test Case_ as used in Google Test is of contradictory sense and thus confusing. 67 68Unfortunately replacing the term _Test Case_ by _Test Suite_ throughout the 69googletest is not easy without breaking dependent projects, as `TestCase` is 70part of the public API at various places. 71 72So for the time being, please be aware of the different definitions of 73the terms: 74 75Meaning | googletest Term | [ISTQB](http://www.istqb.org/) Term 76:----------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------- | :---------------------------------- 77Exercise a particular program path with specific input values and verify the results | [TEST()](#simple-tests) | [Test Case](http://glossary.istqb.org/search/test%20case) 78A set of several tests related to one component | [TestCase](#basic-concepts) | [TestSuite](http://glossary.istqb.org/search/test%20suite) 79 80## Basic Concepts 81 82When using googletest, you start by writing *assertions*, which are statements 83that check whether a condition is true. An assertion's result can be *success*, 84*nonfatal failure*, or *fatal failure*. If a fatal failure occurs, it aborts the 85current function; otherwise the program continues normally. 86 87*Tests* use assertions to verify the tested code's behavior. If a test crashes 88or has a failed assertion, then it *fails*; otherwise it *succeeds*. 89 90A *test case* contains one or many tests. You should group your tests into test 91cases that reflect the structure of the tested code. When multiple tests in a 92test case need to share common objects and subroutines, you can put them into a 93*test fixture* class. 94 95A *test program* can contain multiple test cases. 96 97We'll now explain how to write a test program, starting at the individual 98assertion level and building up to tests and test cases. 99 100## Assertions 101 102googletest assertions are macros that resemble function calls. You test a class 103or function by making assertions about its behavior. When an assertion fails, 104googletest prints the assertion's source file and line number location, along 105with a failure message. You may also supply a custom failure message which will 106be appended to googletest's message. 107 108The assertions come in pairs that test the same thing but have different effects 109on the current function. `ASSERT_*` versions generate fatal failures when they 110fail, and **abort the current function**. `EXPECT_*` versions generate nonfatal 111failures, which don't abort the current function. Usually `EXPECT_*` are 112preferred, as they allow more than one failure to be reported in a test. 113However, you should use `ASSERT_*` if it doesn't make sense to continue when the 114assertion in question fails. 115 116Since a failed `ASSERT_*` returns from the current function immediately, 117possibly skipping clean-up code that comes after it, it may cause a space leak. 118Depending on the nature of the leak, it may or may not be worth fixing - so keep 119this in mind if you get a heap checker error in addition to assertion errors. 120 121To provide a custom failure message, simply stream it into the macro using the 122`<<` operator, or a sequence of such operators. An example: 123 124```c++ 125ASSERT_EQ(x.size(), y.size()) << "Vectors x and y are of unequal length"; 126 127for (int i = 0; i < x.size(); ++i) { 128 EXPECT_EQ(x[i], y[i]) << "Vectors x and y differ at index " << i; 129} 130``` 131 132Anything that can be streamed to an `ostream` can be streamed to an assertion 133macro--in particular, C strings and `string` objects. If a wide string 134(`wchar_t*`, `TCHAR*` in `UNICODE` mode on Windows, or `std::wstring`) is 135streamed to an assertion, it will be translated to UTF-8 when printed. 136 137### Basic Assertions 138 139These assertions do basic true/false condition testing. 140 141Fatal assertion | Nonfatal assertion | Verifies 142-------------------------- | -------------------------- | -------------------- 143`ASSERT_TRUE(condition);` | `EXPECT_TRUE(condition);` | `condition` is true 144`ASSERT_FALSE(condition);` | `EXPECT_FALSE(condition);` | `condition` is false 145 146Remember, when they fail, `ASSERT_*` yields a fatal failure and returns from the 147current function, while `EXPECT_*` yields a nonfatal failure, allowing the 148function to continue running. In either case, an assertion failure means its 149containing test fails. 150 151**Availability**: Linux, Windows, Mac. 152 153### Binary Comparison 154 155This section describes assertions that compare two values. 156 157Fatal assertion | Nonfatal assertion | Verifies 158------------------------ | ------------------------ | -------------- 159`ASSERT_EQ(val1, val2);` | `EXPECT_EQ(val1, val2);` | `val1 == val2` 160`ASSERT_NE(val1, val2);` | `EXPECT_NE(val1, val2);` | `val1 != val2` 161`ASSERT_LT(val1, val2);` | `EXPECT_LT(val1, val2);` | `val1 < val2` 162`ASSERT_LE(val1, val2);` | `EXPECT_LE(val1, val2);` | `val1 <= val2` 163`ASSERT_GT(val1, val2);` | `EXPECT_GT(val1, val2);` | `val1 > val2` 164`ASSERT_GE(val1, val2);` | `EXPECT_GE(val1, val2);` | `val1 >= val2` 165 166Value arguments must be comparable by the assertion's comparison operator or 167you'll get a compiler error. We used to require the arguments to support the 168`<<` operator for streaming to an `ostream`, but it's no longer necessary. If 169`<<` is supported, it will be called to print the arguments when the assertion 170fails; otherwise googletest will attempt to print them in the best way it can. 171For more details and how to customize the printing of the arguments, see 172gMock [recipe](../../googlemock/docs/CookBook.md#teaching-google-mock-how-to-print-your-values).). 173 174These assertions can work with a user-defined type, but only if you define the 175corresponding comparison operator (e.g. `==`, `<`, etc). Since this is 176discouraged by the Google [C++ Style 177Guide](https://google.github.io/styleguide/cppguide.html#Operator_Overloading), 178you may need to use `ASSERT_TRUE()` or `EXPECT_TRUE()` to assert the equality of 179two objects of a user-defined type. 180 181However, when possible, `ASSERT_EQ(actual, expected)` is preferred to 182`ASSERT_TRUE(actual == expected)`, since it tells you `actual` and `expected`'s 183values on failure. 184 185Arguments are always evaluated exactly once. Therefore, it's OK for the 186arguments to have side effects. However, as with any ordinary C/C++ function, 187the arguments' evaluation order is undefined (i.e. the compiler is free to 188choose any order) and your code should not depend on any particular argument 189evaluation order. 190 191`ASSERT_EQ()` does pointer equality on pointers. If used on two C strings, it 192tests if they are in the same memory location, not if they have the same value. 193Therefore, if you want to compare C strings (e.g. `const char*`) by value, use 194`ASSERT_STREQ()`, which will be described later on. In particular, to assert 195that a C string is `NULL`, use `ASSERT_STREQ(c_string, NULL)`. Consider use 196`ASSERT_EQ(c_string, nullptr)` if c++11 is supported. To compare two `string` 197objects, you should use `ASSERT_EQ`. 198 199When doing pointer comparisons use `*_EQ(ptr, nullptr)` and `*_NE(ptr, nullptr)` 200instead of `*_EQ(ptr, NULL)` and `*_NE(ptr, NULL)`. This is because `nullptr` is 201typed while `NULL` is not. See [FAQ](faq.md#why-does-google-test-support-expect_eqnull-ptr-and-assert_eqnull-ptr-but-not-expect_nenull-ptr-and-assert_nenull-ptr) 202for more details. 203 204If you're working with floating point numbers, you may want to use the floating 205point variations of some of these macros in order to avoid problems caused by 206rounding. See [Advanced googletest Topics](advanced.md) for details. 207 208Macros in this section work with both narrow and wide string objects (`string` 209and `wstring`). 210 211**Availability**: Linux, Windows, Mac. 212 213**Historical note**: Before February 2016 `*_EQ` had a convention of calling it 214as `ASSERT_EQ(expected, actual)`, so lots of existing code uses this order. Now 215`*_EQ` treats both parameters in the same way. 216 217### String Comparison 218 219The assertions in this group compare two **C strings**. If you want to compare 220two `string` objects, use `EXPECT_EQ`, `EXPECT_NE`, and etc instead. 221 222| Fatal assertion | Nonfatal assertion | Verifies | 223| ------------------------------- | ------------------------------- | -------------------------------------------------------- | 224| `ASSERT_STREQ(str1, str2);` | `EXPECT_STREQ(str1, str2);` | the two C strings have the same content | 225| `ASSERT_STRNE(str1, str2);` | `EXPECT_STRNE(str1, str2);` | the two C strings have different contents | 226| `ASSERT_STRCASEEQ(str1, str2);` | `EXPECT_STRCASEEQ(str1, str2);` | the two C strings have the same content, ignoring case | 227| `ASSERT_STRCASENE(str1, str2);` | `EXPECT_STRCASENE(str1, str2);` | the two C strings have different contents, ignoring case | 228 229Note that "CASE" in an assertion name means that case is ignored. A `NULL` 230pointer and an empty string are considered *different*. 231 232`*STREQ*` and `*STRNE*` also accept wide C strings (`wchar_t*`). If a comparison 233of two wide strings fails, their values will be printed as UTF-8 narrow strings. 234 235**Availability**: Linux, Windows, Mac. 236 237**See also**: For more string comparison tricks (substring, prefix, suffix, and 238regular expression matching, for example), see 239[this](https://github.com/google/googletest/blob/master/googletest/docs/advanced.md) 240in the Advanced googletest Guide. 241 242## Simple Tests 243 244To create a test: 245 2461. Use the `TEST()` macro to define and name a test function, These are 247 ordinary C++ functions that don't return a value. 2481. In this function, along with any valid C++ statements you want to include, 249 use the various googletest assertions to check values. 2501. The test's result is determined by the assertions; if any assertion in the 251 test fails (either fatally or non-fatally), or if the test crashes, the 252 entire test fails. Otherwise, it succeeds. 253 254```c++ 255TEST(TestCaseName, TestName) { 256 ... test body ... 257} 258``` 259 260`TEST()` arguments go from general to specific. The *first* argument is the name 261of the test case, and the *second* argument is the test's name within the test 262case. Both names must be valid C++ identifiers, and they should not contain 263underscore (`_`). A test's *full name* consists of its containing test case and 264its individual name. Tests from different test cases can have the same 265individual name. 266 267For example, let's take a simple integer function: 268 269```c++ 270int Factorial(int n); // Returns the factorial of n 271``` 272 273A test case for this function might look like: 274 275```c++ 276// Tests factorial of 0. 277TEST(FactorialTest, HandlesZeroInput) { 278 EXPECT_EQ(Factorial(0), 1); 279} 280 281// Tests factorial of positive numbers. 282TEST(FactorialTest, HandlesPositiveInput) { 283 EXPECT_EQ(Factorial(1), 1); 284 EXPECT_EQ(Factorial(2), 2); 285 EXPECT_EQ(Factorial(3), 6); 286 EXPECT_EQ(Factorial(8), 40320); 287} 288``` 289 290googletest groups the test results by test cases, so logically-related tests 291should be in the same test case; in other words, the first argument to their 292`TEST()` should be the same. In the above example, we have two tests, 293`HandlesZeroInput` and `HandlesPositiveInput`, that belong to the same test case 294`FactorialTest`. 295 296When naming your test cases and tests, you should follow the same convention as 297for [naming functions and 298classes](https://google.github.io/styleguide/cppguide.html#Function_Names). 299 300**Availability**: Linux, Windows, Mac. 301 302## Test Fixtures: Using the Same Data Configuration for Multiple Tests 303 304If you find yourself writing two or more tests that operate on similar data, you 305can use a *test fixture*. It allows you to reuse the same configuration of 306objects for several different tests. 307 308To create a fixture: 309 3101. Derive a class from `::testing::Test` . Start its body with `protected:` as 311 we'll want to access fixture members from sub-classes. 3121. Inside the class, declare any objects you plan to use. 3131. If necessary, write a default constructor or `SetUp()` function to prepare 314 the objects for each test. A common mistake is to spell `SetUp()` as 315 **`Setup()`** with a small `u` - Use `override` in C++11 to make sure you 316 spelled it correctly 3171. If necessary, write a destructor or `TearDown()` function to release any 318 resources you allocated in `SetUp()` . To learn when you should use the 319 constructor/destructor and when you should use `SetUp()/TearDown()`, read 320 this [FAQ](faq.md#should-i-use-the-constructordestructor-of-the-test-fixture-or-setupteardown) entry. 3211. If needed, define subroutines for your tests to share. 322 323When using a fixture, use `TEST_F()` instead of `TEST()` as it allows you to 324access objects and subroutines in the test fixture: 325 326```c++ 327TEST_F(TestCaseName, TestName) { 328 ... test body ... 329} 330``` 331 332Like `TEST()`, the first argument is the test case name, but for `TEST_F()` this 333must be the name of the test fixture class. You've probably guessed: `_F` is for 334fixture. 335 336Unfortunately, the C++ macro system does not allow us to create a single macro 337that can handle both types of tests. Using the wrong macro causes a compiler 338error. 339 340Also, you must first define a test fixture class before using it in a 341`TEST_F()`, or you'll get the compiler error "`virtual outside class 342declaration`". 343 344For each test defined with `TEST_F()` , googletest will create a *fresh* test 345fixture at runtime, immediately initialize it via `SetUp()` , run the test, 346clean up by calling `TearDown()` , and then delete the test fixture. Note that 347different tests in the same test case have different test fixture objects, and 348googletest always deletes a test fixture before it creates the next one. 349googletest does **not** reuse the same test fixture for multiple tests. Any 350changes one test makes to the fixture do not affect other tests. 351 352As an example, let's write tests for a FIFO queue class named `Queue`, which has 353the following interface: 354 355```c++ 356template <typename E> // E is the element type. 357class Queue { 358 public: 359 Queue(); 360 void Enqueue(const E& element); 361 E* Dequeue(); // Returns NULL if the queue is empty. 362 size_t size() const; 363 ... 364}; 365``` 366 367First, define a fixture class. By convention, you should give it the name 368`FooTest` where `Foo` is the class being tested. 369 370```c++ 371class QueueTest : public ::testing::Test { 372 protected: 373 void SetUp() override { 374 q1_.Enqueue(1); 375 q2_.Enqueue(2); 376 q2_.Enqueue(3); 377 } 378 379 // void TearDown() override {} 380 381 Queue<int> q0_; 382 Queue<int> q1_; 383 Queue<int> q2_; 384}; 385``` 386 387In this case, `TearDown()` is not needed since we don't have to clean up after 388each test, other than what's already done by the destructor. 389 390Now we'll write tests using `TEST_F()` and this fixture. 391 392```c++ 393TEST_F(QueueTest, IsEmptyInitially) { 394 EXPECT_EQ(q0_.size(), 0); 395} 396 397TEST_F(QueueTest, DequeueWorks) { 398 int* n = q0_.Dequeue(); 399 EXPECT_EQ(n, nullptr); 400 401 n = q1_.Dequeue(); 402 ASSERT_NE(n, nullptr); 403 EXPECT_EQ(*n, 1); 404 EXPECT_EQ(q1_.size(), 0); 405 delete n; 406 407 n = q2_.Dequeue(); 408 ASSERT_NE(n, nullptr); 409 EXPECT_EQ(*n, 2); 410 EXPECT_EQ(q2_.size(), 1); 411 delete n; 412} 413``` 414 415The above uses both `ASSERT_*` and `EXPECT_*` assertions. The rule of thumb is 416to use `EXPECT_*` when you want the test to continue to reveal more errors after 417the assertion failure, and use `ASSERT_*` when continuing after failure doesn't 418make sense. For example, the second assertion in the `Dequeue` test is 419=ASSERT_NE(nullptr, n)=, as we need to dereference the pointer `n` later, which 420would lead to a segfault when `n` is `NULL`. 421 422When these tests run, the following happens: 423 4241. googletest constructs a `QueueTest` object (let's call it `t1` ). 4251. `t1.SetUp()` initializes `t1` . 4261. The first test ( `IsEmptyInitially` ) runs on `t1` . 4271. `t1.TearDown()` cleans up after the test finishes. 4281. `t1` is destructed. 4291. The above steps are repeated on another `QueueTest` object, this time 430 running the `DequeueWorks` test. 431 432**Availability**: Linux, Windows, Mac. 433 434 435## Invoking the Tests 436 437`TEST()` and `TEST_F()` implicitly register their tests with googletest. So, 438unlike with many other C++ testing frameworks, you don't have to re-list all 439your defined tests in order to run them. 440 441After defining your tests, you can run them with `RUN_ALL_TESTS()` , which 442returns `0` if all the tests are successful, or `1` otherwise. Note that 443`RUN_ALL_TESTS()` runs *all tests* in your link unit -- they can be from 444different test cases, or even different source files. 445 446When invoked, the `RUN_ALL_TESTS()` macro: 447 4481. Saves the state of all googletest flags 449 450* Creates a test fixture object for the first test. 451 452* Initializes it via `SetUp()`. 453 454* Runs the test on the fixture object. 455 456* Cleans up the fixture via `TearDown()`. 457 458* Deletes the fixture. 459 460* Restores the state of all googletest flags 461 462* Repeats the above steps for the next test, until all tests have run. 463 464If a fatal failure happens the subsequent steps will be skipped. 465 466> IMPORTANT: You must **not** ignore the return value of `RUN_ALL_TESTS()`, or 467> you will get a compiler error. The rationale for this design is that the 468> automated testing service determines whether a test has passed based on its 469> exit code, not on its stdout/stderr output; thus your `main()` function must 470> return the value of `RUN_ALL_TESTS()`. 471> 472> Also, you should call `RUN_ALL_TESTS()` only **once**. Calling it more than 473> once conflicts with some advanced googletest features (e.g. thread-safe [death 474> tests](advanced#death-tests)) and thus is not supported. 475 476**Availability**: Linux, Windows, Mac. 477 478## Writing the main() Function 479 480In `google3`, the simplest approach is to use the default main() function 481provided by linking in `"//testing/base/public:gtest_main"`. If that doesn't 482cover what you need, you should write your own main() function, which should 483return the value of `RUN_ALL_TESTS()`. Link to `"//testing/base/public:gunit"`. 484You can start from this boilerplate: 485 486```c++ 487#include "this/package/foo.h" 488#include "gtest/gtest.h" 489 490namespace { 491 492// The fixture for testing class Foo. 493class FooTest : public ::testing::Test { 494 protected: 495 // You can remove any or all of the following functions if its body 496 // is empty. 497 498 FooTest() { 499 // You can do set-up work for each test here. 500 } 501 502 ~FooTest() override { 503 // You can do clean-up work that doesn't throw exceptions here. 504 } 505 506 // If the constructor and destructor are not enough for setting up 507 // and cleaning up each test, you can define the following methods: 508 509 void SetUp() override { 510 // Code here will be called immediately after the constructor (right 511 // before each test). 512 } 513 514 void TearDown() override { 515 // Code here will be called immediately after each test (right 516 // before the destructor). 517 } 518 519 // Objects declared here can be used by all tests in the test case for Foo. 520}; 521 522// Tests that the Foo::Bar() method does Abc. 523TEST_F(FooTest, MethodBarDoesAbc) { 524 const std::string input_filepath = "this/package/testdata/myinputfile.dat"; 525 const std::string output_filepath = "this/package/testdata/myoutputfile.dat"; 526 Foo f; 527 EXPECT_EQ(f.Bar(input_filepath, output_filepath), 0); 528} 529 530// Tests that Foo does Xyz. 531TEST_F(FooTest, DoesXyz) { 532 // Exercises the Xyz feature of Foo. 533} 534 535} // namespace 536 537int main(int argc, char **argv) { 538 ::testing::InitGoogleTest(&argc, argv); 539 return RUN_ALL_TESTS(); 540} 541``` 542 543 544The `::testing::InitGoogleTest()` function parses the command line for 545googletest flags, and removes all recognized flags. This allows the user to 546control a test program's behavior via various flags, which we'll cover in 547[AdvancedGuide](advanced.md). You **must** call this function before calling 548`RUN_ALL_TESTS()`, or the flags won't be properly initialized. 549 550On Windows, `InitGoogleTest()` also works with wide strings, so it can be used 551in programs compiled in `UNICODE` mode as well. 552 553But maybe you think that writing all those main() functions is too much work? We 554agree with you completely and that's why Google Test provides a basic 555implementation of main(). If it fits your needs, then just link your test with 556gtest\_main library and you are good to go. 557 558NOTE: `ParseGUnitFlags()` is deprecated in favor of `InitGoogleTest()`. 559 560 561## Known Limitations 562 563* Google Test is designed to be thread-safe. The implementation is thread-safe 564 on systems where the `pthreads` library is available. It is currently 565 _unsafe_ to use Google Test assertions from two threads concurrently on 566 other systems (e.g. Windows). In most tests this is not an issue as usually 567 the assertions are done in the main thread. If you want to help, you can 568 volunteer to implement the necessary synchronization primitives in 569 `gtest-port.h` for your platform. 570