1 2 3# Introduction: Why Google C++ Testing Framework? # 4 5_Google C++ Testing Framework_ helps you write better C++ tests. 6 7No matter whether you work on Linux, Windows, or a Mac, if you write C++ code, 8Google Test can help you. 9 10So what makes a good test, and how does Google C++ Testing Framework fit in? We believe: 11 1. Tests should be _independent_ and _repeatable_. It's a pain to debug a test that succeeds or fails as a result of other tests. Google C++ Testing Framework isolates the tests by running each of them on a different object. When a test fails, Google C++ Testing Framework allows you to run it in isolation for quick debugging. 12 1. Tests should be well _organized_ and reflect the structure of the tested code. Google C++ Testing Framework groups related tests into test cases that can share data and subroutines. This common pattern is easy to recognize and makes tests easy to maintain. Such consistency is especially helpful when people switch projects and start to work on a new code base. 13 1. Tests should be _portable_ and _reusable_. The open-source community has a lot of code that is platform-neutral, its tests should also be platform-neutral. Google C++ Testing Framework works on different OSes, with different compilers (gcc, MSVC, and others), with or without exceptions, so Google C++ Testing Framework tests can easily work with a variety of configurations. (Note that the current release only contains build scripts for Linux - we are actively working on scripts for other platforms.) 14 1. When tests fail, they should provide as much _information_ about the problem as possible. Google C++ Testing Framework doesn't stop at the first test failure. Instead, it only stops the current test and continues with the next. You can also set up tests that report non-fatal failures after which the current test continues. Thus, you can detect and fix multiple bugs in a single run-edit-compile cycle. 15 1. The testing framework should liberate test writers from housekeeping chores and let them focus on the test _content_. Google C++ Testing Framework automatically keeps track of all tests defined, and doesn't require the user to enumerate them in order to run them. 16 1. Tests should be _fast_. With Google C++ Testing Framework, you can reuse shared resources across tests and pay for the set-up/tear-down only once, without making tests depend on each other. 17 18Since Google C++ Testing Framework is based on the popular xUnit 19architecture, you'll feel right at home if you've used JUnit or PyUnit before. 20If not, it will take you about 10 minutes to learn the basics and get started. 21So let's go! 22 23_Note:_ We sometimes refer to Google C++ Testing Framework informally 24as _Google Test_. 25 26# Setting up a New Test Project # 27 28To write a test program using Google Test, you need to compile Google 29Test into a library and link your test with it. We provide build 30files for some popular build systems: `msvc/` for Visual Studio, 31`xcode/` for Mac Xcode, `make/` for GNU make, `codegear/` for Borland 32C++ Builder, and the autotools script (deprecated) and 33`CMakeLists.txt` for CMake (recommended) in the Google Test root 34directory. If your build system is not on this list, you can take a 35look at `make/Makefile` to learn how Google Test should be compiled 36(basically you want to compile `src/gtest-all.cc` with `GTEST_ROOT` 37and `GTEST_ROOT/include` in the header search path, where `GTEST_ROOT` 38is the Google Test root directory). 39 40Once you are able to compile the Google Test library, you should 41create a project or build target for your test program. Make sure you 42have `GTEST_ROOT/include` in the header search path so that the 43compiler can find `"gtest/gtest.h"` when compiling your test. Set up 44your test project to link with the Google Test library (for example, 45in Visual Studio, this is done by adding a dependency on 46`gtest.vcproj`). 47 48If you still have questions, take a look at how Google Test's own 49tests are built and use them as examples. 50 51# Basic Concepts # 52 53When using Google Test, you start by writing _assertions_, which are statements 54that check whether a condition is true. An assertion's result can be _success_, 55_nonfatal failure_, or _fatal failure_. If a fatal failure occurs, it aborts 56the current function; otherwise the program continues normally. 57 58_Tests_ use assertions to verify the tested code's behavior. If a test crashes 59or has a failed assertion, then it _fails_; otherwise it _succeeds_. 60 61A _test case_ contains one or many tests. You should group your tests into test 62cases that reflect the structure of the tested code. When multiple tests in a 63test case need to share common objects and subroutines, you can put them into a 64_test fixture_ class. 65 66A _test program_ can contain multiple test cases. 67 68We'll now explain how to write a test program, starting at the individual 69assertion level and building up to tests and test cases. 70 71# Assertions # 72 73Google Test assertions are macros that resemble function calls. You test a 74class or function by making assertions about its behavior. When an assertion 75fails, Google Test prints the assertion's source file and line number location, 76along with a failure message. You may also supply a custom failure message 77which will be appended to Google Test's message. 78 79The assertions come in pairs that test the same thing but have different 80effects on the current function. `ASSERT_*` versions generate fatal failures 81when they fail, and **abort the current function**. `EXPECT_*` versions generate 82nonfatal failures, which don't abort the current function. Usually `EXPECT_*` 83are preferred, as they allow more than one failures to be reported in a test. 84However, you should use `ASSERT_*` if it doesn't make sense to continue when 85the assertion in question fails. 86 87Since a failed `ASSERT_*` returns from the current function immediately, 88possibly skipping clean-up code that comes after it, it may cause a space leak. 89Depending on the nature of the leak, it may or may not be worth fixing - so 90keep this in mind if you get a heap checker error in addition to assertion 91errors. 92 93To provide a custom failure message, simply stream it into the macro using the 94`<<` operator, or a sequence of such operators. An example: 95``` 96ASSERT_EQ(x.size(), y.size()) << "Vectors x and y are of unequal length"; 97 98for (int i = 0; i < x.size(); ++i) { 99 EXPECT_EQ(x[i], y[i]) << "Vectors x and y differ at index " << i; 100} 101``` 102 103Anything that can be streamed to an `ostream` can be streamed to an assertion 104macro--in particular, C strings and `string` objects. If a wide string 105(`wchar_t*`, `TCHAR*` in `UNICODE` mode on Windows, or `std::wstring`) is 106streamed to an assertion, it will be translated to UTF-8 when printed. 107 108## Basic Assertions ## 109 110These assertions do basic true/false condition testing. 111 112| **Fatal assertion** | **Nonfatal assertion** | **Verifies** | 113|:--------------------|:-----------------------|:-------------| 114| `ASSERT_TRUE(`_condition_`)`; | `EXPECT_TRUE(`_condition_`)`; | _condition_ is true | 115| `ASSERT_FALSE(`_condition_`)`; | `EXPECT_FALSE(`_condition_`)`; | _condition_ is false | 116 117Remember, when they fail, `ASSERT_*` yields a fatal failure and 118returns from the current function, while `EXPECT_*` yields a nonfatal 119failure, allowing the function to continue running. In either case, an 120assertion failure means its containing test fails. 121 122_Availability_: Linux, Windows, Mac. 123 124## Binary Comparison ## 125 126This section describes assertions that compare two values. 127 128| **Fatal assertion** | **Nonfatal assertion** | **Verifies** | 129|:--------------------|:-----------------------|:-------------| 130|`ASSERT_EQ(`_val1_`, `_val2_`);`|`EXPECT_EQ(`_val1_`, `_val2_`);`| _val1_ `==` _val2_ | 131|`ASSERT_NE(`_val1_`, `_val2_`);`|`EXPECT_NE(`_val1_`, `_val2_`);`| _val1_ `!=` _val2_ | 132|`ASSERT_LT(`_val1_`, `_val2_`);`|`EXPECT_LT(`_val1_`, `_val2_`);`| _val1_ `<` _val2_ | 133|`ASSERT_LE(`_val1_`, `_val2_`);`|`EXPECT_LE(`_val1_`, `_val2_`);`| _val1_ `<=` _val2_ | 134|`ASSERT_GT(`_val1_`, `_val2_`);`|`EXPECT_GT(`_val1_`, `_val2_`);`| _val1_ `>` _val2_ | 135|`ASSERT_GE(`_val1_`, `_val2_`);`|`EXPECT_GE(`_val1_`, `_val2_`);`| _val1_ `>=` _val2_ | 136 137In the event of a failure, Google Test prints both _val1_ and _val2_. 138 139Value arguments must be comparable by the assertion's comparison 140operator or you'll get a compiler error. We used to require the 141arguments to support the `<<` operator for streaming to an `ostream`, 142but it's no longer necessary since v1.6.0 (if `<<` is supported, it 143will be called to print the arguments when the assertion fails; 144otherwise Google Test will attempt to print them in the best way it 145can. For more details and how to customize the printing of the 146arguments, see this Google Mock [recipe](../../googlemock/docs/CookBook.md#teaching-google-mock-how-to-print-your-values).). 147 148These assertions can work with a user-defined type, but only if you define the 149corresponding comparison operator (e.g. `==`, `<`, etc). If the corresponding 150operator is defined, prefer using the `ASSERT_*()` macros because they will 151print out not only the result of the comparison, but the two operands as well. 152 153Arguments are always evaluated exactly once. Therefore, it's OK for the 154arguments to have side effects. However, as with any ordinary C/C++ function, 155the arguments' evaluation order is undefined (i.e. the compiler is free to 156choose any order) and your code should not depend on any particular argument 157evaluation order. 158 159`ASSERT_EQ()` does pointer equality on pointers. If used on two C strings, it 160tests if they are in the same memory location, not if they have the same value. 161Therefore, if you want to compare C strings (e.g. `const char*`) by value, use 162`ASSERT_STREQ()` , which will be described later on. In particular, to assert 163that a C string is `NULL`, use `ASSERT_STREQ(NULL, c_string)` . However, to 164compare two `string` objects, you should use `ASSERT_EQ`. 165 166Macros in this section work with both narrow and wide string objects (`string` 167and `wstring`). 168 169_Availability_: Linux, Windows, Mac. 170 171_Historical note_: Before February 2016 `*_EQ` had a convention of calling it as 172`ASSERT_EQ(expected, actual)`, so lots of existing code uses this order. 173Now `*_EQ` treats both parameters in the same way. 174 175## String Comparison ## 176 177The assertions in this group compare two **C strings**. If you want to compare 178two `string` objects, use `EXPECT_EQ`, `EXPECT_NE`, and etc instead. 179 180| **Fatal assertion** | **Nonfatal assertion** | **Verifies** | 181|:--------------------|:-----------------------|:-------------| 182| `ASSERT_STREQ(`_str1_`, `_str2_`);` | `EXPECT_STREQ(`_str1_`, `_str_2`);` | the two C strings have the same content | 183| `ASSERT_STRNE(`_str1_`, `_str2_`);` | `EXPECT_STRNE(`_str1_`, `_str2_`);` | the two C strings have different content | 184| `ASSERT_STRCASEEQ(`_str1_`, `_str2_`);`| `EXPECT_STRCASEEQ(`_str1_`, `_str2_`);` | the two C strings have the same content, ignoring case | 185| `ASSERT_STRCASENE(`_str1_`, `_str2_`);`| `EXPECT_STRCASENE(`_str1_`, `_str2_`);` | the two C strings have different content, ignoring case | 186 187Note that "CASE" in an assertion name means that case is ignored. 188 189`*STREQ*` and `*STRNE*` also accept wide C strings (`wchar_t*`). If a 190comparison of two wide strings fails, their values will be printed as UTF-8 191narrow strings. 192 193A `NULL` pointer and an empty string are considered _different_. 194 195_Availability_: Linux, Windows, Mac. 196 197See also: For more string comparison tricks (substring, prefix, suffix, and 198regular expression matching, for example), see the [Advanced Google Test Guide](AdvancedGuide.md). 199 200# Simple Tests # 201 202To create a test: 203 1. Use the `TEST()` macro to define and name a test function, These are ordinary C++ functions that don't return a value. 204 1. In this function, along with any valid C++ statements you want to include, use the various Google Test assertions to check values. 205 1. The test's result is determined by the assertions; if any assertion in the test fails (either fatally or non-fatally), or if the test crashes, the entire test fails. Otherwise, it succeeds. 206 207``` 208TEST(test_case_name, test_name) { 209 ... test body ... 210} 211``` 212 213 214`TEST()` arguments go from general to specific. The _first_ argument is the 215name of the test case, and the _second_ argument is the test's name within the 216test case. Both names must be valid C++ identifiers, and they should not contain underscore (`_`). A test's _full name_ consists of its containing test case and its 217individual name. Tests from different test cases can have the same individual 218name. 219 220For example, let's take a simple integer function: 221``` 222int Factorial(int n); // Returns the factorial of n 223``` 224 225A test case for this function might look like: 226``` 227// Tests factorial of 0. 228TEST(FactorialTest, HandlesZeroInput) { 229 EXPECT_EQ(1, Factorial(0)); 230} 231 232// Tests factorial of positive numbers. 233TEST(FactorialTest, HandlesPositiveInput) { 234 EXPECT_EQ(1, Factorial(1)); 235 EXPECT_EQ(2, Factorial(2)); 236 EXPECT_EQ(6, Factorial(3)); 237 EXPECT_EQ(40320, Factorial(8)); 238} 239``` 240 241Google Test groups the test results by test cases, so logically-related tests 242should be in the same test case; in other words, the first argument to their 243`TEST()` should be the same. In the above example, we have two tests, 244`HandlesZeroInput` and `HandlesPositiveInput`, that belong to the same test 245case `FactorialTest`. 246 247_Availability_: Linux, Windows, Mac. 248 249# Test Fixtures: Using the Same Data Configuration for Multiple Tests # 250 251If you find yourself writing two or more tests that operate on similar data, 252you can use a _test fixture_. It allows you to reuse the same configuration of 253objects for several different tests. 254 255To create a fixture, just: 256 1. Derive a class from `::testing::Test` . Start its body with `protected:` or `public:` as we'll want to access fixture members from sub-classes. 257 1. Inside the class, declare any objects you plan to use. 258 1. If necessary, write a default constructor or `SetUp()` function to prepare the objects for each test. A common mistake is to spell `SetUp()` as `Setup()` with a small `u` - don't let that happen to you. 259 1. If necessary, write a destructor or `TearDown()` function to release any resources you allocated in `SetUp()` . To learn when you should use the constructor/destructor and when you should use `SetUp()/TearDown()`, read this [FAQ entry](FAQ.md#should-i-use-the-constructordestructor-of-the-test-fixture-or-the-set-uptear-down-function). 260 1. If needed, define subroutines for your tests to share. 261 262When using a fixture, use `TEST_F()` instead of `TEST()` as it allows you to 263access objects and subroutines in the test fixture: 264``` 265TEST_F(test_case_name, test_name) { 266 ... test body ... 267} 268``` 269 270Like `TEST()`, the first argument is the test case name, but for `TEST_F()` 271this must be the name of the test fixture class. You've probably guessed: `_F` 272is for fixture. 273 274Unfortunately, the C++ macro system does not allow us to create a single macro 275that can handle both types of tests. Using the wrong macro causes a compiler 276error. 277 278Also, you must first define a test fixture class before using it in a 279`TEST_F()`, or you'll get the compiler error "`virtual outside class 280declaration`". 281 282For each test defined with `TEST_F()`, Google Test will: 283 1. Create a _fresh_ test fixture at runtime 284 1. Immediately initialize it via `SetUp()` , 285 1. Run the test 286 1. Clean up by calling `TearDown()` 287 1. Delete the test fixture. Note that different tests in the same test case have different test fixture objects, and Google Test always deletes a test fixture before it creates the next one. Google Test does not reuse the same test fixture for multiple tests. Any changes one test makes to the fixture do not affect other tests. 288 289As an example, let's write tests for a FIFO queue class named `Queue`, which 290has the following interface: 291``` 292template <typename E> // E is the element type. 293class Queue { 294 public: 295 Queue(); 296 void Enqueue(const E& element); 297 E* Dequeue(); // Returns NULL if the queue is empty. 298 size_t size() const; 299 ... 300}; 301``` 302 303First, define a fixture class. By convention, you should give it the name 304`FooTest` where `Foo` is the class being tested. 305``` 306class QueueTest : public ::testing::Test { 307 protected: 308 virtual void SetUp() { 309 q1_.Enqueue(1); 310 q2_.Enqueue(2); 311 q2_.Enqueue(3); 312 } 313 314 // virtual void TearDown() {} 315 316 Queue<int> q0_; 317 Queue<int> q1_; 318 Queue<int> q2_; 319}; 320``` 321 322In this case, `TearDown()` is not needed since we don't have to clean up after 323each test, other than what's already done by the destructor. 324 325Now we'll write tests using `TEST_F()` and this fixture. 326``` 327TEST_F(QueueTest, IsEmptyInitially) { 328 EXPECT_EQ(0, q0_.size()); 329} 330 331TEST_F(QueueTest, DequeueWorks) { 332 int* n = q0_.Dequeue(); 333 EXPECT_EQ(NULL, n); 334 335 n = q1_.Dequeue(); 336 ASSERT_TRUE(n != NULL); 337 EXPECT_EQ(1, *n); 338 EXPECT_EQ(0, q1_.size()); 339 delete n; 340 341 n = q2_.Dequeue(); 342 ASSERT_TRUE(n != NULL); 343 EXPECT_EQ(2, *n); 344 EXPECT_EQ(1, q2_.size()); 345 delete n; 346} 347``` 348 349The above uses both `ASSERT_*` and `EXPECT_*` assertions. The rule of thumb is 350to use `EXPECT_*` when you want the test to continue to reveal more errors 351after the assertion failure, and use `ASSERT_*` when continuing after failure 352doesn't make sense. For example, the second assertion in the `Dequeue` test is 353`ASSERT_TRUE(n != NULL)`, as we need to dereference the pointer `n` later, 354which would lead to a segfault when `n` is `NULL`. 355 356When these tests run, the following happens: 357 1. Google Test constructs a `QueueTest` object (let's call it `t1` ). 358 1. `t1.SetUp()` initializes `t1` . 359 1. The first test ( `IsEmptyInitially` ) runs on `t1` . 360 1. `t1.TearDown()` cleans up after the test finishes. 361 1. `t1` is destructed. 362 1. The above steps are repeated on another `QueueTest` object, this time running the `DequeueWorks` test. 363 364_Availability_: Linux, Windows, Mac. 365 366_Note_: Google Test automatically saves all _Google Test_ flags when a test 367object is constructed, and restores them when it is destructed. 368 369# Invoking the Tests # 370 371`TEST()` and `TEST_F()` implicitly register their tests with Google Test. So, unlike with many other C++ testing frameworks, you don't have to re-list all your defined tests in order to run them. 372 373After defining your tests, you can run them with `RUN_ALL_TESTS()` , which returns `0` if all the tests are successful, or `1` otherwise. Note that `RUN_ALL_TESTS()` runs _all tests_ in your link unit -- they can be from different test cases, or even different source files. 374 375When invoked, the `RUN_ALL_TESTS()` macro: 376 1. Saves the state of all Google Test flags. 377 1. Creates a test fixture object for the first test. 378 1. Initializes it via `SetUp()`. 379 1. Runs the test on the fixture object. 380 1. Cleans up the fixture via `TearDown()`. 381 1. Deletes the fixture. 382 1. Restores the state of all Google Test flags. 383 1. Repeats the above steps for the next test, until all tests have run. 384 385In addition, if the text fixture's constructor generates a fatal failure in 386step 2, there is no point for step 3 - 5 and they are thus skipped. Similarly, 387if step 3 generates a fatal failure, step 4 will be skipped. 388 389_Important_: You must not ignore the return value of `RUN_ALL_TESTS()`, or `gcc` 390will give you a compiler error. The rationale for this design is that the 391automated testing service determines whether a test has passed based on its 392exit code, not on its stdout/stderr output; thus your `main()` function must 393return the value of `RUN_ALL_TESTS()`. 394 395Also, you should call `RUN_ALL_TESTS()` only **once**. Calling it more than once 396conflicts with some advanced Google Test features (e.g. thread-safe death 397tests) and thus is not supported. 398 399_Availability_: Linux, Windows, Mac. 400 401# Writing the main() Function # 402 403You can start from this boilerplate: 404``` 405#include "this/package/foo.h" 406#include "gtest/gtest.h" 407 408namespace { 409 410// The fixture for testing class Foo. 411class FooTest : public ::testing::Test { 412 protected: 413 // You can remove any or all of the following functions if its body 414 // is empty. 415 416 FooTest() { 417 // You can do set-up work for each test here. 418 } 419 420 virtual ~FooTest() { 421 // You can do clean-up work that doesn't throw exceptions here. 422 } 423 424 // If the constructor and destructor are not enough for setting up 425 // and cleaning up each test, you can define the following methods: 426 427 virtual void SetUp() { 428 // Code here will be called immediately after the constructor (right 429 // before each test). 430 } 431 432 virtual void TearDown() { 433 // Code here will be called immediately after each test (right 434 // before the destructor). 435 } 436 437 // Objects declared here can be used by all tests in the test case for Foo. 438}; 439 440// Tests that the Foo::Bar() method does Abc. 441TEST_F(FooTest, MethodBarDoesAbc) { 442 const string input_filepath = "this/package/testdata/myinputfile.dat"; 443 const string output_filepath = "this/package/testdata/myoutputfile.dat"; 444 Foo f; 445 EXPECT_EQ(0, f.Bar(input_filepath, output_filepath)); 446} 447 448// Tests that Foo does Xyz. 449TEST_F(FooTest, DoesXyz) { 450 // Exercises the Xyz feature of Foo. 451} 452 453} // namespace 454 455int main(int argc, char **argv) { 456 ::testing::InitGoogleTest(&argc, argv); 457 return RUN_ALL_TESTS(); 458} 459``` 460 461The `::testing::InitGoogleTest()` function parses the command line for Google 462Test flags, and removes all recognized flags. This allows the user to control a 463test program's behavior via various flags, which we'll cover in [AdvancedGuide](AdvancedGuide.md). 464You must call this function before calling `RUN_ALL_TESTS()`, or the flags 465won't be properly initialized. 466 467On Windows, `InitGoogleTest()` also works with wide strings, so it can be used 468in programs compiled in `UNICODE` mode as well. 469 470But maybe you think that writing all those main() functions is too much work? We agree with you completely and that's why Google Test provides a basic implementation of main(). If it fits your needs, then just link your test with gtest\_main library and you are good to go. 471 472## Important note for Visual C++ users ## 473If you put your tests into a library and your `main()` function is in a different library or in your .exe file, those tests will not run. The reason is a [bug](https://connect.microsoft.com/feedback/viewfeedback.aspx?FeedbackID=244410&siteid=210) in Visual C++. When you define your tests, Google Test creates certain static objects to register them. These objects are not referenced from elsewhere but their constructors are still supposed to run. When Visual C++ linker sees that nothing in the library is referenced from other places it throws the library out. You have to reference your library with tests from your main program to keep the linker from discarding it. Here is how to do it. Somewhere in your library code declare a function: 474``` 475__declspec(dllexport) int PullInMyLibrary() { return 0; } 476``` 477If you put your tests in a static library (not DLL) then `__declspec(dllexport)` is not required. Now, in your main program, write a code that invokes that function: 478``` 479int PullInMyLibrary(); 480static int dummy = PullInMyLibrary(); 481``` 482This will keep your tests referenced and will make them register themselves at startup. 483 484In addition, if you define your tests in a static library, add `/OPT:NOREF` to your main program linker options. If you use MSVC++ IDE, go to your .exe project properties/Configuration Properties/Linker/Optimization and set References setting to `Keep Unreferenced Data (/OPT:NOREF)`. This will keep Visual C++ linker from discarding individual symbols generated by your tests from the final executable. 485 486There is one more pitfall, though. If you use Google Test as a static library (that's how it is defined in gtest.vcproj) your tests must also reside in a static library. If you have to have them in a DLL, you _must_ change Google Test to build into a DLL as well. Otherwise your tests will not run correctly or will not run at all. The general conclusion here is: make your life easier - do not write your tests in libraries! 487 488# Where to Go from Here # 489 490Congratulations! You've learned the Google Test basics. You can start writing 491and running Google Test tests, read some [samples](Samples.md), or continue with 492[AdvancedGuide](AdvancedGuide.md), which describes many more useful Google Test features. 493 494# Known Limitations # 495 496Google Test is designed to be thread-safe. The implementation is 497thread-safe on systems where the `pthreads` library is available. It 498is currently _unsafe_ to use Google Test assertions from two threads 499concurrently on other systems (e.g. Windows). In most tests this is 500not an issue as usually the assertions are done in the main thread. If 501you want to help, you can volunteer to implement the necessary 502synchronization primitives in `gtest-port.h` for your platform. 503