1/* 2Package framework implements a model for developing end-to-end test suites. The 3model includes a top level Framework which TestSuites can be added to. TestSuites 4include conditions under which the suite will run and a list of TestCase 5implementations to run. TestCases can be implemented with methods that run 6before/after each and all tests. 7 8Writing Tests 9 10Tests follow a similar patterns as go tests. They are functions that must start 11with 'Test' and instead of a *testing.T argument, a *framework.F is passed and 12they must have a receiver that implements the TestCase interface. 13A crude example as follows: 14 15 // foo_test.go 16 type MyTestCase struct { 17 framework.TC 18 } 19 20 func (tc *MyTestCase) TestMyFoo(f *framework.F) { 21 f.T().Log("bar") 22 } 23 24 func TestCalledFromGoTest(t *testing.T){ 25 framework.New().AddSuites(&framework.TestSuite{ 26 Component: "foo", 27 Cases: []framework.TestCase{ 28 new(MyTestCase), 29 }, 30 }).Run(t) 31 } 32 33Test cases should embed the TC struct which satisfies the TestCase interface. 34Optionally a TestCase can also implement the Name() function which returns 35a string to name the test case. By default the name is the name of the struct 36type, which in the above example would be "MyTestCase" 37 38Test cases may also optionally implement additional interfaces to define setup 39and teardown logic: 40 41 BeforeEachTest 42 AfterEachTest 43 BeforeAllTests 44 AfterAllTests 45 46The test case struct allows you to setup and teardown state in the struct that 47can be consumed by the tests. For example: 48 49 type ComplexNomadTC struct { 50 framework.TC 51 jobID string 52 } 53 54 func (tc *ComplexNomadTC) BeforeEach(f *framework.F){ 55 // Do some complex job setup with a unique prefix string 56 jobID, err := doSomeComplexSetup(tc.Nomad(), f.ID()) 57 f.NoError(err) 58 f.Set("jobID", jobID) 59 } 60 61 func (tc *ComplexNomadTC) TestSomeScenario(f *framework.F){ 62 jobID := f.Value("jobID").(string) 63 doTestThingWithJob(f, tc.Nomad(), jobID) 64 } 65 66 func (tc *ComplexNomadTC) TestOtherScenario(f *framework.F){ 67 jobID := f.Value("jobID").(string) 68 doOtherTestThingWithJob(f, tc.Nomad(), jobID) 69 } 70 71 func (tc *ComplexNomadTC) AfterEach(f *framework.F){ 72 jobID := f.Value("jobID").(string) 73 _, _, err := tc.Nomad().Jobs().Deregister(jobID, true, nil) 74 f.NoError(err) 75 } 76 77As demonstrated in the previous example, TC also exposes functions that return 78configured api clients including Nomad, Consul and Vault. If Consul or Vault 79are not provisioned their respective getter functions will return nil. 80 81Testify Integration 82 83Test cases expose a T() function to fetch the current *testing.T context. 84While this means the author is able to most other testing libraries, 85github.com/stretch/testify is recommended and integrated into the framework. 86The TC struct also embeds testify assertions that are preconfigured with the 87current testing context. Additionally TC comes with a Require() method that 88yields a testify Require if that flavor is desired. 89 90 func (tc *MyTestCase) TestWithTestify() { 91 err := someErrFunc() 92 tc.NoError(err) 93 // Or tc.Require().NoError(err) 94 } 95 96Parallelism 97 98The test framework honors go test's parallel feature under certain conditions. 99A TestSuite can be created with the Parallel field set to true to enable 100parallel execution of the test cases of the suite. Tests within a test case 101will be executed sequentially unless f.T().Parallel() is called. Note that if 102multiple tests are to be executed in parallel, access to TC is not syncronized. 103The *framework.F offers a way to store state between before/after each method if 104desired. 105 106 func (tc *MyTestCase) BeforeEach(f *framework.F){ 107 jobID, _ := doSomeComplexSetup(tc.Nomad(), f.ID()) 108 f.Set("jobID", jobID) 109 } 110 111 func (tc *MyTestCase) TestParallel(f *framework.F){ 112 f.T().Parallel() 113 jobID := f.Value("jobID").(string) 114 } 115 116Since test cases have the potential to work with a shared Nomad cluster in parallel 117any resources created or destroyed must be prefixed with a unique identifier for 118each test case. The framework.F struct exposes an ID() function that will return a 119string that is unique with in a test. Therefore, multiple tests with in the case 120can reliably create unique IDs between tests and setup/teardown. The string 121returned is 8 alpha numeric characters. 122 123*/ 124package framework 125