README.rst
1====================================
2python-novaclient functional testing
3====================================
4
5Idea
6----
7
8Over time we have noticed two issues with novaclient unit tests.
9
10* Does not exercise the CLI
11* We can get the expected server behavior wrong, and test the wrong thing.
12
13We are using functional tests, run against a running cloud
14(primarily devstack), to address these two cases.
15
16Additionally these functional tests can be considered example uses
17of python-novaclient.
18
19These tests started out in tempest as read only nova CLI tests, to make sure
20the CLI didn't simply stacktrace when being used (which happened on
21multiple occasions).
22
23
24Testing Theory
25--------------
26
27We are treating python-novaclient as legacy code, so we do not want to spend a
28lot of effort adding in missing features. In the future the CLI will move to
29python-openstackclient, and the python API will be based on the OpenStack
30SDK project. But until that happens we still need better functional testing,
31to prevent regressions etc.
32
33
34Since python-novaclient has two uses, CLI and python API, we should have two
35sets of functional tests. CLI and python API. The python API tests should
36never use the CLI. But the CLI tests can use the python API where adding
37native support to the CLI for the required functionality would involve a
38non trivial amount of work.
39
40Functional Test Guidelines
41--------------------------
42
43* Consume credentials via standard client environmental variables::
44
45 OS_USERNAME
46 OS_PASSWORD
47 OS_TENANT_NAME
48 OS_AUTH_URL
49
50* Usage of insecure SSL can be configured via the standard client environment
51 variable::
52
53 OS_INSECURE
54
55* Try not to require an additional configuration file
56