README.rst
1=====================================
2Convenient API for Transactions Tests
3=====================================
4
5.. contents::
6
7----
8
9Introduction
10============
11
12The YAML and JSON files in this directory are platform-independent tests that
13drivers can use to prove their conformance to the Convenient API for
14Transactions spec. They are designed with the intention of sharing some
15test-runner code with the CRUD, Command Monitoring, and Transaction spec tests.
16
17Several prose tests, which are not easily expressed in YAML, are also presented
18in this file. Those tests will need to be manually implemented by each driver.
19
20Server Fail Point
21=================
22
23See: `Server Fail Point <../../transactions/tests#server-fail-point>`_ in the
24Transactions spec test suite.
25
26Test Format
27===========
28
29Each YAML file has the following keys:
30
31- ``runOn`` (optional): An array of server version and/or topology requirements
32 for which the tests can be run. If the test environment satisfies one or more
33 of these requirements, the tests may be executed; otherwise, this file should
34 be skipped. If this field is omitted, the tests can be assumed to have no
35 particular requirements and should be executed. Each element will have some or
36 all of the following fields:
37
38 - ``minServerVersion`` (optional): The minimum server version (inclusive)
39 required to successfully run the tests. If this field is omitted, it should
40 be assumed that there is no lower bound on the required server version.
41
42 - ``maxServerVersion`` (optional): The maximum server version (inclusive)
43 against which the tests can be run successfully. If this field is omitted,
44 it should be assumed that there is no upper bound on the required server
45 version.
46
47 - ``topology`` (optional): An array of server topologies against which the
48 tests can be run successfully. Valid topologies are "single", "replicaset",
49 and "sharded". If this field is omitted, the default is all topologies (i.e.
50 ``["single", "replicaset", "sharded"]``).
51
52- ``database_name`` and ``collection_name``: The database and collection to use
53 for testing.
54
55- ``data``: The data that should exist in the collection under test before each
56 test run.
57
58- ``tests``: An array of tests that are to be run independently of each other.
59 Each test will have some or all of the following fields:
60
61 - ``description``: The name of the test.
62
63 - ``skipReason`` (optional): If present, the test should be skipped and the
64 string value will specify a reason.
65
66 - ``failPoint`` (optional): The ``configureFailPoint`` command document to run
67 to configure a fail point on the primary server. This option and
68 ``useMultipleMongoses: true`` are mutually exclusive.
69
70 - ``useMultipleMongoses`` (optional): If ``true``, the MongoClient for this
71 test should be initialized with multiple mongos seed addresses. If ``false``
72 or omitted, only a single mongos address should be specified. This field has
73 no effect for non-sharded topologies.
74
75 - ``clientOptions`` (optional): Names and values of options to pass to
76 ``MongoClient()``.
77
78 - ``sessionOptions`` (optional): Names and values of options to pass to
79 ``MongoClient.startSession()``.
80
81 - ``operations``: Array of documents, each describing an operation to be
82 executed. Each document has the following fields:
83
84 - ``name``: The name of the operation on ``object``.
85
86 - ``object``: The name of the object on which to perform the operation. The
87 value will be either "collection" or "session0".
88
89 - ``arguments`` (optional): Names and values of arguments to pass to the
90 operation.
91
92 - ``result`` (optional): The return value from the operation. This will
93 correspond to an operation's result object as defined in the CRUD
94 specification. If the operation is expected to return an error, the
95 ``result`` is a single document that has one or more of the following
96 fields:
97
98 - ``errorContains``: A substring of the expected error message.
99
100 - ``errorCodeName``: The expected "codeName" field in the server
101 error response.
102
103 - ``errorLabelsContain``: A list of error label strings that the
104 error is expected to have.
105
106 - ``errorLabelsOmit``: A list of error label strings that the
107 error is expected not to have.
108
109 - ``expectations`` (optional): List of command-started events.
110
111 - ``outcome``: Document describing the return value and/or expected state of
112 the collection after the operation is executed. Contains the following
113 fields:
114
115 - ``collection``:
116
117 - ``data``: The data that should exist in the collection after the
118 operations have run.
119
120``withTransaction`` Operation
121`````````````````````````````
122
123These tests introduce a ``withTransaction`` operation, which may have the
124following fields:
125
126- ``callback``: Document containing the following field:
127
128 - ``operations``: Array of documents, each describing an operation to be
129 executed. Elements in this array will follow the same structure as the
130 ``operations`` field defined above (and in the CRUD and Transactions specs).
131
132 Note that drivers are expected to evaluate ``error` and ``result``
133 assertions when executing operations within ``callback.operations``.
134
135- ``options`` (optional): Names and values of options to pass to
136 ``withTransaction()``, which will in turn be used for ``startTransaction()``.
137
138Use as Integration Tests
139========================
140
141Testing against a replica set will require server version 4.0.0 or later. The
142replica set should include a primary, a secondary, and an arbiter. Including a
143secondary ensures that server selection in a transaction works properly.
144Including an arbiter helps ensure that no new bugs have been introduced related
145to arbiters.
146
147Testing against a sharded cluster will require server version 4.1.6 or later.
148A driver that implements support for sharded transactions MUST also run these
149tests against a MongoDB sharded cluster with multiple mongoses. Including
150multiple mongoses (and initializing the MongoClient with multiple mongos seeds!)
151ensures that mongos transaction pinning works properly.
152
153See: `Use as Integration Tests <../../transactions/tests#use-as-integration-tests>`_
154in the Transactions spec test suite for instructions on executing each test.
155
156Take note of the following:
157
158- Most tests will consist of a single "withTransaction" operation to be invoked
159 on the "session0" object. The ``callback`` argument of that operation will
160 resemble the ``operations`` array found in transaction spec tests.
161
162Command-Started Events
163``````````````````````
164
165See: `Command-Started Events <../../transactions/tests#command-started-events>`_
166in the Transactions spec test suite for instructions on asserting
167command-started events.
168
169Prose Tests
170===========
171
172Callback Raises a Custom Error
173``````````````````````````````
174
175Write a callback that raises a custom exception or error that does not include
176either UnknownTransactionCommitResult or TransientTransactionError error labels.
177Execute this callback using ``withTransaction`` and assert that the callback's
178error bypasses any retry logic within ``withTransaction`` and is propagated to
179the caller of ``withTransaction``.
180
181Callback Returns a Value
182````````````````````````
183
184Write a callback that returns a custom value (e.g. boolean, string, object).
185Execute this callback using ``withTransaction`` and assert that the callback's
186return value is propagated to the caller of ``withTransaction``.
187
188Retry Timeout is Enforced
189`````````````````````````
190
191Drivers should test that ``withTransaction`` enforces a non-configurable timeout
192before retrying both commits and entire transactions. Specifically, three cases
193should be checked:
194
195 * If the callback raises an error with the TransientTransactionError label and
196 the retry timeout has been exceeded, ``withTransaction`` should propagate the
197 error to its caller.
198 * If committing raises an error with the UnknownTransactionCommitResult label,
199 the error is not a write concern timeout, and the retry timeout has been
200 exceeded, ``withTransaction`` should propagate the error to its caller.
201 * If committing raises an error with the TransientTransactionError label and
202 the retry timeout has been exceeded, ``withTransaction`` should propagate the
203 error to its caller. This case may occur if the commit was internally retried
204 against a new primary after a failover and the second primary returned a
205 NoSuchTransaction error response.
206
207 If possible, drivers should implement these tests without requiring the test
208 runner to block for the full duration of the retry timeout. This might be done
209 by internally modifying the timeout value used by ``withTransaction`` with some
210 private API or using a mock timer.
211
212Changelog
213=========
214
215:2019-03-01: Add top-level ``runOn`` field to denote server version and/or
216 topology requirements requirements for the test file. Removes the
217 ``minServerVersion`` top-level field, which is now expressed within
218 ``runOn`` elements.
219
220 Add test-level ``useMultipleMongoses`` field.
221