1=====================
2Retryable Reads Tests
3=====================
4
5.. contents::
6
7----
8
9Introduction
10============
11
12The YAML and JSON files in this directory tree are platform-independent tests
13that drivers can use to prove their conformance to the Retryable Reads spec.
14
15Prose tests, which are not easily expressed in YAML, are also presented
16in this file. Those tests will need to be manually implemented by each driver.
17
18Tests will require a MongoClient created with options defined in the tests.
19Integration tests will require a running MongoDB cluster with server versions
204.0 or later.
21
22N.B. The spec specifies 3.6 as the minimum server version: however,
23``failCommand`` is not supported on 3.6, so for now, testing requires MongoDB
244.0. Once `DRIVERS-560`_ is resolved, we will attempt to adapt its live failure
25integration tests to test Retryable Reads on MongoDB 3.6.
26
27.. _DRIVERS-560: https://jira.mongodb.org/browse/DRIVERS-560
28
29Server Fail Point
30=================
31
32See: `Server Fail Point`_ in the Transactions spec test suite.
33
34.. _Server Fail Point: ../../transactions/tests#server-fail-point
35
36Disabling Fail Point after Test Execution
37-----------------------------------------
38
39After each test that configures a fail point, drivers should disable the
40``failCommand`` fail point to avoid spurious failures in
41subsequent tests. The fail point may be disabled like so::
42
43 db.runCommand({
44 configureFailPoint: "failCommand",
45 mode: "off"
46 });
47
48Network Error Tests
49===================
50
51Network error tests are expressed in YAML and should be run against a standalone,
52shard cluster, or single-node replica set.
53
54
55Test Format
56-----------
57
58Each YAML file has the following keys:
59
60- ``runOn`` (optional): An array of server version and/or topology requirements
61 for which the tests can be run. If the test environment satisfies one or more
62 of these requirements, the tests may be executed; otherwise, this file should
63 be skipped. If this field is omitted, the tests can be assumed to have no
64 particular requirements and should be executed. Each element will have some or
65 all of the following fields:
66
67 - ``minServerVersion`` (optional): The minimum server version (inclusive)
68 required to successfully run the tests. If this field is omitted, it should
69 be assumed that there is no lower bound on the required server version.
70
71 - ``maxServerVersion`` (optional): The maximum server version (inclusive)
72 against which the tests can be run successfully. If this field is omitted,
73 it should be assumed that there is no upper bound on the required server
74 version.
75
76 - ``topology`` (optional): An array of server topologies against which the
77 tests can be run successfully. Valid topologies are "single", "replicaset",
78 and "sharded". If this field is omitted, the default is all topologies (i.e.
79 ``["single", "replicaset", "sharded"]``).
80
81- ``database_name`` and ``collection_name``: Optional. The database and
82 collection to use for testing.
83
84- ``bucket_name``: Optional. The GridFS bucket name to use for testing.
85
86- ``data``: The data that should exist in the collection(s) under test before
87 each test run.
88
89- ``tests``: An array of tests that are to be run independently of each other.
90 Each test will have some or all of the following fields:
91
92 - ``description``: The name of the test.
93
94 - ``clientOptions``: Optional, parameters to pass to MongoClient().
95
96 - ``useMultipleMongoses`` (optional): If ``true``, the MongoClient for this
97 test should be initialized with multiple mongos seed addresses. If ``false``
98 or omitted, only a single mongos address should be specified. This field has
99 no effect for non-sharded topologies.
100
101 - ``skipReason``: Optional, string describing why this test should be skipped.
102
103 - ``failPoint``: Optional, a server fail point to enable, expressed as the
104 configureFailPoint command to run on the admin database.
105
106 - ``operations``: An array of documents describing an operation to be
107 executed. Each document has the following fields:
108
109 - ``name``: The name of the operation on ``object``.
110
111 - ``object``: The name of the object to perform the operation on. Can be
112 "database", "collection", "client", or "gridfsbucket."
113
114 - ``arguments``: Optional, the names and values of arguments.
115
116 - ``result``: Optional. The return value from the operation, if any. This
117 field may be a scalar (e.g. in the case of a count), a single document, or
118 an array of documents in the case of a multi-document read.
119
120 - ``error``: Optional. If ``true``, the test should expect an error or
121 exception.
122
123 - ``expectations``: Optional list of command-started events.
124
125GridFS Tests
126------------
127
128GridFS tests are denoted by when the YAML file contains ``bucket_name``.
129
130``fs.files`` and ``fs.chunks`` should be created in the database
131specified by ``database_name``. This could be done via inserts or by
132creating GridFSBuckets—using the GridFS ``bucketName`` (see
133`GridFSBucket spec`_) specified by ``bucket_name`` field in the YAML
134file—and calling ``upload_from_stream_with_id`` with the appropriate
135data.
136
137``Download`` tests should be tested against ``GridFS.download_to_stream``.
138``DownloadByName`` tests should be tested against
139``GridFS.download_to_stream_by_name``.
140
141
142.. _GridFSBucket spec: https://github.com/mongodb/specifications/blob/master/source/gridfs/gridfs-spec.rst#configurable-gridfsbucket-class
143
144Speeding Up Tests
145-----------------
146
147Drivers may benefit reducing `minHeartbeatFrequencyMS`_ in order to speed up
148tests. Python was able to decrease the run time of the tests greatly by lowering
149the SDAM's ``minHeartbeatFrequencyMS`` from 500ms to 50ms, thus decreasing the
150waiting time after a "not master" error:
151
152.. _minHeartbeatFrequencyMS: https://github.com/mongodb/specifications/blob/master/source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst#minheartbeatfrequencyms
153Optional Enumeration Commands
154=============================
155
156A driver only needs to test the optional enumeration commands it has chosen to
157implement (e.g. ``Database.listCollectionNames()``).
158
159Changelog
160=========
161
162:2019-03-19: Add top-level ``runOn`` field to denote server version and/or
163 topology requirements requirements for the test file. Removes the
164 ``minServerVersion`` and ``topology`` top-level fields, which are
165 now expressed within ``runOn`` elements.
166
167 Add test-level ``useMultipleMongoses`` field.
168