README.md
1Regression tests
2================
3
4### [test_framework/authproxy.py](test_framework/authproxy.py)
5Taken from the [python-bitcoinrpc repository](https://github.com/jgarzik/python-bitcoinrpc).
6
7### [test_framework/test_framework.py](test_framework/test_framework.py)
8Base class for new regression tests.
9
10### [test_framework/util.py](test_framework/util.py)
11Generally useful functions.
12
13### [test_framework/mininode.py](test_framework/mininode.py)
14Basic code to support p2p connectivity to a bitcoind.
15
16### [test_framework/comptool.py](test_framework/comptool.py)
17Framework for comparison-tool style, p2p tests.
18
19### [test_framework/script.py](test_framework/script.py)
20Utilities for manipulating transaction scripts (originally from python-bitcoinlib)
21
22### [test_framework/blockstore.py](test_framework/blockstore.py)
23Implements disk-backed block and tx storage.
24
25### [test_framework/key.py](test_framework/key.py)
26Wrapper around OpenSSL EC_Key (originally from python-bitcoinlib)
27
28### [test_framework/bignum.py](test_framework/bignum.py)
29Helpers for script.py
30
31### [test_framework/blocktools.py](test_framework/blocktools.py)
32Helper functions for creating blocks and transactions.
33
34P2P test design notes
35---------------------
36
37## Mininode
38
39* ```mininode.py``` contains all the definitions for objects that pass
40over the network (```CBlock```, ```CTransaction```, etc, along with the network-level
41wrappers for them, ```msg_block```, ```msg_tx```, etc).
42
43* P2P tests have two threads. One thread handles all network communication
44with the bitcoind(s) being tested (using python's asyncore package); the other
45implements the test logic.
46
47* ```NodeConn``` is the class used to connect to a bitcoind. If you implement
48a callback class that derives from ```NodeConnCB``` and pass that to the
49```NodeConn``` object, your code will receive the appropriate callbacks when
50events of interest arrive.
51
52* You can pass the same handler to multiple ```NodeConn```'s if you like, or pass
53different ones to each -- whatever makes the most sense for your test.
54
55* Call ```NetworkThread.start()``` after all ```NodeConn``` objects are created to
56start the networking thread. (Continue with the test logic in your existing
57thread.)
58
59* RPC calls are available in p2p tests.
60
61* Can be used to write free-form tests, where specific p2p-protocol behavior
62is tested. Examples: ```p2p-accept-block.py```, ```maxblocksinflight.py```.
63
64## Comptool
65
66* Testing framework for writing tests that compare the block/tx acceptance
67behavior of a bitcoind against 1 or more other bitcoind instances, or against
68known outcomes, or both.
69
70* Set the ```num_nodes``` variable (defined in ```ComparisonTestFramework```) to start up
711 or more nodes. If using 1 node, then ```--testbinary``` can be used as a command line
72option to change the bitcoind binary used by the test. If using 2 or more nodes,
73then ```--refbinary``` can be optionally used to change the bitcoind that will be used
74on nodes 2 and up.
75
76* Implement a (generator) function called ```get_tests()``` which yields ```TestInstance```s.
77Each ```TestInstance``` consists of:
78 - a list of ```[object, outcome, hash]``` entries
79 * ```object``` is a ```CBlock```, ```CTransaction```, or
80 ```CBlockHeader```. ```CBlock```'s and ```CTransaction```'s are tested for
81 acceptance. ```CBlockHeader```s can be used so that the test runner can deliver
82 complete headers-chains when requested from the bitcoind, to allow writing
83 tests where blocks can be delivered out of order but still processed by
84 headers-first bitcoind's.
85 * ```outcome``` is ```True```, ```False```, or ```None```. If ```True```
86 or ```False```, the tip is compared with the expected tip -- either the
87 block passed in, or the hash specified as the optional 3rd entry. If
88 ```None``` is specified, then the test will compare all the bitcoind's
89 being tested to see if they all agree on what the best tip is.
90 * ```hash``` is the block hash of the tip to compare against. Optional to
91 specify; if left out then the hash of the block passed in will be used as
92 the expected tip. This allows for specifying an expected tip while testing
93 the handling of either invalid blocks or blocks delivered out of order,
94 which complete a longer chain.
95 - ```sync_every_block```: ```True/False```. If ```False```, then all blocks
96 are inv'ed together, and the test runner waits until the node receives the
97 last one, and tests only the last block for tip acceptance using the
98 outcome and specified tip. If ```True```, then each block is tested in
99 sequence and synced (this is slower when processing many blocks).
100 - ```sync_every_transaction```: ```True/False```. Analogous to
101 ```sync_every_block```, except if the outcome on the last tx is "None",
102 then the contents of the entire mempool are compared across all bitcoind
103 connections. If ```True``` or ```False```, then only the last tx's
104 acceptance is tested against the given outcome.
105
106* For examples of tests written in this framework, see
107 ```invalidblockrequest.py``` and ```p2p-fullblocktest.py```.
108
109