1# Block and Transaction Broadcasting with ZeroMQ 2 3[ZeroMQ](https://zeromq.org/) is a lightweight wrapper around TCP 4connections, inter-process communication, and shared-memory, 5providing various message-oriented semantics such as publish/subscribe, 6request/reply, and push/pull. 7 8The Bitcoin Core daemon can be configured to act as a trusted "border 9router", implementing the bitcoin wire protocol and relay, making 10consensus decisions, maintaining the local blockchain database, 11broadcasting locally generated transactions into the network, and 12providing a queryable RPC interface to interact on a polled basis for 13requesting blockchain related data. However, there exists only a 14limited service to notify external software of events like the arrival 15of new blocks or transactions. 16 17The ZeroMQ facility implements a notification interface through a set 18of specific notifiers. Currently there are notifiers that publish 19blocks and transactions. This read-only facility requires only the 20connection of a corresponding ZeroMQ subscriber port in receiving 21software; it is not authenticated nor is there any two-way protocol 22involvement. Therefore, subscribers should validate the received data 23since it may be out of date, incomplete or even invalid. 24 25ZeroMQ sockets are self-connecting and self-healing; that is, 26connections made between two endpoints will be automatically restored 27after an outage, and either end may be freely started or stopped in 28any order. 29 30Because ZeroMQ is message oriented, subscribers receive transactions 31and blocks all-at-once and do not need to implement any sort of 32buffering or reassembly. 33 34## Prerequisites 35 36The ZeroMQ feature in Bitcoin Core requires the ZeroMQ API >= 4.0.0 37[libzmq](https://github.com/zeromq/libzmq/releases). 38For version information, see [dependencies.md](dependencies.md). 39Typically, it is packaged by distributions as something like 40*libzmq3-dev*. The C++ wrapper for ZeroMQ is *not* needed. 41 42In order to run the example Python client scripts in the `contrib/zmq/` 43directory, one must also install [PyZMQ](https://github.com/zeromq/pyzmq) 44(generally with `pip install pyzmq`), though this is not necessary for daemon 45operation. 46 47## Enabling 48 49By default, the ZeroMQ feature is automatically compiled in if the 50necessary prerequisites are found. To disable, use --disable-zmq 51during the *configure* step of building bitcoind: 52 53 $ ./configure --disable-zmq (other options) 54 55To actually enable operation, one must set the appropriate options on 56the command line or in the configuration file. 57 58## Usage 59 60Currently, the following notifications are supported: 61 62 -zmqpubhashtx=address 63 -zmqpubhashblock=address 64 -zmqpubrawblock=address 65 -zmqpubrawtx=address 66 -zmqpubsequence=address 67 68The socket type is PUB and the address must be a valid ZeroMQ socket 69address. The same address can be used in more than one notification. 70The same notification can be specified more than once. 71 72The option to set the PUB socket's outbound message high water mark 73(SNDHWM) may be set individually for each notification: 74 75 -zmqpubhashtxhwm=n 76 -zmqpubhashblockhwm=n 77 -zmqpubrawblockhwm=n 78 -zmqpubrawtxhwm=n 79 -zmqpubsequencehwm=address 80 81The high water mark value must be an integer greater than or equal to 0. 82 83For instance: 84 85 $ bitcoind -zmqpubhashtx=tcp://127.0.0.1:28332 \ 86 -zmqpubhashtx=tcp://192.168.1.2:28332 \ 87 -zmqpubrawtx=ipc:///tmp/bitcoind.tx.raw \ 88 -zmqpubhashtxhwm=10000 89 90Each PUB notification has a topic and body, where the header 91corresponds to the notification type. For instance, for the 92notification `-zmqpubhashtx` the topic is `hashtx` (no null 93terminator) and the body is the transaction hash (32 94bytes) for all but `sequence` topic. For `sequence`, the body 95is structured as the following based on the type of message: 96 97 <32-byte hash>C : Blockhash connected 98 <32-byte hash>D : Blockhash disconnected 99 <32-byte hash>R<8-byte LE uint> : Transactionhash removed from mempool for non-block inclusion reason 100 <32-byte hash>A<8-byte LE uint> : Transactionhash added mempool 101 102Where the 8-byte uints correspond to the mempool sequence number. 103 104These options can also be provided in bitcoin.conf. 105 106ZeroMQ endpoint specifiers for TCP (and others) are documented in the 107[ZeroMQ API](http://api.zeromq.org/4-0:_start). 108 109Client side, then, the ZeroMQ subscriber socket must have the 110ZMQ_SUBSCRIBE option set to one or either of these prefixes (for 111instance, just `hash`); without doing so will result in no messages 112arriving. Please see [`contrib/zmq/zmq_sub.py`](/contrib/zmq/zmq_sub.py) for a working example. 113 114The ZMQ_PUB socket's ZMQ_TCP_KEEPALIVE option is enabled. This means that 115the underlying SO_KEEPALIVE option is enabled when using a TCP transport. 116The effective TCP keepalive values are managed through the underlying 117operating system configuration and must be configured prior to connection establishment. 118 119For example, when running on GNU/Linux, one might use the following 120to lower the keepalive setting to 10 minutes: 121 122sudo sysctl -w net.ipv4.tcp_keepalive_time=600 123 124Setting the keepalive values appropriately for your operating environment may 125improve connectivity in situations where long-lived connections are silently 126dropped by network middle boxes. 127 128## Remarks 129 130From the perspective of bitcoind, the ZeroMQ socket is write-only; PUB 131sockets don't even have a read function. Thus, there is no state 132introduced into bitcoind directly. Furthermore, no information is 133broadcast that wasn't already received from the public P2P network. 134 135No authentication or authorization is done on connecting clients; it 136is assumed that the ZeroMQ port is exposed only to trusted entities, 137using other means such as firewalling. 138 139Note that for `*block` topics, when the block chain tip changes, 140a reorganisation may occur and just the tip will be notified. 141It is up to the subscriber to retrieve the chain from the last known 142block to the new tip. Also note that no notification will occur if the tip 143was in the active chain--as would be the case after calling invalidateblock RPC. 144In contrast, the `sequence` topic publishes all block connections and 145disconnections. 146 147There are several possibilities that ZMQ notification can get lost 148during transmission depending on the communication type you are 149using. Bitcoind appends an up-counting sequence number to each 150notification which allows listeners to detect lost notifications. 151 152The `sequence` topic refers specifically to the mempool sequence 153number, which is also published along with all mempool events. This 154is a different sequence value than in ZMQ itself in order to allow a total 155ordering of mempool events to be constructed. 156