1# Consul Benchmark Results 2 3As part of a benchmark, we started a 4 node DigitalOcean cluster to benchmark. 4There are 3 servers, meaning writes must commit to at least 2 servers. 5The cluster uses the 16GB DigitalOcean droplet which has the following specs: 6 7 * 8 CPU Cores, 2Ghz 8 * 16GB RAM 9 * 160GB SSD disk 10 * 1Gbps NIC 11 12# Output 13 14Below is the output for a test run on a benchmark cluster. We ran the benchmark 15several times to warm up the nodes, and this is just a single representative sample. 16 17Note, that a single worker was running the benchmark. This means the "stale" test 18is not representative of total throughput, as the client was only routing to a 19single server. 20 21We also did an initial run where we got lots of noise in the results, so we 22increased the number of requests to try to get a better sample. 23 24``` 25===== PUT test ===== 26GOMAXPROCS=4 boom -m PUT -d "74a31e96-1d0f-4fa7-aa14-7212a326986e" -n 262144 -c 64 http://127.0.0.1:8500/v1/kv/bench 27262144 / 262144 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00 % 28 29Summary: 30 Total: 69.3512 secs. 31 Slowest: 0.0966 secs. 32 Fastest: 0.0026 secs. 33 Average: 0.0169 secs. 34 Requests/sec: 3779.9491 35 Total Data Received: 1048576 bytes. 36 Response Size per Request: 4 bytes. 37 38Status code distribution: 39 [200] 262144 responses 40 41Response time histogram: 42 0.003 [1] | 43 0.012 [66586] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎ 44 0.021 [146064] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎ 45 0.031 [34189] |∎∎∎∎∎∎∎∎∎ 46 0.040 [9178] |∎∎ 47 0.050 [3682] |∎ 48 0.059 [1773] | 49 0.068 [464] | 50 0.078 [124] | 51 0.087 [63] | 52 0.097 [20] | 53 54Latency distribution: 55 10% in 0.0095 secs. 56 25% in 0.0119 secs. 57 50% in 0.0151 secs. 58 75% in 0.0195 secs. 59 90% in 0.0260 secs. 60 95% in 0.0323 secs. 61 99% in 0.0489 secs. 62 63===== GET default test ===== 64GOMAXPROCS=4 boom -n 262144 -c 64 http://127.0.0.1:8500/v1/kv/bench 65262144 / 262144 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00 % 66 67Summary: 68 Total: 34.8371 secs. 69 Slowest: 0.9568 secs. 70 Fastest: 0.0014 secs. 71 Average: 0.0085 secs. 72 Requests/sec: 7524.8570 73 Total Data Received: 36175872 bytes. 74 Response Size per Request: 138 bytes. 75 76Status code distribution: 77 [200] 262144 responses 78 79Response time histogram: 80 0.001 [1] | 81 0.097 [261977] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎ 82 0.192 [38] | 83 0.288 [64] | 84 0.384 [0] | 85 0.479 [0] | 86 0.575 [0] | 87 0.670 [0] | 88 0.766 [0] | 89 0.861 [38] | 90 0.957 [26] | 91 92Latency distribution: 93 10% in 0.0044 secs. 94 25% in 0.0055 secs. 95 50% in 0.0072 secs. 96 75% in 0.0098 secs. 97 90% in 0.0130 secs. 98 95% in 0.0157 secs. 99 99% in 0.0228 secs. 100 101===== GET stale test ===== 102GOMAXPROCS=4 boom -n 262144 -c 64 http://127.0.0.1:8500/v1/kv/bench?stale 103262144 / 262144 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00 % 104 105Summary: 106 Total: 26.8200 secs. 107 Slowest: 0.0838 secs. 108 Fastest: 0.0005 secs. 109 Average: 0.0065 secs. 110 Requests/sec: 9774.1922 111 Total Data Received: 36175872 bytes. 112 Response Size per Request: 138 bytes. 113 114Status code distribution: 115 [200] 262144 responses 116 117Response time histogram: 118 0.001 [1] | 119 0.009 [214210] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎ 120 0.017 [42999] |∎∎∎∎∎∎∎∎ 121 0.026 [3709] | 122 0.034 [589] | 123 0.042 [313] | 124 0.050 [166] | 125 0.059 [102] | 126 0.067 [42] | 127 0.075 [11] | 128 0.084 [2] | 129 130Latency distribution: 131 10% in 0.0031 secs. 132 25% in 0.0041 secs. 133 50% in 0.0056 secs. 134 75% in 0.0079 secs. 135 90% in 0.0109 secs. 136 95% in 0.0134 secs. 137 99% in 0.0203 secs. 138 139===== GET consistent test ===== 140GOMAXPROCS=4 boom -n 262144 -c 64 http://127.0.0.1:8500/v1/kv/bench?consistent 141262144 / 262144 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00 % 142 143Summary: 144 Total: 35.6962 secs. 145 Slowest: 0.0826 secs. 146 Fastest: 0.0016 secs. 147 Average: 0.0087 secs. 148 Requests/sec: 7343.7475 149 Total Data Received: 36175872 bytes. 150 Response Size per Request: 138 bytes. 151 152Status code distribution: 153 [200] 262144 responses 154 155Response time histogram: 156 0.002 [1] | 157 0.010 [183123] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎ 158 0.018 [70460] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎ 159 0.026 [6955] |∎ 160 0.034 [657] | 161 0.042 [391] | 162 0.050 [229] | 163 0.058 [120] | 164 0.066 [121] | 165 0.074 [68] | 166 0.083 [19] | 167 168Latency distribution: 169 10% in 0.0047 secs. 170 25% in 0.0059 secs. 171 50% in 0.0077 secs. 172 75% in 0.0104 secs. 173 90% in 0.0137 secs. 174 95% in 0.0162 secs. 175 99% in 0.0227 secs. 176``` 177 178# Profile 179 180In order to probe performance a bit, we ran the get-stale benchmark on the 181leader itself and collected pprof data. Here's the output of the benchmark: 182 183``` 184===== GET stale test ===== 185GOMAXPROCS=4 boom -n 262144 -c 64 http://127.0.0.1:8500/v1/kv/bench?stale 186262144 / 262144 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00 % 187 188Summary: 189 Total: 16.3139 secs. 190 Slowest: 0.0815 secs. 191 Fastest: 0.0001 secs. 192 Average: 0.0040 secs. 193 Requests/sec: 16068.7946 194 Total Data Received: 36175872 bytes. 195 Response Size per Request: 138 bytes. 196 197Status code distribution: 198 [200] 262144 responses 199 200Response time histogram: 201 0.000 [1] | 202 0.008 [240221] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎ 203 0.016 [18761] |∎∎∎ 204 0.025 [1937] | 205 0.033 [496] | 206 0.041 [293] | 207 0.049 [131] | 208 0.057 [162] | 209 0.065 [127] | 210 0.073 [10] | 211 0.081 [5] | 212 213Latency distribution: 214 10% in 0.0013 secs. 215 25% in 0.0019 secs. 216 50% in 0.0030 secs. 217 75% in 0.0046 secs. 218 90% in 0.0074 secs. 219 95% in 0.0109 secs. 220 99% in 0.0174 secs. 221``` 222 223And here's the [resulting flame graph](results-0.7.1.svg). 224