1# Consul Benchmark Results
2
3As part of a benchmark, we started a 4 node DigitalOcean cluster to benchmark.
4There are 3 servers, meaning writes must commit to at least 2 servers.
5The cluster uses the 16GB DigitalOcean droplet which has the following specs:
6
7 * 8 CPU Cores, 2Ghz
8 * 16GB RAM
9 * 160GB SSD disk
10 * 1Gbps NIC
11
12# Output
13
14Below is the output for a test run on a benchmark cluster. We ran the benchmark
15several times to warm up the nodes, and this is just a single representative sample.
16
17Note, that a single worker was running the benchmark. This means the "stale" test
18is not representative of total throughput, as the client was only routing to a
19single server.
20
21We also did an initial run where we got lots of noise in the results, so we
22increased the number of requests to try to get a better sample.
23
24```
25===== PUT test =====
26GOMAXPROCS=4 boom -m PUT -d "74a31e96-1d0f-4fa7-aa14-7212a326986e" -n 262144 -c 64 http://127.0.0.1:8500/v1/kv/bench
27262144 / 262144 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00 %
28
29Summary:
30  Total:	69.3512 secs.
31  Slowest:	0.0966 secs.
32  Fastest:	0.0026 secs.
33  Average:	0.0169 secs.
34  Requests/sec:	3779.9491
35  Total Data Received:	1048576 bytes.
36  Response Size per Request:	4 bytes.
37
38Status code distribution:
39  [200]	262144 responses
40
41Response time histogram:
42  0.003 [1]	|
43  0.012 [66586]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
44  0.021 [146064]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
45  0.031 [34189]	|∎∎∎∎∎∎∎∎∎
46  0.040 [9178]	|∎∎
47  0.050 [3682]	|∎
48  0.059 [1773]	|
49  0.068 [464]	|
50  0.078 [124]	|
51  0.087 [63]	|
52  0.097 [20]	|
53
54Latency distribution:
55  10% in 0.0095 secs.
56  25% in 0.0119 secs.
57  50% in 0.0151 secs.
58  75% in 0.0195 secs.
59  90% in 0.0260 secs.
60  95% in 0.0323 secs.
61  99% in 0.0489 secs.
62
63===== GET default test =====
64GOMAXPROCS=4 boom -n 262144 -c 64 http://127.0.0.1:8500/v1/kv/bench
65262144 / 262144 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00 %
66
67Summary:
68  Total:	34.8371 secs.
69  Slowest:	0.9568 secs.
70  Fastest:	0.0014 secs.
71  Average:	0.0085 secs.
72  Requests/sec:	7524.8570
73  Total Data Received:	36175872 bytes.
74  Response Size per Request:	138 bytes.
75
76Status code distribution:
77  [200]	262144 responses
78
79Response time histogram:
80  0.001 [1]	|
81  0.097 [261977]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
82  0.192 [38]	|
83  0.288 [64]	|
84  0.384 [0]	|
85  0.479 [0]	|
86  0.575 [0]	|
87  0.670 [0]	|
88  0.766 [0]	|
89  0.861 [38]	|
90  0.957 [26]	|
91
92Latency distribution:
93  10% in 0.0044 secs.
94  25% in 0.0055 secs.
95  50% in 0.0072 secs.
96  75% in 0.0098 secs.
97  90% in 0.0130 secs.
98  95% in 0.0157 secs.
99  99% in 0.0228 secs.
100
101===== GET stale test =====
102GOMAXPROCS=4 boom -n 262144 -c 64 http://127.0.0.1:8500/v1/kv/bench?stale
103262144 / 262144 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00 %
104
105Summary:
106  Total:	26.8200 secs.
107  Slowest:	0.0838 secs.
108  Fastest:	0.0005 secs.
109  Average:	0.0065 secs.
110  Requests/sec:	9774.1922
111  Total Data Received:	36175872 bytes.
112  Response Size per Request:	138 bytes.
113
114Status code distribution:
115  [200]	262144 responses
116
117Response time histogram:
118  0.001 [1]	|
119  0.009 [214210]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
120  0.017 [42999]	|∎∎∎∎∎∎∎∎
121  0.026 [3709]	|
122  0.034 [589]	|
123  0.042 [313]	|
124  0.050 [166]	|
125  0.059 [102]	|
126  0.067 [42]	|
127  0.075 [11]	|
128  0.084 [2]	|
129
130Latency distribution:
131  10% in 0.0031 secs.
132  25% in 0.0041 secs.
133  50% in 0.0056 secs.
134  75% in 0.0079 secs.
135  90% in 0.0109 secs.
136  95% in 0.0134 secs.
137  99% in 0.0203 secs.
138
139===== GET consistent test =====
140GOMAXPROCS=4 boom -n 262144 -c 64 http://127.0.0.1:8500/v1/kv/bench?consistent
141262144 / 262144 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00 %
142
143Summary:
144  Total:	35.6962 secs.
145  Slowest:	0.0826 secs.
146  Fastest:	0.0016 secs.
147  Average:	0.0087 secs.
148  Requests/sec:	7343.7475
149  Total Data Received:	36175872 bytes.
150  Response Size per Request:	138 bytes.
151
152Status code distribution:
153  [200]	262144 responses
154
155Response time histogram:
156  0.002 [1]	|
157  0.010 [183123]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
158  0.018 [70460]	|∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
159  0.026 [6955]	|∎
160  0.034 [657]	|
161  0.042 [391]	|
162  0.050 [229]	|
163  0.058 [120]	|
164  0.066 [121]	|
165  0.074 [68]	|
166  0.083 [19]	|
167
168Latency distribution:
169  10% in 0.0047 secs.
170  25% in 0.0059 secs.
171  50% in 0.0077 secs.
172  75% in 0.0104 secs.
173  90% in 0.0137 secs.
174  95% in 0.0162 secs.
175  99% in 0.0227 secs.
176```
177
178# Profile
179
180In order to probe performance a bit, we ran the get-stale benchmark on the
181leader itself and collected pprof data. Here's the output of the benchmark:
182
183```
184===== GET stale test =====
185GOMAXPROCS=4 boom -n 262144 -c 64 http://127.0.0.1:8500/v1/kv/bench?stale
186262144 / 262144 Booooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00 %
187
188Summary:
189  Total:        16.3139 secs.
190  Slowest:      0.0815 secs.
191  Fastest:      0.0001 secs.
192  Average:      0.0040 secs.
193  Requests/sec: 16068.7946
194  Total Data Received:  36175872 bytes.
195  Response Size per Request:    138 bytes.
196
197Status code distribution:
198  [200] 262144 responses
199
200Response time histogram:
201  0.000 [1]     |
202  0.008 [240221]        |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
203  0.016 [18761] |∎∎∎
204  0.025 [1937]  |
205  0.033 [496]   |
206  0.041 [293]   |
207  0.049 [131]   |
208  0.057 [162]   |
209  0.065 [127]   |
210  0.073 [10]    |
211  0.081 [5]     |
212
213Latency distribution:
214  10% in 0.0013 secs.
215  25% in 0.0019 secs.
216  50% in 0.0030 secs.
217  75% in 0.0046 secs.
218  90% in 0.0074 secs.
219  95% in 0.0109 secs.
220  99% in 0.0174 secs.
221```
222
223And here's the [resulting flame graph](results-0.7.1.svg).
224