README.rst
1*************************************************
2HATop - An Interactive ncurses Client for HAProxy
3*************************************************
4
5HATop is an interactive ncurses client and real-time monitoring,
6statistics displaying tool for the HAProxy TCP/HTTP load balancer.
7
8HATop's appearance is similar to top(1). It supports various modes
9for detailed statistics of all configured proxies and services in near
10realtime. In addition, it features an interactive CLI for the haproxy
11unix socket. This allows administrators to control the given haproxy
12instance (change server weight, put servers into maintenance mode, ...)
13directly out of hatop and monitor the results immediately.
14
15*It is important to understand that when multiple haproxy processes are started
16on the same socket, any process may pick up the request and thus hatop will
17output stats owned solely by that process. The current haproxy-internal
18process id is displayed top right.*
19
20
21Installation
22============
23
24See ``INSTALL`` or refer to :ref:`install`
25
26
27Command line options
28====================
29
30Invoking hatop without options or with ``-h / --help`` results in:
31
32::
33
34 $ hatop --help
35 Usage: hatop (-s SOCKET| -t HOST:PORT) [OPTIONS]...
36
37 Options:
38 --version show program's version number and exit
39 -h, --help show this help message and exit
40
41 Mandatory:
42 -s SOCKET, --unix-socket=SOCKET
43 path to the haproxy unix socket
44 -t TCP_SOCKET, --tcp-socket=TCP_SOCKET
45 address of the haproxy tcp stats socket
46
47 Optional:
48 -i INTERVAL, --update-interval=INTERVAL
49 update interval in seconds (1-30, default: 3)
50 -m MODE, --mode=MODE
51 start in specific mode (1-5, default: 1)
52 -n, --read-only disable the cli and query for stats only
53
54 Filters:
55 Note: All filter options may be given multiple times.
56
57 -f FILTER, --filter=FILTER
58 stat filter in format "<iid> <type> <sid>"
59 -p PROXY, --proxy=PROXY
60 proxy filter in format "<pxname>"
61
62
63Display mode reference
64======================
65
66See also: :ref:`screenshots`
67
68::
69
70 ID Mode Description
71
72 1 STATUS The default mode with health, session and queue statistics
73 2 TRAFFIC Display connection and request rates as well as traffic stats
74 3 HTTP Display various statistical information related to HTTP
75 4 ERRORS Display health info, various error counters and downtimes
76 5 CLI Display embedded command line client for the unix socket
77
78
79Keybind reference
80=================
81
82See also: :ref:`keybinds`
83
84::
85
86 Key Action
87
88 Hh? Display this help screen
89 CTRL-C / Qq Quit
90
91 TAB Cycle mode forwards
92 SHIFT-TAB Cycle mode backwards
93 ALT-n / ESC-n Switch to mode n, where n is the numeric mode id
94 ESC-ESC Jump to previous mode
95
96 ENTER Display hotkey menu for selected service
97 SPACE Copy and paste selected service identifier to the CLI
98
99You can scroll the stat views using ``UP / DOWN / PGUP / PGDOWN / HOME / END``.
100
101The reverse colored cursor line is used to select a given service instance.
102
103An unique identifier ``[#<iid>/<#sid>]`` of the selected
104service is displayed bottom right.
105
106You can hit ``SPACE`` to copy and paste the identifier in string format
107``pxname/svname`` to the CLI for easy re-use with some commands.
108
109For example:
110
1111. Open the CLI
1122. Type "disable server "
1133. Switch back to some stat view using TAB / SHIFT-TAB
1144. Select the server instance using UP / DOWN
1155. Hit SPACE
116
117The result is this command line::
118
119 > disable server <pxname>/<svname>
120
121Hotkeys for common administrative actions
122-----------------------------------------
123::
124
125 Hotkey Action
126
127 F4 Restore initial server weight
128
129 F5 Decrease server weight: - 10
130 F6 Decrease server weight: - 1
131 F7 Increase server weight: + 1
132 F8 Increase server weight: + 10
133
134 F9 Enable server (return from maintenance mode)
135 F10 Disable server (put into maintenance mode)
136
137Hotkey actions and server responses are logged on the CLI viewport.
138
139You can scroll the output on the CLI view using ``PGUP / PGDOWN``.
140
141A brief keybind reference is logged there directly after startup...
142
143
144Header reference
145================
146
147See also: :ref:`screenshots`
148
149::
150
151 Node configured name of the haproxy node
152 Uptime runtime since haproxy was initially started
153 Pipes pipes are currently used for kernel-based tcp slicing
154 Procs number of haproxy processes
155 Tasks number of actice process tasks
156 Queue number of queued process tasks (run queue)
157 Proxies number of configured proxies
158 Services number of configured services
159
160In multiple modes
161-----------------
162::
163
164 NAME name of the proxy and its services
165 W configured weight of the service
166 STATUS service status (UP/DOWN/NOLB/MAINT/MAINT(via)...)
167 CHECK status of last health check (see status reference below)
168
169In STATUS mode
170--------------
171::
172
173 ACT server is active (server), number of active servers (backend)
174 BCK server is backup (server), number of backup servers (backend)
175 QCUR current queued requests
176 QMAX max queued requests
177 SCUR current sessions
178 SMAX max sessions
179 SLIM sessions limit
180 STOT total sessions
181
182In TRAFFIC mode
183---------------
184::
185
186 LBTOT total number of times a server was selected
187 RATE number of sessions per second over last elapsed second
188 RLIM limit on new sessions per second
189 RMAX max number of new sessions per second
190 BIN bytes in (IEEE 1541-2002)
191 BOUT bytes out (IEEE 1541-2002)
192
193In HTTP mode
194------------
195::
196
197 RATE HTTP requests per second over last elapsed second
198 RMAX max number of HTTP requests per second observed
199 RTOT total number of HTTP requests received
200 1xx number of HTTP responses with 1xx code
201 2xx number of HTTP responses with 2xx code
202 3xx number of HTTP responses with 3xx code
203 4xx number of HTTP responses with 4xx code
204 5xx number of HTTP responses with 5xx code
205 ?xx number of HTTP responses with other codes (protocol error)
206
207In ERRORS mode
208--------------
209::
210
211 CF number of failed checks
212 CD number of UP->DOWN transitions
213 CL last status change
214 ECONN connection errors
215 EREQ request errors
216 ERSP response errors
217 DREQ denied requests
218 DRSP denied responses
219 DOWN total downtime
220
221
222Health check status reference
223=============================
224::
225
226 UNK unknown
227 INI initializing
228 SOCKERR socket error
229 L4OK check passed on layer 4, no upper layers testing enabled
230 L4TMOUT layer 1-4 timeout
231 L4CON layer 1-4 connection problem, for example
232 "Connection refused" (tcp rst) or "No route to host" (icmp)
233 L6OK check passed on layer 6
234 L6TOUT layer 6 (SSL) timeout
235 L6RSP layer 6 invalid response - protocol error
236 L7OK check passed on layer 7
237 L7OKC check conditionally passed on layer 7, for example 404 with
238 disable-on-404
239 L7TOUT layer 7 (HTTP/SMTP) timeout
240 L7RSP layer 7 invalid response - protocol error
241 L7STS layer 7 response error, for example HTTP 5xx
242
243
244Authors
245=======
246
247HATop was originally written by John Feuerstein <john@feurix.com>,
248known on GitHub as @feurix.
249
250As of 2020, however, support for Python 2.x is being phased out
251of modern distributions of Linux, the last code interaction on the
252upstream repository (<https://github.com/feurix/hatop>) is from
253over four years ago, and no issue or pull request has been able to
254elicit a response from the author.
255
256As of February 29th (2020), I have forked this repository into my own
257GitHub org, at <https://github.com/jhunt/hatop>, with the intent
258of merging some of the upstream pull requests, and continuing
259ongoing maintenance of this tool.
260
261I have been an avid user of HATop since I first discovered it many
262many years ago, and would like to continue to be able to use it
263for many many more.
264
265Contributors
266============
267
268The following people have contributed to HATop:
269
270 - Andrew Hayworth
271 - Cyril Bonté
272 - James Hunt
273 - John Feuerstein
274 - Louis Charreau
275 - Matt Behrens
276