1************************************************* 2HATop - An Interactive ncurses Client for HAProxy 3************************************************* 4 5HATop is an interactive ncurses client and real-time monitoring, 6statistics displaying tool for the HAProxy TCP/HTTP load balancer. 7 8HATop's appearance is similar to top(1). It supports various modes 9for detailed statistics of all configured proxies and services in near 10realtime. In addition, it features an interactive CLI for the haproxy 11unix socket. This allows administrators to control the given haproxy 12instance (change server weight, put servers into maintenance mode, ...) 13directly out of hatop and monitor the results immediately. 14 15*It is important to understand that when multiple haproxy processes are started 16on the same socket, any process may pick up the request and thus hatop will 17output stats owned solely by that process. The current haproxy-internal 18process id is displayed top right.* 19 20 21Installation 22============ 23 24See ``INSTALL`` or refer to :ref:`install` 25 26 27Command line options 28==================== 29 30Invoking hatop without options or with ``-h / --help`` results in: 31 32:: 33 34 $ hatop --help 35 Usage: hatop (-s SOCKET| -t HOST:PORT) [OPTIONS]... 36 37 Options: 38 --version show program's version number and exit 39 -h, --help show this help message and exit 40 41 Mandatory: 42 -s SOCKET, --unix-socket=SOCKET 43 path to the haproxy unix socket 44 -t TCP_SOCKET, --tcp-socket=TCP_SOCKET 45 address of the haproxy tcp stats socket 46 47 Optional: 48 -i INTERVAL, --update-interval=INTERVAL 49 update interval in seconds (1-30, default: 3) 50 -m MODE, --mode=MODE 51 start in specific mode (1-5, default: 1) 52 -n, --read-only disable the cli and query for stats only 53 54 Filters: 55 Note: All filter options may be given multiple times. 56 57 -f FILTER, --filter=FILTER 58 stat filter in format "<iid> <type> <sid>" 59 -p PROXY, --proxy=PROXY 60 proxy filter in format "<pxname>" 61 62 63Display mode reference 64====================== 65 66See also: :ref:`screenshots` 67 68:: 69 70 ID Mode Description 71 72 1 STATUS The default mode with health, session and queue statistics 73 2 TRAFFIC Display connection and request rates as well as traffic stats 74 3 HTTP Display various statistical information related to HTTP 75 4 ERRORS Display health info, various error counters and downtimes 76 5 CLI Display embedded command line client for the unix socket 77 78 79Keybind reference 80================= 81 82See also: :ref:`keybinds` 83 84:: 85 86 Key Action 87 88 Hh? Display this help screen 89 CTRL-C / Qq Quit 90 91 TAB Cycle mode forwards 92 SHIFT-TAB Cycle mode backwards 93 ALT-n / ESC-n Switch to mode n, where n is the numeric mode id 94 ESC-ESC Jump to previous mode 95 96 ENTER Display hotkey menu for selected service 97 SPACE Copy and paste selected service identifier to the CLI 98 99You can scroll the stat views using ``UP / DOWN / PGUP / PGDOWN / HOME / END``. 100 101The reverse colored cursor line is used to select a given service instance. 102 103An unique identifier ``[#<iid>/<#sid>]`` of the selected 104service is displayed bottom right. 105 106You can hit ``SPACE`` to copy and paste the identifier in string format 107``pxname/svname`` to the CLI for easy re-use with some commands. 108 109For example: 110 1111. Open the CLI 1122. Type "disable server " 1133. Switch back to some stat view using TAB / SHIFT-TAB 1144. Select the server instance using UP / DOWN 1155. Hit SPACE 116 117The result is this command line:: 118 119 > disable server <pxname>/<svname> 120 121Hotkeys for common administrative actions 122----------------------------------------- 123:: 124 125 Hotkey Action 126 127 F4 Restore initial server weight 128 129 F5 Decrease server weight: - 10 130 F6 Decrease server weight: - 1 131 F7 Increase server weight: + 1 132 F8 Increase server weight: + 10 133 134 F9 Enable server (return from maintenance mode) 135 F10 Disable server (put into maintenance mode) 136 137Hotkey actions and server responses are logged on the CLI viewport. 138 139You can scroll the output on the CLI view using ``PGUP / PGDOWN``. 140 141A brief keybind reference is logged there directly after startup... 142 143 144Header reference 145================ 146 147See also: :ref:`screenshots` 148 149:: 150 151 Node configured name of the haproxy node 152 Uptime runtime since haproxy was initially started 153 Pipes pipes are currently used for kernel-based tcp slicing 154 Procs number of haproxy processes 155 Tasks number of actice process tasks 156 Queue number of queued process tasks (run queue) 157 Proxies number of configured proxies 158 Services number of configured services 159 160In multiple modes 161----------------- 162:: 163 164 NAME name of the proxy and its services 165 W configured weight of the service 166 STATUS service status (UP/DOWN/NOLB/MAINT/MAINT(via)...) 167 CHECK status of last health check (see status reference below) 168 169In STATUS mode 170-------------- 171:: 172 173 ACT server is active (server), number of active servers (backend) 174 BCK server is backup (server), number of backup servers (backend) 175 QCUR current queued requests 176 QMAX max queued requests 177 SCUR current sessions 178 SMAX max sessions 179 SLIM sessions limit 180 STOT total sessions 181 182In TRAFFIC mode 183--------------- 184:: 185 186 LBTOT total number of times a server was selected 187 RATE number of sessions per second over last elapsed second 188 RLIM limit on new sessions per second 189 RMAX max number of new sessions per second 190 BIN bytes in (IEEE 1541-2002) 191 BOUT bytes out (IEEE 1541-2002) 192 193In HTTP mode 194------------ 195:: 196 197 RATE HTTP requests per second over last elapsed second 198 RMAX max number of HTTP requests per second observed 199 RTOT total number of HTTP requests received 200 1xx number of HTTP responses with 1xx code 201 2xx number of HTTP responses with 2xx code 202 3xx number of HTTP responses with 3xx code 203 4xx number of HTTP responses with 4xx code 204 5xx number of HTTP responses with 5xx code 205 ?xx number of HTTP responses with other codes (protocol error) 206 207In ERRORS mode 208-------------- 209:: 210 211 CF number of failed checks 212 CD number of UP->DOWN transitions 213 CL last status change 214 ECONN connection errors 215 EREQ request errors 216 ERSP response errors 217 DREQ denied requests 218 DRSP denied responses 219 DOWN total downtime 220 221 222Health check status reference 223============================= 224:: 225 226 UNK unknown 227 INI initializing 228 SOCKERR socket error 229 L4OK check passed on layer 4, no upper layers testing enabled 230 L4TMOUT layer 1-4 timeout 231 L4CON layer 1-4 connection problem, for example 232 "Connection refused" (tcp rst) or "No route to host" (icmp) 233 L6OK check passed on layer 6 234 L6TOUT layer 6 (SSL) timeout 235 L6RSP layer 6 invalid response - protocol error 236 L7OK check passed on layer 7 237 L7OKC check conditionally passed on layer 7, for example 404 with 238 disable-on-404 239 L7TOUT layer 7 (HTTP/SMTP) timeout 240 L7RSP layer 7 invalid response - protocol error 241 L7STS layer 7 response error, for example HTTP 5xx 242 243 244Authors 245======= 246 247HATop was originally written by John Feuerstein <john@feurix.com>, 248known on GitHub as @feurix. 249 250As of 2020, however, support for Python 2.x is being phased out 251of modern distributions of Linux, the last code interaction on the 252upstream repository (<https://github.com/feurix/hatop>) is from 253over four years ago, and no issue or pull request has been able to 254elicit a response from the author. 255 256As of February 29th (2020), I have forked this repository into my own 257GitHub org, at <https://github.com/jhunt/hatop>, with the intent 258of merging some of the upstream pull requests, and continuing 259ongoing maintenance of this tool. 260 261I have been an avid user of HATop since I first discovered it many 262many years ago, and would like to continue to be able to use it 263for many many more. 264 265Contributors 266============ 267 268The following people have contributed to HATop: 269 270 - Andrew Hayworth 271 - Cyril Bonté 272 - James Hunt 273 - John Feuerstein 274 - Louis Charreau 275 - Matt Behrens 276