1NAME
2
3 Hijk - Fast & minimal low-level HTTP client
4
5SYNOPSIS
6
7 A simple GET request:
8
9 use Hijk ();
10 my $res = Hijk::request({
11 method => "GET",
12 host => "example.com",
13 port => "80",
14 path => "/flower",
15 query_string => "color=red"
16 });
17
18 if (exists $res->{error} and $res->{error} & Hijk::Error::TIMEOUT) {
19 die "Oh noes we had some sort of timeout";
20 }
21
22 die "Expecting an 'OK' response" unless $res->{status} == 200;
23
24 say $res->{body};
25
26 A POST request, you have to manually set the appropriate headers, URI
27 escape your values etc.
28
29 use Hijk ();
30 use URI::Escape qw(uri_escape);
31
32 my $res = Hijk::request({
33 method => "POST",
34 host => "example.com",
35 port => "80",
36 path => "/new",
37 head => [ "Content-Type" => "application/x-www-form-urlencoded" ],
38 query_string => "type=flower&bucket=the%20one%20out%20back",
39 body => "description=" . uri_escape("Another flower, let's hope it's exciting"),
40 });
41
42 die "Expecting an 'OK' response" unless $res->{status} == 200;
43
44DESCRIPTION
45
46 Hijk is a fast & minimal low-level HTTP client intended to be used
47 where you control both the client and the server, e.g. for talking to
48 some internal service from a frontend user-facing web application.
49
50 It is NOT a general HTTP user agent, it doesn't support redirects,
51 proxies, SSL and any number of other advanced HTTP features like (in
52 roughly descending order of feature completeness) LWP::UserAgent,
53 WWW::Curl, HTTP::Tiny, HTTP::Lite or Furl. This library is basically
54 one step above manually talking HTTP over sockets.
55
56 Having said that it's lightning fast and extensively used in production
57 at Booking.com <https://www.booking.com> where it's used as the go-to
58 transport layer for talking to internal services. It uses non-blocking
59 sockets and correctly handles all combinations of connect/read timeouts
60 and other issues you might encounter from various combinations of parts
61 of your system going down or becoming otherwise unavailable.
62
63FUNCTION: Hijk::request( $args :HashRef ) :HashRef
64
65 Hijk::request is the only function you should use. It (or anything else
66 in this package for that matter) is not exported, so you have to use
67 the fully qualified name.
68
69 It takes a HashRef of arguments and either dies or returns a HashRef as
70 a response.
71
72 The HashRef argument to it must contain some of the key-value pairs
73 from the following list. The value for host and port are mandatory, but
74 others are optional with default values listed below.
75
76 protocol => "HTTP/1.1", # (or "HTTP/1.0")
77 host => ...,
78 port => ...,
79 connect_timeout => undef,
80 read_timeout => undef,
81 read_length => 10240,
82 method => "GET",
83 path => "/",
84 query_string => "",
85 head => [],
86 body => "",
87 socket_cache => \%Hijk::SOCKET_CACHE, # (undef to disable, or \my %your_socket_cache)
88 on_connect => undef, # (or sub { ... })
89 parse_chunked => 0,
90 head_as_array => 0,
91 no_default_host_header => 1,
92
93 Notice how Hijk does not take a full URI string as input, you have to
94 specify the individual parts of the URL. Users who need to parse an
95 existing URI string to produce a request should use the URI module to
96 do so.
97
98 The value of head is an ArrayRef of key-value pairs instead of a
99 HashRef, this way you can decide in which order the headers are sent,
100 and you can send the same header name multiple times. For example:
101
102 head => [
103 "Content-Type" => "application/json",
104 "X-Requested-With" => "Hijk",
105 ]
106
107 Will produce these request headers:
108
109 Content-Type: application/json
110 X-Requested-With: Hijk
111
112 In addition Hijk will provide a Host header for you by default with the
113 host value you pass to request(). To suppress this (e.g. to send custom
114 Host requests) pass a true value to the no_default_host_header option
115 and provide your own Host header in the head ArrayRef (or don't, if you
116 want to construct a Host-less request knock yourself out...).
117
118 Hijk doesn't escape any values for you, it just passes them through
119 as-is. You can easily produce invalid requests if e.g. any of these
120 strings contain a newline, or aren't otherwise properly escaped.
121
122 The value of connect_timeout or read_timeout is in floating point
123 seconds, and is used as the time limit for connecting to the host, and
124 reading the response back from it, respectively. The default value for
125 both is undef, meaning no timeout limit. If you don't supply these
126 timeouts and the host really is unreachable or slow, we'll reach the
127 TCP timeout limit before returning some other error to you.
128
129 The default protocol is HTTP/1.1, but you can also specify HTTP/1.0.
130 The advantage of using HTTP/1.1 is support for keep-alive, which
131 matters a lot in environments where the connection setup represents
132 non-trivial overhead. Sometimes that overhead is negligible (e.g. on
133 Linux talking to an nginx on the local network), and keeping open
134 connections down and reducing complexity is more important, in those
135 cases you can either use HTTP/1.0, or specify Connection: close in the
136 request, but just using HTTP/1.0 is an easy way to accomplish the same
137 thing.
138
139 By default we will provide a socket_cache for you which is a global
140 singleton that we maintain keyed on join($;, $$, $host, $port).
141 Alternatively you can pass in socket_cache hash of your own which we'll
142 use as the cache. To completely disable the cache pass in undef.
143
144 The optional on_connect callback is intended to be used for you to
145 figure out from production traffic what you should set the
146 connect_timeout. I.e. you can start a timer when you call
147 Hijk::request() that you end when on_connect is called, that's how long
148 it took us to get a connection. If you start another timer in that
149 callback that you end when Hijk::request() returns to you that'll give
150 you how long it took to send/receive data after we constructed the
151 socket, i.e. it'll help you to tweak your read_timeout. The on_connect
152 callback is provided with no arguments, and is called in void context.
153
154 We have experimental support for parsing chunked responses encoding.
155 historically Hijk didn't support this at all and if you wanted to use
156 it with e.g. nginx you had to add chunked_transfer_encoding off to the
157 nginx config file.
158
159 Since you may just want to do that instead of having Hijk do more work
160 to parse this out with a more complex and experimental codepath you
161 have to explicitly enable it with parse_chunked. Otherwise Hijk will
162 die when it encounters chunked responses. The parse_chunked option may
163 be turned on by default in the future.
164
165 The return value is a HashRef representing a response. It contains the
166 following key-value pairs.
167
168 proto => :Str
169 status => :StatusCode
170 body => :Str
171 head => :HashRef (or :ArrayRef with "head_as_array")
172 error => :PositiveInt
173 error_message => :Str
174 errno_number => :Int
175 errno_string => :Str
176
177 For example, to send a request to http://example.com/flower?color=red,
178 pass the following parameters:
179
180 my $res = Hijk::request({
181 host => "example.com",
182 port => "80",
183 path => "/flower",
184 query_string => "color=red"
185 });
186 die "Response is not 'OK'" unless $res->{status} == 200;
187
188 Notice that you do not need to put the leading "?" character in the
189 query_string. You do, however, need to properly uri_escape the content
190 of query_string.
191
192 Again, Hijk doesn't escape any values for you, so these values MUST be
193 properly escaped before being passed in, unless you want to issue
194 invalid requests.
195
196 By default the head in the response is a HashRef rather then an
197 ArrayRef. This makes it easier to retrieve specific header fields, but
198 it means that we'll clobber any duplicated header names with the most
199 recently seen header value. To get the returned headers as an ArrayRef
200 instead specify head_as_array.
201
202 If you want to fiddle with the read_length value it controls how much
203 we POSIX::read($fd, $buf, $read_length) at a time.
204
205 We currently don't support servers returning a http body without an
206 accompanying Content-Length header; bodies MUST have a Content-Length
207 or we won't pick them up.
208
209ERROR CODES
210
211 If we had a recoverable error we'll include an "error" key whose value
212 is a bitfield that you can check against Hijk::Error::* constants.
213 Those are:
214
215 Hijk::Error::CONNECT_TIMEOUT
216 Hijk::Error::READ_TIMEOUT
217 Hijk::Error::TIMEOUT
218 Hijk::Error::CANNOT_RESOLVE
219 Hijk::Error::REQUEST_SELECT_ERROR
220 Hijk::Error::REQUEST_WRITE_ERROR
221 Hijk::Error::REQUEST_ERROR
222 Hijk::Error::RESPONSE_READ_ERROR
223 Hijk::Error::RESPONSE_BAD_READ_VALUE
224 Hijk::Error::RESPONSE_ERROR
225
226 In addition we might return error_message, errno_number and
227 errno_string keys, see the discussion of Hijk::Error::REQUEST_* and
228 Hijk::Error::RESPONSE_* errors below.
229
230 The Hijk::Error::TIMEOUT constant is the same as
231 Hijk::Error::CONNECT_TIMEOUT | Hijk::Error::READ_TIMEOUT. It's there
232 for convenience so you can do:
233
234 .. if exists $res->{error} and $res->{error} & Hijk::Error::TIMEOUT;
235
236 Instead of the more verbose:
237
238 .. if exists $res->{error} and $res->{error} & (Hijk::Error::CONNECT_TIMEOUT | Hijk::Error::READ_TIMEOUT)
239
240 We'll return Hijk::Error::CANNOT_RESOLVE if we can't gethostbyname()
241 the host you've provided.
242
243 If we fail to do a select() or write() during when sending the response
244 we'll return Hijk::Error::REQUEST_SELECT_ERROR or
245 Hijk::Error::REQUEST_WRITE_ERROR, respectively. Similarly to
246 Hijk::Error::TIMEOUT the Hijk::Error::REQUEST_ERROR constant is a union
247 of these two, and any other request errors we might add in the future.
248
249 When we're getting the response back we'll return
250 Hijk::Error::RESPONSE_READ_ERROR when we can't read() the response, and
251 Hijk::Error::RESPONSE_BAD_READ_VALUE when the value we got from read()
252 is 0. The Hijk::Error::RESPONSE_ERROR constant is a union of these two
253 and any other response errors we might add in the future.
254
255 Some of these Hijk::Error::REQUEST_* and Hijk::Error::RESPONSE_* errors
256 are re-thrown errors from system calls. In that case we'll also pass
257 along error_message which is a short human readable error message about
258 the error, as well as errno_number & errno_string, which are $!+0 and
259 "$!" at the time we had the error.
260
261 Hijk might encounter other errors during the course of the request and
262 WILL call die if that happens, so if you don't want your program to
263 stop when a request like that fails wrap it in eval.
264
265 Having said that the point of the Hijk::Error::* interface is that all
266 errors that happen during normal operation, i.e. making valid requests
267 against servers where you can have issues like timeouts, network blips
268 or the server thread on the other end being suddenly kill -9'd should
269 be caught, categorized and returned in a structural way by Hijk.
270
271 We're not currently aware of any issues that occur in such normal
272 operations that aren't classified as a Hijk::Error::*, and if we find
273 new issues that fit the criteria above we'll likely just make a new
274 Hijk::Error::* for it.
275
276 We're just not trying to guarantee that the library can never die, and
277 aren't trying to catch truly exceptional issues like e.g. fcntl()
278 failing on a valid socket.
279
280AUTHORS
281
282 Kang-min Liu <gugod@gugod.org>
283
284 Ævar Arnfjörð Bjarmason <avar@cpan.org>
285
286 Borislav Nikolov <jack@sofialondonmoskva.com>
287
288 Damian Gryski <damian@gryski.com>
289
290COPYRIGHT
291
292 Copyright (c) 2013- Kang-min Liu <gugod@gugod.org>.
293
294LICENCE
295
296 The MIT License
297
298DISCLAIMER OF WARRANTY
299
300 BECAUSE THIS SOFTWARE IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
301 FOR THE SOFTWARE, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT
302 WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER
303 PARTIES PROVIDE THE SOFTWARE "AS IS" WITHOUT WARRANTY OF ANY KIND,
304 EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
305 WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE
306 ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE SOFTWARE IS WITH
307 YOU. SHOULD THE SOFTWARE PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL
308 NECESSARY SERVICING, REPAIR, OR CORRECTION.
309
310 IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
311 WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
312 REDISTRIBUTE THE SOFTWARE AS PERMITTED BY THE ABOVE LICENCE, BE LIABLE
313 TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL, OR
314 CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
315 SOFTWARE (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
316 RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
317 FAILURE OF THE SOFTWARE TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
318 SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
319 DAMAGES.
320
321
){kind=link}
){kind=link}