1.. _callbacks: 2 3Callbacks 4========= 5 6For more fine-grained control, libcurl allows a number of callbacks to be 7associated with each connection. In pycurl, callbacks are defined using the 8``setopt()`` method for Curl objects with options ``WRITEFUNCTION``, 9``READFUNCTION``, ``HEADERFUNCTION``, ``PROGRESSFUNCTION``, 10``XFERINFOFUNCTION``, ``IOCTLFUNCTION``, or 11``DEBUGFUNCTION``. These options correspond to the libcurl options with ``CURLOPT_`` 12prefix removed. A callback in pycurl must be either a regular Python 13function, a class method or an extension type function. 14 15There are some limitations to some of the options which can be used 16concurrently with the pycurl callbacks compared to the libcurl callbacks. 17This is to allow different callback functions to be associated with different 18Curl objects. More specifically, ``WRITEDATA`` cannot be used with 19``WRITEFUNCTION``, ``READDATA`` cannot be used with ``READFUNCTION``, 20``WRITEHEADER`` cannot be used with ``HEADERFUNCTION``. 21In practice, these limitations can be overcome by having a 22callback function be a class instance method and rather use the class 23instance attributes to store per object data such as files used in the 24callbacks. 25 26The signature of each callback used in PycURL is documented below. 27 28 29Error Reporting 30--------------- 31 32PycURL callbacks are invoked as follows: 33 34Python application -> ``perform()`` -> libcurl (C code) -> Python callback 35 36Because callbacks are invoked by libcurl, they should not raise exceptions 37on failure but instead return appropriate values indicating failure. 38The documentation for individual callbacks below specifies expected success and 39failure return values. 40 41Unhandled exceptions propagated out of Python callbacks will be intercepted 42by PycURL or the Python runtime. This will fail the callback with a 43generic failure status, in turn failing the ``perform()`` operation. 44A failing ``perform()`` will raise ``pycurl.error``, but the error code 45used depends on the specific callback. 46 47Rich context information like exception objects can be stored in various ways, 48for example the following example stores OPENSOCKET callback exception on the 49Curl object:: 50 51 import pycurl, random, socket 52 53 class ConnectionRejected(Exception): 54 pass 55 56 def opensocket(curl, purpose, curl_address): 57 # always fail 58 curl.exception = ConnectionRejected('Rejecting connection attempt in opensocket callback') 59 return pycurl.SOCKET_BAD 60 61 # the callback must create a socket if it does not fail, 62 # see examples/opensocketexception.py 63 64 c = pycurl.Curl() 65 c.setopt(c.URL, 'http://pycurl.io') 66 c.exception = None 67 c.setopt(c.OPENSOCKETFUNCTION, 68 lambda purpose, address: opensocket(c, purpose, address)) 69 70 try: 71 c.perform() 72 except pycurl.error as e: 73 if e.args[0] == pycurl.E_COULDNT_CONNECT and c.exception: 74 print(c.exception) 75 else: 76 print(e) 77 78 79WRITEFUNCTION 80------------- 81 82.. function:: WRITEFUNCTION(byte string) -> number of characters written 83 84 Callback for writing data. Corresponds to `CURLOPT_WRITEFUNCTION`_ 85 in libcurl. 86 87 On Python 3, the argument is of type ``bytes``. 88 89 The ``WRITEFUNCTION`` callback may return the number of bytes written. 90 If this number is not equal to the size of the byte string, this signifies 91 an error and libcurl will abort the request. Returning ``None`` is an 92 alternate way of indicating that the callback has consumed all of the 93 string passed to it and, hence, succeeded. 94 95 `write_test.py test`_ shows how to use ``WRITEFUNCTION``. 96 97 98Example: Callbacks for document header and body 99~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 100 101This example prints the header data to stderr and the body data to stdout. 102Also note that neither callback returns the number of bytes written. For 103WRITEFUNCTION and HEADERFUNCTION callbacks, returning None implies that all 104bytes where written. 105 106:: 107 108 ## Callback function invoked when body data is ready 109 def body(buf): 110 # Print body data to stdout 111 import sys 112 sys.stdout.write(buf) 113 # Returning None implies that all bytes were written 114 115 ## Callback function invoked when header data is ready 116 def header(buf): 117 # Print header data to stderr 118 import sys 119 sys.stderr.write(buf) 120 # Returning None implies that all bytes were written 121 122 c = pycurl.Curl() 123 c.setopt(pycurl.URL, "http://www.python.org/") 124 c.setopt(pycurl.WRITEFUNCTION, body) 125 c.setopt(pycurl.HEADERFUNCTION, header) 126 c.perform() 127 128 129HEADERFUNCTION 130-------------- 131 132.. function:: HEADERFUNCTION(byte string) -> number of characters written 133 134 Callback for writing received headers. Corresponds to 135 `CURLOPT_HEADERFUNCTION`_ in libcurl. 136 137 On Python 3, the argument is of type ``bytes``. 138 139 The ``HEADERFUNCTION`` callback may return the number of bytes written. 140 If this number is not equal to the size of the byte string, this signifies 141 an error and libcurl will abort the request. Returning ``None`` is an 142 alternate way of indicating that the callback has consumed all of the 143 string passed to it and, hence, succeeded. 144 145 `header_test.py test`_ shows how to use ``WRITEFUNCTION``. 146 147 148READFUNCTION 149------------ 150 151.. function:: READFUNCTION(number of characters to read) -> byte string 152 153 Callback for reading data. Corresponds to `CURLOPT_READFUNCTION`_ in 154 libcurl. 155 156 On Python 3, the callback must return either a byte string or a Unicode 157 string consisting of ASCII code points only. 158 159 In addition, ``READFUNCTION`` may return ``READFUNC_ABORT`` or 160 ``READFUNC_PAUSE``. See the libcurl documentation for an explanation 161 of these values. 162 163 The `file_upload.py example`_ in the distribution contains example code for 164 using ``READFUNCTION``. 165 166 167.. _SEEKFUNCTION: 168 169SEEKFUNCTION 170------------ 171 172.. function:: SEEKFUNCTION(offset, origin) -> status 173 174 Callback for seek operations. Corresponds to `CURLOPT_SEEKFUNCTION`_ 175 in libcurl. 176 177 178IOCTLFUNCTION 179------------- 180 181.. function:: IOCTLFUNCTION(ioctl cmd) -> status 182 183 Callback for I/O operations. Corresponds to `CURLOPT_IOCTLFUNCTION`_ 184 in libcurl. 185 186 *Note:* this callback is deprecated. Use :ref:`SEEKFUNCTION <SEEKFUNCTION>` instead. 187 188 189DEBUGFUNCTION 190------------- 191 192.. function:: DEBUGFUNCTION(debug message type, debug message byte string) -> None 193 194 Callback for debug information. Corresponds to `CURLOPT_DEBUGFUNCTION`_ 195 in libcurl. 196 197 *Changed in version 7.19.5.2:* The second argument to a ``DEBUGFUNCTION`` 198 callback is now of type ``bytes`` on Python 3. Previously the argument was 199 of type ``str``. 200 201 `debug_test.py test`_ shows how to use ``DEBUGFUNCTION``. 202 203 204Example: Debug callbacks 205~~~~~~~~~~~~~~~~~~~~~~~~ 206 207This example shows how to use the debug callback. The debug message type is 208an integer indicating the type of debug message. The VERBOSE option must be 209enabled for this callback to be invoked. 210 211:: 212 213 def test(debug_type, debug_msg): 214 print "debug(%d): %s" % (debug_type, debug_msg) 215 216 c = pycurl.Curl() 217 c.setopt(pycurl.URL, "https://curl.haxx.se/") 218 c.setopt(pycurl.VERBOSE, 1) 219 c.setopt(pycurl.DEBUGFUNCTION, test) 220 c.perform() 221 222 223PROGRESSFUNCTION 224---------------- 225 226.. function:: PROGRESSFUNCTION(download total, downloaded, upload total, uploaded) -> status 227 228 Callback for progress meter. Corresponds to `CURLOPT_PROGRESSFUNCTION`_ 229 in libcurl. 230 231 ``PROGRESSFUNCTION`` receives amounts as floating point arguments to the 232 callback. Since libcurl 7.32.0 ``PROGRESSFUNCTION`` is deprecated; 233 ``XFERINFOFUNCTION`` should be used instead which receives amounts as 234 long integers. 235 236 ``NOPROGRESS`` option must be set for False libcurl to invoke a 237 progress callback, as PycURL by default sets ``NOPROGRESS`` to True. 238 239 240XFERINFOFUNCTION 241---------------- 242 243.. function:: XFERINFOFUNCTION(download total, downloaded, upload total, uploaded) -> status 244 245 Callback for progress meter. Corresponds to `CURLOPT_XFERINFOFUNCTION`_ 246 in libcurl. 247 248 ``XFERINFOFUNCTION`` receives amounts as long integers. 249 250 ``NOPROGRESS`` option must be set for False libcurl to invoke a 251 progress callback, as PycURL by default sets ``NOPROGRESS`` to True. 252 253 254Example: Download/upload progress callback 255~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 256 257This example shows how to use the progress callback. When downloading a 258document, the arguments related to uploads are zero, and vice versa. 259 260:: 261 262 ## Callback function invoked when download/upload has progress 263 def progress(download_t, download_d, upload_t, upload_d): 264 print "Total to download", download_t 265 print "Total downloaded", download_d 266 print "Total to upload", upload_t 267 print "Total uploaded", upload_d 268 269 c = pycurl.Curl() 270 c.setopt(c.URL, "http://slashdot.org/") 271 c.setopt(c.NOPROGRESS, False) 272 c.setopt(c.XFERINFOFUNCTION, progress) 273 c.perform() 274 275 276OPENSOCKETFUNCTION 277------------------ 278 279.. function:: OPENSOCKETFUNCTION(purpose, address) -> int 280 281 Callback for opening sockets. Corresponds to 282 `CURLOPT_OPENSOCKETFUNCTION`_ in libcurl. 283 284 *purpose* is a ``SOCKTYPE_*`` value. 285 286 *address* is a `namedtuple`_ with ``family``, ``socktype``, ``protocol`` 287 and ``addr`` fields, per `CURLOPT_OPENSOCKETFUNCTION`_ documentation. 288 289 *addr* is an object representing the address. Currently the following 290 address families are supported: 291 292 - ``AF_INET``: *addr* is a 2-tuple of ``(host, port)``. 293 - ``AF_INET6``: *addr* is a 4-tuple of ``(host, port, flow info, scope id)``. 294 - ``AF_UNIX``: *addr* is a byte string containing path to the Unix socket. 295 296 Availability: Unix. 297 298 This behavior matches that of Python's `socket module`_. 299 300 The callback should return a socket object, a socket file descriptor 301 or a Python object with a ``fileno`` property containing the socket 302 file descriptor. 303 304 The callback may be unset by calling :ref:`setopt <setopt>` with ``None`` 305 as the value or by calling :ref:`unsetopt <unsetopt>`. 306 307 `open_socket_cb_test.py test`_ shows how to use ``OPENSOCKETFUNCTION``. 308 309 *Changed in version 7.21.5:* Previously, the callback received ``family``, 310 ``socktype``, ``protocol`` and ``addr`` parameters (``purpose`` was 311 not passed and ``address`` was flattened). Also, ``AF_INET6`` addresses 312 were exposed as 2-tuples of ``(host, port)`` rather than 4-tuples. 313 314 *Changed in version 7.19.3:* ``addr`` parameter added to the callback. 315 316 317CLOSESOCKETFUNCTION 318------------------- 319 320.. function:: CLOSESOCKETFUNCTION(curlfd) -> int 321 322 Callback for setting socket options. Corresponds to 323 `CURLOPT_CLOSESOCKETFUNCTION`_ in libcurl. 324 325 *curlfd* is the file descriptor to be closed. 326 327 The callback should return an ``int``. 328 329 The callback may be unset by calling :ref:`setopt <setopt>` with ``None`` 330 as the value or by calling :ref:`unsetopt <unsetopt>`. 331 332 `close_socket_cb_test.py test`_ shows how to use ``CLOSESOCKETFUNCTION``. 333 334 335SOCKOPTFUNCTION 336--------------- 337 338.. function:: SOCKOPTFUNCTION(curlfd, purpose) -> int 339 340 Callback for setting socket options. Corresponds to `CURLOPT_SOCKOPTFUNCTION`_ 341 in libcurl. 342 343 *curlfd* is the file descriptor of the newly created socket. 344 345 *purpose* is a ``SOCKTYPE_*`` value. 346 347 The callback should return an ``int``. 348 349 The callback may be unset by calling :ref:`setopt <setopt>` with ``None`` 350 as the value or by calling :ref:`unsetopt <unsetopt>`. 351 352 `sockopt_cb_test.py test`_ shows how to use ``SOCKOPTFUNCTION``. 353 354 355SSH_KEYFUNCTION 356--------------- 357 358.. function:: SSH_KEYFUNCTION(known_key, found_key, match) -> int 359 360 Callback for known host matching logic. Corresponds to 361 `CURLOPT_SSH_KEYFUNCTION`_ in libcurl. 362 363 *known_key* and *found_key* are instances of ``KhKey`` class which is a 364 `namedtuple`_ with ``key`` and ``keytype`` fields, corresponding to 365 libcurl's ``struct curl_khkey``:: 366 367 KhKey = namedtuple('KhKey', ('key', 'keytype')) 368 369 On Python 2, the *key* field of ``KhKey`` is a ``str``. On Python 3, the 370 *key* field is ``bytes``. *keytype* is an ``int``. 371 372 *known_key* may be ``None`` when there is no known matching host key. 373 374 ``SSH_KEYFUNCTION`` callback should return a ``KHSTAT_*`` value. 375 376 The callback may be unset by calling :ref:`setopt <setopt>` with ``None`` 377 as the value or by calling :ref:`unsetopt <unsetopt>`. 378 379 `ssh_key_cb_test.py test`_ shows how to use ``SSH_KEYFUNCTION``. 380 381 382TIMERFUNCTION 383------------- 384 385.. function:: TIMERFUNCTION(timeout_ms) -> None 386 387 Callback for installing a timer requested by libcurl. Corresponds to 388 `CURLMOPT_TIMERFUNCTION`_. 389 390 The application should arrange for a non-repeating timer to fire in 391 ``timeout_ms`` milliseconds, at which point the application should call 392 either :ref:`socket_action <multi-socket_action>` or 393 :ref:`perform <multi-perform>`. 394 395 See ``examples/multi-socket_action-select.py`` for an example program 396 that uses the timer function and the socket function. 397 398 399SOCKETFUNCTION 400-------------- 401 402.. function:: SOCKETFUNCTION(what, sock_fd, multi, socketp) -> None 403 404 Callback notifying the application about activity on libcurl sockets. 405 Corresponds to `CURLMOPT_SOCKETFUNCTION`_. 406 407 Note that the PycURL callback takes ``what`` as the first argument and 408 ``sock_fd`` as the second argument, whereas the libcurl callback takes 409 ``sock_fd`` as the first argument and ``what`` as the second argument. 410 411 The ``userp`` ("private callback pointer") argument, as described in the 412 ``CURLMOPT_SOCKETFUNCTION`` documentation, is set to the ``CurlMulti`` 413 instance. 414 415 The ``socketp`` ("private socket pointer") argument, as described in the 416 ``CURLMOPT_SOCKETFUNCTION`` documentation, is set to the value provided 417 to the :ref:`assign <multi-assign>` method for the corresponding 418 ``sock_fd``, or ``None`` if no value was assigned. 419 420 See ``examples/multi-socket_action-select.py`` for an example program 421 that uses the timer function and the socket function. 422 423 424.. _CURLOPT_HEADERFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_HEADERFUNCTION.html 425.. _CURLOPT_WRITEFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_WRITEFUNCTION.html 426.. _CURLOPT_READFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_READFUNCTION.html 427.. _CURLOPT_PROGRESSFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_PROGRESSFUNCTION.html 428.. _CURLOPT_XFERINFOFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_XFERINFOFUNCTION.html 429.. _CURLOPT_DEBUGFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_DEBUGFUNCTION.html 430.. _CURLOPT_SEEKFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_SEEKFUNCTION.html 431.. _CURLOPT_IOCTLFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_IOCTLFUNCTION.html 432.. _file_upload.py example: https://github.com/pycurl/pycurl/blob/master/examples/file_upload.py 433.. _write_test.py test: https://github.com/pycurl/pycurl/blob/master/tests/write_test.py 434.. _header_test.py test: https://github.com/pycurl/pycurl/blob/master/tests/header_test.py 435.. _debug_test.py test: https://github.com/pycurl/pycurl/blob/master/tests/debug_test.py 436.. _CURLOPT_SSH_KEYFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_SSH_KEYFUNCTION.html 437.. _namedtuple: https://docs.python.org/library/collections.html#collections.namedtuple 438.. _CURLOPT_SOCKOPTFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_SOCKOPTFUNCTION.html 439.. _sockopt_cb_test.py test: https://github.com/pycurl/pycurl/blob/master/tests/sockopt_cb_test.py 440.. _ssh_key_cb_test.py test: https://github.com/pycurl/pycurl/blob/master/tests/ssh_key_cb_test.py 441.. _CURLOPT_CLOSESOCKETFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_CLOSESOCKETFUNCTION.html 442.. _close_socket_cb_test.py test: https://github.com/pycurl/pycurl/blob/master/tests/close_socket_cb_test.py 443.. _CURLOPT_OPENSOCKETFUNCTION: https://curl.haxx.se/libcurl/c/CURLOPT_OPENSOCKETFUNCTION.html 444.. _open_socket_cb_test.py test: https://github.com/pycurl/pycurl/blob/master/tests/open_socket_cb_test.py 445.. _socket module: https://docs.python.org/library/socket.html 446.. _CURLMOPT_TIMERFUNCTION: https://curl.se/libcurl/c/CURLMOPT_TIMERFUNCTION.html 447.. _CURLMOPT_SOCKETFUNCTION: https://curl.se/libcurl/c/CURLMOPT_SOCKETFUNCTION.html 448