1 _ _ ____ _ 2 ___| | | | _ \| | 3 / __| | | | |_) | | 4 | (__| |_| | _ <| |___ 5 \___|\___/|_| \_\_____| 6 7 Things that could be nice to do in the future 8 9 Things to do in project curl. Please tell us what you think, contribute and 10 send us patches that improve things! 11 12 Be aware that these are things that we could do, or have once been considered 13 things we could do. If you want to work on any of these areas, please 14 consider bringing it up for discussions first on the mailing list so that we 15 all agree it is still a good idea for the project! 16 17 All bugs documented in the KNOWN_BUGS document are subject for fixing! 18 19 1. libcurl 20 1.1 TFO support on Windows 21 1.2 Consult %APPDATA% also for .netrc 22 1.3 struct lifreq 23 1.4 alt-svc sharing 24 1.5 get rid of PATH_MAX 25 1.6 native IDN support on macOS 26 1.7 Support HTTP/2 for HTTP(S) proxies 27 1.8 CURLOPT_RESOLVE for any port number 28 1.9 Cache negative name resolves 29 1.10 auto-detect proxy 30 1.11 minimize dependencies with dynamically loaded modules 31 1.12 updated DNS server while running 32 1.13 c-ares and CURLOPT_OPENSOCKETFUNCTION 33 1.14 Typesafe curl_easy_setopt() 34 1.15 Monitor connections in the connection pool 35 1.16 Try to URL encode given URL 36 1.17 Add support for IRIs 37 1.18 try next proxy if one does not work 38 1.19 provide timing info for each redirect 39 1.20 SRV and URI DNS records 40 1.21 netrc caching and sharing 41 1.22 CURLINFO_PAUSE_STATE 42 1.23 Offer API to flush the connection pool 43 1.24 TCP Fast Open for windows 44 1.25 Expose tried IP addresses that failed 45 1.27 hardcode the "localhost" addresses 46 1.28 FD_CLOEXEC 47 1.29 Upgrade to websockets 48 1.30 config file parsing 49 1.31 erase secrets from heap/stack after use 50 1.32 add asynch getaddrinfo support 51 52 2. libcurl - multi interface 53 2.1 More non-blocking 54 2.2 Better support for same name resolves 55 2.3 Non-blocking curl_multi_remove_handle() 56 2.4 Split connect and authentication process 57 2.5 Edge-triggered sockets should work 58 2.6 multi upkeep 59 2.7 Virtual external sockets 60 2.8 dynamically decide to use socketpair 61 62 3. Documentation 63 3.1 Improve documentation about fork safety 64 3.2 Provide cmake config-file 65 66 4. FTP 67 4.1 HOST 68 4.2 Alter passive/active on failure and retry 69 4.3 Earlier bad letter detection 70 4.5 ASCII support 71 4.6 GSSAPI via Windows SSPI 72 4.7 STAT for LIST without data connection 73 4.8 Option to ignore private IP addresses in PASV response 74 75 5. HTTP 76 5.1 Better persistency for HTTP 1.0 77 5.2 Set custom client ip when using haproxy protocol 78 5.3 Rearrange request header order 79 5.4 Allow SAN names in HTTP/2 server push 80 5.5 auth= in URLs 81 5.6 alt-svc should fallback if alt-svc does not work 82 83 6. TELNET 84 6.1 ditch stdin 85 6.2 ditch telnet-specific select 86 6.3 feature negotiation debug data 87 88 7. SMTP 89 7.2 Enhanced capability support 90 7.3 Add CURLOPT_MAIL_CLIENT option 91 92 8. POP3 93 8.2 Enhanced capability support 94 95 9. IMAP 96 9.1 Enhanced capability support 97 98 10. LDAP 99 10.1 SASL based authentication mechanisms 100 10.2 CURLOPT_SSL_CTX_FUNCTION for LDAPS 101 10.3 Paged searches on LDAP server 102 103 11. SMB 104 11.1 File listing support 105 11.2 Honor file timestamps 106 11.3 Use NTLMv2 107 11.4 Create remote directories 108 109 12. FILE 110 12.1 Directory listing for FILE: 111 112 13. SSL 113 13.1 TLS-PSK with OpenSSL 114 13.2 Provide mutex locking API 115 13.4 Cache/share OpenSSL contexts 116 13.5 Export session ids 117 13.6 Provide callback for cert verification 118 13.8 Support DANE 119 13.9 TLS record padding 120 13.10 Support Authority Information Access certificate extension (AIA) 121 13.11 Support intermediate & root pinning for PINNEDPUBLICKEY 122 13.13 Make sure we forbid TLS 1.3 post-handshake authentication 123 13.14 Support the clienthello extension 124 125 14. GnuTLS 126 14.2 check connection 127 128 15. Schannel 129 15.1 Extend support for client certificate authentication 130 15.2 Extend support for the --ciphers option 131 15.4 Add option to allow abrupt server closure 132 133 16. SASL 134 16.1 Other authentication mechanisms 135 16.2 Add QOP support to GSSAPI authentication 136 137 17. SSH protocols 138 17.1 Multiplexing 139 17.2 Handle growing SFTP files 140 17.4 Support CURLOPT_PREQUOTE 141 17.5 SSH over HTTPS proxy with more backends 142 143 18. Command line tool 144 18.1 sync 145 18.2 glob posts 146 18.3 prevent file overwriting 147 18.4 --proxycommand 148 18.5 UTF-8 filenames in Content-Disposition 149 18.6 Option to make -Z merge lined based outputs on stdout 150 18.7 at least N milliseconds between requests 151 18.8 Consider convenience options for JSON and XML? 152 18.9 Choose the name of file in braces for complex URLs 153 18.10 improve how curl works in a windows console window 154 18.11 Windows: set attribute 'archive' for completed downloads 155 18.12 keep running, read instructions from pipe/socket 156 18.13 Ratelimit or wait between serial requests 157 18.14 --dry-run 158 18.15 --retry should resume 159 18.16 send only part of --data 160 18.17 consider file name from the redirected URL with -O ? 161 18.18 retry on network is unreachable 162 18.19 expand ~/ in config files 163 18.20 host name sections in config files 164 18.21 retry on the redirected-to URL 165 18.23 Set the modification date on an uploaded file 166 18.24 Use multiple parallel transfers for a single download 167 18.25 Prevent terminal injection when writing to terminal 168 18.26 Custom progress meter update interval 169 170 19. Build 171 19.1 roffit 172 19.2 Enable PIE and RELRO by default 173 19.3 Do not use GNU libtool on OpenBSD 174 19.4 Package curl for Windows in a signed installer 175 19.5 make configure use --cache-file more and better 176 177 20. Test suite 178 20.1 SSL tunnel 179 20.2 nicer lacking perl message 180 20.3 more protocols supported 181 20.4 more platforms supported 182 20.5 Add support for concurrent connections 183 20.6 Use the RFC6265 test suite 184 20.7 Support LD_PRELOAD on macOS 185 20.8 Run web-platform-tests url tests 186 20.9 Bring back libssh tests on Travis 187 188 21. MQTT 189 21.1 Support rate-limiting 190 191============================================================================== 192 1931. libcurl 194 1951.1 TFO support on Windows 196 197 TCP Fast Open is supported on several platforms but not on Windows. Work on 198 this was once started but never finished. 199 200 See https://github.com/curl/curl/pull/3378 201 2021.2 Consult %APPDATA% also for .netrc 203 204 %APPDATA%\.netrc is not considered when running on Windows. should not it? 205 206 See https://github.com/curl/curl/issues/4016 207 2081.3 struct lifreq 209 210 Use 'struct lifreq' and SIOCGLIFADDR instead of 'struct ifreq' and 211 SIOCGIFADDR on newer Solaris versions as they claim the latter is obsolete. 212 To support IPv6 interface addresses for network interfaces properly. 213 2141.4 alt-svc sharing 215 216 The share interface could benefit from allowing the alt-svc cache to be 217 possible to share between easy handles. 218 219 See https://github.com/curl/curl/issues/4476 220 2211.5 get rid of PATH_MAX 222 223 Having code use and rely on PATH_MAX is not nice: 224 https://insanecoding.blogspot.com/2007/11/pathmax-simply-isnt.html 225 226 Currently the libssh2 SSH based code uses it, but to remove PATH_MAX from 227 there we need libssh2 to properly tell us when we pass in a too small buffer 228 and its current API (as of libssh2 1.2.7) does not. 229 2301.6 native IDN support on macOS 231 232 On recent macOS versions, the getaddrinfo() function itself has built-in IDN 233 support. By setting the AI_CANONNAME flag, the function will return the 234 encoded name in the ai_canonname struct field in the returned information. 235 This could be used by curl on macOS when built without a separate IDN library 236 and an IDN host name is used in a URL. 237 238 See initial work in https://github.com/curl/curl/pull/5371 239 2401.7 Support HTTP/2 for HTTP(S) proxies 241 242 Support for doing HTTP/2 to HTTP and HTTPS proxies is still missing. 243 244 See https://github.com/curl/curl/issues/3570 245 2461.8 CURLOPT_RESOLVE for any port number 247 248 This option allows applications to set a replacement IP address for a given 249 host + port pair. Consider making support for providing a replacement address 250 for the host name on all port numbers. 251 252 See https://github.com/curl/curl/issues/1264 253 2541.9 Cache negative name resolves 255 256 A name resolve that has failed is likely to fail when made again within a 257 short period of time. Currently we only cache positive responses. 258 2591.10 auto-detect proxy 260 261 libcurl could be made to detect the system proxy setup automatically and use 262 that. On Windows, macOS and Linux desktops for example. 263 264 The pull-request to use libproxy for this was deferred due to doubts on the 265 reliability of the dependency and how to use it: 266 https://github.com/curl/curl/pull/977 267 268 libdetectproxy is a (C++) library for detecting the proxy on Windows 269 https://github.com/paulharris/libdetectproxy 270 2711.11 minimize dependencies with dynamically loaded modules 272 273 We can create a system with loadable modules/plug-ins, where these modules 274 would be the ones that link to 3rd party libs. That would allow us to avoid 275 having to load ALL dependencies since only the necessary ones for this 276 app/invoke/used protocols would be necessary to load. See 277 https://github.com/curl/curl/issues/349 278 2791.12 updated DNS server while running 280 281 If /etc/resolv.conf gets updated while a program using libcurl is running, it 282 is may cause name resolves to fail unless res_init() is called. We should 283 consider calling res_init() + retry once unconditionally on all name resolve 284 failures to mitigate against this. Firefox works like that. Note that Windows 285 does not have res_init() or an alternative. 286 287 https://github.com/curl/curl/issues/2251 288 2891.13 c-ares and CURLOPT_OPENSOCKETFUNCTION 290 291 curl will create most sockets via the CURLOPT_OPENSOCKETFUNCTION callback and 292 close them with the CURLOPT_CLOSESOCKETFUNCTION callback. However, c-ares 293 does not use those functions and instead opens and closes the sockets 294 itself. This means that when curl passes the c-ares socket to the 295 CURLMOPT_SOCKETFUNCTION it is not owned by the application like other sockets. 296 297 See https://github.com/curl/curl/issues/2734 298 2991.14 Typesafe curl_easy_setopt() 300 301 One of the most common problems in libcurl using applications is the lack of 302 type checks for curl_easy_setopt() which happens because it accepts varargs 303 and thus can take any type. 304 305 One possible solution to this is to introduce a few different versions of the 306 setopt version for the different kinds of data you can set. 307 308 curl_easy_set_num() - sets a long value 309 310 curl_easy_set_large() - sets a curl_off_t value 311 312 curl_easy_set_ptr() - sets a pointer 313 314 curl_easy_set_cb() - sets a callback PLUS its callback data 315 3161.15 Monitor connections in the connection pool 317 318 libcurl's connection cache or pool holds a number of open connections for the 319 purpose of possible subsequent connection reuse. It may contain a few up to a 320 significant amount of connections. Currently, libcurl leaves all connections 321 as they are and first when a connection is iterated over for matching or 322 reuse purpose it is verified that it is still alive. 323 324 Those connections may get closed by the server side for idleness or they may 325 get a HTTP/2 ping from the peer to verify that they are still alive. By adding 326 monitoring of the connections while in the pool, libcurl can detect dead 327 connections (and close them) better and earlier, and it can handle HTTP/2 328 pings to keep such ones alive even when not actively doing transfers on them. 329 3301.16 Try to URL encode given URL 331 332 Given a URL that for example contains spaces, libcurl could have an option 333 that would try somewhat harder than it does now and convert spaces to %20 and 334 perhaps URL encoded byte values over 128 etc (basically do what the redirect 335 following code already does). 336 337 https://github.com/curl/curl/issues/514 338 3391.17 Add support for IRIs 340 341 IRIs (RFC 3987) allow localized, non-ascii, names in the URL. To properly 342 support this, curl/libcurl would need to translate/encode the given input 343 from the input string encoding into percent encoded output "over the wire". 344 345 To make that work smoothly for curl users even on Windows, curl would 346 probably need to be able to convert from several input encodings. 347 3481.18 try next proxy if one does not work 349 350 Allow an application to specify a list of proxies to try, and failing to 351 connect to the first go on and try the next instead until the list is 352 exhausted. Browsers support this feature at least when they specify proxies 353 using PACs. 354 355 https://github.com/curl/curl/issues/896 356 3571.19 provide timing info for each redirect 358 359 curl and libcurl provide timing information via a set of different 360 time-stamps (CURLINFO_*_TIME). When curl is following redirects, those 361 returned time value are the accumulated sums. An improvement could be to 362 offer separate timings for each redirect. 363 364 https://github.com/curl/curl/issues/6743 365 3661.20 SRV and URI DNS records 367 368 Offer support for resolving SRV and URI DNS records for libcurl to know which 369 server to connect to for various protocols (including HTTP!). 370 3711.21 netrc caching and sharing 372 373 The netrc file is read and parsed each time a connection is setup, which 374 means that if a transfer needs multiple connections for authentication or 375 redirects, the file might be reread (and parsed) multiple times. This makes 376 it impossible to provide the file as a pipe. 377 3781.22 CURLINFO_PAUSE_STATE 379 380 Return information about the transfer's current pause state, in both 381 directions. https://github.com/curl/curl/issues/2588 382 3831.23 Offer API to flush the connection pool 384 385 Sometimes applications want to flush all the existing connections kept alive. 386 An API could allow a forced flush or just a forced loop that would properly 387 close all connections that have been closed by the server already. 388 3891.24 TCP Fast Open for windows 390 391 libcurl supports the CURLOPT_TCP_FASTOPEN option since 7.49.0 for Linux and 392 Mac OS. Windows supports TCP Fast Open starting with Windows 10, version 1607 393 and we should add support for it. 394 3951.25 Expose tried IP addresses that failed 396 397 When libcurl fails to connect to a host, it should be able to offer the 398 application the list of IP addresses that were used in the attempt. 399 400 https://github.com/curl/curl/issues/2126 401 4021.27 hardcode the "localhost" addresses 403 404 There's this new spec getting adopted that says "localhost" should always and 405 unconditionally be a local address and not get resolved by a DNS server. A 406 fine way for curl to fix this would be to simply hard-code the response to 407 127.0.0.1 and/or ::1 (depending on what IP versions that are requested). This 408 is what the browsers probably will do with this hostname. 409 410 https://bugzilla.mozilla.org/show_bug.cgi?id=1220810 411 412 https://tools.ietf.org/html/draft-ietf-dnsop-let-localhost-be-localhost-02 413 4141.28 FD_CLOEXEC 415 416 It sets the close-on-exec flag for the file descriptor, which causes the file 417 descriptor to be automatically (and atomically) closed when any of the 418 exec-family functions succeed. Should probably be set by default? 419 420 https://github.com/curl/curl/issues/2252 421 4221.29 Upgrade to websockets 423 424 libcurl could offer a smoother path to get to a websocket connection. 425 See https://github.com/curl/curl/issues/3523 426 427 Michael Kaufmann suggestion here: 428 https://curl.se/video/curlup-2017/2017-03-19_05_Michael_Kaufmann_Websocket_support_for_curl.mp4 429 4301.30 config file parsing 431 432 Consider providing an API, possibly in a separate companion library, for 433 parsing a config file like curl's -K/--config option to allow applications to 434 get the same ability to read curl options from files. 435 436 See https://github.com/curl/curl/issues/3698 437 4381.31 erase secrets from heap/stack after use 439 440 Introducing a concept and system to erase secrets from memory after use, it 441 could help mitigate and lessen the impact of (future) security problems etc. 442 However: most secrets are passed to libcurl as clear text from the 443 application and then clearing them within the library adds nothing... 444 445 https://github.com/curl/curl/issues/7268 446 4471.32 add asynch getaddrinfo support 448 449 Use getaddrinfo_a() to provide an asynch name resolver backend to libcurl 450 that does not use threads and does not depend on c-ares. The getaddrinfo_a 451 function is (probably?) glibc specific but that is a widely used libc among 452 our users. 453 454 https://github.com/curl/curl/pull/6746 455 4562. libcurl - multi interface 457 4582.1 More non-blocking 459 460 Make sure we do not ever loop because of non-blocking sockets returning 461 EWOULDBLOCK or similar. Blocking cases include: 462 463 - Name resolves on non-windows unless c-ares or the threaded resolver is used. 464 465 - The threaded resolver may block on cleanup: 466 https://github.com/curl/curl/issues/4852 467 468 - file:// transfers 469 470 - TELNET transfers 471 472 - GSSAPI authentication for FTP transfers 473 474 - The "DONE" operation (post transfer protocol-specific actions) for the 475 protocols SFTP, SMTP, FTP. Fixing multi_done() for this is a worthy task. 476 477 - curl_multi_remove_handle for any of the above. See section 2.3. 478 4792.2 Better support for same name resolves 480 481 If a name resolve has been initiated for name NN and a second easy handle 482 wants to resolve that name as well, make it wait for the first resolve to end 483 up in the cache instead of doing a second separate resolve. This is 484 especially needed when adding many simultaneous handles using the same host 485 name when the DNS resolver can get flooded. 486 4872.3 Non-blocking curl_multi_remove_handle() 488 489 The multi interface has a few API calls that assume a blocking behavior, like 490 add_handle() and remove_handle() which limits what we can do internally. The 491 multi API need to be moved even more into a single function that "drives" 492 everything in a non-blocking manner and signals when something is done. A 493 remove or add would then only ask for the action to get started and then 494 multi_perform() etc still be called until the add/remove is completed. 495 4962.4 Split connect and authentication process 497 498 The multi interface treats the authentication process as part of the connect 499 phase. As such any failures during authentication will not trigger the relevant 500 QUIT or LOGOFF for protocols such as IMAP, POP3 and SMTP. 501 5022.5 Edge-triggered sockets should work 503 504 The multi_socket API should work with edge-triggered socket events. One of 505 the internal actions that need to be improved for this to work perfectly is 506 the 'maxloops' handling in transfer.c:readwrite_data(). 507 5082.6 multi upkeep 509 510 In libcurl 7.62.0 we introduced curl_easy_upkeep. It unfortunately only works 511 on easy handles. We should introduces a version of that for the multi handle, 512 and also consider doing "upkeep" automatically on connections in the 513 connection pool when the multi handle is in used. 514 515 See https://github.com/curl/curl/issues/3199 516 5172.7 Virtual external sockets 518 519 libcurl performs operations on the given file descriptor that presumes it is 520 a socket and an application cannot replace them at the moment. Allowing an 521 application to fully replace those would allow a larger degree of freedom and 522 flexibility. 523 524 See https://github.com/curl/curl/issues/5835 525 5262.8 dynamically decide to use socketpair 527 528 For users who do not use curl_multi_wait() or do not care for 529 curl_multi_wakeup(), we could introduce a way to make libcurl NOT 530 create a socketpair in the multi handle. 531 532 See https://github.com/curl/curl/issues/4829 533 5343. Documentation 535 5363.1 Improve documentation about fork safety 537 538 See https://github.com/curl/curl/issues/6968 539 5403.2 Provide cmake config-file 541 542 A config-file package is a set of files provided by us to allow applications 543 to write cmake scripts to find and use libcurl easier. See 544 https://github.com/curl/curl/issues/885 545 5464. FTP 547 5484.1 HOST 549 550 HOST is a command for a client to tell which host name to use, to offer FTP 551 servers named-based virtual hosting: 552 553 https://tools.ietf.org/html/rfc7151 554 5554.2 Alter passive/active on failure and retry 556 557 When trying to connect passively to a server which only supports active 558 connections, libcurl returns CURLE_FTP_WEIRD_PASV_REPLY and closes the 559 connection. There could be a way to fallback to an active connection (and 560 vice versa). https://curl.se/bug/feature.cgi?id=1754793 561 5624.3 Earlier bad letter detection 563 564 Make the detection of (bad) %0d and %0a codes in FTP URL parts earlier in the 565 process to avoid doing a resolve and connect in vain. 566 5674.5 ASCII support 568 569 FTP ASCII transfers do not follow RFC959. They do not convert the data 570 accordingly. 571 5724.6 GSSAPI via Windows SSPI 573 574 In addition to currently supporting the SASL GSSAPI mechanism (Kerberos V5) 575 via third-party GSS-API libraries, such as Heimdal or MIT Kerberos, also add 576 support for GSSAPI authentication via Windows SSPI. 577 5784.7 STAT for LIST without data connection 579 580 Some FTP servers allow STAT for listing directories instead of using LIST, 581 and the response is then sent over the control connection instead of as the 582 otherwise usedw data connection: https://www.nsftools.com/tips/RawFTP.htm#STAT 583 584 This is not detailed in any FTP specification. 585 5864.8 Option to ignore private IP addresses in PASV response 587 588 Some servers respond with and some other FTP client implementations can 589 ignore private (RFC 1918 style) IP addresses when received in PASV responses. 590 To consider for libcurl as well. See https://github.com/curl/curl/issues/1455 591 5925. HTTP 593 5945.1 Better persistency for HTTP 1.0 595 596 "Better" support for persistent connections over HTTP 1.0 597 https://curl.se/bug/feature.cgi?id=1089001 598 5995.2 Set custom client ip when using haproxy protocol 600 601 This would allow testing servers with different client ip addresses (without 602 using x-forward-for header). 603 604 https://github.com/curl/curl/issues/5125 605 6065.3 Rearrange request header order 607 608 Server implementors often make an effort to detect browser and to reject 609 clients it can detect to not match. One of the last details we cannot yet 610 control in libcurl's HTTP requests, which also can be exploited to detect 611 that libcurl is in fact used even when it tries to impersonate a browser, is 612 the order of the request headers. I propose that we introduce a new option in 613 which you give headers a value, and then when the HTTP request is built it 614 sorts the headers based on that number. We could then have internally created 615 headers use a default value so only headers that need to be moved have to be 616 specified. 617 6185.4 Allow SAN names in HTTP/2 server push 619 620 curl only allows HTTP/2 push promise if the provided :authority header value 621 exactly matches the host name given in the URL. It could be extended to allow 622 any name that would match the Subject Alternative Names in the server's TLS 623 certificate. 624 625 See https://github.com/curl/curl/pull/3581 626 6275.5 auth= in URLs 628 629 Add the ability to specify the preferred authentication mechanism to use by 630 using ;auth=<mech> in the login part of the URL. 631 632 For example: 633 634 http://test:pass;auth=NTLM@example.com would be equivalent to specifying 635 --user test:pass;auth=NTLM or --user test:pass --ntlm from the command line. 636 637 Additionally this should be implemented for proxy base URLs as well. 638 6395.6 alt-svc should fallback if alt-svc does not work 640 641 The alt-svc: header provides a set of alternative services for curl to use 642 instead of the original. If the first attempted one fails, it should try the 643 next etc and if all alternatives fail go back to the original. 644 645 See https://github.com/curl/curl/issues/4908 646 6476. TELNET 648 6496.1 ditch stdin 650 651 Reading input (to send to the remote server) on stdin is a crappy solution 652 for library purposes. We need to invent a good way for the application to be 653 able to provide the data to send. 654 6556.2 ditch telnet-specific select 656 657 Move the telnet support's network select() loop go away and merge the code 658 into the main transfer loop. Until this is done, the multi interface will not 659 work for telnet. 660 6616.3 feature negotiation debug data 662 663 Add telnet feature negotiation data to the debug callback as header data. 664 665 6667. SMTP 667 6687.2 Enhanced capability support 669 670 Add the ability, for an application that uses libcurl, to obtain the list of 671 capabilities returned from the EHLO command. 672 6737.3 Add CURLOPT_MAIL_CLIENT option 674 675 Rather than use the URL to specify the mail client string to present in the 676 HELO and EHLO commands, libcurl should support a new CURLOPT specifically for 677 specifying this data as the URL is non-standard and to be honest a bit of a 678 hack ;-) 679 680 Please see the following thread for more information: 681 https://curl.se/mail/lib-2012-05/0178.html 682 683 6848. POP3 685 6868.2 Enhanced capability support 687 688 Add the ability, for an application that uses libcurl, to obtain the list of 689 capabilities returned from the CAPA command. 690 6919. IMAP 692 6939.1 Enhanced capability support 694 695 Add the ability, for an application that uses libcurl, to obtain the list of 696 capabilities returned from the CAPABILITY command. 697 69810. LDAP 699 70010.1 SASL based authentication mechanisms 701 702 Currently the LDAP module only supports ldap_simple_bind_s() in order to bind 703 to an LDAP server. However, this function sends username and password details 704 using the simple authentication mechanism (as clear text). However, it should 705 be possible to use ldap_bind_s() instead specifying the security context 706 information ourselves. 707 70810.2 CURLOPT_SSL_CTX_FUNCTION for LDAPS 709 710 CURLOPT_SSL_CTX_FUNCTION works perfectly for HTTPS and email protocols, but 711 it has no effect for LDAPS connections. 712 713 https://github.com/curl/curl/issues/4108 714 71510.3 Paged searches on LDAP server 716 717 https://github.com/curl/curl/issues/4452 718 71911. SMB 720 72111.1 File listing support 722 723 Add support for listing the contents of a SMB share. The output should 724 probably be the same as/similar to FTP. 725 72611.2 Honor file timestamps 727 728 The timestamp of the transferred file should reflect that of the original 729 file. 730 73111.3 Use NTLMv2 732 733 Currently the SMB authentication uses NTLMv1. 734 73511.4 Create remote directories 736 737 Support for creating remote directories when uploading a file to a directory 738 that does not exist on the server, just like --ftp-create-dirs. 739 740 74112. FILE 742 74312.1 Directory listing for FILE: 744 745 Add support for listing the contents of a directory accessed with FILE. The 746 output should probably be the same as/similar to FTP. 747 748 74913. SSL 750 75113.1 TLS-PSK with OpenSSL 752 753 Transport Layer Security pre-shared key ciphersuites (TLS-PSK) is a set of 754 cryptographic protocols that provide secure communication based on pre-shared 755 keys (PSKs). These pre-shared keys are symmetric keys shared in advance among 756 the communicating parties. 757 758 https://github.com/curl/curl/issues/5081 759 76013.2 Provide mutex locking API 761 762 Provide a libcurl API for setting mutex callbacks in the underlying SSL 763 library, so that the same application code can use mutex-locking 764 independently of OpenSSL or GnutTLS being used. 765 76613.4 Cache/share OpenSSL contexts 767 768 "Look at SSL cafile - quick traces look to me like these are done on every 769 request as well, when they should only be necessary once per SSL context (or 770 once per handle)". The major improvement we can rather easily do is to make 771 sure we do not create and kill a new SSL "context" for every request, but 772 instead make one for every connection and re-use that SSL context in the same 773 style connections are re-used. It will make us use slightly more memory but 774 it will libcurl do less creations and deletions of SSL contexts. 775 776 Technically, the "caching" is probably best implemented by getting added to 777 the share interface so that easy handles who want to and can reuse the 778 context specify that by sharing with the right properties set. 779 780 https://github.com/curl/curl/issues/1110 781 78213.5 Export session ids 783 784 Add an interface to libcurl that enables "session IDs" to get 785 exported/imported. Cris Bailiff said: "OpenSSL has functions which can 786 serialise the current SSL state to a buffer of your choice, and recover/reset 787 the state from such a buffer at a later date - this is used by mod_ssl for 788 apache to implement and SSL session ID cache". 789 79013.6 Provide callback for cert verification 791 792 OpenSSL supports a callback for customised verification of the peer 793 certificate, but this does not seem to be exposed in the libcurl APIs. Could 794 it be? There's so much that could be done if it were! 795 79613.8 Support DANE 797 798 DNS-Based Authentication of Named Entities (DANE) is a way to provide SSL 799 keys and certs over DNS using DNSSEC as an alternative to the CA model. 800 https://www.rfc-editor.org/rfc/rfc6698.txt 801 802 An initial patch was posted by Suresh Krishnaswamy on March 7th 2013 803 (https://curl.se/mail/lib-2013-03/0075.html) but it was a too simple 804 approach. See Daniel's comments: 805 https://curl.se/mail/lib-2013-03/0103.html . libunbound may be the 806 correct library to base this development on. 807 808 Björn Stenberg wrote a separate initial take on DANE that was never 809 completed. 810 81113.9 TLS record padding 812 813 TLS (1.3) offers optional record padding and OpenSSL provides an API for it. 814 I could make sense for libcurl to offer this ability to applications to make 815 traffic patterns harder to figure out by network traffic observers. 816 817 See https://github.com/curl/curl/issues/5398 818 81913.10 Support Authority Information Access certificate extension (AIA) 820 821 AIA can provide various things like CRLs but more importantly information 822 about intermediate CA certificates that can allow validation path to be 823 fulfilled when the HTTPS server does not itself provide them. 824 825 Since AIA is about downloading certs on demand to complete a TLS handshake, 826 it is probably a bit tricky to get done right. 827 828 See https://github.com/curl/curl/issues/2793 829 83013.11 Support intermediate & root pinning for PINNEDPUBLICKEY 831 832 CURLOPT_PINNEDPUBLICKEY does not consider the hashes of intermediate & root 833 certificates when comparing the pinned keys. Therefore it is not compatible 834 with "HTTP Public Key Pinning" as there also intermediate and root 835 certificates can be pinned. This is useful as it prevents webadmins from 836 "locking themselves out of their servers". 837 838 Adding this feature would make curls pinning 100% compatible to HPKP and 839 allow more flexible pinning. 840 84113.13 Make sure we forbid TLS 1.3 post-handshake authentication 842 843 RFC 8740 explains how using HTTP/2 must forbid the use of TLS 1.3 844 post-handshake authentication. We should make sure to live up to that. 845 846 See https://github.com/curl/curl/issues/5396 847 84813.14 Support the clienthello extension 849 850 Certain stupid networks and middle boxes have a problem with SSL handshake 851 packets that are within a certain size range because how that sets some bits 852 that previously (in older TLS version) were not set. The clienthello 853 extension adds padding to avoid that size range. 854 855 https://tools.ietf.org/html/rfc7685 856 https://github.com/curl/curl/issues/2299 857 85814. GnuTLS 859 86014.2 check connection 861 862 Add a way to check if the connection seems to be alive, to correspond to the 863 SSL_peak() way we use with OpenSSL. 864 86515. Schannel 866 86715.1 Extend support for client certificate authentication 868 869 The existing support for the -E/--cert and --key options could be 870 extended by supplying a custom certificate and key in PEM format, see: 871 - Getting a Certificate for Schannel 872 https://msdn.microsoft.com/en-us/library/windows/desktop/aa375447.aspx 873 87415.2 Extend support for the --ciphers option 875 876 The existing support for the --ciphers option could be extended 877 by mapping the OpenSSL/GnuTLS cipher suites to the Schannel APIs, see 878 - Specifying Schannel Ciphers and Cipher Strengths 879 https://msdn.microsoft.com/en-us/library/windows/desktop/aa380161.aspx 880 88115.4 Add option to allow abrupt server closure 882 883 libcurl w/schannel will error without a known termination point from the 884 server (such as length of transfer, or SSL "close notify" alert) to prevent 885 against a truncation attack. Really old servers may neglect to send any 886 termination point. An option could be added to ignore such abrupt closures. 887 888 https://github.com/curl/curl/issues/4427 889 89016. SASL 891 89216.1 Other authentication mechanisms 893 894 Add support for other authentication mechanisms such as OLP, 895 GSS-SPNEGO and others. 896 89716.2 Add QOP support to GSSAPI authentication 898 899 Currently the GSSAPI authentication only supports the default QOP of auth 900 (Authentication), whilst Kerberos V5 supports both auth-int (Authentication 901 with integrity protection) and auth-conf (Authentication with integrity and 902 privacy protection). 903 904 90517. SSH protocols 906 90717.1 Multiplexing 908 909 SSH is a perfectly fine multiplexed protocols which would allow libcurl to do 910 multiple parallel transfers from the same host using the same connection, 911 much in the same spirit as HTTP/2 does. libcurl however does not take 912 advantage of that ability but will instead always create a new connection for 913 new transfers even if an existing connection already exists to the host. 914 915 To fix this, libcurl would have to detect an existing connection and "attach" 916 the new transfer to the existing one. 917 91817.2 Handle growing SFTP files 919 920 The SFTP code in libcurl checks the file size *before* a transfer starts and 921 then proceeds to transfer exactly that amount of data. If the remote file 922 grows while the transfer is in progress libcurl will not notice and will not 923 adapt. The OpenSSH SFTP command line tool does and libcurl could also just 924 attempt to download more to see if there is more to get... 925 926 https://github.com/curl/curl/issues/4344 927 92817.4 Support CURLOPT_PREQUOTE 929 930 The two other QUOTE options are supported for SFTP, but this was left out for 931 unknown reasons! 932 93317.5 SSH over HTTPS proxy with more backends 934 935 The SSH based protocols SFTP and SCP did not work over HTTPS proxy at 936 all until PR https://github.com/curl/curl/pull/6021 brought the 937 functionality with the libssh2 backend. Presumably, this support 938 can/could be added for the other backends as well. 939 94018. Command line tool 941 94218.1 sync 943 944 "curl --sync http://example.com/feed[1-100].rss" or 945 "curl --sync http://example.net/{index,calendar,history}.html" 946 947 Downloads a range or set of URLs using the remote name, but only if the 948 remote file is newer than the local file. A Last-Modified HTTP date header 949 should also be used to set the mod date on the downloaded file. 950 95118.2 glob posts 952 953 Globbing support for -d and -F, as in 'curl -d "name=foo[0-9]" URL'. 954 This is easily scripted though. 955 95618.3 prevent file overwriting 957 958 Add an option that prevents curl from overwriting existing local files. When 959 used, and there already is an existing file with the target file name 960 (either -O or -o), a number should be appended (and increased if already 961 existing). So that index.html becomes first index.html.1 and then 962 index.html.2 etc. 963 96418.4 --proxycommand 965 966 Allow the user to make curl run a command and use its stdio to make requests 967 and not do any network connection by itself. Example: 968 969 curl --proxycommand 'ssh pi@raspberrypi.local -W 10.1.1.75 80' \ 970 http://some/otherwise/unavailable/service.php 971 972 See https://github.com/curl/curl/issues/4941 973 97418.5 UTF-8 filenames in Content-Disposition 975 976 RFC 6266 documents how UTF-8 names can be passed to a client in the 977 Content-Disposition header, and curl does not support this. 978 979 https://github.com/curl/curl/issues/1888 980 98118.6 Option to make -Z merge lined based outputs on stdout 982 983 When a user requests multiple lined based files using -Z and sends them to 984 stdout, curl will not "merge" and send complete lines fine but may send 985 partial lines from several sources. 986 987 https://github.com/curl/curl/issues/5175 988 98918.7 at least N milliseconds between requests 990 991 Allow curl command lines issue a lot of request against services that limit 992 users to no more than N requests/second or similar. Could be implemented with 993 an option asking that at least a certain time has elapsed since the previous 994 request before the next one will be performed. Example: 995 996 $ curl "https://example.com/api?input=[1-1000]" -d yadayada --after 500 997 998 See https://github.com/curl/curl/issues/3920 999 100018.8 Consider convenience options for JSON and XML? 1001 1002 Could we add `--xml` or `--json` to add headers needed to call rest API: 1003 1004 `--xml` adds -H 'Content-Type: application/xml' -H "Accept: application/xml" and 1005 `--json` adds -H 'Content-Type: application/json' -H "Accept: application/json" 1006 1007 Setting Content-Type when doing a GET or any other method without a body 1008 would be a bit strange I think - so maybe only add CT for requests with body? 1009 Maybe plain `--xml` and ` --json` are a bit too brief and generic. Maybe 1010 `--http-json` etc? 1011 1012 See https://github.com/curl/curl/issues/5203 1013 101418.9 Choose the name of file in braces for complex URLs 1015 1016 When using braces to download a list of URLs and you use complicated names 1017 in the list of alternatives, it could be handy to allow curl to use other 1018 names when saving. 1019 1020 Consider a way to offer that. Possibly like 1021 {partURL1:name1,partURL2:name2,partURL3:name3} where the name following the 1022 colon is the output name. 1023 1024 See https://github.com/curl/curl/issues/221 1025 102618.10 improve how curl works in a windows console window 1027 1028 If you pull the scrollbar when transferring with curl in a Windows console 1029 window, the transfer is interrupted and can get disconnected. This can 1030 probably be improved. See https://github.com/curl/curl/issues/322 1031 103218.11 Windows: set attribute 'archive' for completed downloads 1033 1034 The archive bit (FILE_ATTRIBUTE_ARCHIVE, 0x20) separates files that shall be 1035 backed up from those that are either not ready or have not changed. 1036 1037 Downloads in progress are neither ready to be backed up, nor should they be 1038 opened by a different process. Only after a download has been completed it's 1039 sensible to include it in any integer snapshot or backup of the system. 1040 1041 See https://github.com/curl/curl/issues/3354 1042 104318.12 keep running, read instructions from pipe/socket 1044 1045 Provide an option that makes curl not exit after the last URL (or even work 1046 without a given URL), and then make it read instructions passed on a pipe or 1047 over a socket to make further instructions so that a second subsequent curl 1048 invoke can talk to the still running instance and ask for transfers to get 1049 done, and thus maintain its connection pool, DNS cache and more. 1050 105118.13 Ratelimit or wait between serial requests 1052 1053 Consider a command line option that can make curl do multiple serial requests 1054 slow, potentially with a (random) wait between transfers. There's also a 1055 proposed set of standard HTTP headers to let servers let the client adapt to 1056 its rate limits: 1057 https://www.ietf.org/id/draft-polli-ratelimit-headers-02.html 1058 1059 See https://github.com/curl/curl/issues/5406 1060 106118.14 --dry-run 1062 1063 A command line option that makes curl show exactly what it would do and send 1064 if it would run for real. 1065 1066 See https://github.com/curl/curl/issues/5426 1067 106818.15 --retry should resume 1069 1070 When --retry is used and curl actually retries transfer, it should use the 1071 already transferred data and do a resumed transfer for the rest (when 1072 possible) so that it does not have to transfer the same data again that was 1073 already transferred before the retry. 1074 1075 See https://github.com/curl/curl/issues/1084 1076 107718.16 send only part of --data 1078 1079 When the user only wants to send a small piece of the data provided with 1080 --data or --data-binary, like when that data is a huge file, consider a way 1081 to specify that curl should only send a piece of that. One suggested syntax 1082 would be: "--data-binary @largefile.zip!1073741823-2147483647". 1083 1084 See https://github.com/curl/curl/issues/1200 1085 108618.17 consider file name from the redirected URL with -O ? 1087 1088 When a user gives a URL and uses -O, and curl follows a redirect to a new 1089 URL, the file name is not extracted and used from the newly redirected-to URL 1090 even if the new URL may have a much more sensible file name. 1091 1092 This is clearly documented and helps for security since there's no surprise 1093 to users which file name that might get overwritten. But maybe a new option 1094 could allow for this or maybe -J should imply such a treatment as well as -J 1095 already allows for the server to decide what file name to use so it already 1096 provides the "may overwrite any file" risk. 1097 1098 This is extra tricky if the original URL has no file name part at all since 1099 then the current code path will error out with an error message, and we cannot 1100 *know* already at that point if curl will be redirected to a URL that has a 1101 file name... 1102 1103 See https://github.com/curl/curl/issues/1241 1104 110518.18 retry on network is unreachable 1106 1107 The --retry option retries transfers on "transient failures". We later added 1108 --retry-connrefused to also retry for "connection refused" errors. 1109 1110 Suggestions have been brought to also allow retry on "network is unreachable" 1111 errors and while totally reasonable, maybe we should consider a way to make 1112 this more configurable than to add a new option for every new error people 1113 want to retry for? 1114 1115 https://github.com/curl/curl/issues/1603 1116 111718.19 expand ~/ in config files 1118 1119 For example .curlrc could benefit from being able to do this. 1120 1121 See https://github.com/curl/curl/issues/2317 1122 112318.20 host name sections in config files 1124 1125 config files would be more powerful if they could set different 1126 configurations depending on used URLs, host name or possibly origin. Then a 1127 default .curlrc could a specific user-agent only when doing requests against 1128 a certain site. 1129 113018.21 retry on the redirected-to URL 1131 1132 When curl is told to --retry a failed transfer and follows redirects, it 1133 might get a HTTP 429 response from the redirected-to URL and not the original 1134 one, which then could make curl decide to rather retry the transfer on that 1135 URL only instead of the original operation to the original URL. 1136 1137 Perhaps extra emphasized if the original transfer is a large POST that 1138 redirects to a separate GET, and that GET is what gets the 529 1139 1140 See https://github.com/curl/curl/issues/5462 1141 114218.23 Set the modification date on an uploaded file 1143 1144 For SFTP and possibly FTP, curl could offer an option to set the 1145 modification time for the uploaded file. 1146 1147 See https://github.com/curl/curl/issues/5768 1148 114918.24 Use multiple parallel transfers for a single download 1150 1151 To enhance transfer speed, downloading a single URL can be split up into 1152 multiple separate range downloads that get combined into a single final 1153 result. 1154 1155 An ideal implementation would not use a specified number of parallel 1156 transfers, but curl could: 1157 - First start getting the full file as transfer A 1158 - If after N seconds have passed and the transfer is expected to continue for 1159 M seconds or more, add a new transfer (B) that asks for the second half of 1160 A's content (and stop A at the middle). 1161 - If splitting up the work improves the transfer rate, it could then be done 1162 again. Then again, etc up to a limit. 1163 1164 This way, if transfer B fails (because Range: is not supported) it will let 1165 transfer A remain the single one. N and M could be set to some sensible 1166 defaults. 1167 1168 See https://github.com/curl/curl/issues/5774 1169 117018.25 Prevent terminal injection when writing to terminal 1171 1172 curl could offer an option to make escape sequence either non-functional or 1173 avoid cursor moves or similar to reduce the risk of a user getting tricked by 1174 clever tricks. 1175 1176 See https://github.com/curl/curl/issues/6150 1177 117818.26 Custom progress meter update interval 1179 1180 Users who are for example doing large downloads in CI or remote setups might 1181 want the occasional progress meter update to see that the transfer is 1182 progressing and has not stuck, but they may not appreciate the 1183 many-times-a-second frequency curl can end up doing it with now. 1184 118519. Build 1186 118719.1 roffit 1188 1189 Consider extending 'roffit' to produce decent ASCII output, and use that 1190 instead of (g)nroff when building src/tool_hugehelp.c 1191 119219.2 Enable PIE and RELRO by default 1193 1194 Especially when having programs that execute curl via the command line, PIE 1195 renders the exploitation of memory corruption vulnerabilities a lot more 1196 difficult. This can be attributed to the additional information leaks being 1197 required to conduct a successful attack. RELRO, on the other hand, masks 1198 different binary sections like the GOT as read-only and thus kills a handful 1199 of techniques that come in handy when attackers are able to arbitrarily 1200 overwrite memory. A few tests showed that enabling these features had close 1201 to no impact, neither on the performance nor on the general functionality of 1202 curl. 1203 120419.3 Do not use GNU libtool on OpenBSD 1205 When compiling curl on OpenBSD with "--enable-debug" it will give linking 1206 errors when you use GNU libtool. This can be fixed by using the libtool 1207 provided by OpenBSD itself. However for this the user always needs to invoke 1208 make with "LIBTOOL=/usr/bin/libtool". It would be nice if the script could 1209 have some magic to detect if this system is an OpenBSD host and then use the 1210 OpenBSD libtool instead. 1211 1212 See https://github.com/curl/curl/issues/5862 1213 121419.4 Package curl for Windows in a signed installer 1215 1216 See https://github.com/curl/curl/issues/5424 1217 121819.5 make configure use --cache-file more and better 1219 1220 The configure script can be improved to cache more values so that repeated 1221 invokes run much faster. 1222 1223 See https://github.com/curl/curl/issues/7753 1224 122520. Test suite 1226 122720.1 SSL tunnel 1228 1229 Make our own version of stunnel for simple port forwarding to enable HTTPS 1230 and FTP-SSL tests without the stunnel dependency, and it could allow us to 1231 provide test tools built with either OpenSSL or GnuTLS 1232 123320.2 nicer lacking perl message 1234 1235 If perl was not found by the configure script, do not attempt to run the tests 1236 but explain something nice why it does not. 1237 123820.3 more protocols supported 1239 1240 Extend the test suite to include more protocols. The telnet could just do FTP 1241 or http operations (for which we have test servers). 1242 124320.4 more platforms supported 1244 1245 Make the test suite work on more platforms. OpenBSD and Mac OS. Remove 1246 fork()s and it should become even more portable. 1247 124820.5 Add support for concurrent connections 1249 1250 Tests 836, 882 and 938 were designed to verify that separate connections 1251 are not used when using different login credentials in protocols that 1252 should not re-use a connection under such circumstances. 1253 1254 Unfortunately, ftpserver.pl does not appear to support multiple concurrent 1255 connections. The read while() loop seems to loop until it receives a 1256 disconnect from the client, where it then enters the waiting for connections 1257 loop. When the client opens a second connection to the server, the first 1258 connection has not been dropped (unless it has been forced - which we 1259 should not do in these tests) and thus the wait for connections loop is never 1260 entered to receive the second connection. 1261 126220.6 Use the RFC6265 test suite 1263 1264 A test suite made for HTTP cookies (RFC 6265) by Adam Barth is available at 1265 https://github.com/abarth/http-state/tree/master/tests 1266 1267 It'd be really awesome if someone would write a script/setup that would run 1268 curl with that test suite and detect deviances. Ideally, that would even be 1269 incorporated into our regular test suite. 1270 127120.7 Support LD_PRELOAD on macOS 1272 1273 LD_RELOAD does not work on macOS, but there are tests which require it to run 1274 properly. Look into making the preload support in runtests.pl portable such 1275 that it uses DYLD_INSERT_LIBRARIES on macOS. 1276 127720.8 Run web-platform-tests url tests 1278 1279 Run web-platform-tests url tests and compare results with browsers on wpt.fyi 1280 1281 It would help us find issues to fix and help us document where our parser 1282 differs from the WHATWG URL spec parsers. 1283 1284 See https://github.com/curl/curl/issues/4477 1285 128620.9 Bring back libssh tests on Travis 1287 1288 In https://github.com/curl/curl/pull/7012 we remove the libssh builds and 1289 tests from Travis CI due to them not working. This should be remedied and 1290 libssh builds be brought back. 1291 1292 129321. MQTT 1294 129521.1 Support rate-limiting 1296 1297 The rate-limiting logic is done in the PERFORMING state in multi.c but MQTT 1298 is not (yet) implemented to use that! 1299