12021-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 2 3 NEWS: Prepare for release v2.0.0 4 52021-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 6 7 * docs/announce_2.0.0.txt: New file 8 92021-09-12 Tim Rühsen <tim.ruehsen@gmx.de> 10 11 * configure.ac: Update versions 12 132021-08-07 Jeffrey Marius <getjff@gmail.com> 14 15 Fix issue #467 Adding wiki as submodule to repo 16 17 * bootstrap.conf: Add clone_wiki() and call it from 18 bootstrap_post_import_hook(). 19 20 Signed-off-by: Jeffrey Marius <getjff@gmail.com> 21 222021-08-08 Tim Rühsen <tim.ruehsen@gmx.de> 23 24 * tests/test--page-requisites.c: Add tests for new rel parsing 25 262021-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 27 28 Fix the rel attribute handling for link tags 29 302021-07-07 Michael Roosz <michael.roosz@check24.de> 31 32 * libwget/http.c (wget_http_response_cb): Fix chunked transfer 33 342020-11-22 Sergei Litvin <litvindev@gmail.com> 35 36 Fix parsing comments in <style> content 37 38 * libwget/xml.c: Add new function getStyleContent(), 39 (parseXML): Use getStyleContent() instead of getContent(), 40 update link in comment. 41 422021-05-30 Tim Rühsen <tim.ruehsen@gmx.de> 43 44 * fuzz/libwget_html_url_fuzzer.dict: Add 'data:' to dictionary 45 462021-05-30 Michael Roosz <michael.roosz@check24.de> 47 48 * libwget/html_url.c (html_get_url): Support data: URL in srcset attribute 49 502021-05-30 Tim Rühsen <tim.ruehsen@gmx.de> 51 52 * docs/wget2.md: Mention that -P creates the prefix directory 53 542021-05-16 Tim Rühsen <tim.ruehsen@gmx.de> 55 56 * unit-tests/test.c: Free memory in test_parse_header_line() 57 582021-05-16 Tim Rühsen <tim.ruehsen@gmx.de> 59 60 Fix request path escaping 61 62 * include/wget/wget.h: Remove function wget_iri_isunreserved_path(). 63 * libwget/iri.c: Remove function wget_iri_isunreserved_path(). 64 (iri_ctype): Extend array with unreserved characters. 65 (wget_iri_isunreserved): Simplify code. 66 (wget_iri_escape): Use macro iri_isunreserved instead of wget_iri_isunreserved(). 67 (wget_iri_escape_path): Add RFC links to function comment, 68 Fix check whether char needs percent-encoding or not, 69 Use macro iri_isunreserved instead of wget_iri_isunreserved(). 70 712021-03-27 Tim Rühsen <tim.ruehsen@gmx.de> 72 73 * libwget/http_parse.c: Remove old comment from RFC 2183. 74 752021-03-27 Tim Rühsen <tim.ruehsen@gmx.de> 76 77 Add test case for Content-Disposition 78 79 * libwget/http_parse.c: Update comment with BNF of RFC 6266. 80 * unit-tests/test.c: Add test for a filename containing space. 81 822021-02-13 Tim Rühsen <tim.ruehsen@gmx.de> 83 84 * .gitlab-ci.yml: Drop manywarnings and SSL support on FreeBSD CI 85 862021-02-08 Tim Rühsen <tim.ruehsen@gmx.de> 87 88 * libwget/ssl_openssl.c: Include openssl/asn1.h for ASN1* functions. 89 902021-02-07 Michael Roosz <michael.roosz@check24.de> 91 92 * src/wget.c (prepare_file): Skip file name filter if config.filter_urls is active 93 942021-02-07 Tim Rühsen <tim.ruehsen@gmx.de> 95 96 Fix building without TLS library 97 98 This changes configure semantics to fail if no TLS library 99 is found and no --with-ssl=none was explicitly requested. 100 It protects users from accidentally building without TLS. 101 102 * .gitlab-ci.yml: Tarball build using --with-ssl=none. 103 * configure.ac: Stop with error if no TLS was found and --with-ssl 104 was not used. 105 * tests/libtest.c: Fix issues with conditionally declared variables. 106 1072021-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 108 109 * .gitlab-ci.yml: Allow failure for FreeBSD runner 110 1112021-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 112 113 * libwget/ssl_openssl.c: Fix for openssl/clang 114 1152021-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 116 117 * tests/test-ocsp-server.c: Skip cert revocation check for WolfSSL 118 1192021-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 120 121 * tests/test-post-handshake-auth.c (main.c): Fix warning when building with non-gnutls libraries 122 1232021-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 124 125 Separate GnuTLS in wget2 and in test suite 126 127 * configure.ac: Separate GnuTLS in wget2 and in test suite. 128 * tests/libtest.c: Use WITH_GNUTLS_OCSP insteadof WITH_OCSP. 129 1302021-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 131 132 * configure.ac: Improve searching for SSL/TLS libraries 133 1342021-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 135 136 * gnulib: Update 137 1382021-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 139 140 * libwget/ssl_openssl.c (get_tls_version): Fix error in case construct for LibreSSL 141 1422021-01-31 Tim Rühsen <tim.ruehsen@gmx.de> 143 144 * tests/test-chunked.c: Extend test with hand-crafted chunked pattern 145 1462021-01-31 Tim Rühsen <tim.ruehsen@gmx.de> 147 148 * libwget/http.c (wget_http_get_response_cb): Remove superfluous space in debug msg 149 1502021-01-24 Tim Rühsen <tim.ruehsen@gmx.de> 151 152 * libwget/hsts.c (hsts_db_add_entry): Fix possible NULL access 153 1542021-01-24 Michael Roosz <michael.roosz@check24.de> 155 156 * libwget/http.c (wget_http_get_response_cb): Fix NULL ptr access 157 1582021-01-24 Michael Roosz <michael.roosz@check24.de> 159 160 * libwget/http.c (wget_http_response): Fix buffer overflow 161 1622021-01-24 Tim Rühsen <tim.ruehsen@gmx.de> 163 164 * configure.ac: Detect unresolved AX_ macros 165 1662021-01-24 Tim Rühsen <tim.ruehsen@gmx.de> 167 168 * configure.ac: Use noyywrap for AC_PROG_LEX 169 1702020-12-29 Michael Roosz <michael.roosz@check24.de> 171 172 Fixed error log for non-fatal inflate() return code Z_BUF_ERROR 173 174 * libwget/decompressor.c (gzip_decompress): Return 0 for Z_BUF_ERROR. 175 1762021-01-10 Tim Rühsen <tim.ruehsen@gmx.de> 177 178 * configure.ac: Fix for autoconf 2.70 179 1802021-01-10 Tim Rühsen <tim.ruehsen@gmx.de> 181 182 * bootstrap: Update from gnulib 183 1842021-01-10 Tim Rühsen <tim.ruehsen@gmx.de> 185 186 * gnulib: Update 187 1882021-01-22 Darshit Shah <darnir@gnu.org> 189 190 Update copyright year 191 1922021-01-03 Darshit Shah <git@darnir.net> 193 194 Update POTFILES.in 195 1962021-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 197 198 * AUTHORS: Add Michael Heerklotz [skip ci] 199 2002021-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 201 202 * AUTHORS: Add Michael Roosz [skip ci] 203 2042021-01-01 Tim Rühsen <tim.ruehsen@gmx.de> 205 206 * .travis.yml: Remove unneeded packages for Linux 207 2082021-01-01 Tim Rühsen <tim.ruehsen@gmx.de> 209 210 * configure.ac: Small cleanups 211 2122021-01-01 Tim Rühsen <tim.ruehsen@gmx.de> 213 214 * configure.ac: Remove superfluous 'x' ahead of variables 215 2162020-12-27 Tim Rühsen <tim.ruehsen@gmx.de> 217 218 Fix combination of redirections with --retry-on-http-error 219 220 * src/wget.c: Move some code logic around. 221 * tests/test-interrupt-response.c: Correct expected wget2 exit status. 222 * tests/test-redirection.c: Add tests for --retry-on-http-error. 223 224 Reported-by: Sergei Litvin 225 2262020-12-27 Tim Rühsen <tim.ruehsen@gmx.de> 227 228 * src/options.c (print_version): Print https and ssl properly for OpenSSL and WolfSSL 229 2302020-12-27 Tim Rühsen <tim.ruehsen@gmx.de> 231 232 Reduce duplicated code in check_status_code_list(). 233 234 * src/wget.c (check_status_code_list): Call check_statuscode_list(). 235 2362020-12-27 Tim Rühsen <tim.ruehsen@gmx.de> 237 238 * src/wget.c (downloader_thread): Add check_statuscode_list() for readability 239 2402020-12-27 Tim Rühsen <tim.ruehsen@gmx.de> 241 242 Rename config.http_retry_on_error to config.retry_on_http_error 243 244 * src/options.c (options,deinit): Rename. 245 * src/wget.c (downloader_thread): Rename. 246 * src/wget_options.h (struct config): Rename. 247 2482020-12-27 Tim Rühsen <tim.ruehsen@gmx.de> 249 250 * tests/libtest.c (wget_test): Add --tries=1 to wget2 command line 251 2522020-12-25 Tim Rühsen <tim.ruehsen@gmx.de> 253 254 * AUTHORS: Add Sergei Litvin 255 2562020-12-25 Tim Rühsen <tim.ruehsen@gmx.de> 257 258 * tests/libtest.c (wget_test_start_server): Fix warning 'break will never be executed' 259 2602020-12-25 Tim Rühsen <tim.ruehsen@gmx.de> 261 262 * tests/libtest.c (wget_test_start_server): Fix unused variable warning 263 2642020-12-25 Tim Rühsen <tim.ruehsen@gmx.de> 265 266 * configure.ac: Use AC_CONFIG_MACRO_DIR only once 267 268 This fixes #547 (autotools 2.70 is more restrictive). 269 Reported by: Gabriele Balducci 270 2712020-12-20 Tim Rühsen <tim.ruehsen@gmx.de> 272 273 Fix clang 11 warnings in WolfSSL code. 274 275 * libwget/hashfile.c (wget_hash_fast): Cast size_t to unsigned int. 276 (wget_hash): Likewise. 277 * libwget/ssl_wolfssl.c (ShowX509): Use XFREE without trailing semicolon. 278 (wget_ssl_open): Use 64bit to 32bit casts to pacify clang 11. 279 2802020-12-20 Tim Rühsen <tim.ruehsen@gmx.de> 281 282 Fix findings from scan-build 11 283 284 * libwget/dns.c (wget_dns_resolve): 'rc' was set but not used. 285 * libwget/http.c (wget_http_get_response_cb): 'rc' was set but not used. 286 * libwget/net.c (wget_tcp_connect): 'rc' was set but not used. 287 * libwget/ssl_gnutls.c (send_ocsp_request): 'rc' was set but not used. 288 2892020-12-20 Tim Rühsen <tim.ruehsen@gmx.de> 290 291 Fix findings from scan-build 11 292 293 * examples/batch_loader.c (_normalize_location): 'rc' was set but not used. 294 * examples/check_url_types.c (_normalize_location): 'rc' was set but not used. 295 2962020-12-20 Tim Rühsen <tim.ruehsen@gmx.de> 297 298 Fix findings from scan-build 11 299 300 * src/job.c (job_validate_file): 'rc' was set but not used. 301 * src/options.c (parse_proxy): Fix return value. 302 * src/wget.c (input_thread): 'rc' was set but not used. 303 3042020-12-20 Tim Rühsen <tim.ruehsen@gmx.de> 305 306 Fix findings from scan-build 11 307 308 * tests/libtest.c (_parse_hostname): Fix += into +. 309 (_answer_to_connection): Limit scope of body_len. 310 3112020-12-13 Tim Rühsen <tim.ruehsen@gmx.de> 312 313 * libwget/http_parse.c (wget_http_parse_response_header): Fix (harmless !?) UB 314 3152020-12-13 Tim Rühsen <tim.ruehsen@gmx.de> 316 317 * configure.ac: Add -Wno-implicit-int-float-conversion for clang >= 11 318 3192020-12-06 Tim Rühsen <tim.ruehsen@gmx.de> 320 321 Fix type warnings on 32bit systems 322 323 * libwget/http.c (wget_http_get_response_cb): Fix printf format from %lX to %zX. 324 * libwget/ssl_openssl.c (wget_ssl_deinit): Fix sesslen to size_t. 325 * src/stats_site.c (print_csv_entry): Use %lld and cast to long long. 326 3272020-10-31 Tim Rühsen <tim.ruehsen@gmx.de> 328 329 Test for bashisms in Debian Testing 330 3312020-10-29 Tim Rühsen <tim.ruehsen@gmx.de> 332 333 * configure.ac: Fix configure.ac bashisms 334 335 Reported-by: Brian Inglis 336 3372020-10-26 Tim Rühsen <tim.ruehsen@gmx.de> 338 339 * bootstrap.conf: Remove python as dependency 340 341 /usr/bin/python don't exist here any more. 'python2' and 'python3' do 342 exist. 343 3442020-10-26 Tim Rühsen <tim.ruehsen@gmx.de> 345 346 * gnulib: Update 347 3482020-10-25 Tim Rühsen <tim.ruehsen@gmx.de> 349 350 Fix contrib/mingw 351 3522020-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 353 354 * .travis.yml: libunistring0 -> libunistring 355 3562020-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 357 358 Use brew upgrade for Travis CI OSX 359 3602020-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 361 362 Update Travis CI to bionic 363 3642020-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 365 366 * .travis.sh: Output tail -1000 of test logs 367 3682020-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 369 370 * libwget/hashmap.c (wget_hashmap_iterator_next): Fix segfault 371 3722020-08-02 Tim Rühsen <tim.ruehsen@gmx.de> 373 374 Implement --download-attr=[strippath|usepath] 375 376 * docs/wget2.md: Amend docs. 377 * src/options.c: Add parse_download_attr(), 378 amend option --download-attr. 379 * src/wget.c (queue_url_from_remote): Strip path if 380 DOWNLOAD_ATTR_STRIPPATH is set. 381 * src/wget_options.h: Add enum download_attr_mode. 382 * tests/test-download-attr.c: Test strippath and usepath. 383 3842020-08-02 Tim Rühsen <tim.ruehsen@gmx.de> 385 386 * libwget/ssl_openssl.c: Fix indentation, remove leading _ from variable 387 3882020-06-06 Ander Juaristi <a@juaristi.eus> 389 390 Update OCSP responses and certs 391 392 tests/certs/ocsp/*: Fix the cert stack for the testing HTTPS+OCSP server. 393 3942020-06-06 Ander Juaristi <a@juaristi.eus> 395 396 libtest: send normal cert chain in OCSP stapling HTTPS server 397 398 * tests/libtest.c (_ocsp_stap_cert_callback): rename variable 399 'pcert' -> 'certs'. 400 (_http_server_start): send normal cert chain in responses. 401 4022020-05-23 Ander Juaristi <a@juaristi.eus> 403 404 OpenSSL: Enable test-ocsp-stap 405 406 This commit enables the test `test-ocsp-stap` 407 408 It was previously skipped for OpenSSL since OCSP stapling was not 409 yet implemented in OpenSSL. 410 411 This commit also modifies these tests to pass the --no-ocsp-date flag to 412 wget2. 413 414 * tests/test-ocsp-stap.c (main): remove ifdef clause; pass 415 --no-ocsp-date flag. 416 4172020-05-09 Ander Juaristi <a@juaristi.eus> 418 419 OpenSSL: Implement OCSP stapling 420 421 * libwget/ssl_openssl.c (ocsp_resp_cb): new function: stapled OCSP 422 response callback. 423 (check_ocsp_response): do not check OCSP nonce here. 424 (verify_ocsp): check OCSP nonce here. 425 (openssl_init): set stapled OCSP callback function, ocsp_resp_cb. 426 (openssl_open): send status_request extension in handshake. 427 * tests/libtest.c (_http_server_start): Fix callback macro 428 4292020-08-02 Tim Rühsen <tim.ruehsen@gmx.de> 430 431 * .gitlab-ci.yml: Create /usr/bin/python before bootstrap 432 4332020-08-02 Tim Rühsen <tim.ruehsen@gmx.de> 434 435 * gnulib: Update 436 4372020-07-25 Tim Rühsen <tim.ruehsen@gmx.de> 438 439 * tests/libtest.c: Fix MHD 0.97.1 ABI/API breakage 440 4412020-07-25 Tim Rühsen <tim.ruehsen@gmx.de> 442 443 * src/options.c (init): Fix spelling 444 4452020-06-23 Tim Rühsen <tim.ruehsen@gmx.de> 446 447 * src/options.c: Fix crash introduced in fa3c71c85 448 4492020-05-25 Tim Rühsen <tim.ruehsen@gmx.de> 450 451 * libwget/decompressor.c (zstd_decompress): Avoid calls to sink with 0 length 452 453 Calls to sink() with length == 0 made -S output the header several times when 454 Content-Encoding was zstd. 455 4562020-05-24 Tim Rühsen <tim.ruehsen@gmx.de> 457 458 * libwget/ssl_gnutls.c (wget_ssl_init): Add debug msgs regarding system cert store 459 4602020-04-26 huangzsh7 <huangzsh7@gmail.com> 461 462 Delete obsolete code comment 463 464 * src/wget.c (test_modify_hsts): Delete obsolete code comment 465 4662020-04-22 Tim Rühsen <tim.ruehsen@gmx.de> 467 468 Remove CR and LF from URLs 469 470 * libwget/iri.c (iri_unescape_inline): Remove CR/LF from URL. 471 * tests/test-base.c: Add CR+LF into URL. 472 473 References 474 https://gitlab.com/gnuwget/wget2/-/issues/522 475 4762020-04-19 huangzsh7 <huangzsh7@gmail.com> 477 478 Create test cases 479 480 * tests/test-wget-1.c (main): Create test cases for --chunk-size with --progress 481 4822020-04-19 huangzsh7 <huangzsh7@gmail.com> 483 484 Add name in AUTHORS for CommitCheck 485 4862020-04-19 huangzsh7 <huangzsh7@gmail.com> 487 488 Fix segmentation faults when chunk-size and progress=bar are present 489 490 * src/wget.c (http_send_request): Keep the received response header when use chunk-size and progress=bar 491 4922020-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 493 494 Fix warning when converting void * to function pointer 495 496 * unit-tests/test-dl.c (test_fn_check): Add #pragma to suppress 497 -pedantic when converting void * to function pointer. 498 499 POSIX requires a conversion from 'void *' into a function pointer to work. 500 But -pedantic throws 'ISO C forbids conversion of object pointer to function 501 pointer type'. 502 5032020-04-11 Andreas Schwab <schwab@suse.de> 504 505 Fix strict aliasing violation 506 507 * unit-tests/test-dl.c (test_fn_check): Avoid strict aliasing 508 violation. 509 5102020-04-11 Andreas Schwab <schwab@suse.de> 511 512 Fix undefined behaviour in test_buffer_printf 513 514 * unit-tests/test.c (test_buffer_printf): Cast argument to printf 515 function to correct type. 516 5172020-03-25 Ander Juaristi <a@juaristi.eus> 518 519 test-ocsp-stap: Skip 520 521 OCSP stapling is not yet supported in OpenSSL. 522 523 * tests/test-ocsp-stap.c (main): skip this test in OpenSSL 524 5252020-01-12 Ander Juaristi <a@juaristi.eus> 526 527 OpenSSL: Update stats 528 529 * libwget/ssl_openssl.c (openssl_revocation_check_fn): update stats 530 (wget_ssl_open): update stats 531 5322019-09-16 Ander Juaristi <a@juaristi.eus> 533 534 OpenSSL: Add OCSP support 535 536 * libwget/ssl_gnutls.c: replace _config with config 537 * libwget/ssl_openssl.c (wget_ssl_set_config_int): new options 538 WGET_SSL_OCSP_NONCE and WGET_SSL_OCSP_DATE. 539 (openssl_revocation_check_fn): implement OCSP in the OpenSSL backend, 540 and update stats 541 (ssl_set_alpn_selected_protocol): update stats 542 (verify_hpkp): update stats 543 (wget_ssl_open): update stats 544 5452019-10-19 Ander Juaristi <a@juaristi.eus> 546 547 Include serverAuth extension in OCSP certificates 548 549 Since the OCSP tests use the same certificate chain for both the OCSP server and 550 the HTTPS server, the 'serverAuth' flags needs to be added, in addition to the OCSP flags. 551 552 OpenSSL will refuse to run a TLS handshake for HTTPS if that flag is not present. 553 554 * tests/certs/ocsp/demoCA/*, tests/certs/ocsp/*.pem: change testing certificates 555 * tests/certs/ocsp/ocsp_resp_revoked.der, 556 tests/certs/ocsp/ocsp_stapled_resp.der 557 tests/certs/ocsp/ocsp_resp_ok.der: update OCSP responses to contain 558 the new certificates 559 * tests/certs/ocsp/interm.cnf: add the serverAuth extension 560 * tests/certs/ocsp/root-template.txt: likewise 561 5622019-09-16 Ander Juaristi <a@juaristi.eus> 563 564 Adapt configure.ac to support multiple SSL/TLS backends 565 566 This patch modifies `configure.ac` to support GnuTLS, OpenSSL and wolfSSL as the SSL/TLS backends. 567 568 Either of these can be enabled at compile time the `configure` switch `--with-ssl`. Example: 569 570 ./configure --with-ssl=openssl 571 572 If `--with-ssl` is omitted, the default behavior will be to build with GnuTLS if available, and without 573 TLS if not. TLS can be explicitly disabled with `--without-ssl`. 574 575 Regardless of which specific backend has been selected, the TLS and OCSP tests will always be built with GnuTLS, 576 as that is the only backend currently supported in the test suite. 577 578 It renames the constant `HAVE_GNUTLS_OCSP_H` to a more generic `WITH_OCSP`, that captures 579 the notion that OCSP has been enabled at compile-time, hiding the specific backend. 580 581 Changed files 582 583 * configure.ac: honor --with-ssl option, but keep on testing for GnuTLS 584 for the test suite. 585 * libwget/ssl_gnutls.c: replace HAVE_GNUTLS_OCSP_H with WITH_OCSP 586 * libwget/ssl_openssl.c: likewise 587 * tests/libtest.c: replace HAVE_GNUTLS_OCSP_H with WITH_OCSP. 588 Check for WITH_GNUTLS_IN_TESTSUITE. 589 5902020-03-28 Tim Rühsen <tim.ruehsen@gmx.de> 591 592 * src/wget.c: Remove undefining FILE_SYSTEM_PREFIX_LEN 593 5942020-03-28 Tim Rühsen <tim.ruehsen@gmx.de> 595 596 * gnulib: Update 597 5982020-03-17 Tim Rühsen <tim.ruehsen@gmx.de> 599 600 * libwget/dns_cache.c (wget_dns_cache_add): Fix bitwise to logical OR (cppcheck) 601 6022020-03-17 Tim Rühsen <tim.ruehsen@gmx.de> 603 604 * include/wget/wget.h (wget_hsts_host_match_fn): Fix argument names (cppcheck) 605 6062020-03-17 Tim Rühsen <tim.ruehsen@gmx.de> 607 608 * src/wget.c: Work around redundant define in gnulib 609 6102020-03-17 Tim Rühsen <tim.ruehsen@gmx.de> 611 612 * libwget/net.c (struct _stats_data_t): Remove unused struct 613 6142020-03-17 Tim Rühsen <tim.ruehsen@gmx.de> 615 616 * libwget/metalink.c (add_mirror): Code cleanup 617 6182020-03-17 Tim Rühsen <tim.ruehsen@gmx.de> 619 620 * libwget/metalink.c (metalink_parse): Check return value of wget_strmemcpy_a 621 6222020-03-17 Tim Rühsen <tim.ruehsen@gmx.de> 623 624 * libwget/html_url.c (html_get_url): Check return value of wget_strmemcpy_a 625 6262020-03-17 Tim Rühsen <tim.ruehsen@gmx.de> 627 628 * libwget/net.c (wget_tcp_set_bind_address): Simplify code 629 6302020-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 631 632 * src/options.c (options): Fix indentation and alignment 633 6342020-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 635 636 Fix indentation for some tests 637 638 * tests/test-parse-css.c: Fix indentation. 639 * tests/test-parse-html-css.c: Likewise. 640 * tests/test-parse-rss.c: Likewise. 641 6422020-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 643 644 Add tests for --download-attr 645 646 * tests/Makefile.am: Add tests/test-download-attr.c. 647 * tests/test-download-attr.c: New file. 648 6492020-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 650 651 Add new option --download-attr 652 653 * src/options.c (options): Add new option --download-attr. 654 * src/wget.c (html_parse): Check for config.download_attr. 655 * src/wget_options.h (struct config): Add new member download_attr. 656 6572020-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 658 659 Implement HTML download attribute 660 661 * bootstrap.conf: Add gnulib module 'filename'. 662 * src/blacklist.c (get_local_filename_real): Remove const from return type. 663 (get_local_filename): Likewise. 664 * src/wget.c: Include "filename.h". 665 (queue_url_from_remote): Add param 'download_name'. 666 (queue_url_from_remote): Change local filename to that given by 'download' tag. 667 (html_parse): Create C string from 'download' value. 668 * src/wget_blacklist.h (blacklist_entry): Remove const from local_filename. 669 6702020-02-23 Tim Rühsen <tim.ruehsen@gmx.de> 671 672 Parse HTML download attribute 673 674 * examples/print_html_urls.c: Print download attribute. 675 * include/wget/wget.h (wget_html_parsed_url): Add download member. 676 * libwget/html_url.c (html_get_url): Parse download attribute. 677 6782020-03-15 Tim Rühsen <tim.ruehsen@gmx.de> 679 680 * libwget/http.c (wget_http_send_request): Fix unused variable without WITH_LIBNGHTTP2 681 6822020-02-26 Tim Rühsen <tim.ruehsen@gmx.de> 683 684 * libwget/http.c: Implement data upload for HTTP/2 685 6862020-02-25 Tim Rühsen <tim.ruehsen@gmx.de> 687 688 * libwget/printf.c (wget_vsnprintf): Fix size in call to wget_buffer_init 689 6902020-02-25 Tim Rühsen <tim.ruehsen@gmx.de> 691 692 * libwget/xml.c (parseXML): Remove use of VLA 693 6942020-02-25 Tim Rühsen <tim.ruehsen@gmx.de> 695 696 * libwget/ssl_wolfssl.c (wget_ssl_open): Safer code with wget_strmemcpy_a 697 6982020-02-25 Tim Rühsen <tim.ruehsen@gmx.de> 699 700 * libwget/printf.c (wget_printf): Optimize code 701 7022020-02-25 Tim Rühsen <tim.ruehsen@gmx.de> 703 704 * libwget/net.c (wget_tcp_set_bind_address): Safer code with wget_strmemcpy_a 705 7062020-02-25 Tim Rühsen <tim.ruehsen@gmx.de> 707 708 * libwget/metalink.c (add_mirror): Safer code with wget_strmemcpy_a 709 7102020-02-25 Tim Rühsen <tim.ruehsen@gmx.de> 711 712 * libwget/http.c (parse_proxies): Safer code with wget_strmemcpy 713 7142020-02-25 Tim Rühsen <tim.ruehsen@gmx.de> 715 716 * libwget/html_url.c (html_get_url): Simplify code with wget_strmemcpy_a 717 7182020-02-23 Tim Rühsen <tim.ruehsen@gmx.de> 719 720 * tests/valgrind-suppressions: Add GnuTLS suppression 721 7222020-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 723 724 * src/wget.c (sitemap_parse_text): Simplify code by using wget_strmemcpy_a 725 7262020-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 727 728 * src/options.c (struct optionw): Use fixed length for long_name 729 730 This reduces reallocations in the executable, reducing size by ~4k (stripped). 731 7322020-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 733 734 Add new function wget_strmemcpy_a 735 736 * include/wget/wget.h: Add prototype of wget_strmemcpy_a. 737 * src/mem.c: Implement wget_strmemcpy_a. 738 * libwget/http_parse.c (wget_http_parse_header_line): Use wget_strmemcpy_a. 739 * src/utils.c (mkdir_path): Likewise. 740 7412020-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 742 743 Let wget_strmemcpy return the number of copied bytes 744 745 * include/wget/wget.h: Set return type of wget_strmemcpy to size_t. 746 * libwget/mem.c (wget_strmemcpy): Return length of destination string. 747 * unit-tests/test.c: Amend tests of wget_strmemcpy. 748 7492020-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 750 751 * .gitlab-ci.yml (Scan-Build/Debian): Do not use security checker 752 753 The security checker complains about not using the C11 _s functions. 754 We don't want to use them, see also 755 http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1969.htm 756 7572019-12-09 Suhas K S <suhasks123@gmail.com> 758 759 Add option --bind-interface 760 761 * libwget/net.h (wget_tcp_st): New variable 762 const char *bind_interface. 763 * libwget/net.c: Add function wget_tcp_bind_interface(). 764 (set_socket_options): Add parameter wget_tcp *tcp and 765 implement binding of tcp to bind_interface. 766 * libwget/init.c (wget_global_init): Call wget_tcp_bind_interface(). 767 * include/wget/wget.h: Add wget_tcp_bind_interface() and 768 WGET_BIND_INTERFACE. 769 * src/options.c: Add --bind-interface handling. 770 (init): Call wget_tcp_bind_interface(). 771 * src/wget2_options.h (config): Add const char *bind_interface. 772 * docs/wget2.md: Add documentation for --bind-interface. 773 774 This finishes task #430 775 7762020-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 777 778 tests/valgrind-suppressions: Add GnuTLS suppression 779 7802020-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 781 782 Amend docs regarding -q -O- [skip ci] 783 7842020-02-16 Tim Rühsen <tim.ruehsen@gmx.de> 785 786 New configure switch --disable-manylibs 787 788 * NEWS: Add --disable-manylibs 789 * configure.ac: Implement --disable-manylibs 790 * libwget/Makefile.am: Likewise 791 792 This switches off building small libraries from libwget functionality groups. 793 7942019-08-22 Tim Rühsen <tim.ruehsen@gmx.de> 795 796 Generate library libwget_decompress 797 798 * libwget/Makefile.am: Add libwget_decompress.la 799 * libwget/test_linking_compress.c: New file 800 8012019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 802 803 Generate library libwget_netrc 804 805 * libwget/Makefile.am: Add libwget_netrc.la 806 * libwget/test_linking_netrc.c: New file 807 8082019-08-14 Tim Rühsen <tim.ruehsen@gmx.de> 809 810 Generate library libwget_metalink 811 812 * libwget/Makefile.am: Add libwget_metalink.la 813 * libwget/test_linking_metalink.c: New file 814 8152019-08-14 Tim Rühsen <tim.ruehsen@gmx.de> 816 817 Generate library libwget_xml 818 819 * libwget/Makefile.am: Add libwget_xml.la 820 * libwget/test_linking_xml.c: New file 821 8222019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 823 824 Generate library libwget_http_parse 825 826 * libwget/Makefile.am: Add libwget_http_parse.la 827 * libwget/test_linking_htp_parse.c: New file 828 8292019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 830 831 Generate library libwget_tls_session 832 833 * libwget/Makefile.am: Add libwget_tls_session.la 834 * libwget/test_linking_tls_session.c: New file 835 8362019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 837 838 Generate library libwget_ocsp 839 840 * libwget/Makefile.am: Add libwget_ocsp.la 841 * libwget/test_linking_ocsp.c: New file 842 8432019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 844 845 Generate library libwget_hsts 846 847 * libwget/Makefile.am: Add libwget_hsts.la 848 * libwget/test_linking_hsts.c: New file 849 8502019-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 851 852 Generate library libwget_robots 853 854 * libwget/Makefile.am: Add libwget_robots.la 855 8562019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 857 858 Generate library libwget_dns 859 860 * libwget/Makefile.am: Add libwget_dns.la 861 * libwget/test_linking_dns.c: New file 862 8632019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 864 865 Generate library libwget_dnscache 866 867 * libwget/Makefile.am: Add libwget_dnscache.la 868 * libwget/test_linking_dnscache.c: New file 869 8702019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 871 872 Generate library libwget_logger 873 874 * libwget/Makefile.am: Add libwget_logger.la 875 * libwget/test_linking_logger.c: New file 876 8772019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 878 879 Generate library libwget_hpkp_db 880 881 * libwget/Makefile.am: Add libwget_hpkp_db.la 882 * libwget/test_linking_hpkp_db.c: New file 883 8842019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 885 886 Generate library libwget_hashfile 887 888 * libwget/Makefile.am: Add libwget_hashfile.la 889 * libwget/test_linking_hashfile.c: New file 890 8912019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 892 893 Generate library libwget_io 894 895 * libwget/Makefile.am: Add libwget_io.la 896 * libwget/test_linking_io.c: New file 897 8982019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 899 900 Generate library libwget_progress 901 902 * libwget/Makefile.am: Add libwget_progress.la 903 * libwget/test_linking_progress.c: New file 904 9052019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 906 907 Generate library libwget_thread 908 909 * libwget/Makefile.am: Add libwget_thread.la 910 * libwget/test_linking_thread.c: New file 911 9122019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 913 914 Generate library libwget_css 915 916 * libwget/Makefile.am: Add libwget_css.la 917 * libwget/test_linking_css.c: New file 918 9192019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 920 921 Generate library libwget_iri 922 923 * libwget/Makefile.am: Add libwget_iri.la 924 * libwget/test_linking_iri.c: New file 925 9262019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 927 928 Generate library libwget_encoding 929 930 * libwget/Makefile.am: Add libwget_encoding.la 931 * libwget/test_linking_encoding.c: New file 932 9332019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 934 935 Generate library libwget_common 936 937 * libwget/Makefile.am: Add libwget_common.la 938 * libwget/test_linking_common.c: New file 939 9402019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 941 942 Generate library libwget_ip 943 944 * libwget/Makefile.am: Add libwget_ip.la 945 9462019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 947 948 Generate library libwget_alloc 949 950 * libwget/Makefile.am: Add libwget_alloc.la 951 9522019-05-16 Tim Rühsen <tim.ruehsen@gmx.de> 953 954 Patch libtool to add -no-whole-archive 955 956 * cfg.mk: Exclude contrib/libtool.patch from 'make syntax-check' 957 * configure.ac: Patch libtool 958 * contrib/libtool.patch: New file 959 * lib/Makefile.am: Unset $LIBS 960 * libwget/Makefile.am: Use -no-whole-archive 961 962 This patch keeps the libwget libraries short by not adding 963 *everything* from lib/libgnu.a. 964 9652020-02-12 Rohan Fletcher <rohfle@gmail.com> 966 967 Skip HTTP/2 testing for test-interrupt-response 968 969 * tests/test-interrupt-response.c: Set WGET_TEST_SKIP_H2 970 9712020-02-11 Tim Rühsen <tim.ruehsen@gmx.de> 972 973 * README.md: Fix badge for Build Status [skip ci] 974 9752020-02-11 Tim Rühsen <tim.ruehsen@gmx.de> 976 977 Build own badges for coverage and fuzz-coverage 978 979 * contrib/make-coverage-badge: Add script to create badges 980 * .gitlab-ci.yml: Create coverage and fuzz-covrage badges 981 * README.md: Display badges 982 9832020-02-11 Tim Rühsen <tim.ruehsen@gmx.de> 984 985 * fuzz/*_fuzer.in/*: Update OSS-Fuzz corpora from upstream 986 9872020-02-11 Tim Rühsen <tim.ruehsen@gmx.de> 988 989 * fuzz/Makefile.am: Remove -all-static from AM_LDFLAGS 990 9912020-02-11 Tim Rühsen <tim.ruehsen@gmx.de> 992 993 * fuzz/README.md: Update docs 994 9952020-02-09 Tim Rühsen <tim.ruehsen@gmx.de> 996 997 * .gitlab-ci.yml: Add 'linux' to 'tags:' 998 9992020-02-10 Tim Rühsen <tim.ruehsen@gmx.de> 1000 1001 * include/wget/wget.h: Make more use of 'restrict' keyword 1002 10032020-02-04 Tim Rühsen <tim.ruehsen@gmx.de> 1004 1005 * libwget/ssl_gnutls.c (safe_ctime): Suppress -Wformat-y2k around strftime %c 1006 10072020-01-31 Tim Rühsen <tim.ruehsen@gmx.de> 1008 1009 * src/wget.c (http_receive_response): Set mtime to last_modified-1 on partial download 1010 10112020-01-31 Tim Rühsen <tim.ruehsen@gmx.de> 1012 1013 * tests/test-interrupt-response.c: Fix for wget2, add one more test 1014 10152020-01-29 Rohan Fletcher <rohfle@gmail.com> 1016 1017 Added test simulations of aborts while sending response body 1018 1019 * tests/Makefile.am: add tests/test-interrupt-response test 1020 * tests/libtest.c (_callback_interruptable, _answer_to_connection): 1021 add interruptable response callback, enable in wget_test_url_t 1022 settings 1023 * tests/libtest.h: added interrupt_response_mode and 1024 interrupt_response_after_nbytes to wget_test_url_t 1025 * tests/test-interrupt-response.c: add test for interruptable 1026 responses and interrupt of responses while timestamping active 1027 10282020-01-31 Tim Rühsen <tim.ruehsen@gmx.de> 1029 1030 * tests/gpg/pubring.kbx: Update 1031 10322020-01-31 Tim Rühsen <tim.ruehsen@gmx.de> 1033 1034 * tests/libtest.c (wget_test): Reduce timeout from 10s to 3s 1035 10362020-01-31 Tim Rühsen <tim.ruehsen@gmx.de> 1037 1038 * tests/libtest.c (wget_test): Amend error output to be more informative / prominent 1039 10402020-01-31 Tim Rühsen <tim.ruehsen@gmx.de> 1041 1042 Amend retry strategy for job failures 1043 1044 This introduces retries and and pauses for JOBs. 1045 Formerly, we only had a host failure strategy. 1046 1047 * src/wget_job.h (struct JOB): Add retry_ts and failures 1048 * src/host.c (_search_queue_for_free_job): Check for pause after failure 1049 * src/wget.c (downloader_thread): Rearrange code for job retry strategy 1050 10512020-01-29 Tim Rühsen <tim.ruehsen@gmx.de> 1052 1053 * AUTHORS: Add Rohan Fletcher [skip ci] 1054 10552020-01-27 Suhas K S <suhasks123@gmail.com> 1056 1057 Add --background support for Windows 1058 1059 * src/wget.c: Add function make_section_name(), 1060 fake_fork_child(), fake_fork(), fork_to_background(). 1061 Add struct fake_fork_info. 1062 * docs/wget2.md: Remove [Not Supported on Windows yet] 1063 from --background. 1064 1065 Closes issue #457 1066 1067 Copyright-paperwork-exempt: Yes 1068 10692020-01-25 Tim Rühsen <tim.ruehsen@gmx.de> 1070 1071 * .gitlab-ci.yml: Use tag 'linux' instead 'docker' 1072 10732020-01-20 Tim Rühsen <tim.ruehsen@gmx.de> 1074 1075 Update gnulib to fix FreeBSD CI runner 1076 1077 * gnulib: Update 1078 * .gitlab-ci.yml: Remove --enable-threads=isoc from ./configure 1079 10802020-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 1081 1082 * gnulib: Update to fix use-after-free in glob() 1083 1084 This fixes OSS-Fuzz issue 20176. 1085 10862020-01-10 Darshit Shah <darnir@gnu.org> 1087 1088 * Update copyright year to 2020 1089 10902020-01-08 Tim Rühsen <tim.ruehsen@gmx.de> 1091 1092 * .gitlab-ci.-yml: Add --enable-threads=isoc to FreeBSD configure run 1093 10942020-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 1095 1096 * .gitlab-ci.yml: Do not use -Werror for ./configure run 1097 10982020-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 1099 1100 Update Makefile.am for latest gnulib 1101 1102 * examples/Makefile.am: Amend LDADD assignment 1103 * fuzz/Makefile.am: Likewise 1104 * libwget/Makefile.am: Likewise 1105 * src/Makefile.am: Likewise 1106 * tests/Makefile.am: Likewise 1107 * unit-tests/Makefile.am: Likwise 1108 11092020-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 1110 1111 Fix clang warning '-Wunreachable-code' in tests/ 1112 1113 * tests/test--https-enforce-hard3.c: Ignore warning via #pragma 1114 * tests/test--https-enforce-soft2.c: Likewise 1115 11162020-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 1117 1118 * src/wget.c: Suppress clang warning '-Watomic-implicit-seq-cst' 1119 11202020-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 1121 1122 * libwget/css_tokenizer.lex: Suppress clang warning '-Wextra-semi-stmt' 1123 11242020-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 1125 1126 * .gitlab-ci.yml: CFLAGS_DEFAULT: -O1 -g -ggdb3 1127 1128 The sanitizer does not work with -O0, changed to -O1. 1129 11302020-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 1131 1132 * .gitlab-ci.yml: Remove manual copying of gnulib directory 1133 11342020-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 1135 1136 * gnulib: Update 1137 11382020-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 1139 1140 Move sanitizer CI runner to Debian unstable 1141 11422020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1143 1144 * configure.ac: Enable more sanitizer options for UBSAN and ASAN 1145 11462020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1147 1148 * libwget/bitmap.c: Fix clang sanitizer's false positive (buffer overflow) 1149 11502020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1151 1152 * src/gpgme.c: Fix implicit conversion 1153 11542020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1155 1156 * tests/libtest.c: Fix implicit conversion 1157 11582020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1159 1160 * src/wget.c: Fix implicit conversion -1 to size_t 1161 11622020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1163 1164 * unit-tests/test.c: Fix implicit conversions -1 to size_t 1165 11662020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1167 1168 * libwget/buffer_printf: Fix implicit conversionsi (use unsigned flag values) 1169 11702020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1171 1172 * libwget/buffer_printf: Fix implicit conversions 1173 11742020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1175 1176 * libwget/http.c (wget_http_get_response_cb): Fix implicit conversion -1 to size_t 1177 11782020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1179 1180 * src/options.c (parse_numbytes): Fix 8.35133e+19 outside long long range 1181 11822020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1183 1184 * libwget/base64.c (wget_base64_decode): Fix implicit conversion 218 to char 1185 11862020-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 1187 1188 * libwget/iri.c (wget_iri_parse_base): Fix implicit conversion -1 to size_t 1189 11902019-12-19 Tim Rühsen <tim.ruehsen@gmx.de> 1191 1192 * src/options.c (parse_numbytes): Fix implicit conversion to double 1193 11942019-12-19 Tim Rühsen <tim.ruehsen@gmx.de> 1195 1196 * unit-tests/test.c (test_mem): Fix buffer overflow in test 1197 11982019-12-10 Tim Rühsen <tim.ruehsen@gmx.de> 1199 1200 * contrib/commit-check: Don't say 'not found' if found [skip ci] 1201 12022019-12-08 Tim Rühsen <tim.ruehsen@gmx.de> 1203 1204 * include/wget/wget.h: Fix old types in comments [skip ci] 1205 12062019-12-07 Tim Rühsen <tim.ruehsen@gmx.de> 1207 1208 * tests/test-cut-get-vars.c: Add three more test variations 1209 12102019-12-07 Tim Rühsen <tim.ruehsen@gmx.de> 1211 1212 Fix --cut-file-get-vars creating dirs on single file download 1213 1214 * tests/test-cut-get-vars.c: Amend to reproduce issue #490. 1215 * src/blacklist.c (get_local_filename_real): Generate basename 1216 instead of filename, without query. 1217 * include/wget/wget.h: Add flag WGET_IRI_WITH_QUERY. 1218 Add param 'flags' to wget_iri_get_basename. 1219 * libwget/iri.c: Add param 'flags' to wget_iri_get_basename. 1220 Implement flag WGET_IRI_WITH_QUERY. 1221 * fuzz/libwget_iri_fuzzer.c (test): Use WGET_IRI_WITH_QUERY. 1222 1223 Closes #490 1224 12252019-12-07 Tim Rühsen <tim.ruehsen@gmx.de> 1226 1227 * tests/Makefile.am: Add test-cut-get-vars 1228 12292019-12-07 Tim Rühsen <tim.ruehsen@gmx.de> 1230 1231 Rename wget_iri_get_filename -> wget_iri_get_basename 1232 1233 * fuzz/libwget_iri_fuzzer.c: Likewise. 1234 * include/wget/wget.h: Likewise. 1235 * libwget/iri.c: Likewise. 1236 * src/blacklist.c: Likewise. 1237 12382019-12-06 Tim Rühsen <tim.ruehsen@gmx.de> 1239 1240 Fix OpenSSL for Win32 1241 1242 * libwget/ssl_openssl.c (wget_ssl_open): Use FD_TO_SOCKET to 1243 convert POSIX fd to WIN32 socket. 1244 (ssl_transfer): Use SOCKET_TO_FD to convert WIN32 socket to POSIX fd. 1245 1246 Reported-by: Gisle Vanem 1247 12482019-12-03 Tim Rühsen <tim.ruehsen@gmx.de> 1249 1250 Code cleanup 1251 1252 * src/plugin.c (plugin_db_forward_downloaded_file): 1253 Change size from uint64_t to int64_t. 1254 * src/wget_plugin.h (plugin_db_forward_downloaded_file): 1255 Likewise. 1256 * src/wget.c (process_response): Likewise. 1257 Remove n_recurse_iris. 1258 12592019-12-03 Tim Rühsen <tim.ruehsen@gmx.de> 1260 1261 Add two new tests for -c 1262 1263 * tests/test-wget-1.c: Test -c with existing empty file. 1264 Test -c with existing complete file. 1265 12662019-12-03 TheAlakazam <piyushjaipuriyar@gmail.com> 1267 1268 Fix -c with existing zero-length file 1269 1270 * src/wget.c (prepare_file): Replace `> 0` by `>= 0`. 1271 (wget_http_request): Likewise. 1272 1273 Copyright-paperwork-exempt: Yes 1274 12752019-12-01 Tim Rühsen <tim.ruehsen@gmx.de> 1276 1277 Remove usage of strerror_r due to diverting GNU/POSIX behavior 1278 1279 * libwget/http_highlevel.c: Likewise 1280 * libwget/net.c: Likewise 1281 * src/wget.c: Likewise 1282 12832019-11-30 Tim Rühsen <tim.ruehsen@gmx.de> 1284 1285 * gnulib: Update 1286 1287 Fixes #492 (library -lnsl-lintl: not found) 1288 12892019-11-21 Tim Rühsen <tim.ruehsen@gmx.de> 1290 1291 Replace strerror() by strerror_r() in multi-threaded code 1292 1293 * libwget/http_highlevel.c: Likewise 1294 * libwget/net.c: Likewise 1295 * src/wget.c: Likewise 1296 12972019-11-15 Tim Rühsen <tim.ruehsen@gmx.de> 1298 1299 * configure.ac: Fix for 'Error on unknown value for --with-ssl' 1300 13012019-11-15 Tim Rühsen <tim.ruehsen@gmx.de> 1302 1303 Fix for LibreSSL 1304 1305 * libwget/ssl_openssl.c: Some small fixes for LibreSSL 1306 1307 For LibreSSL you need to './configure --with-ssl=openssl'. 1308 13092019-11-15 Tim Rühsen <tim.ruehsen@gmx.de> 1310 1311 * configure.ac: Error on unknown value for --with-ssl 1312 13132019-11-13 Tim Rühsen <tim.ruehsen@gmx.de> 1314 1315 * NEWS: Update 1316 13172019-11-05 Darshit Shah <darnir@gnu.org> 1318 1319 Support terminal hyperlinks in output 1320 1321 Some terminal emulators now allow printing hyperlinks through escape 1322 sequences. See 1323 https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda for 1324 more information. 1325 1326 Hyperlink support cannot be gauged accurately. And while most terminals 1327 just ignore the extra escape sequences, older versions of vte actually 1328 print the URI along with some garbage. So, in this patch, the hyperlink 1329 support is disabled by default and used only if the user explicitly 1330 requests it. Once there is a way to query for feature support in 1331 terminals, this can be intelligently enabled automatically. 1332 1333 * bootstrap.conf: Add new module xgethostname 1334 * src/wget_options.h: Add a new option hyperlink and a global variable 1335 hostname 1336 * src/options.c: Add a new command line option --hyperlink. This is the 1337 same option name as `ls` from coreutils for consistency across 1338 programs 1339 (init): If the user requests hyperlink support, get the hostname and 1340 store it. This is to prevent having to get the hostname for each 1341 downloaded file 1342 * src/wget.c (prepare_file): Print the filename with a hyperlink if the 1343 user requests it. 1344 * docs/wget2.md: Add documentation entry for hyperlinks 1345 13462019-10-28 Archit Pandey <archpndy@gmail.com> 1347 1348 Added --convert-file-only handling 1349 1350 * src/options.c: Add --convert-file-only handling 1351 * src/wget_options.h (config): Add bool convert_file_only 1352 * src/wget.c (html_parse): conversions initialized when 1353 convert_file_only is set 1354 (main): Call convert_links when config.convert_file_only 1355 (convert_links): move conversion logic to convert_link_whole and 1356 convert_link_file_only 1357 New function convert_link_whole: convert whole URL 1358 New function convert_link_file_only: convert filename part of URLs 1359 * tests/test-convert-file-only.c: testcase for --convert-file-only 1360 with -E 1361 * tests/Makefile.am: add test-convert-file-only object 1362 * docs/wget2.md: Removed [unimplemented-option] from 1363 --convert-file-only 1364 13652019-11-06 Tim Rühsen <tim.ruehsen@gmx.de> 1366 1367 * configure.ac: Fix 1368 13692019-11-06 Tim Rühsen <tim.ruehsen@gmx.de> 1370 1371 * tests/libtest.c (wget_test): Fix output, make use of wget_read_file() 1372 13732019-11-02 Archit Pandey <archpndy@gmail.com> 1374 1375 Add --ignore-length handling 1376 1377 * include/wget/wget.h (wget_http_request): Add bool 1378 response_ignorelength 1379 (wget_http_response): Add bool length_inconsistent 1380 * src/options.c: Add --ignore-length handling 1381 * src/wget_options.h (config): Add bool ignore_length 1382 (exit_status_e): Add WGET_EXIT_INCONSISTENT_LENGTH 1383 * libwget/http.c (wget_http_response_cb): Set length_inconsistent 1384 when mismatch with Content-Length 1385 * src/wget.c (wget_send_request): Set ignore-length in request 1386 * tests/test-ignore-length.c: test --ignore-length 1387 * tests/Makefile.am: add test-ignore-length 1388 * docs/wget2.md: Removed [unimplemented-option] from 1389 --ignore-length 1390 13912019-10-20 Darshit Shah <darnir@gnu.org> 1392 1393 src/libwget.doxy.in: Update doxygen file 1394 13952019-10-20 Darshit Shah <darnir@gnu.org> 1396 1397 Explicitly set the FILE_PATTERNS for doxygen 1398 1399 * docs/libwget.doxy.in: Set FILE_PATTERNS to workaround bug in doxygen 1400 1.8.16 1401 1402 Fix-by: Tim Ruehsen <rockdaboot@gmail.com> 1403 14042019-10-03 Darshit Shah <darnir@gnu.org> 1405 1406 Add license text to files 1407 14082019-09-13 Darshit Shah <darnir@gnu.org> 1409 1410 Fix license of all tests 1411 1412 All the tests in tests/ are part of Wget2 the binary, not libwget. Fix 1413 the license and copyright information accordingly. 1414 1415 * tests/libtest.c: libwget -> Wget. LGPLv3+ -> GPLv3+ 1416 * tests/test--accept.c: Same 1417 * tests/test--filter-mime-type.c: Same 1418 * tests/test--follow-tags.c: Same 1419 * tests/test--https-enforce-hard1.c: Same 1420 * tests/test--https-enforce-hard2.c: Same 1421 * tests/test--https-enforce-hard3.c: Same 1422 * tests/test--https-enforce-soft1.c: Same 1423 * tests/test--https-enforce-soft2.c: Same 1424 * tests/test--https-enforce-soft3.c: Same 1425 * tests/test--page-requisites.c: Same 1426 * tests/test--post-file.c: Same 1427 * tests/test--save-content-on.c: Same 1428 * tests/test--spider-r.c: Same 1429 * tests/test-E-k.c: Same 1430 * tests/test-auth-basic.c: Same 1431 * tests/test-auth-digest.c: Same 1432 * tests/test-bad-chunk.c: Same 1433 * tests/test-base.c: Same 1434 * tests/test-c-r.c: Same 1435 * tests/test-chunked.c: Same 1436 * tests/test-compression.c: Same 1437 * tests/test-cut-dirs.c: Same 1438 * tests/test-cut-get-vars.c: Same 1439 * tests/test-directory-clash.c: Same 1440 * tests/test-ftp.c: Same 1441 * tests/test-ftps.c: Same 1442 * tests/test-gpg-bad.c: Same 1443 * tests/test-gpg-invalid.c: Same 1444 * tests/test-gpg-missing.c: Same 1445 * tests/test-gpg-save-failed.c: Same 1446 * tests/test-gpg-styles.c: Same 1447 * tests/test-gpg-valid.c: Same 1448 * tests/test-gpg-verify-no-file.c: Same 1449 * tests/test-gzip.c: Same 1450 * tests/test-i-http.c: Same 1451 * tests/test-i-https.c: Same 1452 * tests/test-idn-cmd.c: Same 1453 * tests/test-idn-meta.c: Same 1454 * tests/test-idn-robots.c: Same 1455 * tests/test-include-and-exclude-directories.c: Same 1456 * tests/test-iri-disabled.c: Same 1457 * tests/test-iri-forced-remote.c: Same 1458 * tests/test-iri-list.c: Same 1459 * tests/test-iri-percent.c: Same 1460 * tests/test-iri-subdir.c: Same 1461 * tests/test-iri.c: Same 1462 * tests/test-k.c: Same 1463 * tests/test-limit-rate-http2.c: Same 1464 * tests/test-limit-rate.c: Same 1465 * tests/test-meta-robots.c: Same 1466 * tests/test-metalink.c: Same 1467 * tests/test-np.c: Same 1468 * tests/test-ocsp-server.c: Same 1469 * tests/test-ocsp-stap.c: Same 1470 * tests/test-p-nc.c: Same 1471 * tests/test-parse-css.c: Same 1472 * tests/test-parse-html-css.c: Same 1473 * tests/test-parse-rss.c: Same 1474 * tests/test-plugin-dummy.c: Same 1475 * tests/test-plugin-failure.c: Same 1476 * tests/test-plugin-interception.c: Same 1477 * tests/test-plugin-nonexistance.c: Same 1478 * tests/test-plugin.c: Same 1479 * tests/test-post-handhshake-auth.c: Same 1480 * tests/test-redirection.c: Same 1481 * tests/test-restrict-ascii.c: Same 1482 * tests/test-robots.c: Same 1483 * tests/test-stats-dns.c: Same 1484 * tests/test-stats-ocsp.c: Same 1485 * tests/test-stats-server.c: Same 1486 * tests/test-stats-site.c: Same 1487 * tests/test-stats-tls.c: Same 1488 * tests/test-timestamping.c: Same 1489 * tests/test-unlink.c: Same 1490 * tests/test-wget-1.c: Same 1491 14922019-09-13 Darshit Shah <darnir@gnu.org> 1493 1494 * README: Symlink to README.md 1495 14962019-09-13 Darshit Shah <darnir@gnu.org> 1497 1498 * README.md: Add license information 1499 15002019-10-29 Tim Rühsen <tim.ruehsen@gmx.de> 1501 1502 * include/wget/wget.h (wget_bar_slot_begin): Set 'filename' not NONNULL 1503 15042019-10-29 Tim Rühsen <tim.ruehsen@gmx.de> 1505 1506 * tests/libtest.c (_answer_to_connection): Switch off Content-Length sanity checks in MHD 1507 15082019-10-29 Tim Rühsen <tim.ruehsen@gmx.de> 1509 1510 * tests/libtest.c (wget_test_start_server): Print MHD info 1511 15122019-10-29 Tim Rühsen <tim.ruehsen@gmx.de> 1513 1514 * tests/libtest.c (_http_server_start): Fix checks for MHD options 1515 15162019-10-28 Tim Rühsen <tim.ruehsen@gmx.de> 1517 1518 * libwget/http.c (wget_http_send_request): Fix hard-coded GET for HTTP/2 1519 15202019-10-28 Tim Rühsen <tim.ruehsen@gmx.de> 1521 1522 * tests/libtest.c: Fix CI failure 1523 15242019-10-28 Tim Rühsen <tim.ruehsen@gmx.de> 1525 1526 * tests/libtest.c (_http_server_start): Make use of MHD_OPTION_SERVER_INSANITY 1527 15282019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1529 1530 Add wget_test_set_executable() to test suite 1531 1532 * tests/libtest.h: Add declaration of wget_test_set_executable() 1533 * tests/libtest.c: Add definition of wget_test_set_executable(), 1534 add global variable 'global_executable', 1535 (wget_test): Use 'global_executable' to initialize 'executable', 1536 (wget_test): Use 'executable' if set 1537 15382019-10-27 Archit Pandey <archpndy@gmail.com> 1539 1540 Added options --method, --body-data, --body-file 1541 1542 * src/wget_options.h (config): Added method, body_data, body_file 1543 * src/options.c (options): Added --method, --body-data, --body-file 1544 handling 1545 * src/wget_job.h (JOB): New variable bool redirect_get 1546 * src/wget.c (http_create_request): Added support for user 1547 provided HTTP method and request body 1548 (process_response_header): Set redirect_get to TRUE for 30x 1549 response codes 1550 * tests/test-redirection.c: Added test for redirection with 1551 --method=POST 1552 * docs/wget2.md: Removed [unimplemented-option] for --method, 1553 --body-data, --body-file 1554 15552019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1556 1557 Add method check for test suite 1558 1559 * tests/libtest.h: Add expected_method to wget_test_url_t 1560 * tests/libtest.c (_answer_to_connection): Check method against expected_method 1561 15622019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1563 1564 * tests/libtest.c (_answer_to_connection): Free query.params 1565 15662019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1567 1568 Fix typo: test-post-handhshake-auth -> test-post-handshake-auth 1569 1570 * tests/Makefile.am: Fix typo in handhshake -> handshake 1571 * tests/test-post-handhshake-auth.c: Rename to tests/test-post-handshake-auth.c 1572 15732019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1574 1575 * tests/libtest.c (_answer_to_connection): More code cleanup 1576 15772019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1578 1579 * tests/libtest.c (_answer_to_connection): Simplify URL expansion 1580 15812019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1582 1583 * tests/libtest.c (_answer_to_connection): Code cleanup 1584 15852019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1586 1587 * tests/test-unlink.c (main): Remove double entry from urls 1588 15892019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1590 1591 * tests/libtest.c (_answer_to_connection): Set given status code for response 1592 15932019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1594 1595 * tests/test-metalink.c: Fix test for HTTP/2 1596 15972019-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 1598 1599 * tests/test-E-k.c: Fix for HTTP/2 test 1600 16012019-10-26 Tim Rühsen <tim.ruehsen@gmx.de> 1602 1603 SKIP tests failing with MHD 0.9.67 1604 1605 * tests/test--https-enforce-hard3.c: Likewise 1606 * tests/test--https-enforce-soft2.c: Likewise 1607 16082019-10-23 Tim Rühsen <tim.ruehsen@gmx.de> 1609 1610 * libwget/ssl_gnutls.c: Fix loading CA certs when 'system' is not supported 1611 16122019-10-23 Tim Rühsen <tim.ruehsen@gmx.de> 1613 1614 Fix cookie file header to be recognized by 'file' 1615 1616 * libwget/cookie.c: Use uppercase 'Cookie' and 'File' 1617 * tests/test-cookies.c: Likewise 1618 16192019-10-26 Tim Rühsen <tim.ruehsen@gmx.de> 1620 1621 Add new test file test-E-k.c 1622 1623 * tests/Makefile.am: Add test-E-k.c to tests 1624 * tests/test-E-k.c: New file 1625 16262019-10-26 Tim Rühsen <tim.ruehsen@gmx.de> 1627 1628 Fix conversion (-k) in combination with -E 1629 1630 * src/wget.c (prepare_file): Adjust filename in blacklist if changed, 1631 (conversions): Change type to wget_stringmap, 1632 (convert_links): Use stringmap instead of vector, 1633 (hash_conversion): New function to hash conversion key, 1634 (compare_conversion): New function to compare conversion entries, 1635 (remember_for_conversion): Use stringmap instead of vector 1636 16372019-10-26 Tim Rühsen <tim.ruehsen@gmx.de> 1638 1639 Add new function blacklist_set_filename() 1640 1641 * src/blacklist.c: Implement blacklist_set_filename() 1642 * src/wget_blacklist.h: Add declaration of blacklist_set_filename() 1643 16442019-10-25 Tim Rühsen <tim.ruehsen@gmx.de> 1645 1646 Fix type of 2nd param of wget_stringmap_iterator_next() to void ** 1647 1648 * include/wget/wget.h: Likewise 1649 * unit-tests/test.c: Likewise 1650 16512019-10-19 Tim Rühsen <tim.ruehsen@gmx.de> 1652 1653 Make metalink->iri 'const' 1654 1655 * include/wget/wget.h: Make metalink->iri 'const' 1656 * libwget/metalink.c (add_mirror): Code cleanup 1657 * src/wget.c (establish_connection): Skip HTTP mirrors if HTTPS requested, 1658 (queue_url_from_local): Lock mutex a bit later 1659 16602019-10-19 Tim Rühsen <tim.ruehsen@gmx.de> 1661 1662 Move applying HSTS to queue_url*() functions 1663 1664 * src/wget.c: Add new function test_modify_hsts(), 1665 (queue_url_from_local): Call test_modify_hsts(), 1666 (queue_url_from_remote): Call test_modify_hsts(), 1667 (try_connection): Remove code for applying HSTS 1668 16692019-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 1670 1671 Use const wget_iri instances 1672 1673 * src/wget_host.h (host_add, host_get): Add 'const' to param 1674 * src/host.c: Likewise 1675 * src/stats_site.c: Add 'const' to wget_iri types 1676 * src/wget.c: Likewise 1677 * src/wget_job.h (struct JOB): Add 'const' to 'iri' 1678 16792019-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 1680 1681 Amend job_init() to take blacklist_entry instead of wget_iri 1682 1683 * src/wget_job.h: Amend job_init() to take blacklist_entry instead of wget_iri 1684 * src/job.c: Likewise 1685 * src/host.c (host_add_robotstxt_job): Amend call to job_init() 1686 * src/wget.c (queue_url_from_local): Likewise, 1687 (queue_url_from_remote): Likewise 1688 16892019-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 1690 1691 Deduplicate code for creating robots.txt jobs 1692 1693 * src/wget.c (queue_url_from_local): Remove duplicated code, 1694 (queue_url_from_remote): Likewise, 1695 (set_file_metadata): Amend calling wget_iri_get_connection_part() 1696 * include/wget/wget.h (wget_iri_parse_base): Add 'const' to 'base' param, 1697 (wget_iri_get_connection_part): Add 'const' to 'iri' param, 1698 add second param 'buf' 1699 * fuzz/libwget_iri_fuzzer.c (test): Amend calling wget_iri_get_connection_part() 1700 * libwget/iri.c (wget_iri_get_connection_part): Add param 'buf', 1701 add 'const' to 'iri' param, amend docs, 1702 (wget_iri_relative_to_abs): Add 'const' to 'base' param, 1703 amend calling wget_iri_get_connection_part(), 1704 (wget_iri_parse_base): Add 'const' to 'iri' param 1705 * src/host.c (host_add_robotstxt_job): Add code removed in wget.c 1706 * src/wget_host.h: Don't include wget_blacklist.h, 1707 amend signature of host_add_robotstxt_job() 1708 * src/wget_job.h (struct JOB): Add 'const' to 'blacklist_entry' 1709 17102019-10-16 Tim Rühsen <tim.ruehsen@gmx.de> 1711 1712 Blacklist changes 1713 1714 * src/wget_main.h: Remove decalration of get_local_filename() 1715 * src/wget_job.h (struct JOB): Remove local_filename, add blacklist_entry 1716 * src/wget_host.h: Amend declaration of host_add_robotstxt_job() 1717 * src/wget_blacklist.h (struct blacklist_entry): Add const to iri, 1718 add const to param of blacklist_add(), 1719 add declaration of blacklist_get() 1720 * src/job.c (job_free): Do not free job->local_filename 1721 * src/host.c (host_add_robotstxt_job): Param blacklist_entry instead of wget_iri 1722 * src/gpgme.c (determine_base_file): Fix calls to determine_base_file() 1723 * src/blacklist.c: Add new function blacklist_get(), 1724 (blacklist_add): Add const to param, call get_local_filename(), 1725 (blacklist_init): Amend destructors, 1726 Remove free_key(), 1727 Add free_value(), 1728 Add function get_local_filename(), 1729 Add function get_local_filename_real() 1730 * src/wget.c: Remove function get_local_filename(), 1731 Remove function get_local_filename_real(), 1732 Use blacklist_entry instead of local_filename 1733 17342019-10-15 Tim Rühsen <tim.ruehsen@gmx.de> 1735 1736 * src/wget.c: Rename add_url() -> queue_url_from_remote() 1737 17382019-10-15 Tim Rühsen <tim.ruehsen@gmx.de> 1739 1740 * src/wget.c: Rename add_url_to_queue() -> queue_url_from_local() 1741 17422019-10-15 Tim Rühsen <tim.ruehsen@gmx.de> 1743 1744 Cleanup blacklist code 1745 1746 * src/wget_blacklist.h: Add new structure 'blacklist_entry', 1747 change return type of blacklist_add() to 'blacklist_entry *' 1748 * src/blacklist.c: Store 'blacklist_entry' instead of 'wget_iri', 1749 remove mutexes 1750 * src/wget.c: Amend code in add_url_to_queue() and add_url() 1751 17522019-10-14 Tim Rühsen <tim.ruehsen@gmx.de> 1753 1754 * src/wget_blacklist.h: Remove unused declaration of in_blacklist() 1755 17562019-10-23 Tim Rühsen <tim.ruehsen@gmx.de> 1757 1758 * configure.ac: Add pkg-config support for GPGME 1759 17602019-10-22 Archit Pandey <archpndy@gmail.com> 1761 1762 Using --robots=off downloads robots.txt 1763 1764 * src/wget.c (add_url_to_queue): remove config.robots check, 1765 (add_url): add config.robots check when disallowing URLs 1766 * tests/test-robots-off.c: New file to test --robots=off behavior 1767 * tests/Makefile.am: Added test-robots-off 1768 * tests/test-iri-percent.c: Add check for robots.txt 1769 * docs/wget2.md: Describe --robots=off behaviour 1770 17712019-10-21 Archit Pandey <archpndy@gmail.com> 1772 1773 Moved WGET_E_GPG_* definitions out of libwget 1774 1775 * libwget/error.c (wget_strerror): Remove WGET_E_GPG_* symbols 1776 * include/wget/wget.h (wget_error): Remove WGET_E_GPG_* symbol definitions 1777 * src/wget_gpgme.h: New enum wget_gpg_error to hold WGET_E_GPG_* symbol definitions 1778 * AUHOTRS: Add archpndy [at] gmail.com 1779 1780 Copyright-paperwork-exempt: Yes 1781 17822019-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 1783 1784 * src/wget.c (prepare_file): Fix regression from commit 3c6f61a8 1785 17862019-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 1787 1788 * src/wget.c (prepare_file): Check '.htm' for -E/--adjust-extension 1789 17902019-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 1791 1792 * src/wget.c (prepare_file): Simplify code to append extension (-E) 1793 17942019-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 1795 1796 * .gitlab-ci.yml: Add automated coverity scan 1797 1798 This only works on a branch called 'coverity-scan' 1799 18002019-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 1801 1802 * tests/libtest.c (wget_test_check_file_system): Use memcmp() instead of strcmp() 1803 18042019-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 1805 1806 * include/wget/wget.h: Add WGET_GCC_UNUSED_RESULT to wget_hashmap_get and wget_stringmap_get 1807 18082019-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 1809 1810 * src/stats_site.c (stats_site_add): Check return value of wget_stringmap_get() 1811 18122019-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 1813 1814 * src/wget.c (in_directory_pattern_list): Return false if fname is NULL 1815 18162019-10-09 Tim Rühsen <tim.ruehsen@gmx.de> 1817 1818 Fix NULL dereference in html_parse() 1819 1820 * src/wget.c (html_parse): Add 'filename' to list of params 1821 1822 Under certain conditions (--convert-links) existing files are 1823 scanned for URLs with 'job' being NULL. We still need a filename 1824 without getting it from job->local_filename, which would cause a 1825 NULL pointer dereference. 1826 18272019-10-09 Tim Rühsen <tim.ruehsen@gmx.de> 1828 1829 * libwget/hsts.c (new_hsts): Check memory allocation 1830 18312019-10-08 Tim Rühsen <tim.ruehsen@gmx.de> 1832 1833 libwget/http.c (wget_http_get_response_cb): Don't ignore response body on header callback failure 1834 18352019-10-08 Tim Rühsen <tim.ruehsen@gmx.de> 1836 1837 Fix directory clash with --no-clobber 1838 1839 * src/wget.c: New functions is_file() and is_directory(), 1840 (prepare_file): Tune clash checks 1841 * tests/test-directory-clash.c: Add two more tests 1842 18432019-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 1844 1845 * libwget/http_parse.c (wget_http_parse_connection): Allow list of tokens 1846 1847 This change detects the keep-alive flag in headers like 1848 Connection: Upgrade, Keep-Alive 1849 18502019-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 1851 1852 * src/wget.c (html_parse): Limit to page requisites only for leaf pages 1853 18542019-10-04 Tim Rühsen <tim.ruehsen@gmx.de> 1855 1856 Rename --retry-on-http-status to --retry-on-http-error 1857 1858 * docs/wget2.md: Likewise 1859 * libwget/http.c: Likewise 1860 * src/options.c: Likewise 1861 * src/wget.c: Likewise 1862 * src/wget_options.h: Likewise 1863 1864 Needed for backward compatibility with Wget 1.x. 1865 18662019-09-29 Tim Rühsen <tim.ruehsen@gmx.de> 1867 1868 * tests/test-compression.c: Replace malloc() by fixed size array 1869 18702019-10-03 Darshit Shah <darnir@gnu.org> 1871 1872 * configure.ac: Update minimum required version 1873 18742019-10-03 Darshit Shah <darnir@gnu.org> 1875 1876 Update Gnulib 1877 1878 * gnulib: Update submodule 1879 * bootstrap.conf: Add module access 1880 18812019-09-25 Tim Rühsen <tim.ruehsen@gmx.de> 1882 1883 Fix bug in wget_dns_cache_add() 1884 1885 * libwget/dns_cache.c (wget_dns_cache_add): 1886 wget_hashmap_get() gets a cache_entry, not a addrinfo 1887 18882019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1889 1890 Remove leading WG_ from exit_status_e enums 1891 1892 * src/wget_options.h: Likewise 1893 * src/wget.c: Likewise 1894 * src/options.c: Likewise 1895 18962019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1897 1898 Introduce enum type gpg_verify_mode 1899 1900 * src/wget_options.h: Add enum type gpg_verify_mode, 1901 remove leading WGET_ from GPG_VERIFY enums 1902 * src/options.c: Use new enum type 1903 * src/wget.c: Likewise 1904 19052019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1906 1907 * src/wget_options.h (struct config): Move members from char to bool 1908 19092019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1910 1911 * src/wget_options.h: Use bool for ocsp_date and ocsp_nonce 1912 19132019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1914 1915 Introduce enum type https_enforce_mode 1916 1917 * src/options.h: Add enum type https_enforce_mode, 1918 remove leading WGET_ from HTTPS_ENFORCE enums 1919 * src/options.c: Use new enum type 1920 * src/wget.c: Likewise 1921 19222019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1923 1924 * src/wget_stats.h: Remove unused macros 1925 19262019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1927 1928 * src/wget.c: Improve C99 compliancy 1929 19302019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1931 1932 * src/stats_site.c: Improve C99 compliancy 1933 19342019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1935 1936 * src/plugin.c: Improve C99 compliancy 1937 19382019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1939 1940 * src/options.c: Improve C99 compliancy 1941 19422019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1943 1944 * src/log.c: Improve C99 compliancy 1945 19462019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1947 1948 * src/job.c: Improve C99 compliancy 1949 19502019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1951 1952 * src/gpgme.c: Improve C99 compliancy 1953 19542019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1955 1956 * src/bar.c: Improve C99 compliancy 1957 19582019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1959 1960 * libwget/xml.c: Improve C99 compliancy 1961 19622019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1963 1964 * libwget/utils.c: Improve C99 compliancy 1965 19662019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1967 1968 * libwget/ssl_wolfssl.c: Improve C99 compliancy 1969 19702019-09-24 Tim Rühsen <tim.ruehsen@gmx.de> 1971 1972 * libwget/ssl_openssl.c: Improve C99 compliancy 1973 19742019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 1975 1976 * libwget/io.c: Rely on gnulib poll module 1977 19782019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 1979 1980 * libwget/ssl_gnutls.c: Improve C99 compliancy 1981 19822019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 1983 1984 * libwget/sitemap_url.c: Improve C99 compliancy 1985 19862019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 1987 1988 * libwget/rss_url.c: Improve C99 compliancy 1989 19902019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 1991 1992 * libwget/random.c: Improve C99 compliancy 1993 19942019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 1995 1996 * libwget/net.c: Improve C99 compliancy 1997 19982019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 1999 2000 * libwget/logger.c: Improve C99 compliancy 2001 20022019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 2003 2004 * libwget/log.c: Improve C99 compliancy 2005 20062019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 2007 2008 * libwget/iri.c: Improve C99 compliancy 2009 20102019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 2011 2012 * libwget/io.c: Improve C99 compliancy 2013 20142019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 2015 2016 * libwget/init.c: Improve C99 compliancy 2017 20182019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 2019 2020 * libwget/http_highlevel.c: Improve C99 compliancy 2021 20222019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 2023 2024 Use int64_t for time_t to fix 32bit time_t limitations 2025 2026 * include/wget/wget.h: Use int64_t instead of time_t 2027 * libwget/hpkp.c: Likewise 2028 * libwget/hsts.c: Likewise 2029 * libwget/http_parse.c: Likewise 2030 * libwget/ocsp.c: Likewise 2031 * libwget/ssl_gnutls.c: Likewise 2032 * libwget/tls_session.c: Likewise 2033 * src/options.c: Likewise 2034 * src/stats_site.c: Likewise 2035 * src/wget.c: Likewise 2036 * tests/libtest.c: Likewise 2037 * tests/libtest.h: Likewise 2038 * tests/test-plugin-dummy.c: Likewise 2039 * unit-tests/test.c: Likewise 2040 20412019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 2042 2043 * libwget/decompressor.c (lzip_decompress): Fix warning 2044 20452019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 2046 2047 Fix http date functions to use int64_t instead of time_t 2048 2049 * include/wget/wget.h: Change time_t -> int64_t for wget_http_print_date() 2050 and for wget_http_parse_full_date() 2051 * libwget/http_parse.c: Likewise 2052 20532019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 2054 2055 * tests/libtest.c (wget_test): Print expected+real content of diverting files 2056 20572019-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 2058 2059 Fix time_t to int64_t in cookie code 2060 2061 * libwget/cookie.c: Likewise 2062 * libwget/cookie.h: Likewise 2063 20642019-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 2065 2066 * unit-tests/test-dl.c: Add missing len param to plugin function call 2067 20682019-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 2069 2070 * .gitlab-ci.yml: Add Valgrind/i386 runner 2071 20722019-09-21 Tim Rühsen <tim.ruehsen@gmx.de> 2073 2074 Add unit test test-decompress.c 2075 2076 * unit-tests/Makefile.am: Add test-decompress.c 2077 * unit-tests/test-decompress.c: New file 2078 20792019-09-21 Tim Rühsen <tim.ruehsen@gmx.de> 2080 2081 * libwget/decompressor.c (wget_decompress_open): Fix xz decompression 2082 20832019-09-21 Tim Rühsen <tim.ruehsen@gmx.de> 2084 2085 Modify wget_decompress() 2nd arg to const 2086 2087 * include/wget/wget.h: Likewise 2088 * libwget/decompressor.c: Likewise 2089 20902019-09-20 Tim Rühsen <tim.ruehsen@gmx.de> 2091 2092 * libwget/decompressor.c: Make use LZ_decompress_finish() 2093 20942019-09-19 Tim Rühsen <tim.ruehsen@gmx.de> 2095 2096 * README.md: Add OSS-Fuzz badge [skip ci] 2097 20982019-09-19 Tim Rühsen <tim.ruehsen@gmx.de> 2099 2100 Implement Accept-Encoding: lzip including decompressor 2101 2102 * README.md: Mention lzip 2103 * configure.ac: Detect lzip 2104 * docs/wget2.md: Document 'lzip' 2105 * include/wget/wget.h: Add wget_content_encoding_lzip 2106 * libwget/decompressor.c: Implement lzip decompression 2107 * libwget/http_parse.c (wget_http_parse_content_encoding): Add "lzip" 2108 * src/options.c: Add code to support lzip 2109 * src/wget.c (http_create_request): Add lzip to Accept-Encoding: if available 2110 * tests/test-compression.c: Add test vector for lzip decompression 2111 21122019-09-19 Tim Rühsen <tim.ruehsen@gmx.de> 2113 2114 * contrib/mk_authors: Simplify getting the latest git tag [skip ci] 2115 21162019-09-19 Tim Rühsen <tim.ruehsen@gmx.de> 2117 2118 * contrib/mk_authors: Fix getting the latest git tag [skip ci] 2119 21202019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2121 2122 * libwget/http.c: Improve C99 compliancy 2123 21242019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2125 2126 * libwget/html_url.c: Improve C99 compliancy 2127 21282019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2129 2130 * libwget/hashmap.c: Improve C99 compliancy 2131 21322019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2133 2134 * libwget/hashfile.c: Improve C99 compliancy 2135 21362019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2137 2138 * libwget/dns_cache.c: Improve C99 compliancy 2139 21402019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2141 2142 * libwget/dns.c: Improve C99 compliancy 2143 21442019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2145 2146 * libwget/css_url.c: Improve C99 compliancy 2147 21482019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2149 2150 * libwget/console.c: Improve C99 compliancy 2151 21522019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2153 2154 * libwget/buffer_printf.c: Improve C99 compliancy 2155 21562019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2157 2158 * libwget/buffer.c: Improve C99 compliancy 2159 21602019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2161 2162 * libwget/base64.c: Improve C99 compliancy 2163 21642019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2165 2166 Remove use of !! in src/ 2167 2168 * src/host.c: Change return type of _search_host_for_free_job() to bool 2169 * src/options.c (selftest_options): Make use of bool 2170 * src/stats_site.c (stats_site_add): Use '!= 0' instead of !! 2171 * src/wget.c: (_get_local_filename): Change type of directories to bool 2172 * src/wget_options.h (struct config): Change type of recursive to bool 2173 21742019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2175 2176 * libwget/bar.c: Improve C99 compliancy 2177 21782019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2179 2180 * libwget/atom_url.c: Improve C99 compliancy 2181 21822019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2183 2184 Change type to bool for wget_tcp_set_ssl/wget_tcp_get_ssl 2185 2186 * include/wget/wget.h: Likewise 2187 * libwget/net.c: Likewise 2188 21892019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2190 2191 Use '!= 0' instead of 'git commit --amend' in libwget/ssl_gnutls.c 2192 2193 * include/wget/wget.h (wget_tls_stats_data): Change type of tfo and false_start to bool 2194 * libwget/ssl_gnutls.c: Use bool in _session_context, 2195 remove use of !! 2196 21972019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2198 2199 Change return type of wget_tcp_get_tls_false_start() to bool 2200 2201 * include/wget/wget.h: Likewise 2202 * libwget/net.c: Likewise 2203 22042019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2205 2206 Change second param type of wget_tcp_set_tls_false_start() to bool 2207 2208 * include/wget/wget.h: Likewise 2209 * libwget/net.c: Likewise 2210 * src/wget_options.h (struct config): Change type of tls_false_start to bool 2211 22122019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2213 2214 Change return type of wget_tcp_get_tcp_fastopen() to bool 2215 2216 * include/wget/wget.h: Likewise 2217 * libwget/net.c: Likewise 2218 22192019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2220 2221 Change second param type of wget_tcp_set_tcp_fastopen() to bool 2222 2223 * include/wget/wget.h: Likewise 2224 * libwget/net.c: Likewise 2225 * libwget/init.c (wget_global_init): Fix param type to wget_tcp_set_tcp_fastopen() 2226 * src/wget_options.h (struct config): Change type of tcp_fastopen to bool 2227 22282019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2229 2230 * libwget/logger.c: Use '!= 0' instead of !! 2231 22322019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2233 2234 * libwget/init.c: Use '!= 0' instead of !! 2235 22362019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2237 2238 * libwget/http.c: Use '!= 0' instead of !! 2239 22402019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2241 2242 Use bool instead of char in HSTS code 2243 2244 * include/wget/wget.h: Change param to bool in wget_hsts_db_add_fn(), 2245 (struct wget_http_response_st): Use bool for hsts_include_subdomains and keep_alive, 2246 change param to bool in wget_http_parse_strict_transport_security(), 2247 change param to bool in wget_http_parse_connection() 2248 * libwget/hsts.c: change param to bool in new_hsts(), 2249 change param to bool in wget_hsts_db_add() 2250 * libwget/http_parse.c: change param to bool in wget_http_parse_strict_transport_security(), 2251 change param to bool in wget_http_parse_connection() 2252 * tests/test-plugin-dummy.c: change param to bool in test_hsts_db_add() 2253 * unit-tests/test.c (test_hsts): Change type of include_subdomains to bool 2254 22552019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2256 2257 Change return type of wget_str_needs_encoding() to bool 2258 2259 * include/wget/wget.h: Likewise 2260 * libwget/encoding.c: Likewise 2261 22622019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2263 2264 * libwget/cookie_parse.c (cookie_normalize_cookie): Use '!= 0' instead of !! 2265 22662019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2267 2268 Use bool param for wget_cookie_set_keep_session_cookies() 2269 2270 * include/wget/wget.h: Change second param of wget_cookie_set_keep_session_cookies() to bool 2271 * libwget/cookie.c: Likewise 2272 * libwget/init.c (struct _CONFIG): Use bool for cookies_enabled, keep_session_cookies 2273 22742019-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 2275 2276 * libwget/base64.c (_isbase64): Use '!= 0' instead of !! 2277 22782019-09-14 Tim Rühsen <tim.ruehsen@gmx.de> 2279 2280 Add new cookie test 2281 2282 * tests/Makefile.am: Add test-cookies 2283 * tests/test-cookies.c: New file 2284 22852019-09-14 Tim Rühsen <tim.ruehsen@gmx.de> 2286 2287 Add new wget_test() option WGET_TEST_CLEAN_DIRECTORY 2288 2289 * tests/libtest.c: Implement WGET_TEST_CLEAN_DIRECTORY 2290 * tests/libtest.h: Add WGET_TEST_CLEAN_DIRECTORY 2291 22922019-09-16 Tim Rühsen <tim.ruehsen@gmx.de> 2293 2294 Fix for reproducible builds 2295 2296 * .gitignore: Remove src/version-text.h 2297 * cfg.mk: Add rule to update copyright in src/options.c 2298 * configure.ac: Remove YEAR 2299 * src/Makefile.am: Remove version-text.h.in 2300 * src/options.c: Add version text 2301 * src/version-text.h.in: Deleted 2302 2303 Reported-by: Bernhard M. Wiedemann 2304 23052019-09-14 Darshit Shah <darnir@gnu.org> 2306 2307 * libwget/cookie_parse.c: Fix spelling issue 2308 23092019-09-13 Darshit Shah <darnir@gnu.org> 2310 2311 Don't call libwget as Wget / Wget2. 2312 2313 Functions in Libwget may be called from other client applications as 2314 well. They should not report that the job was done by Wget / Wget2, but 2315 instead they should state that it was doen by libwget. 2316 2317 * libwget/bar.c(_bar_update): Remove reference to Wget 2318 * libwget/list.c: Same 2319 * libwget/cookie.c(cookie_db_save): Wget -> libwget 2320 * libwget/error.c(wget_strerror): Same 2321 * libwget/hpkp_db.c(hpkp_db_save): Same 2322 * libwget/hsts.c(hsts_db_save): Same 2323 * libwget/ocsp.c(ocsp_db_save_hosts): Same 2324 * libwget/printf.c: Same 2325 * libwget/ssl_gnutls.c(wget_ssl_open: Same 2326 * libwget/tls_session.c(tls_session_db_save): Same 2327 23282019-09-12 Ander Juaristi <a@juaristi.eus> 2329 2330 OpenSSL: Release cert chain (more memleaks) 2331 2332 * libwget/ssl_openssl.c (_openssl_revocation_check_fn): free 2333 cert chain in all error paths 2334 23352019-09-04 Ander Juaristi <a@juaristi.eus> 2336 2337 OpenSSL: fix memory leaks 2338 23392019-09-11 Darshit Shah <darnir@gnu.org> 2340 2341 Remove gettext support in tests/ 2342 2343 * tests/libtest.h: Remove gettext support entirely 2344 * tests/test-iri-subdir.c: Needs the string.h header 2345 * tests/test-limit-rate-http2.c: Same 2346 * tests/test-limit-rate.c: Same 2347 * tests/test-metalink.c: Same 2348 * tests/gpg-test-util.h: Same 2349 * tests/test-gpg-save-failed.c: Same 2350 * tests/test-include-and-exclude-directories.c: Same 2351 23522019-09-10 Darshit Shah <darnir@gnu.org> 2353 2354 Use gnulib's update-copyright module 2355 2356 * bootstrap.conf: Add update-copyright module 2357 * cfg.mk: Setup environment for update-copyright 2358 * contrib/update_copyright: Remove script no longer used 2359 * libwget/ssl_openssl: Fix copyright year 2360 * tests/test-post-handshare-auth.c: Fix copyright years 2361 * contrib/commit-check: Add copyright and license 2362 * contrib/mail.sh: Same 2363 * src/version-text.h.in: Same 2364 23652019-09-08 Darshit Shah <darnir@gnu.org> 2366 2367 Update Copyright statements to be compatible with update-copyright module 2368 23692019-09-10 Darshit Shah <darnir@gnu.org> 2370 2371 Update cfg.mk. Add comments for future 2372 2373 * cfg.mk: Multiple changes. 2374 (sc_GPL_version): Don't exclude all files when only one has a parsing 2375 issue 2376 (sc_po_check): Tests should always output English. Their output is for 2377 developers, not users 2378 (sc_prohibit_magic_number_exit): tests/test-plugin.c has a parsing 2379 issue that has been reported to gnulib. Others should be checked. 2380 (sc_trailing_blank): Fixed the relevant files instead of ignoring them 2381 (sc_two_space_separator_in_usage): Doesn't seem to trigger on removal 2382 (sc_prohibit_empty_lines_at_EOF): Same 2383 (sc_prohibit_sprintf): Same 2384 (sc_prohibit_printf): Re-arrange within file 2385 (sc_prohibit_free): Same 2386 (sc_prohibit_alloc): Same 2387 (sc_gettext_printf): Same 2388 * docs/DoxygenLayout.xml: Remove trailing whitespaces 2389 * docs/libwget.doxy.in: Same 2390 * tests/libtest.c: Don't internationalize test suite strings 2391 * tests/test-*.c: Use EXIT_FAILURE and EXIT_SUCCESS instead of 1 and 0 2392 23932019-09-10 Darshit Shah <darnir@gnu.org> 2394 2395 * cfg.mk: We don't cast arguments to free anywhere 2396 23972019-09-10 Tim Rühsen <tim.ruehsen@gmx.de> 2398 2399 Revert "* cfg.mk: We don't need to exclude gnulib from syntax-check" [ci skip] 2400 2401 This reverts commit 074983606b3536d0cf2677beb0ed523dd43145e3. 2402 2403 Also added a comment in cfg.mk to explain why 2404 24052019-08-22 Ander Juaristi <a@juaristi.eus> 2406 2407 * libwget/ssl_openssl.c: Implement ALPN 2408 24092019-09-07 Darshit Shah <darnir@gnu.org> 2410 2411 * cfg.mk: We don't need to exclude gnulib from syntax-check 2412 24132019-09-06 Tim Rühsen <tim.ruehsen@gmx.de> 2414 2415 * libwget/init.c (wget_global_get_ptr): Fix return type for WGET_COOKIE_DB 2416 24172019-09-06 Tim Rühsen <tim.ruehsen@gmx.de> 2418 2419 * libwget/dns.c: Add constructor & destructor for global mutex 2420 24212019-09-06 Tim Rühsen <tim.ruehsen@gmx.de> 2422 2423 * examples/http_get.c: Use https:// instead of http:// 2424 24252019-09-02 Tim Rühsen <tim.ruehsen@gmx.de> 2426 2427 Rename wget_html_parsed_url.dir -> wget_html_parsed_url.tag 2428 2429 * include/wget/wget.h: Rename wget_html_parsed_url.dir -> wget_html_parsed_url.tag 2430 * examples/print_html_urls.c: Likewise 2431 * libwget/html_url.c: Likewise 2432 * src/wget.c: Likewise 2433 24342019-09-01 Ander Juaristi <a@juaristi.eus> 2435 2436 OpenSSL: Fix CRL checking 2437 2438 Check CRLs properly on the OpenSSL backend. CRLs were previously bypassed 2439 because OpenSSL's internal certificate verification function was being overridden. 2440 2441 We need to provide our own verification callback for HPKP and OCSP, but this callback is now 2442 introduced later in the verification chain, so that it does no longer override OpenSSL's checks. 2443 2444 This commit also re-enables a CRL test on OpenSSL, which would previously only run on the GnuTLS backend. 2445 2446 * libwget/ssl_openssl.c (openssl_init): Register verification callback, rather than revocation one. 2447 (_openssl_revocation_check_fn): Adhere to verification callback interface. 2448 * tests/test-i-https.c: Enable CRL test for OpenSSL (but keep it disabled for wolfSSL) 2449 2450 Signed-off-by: Ander Juaristi <a@juaristi.eus> 2451 24522019-08-30 Tim Rühsen <tim.ruehsen@gmx.de> 2453 2454 Add script to generate list of authors by # of commits 2455 2456 * .mailmap: Map authors when using different names/email addresses 2457 * contrib/mk_authors: Create list of authors sorted by # of commits 2458 * contrib/release: Mention contrib/mk_authors 2459 24602019-08-30 Tim Rühsen <tim.ruehsen@gmx.de> 2461 2462 Release 1.99.2 (beta) 2463 24642019-08-30 Tim Rühsen <tim.ruehsen@gmx.de> 2465 2466 Only use clang's Nullability feature with enabled manywarnings 2467 24682019-08-21 Darshit Shah <darnir@gnu.org> 2469 2470 * bootstrap.conf: Add gnulib modules for non-portable function calls 2471 24722019-08-21 Darshit Shah <darnir@gnu.org> 2473 2474 Remove portability warnings from configure 2475 2476 * configure.ac: Don't warn when we use GNU Make extensions. 2477 We have already dug into this by disabling the relevant syntax-check 2478 rule as well. The recent commit adding support for gcov based code 2479 coverage adds more GNU make specific rules. 2480 24812019-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 2482 2483 Fix hashing with Gcrypt backend 2484 24852019-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 2486 2487 Fix hashing with Nettle backend 2488 24892019-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 2490 2491 Fix hashing with OpenSSL backend 2492 24932019-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 2494 2495 Fix hashing with GnuTLS backend 2496 2497 * libwget/hashfile.c: Fix the GnuTLS functions 2498 24992019-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 2500 2501 * libwget/hashfile.c (wget_hash_file_fd): Do not close fd on error 2502 25032019-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 2504 2505 Fix hashing with WolfTLS backend 2506 2507 * include/wget/wget.h: Change param for wget_hash_init(), 2508 change param and return type for wget_hash_deinit() 2509 * libwget/hashfile.c: Rewrite hashing functions for WITH_LIBWOLFCRYPT 2510 * unit-tests/test.c: Add tests for incremental hashing 2511 25122019-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 2513 2514 * configure.ac: Fix TLS/hashing detection 2515 25162019-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 2517 2518 * tests/test-i-https.c: CRL test only for GnuTLS 2519 25202019-08-25 Tim Rühsen <tim.ruehsen@gmx.de> 2521 2522 Update test suite certs and build script 2523 2524 * tests/certs/revoked-template.txt: New file 2525 * tests/certs/ca-template.txt: Update 2526 * tests/certs/generate_certs.sh: Use revoked-template.txt 2527 * tests/certs/server-template.txt: Update 2528 * tests/certs/*.pem: Update, generated by generate_certs.sh 2529 25302019-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 2531 2532 * libwget/ssl_openssl.c: Fix some assignment warnings 2533 25342019-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 2535 2536 * libwget/ssl_openssl.c: Apply fixes due to rebasing 2537 25382019-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 2539 2540 * libwget/hashfile.c: Make use of WGET_E_UNSUPPORTED 2541 25422019-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 2543 2544 * libwget/error.c (wget_strerror): Add WGET_E_UNSUPPORTED 2545 25462019-08-22 Ander Juaristi <ajuaristi@gmx.es> 2547 2548 Add more hash backends 2549 25502019-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 2551 2552 * libwget/ssl_openssl.c: Do not translate debug messages 2553 25542019-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 2555 2556 * libwget/ssl_openssl.c (verify_hpkp): Add switch default 2557 25582019-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 2559 2560 * libwget/ssl_openssl.c: Fix indentation 2561 25622019-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 2563 2564 * libwget/ssl_openssl.c (openssl_load_trust_files_from_directory): Cleanup 2565 25662019-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 2567 2568 * .gitlab-ci.yml: Add OpenSSL runner 2569 25702019-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 2571 2572 * .gitlab-ci.yml: Fix WolfSSL CI build 2573 25742019-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 2575 2576 * libwget/ssl_openssl.c (openssl_load_trust_file): Use buffer printf instead of VLA 2577 25782019-07-07 Tim Rühsen <tim.ruehsen@gmx.de> 2579 2580 * libwget/ssl_openssl.c: Rebase for new stats API 2581 25822019-06-17 Ander Juaristi <a@juaristi.eus> 2583 2584 Rebase wolfssl branch 2585 2586 * configure.ac: Pretty print 'SSL/TLS support' 2587 * libwget/Makefile.am: Cleanup logic 2588 25892019-06-15 Ander Juaristi <a@juaristi.eus> 2590 2591 * configure.ac: Add SSL/TLS option wolfssl 2592 25932018-09-16 Ander Juaristi <a@juaristi.eus> 2594 2595 SSL/TLS: Add OpenSSL support 2596 2597 * configure.ac: Add --with-ssl=[no,openssl,gnutls], remove --without-gnutls 2598 * libwget/Makefile.am: Add ssl_openssl.c 2599 * libwget/ssl_openssl.c: New file 2600 * po/POTFILES.in: Add libwget/ssl_openssl.c 2601 26022019-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 2603 2604 * configure.ac: Add -Wno-used-but-marked-unused to GNULIB_WARN_FLAGS 2605 26062019-08-27 DalmeGNU <dalmemail@gmail.com> 2607 2608 * benchmarks/sources/curl.bench.sh: Build curl with GnuTLS, in order to have SSL support 2609 26102019-08-26 Tim Rühsen <tim.ruehsen@gmx.de> 2611 2612 * src/options.c (_preload_dns_cache): Don't wait for stdin while testing 2613 26142019-08-25 Tim Rühsen <tim.ruehsen@gmx.de> 2615 2616 * contrib/spell-checker: Do not check tests/certs/README.md 2617 26182019-08-25 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 2619 2620 * tests/certs/README.md: Added README for certificate generation 2621 26222019-08-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 2623 2624 Regenerating certificates for openssl compatibility 2625 2626 * tests/certs/generate_certs.sh: Script for certificate generation 2627 * tests/certs/x509-ca-cert.pem: CA certificate 2628 * tests/certs/x509-ca-key.pem: CA key 2629 * tests/certs/x509-server-cert.pem: Server certificate 2630 * tests/certs/x509-server-key.pem: Server key 2631 * tests/certs/x509-server-crl.pem: CRL for the server certificate 2632 * tests/certs/ca-template.txt: Template for CA Certificate 2633 * tests/certs/server-template.txt: Template for Server Certificate 2634 26352019-08-23 DalmeGNU <dalmemail@gmail.com> 2636 2637 Recursive mode now works with -N --no-if-modified-since 2638 2639 * src/wget.c: Likewise 2640 * tests/test-timestamping.c: Added tests for -N -r (with If-Modified-Since) and amended the ones without If-Modified-Since 2641 * tests/test-include-and-exclude-directories.c: Amended tests that use -N --no-if-modified-since -r 2642 * tests/test--filter-mime-type.c: Amended tests that use -N --no-if-modified-since -r 2643 26442019-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 2645 2646 Update NEWS 2647 26482019-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 2649 2650 * include/wget/wget.h: Fix typos 2651 26522019-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 2653 2654 * contrib/spell-checker: Abort on error (set -e) 2655 26562019-08-22 DalmeGNU <dalmemail@gmail.com> 2657 2658 Consider files as 'application/octet-stream' if MIME is not present 2659 2660 * src/wget.c: Likewise, following RFC 7231 2661 * tests/test--filter-mime-type.c: Added tests to ensure this behavior is present 2662 * docs/wget2.md: Added this new behavior to the docs 2663 26642019-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 2665 2666 * libwget/net.h: Include netdb.h for struct addrinfo [skip ci] 2667 26682019-08-18 DalmeGNU <dalmemail@gmail.com> 2669 2670 Implement --no-if-modified-since 2671 2672 * src/options.c: Added new option --if-modified-since, and prevented certain combination of flags 2673 * src/wget.c: Added -N --no-if-modified-since implementation itself 2674 * src/wget_job.h: Added bool 'recursive_send_head' to JOB structure 2675 * src/wget_options.h: Added char 'if_modified_since' to config structure 2676 * tests/test--accept.c: Added tests that make use of -N --no-if-modified-since 2677 * tests/test--filter-mime-type.c: Added tests that make use of -N --no-if-modified-since 2678 * tests/test-include-and-exclude-directories.c: Added tests that make use of -N --no-if-modified-since 2679 * tests/test-timestamping.c: Added tests for -N --no-if-modified-since 2680 * docs/wget2.md: Unmarked --no-if-modified-since as "Not Implemented Yet" 2681 26822019-08-17 DalmeGNU <dalmemail@gmail.com> 2683 2684 * src/options.c: Prevent certain combination of flags with -N (as in wget1.x) 2685 26862019-08-15 DalmeGNU <dalmemail@gmail.com> 2687 2688 Move -N mode tests into a separate file 2689 2690 * tests/test-wget-1.c: Removed -N (timestamping) tests 2691 * tests/test-timestamping.c: Added -N tests from tests/test-wget-1.c 2692 * tests/Makefile.am: Added tests/test-timestamping.c to Makefile 2693 26942019-08-22 Tim Rühsen <tim.ruehsen@gmx.de> 2695 2696 * .gitlab-ci.yml: Use --disable-threads on FreeBSD instead MinGW CI runner 2697 26982019-08-20 DalmeGNU <dalmemail@gmail.com> 2699 2700 * tests/libtest.c: Create directories if needed when creating WGET_TEST_EXISTING_FILES 2701 27022019-08-22 Tim Rühsen <tim.ruehsen@gmx.de> 2703 2704 * src/utils.c (mkdir_path): Do not write 'fname' param 2705 27062019-08-22 Tim Rühsen <tim.ruehsen@gmx.de> 2707 2708 * src/Makefile.am: Add -DMALLOC_RETURNS_NONNULL to AM_CFLAGS 2709 27102019-08-22 Tim Rühsen <tim.ruehsen@gmx.de> 2711 2712 * configure.ac: AC_SUBST BZ2_LIBS 2713 27142019-08-22 Tim Rühsen <tim.ruehsen@gmx.de> 2715 2716 * libwget/decompressor.c: Improve C99 compliance, check memory allocation 2717 27182019-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 2719 2720 * tests/libtest.c (_insert_ports): Fix for reject_https_connection 2721 27222019-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 2723 2724 Fix --https-enforce tests 2725 2726 * tests/test--https-enforce-hard3.c: Add --default-http-port={{port}} 2727 * tests/test--https-enforce-soft1.c: Use WGET_TEST_HTTPS_REJECT_CONNECTIONS 2728 * tests/test--https-enforce-soft2.c: Use WGET_TEST_HTTP_REJECT_CONNECTIONS 2729 * tests/test--https-enforce-soft3.c: Remove space (cleanup only) 2730 27312019-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 2732 2733 * tests/test--https-enforce-hard2.c: Allow error code 4 and 5 2734 27352019-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 2736 2737 * tests/libtest.c: Add WGET_TEST_EXPECTED_ERROR_CODE2 2738 27392019-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 2740 2741 * tests/test--https-enforce-hard3.c: Make test behave stable 2742 27432019-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 2744 2745 * tests/test--https-enforce-hard2.c: Make test behave stable 2746 27472019-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 2748 2749 Add WGET_TEST_HTTP_REJECT_CONNECTIONS to test suite 2750 2751 * tests/libtest.h: Add WGET_TEST_HTTP_REJECT_CONNECTIONS 2752 * tests/libtest.c: Implement WGET_TEST_HTTP_REJECT_CONNECTIONS 2753 27542019-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 2755 2756 * libwget/iri.c (wget_iri_set_scheme): Fix amending https port 2757 27582019-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 2759 2760 * libwget/Makefile.am: Fix 'make code-coverage-capture' for gcc 9 2761 27622019-08-20 Tim Rühsen <tim.ruehsen@gmx.de> 2763 2764 * libwget/buffer.c (wget_buffer_deinit): Fix NONNULL issue 2765 27662019-08-20 Tim Rühsen <tim.ruehsen@gmx.de> 2767 2768 * src/wget.c (sitemap_parse_xml_gz): Use stack memory for wget_buffer 2769 27702019-08-20 Tim Rühsen <tim.ruehsen@gmx.de> 2771 2772 * libwget/xml.c (wget_xml_parse_file): Use stack memory for wget_buffer 2773 27742019-08-20 Tim Rühsen <tim.ruehsen@gmx.de> 2775 2776 * libwget/css.c (wget_css_parse_file): Use stack memory for wget_buffer 2777 27782019-08-20 Tim Rühsen <tim.ruehsen@gmx.de> 2779 2780 * libwget/buffer.c: Add examples to documentation 2781 27822019-08-19 Tim Rühsen <tim.ruehsen@gmx.de> 2783 2784 Cleanup + amend return vales of buffer functions 2785 2786 * include/wget/wget.h: Add wget_buffer.error 2787 * libwget/buffer.c: Amend docs, 2788 change return type of wget_buffer_init(), 2789 amend wget_buffer_alloc(), 2790 set buf.error in _buffer_realloc() 2791 * libwget/printf.c: Check buf.error, return (size_t)-1 2792 * unit-tests/test.c: Amend tests appropriately 2793 27942019-08-19 Tim Rühsen <tim.ruehsen@gmx.de> 2795 2796 * libwget/buffer.c (_buffer_realloc): Make use of realloc() if possible 2797 27982019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2799 2800 Fix more docs [skip ci] 2801 28022019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2803 2804 Fix docs for wget_bitmap_init() [skip ci] 2805 28062019-08-14 DalmeGNU <dalmemail@gmail.com> 2807 2808 Improvements on the progress bar 2809 2810 * docs/wget2.md: There is no default progress value 2811 * src/options.c: If no progress type specifier is given return an error 2812 * src/wget.c: Set global file size and already downloaded amount when continuing a download 2813 28142019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2815 2816 Fix checking return value of wget_vector_find() 2817 2818 * src/options.c: Likewise 2819 * src/wget.c: Likewise 2820 28212019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2822 2823 * .gitlab-ci.yml: Fix clang build on FreeBSD 2824 28252019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2826 2827 * libwget/bitmap.c: Small cleanup 2828 28292019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2830 2831 * examples/Makefile.am: Add -DMALLOC_RETURNS_NONNULL to AM_CFLAGS and AM_CPPFLAGS 2832 28332019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2834 2835 Introduce WGET_E_IO 2836 2837 * include/wget/wget.h: Introduce WGET_E_IO 2838 * libwget/error.c: Add WGET_E_IO to wget_strerror() 2839 * libwget/hashfile.c: Make use of WGET_E_IO 2840 * libwget/io.c: Likewise 2841 28422019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2843 2844 * libwget/http.c (wget_http_match_no_proxy): Check value from wget_vector_get() 2845 28462019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2847 2848 Rename WGET_E_OPEN_FILE -> WGET_E_OPEN 2849 2850 * include/wget/wget.h: Rename WGET_E_OPEN_FILE -> WGET_E_OPEN 2851 * libwget/io.c: Likewise 2852 * libwget/netrc.c: Likewise 2853 28542019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2855 2856 Cleanup vector functions and docs 2857 28582019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2859 2860 * libwget/test_linking.c: Don't give NULL to wget_plugin_get_name() 2861 28622019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2863 2864 * libwget/http_parse.c (wget_http_parse_response_header): Check memory allocation 2865 28662019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2867 2868 * libwget/http_highlevel.c (wget_http_get): Check vector creation 2869 28702019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2871 2872 * libwget/http.c (wget_http_create_request): Check memory allocation 2873 28742019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2875 2876 * libwget/hpkp_db.c (hpkp_db_load): Check memory allocations 2877 28782019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2879 2880 * libwget/vector.c: Check memory allocation 2881 28822019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2883 2884 * include/wget/wget.h: Add NULLABLE to more functions 2885 28862019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2887 2888 * libwget/io.c: Fix return values, add memory check 2889 28902019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2891 2892 * src/job.c (job_validate_file): Fix check for wget_truncate() return 2893 28942019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2895 2896 * gnulib: Update 2897 28982019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2899 2900 * libwget/iri.c (wget_iri_parse): Avoid casts to suppress warning 2901 29022019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2903 2904 * libwget/iri.c (wget_iri_parse): Return NULL on unknown scheme 2905 29062019-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 2907 2908 * libwget/iri.c (wget_iri_parse): Set scheme to -1 if unknown 2909 29102019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2911 2912 * libwget/netrc.c (wget_netrc_db_load): Check memory allocation 2913 29142019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2915 2916 * libwget/http_parse.c (wget_http_parse_header_line): Check memory allocation 2917 29182019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2919 2920 * libwget/http.c (_parse_no_proxies): Check memory allocation 2921 29222019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2923 2924 * libwget/cookie_parse.c (cookie_normalize_cookie): Check memory allocation 2925 29262019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2927 2928 * examples/batch_loader.c: Check return value of wget_strmemdup() 2929 29302019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2931 2932 * examples/check_url_types.c: Check return value of wget_strmemdup() 2933 29342019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2935 2936 * include/wget/wget.h: Nullable wget_memdup, wget_strdup, wget_strmemdup 2937 29382019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2939 2940 * .gitlab-ci.yml (Debian/HTTP2): Allow failure set to true 2941 29422019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2943 2944 * unit-tests/Makefile.am: Add -DMALLOC_RETURNS_NONNULL to AM_CFLAGS and AM_CPPFLAGS 2945 29462019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2947 2948 * unit-tests/test-dl.c (string_vector_check): Fix va_list leak 2949 29502019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2951 2952 * libwget/tls_session.c: Check memory allocations 2953 29542019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2955 2956 * libwget/http_parse.c: Check memory allocations 2957 29582019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2959 2960 * libwget/netrc.c: Check memory allocations 2961 29622019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2963 2964 * libwget/ocsp.c: Check memory allocations 2965 29662019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2967 2968 * libwget/hsts.c: Check memory allocations 2969 29702019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2971 2972 * libwget/bar.c: Check memory allocations 2973 29742019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2975 2976 * libwget/io.c: Check memory allocations 2977 29782019-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 2979 2980 * libwget/encoding.c (wget_memiconv): Check memory allocations, return WGET_E_* codes 2981 29822019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 2983 2984 * libwget/net.c (wget_tcp_init): Check malloc return value 2985 29862019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 2987 2988 Introduce nullability support for clang's static analyzer 2989 2990 * configure.ac: Add -Wno-nullability-extension -Wno-nullability-completeness to WARN_CFLAGS 2991 * include/wget/wget.h: Add CLANG_VERSION_AT_LEAST, 2992 fix attribute defines for clang, 2993 make use of attributes by using static inline functions for wget allocation, 2994 define NULLABLE for clang 2995 * libwget/xalloc.c: Rename wget allocation function pointers 2996 * src/options.c: Likewise 2997 * unit-tests/test.c: Likewise 2998 29992019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3000 3001 * libwget/http_highlevel.c (wget_http_get): Fix va_list leakage 3002 30032019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3004 3005 * .gitlab-ci.yml: Use clang and enable valist checker with scan-build 3006 30072019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3008 3009 * libwget/test_linking.c: Fix clang warning (needless semicolon) 3010 30112019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3012 3013 * libwget/test_linking.c: Make use of return value of wget_strdup() 3014 30152019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3016 3017 * libwget/Makefile.am: Add -DBUILDING_LIBWGET to AM_CFLAGS 3018 30192019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3020 3021 * include/wget/wget.h: Define WGET_GCC_RETURNS_NONNULL for clang as well 3022 30232019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3024 3025 * libwget/ssl_gnutls.c (wget_ssl_open): Avoid 0 size VLA 3026 30272019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3028 3029 * src/wget.c (http_send_request): Fix possible NULL ptr issue 3030 30312019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3032 3033 * tests/libtest.c (_parse_hostname): Silence clang's scan-build 3034 30352019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3036 3037 * .gitlab-ci.yml (FreeBSD.x86_64): Do not retry after failure 3038 30392019-05-18 Tim Rühsen <tim.ruehsen@gmx.de> 3040 3041 Add convenience library for src/utils.c 3042 3043 * src/Makefile.am: Add libtest-utils.la 3044 * tests/Makefile.am: Add libtest-utils.la to libtest_la_LIBADD 3045 30462019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3047 3048 * src/wget.c (_add_authorize_header): Print error if wget_netrc_db_load() fails 3049 30502019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3051 3052 * libwget/netrc.c: Code cleanup 3053 30542019-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 3055 3056 Add new error code WGET_E_OPEN_FILE 3057 30582019-08-14 Tim Rühsen <tim.ruehsen@gmx.de> 3059 3060 * libwget/metalink.c: Make code more C99 compliant 3061 30622019-08-14 Tim Rühsen <tim.ruehsen@gmx.de> 3063 3064 Split hpkp.c into hpkp_db.c, hpkp.c and hpkp.h 3065 3066 * libwget/Makefile.am: Add hpkp.h and hpkp_db.c 3067 * libwget/hpkp.c: Move HPKP database code to hpkp_db.c 3068 * libwget/hpkp.h: New file 3069 * libwget/hpkp_db.c: New file 3070 30712019-08-14 Tim Rühsen <tim.ruehsen@gmx.de> 3072 3073 Split cookie.c into cookie_parse.c, cookie.c and cookie.h 3074 3075 * libwget/Makefile.am: Add cookie.h and cookie_parse.c 3076 * libwget/cookie.c: Move cookie parsing code to cookie_parse.c 3077 * libwget/cookie.h: New file 3078 * libwget/cookie_parse.c: New file 3079 30802019-08-14 DalmeGNU <dalmemail@gmail.com> 3081 3082 * src/options.c: Don't accept negative numbers as a valid byte amount 3083 30842019-08-13 DalmeGNU <dalmemail@gmail.com> 3085 3086 Implement --start-pos option 3087 3088 * src/wget.c: Added --start-pos implementation 3089 * src/options.c: Added --start-pos option and resolved flag combination with --continue 3090 * src/wget_options.h: Added start_pos to the config structure 3091 * docs/wget2.md: Removed "Not Implemented Yet" label from --start-pos documentation 3092 30932019-08-14 Tim Rühsen <tim.ruehsen@gmx.de> 3094 3095 * libwget/iri.c: Cast enums to avoid warnings with clang 3096 30972019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 3098 3099 Change return type of wget_hpkp_get_n_pins() to int 3100 3101 * include/wget/wget.h: Change return type of wget_hpkp_get_n_pins() to int 3102 * libwget/hpkp.c: Likewise 3103 * unit-tests/test.c: Adjust code, cleanup 3104 31052019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 3106 3107 Remove global wget_iri_schemes[], add wget_iri_scheme enum 3108 3109 * include/wget/wget.h: Remove global wget_iri_schemes[], 3110 add wget_iri_scheme enum 3111 * libwget/iri.c: Add struct iri_scheme, 3112 remove wget_iri_schemes and iri_ports, 3113 new function wget_iri_scheme_get_name(), 3114 fix code 3115 * examples/check_url_types.c: Use comparison instead of wget_strcasecmp 3116 * fuzz/libwget_iri_fuzzer.c: Use WGET_IRI_SCHEME_HTTPS instead of string 3117 * libwget/http.c: Use wget_iri_scheme_get_name() 3118 * libwget/http.h: Change scheme from string to wget_iri_scheme 3119 * libwget/http_parse.c: Fix wget_http_get_scheme() 3120 * src/blacklist.c: Fix hash_iri() 3121 * src/host.c: Fix _host_hash() 3122 * src/options.c: Use WGET_IRI_SCHEME_* instead of string 3123 * src/stats_server.c: Use wget_iri_scheme for scheme member 3124 * src/wget.c: Fix code 3125 * src/wget_host.h: Use wget_iri_scheme fro scheme member 3126 * unit-tests/test.c: Fix tests 3127 31282019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 3129 3130 Use more bool in HPKP code 3131 3132 * include/wget/wget.h: Use bool for HPKP functions where possible 3133 * libwget/hpkp.c: Use bool where possible 3134 * libwget/http_parse.c: Use bool for HPKP functions 3135 31362019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 3137 3138 libwget/tls_session.c: Improve C99 compliance and cleanup 3139 31402019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 3141 3142 Use more bool in OCSP code 3143 3144 * include/wget/wget.h (wget_ocsp_db_add_fingerprint_fn): bool for last param 3145 * libwget/ocsp.c: Use bool where possible 3146 * libwget/ssl_gnutls.c: Use bool for OCSP functions 3147 31482019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 3149 3150 libwget/ocsp.c: Improve C99 compliance and cleanup 3151 31522019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 3153 3154 libwget/hsts.c: Improve C99 compliance and cleanup 3155 31562019-08-13 Tim Rühsen <tim.ruehsen@gmx.de> 3157 3158 libwget/hpkp.c: Improve C99 compliance and cleanup 3159 31602019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3161 3162 * .gitlab-ci.yml: Add FreeBSD runner 3163 31642019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3165 3166 Remove variadic macro for wget_printf() 3167 3168 * include/wget/wget.h: Remove variadic macro for wget_printf(), 3169 add prototype for wget_printf() 3170 * libwget/printf.c Implement wget_printf(), 3171 fix some wrongly documented return types 3172 31732019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3174 3175 * tests/libtest.c: Make post_handshake_auth static 3176 31772019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3178 3179 Move server stats code from library into wget2 tool 3180 3181 * fuzz/Makefile.am: Add src/stats_server.c 3182 * src/Makefile.am: Likewise 3183 * unit-tests/Makefile.am: Likewise 3184 * include/wget/wget.h: Move `wget_server_stats_data` to src/stats_server.c, 3185 change wget_server_stats_callback, 3186 change wget_server_set_stats_callback(), 3187 add wget_tcp_get_ip(), 3188 remove host_ips_free() 3189 * libwget/http.c: Move stats code to src/stats_server.c 3190 * libwget/net.c: Add wget_tcp_get_ip() 3191 * src/options.c: Move _hpkp_string() and stats_callback_server() to src/stats_server.c 3192 * src/wget.c (main): Remove call to host_ips_free() 3193 * src/wget_stats.h: Add prototypes of stats_server_add, server_stats_init, server_stats_exit 3194 31952019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3196 3197 Fix variables to wget_stats_format 3198 3199 * src/options.c (parse_stats): Use wget_stats_format instead of char 3200 * src/wget_options.h (stats_args): Likewise 3201 32022019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3203 3204 Fix variable types of wget_report_speed 3205 3206 * src/options.c (parse_report_speed_type): Fix pointer type 3207 * src/wget_options.h (struct config): Amend type of report_speed 3208 32092019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3210 3211 Add typedef 'wget_transfer_encoding' 3212 3213 * include/wget/wget.h: Add typedef 'wget_transfer_encoding' 3214 * libwget/http_parse.c: Make use of wget_transfer_encoding 3215 32162019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3217 3218 Rename HTTP parse callback types 3219 3220 * include/wget/wget.h: Rename HTTP parse callback types 3221 * examples/getstream.c: Likewise 3222 * libwget/http.c: Likewise 3223 * libwget/http_highlevel.c: Likewise 3224 32252019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3226 3227 Rename wget_xml_callback_t -> wget_xml_callback 3228 3229 * include/wget/wget.h: Rename wget_xml_callback_t -> wget_xml_callback 3230 * libwget/xml.c: Likewise 3231 32322019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3233 3234 Rename CSS parse callback types 3235 3236 * include/wget/wget.h: Rename CSS parse callback types 3237 * libwget/css.c: Likewise 3238 32392019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3240 3241 Rename wget_decompressor types 3242 3243 * include/wget/wget.h: Rename wget_decompressor types 3244 * libwget/decompressor.c: Likewise 3245 * libwget/http.c: Likewise 3246 * src/options.c: Likewise 3247 * src/wget_options.h: Likewise 3248 32492019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3250 3251 Rename wget_stringmap types 3252 3253 * include/wget/wget.h: Rename wget_stringmap types 3254 * unit-tests/stringmap_perf.c: Likewise 3255 * unit-tests/test.c: Likewise 3256 32572019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3258 3259 Rename wget_hashmap types 3260 3261 * include/wget/wget.h: Rename wget_hashmap types 3262 * libwget/dns_cache.c: Likewise 3263 * libwget/hashmap.c: Likewise 3264 * libwget/hpkp.c: Likewise 3265 * libwget/hsts.c: Likewise 3266 * libwget/http.c: Likewise 3267 * libwget/netrc.c: Likewise 3268 * libwget/ocsp.c: Likewise 3269 * libwget/stringmap.c: Likewise 3270 * libwget/tls_session.c: Likewise 3271 * src/blacklist.c: Likewise 3272 * src/host.c: Likewise 3273 * src/wget.c: Likewise 3274 32752019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3276 3277 Rename wget_vector_destructor_t -> wget_vector_destructor 3278 3279 * include/wget/wget.h: Rename wget_vector_destructor_t -> wget_vector_destructor 3280 * libwget/http.c: Likewise 3281 * libwget/http_parse.c: Likewise 3282 * libwget/vector.c: Likewise 3283 * src/options.c: Likewise 3284 * src/plugin.c: Likewise 3285 * unit-tests/test.c: Likewise 3286 32872019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3288 3289 Rename wget_vector_browse_t -> wget_vector_browse_fn 3290 3291 * include/wget/wget.h: Rename wget_vector_browse_t -> wget_vector_browse_fn 3292 * libwget/hpkp.c: Likewise 3293 * libwget/vector.c: Likewise 3294 * src/stats_site.c: Likewise 3295 32962019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3297 3298 Rename wget_vector_find_t -> wget_vector_find_fn 3299 3300 * include/wget/wget.h: Rename wget_vector_find_t -> wget_vector_find_fn 3301 * libwget/vector.c: Likewise 3302 33032019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3304 3305 Rename wget_vector_compare_t -> wget_vector_compare_fn 3306 3307 * include/wget/wget.h: Rename wget_vector_compare_t -> wget_vector_compare_fn 3308 * libwget/cookie.c: Likewise 3309 * libwget/hpkp.c: Likewise 3310 * libwget/http_parse.c: Likewise 3311 * libwget/metalink.c: Likewise 3312 * libwget/vector.c: Likewise 3313 * src/options.c: Likewise 3314 * tests/test-plugin-dummy.c: Likewise 3315 * unit-tests/test-dl.c: Likewise 3316 * unit-tests/test.c: Likewise 3317 33182019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3319 3320 Rename wget_logger_func_t -> wget_logger_func 3321 3322 * include/wget/wget.h: Rename wget_logger_func_t -> wget_logger_func 3323 * libwget/init.c: Likewise 3324 * libwget/logger.c: Likewise 3325 33262019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3327 3328 * include/wget/wget.h (wget_buffer): Fix typo 3329 33302019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3331 3332 Rename wget_list_browse_t -> wget_list_browse_fn 3333 3334 * include/wget/wget.h: Rename wget_list_browse_t -> wget_list_browse_fn 3335 * libwget/list.c: Likewise 3336 * src/host.c: Likewise 3337 33382019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3339 3340 Rename wget_update_(load|save)_t -> wget_update_(load|save)_fn 3341 3342 * include/wget/wget.h: Rename wget_update_(load|save)_t -> wget_update_(load|save)_fn 3343 * libwget/cookie.c: Likewise 3344 * libwget/hpkp.c: Likewise 3345 * libwget/hsts.c: Likewise 3346 * libwget/io.c: Likewise 3347 * libwget/tls_session.c: Likewise 3348 33492019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3350 3351 Rename wget_global_get_func_t -> wget_global_func 3352 3353 * include/wget/wget.h: Rename wget_global_get_func_t -> wget_global_func 3354 * libwget/init.c: Likewise 3355 33562019-08-12 Tim Rühsen <tim.ruehsen@gmx.de> 3357 3358 examples/http_get2.c: Cleanup 3359 33602019-08-09 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3361 3362 * tests/test-ocsp-server: Modified to test without OCSP responder's URL 3363 33642019-08-10 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3365 3366 * tests/libtest.c: Freed read response in-case no request is received by the responder 3367 33682019-08-09 Tim Rühsen <tim.ruehsen@gmx.de> 3369 3370 Name unnamed enum to wget_report_speed 3371 3372 * include/wget/wget.h: Name unnamed enum to wget_report_speed, 3373 use wget_report_speed for param of wget_bar_set_speed_type() 3374 * libwget/bar.c: use wget_report_speed for param of wget_bar_set_speed_type() 3375 33762019-08-09 Tim Rühsen <tim.ruehsen@gmx.de> 3377 3378 * libwget/error.c (wget_strerror): Add missing WGET_E_MEMORY handling 3379 33802019-08-09 Tim Rühsen <tim.ruehsen@gmx.de> 3381 3382 wget_strerror(int) -> wget_strerror(wget_error) 3383 3384 * include/wget/wget.h: wget_strerror(int) -> wget_strerror(wget_error) 3385 * libwget/error.c: Likewise 3386 33872019-08-09 Tim Rühsen <tim.ruehsen@gmx.de> 3388 3389 * include/wget/wget.h: Rename wget_error_t -> wget_error 3390 33912019-08-09 Tim Rühsen <tim.ruehsen@gmx.de> 3392 3393 * include/wget/wget.h: Remove unused commented code 3394 33952019-08-09 Tim Rühsen <tim.ruehsen@gmx.de> 3396 3397 * include/wget/wget.h: Remove unused WGET_PREFIX 3398 33992019-08-09 Tim Rühsen <tim.ruehsen@gmx.de> 3400 3401 * libwget/ssl_gnutls.c (wget_ssl_init): Print priorities correctly on error 3402 34032019-08-08 Tim Rühsen <tim.ruehsen@gmx.de> 3404 3405 Changed G_GNUC_WGET prefix to WGET_GCC 3406 3407 * include/wget/wget.h: Changed G_GNUC_WGET prefix to WGET_GCC 3408 * */*.[ch]: Likewise 3409 34102019-08-08 Tim Rühsen <tim.ruehsen@gmx.de> 3411 3412 Add examples/relative_to_absolute_url.c 3413 34142019-08-08 Tim Rühsen <tim.ruehsen@gmx.de> 3415 3416 Allow -1 as len parameter to wget_iri_relative_to_abs() 3417 3418 * libwget/http_highlevel.c (wget_http_get): Use -1 as len 3419 * libwget/iri.c (wget_iri_parse_base): Likewise 3420 * src/wget.c (process_response_header): Likewise 3421 * unit-tests/test.c (test_iri_relative_to_absolute): Likewise 3422 34232019-08-07 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3424 3425 * tests/test-ocsp-stap.c: Removed redundant comment 3426 34272019-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 3428 3429 * .gitlab-ci.yml: Save coverage for HTTP/2 runner as artifact 3430 34312019-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 3432 3433 * unit-tests/Makefile.am: Add CODE_COVERAGE flags 3434 34352019-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 3436 3437 Rename stats types (remove trailing _t) 3438 3439 * include/wget/wget.h: Rename stats types (remove trailing _t) 3440 * libwget/dns.c: Likewise 3441 * libwget/http.c: Likewise 3442 * libwget/net.h: Likewise 3443 * libwget/ssl_gnutls.c: Likewise 3444 * libwget/ssl_none.c: Likewise 3445 * libwget/ssl_wolfssl.c: Likewise 3446 * src/options.c: Likewise 3447 34482019-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 3449 3450 * include/wget/wget.h: Fix type in two comments 3451 34522019-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 3453 3454 Rename wget_hsts_db_t -> wget_hsts_db 3455 3456 * include/wget/wget.h: Rename wget_hsts_db_t -> wget_hsts_db, 3457 * fuzz/libwget_hsts_fuzzer.c: Likewise 3458 * libwget/hsts.c: Likewise 3459 * src/wget_options.h: Likewise 3460 * tests/test-plugin-dummy.c: Likewise 3461 * unit-tests/test.c: Likewise 3462 34632019-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 3464 3465 * include/wget/wget.h: Replace outdated docs for plugins 3466 34672019-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 3468 3469 Rename wget_hpkp_db_t -> wget_hpkp_db and wget_hpkp_t -> wget_hpkp 3470 3471 * include/wget/wget.h: Rename wget_hpkp_db_t -> wget_hpkp_db, 3472 rename wget_hpkp_t -> wget_hpkp 3473 * fuzz/libwget_hpkp_fuzzer.c: Likewise 3474 * libwget/hpkp.c: Likewise 3475 * libwget/http_parse.c: Likewise 3476 * libwget/ssl_gnutls.c: Likewise 3477 * libwget/ssl_wolfssl.c: Likewise 3478 * src/wget_options.h: Likewise 3479 * tests/test-plugin-dummy.c: Likewise 3480 * unit-tests/test.c: Likewise 3481 34822019-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 3483 3484 Rework HPKP DB plugin code 3485 3486 * include/wget/wget.h: Add typedefs for HPKP DB functions, 3487 remove wget_hpkp_db_st, 3488 rewrite wget_hpkp_db_vtable, 3489 add wget_hpkp_set_plugin() 3490 * libwget/hpkp.c: Add one function table for all exposed functions, 3491 implement wget_hpkp_set_plugin(), 3492 call plugin functions if set 3493 * libwget/plugin.c: Remove wget_plugin_add_hpkp_db() 3494 * src/plugin.c: Remove impl_add_hpkp_db() 3495 * tests/test-plugin-dummy.c: Implement dummy hsts db API, 3496 initialize test_hpkp_db_vtable, 3497 call wget_hpkp_set_plugin() in plugin initializer 3498 34992019-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 3500 3501 Rework OCSP DB plugin code 3502 3503 * include/wget/wget.h: Add typedefs for OCSP DB functions, 3504 remove wget_ocsp_db_st, 3505 rewrite wget_ocsp_db_vtable, 3506 add wget_ocsp_set_plugin() 3507 * libwget/ocsp.c: Add one function table for all exposed functions, 3508 implement wget_ocsp_set_plugin(), 3509 call plugin functions if set 3510 * libwget/plugin.c: Remove wget_plugin_add_ocsp_db() 3511 * src/plugin.c: Remove impl_add_ocsp_db() 3512 * tests/test-plugin-dummy.c: Implement dummy hsts db API, 3513 initialize test_ocsp_db_vtable, 3514 call wget_ocsp_set_plugin() in plugin initializer 3515 35162019-08-06 Tim Rühsen <tim.ruehsen@gmx.de> 3517 3518 Rework HSTS DB plugin code 3519 3520 * include/wget/wget.h: Ad typedefs for HSTS DB functions, 3521 remove wget_hsts_db_st, 3522 rewrite wget_hsts_db_vtable, 3523 add wget_hsts_set_plugin() 3524 * libwget/hsts.c: Add one function table for all exposed functions, 3525 implement wget_hsts_set_plugin(), 3526 call plugin functions if set 3527 * libwget/plugin.c: Remove wget_plugin_add_hsts_db() 3528 * src/plugin.c: Remove impl_add_hsts_db() 3529 * tests/test-plugin-dummy.c: Implement dummy hsts db API, 3530 initialize test_hsts_db_vtable, 3531 call wget_hsts_set_plugin() in plugin initializer 3532 35332019-08-05 Tim Rühsen <tim.ruehsen@gmx.de> 3534 3535 Rename plugin types 3536 3537 * include/wget/wget.h: Rename plugin types 3538 * libwget/plugin.c: Likewise 3539 * src/plugin.c: Likeiwse 3540 * src/wget_plugin.h: Likewise 3541 * tests/test-plugin-dummy.c: Likewise 3542 35432019-08-05 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3544 3545 Added support for ax_code_coverage version 32 3546 3547 * Makefile.am: Add code coverage rules as documented in ax_code_coverage.m4 3548 * cfg.mk: Exclude m4 files from sc_GPL_version 3549 * configure.ac: Add AX_CODE_COVERAGE 3550 * fuzz/Makefile.am: Add coverage flags 3551 * libwget/Makefile.am: Likewise 3552 * src/Makefile.am: Likewise 3553 * tests/Makefile.am: Likewise 3554 * m4/ax_ac_append_to_file.m4: New file 3555 * m4/ax_ac_print_to_file.m4: New file 3556 * m4/ax_add_am_macro_static.m4: New file 3557 * m4/ax_am_macros_static.m4: New file 3558 * m4/ax_check_gnu_make.m4: New file 3559 * m4/ax_code_coverage.m4: New file 3560 * m4/ax_file_escapes.m4: New file 3561 35622019-08-04 DalmeGNU <dalmemail@gmail.com> 3563 3564 Background support for fork() compatible OS 3565 3566 * src/wget.c: Added fork_to_background(), in order to implement --background 3567 * src/options.c: Added "--background" to the options list 3568 * src/wget_options.h: Added background member to structure config 3569 * docs/wget2.md: Background option marked as "Not supported on Windows" 3570 35712019-08-04 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3572 3573 Changed validity time for OCSP Stapled Response 3574 3575 * tests/certs/ocsp/generate_stap.sh: Modified to generate longer validity response 3576 * tests/certs/ocsp/ocsp_stapled_resp.der: Regenerated for longer validity 3577 35782019-07-27 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3579 3580 * tests/libtest.c: Resolved tests failure due to OCSP 3581 35822019-06-22 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3583 3584 Add test-limit-rate-http2 3585 3586 * tests/Makefile.am: Added test-limit-rate-http2 3587 * tests/test-limit-rate-http2.c: Cloned 'test-limit-rate' to test for h2 only 3588 35892019-06-22 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3590 3591 * .gitlab-ci.yml: Add Gitlab CI runner for HTTP/2 3592 35932019-06-21 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3594 3595 Making failing tests work on Second Pass 3596 3597 * tests/test--https-enforce-soft3.c: 3598 * tests/test-bad-chunk.c: 3599 * tests/test-chunked.c: 3600 * tests/test-limit-rate.c: 3601 * tests/test-post-handshake-auth.c: Skipped for Second Pass 3602 3603 * tests/test-i-https.c: Added '--no-ca-certificate' in options for expected failure for h2 3604 3605 * tests/test-idn-cmd.c: 3606 * tests/test-idn-meta.c: 3607 * tests/test-idn-robots.c: Dynamic port replacement for http/https proxies 3608 3609 * tests/test-iri.c: 3610 * tests/test-parse-css.c: 3611 * tests/test-parse-html-css.c: Dynamic port replacement for hostname 3612 36132019-06-21 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3614 3615 Adding support for running both http/1.1 and h2 tests 3616 3617 * configure.ac: Modified to have check for availability of MHD-http2 3618 * tests/libtest.c: Modified to run both protocols on existing tests 3619 * tests/libtest.h: Added macros for h2 and modified wget_test_url_t data structure 3620 36212019-06-20 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3622 3623 * tests/libtest.c: Modified Range Header key check to be case-insensitive 3624 36252019-06-20 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3626 3627 * tests/libtest.c: Modified to have compatibility with MHD-http2 3628 36292019-07-25 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3630 3631 * tests/test-ocsp-server.c: Added Revoked OCSP test with --no-check-certificate 3632 36332019-07-25 Tim Rühsen <tim.ruehsen@gmx.de> 3634 3635 * examples/getstream.c: Fix call to wget_http_get() 3636 36372019-07-25 Tim Rühsen <tim.ruehsen@gmx.de> 3638 3639 Fix parsing Location header with trailign digits 3640 3641 * libwget/http_parse.c (wget_http_parse_location): Replace 3642 c_isdigits() by c_isblank() 3643 36442019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3645 3646 * tests/certs/ocsp/README.md: Added README for generating generating certificates and response 3647 36482019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3649 3650 Splitting certs/resp generator script 3651 3652 * tests/certs/ocsp/generate_certs.sh: Added to generate certificates 3653 * tests/certs/ocsp/generate_resp.sh: Added to generate stapled response 3654 36552019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3656 3657 * tests/libtest.c: Removed redundant priority from MHD daemon initialization 3658 36592019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3660 3661 * tests/libtest.c: Added file loading checks for OCSP Stapling PEM/DER files 3662 36632019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3664 3665 * tests/libtest.c: Moved PEM/DER loading out of _ocsp_stap_cert_callback() 3666 36672019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3668 3669 * tests/libtest.c: Dynamically allocating 'privkey_stap' 3670 36712019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3672 3673 * tests/libtest.c: Enforced gnutls_certificate_retrieve_function3 type for _ocsp_stap_cert_callback() 3674 36752019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3676 3677 * tests/libtest.c: Removed ampersand from MHD_OPTION_HTTPS_CERT_CALLBACK2 value 3678 36792019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3680 3681 Added templates, records, and script for OCSP stapling PEM/DER files 3682 3683 * tests/certs/ocsp/demoCA/*: Openssl certificate signing records for root and intermediate 3684 * tests/certs/ocsp/generate_stap.sh: Added script for generating required certificates and response 3685 * tests/certs/ocsp/interm-template.txt: Added template for intermediate certificate 3686 * tests/certs/ocsp/interm.cnf: Added openssl config for intermediate certificate signing 3687 * tests/certs/ocsp/root-template.txt: Added template for root certificate 3688 * tests/certs/ocsp/server-template.txt: Added template for server certificate 3689 36902019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3691 3692 * tests/libtest.c: Moved break into preprocessor else block 3693 36942019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3695 3696 Added required certificates and response 3697 3698 * tests/certs/ocsp/ocsp_stapled_resp.der: Added OCSP stapled response 3699 * tests/certs/ocsp/x509-interm-cert.pem: Added intermediate certificate 3700 * tests/certs/ocsp/x509-interm-key.pem: Added intermediate private key 3701 * tests/certs/ocsp/x509-root-cert.pem: Added root CA certificate 3702 * tests/certs/ocsp/x509-root-key.pem: Added root CA private key 3703 * tests/certs/ocsp/x509-server-cert.pem: Added server certificate 3704 * tests/certs/ocsp/x509-server-key.pem: Added server private key 3705 37062019-07-24 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3707 3708 Added skeleton OCSP Stapling 3709 3710 * tests/Makefile.am: Added test-ocsp-stap 3711 * tests/libtest.c: Added an MHD daemon to send OCSP stapled response 3712 * tests/libtest.h: Added macro WGET_TEST_FEATURE_OCSP_STAPLING 3713 * tests/test-ocsp-stap.c: Added test for OCSP stapling 3714 37152019-07-23 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3716 3717 * tests/certs/ocsp/README.md: Added README for generating generating certificates and responses 3718 37192019-07-23 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3720 3721 Splitting certs/resps generator script 3722 3723 * tests/certs/ocsp/generate_certs.sh: Added to generate certificates 3724 * tests/certs/ocsp/generate_resp.sh: Added to generate responses 3725 37262019-07-21 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3727 3728 Making OCSP Response Global 3729 3730 * tests/libtest.c: Added 'struct ocsp_resp_t' and moved OCSP response to global scope 3731 37322019-07-21 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3733 3734 Added templates, records, and script for OCSP Responder PEM/DER files 3735 3736 * tests/certs/ocsp/demoCA/*: Openssl certificate signing records for root and intermediate 3737 * tests/certs/ocsp/generate_resp.sh: Added script for generating required certificates and responses 3738 * tests/certs/ocsp/interm-template.txt: Added template for intermediate certificate 3739 * tests/certs/ocsp/interm.cnf: Added openssl config for intermediate certificate signing 3740 * tests/certs/ocsp/root-template.txt: Added template for root certificate 3741 * tests/certs/ocsp/server-template.txt: Added template for server certificate 3742 37432019-07-21 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3744 3745 Added required certificates and response 3746 3747 * tests/certs/ocsp/ocsp_resp_ok.der: Added OCSP verified response 3748 * tests/certs/ocsp/ocsp_resp_revoked.der: Added OCSP revoked response 3749 * tests/certs/ocsp/x509-interm-cert.pem: Added intermediate certificate 3750 * tests/certs/ocsp/x509-interm-key.pem: Added intermediate private key 3751 * tests/certs/ocsp/x509-root-cert.pem: Added root CA certificate 3752 * tests/certs/ocsp/x509-root-key.pem: Added root CA private key 3753 * tests/certs/ocsp/x509-server-cert.pem: Added server certificate 3754 * tests/certs/ocsp/x509-server-key.pem: Added server private key 3755 37562019-07-21 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3757 3758 Added test-ocsp-server 3759 3760 * tests/Makefile.am: Modified to run the test 3761 * tests/test-ocsp-server: Added for testing OCSP verified/revoked responses 3762 37632019-07-20 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3764 3765 Added OCSP Responder to the test library 3766 3767 * tests/libtest.c: Added the skeleton responder 3768 * tests/libtest.h: Added required macros and function prototype 3769 37702019-07-20 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3771 3772 Added option --ocsp-date 3773 3774 * docs/wget2.md: Added documentation for --ocsp-date 3775 * include/wget/wget.h: Added macro WGET_SSL_OCSP_DATE 3776 * libwget/ssl_gnutls.c: Modified to use option 3777 * src/options.c: Added option handler 3778 * src/wget_options.h: Declared bool for option 3779 37802019-07-20 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3781 3782 Added option --ocsp-nonce 3783 3784 * docs/wget2.md: Added documentation for --ocsp-nonce 3785 * include/wget/wget.h: Added macro WGET_SSL_OCSP_NONCE 3786 * libwget/ssl_gnutls.c: Modified to use option 3787 * src/options.c: Added option handler 3788 * src/wget_options.h: Declared bool for option 3789 37902019-07-20 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 3791 3792 Added option --ocsp-server 3793 3794 * docs/wget2.md: Added documentation for --ocsp-server 3795 * libwget/ssl_gnutls.c: Modified to use value from option parameter 3796 * src/options.c: Added option handler 3797 * src/wget_options.h: Declared storage for option parameter 3798 37992019-07-23 Tim Rühsen <tim.ruehsen@gmx.de> 3800 3801 * libwget/ssl_gnutls.c (safe_ctime): Fix buffer size 3802 38032019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3804 3805 Rename wget_console_color_t -> wget_console_color 3806 3807 * include/wget/wget.h: Rename wget_console_color_t -> wget_console_color 3808 * libwget/console.c: Likewise 3809 * src/log.c: Likewise 3810 38112019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3812 3813 Rename wget_bar_t -> wget_bar 3814 3815 * include/wget/wget.h: Rename wget_bar_t -> wget_bar 3816 * fuzz/libwget_bar_fuzzer.c: Likewise 3817 * libwget/bar.c: Likewise 3818 * src/bar.c: Likewise 3819 * src/wget_bar.h: Likewise 3820 * unit-tests/test.c: Likewise 3821 38222019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3823 3824 Rename wget_robots_t -> wget_robots 3825 3826 * include/wget/wget.h: Rename wget_robots_t -> wget_robots 3827 * fuzz/libwget_robots_parse_fuzzer.c: Likewise 3828 * libwget/robots.c: Likewise 3829 * libwget/test_linking_robots.c: Likewise 3830 * src/wget_host.h: Likewise 3831 * unit-tests/test.c: Likewise 3832 38332019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3834 3835 Rename wget_metalink_*_t -> wget_metalink_* 3836 3837 * include/wget/wget.h: Rename wget_metalink_*_t -> wget_metalink_* 3838 * libwget/metalink.c: Likewise 3839 * src/job.c: Likewise 3840 * src/wget.c: Likewise 3841 * src/wget_job.h: Likewise 3842 38432019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3844 3845 Rename wget_digest_algorithm_t -> wget_digest_algorithm 3846 3847 * include/wget/wget.h: Rename wget_digest_algorithm_t -> wget_digest_algorithm 3848 * libwget/hash_printf.c: Likewise 3849 * libwget/hashfile.c: Likewise 3850 38512019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3852 3853 Rename wget_hash_hd_t -> wget_hash_hd 3854 3855 * include/wget/wget.h: Rename wget_hash_hd_t -> wget_hash_hd 3856 * libwget/hashfile.c: Likewise 3857 38582019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3859 3860 Rename wget_http_connection_t -> wget_http_connection 3861 3862 * include/wget/wget.h: Rename wget_http_connection_t -> wget_http_connection 3863 * examples/check_url_types.c: Likewise 3864 * examples/http_get.c: Likewise 3865 * examples/http_get2.c: Likewise 3866 * examples/http_multi_get.c: Likewise 3867 * examples/websequencediagram.c: Likewise 3868 * examples/websequencediagram_high.c: Likewise 3869 * fuzz/libwget_http_client_fuzzer.c: Likewise 3870 * libwget/http.c: Likewise 3871 * libwget/http_highlevel.c: Likewise 3872 * libwget/http_parse.c: Likewise 3873 * libwget/ssl_gnutls.c: Likewise 3874 * src/wget.c: Likewise 3875 * src/wget_job.h: Likewise 3876 38772019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3878 3879 Rename wget_http_request_t -> wget_http_request 3880 3881 * include/wget/wget.h: Rename wget_http_request_t -> wget_http_request 3882 * examples/http_get2.c: Likewise 3883 * examples/http_multi_get.c: Likewise 3884 * examples/websequencediagram.c: Likewise 3885 * fuzz/libwget_http_client_fuzzer.c: Likewise 3886 * libwget/http.c: Likewise 3887 * libwget/http_highlevel.c: Likewise 3888 * libwget/http_parse.c: Likewise 3889 * libwget/ssl_gnutls.c: Likewise 3890 * src/wget.c: Likewise 3891 * unit-tests/test.c: Likewise 3892 38932019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3894 3895 Rename wget_http_response_t -> wget_http_response 3896 3897 * include/wget/wget.h: Rename wget_http_response_t -> wget_http_response 3898 * examples/batch_loader.c: Likewise 3899 * examples/check_url_types.c: Likewise 3900 * examples/getstream.c: Likewise 3901 * examples/http_get.c: Likewise 3902 * examples/http_get2.c: Likewise 3903 * examples/http_multi_get.c: Likewise 3904 * examples/websequencediagram.c: Likewise 3905 * examples/websequencediagram_high.c: Likewise 3906 * fuzz/libwget_http_client_fuzzer.c: Likewise 3907 * fuzz/libwget_http_parse_fuzzer.c: Likewise 3908 * libwget/http.c: Likewise 3909 * libwget/http_highlevel.c: Likewise 3910 * libwget/http_parse.c: Likewise 3911 * libwget/ssl_gnutls.c: Likewise 3912 * src/gpgme.c: Likewise 3913 * src/stats_site.c: Likewise 3914 * src/wget.c: Likewise 3915 * src/wget_gpgme.h: Likewise 3916 * src/wget_stats.h: Likewise 3917 * tests/test-cookies-http_state.c: Likewise 3918 * unit-tests/test-cookies-http_state.c: Likewise 3919 * unit-tests/test.c: Likewise 3920 39212019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3922 3923 Rename transfer_encoding_* -> wget_transfer_encoding_* 3924 3925 * include/wget/wget.h: Rename transfer_encoding_* -> wget_transfer_encoding_* 3926 * libwget/http.c: Likewise 3927 * libwget/http_parse.c: Likewise 3928 39292019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3930 3931 Rename wget_http_challenge_t -> wget_http_challenge 3932 3933 * include/wget/wget.h: Rename wget_http_challenge_t -> wget_http_challenge 3934 * libwget/http.c: Likewise 3935 * libwget/http_parse.c: Likewise 3936 * src/options.c: Likewise 3937 * src/wget.c: Likewise 3938 * unit-tests/test.c: Likewise 3939 39402019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3941 3942 Rename wget_http_digest_t -> wget_http_digest 3943 3944 * include/wget/wget.h: Rename wget_http_digest_t -> wget_http_digest 3945 * libwget/http_parse.c: Likewise 3946 39472019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3948 3949 Rename wget_http_link_t -> wget_http_link 3950 3951 * include/wget/wget.h: Rename wget_http_link_t -> wget_http_link 3952 * libwget/http_parse.c: Likewise 3953 * src/wget.c: Likewise 3954 39552019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3956 3957 Rename wget_http_header_param_t -> wget_http_header_param 3958 3959 * include/wget/wget.h: Rename wget_http_header_param_t -> wget_http_header_param 3960 * libwget/http.c: Likewise 3961 * libwget/http_highlevel.c: Likewise 3962 * libwget/http_parse.c: Likewise 3963 * src/options.c: Likewise 3964 * src/wget.c: Likewise 3965 39662019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3967 3968 Rename wget_tcp_t -> wget_tcp 3969 3970 * include/wget/wget.h: Rename wget_tcp_t -> wget_tcp 3971 * libwget/http.h: Likewise 3972 * libwget/net.c: Likewise 3973 * libwget/ssl_gnutls.c: Likewise 3974 * libwget/ssl_none.c: Likewise 3975 * libwget/ssl_wolfssl.c: Likewise 3976 39772019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3978 3979 Rename wget_dns_t -> wget_dns 3980 3981 * include/wget/wget.h: Rename wget_dns_t -> wget_dns 3982 * libwget/dns.c: Likewise 3983 * libwget/net.c: Likewise 3984 * libwget/net.h: Likewise 3985 * src/options.c: Likewise 3986 39872019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3988 3989 Rename wget_dns_cache_t -> wget_dns_cache 3990 3991 * include/wget/wget.h: Rename wget_dns_cache_t -> wget_dns_cache 3992 * libwget/dns.c: Likewise 3993 * libwget/dns_cache.c: Likewise 3994 * libwget/init.c: Likewise 3995 * src/options.c: Likewise 3996 39972019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 3998 3999 Rename wget_html_tag_t -> wget_html_tag 4000 4001 * include/wget/wget.h: Rename wget_html_tag_t -> wget_html_tag 4002 * libwget/html_url.c: Likewise 4003 * src/options.c: Likewise 4004 40052019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 4006 4007 Rename wget_html_parsed_result_t -> wget_html_parsed_result 4008 4009 * include/wget/wget.h: Rename wget_html_parsed_result_t -> wget_html_parsed_result 4010 * examples/check_url_types.c: Likewise 4011 * examples/print_html_urls.c: Likewise 4012 * fuzz/libwget_html_url_fuzzer.c: Likewise 4013 * libwget/html_url.c: Likewise 4014 * src/wget.c: Likewise 4015 40162019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 4017 4018 Rename wget_html_parsed_url_t -> wget_html_parsed_url 4019 4020 * include/wget/wget.h: Rename wget_html_parsed_url_t -> wget_html_parsed_url 4021 * examples/check_url_types.c: Likewise 4022 * examples/print_html_urls.c: Likewise 4023 * libwget/html_url.c: Likewise 4024 * src/wget.c: Likewise 4025 40262019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 4027 4028 Rename wget_string_t -> wget_string 4029 4030 * include/wget/wget.h: Rename wget_string_t -> wget_string 4031 * examples/batch_loader.c: Likewise 4032 * examples/check_url_types.c: Likewise 4033 * examples/print_html_urls.c: Likewise 4034 * libwget/atom_url.c: Likewise 4035 * libwget/robots.c: Likewise 4036 * libwget/rss_url.c: Likewise 4037 * libwget/sitemap_url.c: Likewise 4038 * src/host.c: Likewise 4039 * src/wget.c: Likewise 4040 * unit-tests/test.c: Likewise 4041 40422019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 4043 4044 Rename wget_css_parsed_url_t -> wget_css_parsed_url 4045 4046 * include/wget/wget.h: Rename wget_css_parsed_url_t -> wget_css_parsed_url 4047 * examples/print_css_urls3.c: Likewise 4048 * libwget/css_url.c: Likewise 4049 40502019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 4051 4052 Rename wget_netrc_t -> wget_netrc 4053 4054 * include/wget/wget.h: Rename wget_netrc_t -> wget_netrc 4055 * libwget/netrc.c: Likewise 4056 * src/wget.c: Likewise 4057 * unit-tests/test.c: Likewise 4058 40592019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 4060 4061 Rename wget_netrc_db_t -> wget_netrc_db 4062 4063 * include/wget/wget.h: Rename wget_netrc_db_t -> wget_netrc_db 4064 * fuzz/libwget_netrc_fuzzer.c: Likewise 4065 * libwget/netrc.c: Likewise 4066 * src/wget_options.h: Likewise 4067 * unit-tests/test.c: Likewise 4068 40692019-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 4070 4071 Rename wget_ocsp_db_t -> wget_ocsp_db 4072 4073 * include/wget/wget.h: Rename wget_ocsp_db_t -> wget_ocsp_db 4074 * examples/http_multi_get.c: Likewise 4075 * fuzz/libwget_ocsp_fuzzer.c: Likewise 4076 * libwget/ocsp.c: Likewise 4077 * libwget/plugin.c: Likewise 4078 * libwget/ssl_gnutls.c: Likewise 4079 * libwget/ssl_wolfssl.c: Likewise 4080 * src/plugin.c: Likewise 4081 * src/wget_options.h: Likewise 4082 * src/wget_plugin.h: Likewise 4083 * tests/test-plugin-dummy.c: Likewise 4084 40852019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4086 4087 Rename wget_tls_session_db_t -> wget_tls_session_db 4088 4089 * include/wget/wget.h: Rename wget_tls_session_db_t -> wget_tls_session_db 4090 * fuzz/libwget_tlssess_fuzzer.c: Likewise 4091 * libwget/ssl_gnutls.c: Likewise 4092 * libwget/ssl_wolfssl.c: Likewise 4093 * libwget/tls_session.c: Likewise 4094 * src/wget_options.h: Likewise 4095 40962019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4097 4098 Rename wget_tls_session_t -> wget_tls_session 4099 4100 * include/wget/wget.h: Rename wget_tls_session_t -> wget_tls_session 4101 * libwget/tls_session.c: Likewise 4102 41032019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4104 4105 Rename wget_cookie_t -> wget_cookie 4106 4107 * include/wget/wget.h: Rename wget_cookie_t -> wget_cookie 4108 * fuzz/libwget_cookie_fuzzer.c: Likewise 4109 * libwget/cookie.c: Likewise 4110 * libwget/http_parse.c: Likewise 4111 * unit-tests/test.c: Likewise 4112 41132019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4114 4115 Rename wget_cookie_db_t -> wget_cookie_db 4116 4117 * include/wget/wget.h: Rename wget_cookie_db_t -> wget_cookie_db 4118 * examples/http_get2.c: Likewise 4119 * fuzz/libwget_cookie_fuzzer.c: Likewise 4120 * libwget/cookie.c: Likewise 4121 * libwget/http_highlevel.c: Likewise 4122 * libwget/init.c: Likewise 4123 * src/wget_options.h: Likewise 4124 * tests/test-cookies-http_state.c: Likewise 4125 * unit-tests/test-cookies-http_state.c: Likewise 4126 * unit-tests/test.c: Likewise 4127 41282019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4129 4130 Rename wget_iri_t -> wget_iri 4131 4132 * include/wget/wget.h: Rename wget_iri_t -> wget_iri 4133 * examples/*.c: Likewise 4134 * fuzz/*.c: Likewise 4135 * libwget/*.c: Likewise 4136 * src/*.c: Likewise 4137 * tests/*.c: Likewise 4138 * unit-tests/*.c: Likewise 4139 41402019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4141 4142 Rename wget_decompressor_t -> wget_decompressor 4143 4144 * include/wget/wget.h: Rename wget_decompressor_t -> wget_decompressor 4145 * libwget/decompressor.c: Likewise 4146 * libwget/http.c: Likewise 4147 * src/wget.c: Likewise 4148 41492019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4150 4151 Rename wget_thread_cond_t -> wget_thread_cond 4152 4153 * include/wget/wget.h: Rename wget_thread_cond_t -> wget_thread_cond 4154 * libwget/thread.c: Likewise 4155 * src/wget.c: Likewise 4156 * src/wget_job.h: Likewise 4157 * unit-tests/test-cond.c: Likewise 4158 41592019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4160 4161 Rename wget_thread_mutex_t -> wget_thread_mutex 4162 4163 * include/wget/wget.h: Rename wget_thread_mutex_t -> wget_thread_mutex 4164 * libwget/*.c: Likewise 4165 * src/*.c: Likewise 4166 * unit-tests/test-cond.c: Likewise 4167 41682019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4169 4170 Rename wget_thread_id_t -> wget_thread_id 4171 4172 * include/wget/wget.h: Rename wget_thread_id_t -> wget_thread_id 4173 * libwget/thread.c: Likewise 4174 * src/host.c: Likewise 4175 * src/wget_job.h: Likewise 4176 41772019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4178 4179 Rename wget_thread_t -> wget_thread 4180 4181 * include/wget/wget.h: Rename wget_thread_t -> wget_thread 4182 * examples/batch_loader.c: Likewise 4183 * libwget/thread.c: Likewise 4184 * src/*.c: Likewise 4185 * unit-tests/test-cond.c: Likewise 4186 41872019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4188 4189 Rename wget_stringmap_iterator_t -> wget_stringmap_iterator 4190 4191 * include/wget/wget.h: Likewise 4192 * unit-tests/test.c: Likewise 4193 41942019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4195 4196 Rename wget_stringmap_t -> wget_stringmap 4197 4198 * include/wget/wget.h: Rename wget_stringmap_t -> wget_stringmap 4199 * libwget/stringmap.c: Likewise 4200 * src/options.c: Likewise 4201 * src/plugin.c: Likewise 4202 * src/wget.c: Likewise 4203 * unit-tests/stringmap_perf.c: Likewise 4204 * unit-tests/test.c: Likewise 4205 42062019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4207 4208 Rename wget_hashmap_iterator_t -> wget_hashmap_iterator 4209 4210 * include/wget/wget.h: Rename wget_hashmap_iterator_t -> wget_hashmap_iterator 4211 * libwget/hashmap.c: Likewise 4212 42132019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4214 4215 Rename wget_hashmap_t -> wget_hashmap 4216 4217 * include/wget/wget.h: Rename wget_hashmap_t -> wget_hashmap 4218 * libwget/*.c: Likewise 4219 * src/*.c: Likewise 4220 42212019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4222 4223 Rename wget_vector_t -> wget_vector 4224 4225 * include/wget/wget.h: Rename wget_vector_t -> wget_vector 4226 * examples/print_css_urls3.c: Likewise 4227 * fuzz/*.c: Likewise 4228 * libwget/*.c: Likewise 4229 * src/*.c: Likewise 4230 * tests/*.c: Likewise 4231 * unit-tests/*.c: Likewise 4232 42332019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4234 4235 Rename wget_logger_t -> wget_logger 4236 4237 * include/wget/wget.h: Rename wget_logger_t -> wget_logger 4238 * libwget/log.c: Likewise 4239 * libwget/logger.c: Likewise 4240 * libwget/logger.h: Likewise 4241 42422019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4243 4244 Rename wget_buffer_t -> wget_buffer 4245 4246 * include/wget/wget.h: Rename wget_buffer_t -> wget_buffer 4247 * examples/*.c: Likewise 4248 * fuzz/libwget_iri_fuzzer.c: Likewise 4249 * libwget/*.c: Likewise 4250 * src/*.c: Likewise 4251 * tests/*.c: Likewise 4252 * unit-tests/*.c: Likewise 4253 42542019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4255 4256 Rename wget_bitmap_t -> wget_bitmap 4257 4258 * include/wget/wget.h: Likewise 4259 * libwget/bitmap.c: Likewise 4260 * unit-tests/test.c: Likewise 4261 42622019-07-18 Tim Rühsen <tim.ruehsen@gmx.de> 4263 4264 Rename wget_list_t -> wget_list 4265 4266 * include/wget/wget.h: Likewise 4267 * libwget/list.c: Likewise 4268 * libwget/test_linking.c: Likewise 4269 * src/wget_host.h: Likewise 4270 * src/wget_job.h: Likewise 4271 42722019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4273 4274 * libwget/http.c: Add prototype for _get_body 4275 42762019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4277 4278 Remove wget_tcp_set_stats_site() function 4279 4280 * include/wget/wget.h: Remove wget_stats_callback_t and 4281 wget_tcp_set_stats_site() 4282 * libwget/http.c: Remove wget_tcp_set_stats_site() 4283 42842019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4285 4286 Typedef wget_server_stats_callback_t not as pointer 4287 4288 * include/wget/wget.h: Remove '*' signature from function typedef 4289 * libwget/http.c: Amend casts and variable types 4290 42912019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4292 4293 Typedef wget_tls_stats_callback_t not as pointer 4294 4295 * include/wget/wget.h: Remove '*' signature from function typedef 4296 * libwget/ssl_gnutls.c: Amend casts and variable types 4297 * libwget/ssl_wolfssl.c: Likewise 4298 42992019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4300 4301 Typedef wget_ocsp_stats_callback_t not as pointer 4302 4303 * include/wget/wget.h: Remove '*' signature from function typedef 4304 * libwget/ssl_gnutls.c: Amend casts and variable types 4305 * libwget/ssl_wolfssl.c: Likewise 4306 43072019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4308 4309 Typedef wget_dns_stats_callback_t not as pointer 4310 4311 * include/wget/wget.h: Remove '*' signature from function typedef 4312 * libwget/dns.c: Amend casts and variable types 4313 43142019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4315 4316 Typedef plugin functions not as pointer 4317 4318 * include/wget/wget.h: Remove '*' signature from function typedefs 4319 * libwget/plugin.c: Amend casts and variable types 4320 * src/plugin.c: Likewise 4321 43222019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4323 4324 Typedef http callback functions not as pointer 4325 4326 * include/wget/wget.h: Remove '*' signature from function typedefs 4327 * libwget/http.c: Amend casts and variable types 4328 * libwget/http_highlevel.c: Likewise 4329 43302019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4331 4332 Typedef XML parse functions not as pointer 4333 4334 * include/wget/wget.h: Remove '*' signature from function typedef 4335 * libwget/xml.c: Amend casts and variable types 4336 43372019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4338 4339 Typedef CSS parse functions not as pointer 4340 4341 * include/wget/wget.h: Remove '*' signature from function typedefs 4342 * libwget/css.c: Amend casts and variable types 4343 43442019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4345 4346 Typedef decompressor functions not as pointer 4347 4348 * include/wget/wget.h: Remove '*' signature from function typedefs 4349 * libwget/decompressor.c: Likewise, Amend casts 4350 43512019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4352 4353 * .gitlab-ci.yml: Add CI runner for Debian unstable 4354 43552019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4356 4357 Typedef vector functions not as pointer 4358 4359 * include/wget/wget.h: Remove '*' signature from function typedefs 4360 * libwget/*.c: Amend casts 4361 * src/*.c: Likewise 4362 * tests/*.c: Likewise 4363 * unit-tests/*.c: Likewise 4364 43652019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4366 4367 Typedef wget_logger_func_t not as pointer 4368 4369 * include/wget/wget.h: Remove '*' signature from function typedef 4370 * libwget/init.c: Amend va_arg params 4371 * libwget/logger.c: Amend function params 4372 43732019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4374 4375 Typedef wget_* alloc functions not as pointer 4376 4377 * include/wget/wget.h: Remove '*' signature from function typedefs 4378 * libwget/xalloc.c: Amend declarations 4379 43802019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4381 4382 Typedef wget_list_browse_t not as pointer 4383 4384 * include/wget/wget.h: Remove '*' signature from function typedef 4385 * libwget/list.c: Amend wget_list_browse() argument 4386 * src/host.c: Amend cast 4387 43882019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4389 4390 Typedef wget_update_[load|save]_t not as pointer 4391 4392 * include/wget/wget.h: Remove '*' signature from function typedefs 4393 * libwget/*.c: Amend casts 4394 43952019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4396 4397 Typedef wget_global_get_func_t not as pointer 4398 4399 * include/wget/wget.h: Remove '*' signature from function typedef 4400 * libwget/init.c: Amend function definition 4401 44022019-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 4403 4404 Typedef hashmap functions not as pointer 4405 4406 * include/wget/wget.h: Remove '*' signature from function typedefs 4407 * libwget/*.c: Amend casts 4408 * src/*.c: Likewise 4409 * unit-tests/stringmap_perf.c: Likewise 4410 44112019-07-15 Tim Rühsen <tim.ruehsen@gmx.de> 4412 4413 * libwget/bar.c (_bar_update_speed_stats): Fix division by 0 4414 44152019-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 4416 4417 * src/stats_site.c (site_stats_init): Always free resources 4418 44192019-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 4420 4421 * src/options.c (parse_stats): Fix stats options 4422 4423 Stats options wrote to stdout when a filename was given without format. 4424 44252019-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 4426 4427 * fuzz/*.in: Update fuzz corpora from OSS-Fuzz 4428 44292019-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 4430 4431 * fuzz/README.md: clang-6.0 -> clang 4432 44332019-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 4434 4435 * src/options.c (init): Do not write stats to files when fuzzing 4436 44372019-07-11 Tim Rühsen <tim.ruehsen@gmx.de> 4438 4439 * docs/Makefile.am: Add README.md to DISTCLEANFILES unconditionally 4440 44412019-07-11 Tim Rühsen <tim.ruehsen@gmx.de> 4442 4443 * .travis_setup.sh: Add GNU grep and gawk for OSX 4444 44452019-07-11 Tim Rühsen <tim.ruehsen@gmx.de> 4446 4447 * .travis.sh: Fix sed -i for OSX on TravisCI 4448 44492019-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 4450 4451 * unit-tests/test.c (test_mem): Fix buffer overflow detected by ASAN 4452 44532019-07-09 Tim Rühsen <tim.ruehsen@gmx.de> 4454 4455 * libwget/ssl_wolfssl.c (wget_ssl_init): Exit gracefully on error 4456 44572019-07-09 Tim Rühsen <tim.ruehsen@gmx.de> 4458 4459 * cfg.mk: Add new syntax-check rule 'sc_gettext_printf' 4460 44612019-07-09 Tim Rühsen <tim.ruehsen@gmx.de> 4462 4463 Fix missing gettext/translation for string literals 4464 4465 * libwget/robots.c: Likewise 4466 * libwget/ssl_gnutls.c: Likewise 4467 * libwget/ssl_wolfssl.c: Likewise 4468 * src/gpgme.c: Likewise 4469 * src/host.c: Likewise 4470 * src/options.c: Likewise 4471 * src/wget.c: Likewise 4472 44732019-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 4474 4475 * cfg.mk: Add new syntax-check rule 'sc_prohibit_gettext_debug' 4476 44772019-06-25 Tim Rühsen <tim.ruehsen@gmx.de> 4478 4479 * docs/Makefile.am: Replace $< from explicit rule 4480 44812019-06-25 Tim Rühsen <tim.ruehsen@gmx.de> 4482 4483 * libwget/Makefile.am: Replace $< from explicit rule 4484 44852019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 4486 4487 * src/options.c (_add_tag): Fix OSS-Fuzz issue 15526 4488 44892019-06-28 Tim Rühsen <tim.ruehsen@gmx.de> 4490 4491 * configure.ac: Add pkg-config check for libnettle 4492 44932019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4494 4495 * libwget/utils.c (wget_strnglob): Return NULL if malloc fails 4496 44972019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4498 4499 * include/wget/wget.h: Indentation cleanups 4500 45012019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4502 4503 Rename _wget_bar_st -> wget_bar_st 4504 4505 * include/wget/wget.h: Likewise 4506 * libwget/bar.c: Likewise 4507 45082019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4509 4510 Rename _wget_hash_hd_st -> wget_hash_hd_st 4511 4512 * include/wget/wget.h: Likewise 4513 * libwget/hashfile.c: Likewise 4514 45152019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4516 4517 Rename _wget_http_connection_st -> wget_http_connection_st 4518 4519 * include/wget/wget.h: Likewise 4520 * libwget/http.c: Likewise 4521 45222019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4523 4524 Rename _wget_netrc_db_st -> wget_netrc_db_st 4525 4526 * include/wget/wget.h: Likewise 4527 * libwget/netrc.c: Likewise 4528 45292019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4530 4531 Rename _wget_tls_session_db_st -> wget_tls_sessione_db_st 4532 4533 * include/wget/wget.h: Likewise 4534 * libwget/tls_session.c: Likewise 4535 45362019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4537 4538 Rename _wget_tls_session_st -> wget_tls_sessione_st 4539 4540 * include/wget/wget.h: Likewise 4541 * libwget/tls_session.c: Likewise 4542 45432019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4544 4545 * include/wget/wget.h: Remove comment line 4546 45472019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4548 4549 Rename _wget_hpkp_st -> wget_hpkp_st 4550 4551 * include/wget/wget.h: Likewise 4552 * libwget/hpkp.c: Likewise 4553 45542019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4555 4556 Rename _wget_cookie_st -> wget_cookie_st 4557 4558 * include/wget/wget.h: Likewise 4559 * libwget/cookie.c: Likewise 4560 45612019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4562 4563 * src/wget_options.h (struct config): wget_cookie_db_st -> wget_cookie_db_t 4564 45652019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4566 4567 Rename _wget_cookie_db_st -> wget_cookie_db_st 4568 4569 * include/wget/wget.h: Likewise 4570 * libwget/cookie.c: Likewise 4571 * src/wget_options.h: Likewise 4572 45732019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4574 4575 Rename _wget_decompressor_st -> wget_decompressor_st 4576 4577 * include/wget/wget.h: Likewise 4578 * libwget/decompressor.c: Likewise 4579 45802019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4581 4582 Rename _wget_thread_cond_st -> wget_thread_cond_st 4583 4584 * include/wget/wget.h: Likewise 4585 * libwget/thread.c: Likewise 4586 45872019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4588 4589 Rename _wget_thread_mutex_st -> wget_thread_mutex_st 4590 4591 * include/wget/wget.h: Likewise 4592 * libwget/thread.c: Likewise 4593 45942019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4595 4596 Rename _wget_thread_st -> wget_thread_st 4597 4598 * include/wget/wget.h: Likewise 4599 * libwget/thread.c: Likewise 4600 46012019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4602 4603 Rename _wget_hashmap_iterator_st -> wget_hashmap_iterator_st 4604 4605 * include/wget/wget.h: Likewise 4606 * libwget/hashmap.c: Likewise 4607 46082019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4609 4610 Rename _wget_hashmap_st -> wget_hashmap_st 4611 4612 * include/wget/wget.h: Likewise 4613 * libwget/hashmap.c: Likewise 4614 46152019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4616 4617 Rename _wget_logger_st -> wget_logger_st 4618 4619 * include/wget/wget.h: Likewise 4620 * libwget/logger.c: Likewise 4621 46222019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4623 4624 Rename _wget_list_st -> wget_list_st 4625 4626 * include/wget/wget.h: Likewise 4627 * libwget/list.c: Likewise 4628 46292019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4630 4631 * include/wget/wget.h: Rename _WGET_PREFIX -> WGET_PREFIX 4632 46332019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4634 4635 * include/wget/wget.h: Add G_GNUC_WGET_MALLOC to wget_read_file() 4636 46372019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4638 4639 * libwget/robots.c (wget_robots_parse): Fix corner-case memleak 4640 46412019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4642 4643 Check OOM in wget_http_add_header*() 4644 4645 * include/wget/wget.h: Return int for wget_http_add_header(), 4646 return int for wget_http_add_header_param() 4647 * libwget/http.c; New static function http_add_header(), 4648 (wget_http_add_header_vprintf): Use http_add_header(), 4649 (wget_http_add_header): Return int, use http_add_header(), 4650 (wget_http_add_header_param): Return int, use http_add_header() 4651 46522019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4653 4654 * libwget/http_highlevel.c (wget_http_get): Check return value of wget_vector_add_memdup() 4655 46562019-06-27 Tim Rühsen <tim.ruehsen@gmx.de> 4657 4658 Add memory checks to wget_http_add_header_vprintf() 4659 4660 * include/wget/wget.h: return int for wget_http_add_header_vprintf(), 4661 return int for wget_http_add_header_printf() 4662 * libwget/http.c (wget_http_add_header_vprintf): 4663 check for memory allocations and return error code, 4664 (wget_http_add_header_printf): Return error code 4665 46662019-06-26 Tim Rühsen <tim.ruehsen@gmx.de> 4667 4668 * libwget/html_url.c (_html_get_url): Set url.link_inline 4669 46702019-06-26 Tim Rühsen <tim.ruehsen@gmx.de> 4671 4672 * libwget/html_url.c (_css_parse_uri): Use malloc + wget_vector_add 4673 46742019-06-26 Tim Rühsen <tim.ruehsen@gmx.de> 4675 4676 * libwget/sitemap_url.c (_sitemap_get_url): Use malloc + wget_vector_add 4677 46782019-06-26 Tim Rühsen <tim.ruehsen@gmx.de> 4679 4680 * libwget/rss_url.c (_rss_get_url): Use malloc + wget_vector_add 4681 46822019-06-26 Tim Rühsen <tim.ruehsen@gmx.de> 4683 4684 * libwget/css_url.c (_css_get_url): Use calloc + wget_vector_add 4685 46862019-06-26 Tim Rühsen <tim.ruehsen@gmx.de> 4687 4688 * libwget/atom_url.c (_atom_get_url): Use malloc + wget_vector_add 4689 46902019-06-26 Tim Rühsen <tim.ruehsen@gmx.de> 4691 4692 Simplify vector API 4693 4694 * libwget/vector.c: Use wget_free as default destructor, 4695 renamed _vec_insert_private() -> insert_element, 4696 (insert_element): removed args 'size' and 'alloc', 4697 removed the _noalloc() API, 4698 removed the 'size' argument from functions, 4699 added wget_vector_add_memdup() as convenience function, 4700 renamed _compare() -> compare_element 4701 * include/wget/wget.h: Removed _noalloc() functions 4702 * fuzz/libwget_cookie_fuzzer.c: Amended to new API 4703 * libwget/*.c: Likewise 4704 * src/*.c: Likewise 4705 * tests/*.c: Likewise 4706 * unit-tests/*.c: Likewise 4707 47082019-06-24 Gisle Vanem <gisle.vanem@gmail.com> 4709 4710 * libwget/ssl_gnutls.c: Fix Doxygen indenting of bullet list 4711 4712 Seems Doxgen gets confused unless all bullet-ed items in a list have the same number of spaced. 4713 And the generated HTML looks jagged. 4714 47152019-06-25 Tim Rühsen <tim.ruehsen@gmx.de> 4716 4717 * AUTHORS: Add Gisle Vanem 4718 47192019-06-24 Tim Rühsen <tim.ruehsen@gmx.de> 4720 4721 Add sc_prohibit_alloc to 'make syntax-check' 4722 4723 * cfg.mk: Add sc_prohibit_alloc rule plus exceptions 4724 * src/options.c: Tag exceptions 4725 * unit-tests/test.c: Likewise 4726 4727 This rule checks for calling malloc()/calloc()/realloc(). Instead, either 4728 use wget_malloc(), wget_calloc() or wget_realloc(). 4729 47302019-06-24 Tim Rühsen <tim.ruehsen@gmx.de> 4731 4732 Add sc_prohibit_free to 'make syntax-check' 4733 4734 * cfg.mk: Add sc_prohibit_free rule plus exceptions 4735 * examples/print_css_urls3.c: Remove unneeded comment 4736 * fuzz/libwget_base64_fuzzer.c: Use wget_free() instead of free() 4737 * fuzz/libwget_cookie_fuzzer.c: Likewise 4738 * fuzz/libwget_css_url_fuzzer.c: Likewise 4739 * fuzz/libwget_http_client_fuzzer.c: Likewise 4740 * fuzz/libwget_utils_fuzzer.c: Likewise 4741 * libwget/vector.c: Likewise 4742 * src/options.c: Likewise 4743 * src/wget_main.h: Likewise 4744 * tests/test-compression.c: Likewise 4745 * tests/test-cookies-http_state.c: Likewise 4746 * tests/test-metalink.c: Likewise 4747 * unit-tests/stringmap_perf.c: Likewise 4748 4749 This rule checks for calling free(). Instead, either use wget_free() or 4750 the convenience macro xfree(). 4751 47522019-06-24 Tim Rühsen <tim.ruehsen@gmx.de> 4753 4754 * m4/wget_manywarnings.m4: Remove -Werror-... options 4755 47562019-06-24 Tim Rühsen <tim.ruehsen@gmx.de> 4757 4758 * libwget/net.c: Fix compiler warning on MinGW 4759 47602019-06-24 Tim Rühsen <tim.ruehsen@gmx.de> 4761 4762 * fuzz/wget_options_fuzzer.c: Fix compiler warning on MinGW 4763 47642019-06-24 Tim Rühsen <tim.ruehsen@gmx.de> 4765 4766 * tests/Makefile.am: Avoid using unportable .o reference with libtool 4767 47682019-06-24 Tim Rühsen <tim.ruehsen@gmx.de> 4769 4770 Remove need for C11 thread local storage 4771 4772 * configure.ac: Remove check for threads.h 4773 * libwget/ssl_gnutls.c: Remove need for C11 thread local storage 4774 47752019-06-23 Tim Rühsen <tim.ruehsen@gmx.de> 4776 4777 * fuzz/libwget_http_client_fuzzer.c: Add emulation of freeaddrinfo() 4778 47792019-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 4780 4781 * tests/libtest.c (_answer_to_connection): Add const to conn_info 4782 47832019-06-21 Tim Rühsen <tim.ruehsen@gmx.de> 4784 4785 * tests/Makefile.am: Use top_srcdir instead of .. to fix VPATH build 4786 47872019-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 4788 4789 * src/options.c (deinit): Fix OSS-Fuzz issue 15443 4790 47912019-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 4792 4793 * src/options.c (parse_stats): Fix OSS-Fuzz issue 15438 4794 47952019-06-21 Tim Rühsen <tim.ruehsen@gmx.de> 4796 4797 * tests/Makefile.am: Use top_srcdir instead of .. to fix VPATH build 4798 47992019-06-19 Tim Rühsen <tim.ruehsen@gmx.de> 4800 4801 Inline most of stringmap API 4802 4803 * include/wget/wget.h: Add stringmap API (mostly) as static inline functions 4804 * libwget/stringmap.c: Remove stringmap implementation (mostly) 4805 48062019-06-19 Tim Rühsen <tim.ruehsen@gmx.de> 4807 4808 Add memory checks and error return value to hashmap API 4809 4810 * include/wget/wget.h: wget_hashmap_sethashfunc() returns int 4811 * libwget/hashmap.c: Check memory allocations 4812 * libwget/stringmap.c: wget_stringmap_sethashfunc() returns int 4813 48142019-06-19 Tim Rühsen <tim.ruehsen@gmx.de> 4815 4816 Simplify hashmap and stringmap API 4817 4818 * include/wget/wget.h: Replace _put() by _put_noalloc() functions to 4819 keep memory allocations outside the library functions. 4820 * libwget/stringmap.c: Likewise 4821 * libwget/hashmap.c: Likewise 4822 * libwget/*.c: Likewise 4823 * src/*.c: Likewise 4824 * unit-tests/*.c: Likewise 4825 48262019-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 4827 4828 Separate DNS and DNS caching 4829 4830 * include/wget/wget.h: Rewrite parts of the DNS / DNS caching API 4831 * libwget/dns.c: Likewise 4832 * libwget/dns_cache.c: Likewise 4833 * libwget/init.c (wget_global_init): Amend WGET_DNS_CACHING 4834 * libwget/net.c: Cleanup and amend for the new API 4835 * libwget/net.h: Cleanup 4836 * src/options.c: Use DNS caching instance if caching is requested 4837 48382019-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 4839 4840 Remove --stats-all 4841 4842 * docs/wget2.md Remove docs for --stats-all 4843 * src/options.c: Remove --stats-all and parse_stats_all() 4844 * fuzz/wget_options_fuzzer.dict: remove --stats-all 4845 * tests/Makefile.am: Remove test-stats-all 4846 * tests/test-stats-all.c: Remove file 4847 48482019-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 4849 4850 Rewrite Site statistics --stats-site 4851 4852 * fuzz/Makefile.am: Remove ../src/stats_site.o 4853 * include/wget/wget.h: Remove wget_stats_type_t 4854 * po/POTFILES.in. Remove src/stats.c 4855 * src/Makefile.am: Remove stats_site.c 4856 * src/options.c (parse_stats_all): Remove site stats code, 4857 (init): Add DNS stats code, 4858 (deinit): Cleanup dns stats code 4859 * src/stats.c: Remove file 4860 * src/stats_site.c: iFix for new stats code 4861 * src/wget.c (main): stats_print -> site_stats_print, 4862 (add_statistics): config.stats_site -> config.stats_site_args, 4863 (process_response): Likewise 4864 * src/wget_options.h (struct config): Remove stats_site, add stats_site_args 4865 * src/wget_stats.h: Remove stats_opt_t, 4866 remove stats_print_func_t, 4867 remove stats_callback_setter_t, 4868 remove struct stats_opts, 4869 remove stats_site_opts, 4870 rename stats_init -> site_stats_init, 4871 rename stats_exit -> site_stats_exit, 4872 rename stats_print -> site_stats_print 4873 * unit-tests/Makefile.am: Remove ../src/stats.o 4874 48752019-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 4876 4877 Amend --stats-* syntax 4878 4879 * docs/wget2.md: Amend docs for --stats-* 4880 * src/options.c: Amend option syntax for --stats-* 4881 * tests/stats-test-util.h: Amend tests to succeed 4882 48832019-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 4884 4885 Rewrite Server statistics --stats-server 4886 4887 * fuzz/Makefile.am: Remove ../src/stats_server.o 4888 * include/wget/wget.h: 4889 remove wget_server_stats_t, 4890 rename wget_stats_callback_t -> wget_server_stats_callback_t, 4891 rename wget_tcp_set_stats_server -> wget_server_set_stats_callback, 4892 add wget_server_stats_data_t, 4893 add wget_server_stats_callback_t, 4894 * libwget/http.c: Remove _stats_data_t, 4895 add server_stats_callback, 4896 add server_stats_ctx 4897 * src/Makefile.am: Remove stats_server.c 4898 * src/options.c (parse_stats_all): Remove server stats code, 4899 add stats_callback_server(), 4900 (init): Add DNS stats code, 4901 (deinit): Cleanup dns stats code 4902 * src/stats.c (stats_print_func_t): Remove print_server[], 4903 remove WGET_STATS_TYPE_SERVER 4904 * src/stats_server.c: Remove file 4905 * src/wget_options.h (struct config): Remove stats_server, add stats_server_args 4906 * src/wget_stats.h: Remove stats_server_opts 4907 * unit-tests/Makefile.am: Remove ../src/stats_server.o 4908 49092019-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 4910 4911 Rewrite TLS statistics --stats-tls 4912 4913 * fuzz/Makefile.am: Remove ../src/stats_tls.o 4914 * include/wget/wget.h: Add wget_tls_stats_data_t, 4915 add wget_tls_stats_callback_t, 4916 add wget_ssl_set_stats_callback_tls, 4917 remove WGET_STATS_TYPE_TLS, 4918 remove wget_tls_stats_t, 4919 remove wget_tcp_set_stats_tls(), 4920 remove wget_tcp_get_stats_tls() 4921 * libwget/ssl_gnutls.c: Remove old stats code, add new stats code 4922 * libwget/ssl_none.c: Add/remove function stubs 4923 * libwget/ssl_wolfssl.c: Remove old stats code, add new stats code 4924 * src/Makefile.am: Remove stats_tls.c 4925 * src/options.c (parse_stats_all): Remove TLS stats code, 4926 add stats_callback_tls(), 4927 (init): Add TLS stats code, 4928 * src/stats.c (stats_print_func_t): Remove print_tls[], 4929 remove WGET_STATS_TYPE_TLS 4930 * src/stats_tls.c: Remove file 4931 * src/wget_options.h (struct config): Remove stats_tls, add stats_tls_args 4932 * src/wget_stats.h: Remove stats_tls_opts 4933 * unit-tests/Makefile.am: Remove ../src/stats_tls.o 4934 49352019-06-05 Tim Rühsen <tim.ruehsen@gmx.de> 4936 4937 Rewrite OCSP statistics --stats-ocsp 4938 4939 * fuzz/Makefile.am: Remove ../src/stats_ocsp.o 4940 * include/wget/wget.h: Add wget_ocsp_stats_data_t, 4941 add wget_ocsp_stats_callback_t, 4942 add wget_ssl_set_stats_callback_ocsp, 4943 remove WGET_STATS_TYPE_OCSP, 4944 remove wget_ocsp_stats_t, 4945 remove wget_tcp_set_stats_ocsp(), 4946 remove wget_tcp_get_stats_ocsp() 4947 * libwget/ssl_gnutls.c: Remove old stats code, add new stats code 4948 * libwget/ssl_none.c: Add/remove function stubs 4949 * libwget/ssl_wolfssl.c: Remove old stats code, add new stats code 4950 * src/Makefile.am: Remove stats_ocsp.c 4951 * src/options.c (parse_stats_all): Remove OCSP stats code, 4952 add stats_callback_ocsp(), 4953 (init): Add OCSP stats code, 4954 * src/stats.c (stats_print_func_t): Remove print_ocsp[], 4955 remove WGET_STATS_TYPE_OCSP 4956 * src/stats_ocsp.c: Remove file 4957 * src/wget_options.h (struct config): Remove stats_ocsp, add stats_ocsp_args 4958 * src/wget_stats.h: Remove stats_ocsp_opts 4959 * unit-tests/Makefile.am: Remove ../src/stats_ocsp.o 4960 49612019-06-02 Tim Rühsen <tim.ruehsen@gmx.de> 4962 4963 Rewrite DNS statistics --stats-dns 4964 4965 * fuzz/Makefile.am: Remove ../src/stats_dns.o 4966 * include/wget/wget.h: Add wget_dns_stats_data_t, 4967 add wget_dns_stats_callback_t, 4968 add wget_dns_set_stats_callback, 4969 remove WGET_STATS_TYPE_DNS, 4970 remove wget_dns_stats_t, 4971 remove wget_dns_set_stats(), 4972 remove wget_dns_get_stats() 4973 * libwget/dns.c: Remove _stats_data_t, 4974 amend struct wget_dns_st, 4975 add wget_dns_set_stats_callback(), 4976 remove wget_dns_get_stats() 4977 * src/Makefile.am: Remove stats_dns.c 4978 * src/options.c (parse_stats_all): Remove DNS stats code, 4979 add stats_callback_dns(), 4980 (init): Add DNS stats code, 4981 * src/stats.c (stats_print_func_t): Remove print_dns[], 4982 remove WGET_STATS_TYPE_DNS 4983 * src/stats_dns.c: Remove file 4984 * src/wget_options.h (struct config): Remove stats_dns, add stats_dns_args 4985 * src/wget_stats.h: Remove stats_dns_opts 4986 * tests/stats-test-util.h: Remove "human" 4987 * unit-tests/Makefile.am: Remove ../src/stats_ocsp.o 4988 49892019-05-31 Tim Rühsen <tim.ruehsen@gmx.de> 4990 4991 Put resolver/dns code into own source file 4992 4993 * docs/Makefile.am: Add libwget-dns.3 man page 4994 * examples/batch_loader.c: Remove call to wget_tcp_set_dns_timeout() 4995 * examples/check_url_types.c: Use wget_dns_set_timeout() 4996 * include/wget/wget.h: Add wget_dns_ API 4997 * libwget/Makefile.am: Add dns.c 4998 * libwget/dns.c: New file 4999 * libwget/init.c: Use wget_dns_set_caching() 5000 * libwget/net.c: Remove DNS functions 5001 * libwget/net.h (wget_tcp_st): Add dns member 5002 * po/POTFILES.in: Add libwget/dns.c 5003 * src/options.c: Use new DNS API 5004 * src/stats.c: Add some checks 5005 * src/stats_dns.c: Use wget_dns_get_stats() 5006 50072019-05-17 Tim Rühsen <tim.ruehsen@gmx.de> 5008 5009 * libwget/thread.c: Manually inline wget_get_timemillis() into wget_thread_cond_wait() 5010 50112019-05-14 Tim Rühsen <tim.ruehsen@gmx.de> 5012 5013 Move wget_restrict_file_name() to src/utils.c 5014 5015 * fuzz/libwget_utils_fuzzer.c: Remove test code for wget_restrict_file_name() 5016 * include/wget/wget.h: Remove wget_restrict_file_name() prototype 5017 * libwget/utils.c: Remove wget_restrict_file_name() 5018 * src/utils.c: Add wget_restrict_file_name() 5019 * src/wget_utils.h: Add wget_restrict_file_name() prototype 5020 * tests/Makefile.am: Add ../src/utils.c to libtest_la_SOURCES 5021 * tests/libtest.c: Include ../src/wget_utils.h 5022 50232019-06-21 Tim Rühsen <tim.ruehsen@gmx.de> 5024 5025 Fix several forgotten xmalloc -> wget_malloc 5026 5027 * libwget/css.c (wget_css_parse_file): Rename xmalloc->wget_malloc 5028 * libwget/hashfile.c (wget_hash_init): Likewise 5029 * libwget/ssl_gnutls.c (_cert_verify_hpkp): Likewise 5030 * libwget/xml.c (wget_xml_parse_file): Likewise 5031 50322019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5033 5034 * libwget/hashmap.c (hashmap_rehash): Fix for wget_calloc() 5035 50362019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5037 5038 Remove xrealloc define for wget_realloc 5039 5040 * libwget/private.h: Remove #define 5041 * libwget/*.c: Replace xrealloc by wget_realloc 5042 50432019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5044 5045 Remove xcalloc define for wget_calloc 5046 5047 * libwget/private.h: Remove #define 5048 * libwget/*.c: Replace xcalloc by wget_calloc 5049 50502019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5051 5052 Remove xmalloc define for wget_malloc 5053 5054 * libwget/private.h: Remove #define 5055 * libwget/*.c: Replace xmalloc by wget_malloc 5056 50572019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5058 5059 * libwget/vector.c (wget_vector_create): Check result of malloc() 5060 50612019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5062 5063 * libwget/thread.c: Check result of malloc() 5064 50652019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5066 5067 * libwget/mem.c: Check result of malloc() 5068 50692019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5070 5071 * libwget/list.c (wget_list_append): Check result of malloc() 5072 50732019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5074 5075 * libwget/iri.c (wget_iri_parse,wget_iri_clone): Check result of malloc() 5076 50772019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5078 5079 * libwget/http.c (_server_stats_add): Check result of malloc() 5080 50812019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5082 5083 * libwget/cookie.c (wget_cookie_create_request_header): Check result of malloc() 5084 50852019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5086 5087 * include/wget/wget.h: Add LIBWGET_WARN_UNUSED_RESULT 5088 50892019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5090 5091 Don't ignore wget_malloc() return value 5092 5093 * libwget/test_linking.c: Don't ignore wget_malloc() return value 5094 50952019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5096 5097 Add G_GNUC_WGET_RETURNS_NONNULL in wget.h 5098 5099 * include/wget/wget.h: Add G_GNUC_WGET_RETURNS_NONNULL 5100 * src/Makefile.am: Add -DMALLOC_RETURNS_NONNULL 5101 * src/options.c: Use G_GNUC_WGET_RETURNS_NONNULL for own alloc functions 5102 51032019-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 5104 5105 * libwget/base64.c: Check for malloc failures 5106 51072019-06-19 Tim Rühsen <tim.ruehsen@gmx.de> 5108 5109 Amend buffer functions to handle failed memory allocations 5110 5111 * include/wget/wget.h: wget_buffer_ensure_capacity() returns int 5112 * libwget/buffer.c (wget_buffer_init): Return NULL if malloc fails, 5113 (_buffer_realloc): Return error if malloc fails 5114 * libwget/http.c (wget_http_get_response_cb): Check return value of 5115 wget_buffer_ensure_capacity() 5116 * unit-tests/test.c (test_buffer): Add asserts 5117 51182019-06-18 Tim Rühsen <tim.ruehsen@gmx.de> 5119 5120 Make wget allocation functions overridable function pointers 5121 5122 * include/wget/wget.h: Remove allocation function prototypes, 5123 add function pointer types and 'extern' globals 5124 * libwget/private.h: Use wget_free() instead of free() 5125 * libwget/xalloc.c: Remove function definitions, 5126 add global function pointer variables 5127 * src/options.c: Add own allocation functions 5128 * unit-tests/test.c: Test that function pointers are overridable 5129 51302019-06-18 Tim Rühsen <tim.ruehsen@gmx.de> 5131 5132 Change OOM callback implementation 5133 51342019-06-17 Tim Rühsen <tim.ruehsen@gmx.de> 5135 5136 Fix robots.txt code for stand-alone library API 5137 5138 * fuzz/libwget_robots_parse_fuzzer.c: Amend API call 5139 * include/wget/wget.h: Amend wget_robots_parse() API 5140 * libwget/robots.c: Amend wget_robots_parse() API, 5141 check memory allocations 5142 * libwget/test_linking_robots.c: New file 5143 * src/wget.c (process_response): Amend call to wget_robots_parse() 5144 * unit-tests/test.c (test_robots): Amend call to wget_robots_parse() 5145 51462019-06-16 Tim Rühsen <tim.ruehsen@gmx.de> 5147 5148 Add robots.txt functions, make wget_robots_t opaque 5149 51502019-05-14 Tim Rühsen <tim.ruehsen@gmx.de> 5151 5152 * cfg.mk: Fix syntax-check (bindtextdomain) 5153 51542019-06-19 Tim Rühsen <tim.ruehsen@gmx.de> 5155 5156 Cleanup bitmap API 5157 5158 * wget.h: Add WGET_E_MEMORY = -2, 5159 amend function interface of wget_bitmap_allocate, 5160 rename wget_bitmap_allocate -> wget_bitmap_init 5161 * libwget/bitmap.c: Likewise 5162 * unit-tests/test.c: Amend bitmap tests appropriate 5163 51642019-06-18 Darshit Shah <darnir@gnu.org> 5165 5166 * gnulib: Update gnulib 5167 51682019-06-15 Darshit Shah <darnir@gnu.org> 5169 5170 Don't check for always true headers 5171 5172 * cfg.mk: Enable checking for always_true_header_tests 5173 * libwget/http.c,net.c,ssl_gnutls.c: Don't check for HAVE_SYS_SOCKET_H. 5174 The gnulib module always provides it. 5175 * libwget/ssl_gnutls.c: Don't check for HAVE_SYS_UIO_H. The gnulib 5176 module always provides it. 5177 * bootstrap.conf: Add header sys_uio 5178 * configure.ac: Remove check for sys/uio.h 5179 51802019-06-17 Tim Rühsen <tim.ruehsen@gmx.de> 5181 5182 Add sc_prohibit_sprintf and sc_prohibit_printf and fix code 5183 5184 * cfg.mk: New syntax rules sc_prohibit_sprintf and sc_prohibit_printf 5185 * fuzz/*.c: Apply rules 5186 * libwget/*.c: Likewise 5187 * src/*.c: Likewise 5188 * tests/*.[ch]: Likewise 5189 * unit-tests/*.c: Likewise 5190 51912019-06-17 Tim Rühsen <tim.ruehsen@gmx.de> 5192 5193 * include/wget/wget.h: #define wget_printf() 5194 51952019-06-15 Darshit Shah <darnir@gnu.org> 5196 5197 * cfg.mk: Our makefiles make judicious use of $<, skip the check 5198 51992019-06-17 Tim Rühsen <tim.ruehsen@gmx.de> 5200 5201 * bootstrap.conf: Add gnulib module arpa_inet 5202 5203 On Debian unstable, after updating gnulib there was: 5204 5205 In file included from ../lib/arpa/inet.h:46, 5206 from ip.c:27: 5207 /usr/include/arpa/inet.h:34:8: error: unknown type name 'in_addr_t' 5208 extern in_addr_t inet_addr (const char *__cp) __THROW; 5209 52102019-06-15 Darshit Shah <darnir@gnu.org> 5211 5212 * bootstrap: Patch script to use the python tool 5213 52142019-06-15 Darshit Shah <darnir@gnu.org> 5215 5216 * bootstrap: Update the bootstrap script from gnulib 5217 52182019-06-15 Darshit Shah <darnir@gnu.org> 5219 5220 * gnulib: Update gnulib 5221 52222019-06-13 Tim Rühsen <tim.ruehsen@gmx.de> 5223 5224 * src/wget.c: Fix --https-enforce=hard for metalink 5225 52262019-06-12 Tim Rühsen <tim.ruehsen@gmx.de> 5227 5228 * src/wget.c: Make read_xattr_metadata() easier and safer to use 5229 52302019-06-12 Tim Rühsen <tim.ruehsen@gmx.de> 5231 5232 Fix -nc/--no-clobber 5233 5234 * src/wget.c: New function parse_local_file(), 5235 (add_url_to_queue): Parse local file if needed, 5236 (add_url): Likewise 5237 5238 Closes #451 5239 52402019-06-11 Tim Rühsen <tim.ruehsen@gmx.de> 5241 5242 * libwget/http.c: Moev _fix_broken_server_encoding() outside HTTP/2 code 5243 52442019-06-11 Tim Rühsen <tim.ruehsen@gmx.de> 5245 5246 * libwget/http.c: Fix call to _fix_broken_server_encoding() for HTTP/2 5247 52482019-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 5249 5250 Add --http2-only to resist on HTTP/2 5251 5252 * docs/wget2.md: Document --http2-only 5253 * libwget/ssl_gnutls.c (wget_ssl_open): Error if server doesn't accept h2 5254 * src/options.c: Add --http2-only 5255 * src/wget_options.h: Add config.http2_only 5256 52572019-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 5258 5259 * .gitlab-ci.yml: Fix WolfSSL CI runner 5260 52612019-06-04 Tim Rühsen <tim.ruehsen@gmx.de> 5262 5263 Add WITH_TLS as define and conditional 5264 5265 * configure.ac: Add WITH_TLS as define and conditional 5266 * tests/libtest.c: Use WITH_TLS for conditional code 5267 52682019-06-04 Tim Rühsen <tim.ruehsen@gmx.de> 5269 5270 * tests/test-post-handhshake-auth.c: Check for WITH_GNUTLS 5271 52722019-05-01 Tim Rühsen <tim.ruehsen@gmx.de> 5273 5274 * tests/libtest.c: Enable SSL/TLS tests for WolfSSL 5275 52762019-05-01 Tim Rühsen <tim.ruehsen@gmx.de> 5277 5278 * tests/test-i-https.c: Skip CRL test for WolfSSL 5279 52802019-05-01 Tim Rühsen <tim.ruehsen@gmx.de> 5281 5282 * libwget/ssl_wolfssl.c (wget_ssl_open): Fix return code for testing 5283 52842019-05-01 Tim Rühsen <tim.ruehsen@gmx.de> 5285 5286 Add SSL/TLS engine 'none' 5287 5288 * libwget/Makefile.am: Add building ssl_none.c 5289 * libwget/ssl_gnutls.c: Remove #ifdef WITH_GNUTLS 5290 * libwget/ssl_wolfssl.c: Likewise 5291 * libwget/ssl_none.c: Stub routines for 'no SSL/TLS support' 5292 52932019-05-01 Tim Rühsen <tim.ruehsen@gmx.de> 5294 5295 * .gitlab-ci-yml: Test WolfSSL build on Debian Unstable 5296 52972019-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 5298 5299 WolfSSL: Add ALPN and thus HTTP/2 5300 5301 * libwget/ssl_wolfssl.c (wget_ssl_init): Fix setting minimal TLS version, 5302 (wget_ssl_open): Set ALPN for HTTP/2, add result to statistics 5303 53042019-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 5305 5306 Initial implementation of WolfSSL as TLS backend 5307 5308 * README.md: Mention WolfSSL 5309 * configure.ac: Check for WolfSSL 5310 * libwget/Makefile.am: Conditionally add WolfSSL to libwget 5311 * libwget/ssl_wolfssl.c: New file with WolfSSL implementation 5312 * po/POTFILES.in: Add libwget/ssl_wolfssl.c 5313 5314 ./configure --without-gnutls --with-wolfssl 5315 53162019-05-25 Tim Rühsen <tim.ruehsen@gmx.de> 5317 5318 * .travis.sh: Set gettext version to 0.18.2 5319 53202019-05-24 Tim Rühsen <tim.ruehsen@gmx.de> 5321 5322 Publish tarball via Gitlab CI 5323 5324 * .gitlab-ci.yml: Create https://gnuwget.gitlab.io/wget2/wget2-latest.tar.gz 5325 * README.md: Instruction to download and build wget2 from tarball 5326 53272019-05-24 Tim Rühsen <tim.ruehsen@gmx.de> 5328 5329 * bootstrap.conf: Explicitly set source_base and test_base 5330 53312019-05-18 Tim Rühsen <tim.ruehsen@gmx.de> 5332 5333 Update spell-checker and fix more typos 5334 5335 * contrib/spell-checker: Update to python3 version of codespell 5336 * libwget/robots.c: Fix typo(s) 5337 * src/plugin.c: Likewise 5338 * src/wget_plugin.h: Likewise 5339 * tests/test-plugin-nonexistance.c: Likewise 5340 53412019-05-18 Tim Rühsen <tim.ruehsen@gmx.de> 5342 5343 * po/.gitignore: Update 5344 53452019-05-18 Tim Rühsen <tim.ruehsen@gmx.de> 5346 5347 * bootstrap.conf: Work around VPATH issue 5348 53492019-05-18 Tim Rühsen <tim.ruehsen@gmx.de> 5350 5351 * fuzz/main.c: Fix paths for WIN32 5352 53532019-05-18 Tim Rühsen <tim.ruehsen@gmx.de> 5354 5355 * configure.ac: Update gettext version to 0.19.3 5356 53572019-05-17 Tim Rühsen <tim.ruehsen@gmx.de> 5358 5359 Fix checks to include C11 threads.h 5360 5361 * configure.ac: Check for threads.h 5362 * libwget/ssl_gnutls.c: Add check for HAVE_THREADS_H 5363 53642019-05-15 Tim Rühsen <tim.ruehsen@gmx.de> 5365 5366 * libwget/ssl_gnutls.c: Fix position of _Thread_local 5367 53682019-05-14 Tim Rühsen <tim.ruehsen@gmx.de> 5369 5370 * libwget/bar.c: Replace sprintf/snprintf by wget_ functions 5371 53722019-05-14 Tim Rühsen <tim.ruehsen@gmx.de> 5373 5374 * libwget/thread.c: Remove #include timespec.h 5375 53762019-05-14 Tim Rühsen <tim.ruehsen@gmx.de> 5377 5378 * bootstrap: Fallback to gnulib-tool if python version errors 5379 53802019-05-10 Tim Rühsen <tim.ruehsen@gmx.de> 5381 5382 * libwget/ssl_gnutls.c: Use thread-safe ctime() replacement 5383 53842019-05-06 Tim Rühsen <tim.ruehsen@gmx.de> 5385 5386 * configure.ac: Silence gcc-9 -Wchkp 5387 53882019-04-29 sebma <sebma@users.noreply.github.com> 5389 5390 Test if doxygen OR pandoc is present to build the docs 5391 5392 * configure.ac: Logically split checking for doxygen and pandoc 5393 5394 Copyright-paperwork-exempt: Yes 5395 53962019-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 5397 5398 Add new option --unlink 5399 5400 * bootstrap.conf: Add gnulib module 'link' 5401 * docs/wget2.md: Remove the 'unimplemented' tag for --unlink 5402 * src/options.c: Add --unlink option 5403 * src/wget.c (_prepare_file): Unlink file if requested 5404 * src/wget_options.h (struct config): Add 'unlink' member 5405 * tests/Makefile.am: Add test-unlink 5406 * tests/libtest.c (wget_test): Implement hardlink feature 5407 * tests/libtest.h (wget_test_file_t): Add 'hardlink' member 5408 * tests/test-unlink.c: New file with tests for --unlink 5409 54102019-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 5411 5412 * src/options.c: Fix memleaks from XDG and homedir functions 5413 54142019-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 5415 5416 * src/options.c (get_config_files): Fix memleaks 5417 54182019-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 5419 5420 * docs/wget2.md: Mark unimplemented options with '[Not implemented yet]' 5421 54222019-04-26 Tim Rühsen <tim.ruehsen@gmx.de> 5423 5424 * tests/test-limit-rate.c: Increase tolerance from 200 to 500 ms 5425 5426 Valgrind tests on Travis needed the higher value 5427 54282019-04-25 Tim Rühsen <tim.ruehsen@gmx.de> 5429 5430 * fuzz/wget_options_fuzzer.c: Slightly change magic number 5431 5432 Libfuzzer is too clever and detects the magic number used in the fuzzer. 5433 This leads to weird behavior and false positives. 5434 54352019-04-25 Tim Rühsen <tim.ruehsen@gmx.de> 5436 5437 * fuzz/*.in: Update fuzz corpora from OSS-fuzz 5438 54392019-04-25 Tim Rühsen <tim.ruehsen@gmx.de> 5440 5441 * fuzz/get_ossfuzz_corpora: mkdir *.in directories 5442 54432019-04-25 Tim Rühsen <tim.ruehsen@gmx.de> 5444 5445 * fuzz/get_ossfuzz_corpora: Remove -f from unzip 5446 54472019-04-25 Tim Rühsen <tim.ruehsen@gmx.de> 5448 5449 * libwget/iri.c (_iri_unescape_inline): Fix integer overflow (harmless) 5450 54512019-04-18 Kumar Mallikarjuna <kumarmallikarjuna1@gmail.com> 5452 5453 Add TLS 1.3 post-handshake authentication 5454 * tests/Makefile.am: Loading post-handshake-auth test 5455 * tests/libtest.c: 5456 Added execution of 'WGET_TEST_POST_HANDSHAKE_AUTH' key. 5457 Value: other than 0 (Check for post handshake authentication) 5458 Value: 0 (Don't check) 5459 * tests/libtest.h: Defined 'WGET_TEST_POST_HANDSHAKE_AUTH' 5460 * tests/test-post-handhshake-auth.c: Added to test post handshake authentication support 5461 54622019-04-24 Tim Rühsen <tim.ruehsen@gmx.de> 5463 5464 Fix heap buffer overflow introduced in previous commit 5465 5466 * libwget/iri.c (_iri_unescape_inline): Add missing 'continue' 5467 5468 Found by OSS-FUZZ (Issue 14428) 5469 54702019-04-23 Tim Rühsen <tim.ruehsen@gmx.de> 5471 5472 Add basic HTML entity decoding 5473 5474 * libwget/iri.c (_iri_unescape_inline): Decode basic HTML entities 5475 * tests/test-base.c: Add test for &, #ddd; and #xHH; 5476 5477 This only cares for the basic HTML entities mentioned in RFC1866. 5478 This commit closes Gitlab issue 44, though it is not a full handling 5479 of entities. But not sure if anyone ever requests it. 5480 5481 Fixes #44 5482 54832019-04-22 Rohan Fletcher <rohfle@gmail.com> 5484 5485 Added tests for limit-rate 5486 5487 * tests/test-limit-rate.c: new test file for limit-rate 5488 * tests/Makefile.am: added tests/test-limit-rate.c to build 5489 54902019-04-22 Rohan Fletcher <rohfle@gmail.com> 5491 5492 Progress bar speed now calculated earlier 5493 5494 * libwget/bar.c: added ring_used to _bar_slot_t. 5495 speed calculations now use the first slot while the ring is filling 5496 54972019-04-20 Rohan Fletcher <rohfle@gmail.com> 5498 5499 Added limit-rate feature (#408) 5500 5501 * src/options.c: new --limit-rate command-line args 5502 * src/wget_options.h: new config variable for limit-rate cli arg 5503 * src/wget.c: new limit_transfer_rate function. Added limit rate state 5504 vars to struct _body_callback_context 5505 (_get_body): add call to limit_transfer_rate when limit-rate enabled 5506 (http_send_request): init limit rate state vars 5507 55082019-04-21 Tim Rühsen <tim.ruehsen@gmx.de> 5509 5510 Fix parsing float values for CLI options 5511 5512 * src/options.c: New function _parse_double_modifier(), 5513 (parse_numbytes): use _parse_double_modifier() instead of sscanf(), 5514 (parse_timeout): Likewise 5515 5516 Fixes #432 5517 55182019-03-21 Kumar Mallikarjuna <kumarmallikarjuna314@gmail.com> 5519 5520 Fix for -p -nc Bug 5521 5522 * src/wget.c: Modified to stop redownload if document already exists 5523 * tests/test-p-nc.c: Added test for '-p -nc' 5524 * tests/Makefile.am: Modified to include test-p-nc.c while testing. 5525 5526 Fixes #209 5527 55282019-04-21 Tim Rühsen <tim.ruehsen@gmx.de> 5529 5530 * AUTHORS: Add Kumar Mallikarjuna [ci skip] 5531 55322019-04-18 Tim Rühsen <tim.ruehsen@gmx.de> 5533 5534 * .gitlab-ci.yml: Use buildenv-fedora for FEDORA_BUILD 5535 55362019-04-18 Tim Rühsen <tim.ruehsen@gmx.de> 5537 5538 Change /builds/common to /usr/local 5539 55402019-04-15 Kumar Mallikarjuna <kumarmallikarjuna314@gmail.com> 5541 5542 Fix for --stats-all not working 5543 * src/options.c: Modified 'parse_stats_all()' for 'val' to be non-null. 5544 55452019-04-14 Tim Rühsen <tim.ruehsen@gmx.de> 5546 5547 Add TCP FastOpen Linux 4.11+ style 5548 5549 * libwget/net.c: #define TCP_FASTOPEN_LINUX_411, 5550 (_set_socket_options): Set TCP_FASTOPEN_CONNECT socket option, 5551 (wget_tcp_connect): Add code for Linux 4.11+ 5552 55532019-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 5554 5555 * contrib/mingw: Move WINPATH further up 5556 55572019-04-07 Tim Rühsen <tim.ruehsen@gmx.de> 5558 5559 * contrib/mingw: Fix pkg-config and gnutls 5560 55612019-04-02 Tim Rühsen <tim.ruehsen@gmx.de> 5562 5563 * contrib/spell-checker: Fix for 'te' in example URL 5564 55652019-04-02 Tim Rühsen <tim.ruehsen@gmx.de> 5566 5567 Skip blacklisting URLs from STDIN + tty 5568 5569 * src/wget.c: Add URL_FLG_NO_BLACKLISTING, 5570 (add_url_to_queue): New param 'flags', skip blacklisting 5571 when URL_FLG_NO_BLACKLISTING is given, 5572 (input_thread): Add description, use URL_FLG_NO_BLACKLISTING 5573 5574 This change allows us to use interactive services like 5575 (b="http://te.ttr.is:8003"; echo $b; \ 5576 while read -sN1 a; do echo "$b/$a"; done) | wget2 --input-file=- -qO- 5577 (h,j,k,l = left, down, turn, right; ctrl-c = stop) 5578 55792019-04-02 Tim Rühsen <tim.ruehsen@gmx.de> 5580 5581 * src/wget.c (input_thread): Fix communication with main thread 5582 55832019-03-31 Darshit Shah <darnir@gnu.org> 5584 5585 Add Last-Modified and Content-Type to CSV stats 5586 5587 * src/stats_site.c(struct site_stats_t): Add members last_modified and 5588 mime_type to store the last modified time and content-type respectively 5589 (stats_site_add): Store the mime type and last modification time of the 5590 file into the stats 5591 (free_stats): Free the heap-allocated mime_type field 5592 (print_csv): Add columns for last-modified and content-type to CSV 5593 header 5594 (print_csv_entry): Add the data for the above two columns 5595 55962019-03-26 Darshit Shah <darnir@gnu.org> 5597 5598 Support XDG Desktop Specification for config files 5599 5600 Adds functionality to Wget2 to support the XDG Base Directory 5601 Specification[1]. Also marks as deprecated the old location of the 5602 config file. 5603 5604 * docs/wget2.md: Explicitly mark `$HOME/.wget2rc` as deprecated. 5605 * po/potfiles.in: utils.c now has a translateable stirng 5606 * src/options.c(read_config): Use bool instead of int 5607 (get_config_files): Now takes config_home and user_home as input. 5608 `user_home` is only temporary and will be removed once support for 5609 ~/.wget2rc is removed. 5610 (get_xdg_data_home): Derives the location where the data files are to be 5611 stored according to [1]. 5612 (get_xdg_config_home): Derives the location where the config files are 5613 to be stored according to [1]. 5614 (init): Store all the data files in $XD_DATA_HOME 5615 * src/wget.c(mkdir_path): Move function to utils.c 5616 * src/utils.c(mkdir_path): Move function from wget.c. Can now handle 5617 both file and directory paths. 5618 * src/utils.h: Add declaration for mkdir_path 5619 5620 [1]: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html 5621 56222019-02-16 Darshit Shah <darnir@gnu.org> 5623 5624 Improve handling of initialization files 5625 5626 * docs/wget2.md: Add section on initialization files and the order in 5627 which they are read. 5628 * src/options.c(struct config config): Set system_config to the default 5629 value of "SYSCONFDIR/wget2rc" 5630 (struct optionw): --config now maps to config.user_config and update the 5631 value of the default path 5632 (read_config): Read the configuration for system and user configs, if 5633 they're set 5634 (get_config_files): New function that sets the values for 5635 config.system_config and config.user_config. For the system config it 5636 picks either the file in `$SYSTEM_WGET2RC` or at 5637 `/usr/local/etc/wget2rc`. For the user config, it picks the first file from 5638 `--config`, `$WGET2RC`, `$XDG_CONFIG_HOME/wget/wget2rc`, 5639 `~/.config/wget/wget2rc` or `$HOME/.wget2rc`. 5640 (init): Remove the code for finding all the config files and move it to 5641 `get_config_file`. 5642 (deinit): Free config.user_config and config.system_config. Remove 5643 config.config_files. 5644 * src/wget_options.h(struct config): New char * user_config and 5645 system_config. Remove config.config_files 5646 56472019-02-16 Darshit Shah <darnir@gnu.org> 5648 5649 Get system wget2rc file 5650 5651 * src/Makefile.am: Don't create a separate directory for Wget2 in 5652 SYSCONFDIR 5653 * src/options.c: If SYSTEM_WGET2RC is not set, then try to read the 5654 global Wget2rc from SYSCONFDIR 5655 56562019-03-01 Tim Rühsen <tim.ruehsen@gmx.de> 5657 5658 * libwget/encoding.c (wget_str_to_ascii): Use IDN2_USE_STD3_ASCII_RULES 5659 5660 We convert hostnames and thus have to apply IDN2_USE_STD3_ASCII_RULES. 5661 If we don't do, the result could contain any ascii characters, 5662 e.g. 'evil.c\u2100.example.com' will be converted into 5663 'evil.ca/c.example.com', which seems no good idea. 5664 56652019-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 5666 5667 * fuzz/libwget_http_client_fuzzer.c: Fix for OpenBSD 5668 56692019-02-16 Darshit Shah <darnir@gnu.org> 5670 5671 * docs/wget2.md: Fix typo in identation of --dns-cache 5672 56732019-02-16 Darshit Shah <darnir@gnu.org> 5674 5675 * Pull gnulib forward 5676 56772019-02-14 Tim Rühsen <tim.ruehsen@gmx.de> 5678 5679 * docs/wget2.md: Correct --logfile -> --output-file 5680 56812019-01-23 Leon Klingele <git@leonklingele.de> 5682 5683 * docs/wget2.md: --no-cache also sets the 'Cache-Control: no-cache' header 5684 5685 Copyright-paperwork-exempt: Yes 5686 56872019-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 5688 5689 * .gitlab-ci.yml: Don't enable valgridn tests for Arch runner 5690 56912019-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 5692 5693 * .gitlab-ci.yml: Do not call pacman -Q 5694 56952019-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 5696 5697 * bootstrap.conf: Prerequisite 'rsync' removed 5698 56992019-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 5700 5701 Fix YAML syntax error 5702 57032019-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 5704 5705 Add Debian Testing CI runner 5706 57072019-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 5708 5709 Really use Arch runner in CI 5710 57112019-01-04 Tim Rühsen <tim.ruehsen@gmx.de> 5712 5713 * unit-tests/test.c (test_stringmap): Fix warning (shadowed variable) 5714 57152019-01-04 Tim Rühsen <tim.ruehsen@gmx.de> 5716 5717 Fix hashmap + stringmap docs 5718 5719 * include/wget/wget.h: Add docs for hashmap/stringmap typedefs, 5720 fix argument of wget_hashmap_iterator_free() 5721 * libwget/hashmap.c: Add docs, fix wget_hashmap_iterator_free() 5722 * libwget/stringmap.c: Add docs 5723 * unit-tests/test.c: Fix call to wget_stringmap_iterator_free() 5724 57252019-01-03 Tim Rühsen <tim.ruehsen@gmx.de> 5726 5727 Add iterators for hashmap and stringmap 5728 5729 * include/wget/wget.h: New functions wget_hashmap_iterator_alloc(), 5730 wget_hashmap_iterator_free(), wget_hashmap_iterator_next(), 5731 also for stringmap. 5732 * libwget/hashmap.c: Implementation of the above functions 5733 * libwget/stringmap.c: Add wget_stringmap_iterator_next() 5734 * unit-tests/test.c: Add tests using stringmap iterator 5735 5736 This functionality is sometimes more handy than the *_browse() API 5737 with callback functions and context variables. 5738 57392019-01-03 Tim Rühsen <tim.ruehsen@gmx.de> 5740 5741 * libwget/http.c (wget_http_add_credentials): Code cleanup 5742 57432019-01-02 Tim Rühsen <tim.ruehsen@gmx.de> 5744 5745 Rename container set functions 5746 5747 * include/wget/wget.h: 5748 rename wget_vector_set_growth_policy() to wget_vector_set_resize_factor(), 5749 rename wget_hashmap_set_growth_policy() to wget_hashmap_set_resize_factor(), 5750 rename wget_hashmap_setloadfactor() to wget_hashmap_set_load_factor(), 5751 rename wget_stringmap_setloadfactor() to wget_stringmap_set_load_factor(), 5752 rename wget_stringmap_set_growth_policy() to wget_stringmap_set_resize_factor(). 5753 * libwget/hashmap.c: Likewise 5754 * libwget/stringmap.c: Likewise 5755 * libwget/vector.c: Likewise 5756 57572019-01-02 Tim Rühsen <tim.ruehsen@gmx.de> 5758 5759 Simplify hasmap/stringmap/vector API 5760 5761 * include/wget/wget.h: Remove wget_hashmap_get_null() and 5762 wget_stringmap_get_null(), 5763 change params and return value of wget_hashmap_get() and 5764 wget_stringmap_get(). 5765 * libwget/hashmap.c: Remove wget_hashmap_get_null(), 5766 change params and return value of wget_hashmap_get(), 5767 simplify hashmap_find_entry(). 5768 * libwget/stringmap.c: Remove wget_stringmap_get(), 5769 change params and return value of wget_stringmap_get(). 5770 * libwget/dns_cache.c: Amend for new API 5771 * libwget/hpkp.c: Likewise 5772 * libwget/hsts.c: Likewise 5773 * libwget/http.c: Likewise 5774 * libwget/netrc.c: Likewise 5775 * libwget/ocsp.c: Likewise 5776 * libwget/tls_session.c: Likewise 5777 * src/host.c: Likewise 5778 * src/options.c: Likewise 5779 * src/plugin.c: Likewise 5780 * src/stats_site.c: Likewise 5781 * unit-tests/test.c: Likewise 5782 57832019-01-02 Tim Rühsen <tim.ruehsen@gmx.de> 5784 5785 Use float for container growth policy 5786 5787 * include/wget/wget.h: Use float for *_set_growth_policy() 5788 * libwget/hashmap.c: Use float for wget_hashmap_set_growth_policy() 5789 * libwget/stringmap.c: Use float for wget_stringmap_set_growth_policy() 5790 * libwget/vector.c: Use float for wget_vector_set_growth_policy() 5791 5792 Additionally, the +/- logic has been reversed in the above functions. 5793 57942019-01-02 Tim Rühsen <tim.ruehsen@gmx.de> 5795 5796 Update copyrigght to 2019 5797 57982019-01-02 Tim Rühsen <tim.ruehsen@gmx.de> 5799 5800 * src/version-text.h.in: Use ASCII only chars 5801 58022018-12-30 Tim Rühsen <tim.ruehsen@gmx.de> 5803 5804 Fix xattr behavior (CVE-2018-20483) 5805 5806 * src/options.c (config): Disable --xattr by default 5807 * src/wget.c (set_file_metadata): Store origin URL without user/pw, 5808 store referer just as scheme/host/port 5809 58102018-12-28 Tim Rühsen <tim.ruehsen@gmx.de> 5811 5812 * .gitlab-ci.yml: Spell check 5813 58142018-12-28 Tim Rühsen <tim.ruehsen@gmx.de> 5815 5816 Fix typos found by contrib/spell-checker 5817 58182018-12-28 Tim Rühsen <tim.ruehsen@gmx.de> 5819 5820 * contrib/spell-checker: Add script for spell checking 5821 58222018-12-28 Tim Rühsen <tim.ruehsen@gmx.de> 5823 5824 * fuzz/Makefile.am: Fix order of libraries for linking [skip ci] 5825 58262018-12-27 Rafael Fontenelle <rafaelff@gnome.org> 5827 5828 Fix misspellings 5829 58302018-12-26 Tim Rühsen <tim.ruehsen@gmx.de> 5831 5832 * README.md: Add libzstd as optional dependency 5833 58342018-12-25 Tim Rühsen <tim.ruehsen@gmx.de> 5835 5836 * libwget/utils.c: Cleanup wget_match_tail(_nocase) 5837 58382018-12-20 Tim Rühsen <tim.ruehsen@gmx.de> 5839 5840 * tests/test-redirection.c: Add test for Location URI with spaces in filename 5841 58422018-12-20 Tim Rühsen <tim.ruehsen@gmx.de> 5843 5844 * tests/libtest.c (_answer_to_connection): Allow 3xx for redirection 5845 58462018-12-20 Tim Rühsen <tim.ruehsen@gmx.de> 5847 5848 * src/wget.c (src/wget.c): Small code cleanups 5849 58502018-12-20 Tim Rühsen <tim.ruehsen@gmx.de> 5851 5852 * libwget/http_parse.c (wget_http_parse_location): Being more lenient 5853 5854 Some servers don't percent encode the URI in a Location: header. 5855 Having spaces in the URI leads to magled file names and likely 404. 5856 5857 Fixes #420 5858 58592018-12-19 Tim Rühsen <tim.ruehsen@gmx.de> 5860 5861 Handle ldconfig for unprivileged users 5862 58632018-12-19 Tim Rühsen <tim.ruehsen@gmx.de> 5864 5865 Call 'ldconfig' at the end of 'make install' 5866 58672018-12-13 Tim Rühsen <tim.ruehsen@gmx.de> 5868 5869 Revert "Fix -k/--convert-links" 5870 5871 This reverts commit 89ff57ee937e6c70d3927c00a9dda75f33947238. 5872 5873 The commit fixed a bug by generating a regression at another place. 5874 See https://gitlab.com/gnuwget/wget2/issues/415#note_124641970 5875 58762018-12-12 Tim Rühsen <tim.ruehsen@gmx.de> 5877 5878 libwget/http.c (wget_http_add_credentials): Compare case independent 5879 5880 Fixes an issue with MHD 9.62 that uses a lowercase 'md5' string. 5881 58822018-12-03 Leif Ryge <leif@synthesize.us> 5883 5884 * docs/wget2.md: Fix typo s/exists/exits 5885 5886 Copyright-paperwork-exempt: Yes 5887 58882018-11-18 Tim Rühsen <tim.ruehsen@gmx.de> 5889 5890 * src/wget_xattr.h: Fix getxattr args on OSX/FreeBSD 5891 58922018-11-17 Tim Rühsen <tim.ruehsen@gmx.de> 5893 5894 * libwget/Makefile.am: Do remove css_tokenizer.c with 'make clean' 5895 58962018-11-17 Tim Rühsen <tim.ruehsen@gmx.de> 5897 5898 * contrib/check_options: Exit if not run in bash 5899 59002018-11-17 Tim Rühsen <tim.ruehsen@gmx.de> 5901 5902 * .gitlab-ci.yml: Add CI tarball build on basic Alpine 5903 59042018-11-17 Darshit Shah <darnir@gnu.org> 5905 5906 Fix VPATH build broken in earlier commit 5907 5908 * tests/Makefile.am: Add new file plugin_tests.h to DIST Files 5909 * tests/plugin_tests.h: Use BUILDDIR not SRCDIR for .libs location 5910 59112018-11-14 Darshit Shah <darnir@gnu.org> 5912 5913 Fix bug in dns_cache_preload 5914 5915 * src/options.c(options): --dns-cache-preload does expect a value. It is 5916 not an option. 5917 59182018-11-14 Darshit Shah <darnir@gnu.org> 5919 5920 Use a common header file to deduplicate plugin test boilerplate 5921 5922 * tests/test_plugin.h: Add the OBJECT_DIR and other defines in a single 5923 common file for all plugin tests 5924 * tests/test-plugin*.c: Remove boilerplate code and include 5925 "test_plugin.h" 5926 59272018-11-14 Tim Rühsen <tim.ruehsen@gmx.de> 5928 5929 * docs/wget2.md: Ad -r to example for --filter-mime-type [skip ci] 5930 59312018-10-23 Tim Rühsen <tim.ruehsen@gmx.de> 5932 5933 Use extended file attributes for server's file timestamp 5934 5935 * src/wget_xattr.h: Add fgetxattr definitions (Linux, Apple, FreeBSD) 5936 * src/wget.c (set_file_metadata): Save "user.last_modified" attribute, 5937 (read_xattr_metadata): Add function, 5938 (write_xattr_metadata): Cleanup code, 5939 (http_create_request): Get timestamp from xattr if available 5940 59412018-10-16 Josef Moellers <jmoellers@suse.de> 5942 5943 Handle wide characters in download progress bar 5944 5945 * bootstrap.conf: Add gnulib module 'wcwidth' 5946 * libwget/bar.c: New function inspect_multibyte(), 5947 (_bar_update_slot): Make use of inspect_multibyte() 5948 59492018-10-31 Tim Rühsen <tim.ruehsen@gmx.de> 5950 5951 * src/options.c (init): Avoid error msg if HSTS preload file isn't given 5952 59532018-10-31 Tim Rühsen <tim.ruehsen@gmx.de> 5954 5955 Fix -k/--convert-links 5956 5957 When different URLs result in the same local file so that 5958 -k converts a file several times, that file becomes corrupted. 5959 5960 This commit fixes it and adds a test case. 5961 5962 Reported-by: rgpublic on Gitab.com 5963 Fixes #411 5964 59652018-10-21 Tim Rühsen <tim.ruehsen@gmx.de> 5966 5967 Add valgrind suppressions for TravisCI (due to old libidn) 5968 5969 * fuzz/valgrind-suppressions: Add suppression 5970 * tests/valgrind-suppressions: Add suppression 5971 59722018-10-19 Tim Rühsen <tim.ruehsen@gmx.de> 5973 5974 Suppress some leaks detected by valgrind in fuzz/ 5975 5976 * fuzz/main.c: Use --suppressions and --gen-suppressions 5977 * fuzz/valgrind-suppressions: New suppression file 5978 5979 Some library's memory can't be freed. So we suppress what is needed 5980 to let the CIs survive. 5981 59822018-10-19 Tim Rühsen <tim.ruehsen@gmx.de> 5983 5984 * fuzz/*_fuzzer.in/*: Update fuzzer copora from OSS-Fuzz 5985 59862018-10-19 Tim Rühsen <tim.ruehsen@gmx.de> 5987 5988 * fuzz/get_ossfuzz_corpora: Use faster corpus download from OSS-Fuzz 5989 59902018-10-19 Tim Rühsen <tim.ruehsen@gmx.de> 5991 5992 * .gitignore: Add fuzz/*_fuzer.new directories 5993 59942018-10-19 Tim Rühsen <tim.ruehsen@gmx.de> 5995 5996 * src/options.c (deinit): Free config.accept_regex and config.reject_regex 5997 59982018-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 5999 6000 * contrib/csv2tree: Remove superfluous 'import json' [skip ci] 6001 60022018-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 6003 6004 * include/wget/wget.h: Fix documentation of HPKP return values 6005 60062018-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 6007 6008 * .lgtm.yml: Classify test directories 6009 60102018-10-18 Darshit Shah <darnir@gnu.org> 6011 6012 Fix progress bar display in VTE Terminals 6013 6014 GNOME VTE removed support for VT-100 sequences to save and restore 6015 cursor position that libwget progress display was using. Their suggested 6016 approach is to use the DECSC/DECRC sequences from VT-520 instead of the 6017 SCOSC/SCORC sequences from VT-100. Since the DECSC/DECRC sequences are 6018 also supported by all other terminal emulators, we make the switch here 6019 in order to suport VTE emulators as well. 6020 Upstream Report: https://gitlab.gnome.org/GNOME/vte/issues/48 6021 6022 * libwget/bar.c(_bar_print_slot): Use ESC 7 (DECSC) instead of CSI s (SCOSC) 6023 (wget_bar_write_line): Same 6024 (_restore_cursor_position): Use ESC 8 (DECRC) instead of SCI u (SCORC) 6025 60262018-10-17 Darshit Shah <darnir@gnu.org> 6027 6028 * benchmarks/benches/convert.gp: Add generated file warning 6029 6030 [ci skip] 6031 60322018-10-14 Darshit Shah <darnir@gnu.org> 6033 6034 Remove old benchmark scripts 6035 6036 * benchmarks/bench_https_*: Deleted. Superceded by new scripts 6037 * benchmarks/benchmark.sh: Deleted. Superceded by new scripts 6038 * benchmarks/new_benchmark.sh: Renamed to benchmark.sh 6039 6040 [ci skip] 6041 60422018-10-14 Darshit Shah <darnir@gnu.org> 6043 6044 Add a small harness for Benchmarking code 6045 6046 [ci skip] 6047 This set of script works through the new_benchmark.sh script. 6048 I've tried to keep everything modular, extensible and clean. 6049 6050 * benchmarks/new_benchmark.sh: New benchmark harness 6051 * benchmarks/sources/*.bench.sh: Program specification files 6052 * benchmarks/benches/http{1,2}.sh: Benchmark specification files 6053 * benchmarks/benches/convert.gp: A gnuplot script to convert the 2 6054 column data into four columns for making the errorbars plot 6055 60562018-10-16 Darshit Shah <darnir@gnu.org> 6057 6058 * .gitignore: Ignore version-text.h, a generated file 6059 60602018-10-16 Tim Rühsen <tim.ruehsen@gmx.de> 6061 6062 * .lgtm.yml: Exclude unit-tests/files 6063 60642018-08-17 jmoellers <jmoellers@suse.de> 6065 6066 Fix issue with the number of bytes downloaded vs accounted for in the progress bar. 6067 6068 * include/wget/wget.h(wget_bar_slot_begin): Now accepts another 6069 parameter, `new_file`. 6070 * libwget/bar.c(wget_bar_slot_begin): Support `new_file` parameter. 6071 Allows calling mehod multiple times for the same file. 6072 * src/bar.c(bar_slot_begin): Same 6073 * src/wget_bar.h: Same 6074 * src/wget.c(_get_body): Pass only the difference (number of bytes 6075 actually downloaded since last call) to the progress routines 6076 60772018-08-17 jmoellers <jmoellers@suse.de> 6078 6079 Fix a bug in HTTP/2 downloads that caused incorrect progress display 6080 6081 * libwget/bar.c(wget_bar_slot_begin): Display filename when only 1 file 6082 per slot. Else, display total number of files in that slot. 6083 (wget_bar_slot_downloaded): The length now represents an incremental 6084 value, not the absolute number of bytes. 6085 * src/wget.c(_get_body): Same. 6086 60872018-10-14 Darshit Shah <darnir@gnu.org> 6088 6089 * .gitlab-ci.yml: Skip downloading po files during bootstrap 6090 60912018-10-14 Tim Rühsen <tim.ruehsen@gmx.de> 6092 6093 * src/options.c (deinit): Free config.hsts_preload_file 6094 60952018-10-13 Darshit Shah <darnir@gnu.org> 6096 6097 * bootstrap: Update script from gnulib 6098 60992018-10-13 Darshit Shah <darnir@gnu.org> 6100 6101 Update gnulib 6102 61032018-10-13 Tim Rühsen <tim.ruehsen@gmx.de> 6104 6105 * bootstrap.conf: Use rm -f to avoid error messages 6106 61072018-10-12 Tim Rühsen <tim.ruehsen@gmx.de> 6108 6109 Add copyright,license and mailing list info to --version 6110 6111 * configure.ac: Add creating src/version-text.h from .in file 6112 * src/Makefile.am: Add version-text.h to wget2_SOURCES 6113 * src/options.c (print_version): Print text from version-text.h 6114 * src/version-text.h.in: New template file 6115 61162018-10-12 Tim Rühsen <tim.ruehsen@gmx.de> 6117 6118 Add zstd decompression (RFC8478) 6119 6120 * README.md: Mention zstandard 6121 * configure.ac: Check for libzstd 6122 * docs/wget2.md: Document new --compression type 6123 * include/wget/wget.h: Add wget_content_encoding_zstd 6124 * libwget/decompressor.c: Implement the zstd decompressor 6125 * libwget/http_parse.c (wget_http_parse_content_encoding): Add zstd 6126 * src/options.c (print_version): Add +/-zstd for --version, 6127 (parse_compression): Check for wget_content_encoding_zstd, 6128 (options): Add zstd to help text 6129 * src/wget.c (http_create_request): Add zstd to Accept-Encoding: 6130 * tests/test-compression.c: Add test for zstd 6131 61322018-09-22 Darshit Shah <darnir@gnu.org> 6133 6134 Remove support for libunistring and libidn2 < 0.14 6135 6136 * configure.ac: Remove support for libidn2 < 0.14 and libunistring 6137 * libwget/encoding.c: Same 6138 * README.md: Same 6139 61402018-10-09 Tim Rühsen <tim.ruehsen@gmx.de> 6141 6142 * tests/test-include-and-exclude-directories.c: Use double quote 6143 61442018-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 6145 6146 * tests/libtest.c (_answer_to_connection): Fix check for non-200 status codes 6147 61482018-07-04 Tim Rühsen <tim.ruehsen@gmx.de> 6149 6150 Add option --save-content-on 6151 6152 * docs/wget2.md: Add docs for --save-content-on 6153 * src/options.c (options): Add "save-content-on" 6154 * src/wget.c: 6155 * src/wget_options.h (struct config): Add save_content_on member 6156 * tests/Makefile.am: Add tests/test--save-content-on.c 6157 * tests/test--save-content-on.c: New test file 6158 61592018-10-03 Tim Rühsen <tim.ruehsen@gmx.de> 6160 6161 Add support for Chromium's HSTS Preload List 6162 6163 * README.md: Add libhsts as optional dependency 6164 * configure.ac: Add test for libhsts 6165 * docs/wget2.md: Add text for --hsts-preload and --hsts-preload-file 6166 * libwget/init.c: 6167 * src/options.c (print_version): Add +/-hsts for --version, 6168 (options): Add --hsts-preload and --hsts-preload-file, 6169 (init): Load the HSTS peload list, 6170 (deinit): Unload the HSTS preload list 6171 * src/wget.c (try_connection): Eventually change http to https 6172 * src/wget_options.h (config): Add hsts_preload and hsts_preload_file 6173 61742018-10-01 Tim Rühsen <tim.ruehsen@gmx.de> 6175 6176 * README.md: Add libpcre + libpcre2 as optional deps [skip ci] 6177 61782018-09-28 Tim Rühsen <tim.ruehsen@gmx.de> 6179 6180 * src/wget.c (add_url): Fix -H / --span-hosts 6181 61822018-09-25 DalmeGNU <dalmemail@gmail.com> 6183 6184 * docs/wget2.md: Fixed broken link on guide index 61852018-09-23 Darshit Shah <darnir@gnu.org> 6186 6187 * tests/valgrind-suppressions: Suppress issues from GPGME library 6188 61892018-09-21 Darshit Shah <darnir@gnu.org> 6190 6191 * src/wget.c (process_response): Just an aesthetic improvement to the code 6192 61932018-09-21 Darshit Shah <darnir@gnu.org> 6194 6195 Ensure that robots.txt is honoured even with -c 6196 6197 There was a bug that prevented the robots.txt from being parsed when 6198 resuming an incomplete download. This would cause the robots rules to 6199 not be honoured when resuming a broken download. 6200 6201 * src/wget.c (process_response): Move the robots check out of the 6202 conditional, so it runs in all valid cases 6203 (http_create_request): In case -c has been passed and we see an existing 6204 robots.txt, remove it. As a result, we will always download a fresh 6205 copy of the robots.txt file 6206 * tests/test-robots.c: Add a new test to ensure the robots file is 6207 always being parsed 6208 62092018-08-24 Darshit Shah <darnir@gnu.org> 6210 6211 Add some debug messages when a URI is not added to queue 6212 6213 * src/wget.c: Likewise 6214 * src/blacklist.c: Likewise 6215 62162018-09-21 Darshit Shah <darnir@gnu.org> 6217 6218 Minor changes to the commit-check script 6219 62202018-09-19 Tim Rühsen <tim.ruehsen@gmx.de> 6221 6222 * configure.ac: Fix searching for libpcre2 6223 62242018-09-19 DalmeGNU <dalmemail@gmail.com> 6225 6226 Add Building with alternate GnuTLS version doc 6227 62282018-09-16 DalmeGNU <dalmemail@gmail.com> 6229 6230 Add URL Encoding documentation 6231 62322018-09-16 DalmeGNU <dalmemail@gmail.com> 6233 6234 Update TestSuite.md & fix pipeline 6235 62362018-09-16 DalmeGNU <dalmemail@gmail.com> 6237 6238 Add test suite documentation 6239 62402018-09-14 DalmeGNU <dalmemail@gmail.com> 6241 6242 Add Contributing Guide 6243 62442018-09-14 DalmeGNU <dalmemail@gmail.com> 6245 6246 Add OptionHandling documentation 6247 62482018-09-17 Tim Rühsen <tim.ruehsen@gmx.de> 6249 6250 * src/options.c (options): List all TLS options for --secure-protocol 6251 62522018-09-16 Ander Juaristi <a@juaristi.eus> 6253 6254 Fix SSL/TLS Doxygen documentation 6255 62562018-09-16 Ander Juaristi <a@juaristi.eus> 6257 6258 Update AUTHORS file 6259 62602018-09-15 Tim Rühsen <tim.ruehsen@gmx.de> 6261 6262 * contrib/commit-check: Add support for companies (e.g. redhat.com) 6263 62642018-08-31 DalmeGNU <dalmemail@gmail.com> 6265 6266 Update documentation about how to add a new command line option 6267 6268 * src/options.c: Update comments on how to add a new option 6269 * docs/faq.md: Document how to add a new CLI-option 6270 62712018-09-01 Tim Rühsen <tim.ruehsen@gmx.de> 6272 6273 Fix buffer overflow in -X/-I parsing 6274 6275 * src/options.c (set_char_prefix): Fix backslash handling 6276 * fuzz/wget_options_fuzzer.in/: Add reproducer 6277 6278 This bug was introduced yesterday by commit 6279 756e470de942ec830a9a424a2340b7e986e9fe7c 6280 6281 Found by oss-fuzz (issue 10163). 6282 62832018-09-01 Tim Rühsen <tim.ruehsen@gmx.de> 6284 6285 Fix buffer overflow in -X/-I parsing 6286 6287 * src/options.c (set_char_prefix): Fix scratch buffer size 6288 * fuzz/wget_options_fuzzer.in/: Add reproducer 6289 6290 This bug was introduced yesterday by commit 6291 756e470de942ec830a9a424a2340b7e986e9fe7c 6292 6293 Found by oss-fuzz (issue 10159). 6294 62952018-04-10 Dan <dalmemail@gmail.com> 6296 6297 Add -X/--exclude-directories and -I/--include-directories 6298 6299 * docs/wget2.md: Add docs for -I/--include-directories=list 6300 and -X/--exclude-directories=list 6301 * src/options.c: New function _strchrnul_esc(), 6302 new function _strmemdup_esc(), 6303 (parse_stringlist_expand): Use new functions from above, 6304 new function set_char_prefix(), parse_included_directories() 6305 and parse_excluded_directories, 6306 (options): add exclude-directories and --include-directories 6307 * src/wget.c: New functions match_subdir() and in_directory_pattern_list(), 6308 (add_url): check if path matches directory patterns, 6309 (_prepare_file): add new param 'path', 6310 * src/wget_options.h (struct config): Add member 'exclude_directories' 6311 * tests/Makefile.am: Add 'test-include-and-exclude-directories' 6312 * tests/test-include-and-exclude-directories.c: New file 6313 63142018-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 6315 6316 Rename --dns-caching to --dns-cache (Wget1.x compatibility) 6317 6318 * docs/wget2.md: --dns-caching -> --dns-cache 6319 * src/options.c (options): Likewise 6320 63212018-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 6322 6323 * docs/Makefile.am: Add README.md to DISTCLEANFILES 6324 63252018-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 6326 6327 Improve online documentation 6328 6329 * README.md: Move some dev info to faq.md 6330 * docs/Makefile.am: Generate introduction from README.md 6331 * docs/faq.md: Started with User and Dev sections 6332 * docs/libwget.doxy.in: Remove mainpage, use README.md instead 6333 * docs/mainpage.md: Remove 6334 * docs/wget2.md: Amended intro to options that take lists 6335 63362018-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 6337 6338 * fuzz/libwget_base64_fuzzer.c: Silence scan-build 6339 63402018-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 6341 6342 Document wget_base64_get_(en|de)coded_length() and move to base64.c 6343 6344 * include/wget/wget.h: Declare both function as pure 6345 * libwget/base64.c: Add both functions incl. documentation, 6346 (wget_base64_decode_alloc): Remove hard-coded calculation, 6347 (wget_base64_encode_alloc): Likewise. 6348 * libwget/tls_session.c (_tls_session_save): 6349 Use wget_base64_get_encoded_length() instead hard-coded calculation 6350 * fuzz/libwget_base64_fuzzer.c: 6351 Avoid both functions to be optimized out 6352 63532018-08-21 Tim Rühsen <tim.ruehsen@gmx.de> 6354 6355 * po/POTFILES.in: Add libwget/error.c 6356 63572018-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 6358 6359 * src/wget.c: Set tries to 1 on HTTPS errors when built without TLS support 6360 63612018-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 6362 6363 * src/gpgme.c (_verify_detached_sig): Return WGET_E_GPG_VER_FAIL on signatue failure 6364 63652018-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 6366 6367 New function wget_strerror() 6368 6369 * docs/Makefile.am: Add man page build for error functions 6370 * include/wget/wget.h: Make WGET_E_* an enum wget_error_t, 6371 declare new function wget_strerror() 6372 * libwget/Makefile.am: Add error.c 6373 * libwget/error.c: New file 6374 63752018-08-17 Darshit Shah <darnir@gnu.org> 6376 6377 * src/wget_options.h: Add the const qualifier to valid options 6378 63792018-08-17 Darshit Shah <darnir@gnu.org> 6380 6381 * libwget/http.c(_print_frame_type): Don't print anything when frame type is DATA 6382 63832018-08-17 Tim Rühsen <tim.ruehsen@gmx.de> 6384 6385 Add base64 URL/file safety 6386 6387 * include/wget/wget.h: Add wget_base64_urlencode() 6388 * libwget/base64.c: New function wget_base64_urlencode(), 6389 new function _wget_base64_encode(), 6390 (base64_2_bin): add charactes '-' and '_'. 6391 63922018-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 6393 6394 * AUTHORS: Fix name of Josef Möllers 6395 63962018-08-16 Tim Rühsen <tim.ruehsen@gmx.de> 6397 6398 * contrib/commit-check: Don't check the name 6399 6400 The name can't easily be used as an identifier. 6401 E.g. my name is different through commits: 'Tim Ruehsen' or 'Tim Rühsen' 6402 6403 Checking the email address used for the FSF assignment is enough. 6404 64052018-08-16 Darshit Shah <darnir@gnu.org> 6406 6407 * AUTHORS: Josef Moellers has copyrights assigned 6408 6409 [skip ci] 6410 64112018-08-14 Tim Rühsen <tim.ruehsen@gmx.de> 6412 6413 * Makefile: Perform checks in check-local unconditionally 6414 64152018-08-14 Tim Rühsen <tim.ruehsen@gmx.de> 6416 6417 * Makefile.am: Fix local-check for VPATH build 6418 64192018-08-12 Darshit Shah <darnir@gnu.org> 6420 6421 * Makefile.am: Make check-local fail on error 6422 64232018-08-11 Tim Rühsen <tim.ruehsen@gmx.de> 6424 6425 * AUTHORS: Add Aniketh Gireesh 6426 64272018-08-08 Tim Rühsen <tim.ruehsen@gmx.de> 6428 6429 * docs/wget2.md: Add description for --dns-caching 6430 64312018-08-08 Tim Rühsen <tim.ruehsen@gmx.de> 6432 6433 Add new option --dns-cache-preload 6434 6435 * include/wget/wget.h: New function wget_tcp_dns_cache_add() 6436 * libwget/dns_cache.c (wget_dns_cache_add): Fix docs 6437 * libwget/net.c (_wget_tcp_resolve): Changed params, 6438 new function wget_tcp_dns_cache_add() 6439 * src/options.c (options): Add --dns-cache-preload, 6440 new function _preload_dns_cache(), 6441 (init): Call _preload_dns_cache() 6442 * src/wget_options.h (struct config): Add member dns_cache_preload 6443 64442018-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 6445 6446 * src/wget.c (add_url): Don't follow redirects when --max-redirects=0 6447 64482018-08-05 Tim Rühsen <tim.ruehsen@gmx.de> 6449 6450 * src/wget.c (process_response_header): Prevent read of uninitialized data 6451 64522018-08-05 DalmeGNU <dalmemail@gmail.com> 6453 6454 Add information related to the build process in Haiku [skip ci] 64552018-08-04 Tim Rühsen <tim.ruehsen@gmx.de> 6456 6457 Add example batch_loader 6458 6459 * examples/Makefile.am: Add batch_loader.c 6460 * examples/batch_loader.c: New example code 6461 6462 This example uses 500 threads to process domains / URLS from 6463 stdin. It downloads the URL and creates a stats entry in out.csv. 6464 Check the source for details. 6465 64662018-08-04 Tim Rühsen <tim.ruehsen@gmx.de> 6467 6468 Remove usage of getaddrinfo_a() and gai_* functions 6469 6470 The glibc async functions turned out to be unstable and 6471 error-prone. They also use threads to implement timeouts. 6472 If we need DNS timeouts and/or async DNS lookups, we should 6473 have a look at external resolver libraries. 6474 64752018-08-03 Tim Rühsen <tim.ruehsen@gmx.de> 6476 6477 Fix calling error handler for uncompressing 6478 6479 * ../libwget/decompressor.c (wget_decompress): Fix argument to 6480 error_handler() 6481 * ../libwget/http.c (_on_stream_close_callback): Set stream context 6482 to NULL when freeing 6483 64842018-08-04 Darshit Shah <darnir@gnu.org> 6485 6486 * src/wget.c (main): Add comment explaining why there are nthreads+2 slots 6487 64882018-08-01 Darshit Shah <darnir@gnu.org> 6489 6490 * src/options.c: Remove dot progress from help text 6491 64922018-07-29 Tim Rühsen <tim.ruehsen@gmx.de> 6493 6494 * README.md: Mention 'setarch x86' on Haiku [skip ci] 6495 64962018-07-29 Tim Rühsen <tim.ruehsen@gmx.de> 6497 6498 * configure.ac: Remove -Wabi, too loud with gcc 8 6499 65002018-07-29 Tim Rühsen <tim.ruehsen@gmx.de> 6501 6502 * src/wget.c (downloader_thread): Fix buffer overflow 6503 65042018-07-29 Darshit Shah <darnir@gnu.org> 6505 6506 Rewrte the speed reporting codepath 6507 6508 * include/wget/wget.h (wget_human_readable): Function is not a const 6509 * libwget/bar.c (_BAR_SETTINGS): Remove SPEED_REDRAW_TIME 6510 (struct _speed_report): Remove and add relevant fields to _bar_slot_t 6511 (_bar_update_speed): Removed 6512 (_bar_update_speed_stats): New function to populate the speed ring and 6513 fill in the speed_buf buffer 6514 (_bar_set_progress): Remove old commented out code 6515 (_bar_update_slot): Speed computation is moved to 6516 `_bar_update_speed_stats`. 6517 (wget_bar_set_slots): No longer need to realloc speed ring buffers 6518 (wget_bar_slot_begin): Initialize the ring_pos and the speed rings 6519 (wget_bar_deinit): Variable `speed_r` no longer used 6520 6521 In this implementation, I've made a few functional changes and some 6522 other design changes to how the speed report is computed. First, I've 6523 removed the assumption that the progress bar is updated every 125ms. 6524 This is part of the client side logic and we can't assume that in the 6525 library. Instead, the speed report is computed every time the progress 6526 bar is refreshed. The size of the speed ring and how the computation is 6527 done is documented to allow the library users to make a decision on how 6528 they want to use the progress bar. 6529 6530 While, I've currently removed SPEED_REDRAW_TIME, I do plan to bring it 6531 back soon. This will most likely be done alongside support for computing 6532 the ETA of the download. 6533 6534 In this commit, I have also made it such that the speed is shown only 6535 after the speed ring is filled in entirely. This most likely needs to 6536 change such that we show the speed immediately. However, we document 6537 that it will jitter a lot while the speed ring is being populated. 6538 65392018-07-26 Darshit Shah <darnir@gnu.org> 6540 6541 * libwget/bar.c(_bar_update_speed): Get time only once 6542 65432018-07-23 Darshit Shah <darnir@gnu.org> 6544 6545 * libwget/bar.c(_BAR_DECOR_SIZE): Account for extra space at end 6546 65472018-07-26 Tim Rühsen <tim.ruehsen@gmx.de> 6548 6549 * libwget/tls_session.c: Only update/write when really changed 6550 65512018-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 6552 6553 Remove --gnutls-options 6554 6555 * docs/wget2.md: Remove docs for --gnutls-options 6556 * include/wget/wget.h: Remove WGET_SSL_DIRECT_OPTIONS 6557 * libwget/ssl_gnutls.c: Remove use of WGET_SSL_DIRECT_OPTIONS 6558 * src/options.c: Remove --gnutls-options 6559 * src/wget_options.h: Remove config.gnutls_options 6560 6561 This option was obsoleted by --secure-protocol where you can 6562 also set an explicit priority/cipher string. 6563 65642018-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 6565 6566 Add TLSv1_x to --secure-protocol 6567 6568 * docs/wget2.md: Extend docs for --secure-protocol 6569 * libwget/ssl_gnutls.c (wget_ssl_init): Add priority handling 6570 6571 This commit adds TLSv1_1, TLSv1_2 and TLSv1_3. 6572 65732018-07-15 Tim Rühsen <tim.ruehsen@gmx.de> 6574 6575 Extend OOM handling 6576 6577 * include/wget/wget.h: Amend type of wget_oom_callback_t 6578 * libwget/xalloc.c: Amend OOM handling 6579 * src/options.c (_no_memory): Return EXIT_FAILURE 6580 * unit-tests/test.c (main): Cast to avoid warning 6581 6582 The OOM callback can now return 0 and the allocation function is tried again. 6583 65842018-07-05 Darshit Shah <darnir@gnu.org> 6585 6586 * libwget/bar.c: Set the speed modifier globally 6587 6588 This reduces the number of computations that need to be done on each bar 6589 slot update. The modifier is set only at startup, so we don't need to 6590 recompute it everytime 6591 65922018-07-05 Tim Rühsen <tim.ruehsen@gmx.de> 6593 6594 * .gitlab-ci.yml: Increase version of cache 6595 65962018-07-04 Tim Rühsen <tim.ruehsen@gmx.de> 6597 6598 Add wget_bitmap_t 6599 6600 * include/wget/wget.h: Add bitmap prototypes 6601 * libwget/Makefile.am: Add libwget/bitmap.c 6602 * libwget/bitmap.c: New file for bitmap implementation 6603 * docs/Makefile.am: Add man pages for bitmap functions 6604 * unit-tests/test.c: Add tests for bitmap functions 6605 6606 First version was written by Aniketh Girish 6607 66082018-07-04 Darshit Shah <darnir@gnu.org> 6609 6610 * libwget/bar.c: Use enums for constants instead of defines 6611 66122018-07-04 Darshit Shah <darnir@gnu.org> 6613 6614 * libwget/bar.c: Add a space to empty speed report 6615 66162018-06-11 DalmeGNU <dalmemail@gmail.com> 6617 6618 Implementation of the --retry-on-http-status option 6619 6620 * src/wget_options.h: Add http_retry_on_status (wget_vector_t) to config structure 6621 * src/options.c: Add --retry-on-http-status option 6622 * src/wget.c: Modify downloader_thread() in order to retry a download following the --retry-on-http-status option 6623 * docs/wget2.md: Add documentation for the --retry-on-http-status option 6624 66252018-06-11 DalmeGNU <dalmemail@gmail.com> 6626 6627 * libwget/bar.c: Don't print zero as speed when we can't calculate it 6628 66292018-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 6630 6631 Add new option --keep-extension 6632 6633 * src/options.c: Add option --keep-extension 6634 * src/wget.c (_open_unique): Add code for --keep-extension 6635 * src/wget_options.h (struct config): Add keep_extension member 6636 * docs/wget2.md: Add docs for --keep-extension 6637 6638 The new option --keep-extension changes the behavior for creating a 6639 unique filename if a file already exists. 6640 The standard (default) pattern is to generate <filename>.<N>, the new 6641 pattern is <basename>_<N>.<ext>. 6642 6643 The idea is use such files without renaming when the use depends on the 6644 extension, like on Windows. 6645 66462018-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 6647 6648 * .gitlab-ci.yml: Allow MinGW runner to fail 6649 6650 The MinGW runner fails due to GnuTLS randomizer failing. 6651 See https://gitlab.com/gnutls/gnutls/issues/486 6652 6653 When fixed, we will revert this commit. 6654 66552018-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 6656 6657 * .gitlab-ci.yml: Add config.h to MinGW artifacts 6658 66592018-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 6660 6661 * libwget/http.c (wget_http_get_response_cb): Fix clang warning 6662 66632018-06-20 Tim Rühsen <tim.ruehsen@gmx.de> 6664 6665 * src/wget.c (_add_authorize_header): Fix tests for auth scheme name 6666 6667 Reported-by: @Con7e (gitlab.com) 6668 66692018-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 6670 6671 Fix http2 slowness issue 6672 6673 We saw many WINDOW_UPDATE frames while debugging, so we tried to 6674 fix the window sizes for connection and stream windows to a large value. 6675 6676 This patch also simplifies the receive loop. 6677 66782018-06-13 Tim Rühsen <tim.ruehsen@gmx.de> 6679 6680 * libwget/ssl_gnutls.c (_verify_certificate_callback): Fix grammar of error msg 6681 6682 Reported-by: Nicholas Sielicki 6683 66842018-06-10 DalmeGNU <dalmemail@gmail.com> 6685 6686 * libwget/bar.c (wget_bar_slot_begin): Reset speed ring buffer 6687 6688 This fixes wrong (huge) values for small files on the progress bar. 6689 66902018-05-27 Neil Locketz <neil@nlocketz.com> 6691 6692 Document new arguments 6693 6694 * docs/wget2.md: Document the change in behavior, and what each option 6695 means. 6696 66972018-05-31 Neil Locketz <neil@nlocketz.com> 6698 6699 Add verify-sig missing signature file behavior test. 6700 6701 * tests/Makefile.am: Add new test. 6702 * tests/test-gpg-verify-no-fail.c: Test behavior of the new missing 6703 signature stuff. 6704 67052018-05-27 Neil Locketz <neil@nlocketz.com> 6706 6707 Add arguments fail / nofail to --verify-sig 6708 6709 * src/options.c: Change flag to parse out the signature options. 6710 * src/wget_options.h: Change verify_sig to allow more information 6711 (selected option rather then a boolean value). 6712 * src/wget.c: Check for missing signature behavior before failing for 6713 missing signature. 6714 * src/job.c: Added additional memory cleanup 6715 * src/wget_job.h: Shuffled some fields to make new behavior cleaner and 6716 more convinient. Also added list for remaining signatures that need to 6717 be attempted. 6718 6719 Allows fine-tuning Wget2's behavior when signature files are not found. 6720 67212018-06-07 Tim Rühsen <tim.ruehsen@gmx.de> 6722 6723 * bootstrap.conf: Enable syncing with translationproject.org 6724 67252018-06-04 Tim Rühsen <tim.ruehsen@gmx.de> 6726 6727 Do not localize debug_printf format strings 6728 6729 * libwget/cookie.c: Likewise 6730 * libwget/css_url.c: Likewise 6731 * libwget/hsts.c: Likewise 6732 * libwget/ocsp.c: Likewise 6733 * libwget/ssl_gnutls.c: Likewise 6734 * libwget/tls_session.c: Likewise 6735 * unit-tests/test.c: Likewise 6736 6737 Reported-by: Rafael Fontenelle 6738 67392018-06-02 Yuri Chornoivan <yurchor@ukr.net> 6740 6741 Fix minor typos 6742 6743 * docs/wget2.md: sperated -> separated 6744 * src/bar.c (bar_init): progess -> progress 6745 6746 Copyright-paperwork-exempt: Yes 6747 67482018-05-13 Tim Rühsen <tim.ruehsen@gmx.de> 6749 6750 * contrib/mingw: Add libpsl build + cleanup 6751 67522018-05-07 Tim Rühsen <tim.ruehsen@gmx.de> 6753 6754 Skip -np/--no-parent for CSS URLs 6755 6756 * src/wget.c: Use URL_FLG_REQUISITE for all CSS URLs 6757 * tests/test--page-requisites.c: Add tests for @import and url() in CSS 6758 67592018-05-04 Tim Rühsen <tim.ruehsen@gmx.de> 6760 6761 Add URL_FLG_REQUISITE to add_url() 6762 6763 * src/wget.c: Add flag URL_FLG_REQUISITE, call add_url() with 6764 URL_FLG_REQUISITE in case of page requisites. 6765 67662018-05-03 Tsukasa OI <floss_wget@irq.a4lg.com> 6767 6768 Add test case to fail -p with -np 6769 6770 * tests/test--page-requisites.c: Add test case to fail 6771 67722018-05-07 Tim Rühsen <tim.ruehsen@gmx.de> 6773 6774 * cfg.mk: Escape dots in path components 6775 67762018-05-07 Tim Rühsen <tim.ruehsen@gmx.de> 6777 6778 * po/POTFILES.in: Remove libwget/html_url.c 6779 67802018-05-06 sameeran joshi <joshisameeran17@gmail.com> 6781 6782 * libwget/html_url.c(_urls_to_absolute): Remove unused function 6783 67842018-05-04 Darshit Shah <darnir@gnu.org> 6785 6786 Update AUTHORS 6787 6788 [skip-ci] 6789 67902018-05-04 Tsukasa OI <floss_wget@irq.a4lg.com> 6791 6792 * contrib/commit-check: Detect Copyright-paperwork-exempt correctly 6793 67942018-05-03 Tim Rühsen <tim.ruehsen@gmx.de> 6795 6796 Remove unconditionally set warning flags for GCC 6797 6798 * configure.ac: Add -W(no-)unused-macros and -Wno-missing-field-initializers 6799 to gcc/clang compiler flags 6800 * libwget/Makefile.am: Remove -Wno-unused-macros 6801 * tests/Makefile.am: Remove -Wno-missing-field-initializers 6802 * unit-tests/Makefile.am: Likewise 6803 6804 Reported-by: Dagobert Michelsen 6805 68062018-05-03 Tim Rühsen <tim.ruehsen@gmx.de> 6807 6808 * tests/Makefile.am: Add $(EXEEXT) where missing 6809 68102018-05-03 Tsukasa OI <floss_wget@irq.a4lg.com> 6811 6812 Fix null-pointer dereference on -r -np 6813 6814 * src/wget.c (add_url): Fix null-pointer dereference on -r -np 6815 * tests/test-np.c: Modify test case to detect this condition 6816 6817 Copyright-paperwork-exempt: Yes 6818 68192018-05-01 Tim Rühsen <tim.ruehsen@gmx.de> 6820 6821 * README.md: Small cleanup 6822 68232018-05-01 Darshit Shah <darnir@gnu.org> 6824 6825 * libwget/bar.c(_bar_set_progress): Fix off by one error 6826 68272018-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 6828 6829 Prepare Release 1.99.1 (alpha) 6830 68312018-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 6832 6833 * fuzz/fuzzer.h: Disable pedantic and unused warnings 6834 68352018-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 6836 6837 Update copyright to 2018 6838 68392018-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 6840 6841 * contrib/update_copyright: Update to 2018 6842 68432018-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 6844 6845 Add scripts for testing on gcc and OpenCSW buildfarm 6846 6847 * ci-test: New script with generic code 6848 * ci-test.d/gcc110.fsffrance.org.sh: Script with special code for gcc110 (CentOS 7.4) 6849 * ci-test.d/gcc70.fsffrance.org.sh: Script with special code for gcc70 (NetBSD 5.1) 6850 * ci-test.d/login.opencsw.org.sh: Script with special code for OpenCSW (Solaris) 6851 68522018-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 6853 6854 * Makefile.am: Don't do syntax-check + check_options in tarball build 6855 68562018-04-13 Tim Rühsen <tim.ruehsen@gmx.de> 6857 6858 * tests/valgrind-suppressions: Ignore pthread leak from gai_misc.c 6859 68602018-04-13 Tim Rühsen <tim.ruehsen@gmx.de> 6861 6862 Add G_GNUC_WGET_UNUSED to fuzz code 6863 6864 * fuzz/libwget_xml_parse_buffer_fuzzer.c: Likewise 6865 * fuzz/main.c: Likewise 6866 68672018-04-13 Tim Rühsen <tim.ruehsen@gmx.de> 6868 6869 * fuzz/Makefile.am: Remove hard-coded compiler flags 6870 68712018-04-13 Tim Rühsen <tim.ruehsen@gmx.de> 6872 6873 Remove _GL_INLINE from functions 6874 6875 * libwget/hashmap.c: Likewise 6876 * libwget/iri.c: Likewise 6877 * libwget/ssl_gnutls.c: Likewise 6878 * libwget/utils.c: Likewise 6879 6880 Gcc 4.1.3 on NetBSD 5.1 errors when duplicate _GL_INLINE is given as a 6881 function attribute. 6882 From the gnulib manual: "C code ordinarily should not use inline. Typically it 6883 is better to let the compiler figure out whether to inline, as compilers are 6884 pretty good about optimization nowadays. In this sense, inline is like 6885 register, another keyword that is typically no longer needed" 6886 68872018-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 6888 6889 Exclude fuzz corpora from tarball 6890 6891 * fuzz/Makefile.am: Do not include corpora in tarball 6892 * fuzz/main.c: SKIP if corpora directory isn't found (make check) 6893 6894 The fuzz corpora are thousands of files, not needed for a standard build 6895 from a distribution tarball. The reproducers of former issues are being 6896 included for regression testing. 6897 68982018-04-23 Tim Rühsen <tim.ruehsen@gmx.de> 6899 6900 Add --retry-connrefused 6901 6902 * src/options.c: Add --retry-connrefused 6903 * src/wget.c (establish_connection): Skip final failure 6904 * src/wget_options.h (struct config): Add retry_connrefused 6905 69062018-04-27 Darshit Shah <darnir@gnu.org> 6907 6908 * tests/valgrind-suppressions: Update suppressions for Glibc 2.27 6909 69102018-04-17 Tim Rühsen <tim.ruehsen@gmx.de> 6911 6912 * contrib/mail.sh: Assume bash, print email to file if msmtp not found 6913 69142018-04-17 Darshit Shah <darnir@gnu.org> 6915 6916 * contrib/mail.sh: File got deleted accidentally in an earlier commit 6917 69182018-04-16 Darshit Shah <darnir@gnu.org> 6919 6920 Update the AUTHORS list 6921 69222018-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 6923 6924 * .lgtm.yml: Call ./bootstrap 6925 69262018-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 6927 6928 Dont't recognize %3A as colon in relative URLs 6929 6930 * include/wget/wget.h: New function wget_iri_unescape_url_inline() 6931 * libwget/iri.c: Likewise 6932 * src/wget.c (_normalize_uri): Use wget_iri_unescape_url_inline() 6933 * examples/check_url_types.c: Likewise 6934 * tests/test-base.c: Add test case 6935 * unit-tests/test.c (test_iri_relative_to_absolute): Add test case 6936 6937 Relative URLS like in href='foo%3A/' have been detected and parsed 6938 as absolute URL with (unknown) scheme 'foo:'. 6939 This commit fixes it. 6940 69412018-04-11 Bas van Schaik <github@s.traiectum.net> 6942 6943 * .lgtm.yml: Remove unnneeded package installation and ./bootstrap 6944 6945 - no need to install gnutls-dev; it's installed automatically during the 6946 build/analysis process 6947 - however, the lgtm.yml I used for setting wget2 up on lgtm.com does 6948 install two other packages that are not auto-detected: lzip and 6949 libgpgme11-dev 6950 - no need to run ./bootstrap either; LGTM will do that automatically :) 6951 69522018-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 6953 6954 * .lgtm.yml: New file for static analysis on lgtm.com [ci skip] 6955 69562018-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 6957 6958 * libwget/xml.c: Fix copyright date (test lgtm.com automatism) [ci skip] 6959 69602018-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 6961 6962 * include/wget/wget.h: Fix header guard (found by lgtm.com) 6963 69642018-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 6965 6966 * libwget/ssl_gnutls.c (_do_handshake): Fix error value for older GnuTLS on OSX 6967 69682018-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 6969 6970 * libwget/Makefile: Make css_tokenizer.c generation more portable 6971 69722018-03-15 Jiading Guo <zingdle@gmail.com> 6973 6974 Add --[no-]compression=TYPE 6975 6976 * src/options.c: Add compression option support 6977 * src/wget.c: Generate 'Accept-Encoding' headers accordingly 6978 * src/wget_options.h: Add compression variables 6979 * libwget/decompressor.c: Add xz and "none" types 6980 * include/wget/wget.h: Add xz types 6981 * docs/wget2.md: Add compression option document 6982 * tests/Makefile.am: Add compression test 6983 * tests/libtest.c: Add request header test support 6984 * tests/libtest.h: Add request header test support 6985 * tests/test-compression.c: Add compression test 6986 69872018-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 6988 6989 * libwget/ssl_gnutls.c (_do_handshake): Fix error value for older GnuTLS 6990 69912018-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 6992 6993 * libwget/net.c (wget_tcp_connect): Use WGET_E_* return code 6994 69952018-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 6996 6997 Add Jiading Guo to AUTHORS [ci skip] 6998 69992018-04-10 Tim Rühsen <tim.ruehsen@gmx.de> 7000 7001 * src/wget_job.h: Fix typo in comment [ci skip] 7002 70032018-04-10 Tim Rühsen <tim.ruehsen@gmx.de> 7004 7005 * src/options.c (parse_prefer_family): Fix pointer type 7006 7007 Fixes #373 7008 70092018-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 7010 7011 * configure.ac: Enable hashing fallback to nettle 7012 70132018-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 7014 7015 Fix scan-build for gnulib hash code 7016 7017 * .gitlab-ci.yml: Obsolete configure cache 7018 * include/wget/wget.h: Add WGET_DIGTYPE_MAX 7019 * libwget/hashfile.c: Fix code 7020 70212018-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 7022 7023 * libwget/hashfile.c: Fix warnings 7024 70252018-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 7026 7027 * configure.ac: Fix detection of gnutls/crypto.h 7028 7029 Some versions of GnuTLS do not provide a crypto.h, so we have 7030 to check for it's presence. 7031 This check fails with some versions of GnuTLS since crypto.h 7032 lacks the #include <gnutls/gnutls.h>. 7033 70342018-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 7035 7036 * contrib/mingw: Add fix for lib/pthread_sigmask.c 7037 70382018-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 7039 7040 * configure.ac: Remove gcc's -Wno-incompatible-pointer-types when cross-compiling 7041 70422018-04-03 Tim Rühsen <tim.ruehsen@gmx.de> 7043 7044 * src/options.c: Fix misalignment, found by oss-fuzz 7045 70462018-04-03 Tim Rühsen <tim.ruehsen@gmx.de> 7047 7048 * fuzz/*.in: Update oss-fuzz corpora 7049 70502018-04-02 Darshit Shah <darnir@gnu.org> 7051 7052 * .gitlab-ci.yml: Add new Arch CI which is run only on schedules 7053 7054 [ci skip] 7055 70562018-04-03 Tim Rühsen <tim.ruehsen@gmx.de> 7057 7058 Check for gnutls/crypto.h 7059 7060 * configure.ac: Check for gnutls/crypto.h 7061 * libwget/hashfile.c: Need HAVE_GNUTLS_CRYPTO_H to include gnutls/crypto.h 7062 7063 Found this problem on the OpenCSW Solaris build farm 7064 70652018-04-03 Tim Rühsen <tim.ruehsen@gmx.de> 7066 7067 * libwget/cookie.c: Don't define _U, already defined on Solaris 7068 70692018-04-03 Tim Rühsen <tim.ruehsen@gmx.de> 7070 7071 * README.md: Remove gperf from dependency list 7072 70732018-04-02 Darshit Shah <darnir@gnu.org> 7074 7075 * configure.ac: Add comment explaining IDNA support 7076 70772018-04-02 Darshit Shah <darnir@gnu.org> 7078 7079 * bootstrap.conf: Update dependencies 7080 70812018-04-01 Darshit Shah <darnir@gnu.org> 7082 7083 * .gitlab-ci.yml: Switch commitcheck script to basic alpine container 7084 70852018-04-01 Darshit Shah <darnir@gnu.org> 7086 7087 * .gitlab-ci.yml: Remove flawfinder and change valgrind to Debian 7088 70892018-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 7090 7091 * .gitlab-ci.yml: Do not build lzip tarball 7092 70932018-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 7094 7095 * .gitlab-ci.yml: Copy gnulib subdir instead using git [skip ci] 7096 70972018-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 7098 7099 Use wget_snprintf() in src/*.c 7100 7101 * src/options.c: Use wget_snprintf() instead of snprintf()" 7102 * src/wget.c: Likewise 7103 71042018-03-28 Tim Rühsen <tim.ruehsen@gmx.de> 7105 7106 Use libwget's print-like functions instead the stdio ones 7107 7108 * include/wget/wget.h: Amend return type of wget_bar_print(), wget_bar_vprintf() 7109 * libwget/Makefile.am: Replace fprintf in generated css_tokenizer.c 7110 * libwget/bar.c: Replace *printf by wget_*printf, 7111 (_bar_set_progress): Generate progress bar with memcpy/memset 7112 * libwget/hpkp.c: Replace *printf by wget_*printf 7113 * libwget/cookie.c: Likewise 7114 * libwget/hpkp.c: Likewise 7115 * libwget/hsts.c: Likewise 7116 * libwget/http.c: Likewise 7117 * libwget/http_parse.c: Likewise 7118 * libwget/io.c: Likewise 7119 * libwget/net.c: Likewise 7120 * libwget/ocsp.c: Likewise 7121 * libwget/pipe.c: Likewise 7122 * libwget/ssl_gnutls.c: Likewise 7123 * libwget/utils.c: Likewise 7124 * libwget/xml.c: Likewise 7125 7126 The intention is to not pull in (large) code parts from libc when 7127 linking statically, the performance is better and we have no 7128 portability issues on non-POSIX platforms (e.g. Windows). 7129 71302018-03-28 Tim Rühsen <tim.ruehsen@gmx.de> 7131 7132 Add more printf-like functions 7133 7134 * include/wget/wget.h: Add wget_vfprintf(), wget_fprintf(), 7135 wget_vsnprintf(), wget_snprintf() 7136 * libwget/printf.c: Likewise 7137 71382018-03-28 Tim Rühsen <tim.ruehsen@gmx.de> 7139 7140 * libwget/strscpy.c: Fix return value of strscpy() 7141 71422018-03-28 Tim Rühsen <tim.ruehsen@gmx.de> 7143 7144 * unit-tests/test.c: Add tests for strscpy() 7145 71462018-03-28 Tim Rühsen <tim.ruehsen@gmx.de> 7147 7148 * docs/Makefile.am: Build man pages for hashmap/stringmap API 7149 71502018-03-25 Tim Rühsen <tim.ruehsen@gmx.de> 7151 7152 Document hashmap and stringmap 7153 7154 * include/wget/wget.h: Cleanups 7155 * libwget/hashmap.c: Documentation + cleanups 7156 * libwget/stringmap.c: Documentation + cleanups 7157 * libwget/vector.c: Fix doxygen issue 7158 71592018-03-25 Tim Rühsen <tim.ruehsen@gmx.de> 7160 7161 * libwget/vector.c: Small cleanups 7162 71632018-03-25 Tim Rühsen <tim.ruehsen@gmx.de> 7164 7165 Remove param 'off' from wget_vector_create() 7166 7167 * libwget/vector.c (wget_vector_create): Remove param 'off', 7168 (wget_vector_set_growth_policy): Add new function 7169 * include/wget/wget.h: Add prototype for wget_vector_set_growth_policy() 7170 * */*.c: Remove param 'off' from calls to wget_vector_create() 7171 71722018-03-24 Tim Rühsen <tim.ruehsen@gmx.de> 7173 7174 Add docs for vector routines 7175 7176 * docs/Makefile.am: Add man page 7177 * include/wget/wget.h: Amend some vector functions 7178 * libwget/vector.c: Add checks for parameters, 7179 add documentation 7180 71812018-03-23 Tim Rühsen <tim.ruehsen@gmx.de> 7182 7183 * .gitlab-ci.yml: Remove stages, fix configure cache 7184 71852018-03-23 Tim Rühsen <tim.ruehsen@gmx.de> 7186 7187 * .gitlab-ci.yml: Add testing on Alpine (musl/busybox) 7188 71892018-03-23 Darshit Shah <darnir@gnu.org> 7190 7191 Merge the Full/Debian and VPATH Runners into one 7192 71932018-03-23 Darshit Shah <darnir@gnu.org> 7194 7195 More efficient use of CI Pipelines 7196 7197 * .gitlab-ci.yml: Split CI runners into 3 stages, build, test and 7198 deploy. Stages are executed in that order. So any breakage will save the 7199 later stages from executing. 7200 72012018-03-23 Tim Rühsen <tim.ruehsen@gmx.de> 7202 7203 * cfg.mk: Exclude tests/gpg and tests/certs from any syntax checks 7204 72052018-03-23 Darshit Shah <darnir@gnu.org> 7206 7207 Fix GPGME detection in configure 7208 72092018-03-23 Darshit Shah <darnir@gnu.org> 7210 7211 Update gnulib submodule 7212 7213 * gnulib: Update submodule 7214 * bootstrap: Update script from gnulib submodule 7215 72162018-03-23 Tim Rühsen <tim.ruehsen@gmx.de> 7217 7218 * tests/test-iri-subdir.c: Add runtime test for working SHIFT_JIS->UTF-8 transcoding 7219 72202018-02-24 Darshit Shah <darnir@gnu.org> 7221 7222 New script to ensure that patches without copyright assignment aren't merged 7223 7224 * AUTHORS: Update file with all known contributors to date. Update 7225 FSF Copyright Assignment status in the first column 7226 * contrib/commit-check: New script. This is to be run on each merge 7227 request to master to ensure that the copyright assignment process 7228 has been completed. 7229 * .gitlab-ci.yml: Add a new CommitCheck CI which ensures that the 7230 author has completed their Copyright Assignments. This runner is 7231 allowed to fail, so it won't affect the CI runners for all forks. 7232 72332018-02-26 Darshit Shah <darnir@gnu.org> 7234 7235 Add new scripts to send Copyright assignment mails 7236 7237 * contrib/assignment_template.txt: Template header text for email 7238 * contrib/mail.sh: Script that composes and sends an email to a new 7239 contributor for Copyright Assignment 7240 * cfg.mk: Add exceptions for assignment_template.txt to syntax-check 7241 72422018-02-25 Darshit Shah <darnir@gnu.org> 7243 7244 Allow a list of possible signature-extensions 7245 7246 * stc/options.c: Modify old option signature-extension to now be a 7247 stringlist. Also rename it to signature-extensions 7248 * src/wget_options.h: Move sig_ext from char* to wget_vector_t 7249 * src/wget.c: Iterate over all signature extensions and request a 7250 signature file 7251 * tests/test-gpg-styles.c: Add new test for signature extensions 7252 * docs/wget2.md: Update documentation for signature-extensions 7253 72542018-03-15 Jiading Guo <zingdle@gmail.com> 7255 7256 Fix lzma decompressor init 7257 7258 * libwget/decompressor.c: Use lzma_auto_decoder() instead to auto detect lzma/xz format. 7259 72602018-03-18 Tim Rühsen <tim.ruehsen@gmx.de> 7261 7262 * bootstrap.conf: Add pkg-config as dependency 7263 72642018-03-15 Tim Rühsen <tim.ruehsen@gmx.de> 7265 7266 Fix decompress_open argument type 7267 7268 * include/wget/wget.h: Fix 'encoding' to wget_content_encoding_type_t, 7269 make wget_content_encoding_by_name() PURE, 7270 make wget_content_encoding_to_name() PURE. 7271 * libwget/decompressor.c: Fix struct _wget_decompressor_st encoding to 7272 wget_content_encoding_type_t, 7273 fix wget_content_encoding_to_name() for 'identity'. 7274 72752018-03-15 Tim Rühsen <tim.ruehsen@gmx.de> 7276 7277 Add type<->name functions for content-encoding types 7278 7279 * include/wget/wget.h: AdD typedef wget_content_encoding_type_t, 7280 add wget_content_encoding_unknown, 7281 add prototypes for wget_content_encoding_by_name() and 7282 wget_content_encoding_to_name(). 7283 * libwget/decompressor.c: Implement wget_content_encoding_by_name() and 7284 wget_content_encoding_to_name(). 7285 72862018-03-13 Tim Rühsen <tim.ruehsen@gmx.de> 7287 7288 * README.md: Increase GnuTLS version from 2.10.0 to 3.3, 3.5 or 3.6 7289 72902018-02-27 Jay <jbhv12@gmail.com> 7291 7292 error handling in XML parsing 7293 7294 * include/wget/wget.h: removed nonnull attribute and defined error code 7295 * libwget/metalink.c: wget_metalink_parse returns null if XML is malformed 7296 * libwget/xml.c: added coditions to detect malformed XML in parseXML 7297 * po/POTFILES.in: localization support for metalink.c 7298 72992018-03-08 Tim Rühsen <tim.ruehsen@gmx.de> 7300 7301 * tests/test-gzip.c: Fix unused define w/o WITH_ZLIB 7302 73032018-03-08 Tim Rühsen <tim.ruehsen@gmx.de> 7304 7305 * tests/libtest.c (wget_test_start_server): Fix unused var w/o WITH_GNUTLS 7306 73072018-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 7308 7309 * .gitlab-ci.yml: Add -Werror to appropriate runners 7310 73112018-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 7312 7313 * configure.ac: Add -Wno-error to gnulib compiler flags 7314 73152018-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 7316 7317 * src/stats_site.c (print_human): Fix unused-but-used warning 7318 73192018-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 7320 7321 * libwget/http.c (wget_tcp_set_stats_site): Fix unused argument 7322 73232018-03-04 Tim Rühsen <tim.ruehsen@gmx.de> 7324 7325 * contrib/mingw: Add dlfcn-win32 build 7326 73272018-03-02 Tim Rühsen <tim.ruehsen@gmx.de> 7328 7329 * tests/libtest.c (_http_server_start): Check MHD_OPTION_STRICT_FOR_CLIENT 7330 73312018-03-02 Tim Rühsen <tim.ruehsen@gmx.de> 7332 7333 * tests/libtest.c (_http_server_start): Enable strict MHD client mode 7334 7335 Will not allow spaces in the GET target URI from the next release. 7336 73372018-02-26 Tim Rühsen <tim.ruehsen@gmx.de> 7338 7339 * src/http_parse.c (wget_http_parse_response_header): Remove unneeded comment 7340 73412018-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 7342 7343 * tests/test-gzip.c: Add tests for broken server gzip Content-Encoding 7344 73452018-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 7346 7347 * src/http.c (wget_http_get_response_cb): Add gzip workarounds 7348 73492018-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 7350 7351 Support binary content in test suite 7352 7353 * tests/libtest.h (wget_test_file_t): Add content_length member 7354 * tests/libtest.c (_answer_to_connection): Support content_length member 7355 73562018-02-23 Tim Rühsen <tim.ruehsen@gmx.de> 7357 7358 Fix ignorant server's Content-Encoding 7359 7360 * libwget/decompressor.c: Fix segfault on unexpected Content-Encoding 7361 * tests/Makefile.am: Add test-gzip.c 7362 * tests/test-gzip.c: New test 7363 7364 Reported-by: Jiading Guo 7365 7366 73672018-03-02 Tim Rühsen <tim.ruehsen@gmx.de> 7368 7369 Fix header guards 7370 7371 * include/wget/wget.h: Likewise 7372 * include/wget/wgetver.h.in: Likewise 7373 * libwget/*.h: Likewise 7374 * src/*.h: Likewise 7375 * tests/libtest.h: Likewise 7376 7377 Header guards should not start with _. 7378 Private headers now reflect the path within the project. 7379 The two public headers reflect the releative install directory. 7380 73812018-03-02 Tim Rühsen <tim.ruehsen@gmx.de> 7382 7383 Fix doxygen syntax 7384 7385 * src/wget_gpgme.h: Likewise 7386 * libwget/thread.c: Likewise 7387 * src/wget_gpgme.h: Likewise 7388 73892018-03-01 Tim Rühsen <tim.ruehsen@gmx.de> 7390 7391 * libwget/iri.c (wget_iri_get_escaped_resource): Escape space in query 7392 73932018-03-01 Tim Rühsen <tim.ruehsen@gmx.de> 7394 7395 * libwget/http.c: Add links about request line format 7396 73972018-02-25 Tim Rühsen <tim.ruehsen@gmx.de> 7398 7399 Fix oss-fuzz build 7400 7401 * fuzz/Makefile.am: Add ../src/stats_server.o ../src/stats_site.o to oss-fuzz rule 7402 * fuzz/libwget_http_client_fuzzer.c: Fix pedantic C++ errors 7403 74042018-02-23 DalmeGNU <dalmemail@gmail.com> 7405 7406 * src/wget.c: Avoid printing two times the number of errors in the same bar_printf() 74072018-02-21 Vijo Cherian <codervijo@gmail.com> 7408 7409 * src/options.c: Improve --help output 7410 74112018-02-23 Tim Rühsen <tim.ruehsen@gmx.de> 7412 7413 * src/wget.c: Check config.metalink before executing metalink code 7414 74152018-02-22 Tim Rühsen <tim.ruehsen@gmx.de> 7416 7417 * src/ssl_gnutls.c (_do_handshake): Check for GNUTLS_E_PREMATURE_TERMINATION 7418 74192018-02-22 Tim Rühsen <tim.ruehsen@gmx.de> 7420 7421 Fix stats and add docs for stats 7422 7423 * docs/wget2.md: Extend docs for --stats-* 7424 * include/wget/wget.h: Rename WGET_STATS_TLS_TCP_PROTO to WGET_STATS_TLS_HTTP_PROTO, 7425 add WGET_STATS_OCSP_STAPLING. 7426 * libwget/ssl_gnutls.c: add stapling stats member, cleanup variables 7427 * po/POTFILES.in: Remove rsrc/stats_server.c 7428 * src/stats_dns.c: Cleanup 7429 * src/stats_ocsp.c: Add stapling to stats 7430 * src/stats_server.c: Cleanup 7431 * src/stats_site.c: Rename STATS_SCHEME_* to STATS_METHOD_* 7432 * src/stats_tls.c: Cleanup 7433 * src/wget_stats.h: Remove memory alloc from NULL_TO_DASH() 7434 74352018-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 7436 7437 * src/stats_site: Change PGP verification status in CSV output 7438 74392018-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 7440 7441 * src/stats_site.c: Avoid double entries when signature checking is on 7442 74432018-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 7444 7445 * src/gpgme.c: Add and use _print_gpg_error() 7446 74472018-02-20 Darshit Shah <darnir@gnu.org> 7448 7449 * src/wget.c(_get_body): Remove leftover debugging statement 7450 74512018-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 7452 7453 * src/gpgme.c: Some fixes and cleanups 7454 74552018-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 7456 7457 * src/gpgme.c (wget_verify_job): Always initialize info structure 7458 74592018-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 7460 7461 * contrib/csv2tree: New file 7462 74632018-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 7464 7465 * contrib/csv2json: New file 7466 74672018-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 7468 7469 Remove JSON format for stats options 7470 7471 * include/wget/wget.h: Remove WGET_STATS_FORMAT_JSON 7472 * src/stats.c: Remove JSON code 7473 * src/stats_dns.c: Likewise 7474 * src/stats_ocsp.c: Likewise 7475 * src/stats_server.c: Likewise 7476 * src/stats_site.c: Likewise 7477 * src/stats_tls.c: Likewise 7478 * src/wget_stats.h: Likewise 7479 * tests/stats-test-util.h: Remove JSON tests 7480 We will provide an extra tool to convert CSV into JSON. 7481 74822018-02-19 Tim Rühsen <tim.ruehsen@gmx.de> 7483 7484 Remove tree statistics 7485 7486 * docs/wget2.md: Remove 'tree' format from --stats-site 7487 * po/POTFILES.in: Remove src/stats_site.c 7488 * src/host.c: Remove host_docs, tree_docs 7489 * src/job.c: Add a unique ID for each job 7490 * src/stats.c: Remove TREE stuff 7491 * src/stats_dns.c: Likewise 7492 * src/stats_ocsp.c: Likewise 7493 * src/stats_server.c: Likewise 7494 * src/stats_tls.c: Likewise 7495 * src/stats_site.c: Likewise, 7496 add site_stats_t, cleanup CSV and HUMAN output 7497 * src/wget.c: Set job ID and ParentID, remove TREE stuff 7498 * src/wget_host.h: Remove TREE stuff 7499 * src/wget_job.h: add id and parent_id to struct JOB 7500 * src/wget_stats.h: Add json_stats 7501 * tests/stats-test-util.h: Remove TREE test 7502 We decided to provide an external tool to generate TREE and other 7503 formatted output from Wget2's CSV output. 7504 75052018-02-17 Tim Rühsen <tim.ruehsen@gmx.de> 7506 7507 * stats_*.c: Add missing entry for WGET_STATS_FORMAT_TREE 7508 75092018-02-17 Tim Rühsen <tim.ruehsen@gmx.de> 7510 7511 Move external stats_opts_t from stat.c to wget_stats.h 7512 7513 * src/stats.c: Remove external stats_opts_t 7514 * src/wget_stats.h: Add external stats_opts_t 7515 75162018-02-17 Tim Rühsen <tim.ruehsen@gmx.de> 7517 7518 * src/stats.c (stats_init): Fix memory leak 7519 75202018-02-17 Tim Rühsen <tim.ruehsen@gmx.de> 7521 7522 * src/stats.c (stats_exit): Only deinit after init 7523 75242018-02-17 Tim Rühsen <tim.ruehsen@gmx.de> 7525 7526 Move Site stats code to own source file 7527 75282018-02-17 Tim Rühsen <tim.ruehsen@gmx.de> 7529 7530 Move Server stats code to own source file 7531 75322018-02-17 Tim Rühsen <tim.ruehsen@gmx.de> 7533 7534 Move OCSP stats code to own source file 7535 75362018-02-16 Tim Rühsen <tim.ruehsen@gmx.de> 7537 7538 Move TLS stats code to own source file 7539 75402018-02-16 Tim Rühsen <tim.ruehsen@gmx.de> 7541 7542 Clean up DNS stats code 7543 7544 * src/stats_dns.c: Renames and cleanups 7545 * src/wget_stats.h: Likewise 7546 75472018-02-16 Tim Rühsen <tim.ruehsen@gmx.de> 7548 7549 Move _shell_expand() into own source file 7550 7551 * src/Makefile.am: Add utils.c and wget_utils.h 7552 * src/options.c: Remove _shell_expand() 7553 * src/stats.c: Likewise 7554 * src/utils.c: New file with shell_expand() 7555 * src/wget_utils.h: Header file for shell_expand() 7556 * fuzz/Makefile.am: Add utils.o 7557 * unit-tests/Makefile.am: Likewise 7558 75592018-02-16 Tim Rühsen <tim.ruehsen@gmx.de> 7560 7561 Move dns stats code into stats_dns.c 7562 75632018-02-16 Tim Rühsen <tim.ruehsen@gmx.de> 7564 7565 Fix stats tests 7566 75672018-02-16 Tim Rühsen <tim.ruehsen@gmx.de> 7568 7569 Refactorings on stats option parsing and printing 7570 75712018-02-14 Darshit Shah <darnir@gnu.org> 7572 7573 Set callback for stats_site and allow freeing non-init mutex 7574 75752018-02-13 Tim Rühsen <tim.ruehsen@gmx.de> 7576 7577 Refactoring of stats callback functions 7578 75792018-02-13 Darshit Shah <darnir@gnu.org> 7580 7581 Handle tree stats case 7582 75832018-02-13 Darshit Shah <darnir@gnu.org> 7584 7585 Some more cleanups 7586 75872018-02-13 Darshit Shah <darnir@gnu.org> 7588 7589 Replace dns data and mutex 7590 75912018-02-13 Darshit Shah <darnir@gnu.org> 7592 7593 Improve stats printing 7594 75952018-02-21 Darshit Shah <darnir@gnu.org> 7596 7597 * src/options.c(print_version): Add gpgme info to wget2 --version 7598 75992018-02-20 Darshit Shah <darnir@gnu.org> 7600 7601 Handle case where wget_bar_init() fails 7602 76032018-02-20 Jay <jbhv12@gmail.com> 7604 7605 added error in progress report 7606 76072018-02-17 Tim Rühsen <tim.ruehsen@gmx.de> 7608 7609 * src/wget.c (process_response): Collect site stats only when requested 7610 76112018-02-17 zingdle <zingdle@gmail.com> 7612 7613 * src/stats.c: Fix handling of output location for stats code 7614 76152018-02-15 Darshit Shah <darnir@gnu.org> 7616 7617 * src/stats.c: Initialize all stats modules, always 7618 7619 This is a terrible hack. But it's the smallest solution I can come up 7620 with, with the existing stats code to fix the problem that none of the 7621 stats are being reported. 7622 7623 The problem is that stats_init() must be called before the command line 7624 is parsed. But stats_init() depends on the command line parsing code to 7625 set the status flags for which stats to report. Hence, it always sees 7626 them all as negative. 7627 7628 This commit will initialize all the stats modules, always. So, the stats 7629 are always collected. Only, we decide at the very end, what, if 7630 anything, to display. As is obvious, this is terrible. It's a waste of 7631 resources to compute all these stats that are simply not required. But, 7632 till the stats code is re-written cleanly, this is what we have to deal 7633 with 7634 76352018-02-15 Darshit Shah <darnir@gnu.org> 7636 7637 * .gitignore: Update gitignore file 7638 76392018-02-15 Darshit Shah <darnir@gnu.org> 7640 7641 Revert "* libwget/thread.c(pthread_mutex_destroy): Survive destroying a non-initialized mutex" 7642 7643 This reverts commit c46669387bcb4caffa792d2d6360b09685bf3449. 7644 76452018-02-15 Darshit Shah <darnir@gnu.org> 7646 7647 * bootstrap.conf: Clean up some files that gnulib-tool.py creates 7648 76492018-02-15 Tim Rühsen <tim.ruehsen@gmx.de> 7650 7651 * src/stats.c (stats_print): Fix possible memory leak 7652 76532018-02-15 Tim Rühsen <tim.ruehsen@gmx.de> 7654 7655 Revert "Delay stats_init till after comand line parsing" 7656 7657 This reverts commit 33e3322a718a330fb6b426c6a640a0f3f7746d36. 7658 76592018-02-14 Tim Rühsen <tim.ruehsen@gmx.de> 7660 7661 * gnulib: Update 7662 76632018-02-14 Tim Rühsen <tim.ruehsen@gmx.de> 7664 7665 Add gnulib module limits-h 7666 7667 * bootstrap.conf: Add limits-h 7668 * libwget/cookie.c: Include limits.h 7669 * libwget/metalink.c: Likewise 7670 76712018-02-14 Tim Rühsen <tim.ruehsen@gmx.de> 7672 7673 * po/POTFILES.in: Remove lib/openat-die.c 7674 76752018-02-14 Darshit Shah <darnir@gnu.org> 7676 7677 * libwget/thread.c(pthread_mutex_destroy): Survive destroying a non-initialized mutex 7678 76792018-02-13 Darshit Shah <darnir@gnu.org> 7680 7681 * src/stats.c: Print to stderr not stdout 7682 76832018-02-14 Tim Rühsen <tim.ruehsen@gmx.de> 7684 7685 Use realpath() instead of canonicalize_filename_mode() 7686 7687 * bootstrap.conf: Use canonicalize-lgpl module 7688 * src/gpgme.c: Likewise 7689 7690 Using realpath() reduces extra code from gnulib. 7691 76922018-02-13 Tim Rühsen <tim.ruehsen@gmx.de> 7693 7694 * configure.ac: Fix help text for --enable-fuzzing 7695 76962018-02-13 Darshit Shah <darnir@gnu.org> 7697 7698 Delay stats_init till after comand line parsing 7699 7700 The stats module depends on the command line parser to set some of its 7701 values. Without that, the module is initialized incorrectly. 7702 77032018-02-09 Darshit Shah <darnir@gnu.org> 7704 7705 src/wget.c(main): Honor --quiet for progress bar 7706 77072018-02-07 Tim Rühsen <tim.ruehsen@gmx.de> 7708 7709 Fix make dist with --disable-doc 7710 7711 * configure.ac: Unconditionally process docs/*.in files 7712 * docs/Makefile.am: Unconditionally put DoxygenLayout.xml into EXTRA_DIST 7713 77142018-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 7715 7716 * configure.ac: Fix --disable-docs for make distcheck 7717 77182018-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 7719 7720 * configure.ac: Fix --disable-doc configure option 7721 77222018-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 7723 7724 * .gitlab-ci.yml: Remove continuation slashes 7725 77262018-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 7727 7728 * .gitlab-ci.yml: Add pipeline (gcc full and gcc minial build) 7729 77302018-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 7731 7732 * cfg.mk: Exclude .gitlab-ci.yml from sc_two_space_separator_in_usage 7733 77342018-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 7735 7736 * libwget/ssl_gnutls.c: Remove obsolete dummy functions 7737 77382018-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 7739 7740 * src/gpgme.c (wget_verify_get_base_file): Fix to build w/o WITH_GPGME 7741 77422018-02-03 Tim Rühsen <tim.ruehsen@gmx.de> 7743 7744 Fix manywarnings for gcc 8 7745 7746 * configure.ac: Amend for gcc-8 7747 * m4/wget_manywarnings.m4: Amend for gcc-8 7748 77492018-02-02 Tim Rühsen <tim.ruehsen@gmx.de> 7750 7751 * libwget/http.c (wget_http_get_response_cb): Fix undefined behavior 7752 77532018-02-02 Tim Rühsen <tim.ruehsen@gmx.de> 7754 7755 * libwget/net.c: Use getaddrinfo() for fuzzing 7756 77572017-11-03 Tim Rühsen <tim.ruehsen@gmx.de> 7758 7759 Add new fuzzer libwget_http_client_fuzzer 7760 7761 * fuzz/Makefile.am: Add libwget_http_client_fuzzer 7762 * fuzz/libwget_http_client_fuzzer.c: New file 7763 * fuzz/libwget_http_client_fuzzer.in: Initial corpora 7764 * fuzz/run-clang.sh: Small cleanup 7765 77662018-01-30 Tim Rühsen <tim.ruehsen@gmx.de> 7767 7768 More tests for --https-enfore 7769 7770 * src/options.c: Add debug lines, fix exit_status to exit_status_t 7771 * src/wget_options.h: Amend prototype of get_exit_status() 7772 * tests/test--https-enforce-soft2.c: New test 7773 * tests/test--https-enforce-soft3.c: New test 7774 * tests/Makefile.am: Add new tests 7775 * src/wget.c (_fallback_to_http): Skip robotstxt jobs 7776 * tests/libtest.c: New option WGET_TEST_HTTPS_REJECT_CONNECTIONS 7777 for wget_test_start_server() 7778 77792018-01-29 Tim Rühsen <tim.ruehsen@gmx.de> 7780 7781 Add test test--https-enforce-hard3.c 7782 77832018-01-29 Tim Rühsen <tim.ruehsen@gmx.de> 7784 7785 * tests/libtest.c (wget_test): Allow {{port}} in options 7786 77872018-01-29 Tim Rühsen <tim.ruehsen@gmx.de> 7788 7789 Add options --default-http-port and --default-https-port 7790 7791 * docs/wget2.md: Add docs for the two options 7792 * include/wget/wget.h: Prototype for wget_iri_set_defaultport() 7793 * libwget/iri.c: New function wget_iri_set_defaultport() 7794 * src/options.c: New options --default-http-port and --default-https-port 7795 * src/wget_options.h (struct config): Add default_http_port and default_https_port 7796 77972018-01-29 Tim Rühsen <tim.ruehsen@gmx.de> 7798 7799 Fix exit code for TLS connect errors 7800 7801 * libwget/ssl_gnutls.c (_do_handshake): Return WGET_E_CONNECT on connect errors 7802 * src/wget.c (establish_connection): Set WG_EXIT_STATUS_NETWORK on connect errors 7803 78042018-01-23 Tim Rühsen <tim.ruehsen@gmx.de> 7805 7806 * src/options.c (parse_https_enforce): Fix wrong casts 7807 78082018-01-23 Tim Rühsen <tim.ruehsen@gmx.de> 7809 7810 Add tests for --https-enforce 7811 7812 * tests/Makefile.am: Add new tests 7813 * tests/libtest.c: Code cleanup, 7814 (_answer_to_connection): implement protocol specific urls, 7815 (wget_test_start_server): implement WGET_TEST_HTTP(S)_ONLY, 7816 (wget_test): Allow absolute URLS for WGET_TEST_REQUEST_URL(S) 7817 * tests/libtest.h (wget_test_url_t): Add https_only and http_only 7818 78192018-01-23 Tim Rühsen <tim.ruehsen@gmx.de> 7820 7821 * src/wget.c: Fix --https-enforce=soft --recursive 7822 78232018-01-19 Tim Rühsen <tim.ruehsen@gmx.de> 7824 7825 Add http:// to URIs without scheme 7826 7827 * libwget/iri.c (wget_iri_parse): Add http:// to uri without scheme 7828 * include/wget/wget.h (wget_iri_t): Add member msize 7829 7830 This commit also fixes a buffer overflow in wget_iri_set_scheme in 7831 combination with wget_iri_clone. 7832 78332018-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 7834 7835 Simplify code for --https-enforce 7836 7837 * libwget/iri.c (wget_iri_set_scheme): Cleanup 7838 * src/host.c (host_add_robotstxt_job): Add param http_fallback 7839 * src/wget_host.h: Likewise 7840 * src/job.c (job_init): Add param http_fallback 7841 * src/wget_job.h: Likewise 7842 * src/wget.c: Remove http_fallback_urls and http_fallback_urls_mutex, 7843 (add_url_to_queue): Set http_fallback for new jobs, 7844 (add_url): Likewise 7845 78462018-01-18 DalmeGNU <dalmemail@gmail.com> 7847 7848 Add new option --https-enforce 7849 7850 * docs/wget2.md: Add docs 7851 * src/options.c: Add --https-enforce and parse_https_enforce() 7852 * src/wget.c: 7853 * src/wget_options.h: Add config.https_enforce and WGET_HTTPS_ENFORCE_* 7854 78552018-01-30 Tim Rühsen <tim.ruehsen@gmx.de> 7856 7857 * libwget/net.c (_wget_tcp_resolve): Handle EAI_ALLDONE 7858 78592018-01-29 Tim Rühsen <tim.ruehsen@gmx.de> 7860 7861 * .gitlab-ci.yml: Save artifacts for the VPATH build 7862 78632018-01-29 Tim Rühsen <tim.ruehsen@gmx.de> 7864 7865 Fix testing options 7866 7867 * Makefile.am: Add wget2 executable from builddir 7868 * contrib/check_options: Use given command line arguments 7869 78702018-01-29 Tim Rühsen <tim.ruehsen@gmx.de> 7871 7872 Suppress ASAN leak detection in getaddrinfo_a() 7873 7874 * .gitlab-ci.yml: export LSAN_OPTIONS=suppression=... 7875 * tests/Makefile.am: Add suppression file to TESTS_ENVIRONMENT 7876 * tests/clang-asan-suppressions: New file with leak suppression 7877 7878 getaddrinfo_a() has leak(s) that we have to suppress when using 7879 the address sanitizer. 7880 78812018-01-27 Tim Rühsen <tim.ruehsen@gmx.de> 7882 7883 * src/wget.c (downloader_thread): Pause on error with --disable-threads 7884 78852018-01-26 Tim Rühsen <tim.ruehsen@gmx.de> 7886 7887 * tests/valgrind-suppressions: Add leaks in glibc's getaddrinfo_a() 7888 78892018-01-26 Tim Rühsen <tim.ruehsen@gmx.de> 7890 7891 * tests/libtest.c (wget_test): Add 10s timeout to wget2 command line 7892 78932018-01-26 Tim Rühsen <tim.ruehsen@gmx.de> 7894 7895 * .gitlab-ci.yml: Use persistent ccache and configure cache 7896 78972018-01-25 Tim Rühsen <tim.ruehsen@gmx.de> 7898 7899 * libwget/io.c: #undef select for _WIN32 build 7900 7901 Reported-by: Gisle Vanem 7902 79032018-01-24 Tim Rühsen <tim.ruehsen@gmx.de> 7904 7905 * fuzz/wget_options_fuzzer.dict: Update with new options 7906 79072018-01-24 Tim Rühsen <tim.ruehsen@gmx.de> 7908 7909 * fuzz/wget_options_fuzzer.in: Add new corpora 7910 79112018-01-24 Tim Rühsen <tim.ruehsen@gmx.de> 7912 7913 * src/options.c (parse_regex_type): Return error instead of exit() 7914 79152018-01-19 Tim Rühsen <tim.ruehsen@gmx.de> 7916 7917 * src/wget.c (main): Minor cleanup 7918 79192018-01-23 Tim Rühsen <tim.ruehsen@gmx.de> 7920 7921 * src/host.c (host_queue_free): Fix double free with --debug 7922 79232018-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 7924 7925 Add --http2-request-window to set max. # of parallel HTTP/2 streams 7926 7927 * docs/wget2.md: Docs for --http2-request-window 7928 * src/options.c: Add --http2-request-window 7929 * src/wget_options.h: Remove unused config.http1_request_window 7930 79312018-01-15 Tim Rühsen <tim.ruehsen@gmx.de> 7932 7933 * contrib/release: Fix and add more checks 7934 79352018-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 7936 7937 Fix option checking 7938 7939 * Makefile.am: Add contrib/check_options to EXTRA_DIST 7940 * contrib/check_options: Fix paths 7941 79422018-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 7943 7944 * src/options.c: Add SECTION_PLUGIN and move plugin options into it 7945 79462018-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 7947 7948 * contrib/check_options: Force /bin/bash to have shopt 7949 79502018-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 7951 7952 * docs/wget2.md: Remove WARC and FTP stuff 7953 79542018-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 7955 7956 * docs/wget2_md2man.sh.in: Remove set -o pipefail (incompatible) 7957 79582018-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 7959 7960 * docs/wget2.md: Add text for plugin options 7961 79622018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 7963 7964 * docs/wget2.md: Add text for --local-db, --input-encoding=encoding 7965 79662018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 7967 7968 * docs/wget2.md: Amend text for --secure-protocol 7969 79702018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 7971 7972 * docs/wget2.md: Add text for --gnutls-options 7973 79742018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 7975 7976 * docs/wget2.md: Add text for --fsync-policy, --hpkp, --hpkp-file 7977 79782018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 7979 7980 * docs/wget2.md: Add text for --dns-caching, --http2, --metalink 7981 79822018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 7983 7984 * docs/wget2.md: Add text for --cookie-suffixes 7985 79862018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 7987 7988 * src/options.c: Update link to public_suffix_list.dat 7989 79902018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 7991 7992 * contrib/check_options: Enable/fix pattern matching for case 7993 79942018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 7995 7996 * docs/wget2.md: Add text for --robots 7997 79982018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 7999 8000 * docs/wget2.md: Fix syntax of --private-key 8001 80022018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8003 8004 * src/options.c: Don't display -n compat option on --help 8005 80062018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8007 8008 * docs/wget2.md: Fix --inet*, Add text for --tcp-fastopen 8009 80102018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8011 8012 * contrib/check_options: Amend collecting options 8013 80142018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8015 8016 Remove --config-file in favor of --config 8017 8018 * docs/wget2.md: Amend docs 8019 * src/options.c: Remove option --config-file 8020 80212018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8022 8023 * docs/wget2.md: Fix option syntax 8024 80252018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8026 8027 * contrib/check_options: Adjust search patter 8028 80292018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8030 8031 * docs/wget2.md: Add text for --ocsp, --ocsp-stapling and --ocsp-file 8032 80332018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8034 8035 * docs/wget2.md: Add text for --tls-session-file and --tls-false-start 8036 80372018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8038 8039 * docs/wget2.md: Add text for --http-proxy and --https-proxy 8040 80412018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8042 8043 * docs/wget2.md: Add space before --tries and --output-document 8044 80452018-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 8046 8047 * docs/wget2.md: Add text for --check-hostname 8048 80492018-01-11 Darshit Shah <darnir@gnu.org> 8050 8051 * docs/wget2_md2man.sh.in: Add bash error flags to script 8052 80532018-01-11 Darshit Shah <darnir@gnu.org> 8054 8055 * docs/wget2.md: Add documentation for --signature-extension 8056 80572018-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 8058 8059 Check wget2 options against wget2 manual 8060 8061 * Makefile.am: Call contrib/check_options in 'make check' 8062 * contrib/check_options: New script to check options 8063 80642018-01-10 Neil Locketz <neil@nlocketz.com> 8065 8066 Delete signature file, and delete files that fail signature validation. 8067 8068 * docs/wget2.md: Added documentation for new '--verify-save-failed' 8069 * src/gpgme.c: Exposed function to determine base downloaded file from 8070 signature JOB. 8071 * src/options.c: Added '--verify-save-failed' to allow users to keep downloaded 8072 files that fail validation. 8073 * src/wget.c: Logic for unlinking signatures, and files failing validation. 8074 * src/wget_gpgme.c: Declaration for new function that determine base downloaded 8075 files. 8076 * tests/Makefile.am: Added New test code, fixed missing SOURCES. 8077 * tests/gpg-test-util.h: Conditionally expect a saved base file if exit 8078 code == 0, never expect a saved signature file. 8079 * tests/test-gpg-save-failed.c: A test that tests the '--verify-save-failed' 8080 flag. 8081 * tests/test-gpg-style.s: Remove signature file from expected output. 8082 8083 Added flag '--verify-save-failed' to disable the "fail delete" behavior. 8084 80852018-01-12 Avinash Sonawane <rootkea@gmail.com> 8086 8087 * README.md: Add libgpgme to build requirements 8088 80892016-11-25 Tim Rühsen <tim.ruehsen@gmx.de> 8090 8091 Prepare alpha release 8092 80932018-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 8094 8095 Update gnulib 8096 80972018-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 8098 8099 Fix warnings 8100 8101 * configure.ac: Add -Wno-assign-enum -Wno-unreachable-code to gnulib warnings 8102 * libwget/thread.c: Add #pragmas to silence warnings from gnulib includes 8103 * libwget/utils.c: Likewise 8104 * src/log.c: Likewise 8105 * src/wget.c: Likewise 8106 * src/bar.c: Make _terminate_thread static 8107 * tests/test-gpg-styles.c: Make urls static 8108 81092018-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 8110 8111 * configure.ac: Add -Wno-float-conversion to gnulib compilation 8112 81132018-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 8114 8115 Make strings translatable for error_printf() 8116 8117 * libwget/*.c: Likewise 8118 * src/*.c: Likewise 8119 * po/POTFILES.in: Add source files 8120 81212018-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 8122 8123 Remove data/ directory 8124 8125 * Makefile.am: Remove data/ from SUBDIRS 8126 * configure.ac: Remove data/Makefile from AC_CONFIG_FILES 8127 * tests/Makefile.am: Remove WGET_DATADIR from DEFS 8128 * unit-tests/Makefile.am: Likewise 8129 * data/effective_tld_names.dat: Move to unit-tests/files/public_suffix_list.dat 8130 and update from upstream 8131 * unit-tests/test.c (test_cookies): Use public_suffix_list.dat 8132 81332018-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 8134 8135 * docs/wget2.md: Add text for --xattr 8136 81372018-01-10 Tim Rühsen <tim.ruehsen@gmx.de> 8138 8139 Add man pages to tarball 8140 81412018-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 8142 8143 Fix building docs 8144 8145 * Makefile.am: Add README.md to EXTRA_DIST 8146 * configure.ac: Generate conditionals WITH_DOXYGEN and WITH_PANDOC 8147 * docs/Makefile.am: Use conditionals for building 8148 81492018-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 8150 8151 Check for pandoc in ./configure 8152 8153 * configure.ac: Check for pandoc 8154 * docs/Makefile.am: Use $(PANDOC) 8155 * docs/wget2_md2man.sh.in: Use $PANDOC shell variable 8156 81572018-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 8158 8159 Generate .texi and .info from markdown manual 8160 8161 * Makefile.am: Convert .md to .texi and .info 8162 * mainpage.md: Rename wget2_manual.md to wget2.md 8163 * wget2_manual.md: Renamed to wget2.md 8164 81652018-01-09 Darshit Shah <darnir@gnu.org> 8166 8167 Add a space after section header string in md manual 8168 81692018-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 8170 8171 Generate wget2 man page 8172 8173 * .gitignore: Add auto-generated docs/wget2_md2man 8174 * README.md: Add pandoc 8175 * configure.ac: Generate docs/wget2_md2man 8176 * docs/Makefile.am: Build wget2.1 8177 * docs/wget2_manual.md: Fix things for a better man page experience 8178 * docs/wget2_md2man.sh.in: Template file for docs/wget2_md2man 8179 81802018-01-10 Tim Rühsen <tim.ruehsen@gmx.de> 8181 8182 * po/POTFILES.in: Add src/stats.c 8183 81842018-01-10 Tim Rühsen <tim.ruehsen@gmx.de> 8185 8186 Fix calls to info_printf() for translation 8187 8188 * src/stats.c: Add _() around static text 8189 * src/wget.c: Likewise 8190 81912018-01-10 Tim Rühsen <tim.ruehsen@gmx.de> 8192 8193 * .gitlab-ci.yml: 'make' before 'make distcheck' in VPATH build 8194 81952018-01-09 Neil Locketz <neil@nlocketz.com> 8196 8197 GPG signature information now presented in --site-stats 8198 8199 * src/stats.c: Site stats output contains relevant GPG signature information 8200 * stc/wget.c: Now exits with just a generic GPG error if any sigs fail validation. 8201 More information presented in --site-stats output. 8202 * src/wget_host.h: Added signature information to DOC struct. 8203 * src/wget_options.h: Removed old exit codes in favor of single GPG error exit code. 8204 * tests/Makefile.am: Added test-gpg-bad to Makefile 8205 * tests/gpg/helloworld.txt.bad.sig: A corrupted signature (I just deleted a byte somewhere) 8206 * tests/test-gpg-bad.c: A test for a bad (corrupted) signature. 8207 * tests/test-gpg-invalid.c: Changed exit code to the single GPG error exit code. 8208 82092017-12-26 Tim Rühsen <tim.ruehsen@gmx.de> 8210 8211 Use hashmap instead of vector for DNS caching 8212 8213 * libwget/dns_cache.c: Likewise 8214 82152017-12-26 Tim Rühsen <tim.ruehsen@gmx.de> 8216 8217 Refactor DNS chaching code out of net.c 8218 8219 * include/wget/wget.h: Add missing wget_dns_cache declarations 8220 * libwget/Makefile.am: Add dns_cache.c 8221 * libwget/dns_cache.c: New file 8222 * libwget/net.c: Remove DNS chaching code 8223 82242018-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 8225 8226 Fix MinGW compilation without threads 8227 8228 * .gitlab-ci.yml: Test without multi-threading 8229 * pthread_sigmask.c.mingw: New file with patch for MinGW 8230 8231 Wine/MinGW seems to have a multi-threadign issue (sometimes hangs). 8232 Disabling threads using ./configure --disable-threads needs a change in 8233 gnulib. It has been reported, but not worked on currently. 8234 So we provide our own solution. 8235 82362018-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 8237 8238 * gnulib: Update 8239 82402018-01-05 Tim Rühsen <tim.ruehsen@gmx.de> 8241 8242 * libwget/ssl_gnutls.c (_print_info): Amend printed error messages 8243 82442018-01-04 Darshit Shah <darnir@gnu.org> 8245 8246 Eliminate redundant variable config.head_progress 8247 82482018-01-04 Darshit Shah <darnir@gnu.org> 8249 8250 Unify functions to print progress stats and progress_head stats 8251 82522018-01-04 Darshit Shah <darnir@gnu.org> 8253 8254 * libwget/iri.c: Don't percent encode the query portion of a IRI 8255 82562018-01-02 Darshit Shah <darnir@gnu.org> 8257 8258 Add support to change the GPG signature file extension 8259 8260 * src/wget_options.h: Define a new option, --signature-extension 8261 * src/options.c: Guard the SECTION_GPG in help_section_t with ifdef's. 8262 Set new option as a string and give it a default value 8263 * src/wget.c(process_response): Use the value in config.sig_ext to 8264 decide the extension of the signature file 8265 * tests/Makefile.am: Add new tests to Makefile 8266 * tests/test-gpg-styles.c: New test file 8267 82682018-01-02 Darshit Shah <darnir@gnu.org> 8269 8270 * src/options.c: Put GPG options in their own help section 8271 82722018-01-04 Darshit Shah <darnir@gnu.org> 8273 8274 * src/wget.c: Gracefully disable progress when built without threads 8275 82762018-01-03 Tim Rühsen <tim.ruehsen@gmx.de> 8277 8278 Fix build to work without multi-threading 8279 8280 * libwget/thread.c: Fix wget_thread_start() and wget_thread_join() 8281 * src/wget.c: Fix typo 8282 * tests/libtest.c: Don't skip tests when multi-threading is disabled 8283 82842018-01-03 Tim Rühsen <tim.ruehsen@gmx.de> 8285 8286 Fix --without-plugin-support build 8287 8288 * configure.ac: Check for dlsym and introduce $FUZZ_LIBS 8289 * fuzz/Makefile.am: Use $FUZZ_LIBS instead of $LIBS 8290 82912018-01-03 Tim Rühsen <tim.ruehsen@gmx.de> 8292 8293 * tests/Makefile.am: Do not build/run GPG tests if gpgme isn't installed 8294 82952018-01-01 Tim Rühsen <tim.ruehsen@gmx.de> 8296 8297 Add test for --filter-mime-type 8298 8299 * tests/Makefile.am: Add test--filter-mime-type 8300 * tests/test--filter-mime-type.c: New test 8301 83022017-12-30 DalmeGNU <dalmemail@gmail.com> 8303 8304 Add --filter-mime-type option 8305 8306 * src/wget.c: Add mime type check before downloading a file 8307 * src/options.c: Add --filter-mime-type option 8308 * src/wget_options.h: Add mime_types to the config structure 8309 * docs/wget2_manual.md: Add documentation related to the new option 8310 83112018-01-01 Darshit Shah <darnir@gnu.org> 8312 8313 Cleaner error messages for GPG Verification 8314 8315 * src/wget.c(process_response): Remove some info_printf statements. They 8316 are printed from gpgme now 8317 * src/gpgme.c: Pass the filename of the signature file (if any) ad use 8318 it to print an error message to the screen. This prevents the double 8319 error messages that were printed and works well with the progress bar 8320 83212017-12-25 Neil Locketz <neil@nlocketz.com> 8322 8323 Fixed problems pointed out in MR. Added tests and doxygen documentation. 8324 8325 * bootstrap.conf: Add canonicalize module. 8326 * cfg.mk: Added exclusions to no new line at EOF for binary files used in tests. 8327 * configure.ac: Check for libgpgme 8328 * docs/wget2_manual.md: Added manual entries for --verify-sig, --gnupg-homedir, and new exit codes. 8329 * fuzz/Makefile.am: Amend linker options 8330 * include/wget/wget.h: Add error codes for GPG signature verification. 8331 * po/POTFILES.in: Add new files. 8332 * src/Makefile.am: Add src/gpgme.c and src/wget_gpgme.h 8333 * src/gpgme.c: Using new error codes, and better signature verification failure detection. 8334 * src/job.c (job_free): Free sig_filename 8335 * src/options.c: Add new options --gnupg-homedir and --verify-sig 8336 * src/wget.c: Verify the signature if possible, exit with the proper status code, 8337 print error messages 8338 * src/wget_gpgme.h: Add documentation. 8339 * src/wget_job.h: Extend struct JOB 8340 * src/wget_options.h: Add new status codes, add new members to struct config 8341 * tests/Makefile.am: Added new tests. 8342 * tests/gpg-test-util.h: Common GPG testing functionality (tries to verify a signature, expects exit code ... etc). 8343 * tests/gpg/helloworld.txt: Text that has been signed in the .sig files. 8344 * tests/gpg/helloworld.txt.{invalid,no-pub,missing,trusted}.sig: Signatures on helloworld.txt with properties described by their names. 8345 * tests/gpg/openpgp-revocs.d/*: Revocation certs for the fake "homedir" 8346 * tests/gpg/private-keys-v1.d/*: The not-so-private private keys used to sign the test file. 8347 * tests/gpg/pubring.kbx: Fake gnupg homedir public key ring. 8348 * tests/gpg/test-gpg-*.c: Tests. 8349 * tests/valgrind-supressions: Supress failures due to oddness with GPGME. 8350 * unit-tests/Makefile.am: Add src/gpgme.o to BASE_OBJS 8351 83522018-01-01 Tim Rühsen <tim.ruehsen@gmx.de> 8353 8354 * cfg.mk: Exclude *gnulib/*.c from sc_copyright_check 8355 83562018-01-01 Tim Rühsen <tim.ruehsen@gmx.de> 8357 8358 * libwget/thread.c: Fix typos (make syntax-check) 8359 83602017-12-31 Tim Rühsen <tim.ruehsen@gmx.de> 8361 8362 Remove unused function wget_vector_deinit() 8363 8364 * include/wget/wget.h: Likewise 8365 * libwget/vector.c: Likewise 8366 83672017-12-31 Tim Rühsen <tim.ruehsen@gmx.de> 8368 8369 * src/stats.c (_free_tree_docs_entry): Use wget_vector_free() 8370 83712017-12-29 Tim Rühsen <tim.ruehsen@gmx.de> 8372 8373 Add docs for thread functions 8374 8375 * docs/Makefile.am: Add man page 8376 * libwget/thread.c: Add documentation 8377 83782017-12-31 Tim Rühsen <tim.ruehsen@gmx.de> 8379 8380 * README.md: Extend list of dependencies 8381 83822017-12-31 Tim Rühsen <tim.ruehsen@gmx.de> 8383 8384 * fuzz/main.c: Fix path separator on MinGW 8385 83862017-12-30 Tim Rühsen <tim.ruehsen@gmx.de> 8387 8388 Revert "* .gitlab-ci.yml: Remove MinGW CI runner" 8389 8390 This reverts commit 89166a38d428ccb02eb263394130dd4acada2b51. 8391 8392 At least on Debian unstable the MinGW build works stable now. 8393 This might be due to the latest fixes in the threading area. 8394 83952017-12-30 DalmeGNU <dalmemail@gmail.com> 8396 8397 * docs/wget2_manual.md: Remove duplicate documentation on the --report-speed option and change it a bit 83982017-12-30 Tim Rühsen <tim.ruehsen@gmx.de> 8399 8400 * libwget/bar.c (wget_bar_set_slots): Fix memleak when adding more slots 8401 84022017-12-29 Tim Rühsen <tim.ruehsen@gmx.de> 8403 8404 * src/wget.c (main): Fix read overflow on progress+spider without TTY 8405 84062017-12-23 DalmeGNU <dalmemail@gmail.com> 8407 8408 Fix the filename encoding for the progress bar in spider mode 8409 8410 * src/wget.c: Use functions from wget2 to get the filename, 8411 refactor get_local_filename(). Shorten the progress info 8412 line 8413 * src/wget_main.h: Use const for get_local_filename() param 8414 84152017-12-28 Tim Rühsen <tim.ruehsen@gmx.de> 8416 8417 * libwget/thread.c: Suppress -Wundef warnings in gnulib includes 8418 84192017-12-27 Tim Rühsen <tim.ruehsen@gmx.de> 8420 8421 Avoid copying initialized glthread structures 8422 8423 * libwget/thread.c: Use malloc+init instead init+memdup 8424 8425 Memdup'ing the glthread structures broke internal pointers, 8426 at least on Windows. 8427 8428 Reported-by: Gisle Vanem 8429 84302017-12-23 DalmeGNU <dalmemail@gmail.com> 8431 8432 Print progress info related to headers when spider+progress are set 8433 8434 * src/options.c: Set config.head_progress if spider+progress are set 8435 * src/wget.c: New function print_head_progress_report(), 8436 new function get_filename() 8437 * src/wget_options.h: New var config.head_progress 8438 84392017-12-25 Tim Rühsen <tim.ruehsen@gmx.de> 8440 8441 * src/bar.c: Let progress thread stop 8442 8443 Glthreads don't have a cancel or kill function, so we use 8444 a global termination flag. 8445 84462017-12-10 DalmeGNU <dalmemail@gmail.com> 8447 8448 Improve progress speed display 8449 8450 * include/wget/wget.h: Use enum for WGET_REPORT_SPEED_* 8451 * libwget/bar.c: Add ringbuffer for smoother speed calculations 8452 * src/wget.c (main): Replace call to print_progress_report() by 8453 call to bar_printf() 8454 84552017-12-20 Darshit Shah <darnir@gnu.org> 8456 8457 libwget/bar.c: Ensure all progress bars have the same size 8458 84592017-12-20 Tim Rühsen <tim.ruehsen@gmx.de> 8460 8461 Add test for multi-threading 8462 8463 * unit-tests/Makefile.am: Add test-cond 8464 * unit-tests/test-cond.c: New test 8465 84662017-12-17 Tim Rühsen <tim.ruehsen@gmx.de> 8467 8468 * contrib/build_fuzzers: Print out a small info 8469 84702017-05-03 Tim Rühsen <tim.ruehsen@gmx.de> 8471 8472 Thread abstraction via gnulib thread wrapper (glthread) 8473 8474 * bootstrap.conf: Add modules cond, lock and thread, 8475 remove module pthread 8476 * include/wget/wget.h: Remove pthread code, 8477 add new wget_thread_* functions, 8478 amend params of wget_thread_* functions, 8479 add init and exit functions for mutex initialization 8480 * */*.[ch]: Amend threading to new API 8481 8482 This change let libwget work with different kinds of 8483 threading libraries incl. pthreads and windows threads. 8484 84852017-12-15 Tim Rühsen <tim.ruehsen@gmx.de> 8486 8487 * configure.ac: Reduce warnings from gnulib code 8488 84892017-12-12 Tim Rühsen <tim.ruehsen@gmx.de> 8490 8491 Fix some more URLs from http:// to https:// 8492 8493 * .travis.sh: Likewise 8494 * .travis_setup.sh: Likewise 8495 * COPYING: Likewise 8496 * COPYING.LESSER: Likewise 8497 * bootstrap: Likewise 8498 * include/wget/wget.h: Likewise 8499 * include/wget/wgetver.h.in: Likewise 8500 * libwget/atom_url.c: Likewise 8501 * libwget/iri.c: Likewise 8502 85032017-12-11 Tim Rühsen <tim.ruehsen@gmx.de> 8504 8505 Fix race condition(s) in downloader logic 8506 8507 * src/host.c: Use new flag job.done, 8508 add more documentation, 8509 protect some debugging lines with hosts_mutex 8510 * src/wget.c: Use job.done instead of job.inuse 8511 * src/wget_host.h: Use void return for host_add_job() 8512 and host_add_robotstxt_job() 8513 * src/wget_job.h (struct JOB): Add job.done 8514 8515 The job.inuse flag has been used for two purposes. This 8516 resulted in sporadic conflicts between threads. While one was 8517 destroying a job structure another got this structure from the 8518 job queue. Best reproducible with '-r --spider' combination. 8519 85202017-12-06 Tim Rühsen <tim.ruehsen@gmx.de> 8521 8522 Fix deadlock in host_remove_job() 8523 8524 * src/host.c (_host_remove_job): New function fixes deadlock 8525 8526 Fixes #314 8527 85282017-12-08 Tim Rühsen <tim.ruehsen@gmx.de> 8529 8530 * fuzz/*_fuzzer.in/*: Update fuzzer corpora 8531 85322017-12-08 Tim Rühsen <tim.ruehsen@gmx.de> 8533 8534 * contrib/build_fuzzers: Add new helper script to build fuzzers 8535 85362017-12-08 Tim Rühsen <tim.ruehsen@gmx.de> 8537 8538 * fuzz/get_ossfuzz_corpora: Fix script to handle many corpora 8539 85402017-12-08 Tim Rühsen <tim.ruehsen@gmx.de> 8541 8542 * src/wget.c: Fix clang warning about 64/32 conversion 8543 85442017-12-08 Tim Rühsen <tim.ruehsen@gmx.de> 8545 8546 * libwget/net.c: Fix clang warning about 64/32 conversion 8547 85482017-12-05 Tim Rühsen <tim.ruehsen@gmx.de> 8549 8550 * gnulib: Update 8551 85522017-11-30 DalmeGNU <dalmemail@gmail.com> 8553 8554 Implement speed reporting in the progress bar and the --report-speed switch 8555 8556 * docs/wget2_manual.md: Add documentation for the --report-speed switch 8557 * include/wget/wget.h: Define the report speed types, add reference to wget_bar_set_speed_type() 8558 * libwget/bar.c: Modify the wget2 progress bar to print speed information 8559 * src/bar.c: Add code to set the type of speed reporting 8560 * src/options.c: Add the --report-speed switch and a function to parse the legal types 8561 * src/wget_options.h: Add 'report_speed' in the config structure 8562 * src/wget.c: Print speed reports at the end of a download thread 8563 85642017-12-06 Tim Rühsen <tim.ruehsen@gmx.de> 8565 8566 Fix several issues foudn by cppcheck 8567 8568 * libwget/bar.c (_wget_bar_st): Remove unused member screen_width 8569 * libwget/css.c: Reduce scope of variables 8570 * libwget/encoding.c: Likewise 8571 * libwget/hpkp.c: Likewise 8572 * libwget/http.c: Likewise 8573 * libwget/http_parse.c: Likewise 8574 * libwget/iri.c: Likewise 8575 * libwget/net.c: Likewise 8576 * libwget/sitemap_url.c: Likewise 8577 * libwget/ssl_gnutls.c: Likewise 8578 * libwget/vector.c: Likewise 8579 * src/dl.c: Likewise 8580 85812017-12-06 Tim Rühsen <tim.ruehsen@gmx.de> 8582 8583 * .gitlab-ci.yml: Add VPATH build 8584 85852017-12-06 Tim Rühsen <tim.ruehsen@gmx.de> 8586 8587 Use BUILDDIR for popen() in test suite 8588 8589 * tests/Makefile.am: Set BUILDDIR 8590 * tests/libtest.c: Use BUILDDIR for executable path 8591 85922017-12-05 Tim Rühsen <tim.ruehsen@gmx.de> 8593 8594 Update link libraries in Makefiles 8595 8596 * fuzz/Makefile.am: Update link libraries and remove redundant code 8597 * libwget/Makefile.am: Likewise 8598 * src/Makefile.am: Likewise 8599 * tests/Makefile.am: Likewise 8600 * unit-tests/Makefile.am: Likewise 8601 86022017-12-04 Tim Rühsen <tim.ruehsen@gmx.de> 8603 8604 * po/POTFILES.in: Add lib/regcomp.c 8605 86062017-12-04 Tim Rühsen <tim.ruehsen@gmx.de> 8607 8608 * libwget/net.c: Fix -Wundef warning on Win32 8609 86102017-12-04 Tim Rühsen <tim.ruehsen@gmx.de> 8611 8612 * src/options.c: Include getpass.h 8613 86142017-12-04 Tim Rühsen <tim.ruehsen@gmx.de> 8615 8616 * bootstrap.conf: Add modules pipe-posix, regex 8617 8618 # Conflicts: 8619 # bootstrap.conf 8620 86212017-12-02 Tim Rühsen <tim.ruehsen@gmx.de> 8622 8623 * .gitlab-ci.yml: Use new Fedora27 CI image 8624 86252017-12-02 Tim Rühsen <tim.ruehsen@gmx.de> 8626 8627 * libwget/hpkp.c: Silence doxygen 8628 86292017-12-02 Tim Rühsen <tim.ruehsen@gmx.de> 8630 8631 * libwget/bar.c: Add missing documentation 8632 86332017-12-02 Tim Rühsen <tim.ruehsen@gmx.de> 8634 8635 * libwget/hashfile.c: Use doxygen 'ingroup' instead of 'defgroup' 8636 86372017-12-02 Tim Rühsen <tim.ruehsen@gmx.de> 8638 8639 * libwget/ssl_gnutls.c: Fix end of doxygen file 8640 86412017-12-02 Tim Rühsen <tim.ruehsen@gmx.de> 8642 8643 Fix some doxygen issues 8644 8645 * libwget/hash_printf.c: Fix name of param 8646 * libwget/hpkp.c: Document missing param, 8647 (wget_hpkp_db_add): Rename param 8648 * libwget/ssl_gnutls.c: Fix \ref syntax warnings 8649 * libwget/strscpy.c: Fix param usage in docs 8650 86512017-08-19 Ander Juaristi <a@juaristi.eus> 8652 8653 doxygen: Document SSL/TLS engine 8654 86552017-11-30 DalmeGNU <dalmemail@gmail.com> 8656 8657 * docs/wget2_manual.md: Add documentation for --max-threads and --chunk-size switches 8658 86592017-11-29 Tim Rühsen <tim.ruehsen@gmx.de> 8660 8661 * .gitlab-ci.yml: Add make distcheck 8662 86632017-11-29 Dan <dalmemail@amaya.tk> 8664 8665 * src/options.c: Prevert --max-threads & --chunk-size flags combination 8666 86672017-11-28 DalmeGNU <dalmemail@gmail.com> 8668 8669 * src/options.c: Don't allow wget2 to show any password dialogs if there are no URL arguments 8670 86712017-11-28 Akash Rawal <argmlac@gmail.com> 8672 8673 * unit-tests/Makefile.am: Make libwget usable in dummy libraries 8674 86752017-11-24 Akash Rawal <argmlac@gmail.com> 8676 8677 Fix 'make distcheck' for VPATH build 8678 8679 * fuzz/Makefile.am: Use $(srcdir) to find files 8680 * tests/Makefile.am: Add stats-test-util.h to *_SOURCES where needed 8681 86822017-11-24 Akash Rawal <argmlac@gmail.com> 8683 8684 * .travis.yml: Fix travis-ci build 8685 86862017-11-28 DalmeGNU <dalmemail@gmail.com> 8687 8688 * src/wget.c: Let the user know if file is completely retrieved after a 416 HTTP error 8689 86902017-11-29 Tim Rühsen <tim.ruehsen@gmx.de> 8691 8692 * fuzz/wget_options_fuzzer.c: Fix for C++ building 8693 86942017-11-21 Tim Rühsen <tim.ruehsen@gmx.de> 8695 8696 * fuzz/Makefile.am: Fix building for oss-fuzz 8697 86982017-11-18 Tim Rühsen <tim.ruehsen@gmx.de> 8699 8700 Add testmode to avoid interactive functions when testing 8701 8702 * fuzz/Makefile.am: Link options fuzzer with testing.o 8703 * fuzz/wget_options_fuzzer.c: Set test mode 8704 * src/Makefile.am: Add testing.c and wget_testing.h 8705 * src/options.c (prompt_for_password, run_use_askpass): 8706 Check for testing mode 8707 * src/testing.c: New file 8708 * src/wget_testing.h: New file 8709 * unit-tests/Makefile.am: Link with testing.o 8710 87112017-11-16 Tim Rühsen <tim.ruehsen@gmx.de> 8712 8713 * libwget/hsts.c: Check some function arguments against NULL 8714 87152017-11-16 Tim Rühsen <tim.ruehsen@gmx.de> 8716 8717 * tests/test--accept.c: Fix new tests for Windows 8718 8719 Reported-by: Gisle Vanem 8720 87212017-11-15 Tim Rühsen <tim.ruehsen@gmx.de> 8722 8723 * src/options.c: Fix reading passwords for fuzzing 8724 87252017-11-15 Tim Rühsen <tim.ruehsen@gmx.de> 8726 8727 Fix resource leaks 8728 8729 * src/options.c (prompt_for_password): Fix indentation, 8730 (run_use_askpass): Add file actions to close pipe fds in child, 8731 (init): Free config.password before calling prompt_for_password(), 8732 (deinit): Free config.use_askpass_bin 8733 87342017-11-14 Avinash Sonawane <rootkea@gmail.com> 8735 8736 Fix warnings 8737 8738 * src/wget.c : Declare 'nthreads' as static variable 8739 * src/wget.c (regex_match_pcre): explicit typecast to PCRE2_SPTR 8740 87412017-11-14 Tim Rühsen <tim.ruehsen@gmx.de> 8742 8743 * tests/test-stats-tls.c: Remove superfluous include 8744 87452017-11-14 Tim Rühsen <tim.ruehsen@gmx.de> 8746 8747 * tests/stats-test-util.h: Fix free() on arbitrary stack content 8748 87492017-11-14 Tim Rühsen <tim.ruehsen@gmx.de> 8750 8751 * src/options.c (run_use_askpass): Remove warning 8752 87532017-11-13 Neil Locketz <neil@nlocketz.com> 8754 8755 Hosts gathered from metalinks are added to hosts hashmap 8756 8757 * src/wget.c: Add metalink mirrors to hosts hashmap 8758 87592017-11-14 Tim Rühsen <tim.ruehsen@gmx.de> 8760 8761 * src/wget.c: Fix check for WITH_LIBPCRE 8762 87632017-11-14 Tim Rühsen <tim.ruehsen@gmx.de> 8764 8765 * src/options.c (run_use_askpass): Use const for 'question' 8766 87672017-11-14 Tim Rühsen <tim.ruehsen@gmx.de> 8768 8769 * src/options.c (init): Fix memleak (harmless) 8770 87712017-11-14 Tim Rühsen <tim.ruehsen@gmx.de> 8772 8773 * README.md: Add lzip to list of prerequisites 8774 87752017-11-13 Tim Rühsen <tim.ruehsen@gmx.de> 8776 8777 * src/options.c: Cleanups, prevent resource leaks in --use-askpass 8778 87792017-11-11 DalmeGNU <dalmemail@gmail.com> 8780 8781 Disable TLS Session Resumption by default 8782 8783 * src/options.c: Set config.tls_resume to false/zero by default 8784 8785 * docs/wget2_manual.md: Add docs for --tls-resume 8786 87872017-11-06 DalmeGNU <dalmemail@gmail.com> 8788 8789 Implementation of --use-askpass switch 8790 8791 * bootstrap.conf: Add 'posix_spawn' and 'spawn-pipe' gnulib modules 8792 * docs/wget2_manual.md: Add docs for --use-askpass 8793 * src/options.c: Add implementation of --use-askpass 8794 * src/wget_options.h (struct config): Add 'use_askpass_bin' member 8795 87962017-11-11 Tim Rühsen <tim.ruehsen@gmx.de> 8797 8798 * bootstrap.conf: Add more prerequisites 8799 88002017-11-10 Tim Rühsen <tim.ruehsen@gmx.de> 8801 8802 * include/wget/wget.h: More documentation added 8803 88042017-11-10 Tim Rühsen <tim.ruehsen@gmx.de> 8805 8806 * src/wget.c: Add Cache-Control: no-cache if --no-cache is given 8807 88082017-11-06 Neil Locketz <neil@nlocketz.com> 8809 8810 Response Time Site Statistic 8811 8812 * include/wget/wget.h: first_data_back field added to represent the time of first bytes received back. 8813 * libwget/http.c: Assigning first_data_back at the appropriate time. 8814 * stc/wget_host.h: Added response_time field to DOC, holds response time for document. 8815 * src/stats.c: Logic to calculate response_time for DOCs, and stat printing. 8816 88172017-11-06 Neil Locketz <neil@nlocketz.com> 8818 8819 Transfer time site statistic 8820 8821 * include/wget/wget.h: Added request_start, and response_end to wget_http_request_t, and wget_http_response_t respectively. 8822 * libwget/http.c: New fields initialized/set with their corresponding values at the right time. 8823 * stc/wget_host.h: Added timestamps to DOC for request timeframes. 8824 * src/stats.c: Logic to calculate request_start and response_end for DOCs. Stats writing. 8825 88262017-11-05 Tim Rühsen <tim.ruehsen@gmx.de> 8827 8828 * src/options.c: Skip DNS resolve for bind address for fuzz/ tests 8829 88302017-11-03 Tim Rühsen <tim.ruehsen@gmx.de> 8831 8832 * src/wget.c (add_statistics): Count status 303, 307, 308 as redirects 8833 88342017-11-03 Tim Rühsen <tim.ruehsen@gmx.de> 8835 8836 * libwget/http.c (wget_http_get_response_cb): Improve gzip workaround 8837 88382017-10-31 DalmeGNU <dalmemail@gmail.com> 8839 8840 Implement --ask-password switch to enable a password prompt 8841 8842 * bootstrap.conf: Add 'getpass' gnulib module 8843 * docs/wget2_manual.md: Amend docs for --ask-password 8844 * src/options.c: Add implementation of --ask-password 8845 * src/wget_options.h (struct config): Add 'password' member 8846 88472017-11-02 Tim Rühsen <tim.ruehsen@gmx.de> 8848 8849 * include/wget/wget.h: Add some struct member documentation 8850 88512017-11-02 Tim Rühsen <tim.ruehsen@gmx.de> 8852 8853 Fix out-of-memory issue in option parsing 8854 8855 * src/options.c (parse_stringlist_expand): Limit number 8856 entries in vector table. 8857 8858 And endless loop situation when reading config files lead 8859 to an out-of-memory situation. 8860 8861 Found by oss-fuzz (issue 4015). 8862 Bug existed since a long time. 8863 88642017-11-01 Tim Rühsen <tim.ruehsen@gmx.de> 8865 8866 * fuzz/wget_options_fuzzer.in: Update fuzzer corpora 8867 88682017-11-01 Tim Rühsen <tim.ruehsen@gmx.de> 8869 8870 Let plugin_db_load_from_envvar() return error 8871 8872 * src/options.c (init): Return error if plugin_db_load_from_envvar() fails 8873 * src/plugin.c (plugin_db_load_from_envvar): Return error instead of void 8874 * src/wget_plugin.h (plugin_db_load_from_envvar): Likewise 8875 88762017-10-23 Neil Locketz <neil@nlocketz.com> 8877 8878 Split up plugin tests into several files 8879 8880 * tests/Makefile.am: Add new files 8881 * tests/test-plugin-failure.c: New file with tests based on invalid input 8882 * tests/test-plugin-interception.c: New file for testing the various things 8883 plugins can intercept 8884 * tests/test-plugin-nonexistance.c: New file for tests with plugins that don't exist) 8885 * tests/test-plugin.c: Moved tests to the above new files 8886 88872017-10-23 Neil Locketz <neil@nlocketz.com> 8888 8889 Split individual stats tests into their own files 8890 8891 * tests/Makefile.am: Add new files 8892 * tests/stats-test-util.h: New file 8893 * tests/test-stats-all.c: New file 8894 * tests/test-stats-dns.c: DNS stats tests 8895 * tests/test-stats-ocsp.c: OCSP stats tests 8896 * tests/test-stats-server.c: Server stats tests 8897 * tests/test-stats-site.c: Site stats tests 8898 * tests/test-stats-tls.c: TLS stats tests 8899 89002017-11-01 Tim Rühsen <tim.ruehsen@gmx.de> 8901 8902 Fix memory leak in wget2 8903 8904 * src/options.c (init): Free config.logfile before assignment 8905 8906 Found by oss-fuzz (issue 3963). 8907 Bug existed since a long time. 8908 Bug is not relevant except for testing. 8909 89102017-10-31 Tim Rühsen <tim.ruehsen@gmx.de> 8911 8912 Fix buffer overflow in option parser 8913 8914 * src/options.c (_parse_option): Add end-of-string check 8915 8916 Found by oss-fuzz (issue 3950). 8917 Bug existed since a long time. 8918 89192017-10-29 Tim Rühsen <tim.ruehsen@gmx.de> 8920 8921 Add support for RFC 7616 HTTP Digest Access Authentication 8922 8923 * docs/Makefile.am: Remove libwget-md5.3 8924 * include/wget/wget.h: Remove wget_md5_printf_hex(), 8925 add wget_hash_printf_hex() 8926 * libwget/Makefile.am: Rename md5.c to hash_printf.c 8927 * libwget/hash_printf.c: New file with wget_hash_printf_hex() 8928 * libwget/http.c (wget_http_add_credentials): Remove hardcoded MD5 algo 8929 * tests/test-metalink.c: Use wget_hash_printf_hex() 8930 8931 RFC 7616 adds/replaces the MD5 digest algorithm with SHA-256. 8932 89332017-04-13 Michael Heerklotz <michael.heerklotz@check24.de> 8934 8935 Added new setting --filter-urls 8936 8937 * docs/wget2_manual.md: Added doc for new setting 8938 * src/options.c: Added new setting 8939 * src/wget.c: Added code for new setting. 8940 * src/wget_options.h: Added new setting 8941 8942 This commit hasn't been copyright-assigned to the FSF. 8943 89442017-04-13 Michael Heerklotz <michael.heerklotz@check24.de> 8945 8946 Added regex filters (--accept-regex, --reject-regex, --regex-type) 8947 8948 * configure.ac: Added detection of libpcre and libpcre2 8949 * include/wget/wget.h: Added regex_type defines 8950 * src/options.c: Added the new options 8951 * src/wget.c: Added regex code 8952 * src/wget_options.h: Added the new options 8953 * tests/test--accept.c: Added tests for the new options 8954 8955 This commit hasn't been copyright-assigned to the FSF. 8956 89572017-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 8958 8959 Fixes for oss-fuzz 8960 8961 * fuzz/Makefile.am: Extend oss-fuzz make script 8962 * fuzz/wget_options_fuzzer.c: Do not opendir() in init() 8963 * src/wget_options.h: Tell C++ that functions are C 8964 * src/wget_plugin.h: Likewise 8965 89662017-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 8967 8968 Add wget_options_fuzzer 8969 8970 * Makefile.am: Fix make target 'fuzz-coverage' 8971 * configure.ac: Enable building static library by default, 8972 add option --enable-fuzzing 8973 * fuzz/Makefile.am: Add wget_options_fuzzer, 8974 check for FUZZING (set with --enable-fuzzing) 8975 * fuzz/README.md: Amend the text 8976 * fuzz/main.c: Use printf() instead of wget_info_printf() 8977 * fuzz/run-clang.sh: Use fuzzer binaries built by 'make' 8978 * fuzz/wget_options_fuzzer.c: New fuzzer 8979 * fuzz/wget_options_fuzzer.dict: New fuzzer dictionary 8980 * fuzz/wget_options_fuzzer.in/*: Initial fuzz corpora 8981 * libwget/net.c: Skip IP address resolution when fuzzing 8982 * src/log.c: Don't create files when fuzzing, 8983 don't print to console when fuzzing 8984 * src/options.c: Add set_exit_status() and get_exit_status(), 8985 don't print --help / --version to console when fuzzing, 8986 do not call exit() - return error instead, 8987 fix recursion level in _read_config(), 8988 don't create files when fuzzing, 8989 fix memory leaks in deinit() 8990 * src/stats.c: Don't create files when fuzzing 8991 * src/wget.c: Remove set_exit_status() 8992 * src/wget_main.h: Remove exit_status_t 8993 * src/wget_options.h: Add exit_status_t 8994 * tests/test-plugin.c: Fix expected exit codes from 1 to 2 8995 8996 This is for application fuzzing (namely code from src/). 8997 The code in src/ had to be prepared in certain ways, but it 8998 doesn't add significant overhead, even makes the code cleaner 8999 in some ways. Several of these changes have already been committed 9000 into the master branch. 9001 90022017-10-22 Tim Rühsen <tim.ruehsen@gmx.de> 9003 9004 Rename WGET_HTML_PARSED_RESULT to wget_html_parsed_result_t 9005 9006 * examples/check_url_types.c: Likewise 9007 * examples/print_html_urls.c: Likewise 9008 * fuzz/libwget_html_url_fuzzer.c: Likewise 9009 * include/wget/wget.h: Likewise 9010 * libwget/html_url.c: Likewise 9011 * src/wget.c: Likewise 9012 90132017-10-22 Tim Rühsen <tim.ruehsen@gmx.de> 9014 9015 Rename WGET_HTML_PARSED_URL to wget_html_parsed_url_t 9016 9017 * examples/check_url_types.c: Likewise 9018 * examples/print_html_urls.c: Likewise 9019 * include/wget/wget.h: Likewise 9020 * libwget/html_url.c: Likewise 9021 * src/wget.c: Likewise 9022 90232017-10-22 Tim Rühsen <tim.ruehsen@gmx.de> 9024 9025 Rename WGET_PARSED_URL to wget_css_parsed_url_t 9026 9027 * examples/print_css_urls3.c: Likewise 9028 * include/wget/wget.h: Likewise 9029 * libwget/css_url.c: Likewise 9030 90312017-10-22 Tim Rühsen <tim.ruehsen@gmx.de> 9032 9033 Rename ROBOTS to wget_robots_t 9034 9035 * fuzz/libwget_robots_parse_fuzzer.c: Rename ROBOTS to wget_robots_t 9036 * include/wget/wget.h: Likewise 9037 * libwget/robots.c: Likewise 9038 * src/wget_host.h: Likewise 9039 * unit-tests/test.c: Likewise 9040 90412017-10-22 Tim Rühsen <tim.ruehsen@gmx.de> 9042 9043 Replace ROBOTS_PATH by wget_string_t 9044 9045 * include/wget/wget.h: Remove struct ROBOTS_PATH 9046 * libwget/robots.c: Replace ROBOTS_PATH by wget_string_t 9047 * src/host.c: Likewise 9048 * src/wget.c: Likewise 9049 * unit-tests/test.c: Likewise 9050 90512017-10-21 Tim Rühsen <tim.ruehsen@gmx.de> 9052 9053 * src/options.c (_read_config): Fix buffer underflow 9054 90552017-10-21 Tim Rühsen <tim.ruehsen@gmx.de> 9056 9057 * gnulib: Update for glob() buffer overflows 9058 90592017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9060 9061 * src/options.c (parse_timeout): Fix two integer overflows 9062 90632017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9064 9065 * src/options.c (parse_numbytes): Fix integer overflow 9066 90672017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9068 9069 Cleanup dl_error* functions 9070 9071 * src/dl.c: Deduplicate code, use const where needed 9072 * src/wget_dl.h: Use const attribute for dl_error_t.msg 9073 90742017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9075 9076 * src/wget_plugin.h: Include needed header files 9077 90782017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9079 9080 * src/options.c (get_home_dir): Allow free'ing 'home' outside the function 9081 90822017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9083 9084 * src/options.c (_read_config): Add errno and strerror to error message 9085 90862017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9087 9088 * unit-tests/test-dl.c: Fix memory leak 9089 90902017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9091 9092 * libwget/net.c (wget_tcp_set_bind_address): Fix allocation policy flag 9093 90942017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9095 9096 * libwget/net.c (_wget_tcp_resolve): Fix read of uninitialized stack memory 9097 90982017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9099 9100 * libwget/http.c (_parse_no_proxies): Remove wrong destructor from vector 'proxies' 9101 91022017-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 9103 9104 * gnulib: Update 9105 91062017-10-14 Tim Rühsen <tim.ruehsen@gmx.de> 9107 9108 * src/options.c: Fine-tune help output 9109 91102017-10-12 Tim Rühsen <tim.ruehsen@gmx.de> 9111 9112 Add wget_strscpy() 9113 9114 * include/wget/wget.h: Add wget_strscpy() 9115 * libwget/strscpy.c: New file 9116 * libwget/Makefile.am: Add strscpy.c 9117 * libwget/html_url.c: Use wget_strscpy() instead of wget_strlcpy() 9118 * libwget/http.c: Likewise 9119 * libwget/iri.c: Likewise 9120 * libwget/metalink.c: Likewise 9121 * libwget/ssl_gnutls.c: Likewise 9122 * libwget/test_linking.c: Likewise 9123 * src/job.c: Likewise 9124 * src/plugin.c: Likewise 9125 * src/wget.c: Likewise 9126 91272017-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 9128 9129 Use MHD_free in test suite if available 9130 9131 * configure.ac: Check for MHD_free(), 9132 fallback to AC_SEARCH_LIBS if pkg-config not available 9133 * tests/libtest.c: Call MHD_free() where appropriate, 9134 small cleanups 9135 91362017-10-10 Avinash Sonawane <rootkea@gmail.com> 9137 9138 * .gitlab-ci/yml: Add scan-build for fuzz/*, tests/* and unit-tests/* 9139 91402017-10-10 Avinash Sonawane <rootkea@gmail.com> 9141 9142 * .gitlab-ci.yml: Generate flawfinder report 9143 91442017-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 9145 9146 Cleanup test library 9147 9148 * tests/libtest.c (_http_server_start): Use wget_error_printf instead of printf, 9149 simplify some code, 9150 gettextize some printf strings, 9151 (wget_test): generate WIN32 command line 9152 91532017-10-09 Tim Rühsen <tim.ruehsen@gmx.de> 9154 9155 Fix usage of wget_global_init() and wget_http_get() 9156 9157 * examples/*.c: Use 0 instead of NULL as terminating param 9158 * fuzz/main.c: Likewise 9159 * include/wget/wget.h: Remove G_GNUC_WGET_NULL_TERMINATED from 9160 wget_global_init() and wget_http_get(). 9161 91622017-10-09 Tim Rühsen <tim.ruehsen@gmx.de> 9163 9164 * tests/libtest.c: Fix free'ing of certain variables 9165 91662017-10-09 Tim Rühsen <tim.ruehsen@gmx.de> 9167 9168 * include/wget/wget.h: Fix usage of G_GNUC_WGET_NONNULL for clang-cl (Windows) 9169 91702017-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 9171 9172 * src/options.c (parse_local_db): Let --local-db be a master switch 9173 91742017-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 9175 9176 Add --no-local-db to test suite 9177 9178 * tests/libtest.c: Add --no-local-db to wget2 options 9179 * tests/test-plugin.c: Fix test 9180 91812017-09-27 Dan <dalmemail@amaya.tk> 9182 9183 Implement '--local-db' option 9184 9185 * src/options.c: New function parse_local_db(), 9186 new option '--local-db', 9187 (init): Check for config settings 9188 * src/wget_options.h (struct config): Add local_db 9189 91902017-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 9191 9192 * .gitlab-ci.yml: Fix scan-build 9193 91942017-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 9195 9196 * libwget/iri.c (wget_iri_relative_to_abs): Fix null pointer dereference 9197 91982017-09-21 Tim Rühsen <tim.ruehsen@gmx.de> 9199 9200 Implement --dns-timeout using getaddrinfo_a() 9201 9202 * configure.ac: Check for getaddrinfo_a() in libanl 9203 * libwget/net.c: Implement _wget_tcp_resolve() with timeout 9204 9205 Currently, there seems to be memory leak in getaddrinfo_a(). 9206 Needs investigation and a bug report. 9207 92082017-10-06 Tim Rühsen <tim.ruehsen@gmx.de> 9209 9210 * .gitlab-ci.yml: Generate scan-build report (clang static analyzer) 9211 92122017-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 9213 9214 * libwget/ssl_gnutls.c (send_ocsp_request): Fix null ptr dereference 9215 92162017-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 9217 9218 * src/wget_bar.h: Remove unneeded G_GNUC_WGET_NONNULL_ALL from bar_slot_begin() 9219 92202017-10-07 Avinash Sonawane <rootkea@gmail.com> 9221 9222 Use snprintf() instead of sprintf() 9223 9224 * tests/libtest.c: Likewise 9225 * unit-tests/test.c: Likewise 9226 92272017-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 9228 9229 * src/wget_xattr.h: Move #include into guarded area 9230 92312017-10-05 Avinash Sonawane <rootkea@gmail.com> 9232 9233 Fix issues reported by clang static analyzer 9234 9235 * unit-tests/test-dl-dummy.c: use wget_strlcpy() intead of strcpy() 9236 * src/plugin.c: Likewise 9237 * tests/libtest.c: Get rid of strcpy() 9238 * unit-tests/test.c: Fix out-of-bound array access 9239 92402017-10-05 Avinash Sonawane <rootkea@gmail.com> 9241 9242 Reduce variable scope 9243 9244 * examples/print_html_urls.c: Likewise 9245 * fuzz/libwget_cookie_fuzzer.c: Likewise 9246 * fuzz/main.c: Likewise 9247 * tests/libtest.c: Likewise 9248 * unit-tests/test.c: Likewise 9249 9250 Found by: cppcheck 9251 92522017-10-07 Avinash Sonawane <rootkea@gmail.com> 9253 9254 Remove dead assignments 9255 9256 * libwget/http.c: Likewise 9257 * libwget/net.c: Likewise 9258 * libwget/ssl_gnutls.c: Likewise 9259 * src/wget.c: Likewise 9260 9261 Found by: Clang static analyzer (scan-build) 9262 92632017-10-05 Tim Rühsen <tim.ruehsen@gmx.de> 9264 9265 * tests/test-plugin.c: Fix check for skipping the test 9266 92672017-09-28 Josef Moellers <jmoellers@suse.de> 9268 9269 Terminate http request on response status code 416. 9270 9271 * libwget/http.c (wget_http_get_response_cb): Stop download on 9272 status code 416, cleanup using HTTP status defines. 9273 * libwget/http.h: Add HTTP status defines from Wget1.x 9274 9275 HTTP status response 416 Requested Range Not Satisfiable. 9276 This sometimes happens when a server responds incorrectly to a range request. 9277 There is not enough information available (Range of the request) to properly 9278 handle this case. If it were, the case that the file has already been 9279 completely downloaded could be detected and an appropriate message could be 9280 generated. I also took the liberty to port the status codes from the old wget 9281 source to libwget/http.h to make the code more readable. 9282 92832017-10-05 Avinash Sonawane <rootkea@gmail.com> 9284 9285 Reduce scope of variables 9286 9287 * libwget/bar.c: Likewise 9288 * libwget/console.c: Likewise 9289 * libwget/encoding.c: Likewise 9290 * libwget/hashmap.c: Likewise 9291 * libwget/io.c: Likewise 9292 * libwget/md5.c: Likewise 9293 * libwget/netrc.c: Likewise 9294 * libwget/ssl_gnutls.c: Likewise 9295 * libwget/xml.c: Likewise 9296 * src/job.c: Likewise 9297 * src/plugin.c: Likewise 9298 * libwget/hashfile.c: Include local private.h 9299 * src/host.c: Remove unused find_free_job_context struct 9300 9301 Found by: cppcheck 9302 93032017-10-04 Tim Rühsen <tim.ruehsen@gmx.de> 9304 9305 Remove testing for MHD in test suite 9306 9307 * tests/libtest.c: Likewise 9308 * tests/libtest.h: Likewise 9309 9310 Reported-by: Gisle Vanem 9311 93122017-10-04 Tim Rühsen <tim.ruehsen@gmx.de> 9313 9314 * fuzz/*.in: Update fuzzer corpora 9315 93162017-10-04 Tim Rühsen <tim.ruehsen@gmx.de> 9317 9318 * fuzz/*: Fix clang-5.0 to clang-6.0 9319 93202017-10-03 Tim Rühsen <tim.ruehsen@gmx.de> 9321 9322 Skip plugin tests if shared build is disabled 9323 9324 The use of -shared in the LDFLAGS for the plugins does not work 9325 with './configure --disable-shared'. Instead of failing the build 9326 this patch SKIPs the plugin tests. 9327 93282017-10-02 Tim Rühsen <tim.ruehsen@gmx.de> 9329 9330 * fuzz/main.c: Remove libtool prefix 'lt-' from directory names 9331 93322017-10-02 Tim Rühsen <tim.ruehsen@gmx.de> 9333 9334 Remove -static from AM_LDFLAGS in test directories 9335 9336 * fuzz/Makefile.am: Remove -static from AM_LDFLAGS 9337 * tests/Makefile.am: Likewise 9338 * unit-tests/Makefile.am: Likewise 9339 9340 'make check' didn't work on CygWin with these settings. 9341 Also, GnuTLS removed these a short time ago from the fuzz/ 9342 directory since there were build issues. 9343 93442017-10-02 Tim Rühsen <tim.ruehsen@gmx.de> 9345 9346 * contrib/mingw: Cleanup 9347 93482017-10-02 Tim Rühsen <tim.ruehsen@gmx.de> 9349 9350 * src/wget.c: Set values for enum actions 9351 93522017-10-02 Tim Rühsen <tim.ruehsen@gmx.de> 9353 9354 * README.md: Add libiconv to dependencies 9355 93562017-10-02 Tim Rühsen <tim.ruehsen@gmx.de> 9357 9358 Extend plugin loading for CygWIN 9359 9360 * src/dl.c: Add PATTERN for CygWin 9361 * tests/test-plugin.c: Define LOCAL_NAME for CygWin 9362 * unit-tests/test-dl.c: Define BUILD_NAME and LOCAL_NAME for CygWin 9363 93642017-10-02 Tim Rühsen <tim.ruehsen@gmx.de> 9365 9366 Make fuzzers survive on CygWin 9367 9368 * fuzz/libwget_hpkp_fuzzer.c: Check for RTLD_NEXT 9369 * fuzz/libwget_hsts_fuzzer.c: Likewise 9370 * fuzz/libwget_netrc_fuzzer.c: Likewise 9371 * fuzz/libwget_ocsp_fuzzer.c: Likewise 9372 * fuzz/libwget_tlssess_fuzzer.c: Likewise 9373 9374 On CygWin RTLD_NEXT doesn't exist, so we can't overwrite/preload fopen(). 9375 93762017-09-29 Tim Rühsen <tim.ruehsen@gmx.de> 9377 9378 * tests/libtest.c (wget_test): Flush stderr after each write 9379 93802017-09-29 Tim Rühsen <tim.ruehsen@gmx.de> 9381 9382 * .gitlab-ci.yml: Remove MinGW CI runner 9383 9384 Due to several instabilities / hangs we temporarily remove 9385 the MinGW CI runner. One problem is the buggy behavior of 9386 MinGW pthread_cond_broadcast() that doesn't wake up every thread. 9387 As a result Wget2 sometimes hangs and so does the test suite. 9388 93892017-09-29 Tim Rühsen <tim.ruehsen@gmx.de> 9390 9391 .gitlab-ci.yml: grep posix version of libgcc_s_seh-1.dll 9392 93932017-09-28 Tim Rühsen <tim.ruehsen@gmx.de> 9394 9395 * .gitlab-ci.yml: Remove --enable-static configure flag from MinGW build 9396 93972017-09-27 Tim Rühsen <tim.ruehsen@gmx.de> 9398 9399 * .gitlab-ci.yml: Fix WINEPATH 9400 94012017-09-22 Avinash Sonawane <rootkea@gmail.com> 9402 9403 * tests/libtest.c (_http_server_start): Remove explicit type conversion 94042017-09-27 Tim Rühsen <tim.ruehsen@gmx.de> 9405 9406 * libwget/netrc.c (wget_netrc_db_load): Use wget_strncasecmp_ascii() 9407 94082017-09-27 Tim Rühsen <tim.ruehsen@gmx.de> 9409 9410 Add GNU extensions to .netrc parsing 9411 9412 * include/wget/wget.h (wget_netrc_t): Add port and force 9413 * libwget/netrc.c (wget_netrc_db_load): Parse port and force 9414 94152017-09-22 DalmeGNU <dalmemail@gmail.com> 9416 9417 Replace error_printf() with debug_printf() in order to solve #254 94182017-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 9419 9420 * .gitlab-ci.yml: Test MinGW with -j1 9421 94222017-09-20 Tim Rühsen <tim.ruehsen@gmx.de> 9423 9424 Use type bool instead of char 9425 9426 * include/wget/wget.h: Use type bool instead of char 9427 * libwget/bar.c: Likewise 9428 * libwget/base64.c: Likewise 9429 * libwget/cookie.c: Likewise 9430 * libwget/encoding.c: Likewise 9431 * libwget/hsts.c: Likewise 9432 * libwget/http.h: Likewise 9433 * libwget/http_highlevel.c: Likewise 9434 * libwget/http_parse.c: Likewise 9435 * libwget/ip.c: Likewise 9436 * libwget/iri.c: Likewise 9437 * libwget/logger.c: Likewise 9438 * libwget/net.h: Likewise 9439 * libwget/ocsp.c: Likewise 9440 * libwget/tls_session.c: Likewise 9441 * libwget/vector.c: Likewise 9442 * src/plugin.c: Likewise 9443 * src/wget_host.h: Likewise 9444 * src/wget_job.h: Likewise 9445 * src/wget_plugin.h: Likewise 9446 * tests/test-plugin-dummy.c: Likewise 9447 94482017-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 9449 9450 Fix HPKP checking 9451 9452 * libwget/ssl_gnutls.c (_verify_certificate_callback): PIN check the 9453 whole cert chain. 9454 9455 We used only cert #0 for HPKP checking. This broke connectivity 9456 to github.com. 9457 Now checking the whole cert chain. 9458 9459 Fixes #293 9460 94612017-09-20 Tim Rühsen <tim.ruehsen@gmx.de> 9462 9463 * libwget/thread.c: Do not call pthread functions with NULL thread value 9464 94652017-09-20 Tim Rühsen <tim.ruehsen@gmx.de> 9466 9467 Fix HTTPS functionality on _WIN32 9468 9469 * libwget/ssl_gnutls.c: Add _win32_send() and _win32_recv(), 9470 set those functions as push and pull functions on _WIN32. 9471 9472 Sets the GnuTLS pull and push functions to make gnulib sockets work. 9473 Tested only on MinGW64. 9474 94752017-09-20 Tim Rühsen <tim.ruehsen@gmx.de> 9476 9477 Remove obsolete server code from library and test suite 9478 9479 * include/wget/wget.h: Remove wget_tcp_listen(), wget_tcp_accept(), 9480 wget_ssl_server_init(), wget_ssl_server_deinit(), wget_ssl_server_open(), 9481 wget_ssl_server_close(). 9482 * libwget/net.c: Remove wget_tcp_listen(), wget_tcp_accept(), 9483 remove tcp->passive, 9484 amend wget_tcp_tls_start() and wget_tcp_tls_stop(). 9485 * libwget/net.h: Remove tcp->passive. 9486 * libwget/ssl_gnutls.c: Remove server variables, wget_ssl_server_init(), 9487 wget_ssl_server_deinit(), wget_ssl_server_open(), wget_ssl_server_close(). 9488 * tests/libtest.c: Remove FTP(S) code and obsolete test server code. 9489 * tests/libtest.h: Remove FTP flags, structures, functions. 9490 94912017-09-08 Avinash Sonawane <rootkea@gmail.com> 9492 9493 * doc/wget2_manual.md: Add docs for --stats-* options 9494 94952017-09-17 Avinash Sonawane <rootkea@gmail.com> 9496 9497 Fix --stats-* for chunked download 9498 9499 * src/stats.c (stats_docs_add): For chunked download add chunk sizes to print total size 9500 * src/stats.c (stats_tree_docs_add): Point to doc with non-HEAD method in case of collision 9501 95022017-09-15 Avinash Sonawane <rootkea@gmail.com> 9503 9504 Add tests for --stats-*=format:file 9505 9506 * tests/test-stats.c: test --stats-site=tree:file 9507 * tests/test-stats.c: test --stats-all=csv:file 9508 95092017-09-15 Avinash Sonawane <rootkea@gmail.com> 9510 9511 * src/stats.c: Refactor stats code 9512 95132017-09-14 Avinash Sonawane <rootkea@gmail.com> 9514 9515 Add HTTP status to --stats-site=csv and --stats-site=json 9516 9517 * src/wget_host.h: Add 'status' to struct DOC 9518 * src/stats.c (print_*_site_entry): Print status 9519 95202017-09-14 Avinash Sonawane <rootkea@gmail.com> 9521 9522 Add new format 'tree' for --stats-site 9523 9524 * include/wget/wget.h: Add WGET_STATS_FORMAT_TREE to wget_stats_format_t enum 9525 * src/options.c (parse_stats): Accept format 'tree' for --stats-site switch 9526 * src/stats.c (stats_print_tree): Print tree-ish o/p for format WGET_STATS_FORMAT_TREE 9527 95282017-09-14 Avinash Sonawane <rootkea@gmail.com> 9529 9530 * src/stats.c (stats_print): For --stats-all=csv:foo, save stats in separate files 9531 95322017-09-12 Avinash Sonawane <rootkea@gmail.com> 9533 9534 Add --stats-all switch as a shorthand to turn ON all the stats-* switches 9535 9536 * src/wget_options.h (struct config): Add member 'stats_all' 9537 * src/options.c: Add new function parse_stats_all to parse --stats-all 9538 * tests/test-stats.c (stats_options[]): Add "--stats-all" 9539 * src/stats.c (stats_print): Append to file if --stats-all is provided 9540 95412017-09-11 Avinash Sonawane <rootkea@gmail.com> 9542 9543 Fix --stats-site o/p for --spider 9544 9545 * src/wget_host.h (struct HOST_DOCS): change datatype of 'docs' from wget_vector_t to wget_hashmap_t 9546 * src/stats.c (stats_docs_add): For --spider replace contents of 'doc' with 'resp' of second GET request 9547 95482017-09-10 Avinash Sonawane <rootkea@gmail.com> 9549 9550 Consolidate --stats-site CSV and JSON o/p 9551 9552 * src/stats.c: Consolidate print_csv() and print_json() 9553 * src/stats.c: Consolidate print_site_stats_csv() and print_site_stats_json() 9554 * src/wget_stats.h: Move stats_format_t enum to include/wget/wget.h 9555 95562017-09-09 Avinash Sonawane <rootkea@gmail.com> 9557 9558 Fix --stats-* JSON o/p 9559 9560 * src/wget_host.h: Add 'json_stats' context struct 9561 * src/stats.c: Pass 'ctx' stats_print_*_entry()s 9562 95632017-09-09 Avinash Sonawane <rootkea@gmail.com> 9564 9565 * src/stats.c: Consolidate code from stats_print_*() functions in stats_print() 9566 95672017-09-08 Tim Rühsen <tim.ruehsen@gmx.de> 9568 9569 * src/stats.c (stats_set_option): Fix memory leak 9570 95712017-09-08 Tim Rühsen <tim.ruehsen@gmx.de> 9572 9573 Fix --stats-site hashing 9574 9575 * src/stats.c (_host_docs_hash): Disable integer sanitizer checks, 9576 avoid memory allocation. 9577 (stats_docs_add): Use calloc instead of malloc. 9578 95792017-09-08 Tim Rühsen <tim.ruehsen@gmx.de> 9580 9581 * src/stats.c: More consolidation 9582 95832017-09-08 Tim Rühsen <tim.ruehsen@gmx.de> 9584 9585 Detangle src/hosts.c and src/stats.c 9586 9587 * src/host.c: Move stuff to src/stats.c 9588 * src/options.c (init): Save --stats-site status in config 9589 * src/stats.c: Insert stuff rom src/host.c 9590 * src/wget.c: Cleaup regarding the above changes 9591 * src/wget_host.h: Add structs and prototypes 9592 * src/wget_options.h: Add config.stats_site 9593 * src/wget_stats.h: Add prototypes 9594 * unit-tests/Makefile.am: Link stats.o where needed 9595 95962017-09-07 Avinash Sonawane <rootkea@gmail.com> 9597 9598 Add JSON o/p for --stats-site 9599 9600 * src/wget_host.h: Add print_site_stats_json() prototype declaration 9601 * src/stats.c (stats_print_json): Handle WGET_STATS_TYPE_SITE case 9602 * src/host.c: Add print_site_stats_json(), hosts_hashmap_json() and 9603 print_json() functions 9604 96052017-09-07 Tim Rühsen <tim.ruehsen@gmx.de> 9606 9607 Detangle stats from options.c 9608 9609 * src/options.c (parse_stats): Call stats_set_option() to set values, 9610 (deinit): Call stats_exit() 9611 * src/stats.c: Move stats_opts here, 9612 new functions stats_set_option() and stats_exit() 9613 * src/wget_options.h: Remove stats_format_t, stats_opts_t, stats_opts 9614 * src/wget_stats.h: Add stats_format_t, 9615 add prototypes for stats_exit() and stats_set_option() 9616 96172017-09-07 Tim Rühsen <tim.ruehsen@gmx.de> 9618 9619 * src/stats.c: Cleanup stats output dispatching 9620 96212017-09-07 Tim Rühsen <tim.ruehsen@gmx.de> 9622 9623 * src/stats.c (stats_server_hpkp): Add G_GNUC_WGET_PURE attribute 9624 96252017-09-07 Tim Rühsen <tim.ruehsen@gmx.de> 9626 9627 * src/stats.c: Reduce number of memory allocations 9628 96292017-09-06 Avinash Sonawane <rootkea@gmail.com> 9630 9631 Add CSV o/p for --stats-site 9632 9633 I'll amend this later. 9634 96352017-09-06 Avinash Sonawane <rootkea@gmail.com> 9636 9637 Collect site stats only if --stats-site switch is supplied 9638 9639 * src/wget.c (wget_tcp_set_stats_site): Set 'stats-site' flag 9640 * src/wget.c (add_statistics): Collect Site stats only if `stats_site` is set 9641 * src/stats.c (stats_init): Call wget_tcp_set_stats_site if --stats-site is supplied 9642 * include/wget/wget.h: Add wget_tcp_set_stats_site() prototype declaration 9643 96442017-09-06 Tim Rühsen <tim.ruehsen@gmx.de> 9645 9646 Rearrange stats struct members to avoid padding 9647 9648 * src/wget_stats.h: Likewise 9649 * libwget/ssl_gnutls.c: Likewise 9650 96512017-09-05 Avinash Sonawane <rootkea@gmail.com> 9652 9653 Add docs 9654 9655 * libwget/http.c: Document wget_tcp_set_stats_server() and wget_tcp_get_stats_server() 9656 * libwget/net.c: Document wget_tcp_set_stats_dns() and wget_tcp_get_stats_dns() 9657 * libwget/ssl_gnutls.c: Document wget_tcp_set_stats_tls() and wget_tcp_get_stats_tls() 9658 * libwget/ssl_gnutls.c: Document wget_tcp_set_stats_ocsp() and wget_tcp_get_stats_ocsp() 9659 96602017-09-05 Avinash Sonawane <rootkea@gmail.com> 9661 9662 Add robots.txt as child of first doc having same host as robots.txt 9663 9664 * src/host.c (tree_docs_add): Likewise 9665 * src/host.c (print_treeish): Backtrack once all children have been printed 9666 * src/wget.c: Remove commented code 9667 * scr/wget_job.h: Remove not needed 'cloned_robot_iri' and 'previous_robot_job' 9668 * src/wget_host.h: Remove not needed 'bool robot_iri' 9669 96702017-09-04 Avinash Sonawane <rootkea@gmail.com> 9671 9672 Add robot_iri to blacklist and do not free it explicitly 9673 9674 * src/wget.c (add_url_to_queue): Likewise 9675 * src/wget.c (add_url): Likewise 9676 * src/host.c (host_remove_job): Don't free robot_iri explicitly 9677 96782017-09-04 Avinash Sonawane <rootkea@gmail.com> 9679 9680 * tests/test-stats.c: Add tests for -r with --stats-* options 9681 96822017-09-03 Tim Rühsen <tim.ruehsen@gmx.de> 9683 9684 Add test 'test-stats' for new stats options 9685 9686 * tests/Makefile.am: Add test-stats.c 9687 * tests/test-stats.c: New file 9688 96892017-09-03 Tim Rühsen <tim.ruehsen@gmx.de> 9690 9691 * src/wget.c (downloader_thread): Add missing brackets around else case 9692 96932017-09-03 Tim Rühsen <tim.ruehsen@gmx.de> 9694 9695 * src/stats.c: Reduce # of allocations in stats_print_* functions 9696 96972017-09-03 Tim Rühsen <tim.ruehsen@gmx.de> 9698 9699 * src/host.c (print_encoding): Add 'const' attribute to function 9700 97012017-09-03 Avinash Sonawane <rootkea@gmail.com> 9702 9703 Do not print sizes in human-readable form 9704 9705 * src/host.c (host_docs_hashmap): Likewise 9706 * libwget/http.c (_free_host_entry): Fix parenhesis 9707 * src/options.c: Amend help text to reflect the progress 9708 97092017-09-02 Avinash Sonawane <rootkea@gmail.com> 9710 9711 * src/host.c (print_treeish): Use context variable instead of local static variable 9712 97132017-09-01 Avinash Sonawane <rootkea@gmail.com> 9714 9715 Assign enum values explicitly 9716 9717 * include/wget/wget.h: Likewise 9718 * src/stats.c (stats_print_human): Remove commented code 9719 97202017-09-01 Avinash Sonawane <rootkea@gmail.com> 9721 9722 * src/stats.c (stats_print_csv): Use buffer to print CSV header 9723 97242017-09-01 Avinash Sonawane <rootkea@gmail.com> 9725 9726 Refactor --stats-tls code 9727 9728 * src/wget_stats.h: Change datatype of 'tls_con' and 'resumed' from char to bool 9729 * libwget/ssl_gnutls.c (stats_data_t struct): Likewise 9730 * src/stats.c (stats_print_*): Use WGET_HTTP_PROTOCOL_* instead of hard-coded values 9731 97322017-09-01 Avinash Sonawane <rootkea@gmail.com> 9733 9734 Add port to DNS stats 9735 9736 * include/wget/wget.h: Add WGET_STAST_DNS_PORT to wget_stats_dns_t enum 9737 * libwget/netc. (wget_tcp_resolve): store port in _stats_data_t struct 9738 * src/wget_stats.h: Add 'port' to dns_stats_t struct 9739 * src/stats.c (stats_callback): Collect port into dns_stats_t struct 9740 * src/stats.c (stats_print_*): Print port 9741 97422017-08-29 Avinash Sonawane <rootkea@gmail.com> 9743 9744 * libwget/http.c: Suppress clang warning about integer overflow 9745 97462017-08-23 Avinash Sonawane <rootkea@gmail.com> 9747 9748 Add parent child relationships and print them in tree-ish form 9749 9750 * include/wget/wget.h: Add prototype for wget_vector_deinit() 9751 * libwget/vector.c: Add wget_vector_deinit() to free 'v->entry' 9752 (can be used for vector maintaining references) 9753 * src/host.c: Refactor tree_docs_add() 9754 * src/host.c: Add hosts_hashmap_tree() and print_treeish() to print tree-ish o/p 9755 * wget/host.h: Add 'root' and 'robot' to HOST struct 9756 * src/wget_job.h (JOB struct): Add 'previous_robot_job' and 'clonned_robot_iri' 9757 97582017-08-20 Avinash Sonawane <rootkea@gmail.com> 9759 9760 Add data structures and functions for tree-ish o/p 9761 9762 * src/wget_host.h: Add tree_docs hashmap to HOST struct 9763 * src/wget_host.h: Add TREE_DOCS struct 9764 * src/host.c: Add _tree_docs_hash() as hash function for 'tree_docs' hashmap 9765 * src/host.c: Add _tree_docs_compare() as compare function for 'tree_docs' hashmap 9766 * src/host.c: Add _free_tree_docs_entry() as destructor for 'tree_docs' hashmap 9767 * src/host.c: (tree_docs_add): Add entry to 'tree_docs' hashmap 9768 * src/host.c: (tree_docs_get): Get entry from 'tree_docs' hashmap 9769 97702017-08-17 Avinash Sonawane <rootkea@gmail.com> 9771 9772 Add doc_size and content-encoding to --stats-site 9773 9774 * src/wget_host.h (DOC struct): Add size_downloaded, size_decompressed and encoding 9775 * src/wget.c (_prepare_file): Simplify code 9776 * src/options.c: Amend --stats-* help text 9777 * src/host.c (host_docs_add): Collect size_downloded, size_decompressed and encoding 9778 97792017-08-16 Avinash Sonawane <rootkea@gmail.com> 9780 9781 * src/host.c: Suppress clang warning about integer overflow 9782 97832017-07-31 Avinash Sonawane <rootkea@gmail.com> 9784 9785 Add --stats-site switch 9786 9787 * include/wget/wget.h: Add WGET_STATS_TYPE_SITE to wget_stats_type_t enum 9788 * src/options.h: Increase stats_opts[] size to 5 9789 * src/options.c: Add --stats-site 9790 * src/wget_host.h: Add 'robot_iri' to DOC struct 9791 * src/host.c: Add new functions print_site_stats(), hosts_hashmap() and host_docs_hashmap() 9792 * src/stats.c (stats_print_human): Print site stats 9793 * src/wget.c: Call host_add() as soon as scheme changes due to HSTS 9794 * src/wget.c: Clone robot_iri 9795 * src/wget_stats.h: Fix indentation 9796 97972017-07-31 Avinash Sonawane <rootkea@gmail.com> 9798 9799 Assign enum values explicitly 9800 9801 * src/wget_options.h (stats_format_t enum): Likewise 9802 * src/options.c (help_section_t enum): Likewise 9803 * libwget/css_tokenizer.h: Likewise 9804 * libwget/bar.c (_bar_slot_status_t enum): likewise 9805 * src/host.c (host_docs_get): Initialize 'host_docsp' 9806 * src/wget.c (add_statistics): Add call to host_docs_add() 9807 98082017-07-30 Avinash Sonawane <rootkea@gmail.com> 9809 9810 Add WGET_STATS_HPKP_ERROR to handle HPKP error 9811 9812 * include/wget/wget.h: Add WGET_STATS_HPKP_ERROR to wget_hpkp_stats_t enum 9813 * src/wget_stats.h (server_stats_t struct): Change datatype of 'hpkp' from char to wget_hpkp_stats_t 9814 * libwget/ssl_gnutls.c (_cert_verify_hpkp): Store WGET_STATS_HPKP_ERROR for error 9815 * src/stats.c (stats_server_hpkp): Return appropriate message depending on 'hpkp' value 9816 98172017-07-28 Avinash Sonawane <rootkea@gmail.com> 9818 9819 Collect --stats-server for unique servers(IPs) only 9820 9821 * src/wget_stats.h: Add member 'scheme' to server_stats_t struct 9822 * src/stats.c (stats_callback): Collect 'scheme' in server_stats_t struct 9823 * src/stats.c (stats_print_*): Print 'scheme' 9824 * include/wget/wget.h: Add WGET_STATS_SERVER_SCHEME to wget_server_stats_t enum 9825 * libwget/hashmap.c (wget_hashmap_contains): Add check for 'h' being non-NULL 9826 * libwget/http.c: Add _host_hash() as hash function for 'hosts' hashmap 9827 * libwget/http.c: Add _host_compare() as compare function for 'hosts' hashmap 9828 * libwget/http.c: Add _free_host_entry() destructor for 'hosts' hashmap 9829 * libwget/http.c (host_add): Add host to 'hosts' hashmap 9830 98312017-07-28 Avinash Sonawane <rootkea@gmail.com> 9832 9833 * include/wget/wget.h: Set explicit enum values 9834 98352017-07-28 Avinash Sonawane <rootkea@gmail.com> 9836 9837 Add IP to --stats-server 9838 9839 * src/wget_stats.h: Add 'ip' to 'server_stats_t' struct 9840 * src/stats.c (stats_callback): Collect IP 9841 * src/stats.c (stats_print_*): Print IP 9842 * src/stats.c (free_server_stats): Free 'ip' 9843 * libwget/net.h (wget_tcp_st struct): Add 'ip' 9844 * libwget/net.c (wget_tcp_connect): Store IP in tcp->ip 9845 * libwget/http.c: Collect IP from conn->tcp->ip 9846 98472017-07-27 Avinash Sonawane <rootkea@gmail.com> 9848 9849 * src/options.c: Amend the help text for --stats-* options 9850 98512017-07-26 Avinash Sonawane <rootkea@gmail.com> 9852 9853 Collect site-based stats 9854 9855 * src/wget_host.h: Add HOST_DOCS and DOC structs 9856 * src/host.c (_host_docs_compare): Add compare function for host_docs hashmap 9857 * src/host.c (_host_docs_hash): Add hash function for host_docs hashmap 9858 * src/host.c (_free_host_docs_entry): Add destructor for host_docs hashmap 9859 * src/host.c (host_docs_add): Add function to add stats in host_docs hashmap 9860 * src/host.c (host_docs_get): Add function to retrieve doc from host_docs hashmap 9861 * src/wget.c (process_response_header): Add call to host_docs_add() 9862 98632017-07-21 Avinash Sonawane <rootkea@gmail.com> 9864 9865 Assign default value if wget_tcp_get_stats_* return NULL 9866 9867 * src/stats.c (stats_callback): Likewise 9868 * src/wget_stats.h: Change datatype of cert_chain_size, nvalid, 9869 ignored, nrevoked from size_t to int 9870 * src/wget_stats.h: Change datatype of hpkp from wget_hpkp_stats_t 9871 to char 9872 * src/options.c (parse_stats): Exit if unknown stats format 9873 * libwget/net.c (wget_tcp_resolve): Initialize 'before_millisecs' 9874 * libwget/ssl_gnutls.c: Add function definitions for wget_tcp_*_stats_ocsp() 9875 98762017-07-20 Avinash Sonawane <rootkea@gmail.com> 9877 9878 Add support for --stats-*[=[format:]filename] 9879 9880 * src/options.c: Add parse_stats() function to parse --stats-*[=[format:]filename] 9881 * src/wget_options.h: Add stats_format_t enum and stats_opts_t struct 9882 * src/stats.c: Add stats_print_human() for "human" formatted output 9883 9884 3 case-insensitive formats: json, csv and human (default) 9885 'stdout' is the default file. '-' stands for 'stdout'. 9886 98872017-07-12 Avinash Sonawane <rootkea@gmail.com> 9888 9889 * src/stats.c: Fix --stats-server and --stats-ocsp JSON output 9890 98912017-07-12 Avinash Sonawane <rootkea@gmail.com> 9892 9893 Add flag tls_stats to separate --stats-tls from --stats-ocsp 9894 9895 * libwget/ssl_gnutls.c: Likewise 9896 * src/http.c (wget_http_open): Add check to see whether stats_callback is non-NULL 9897 98982017-07-12 Avinash Sonawane <rootkea@gmail.com> 9899 9900 * src/stats.c (free_server_stats): free server_stats_t members 9901 99022017-07-11 Avinash Sonawane <rootkea@gmail.com> 9903 9904 Print Server stats and OCSP stats in CSV and JSON format 9905 9906 * src/stats.c: Likewise 9907 * src/wget_stats.h: Remove stats_print*() function declarations 9908 99092017-07-11 Avinash Sonawane <rootkea@gmail.com> 9910 9911 Print --stats-server, if asked, for HPKP_NOMATCH 9912 9913 * libwget/http.c (wget_http_open): Call stats_callback() even for WGET_E_CERTIFICATE, if asked 9914 * src/wget_stats.h (server_stats_t struct): Fix indentation 9915 * src/stats.c (stats_print): Fix info_printfs() for server stats 9916 99172017-07-10 Avinash Sonawane <rootkea@gmail.com> 9918 9919 Add HPKP_NEW to --stats-server 9920 9921 * src/wget_stats.h: Add hpkp_new to server_stats_t struct 9922 * src/stats.c (stats_callback): Collect hpkp_new 9923 * src/stats.c (stats_print): Print hpkp_new 9924 * libwget/http.c (wget_http_get_response_cb): Collect hpkp_new stats 9925 * include/wget/wget.h: Add WGET_STATS_SERVER_HPKP_NEW to wget_server_stats_t enum 9926 99272017-07-10 Avinash Sonawane <rootkea@gmail.com> 9928 9929 Add IGNORED to --stats-ocsp 9930 9931 * libwget/ssl_gnutls.c (_verify_certificate_callback): Add nignored to stats 9932 * src/stats.c (stats_callback): Collect nignored 9933 * src/stast.c (stats_print): Print nignored 9934 99352017-07-08 Avinash Sonawane <rootkea@gmail.com> 9936 9937 Add HPKP stats and hostname to --stats-server 9938 9939 * libwget/ssl_gnutls.c: Collect HPKP stats in tcp struct via session_context 9940 * libwget/http.c: Collect hostname and HPKP stats in _stats_data_t struct 9941 * src/stats.c: Collect hostname in stats_callback() and print it 9942 99432017-07-08 Avinash Sonawane <rootkea@gmail.com> 9944 9945 Add --stats-ocsp option 9946 9947 * src/wget_stats.h: Add ocsp_stats_t struct 9948 * src/stats.c: Add ocsp_stats_v vector to store OCSP stats 9949 * src/stats.c: Add OCSP stats code in stats_init(), stats_callback() and stats_print() 9950 * src/wget_options.h: Add stats_ocsp to struct config 9951 * src/options.c: Add --stats-ocsp 9952 * include/wget/wget.h: Add wget_ocsp_stats_t enum 9953 * include/wget/wget.h: Add WGET_STATS_TYPE_OCSP to wget_stats_type_t enum 9954 * libwget/ssl_gnutls.c: Add wget_tcp_set_stats_ocsp() and wget_tcp_get_stats_ocsp() 9955 99562017-07-07 Avinash Sonawane <rootkea@gmail.com> 9957 9958 Print stats in CSV and JSON format 9959 9960 * src/wget_stats.h: Add prototypes for stats_printcsv() and stats_printjson() 9961 * src/stats.c: Add stats_printcsv() and stats_printjson() functions 9962 * libwget/ssl_gnutls.c: Change datatype of cert_chain_size from unsigned int to size_t 9963 99642017-07-06 Avinash Sonawane <rootkea@gmail.com> 9965 9966 Add --stats-server option 9967 9968 * src/wget_stats.h: Add server_stats_t struct 9969 * src/stats.c: Add server_stats_v vector to store server stats 9970 * src/stats.c: Add server stats code in stats_init(), stats_callback and stats_print() 9971 * src/stats.c: Add free_server_stats() destructor 9972 * src/wget_options.h: Add stats_server to struct config 9973 * src/options.c: Add --stats-server 9974 * include/wget/wget.h: Add wget_server_stats_t and wget_hpkp_stats_t enums 9975 * include/wget/wget.h: Add WGET_STATS_TYPE_SERVER to wget_stats_type_t enum 9976 * libwget/http.c: Add wget_tcp_set_stats_server() and wget_tcp_get_stats_server() 9977 * libwget/http_parse.c: Check for "Content-Security-Policy" header 9978 * libwget/ssl_gnutls.c: Collect HPKP stats in wget_tcp_t struct 9979 * libwget/net.h: Add hpkp to wget_tcp_st struct 9980 99812017-06-16 Avinash Sonawane <rootkea@gmail.com> 9982 9983 Add --stats-dns and --stats-tls options 9984 9985 * src/wget_stats.h: Add dns_stats_t and tls_stats_t structs 9986 * src/stats.c: Add stats_init(), stats_callback() and stats_print() functions 9987 * src/stats.c: Add dns_stats_v and tls_stats_v vectors 9988 * src/stats.c: Add free_tls_stats() and free_tls_stats() functions 9989 * src/wget_options.h: Add stats_dns and stats_tls in config struct 9990 * src/options.c: Add --stats-dns and --stats-tls 9991 * src/wget.c: Add call to stats_init() and stats_print() functions 9992 * src/Makefile.am: Add stats.c and wget_stats.h files 9993 * include/wget/wget.h: Add prototypes for wget_tcp_set_stats_dns() and wget_tcp_set_stats_tls() 9994 * include/wget/wget.h: Add prototypes for wget_tcp_get_stats_dns() and wget_tcp_get_stats_tls() 9995 * include/wget/wget.h: Add wget_stats_type_t, wget_dns_stats_t and wget_tls_stats_t enums 9996 * libwget/ssl_gnutls.c: Add wget_tcp_set_stats_tls() and wget_tcp_get_stats_tls() 9997 * libwget/net.h: Add TLS stats in wget_tcp_st struct 9998 * libwget/net.c: Add wget_tcp_set_stats_dns() and wget_tcp_get_stats_dns() 9999 100002017-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 10001 10002 * tests/libtest.c (_answer_to_connection): Fix memleaks found by coverity 10003 100042017-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 10005 10006 * libwget/net.c: Fix warnings on _WIN32 10007 100082017-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 10009 10010 * src/log.c: Fix unused param warning on _WIN32 10011 100122017-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 10013 10014 * libwget/ssl_gnutls.c (_print_info): Remove call to deprecated GnuTLS function 10015 100162017-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 10017 10018 Change __WIN32 to _WIN32 10019 10020 * tests/test-idn-cmd.c: Likewise 10021 * tests/test-idn-robots.c: Likewise 10022 100232017-09-18 Darshit Shah <darnir@gnu.org> 10024 10025 * tests/libtest.h: Add WGET_TEST_FEATURE_FTP to satisfy clang 10026 100272017-09-18 Darshit Shah <darnir@gnu.org> 10028 10029 Revert "* libwget/net.c: Remove port variable from DNS cache" 10030 10031 This reverts commit b9006672dcbbffb3995a78918d04bfe009f29e2d. 10032 We cannot remove the port variable from the DNS cache. The struct 10033 addrinfo structure contains the port information and hence causes issues 10034 during HTTP->HTTPS redirects 10035 100362017-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 10037 10038 * tests/libtest.c: Cleanups 10039 100402017-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 10041 10042 Fix configure ans Makefile.am for MHD 10043 10044 * Makefile.am: Add tests to SUBDIRS only with MHD 10045 * configure.ac: Fix MHD detection 10046 100472017-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 10048 10049 Fix two IDN tests for MinGW/Windows 10050 10051 * tests/test-idn-cmd.c: Skip UTF-8 on command line 10052 * tests/test-idn-robots.c: Likewise 10053 10054 MinGW mangles UTF-8 characters on the command line, so we skip 10055 those tests. 10056 100572017-09-17 Tim Rühsen <tim.ruehsen@gmx.de> 10058 10059 Add Wget2 dependencies to mingw script 10060 100612017-09-16 Darshit Shah <darnir@gnu.org> 10062 10063 * libwget/net.c: Remove port variable from DNS cache 10064 100652017-09-17 Darshit Shah <darnir@gnu.org> 10066 10067 Use Feature Flags to SKIP unsupported tests 10068 10069 * tests/libtest.{c,h}: Add support for feature flags during test 10070 startup. Check if Wget2 is compiled with the feature and conditionally 10071 SKIP the tests 10072 * tests/test-*.c: Pass the right feature flags for each test. MHD for 10073 HTTP, FTP for FTP, IDN and PLUGIN when applicable 10074 100752017-09-17 Darshit Shah <darnir@gnu.org> 10076 10077 * configure.ac: Support compiling without MHD support 10078 100792017-09-18 Darshit Shah <darnir@gnu.org> 10080 10081 * .gitlab-ci.yml: The Fedora runner should use GCC 10082 100832017-09-18 Darshit Shah <darnir@gnu.org> 10084 10085 * .gitlab-ci.yml: Use new MingW Docker Image 10086 100872017-09-16 Tim Rühsen <tim.ruehsen@gmx.de> 10088 10089 * .gitlab-ci.yml: Adjust WINEPATH, remove CXX 10090 100912017-09-16 Tim Rühsen <tim.ruehsen@gmx.de> 10092 10093 * libwget/metalink.c (_add_file_hash): Skip spaces at begin of mirror URL 10094 100952017-09-16 Darshit Shah <darnir@gnu.org> 10096 10097 Fix bug with combination of host-directories and cut-dirs 10098 10099 * libwget/iri.c(wget_iri_get_path): Change API so that a path separator 10100 is added _only_ if one does not already exist at the end of the buffer. 10101 This should be a non-breaking change. 10102 * src/wget.c(get_local_filename): Append a "/" at the end of the 10103 hostname, always. 10104 101052017-09-16 Darshit Shah <darnir@gnu.org> 10106 10107 * tests/test-cut-dir: Add two new tests with host directories 10108 101092017-09-13 Tim Rühsen <tim.ruehsen@gmx.de> 10110 10111 * libwget/buffer_printf.c (wget_buffer_vprintf_append): Reset precision after use 10112 101132017-09-08 Didik Setiawan <ds@didiksetiawan.com> 10114 10115 Fix socket handle issue on MinGW64 build 10116 10117 Helped-by: Akash Rawal 10118 Helped-by: Tim Rühsen 10119 101202017-09-12 Tim Rühsen <tim.ruehsen@gmx.de> 10121 10122 * tests/libtest.c (_http_server_start): Do not use real randomness 10123 101242017-09-12 Tim Rühsen <tim.ruehsen@gmx.de> 10125 10126 * .gitlab-ci.yml: Adjust build flags 10127 101282017-09-12 Tim Rühsen <tim.ruehsen@gmx.de> 10129 10130 * contrib/mingw: Add /usr/x86_64-w64-mingw32/lib/pkgconfig to PKG_CONFIG_PATH 10131 101322017-08-17 Didik Setiawan <ds@didiksetiawan.com> 10133 10134 Add digest authentication test 10135 10136 * tests/Makefile.am: Add test 'test-auth-digest' 10137 * tests/libtest.c: Fix HTTP server to handle digest 10138 authentication 10139 * tests/libtest.h: Add opaque for digest authentication 10140 * tests/test-auth-digest.c: Add new test to prove digest 10141 authentication functionality 10142 101432017-09-11 Darshit Shah <darnir@gnu.org> 10144 10145 * configure.ac: MHD is used only in tests. Don't link to Wget2 10146 101472017-09-12 Akash Rawal <argmlac@gmail.com> 10148 10149 * .gitlab-ci.yml (MinGW64): Remove hardcoded path to wget2 clone 10150 101512017-09-12 Tim Rühsen <tim.ruehsen@gmx.de> 10152 10153 * contrib/mingw: Include building libmicrohttpd 10154 101552017-09-11 Tim Rühsen <tim.ruehsen@gmx.de> 10156 10157 * .gitlab-ci.yml: Remove MHD clone & bootstrap (now in docker build) 10158 101592017-09-11 Tim Rühsen <tim.ruehsen@gmx.de> 10160 10161 * .gitlab-ci.yml: Remove MHD install, moved into docker build 10162 101632017-09-11 Tim Rühsen <tim.ruehsen@gmx.de> 10164 10165 Use gnulib-tool.py to speed up ./bootstrap 10166 10167 * README.md: Add python to optional dependencies 10168 * bootstrap: Use gnulib-tool.py if possible 10169 * gnulib: Update 10170 10171 gnulib-tool.py is much faster than the old gnulib-tool. 10172 101732017-09-10 Avinash Sonawane <rootkea@gmail.com> 10174 10175 * src/wget.c: If not -r then don't send second GET request for --spider 10176 101772017-09-10 Didik Setiawan <ds@didiksetiawan.com> 10178 10179 Fix Clang warnings on MHD code 10180 10181 Reported-by: Avinash Sonawane 10182 101832017-09-05 Darshit Shah <darnir@gnu.org> 10184 10185 * .gitlab-ci.yaml: Set default CFLAGS to retain the human readable symbols 10186 101872017-09-05 Darshit Shah <darnir@gnu.org> 10188 10189 Prevent heap-use-after-free of default_challenges 10190 10191 * src/wget_job.h: Add new field to JOB struct, challenges_alloc. This 10192 field indicates whether the JOB object owns the challenges struct. 10193 * src/job.c(job_free): Free the challenges struct of the job only if it 10194 is owned. Else, let the owner handle freeing it. 10195 * src/wget.c: Mark the challenges vector as owned when it is taken from 10196 the response. If it is the default vector, it is not owned. 10197 101982017-09-03 Didik Setiawan <ds@didiksetiawan.com> 10199 10200 Remove redundant debugging option 10201 10202 * tests/test--post-file.c: Remove redundant debugging option 10203 * tests/test-auth-basic.c: Likewise 10204 * tests/test-metalink.c: Likewise 10205 * tests/test-redirection.c: Likewise 10206 * tests/test-wget-1.c: Likewise 10207 10208 Debugging option already provided in tests/libtest.c, so we don't need 10209 to adding it in each test file 10210 102112017-09-03 Didik Setiawan <ds@didiksetiawan.com> 10212 10213 Fix expected error code regarding wrong authentication test 10214 10215 * tests/test-auth-basic.c: Fix expected error code regarding wrong 10216 authentication test which use basic scheme 10217 102182017-09-03 Didik Setiawan <ds@didiksetiawan.com> 10219 10220 * test/libtest.c: Remove unnecessary old http server code 10221 102222017-09-03 Didik Setiawan <ds@didiksetiawan.com> 10223 10224 * test/libtest.c: Ensure dinfo variable to be initialized before usage 10225 102262017-08-23 Didik Setiawan <ds@didiksetiawan.com> 10227 10228 * tests/libtest.c: Add server info into HTTP header 10229 102302017-08-04 Didik Setiawan <ds@didiksetiawan.com> 10231 10232 Fix HTTP server to handle chunked transfer encoding 10233 10234 * tests/libtest.c: Fix HTTP server to handle chunked 10235 transfer encoding 10236 * tests/test-bad-chunk.c: Modify test to conform with new 10237 server 10238 * tests/test-chunked.c: Likewise 10239 102402017-07-19 Didik Setiawan <ds@didiksetiawan.com> 10241 10242 * tests/libtest.c: Fix -Wunused-param caused by MHD API 10243 102442017-07-14 Didik Setiawan <ds@didiksetiawan.com> 10245 10246 * tests/libtest.c: Add HTTPS server handler 10247 102482017-07-13 Didik Setiawan <ds@didiksetiawan.com> 10249 10250 Fix HTTP server to handle basic authentication 10251 10252 * tests/libtest.c: Add basic authentication test handler 10253 * tests/test-auth-basic.c: Add test for wrong credentials 10254 102552017-07-11 Didik Setiawan <ds@didiksetiawan.com> 10256 10257 Fix HTTP server response to handle 302 redirection 10258 10259 * tests/libtest.c: Fix HTTP server response to handle 302 10260 status code 10261 102622017-07-10 Didik Setiawan <ds@didiksetiawan.com> 10263 10264 Fix HTTP server response to handle 404 Not Found 10265 10266 * tests/libtest.c: Fix HTTP server response to handle 404 10267 Not Found status code with non-empty body 10268 102692017-07-09 Didik Setiawan <ds@didiksetiawan.com> 10270 10271 Fix HTTP server response to handle Byte serving 10272 10273 * tests/libtest.c: Fix HTTP server response to handle Byte 10274 serving 10275 102762017-07-05 Didik Setiawan <ds@didiksetiawan.com> 10277 10278 Fix HTTP server response to handle If-Modified-Since 10279 10280 * tests/libtest.c: Fix HTTP server response to handle 10281 If-Modified-Since 10282 102832017-06-25 Didik Setiawan <ds@didiksetiawan.com> 10284 10285 Fix requested URL query string which contains space 10286 10287 * tests/libtest.c: Fix requested URL query string which 10288 contains space 10289 102902017-06-23 Didik Setiawan <ds@didiksetiawan.com> 10291 10292 Fix empty requested URL regarding IDN hostname 10293 10294 * tests/libtest.c: Fix empty requested URL by redirect 10295 default page to index.html regarding IDN hostname 10296 102972017-06-22 Didik Setiawan <ds@didiksetiawan.com> 10298 10299 Fix empty requested URL 10300 10301 * tests/libtest.c: Fix empty requested URL by redirect 10302 default page to index.html 10303 103042017-06-21 Didik Setiawan <ds@didiksetiawan.com> 10305 10306 Fix remote URL to handle IRI object 10307 10308 * tests/libtest.c: Fix remote URL which contains IRI object 10309 using Libmicrohttpd function (MHD_http_unescape) 10310 103112017-06-18 Didik Setiawan <ds@didiksetiawan.com> 10312 10313 Fix HTTP server response to handle directory creation 10314 10315 * tests/libtest.c: Fix HTTP server response regarding 10316 directory creation on recursive download 10317 103182017-06-18 Didik Setiawan <ds@didiksetiawan.com> 10319 10320 Fix HTTP server response to include available header 10321 10322 * tests/libtest.c: Fix HTTP server response to include 10323 available header. This solved many cases which involving 10324 proper HTTP header response such as for Chunked 10325 "Transfer-Encoding" and "Redirection" 10326 103272017-06-17 Didik Setiawan <ds@didiksetiawan.com> 10328 10329 Fix HTTP server to get available port 10330 10331 * tests/libtest.c: Automatically get available port for HTTP 10332 server using Libmicrohttpd function 10333 103342017-06-16 Didik Setiawan <ds@didiksetiawan.com> 10335 10336 Fix requested URL which contains query string 10337 10338 * tests/libtest.c: Fix requested URL which contains 10339 query string 10340 103412017-06-14 Didik Setiawan <ds@didiksetiawan.com> 10342 10343 Fix HTTP server response to handle 404 Not Found 10344 10345 * tests/libtest.c: Fix HTTP server response to handle 404 10346 Not Found status code with empty body 10347 103482017-06-13 Didik Setiawan <ds@didiksetiawan.com> 10349 10350 * tests/libtest.c: Create Libmicrohttpd function wrapper 10351 103522017-06-05 Didik Setiawan <ds@didiksetiawan.com> 10353 10354 Initial system preparation for Libmicrohttpd integration 10355 10356 * .gitlab-ci.yml: Add Libmicrohttpd package for CI testing 10357 using Debian/gcc, Fedora/clang and MinGW64 10358 * .travis.sh: Add Libmicrohttpd package for OSX/clang 10359 * .travis_setup.sh: Add Libmicrohttpd package for Debian/gcc 10360 and clang 10361 * README.md: Add info which mentioning Libmicrohttpd as 10362 optional requirement 10363 * configure.ac: Check for Libmicrohttpd availability 10364 103652017-09-06 Tim Rühsen <tim.ruehsen@gmx.de> 10366 10367 * docs/wget2_manual.md: Fix several small things 10368 103692017-09-06 Tim Rühsen <tim.ruehsen@gmx.de> 10370 10371 Remove redundant pointer syntax from function pointer calls 10372 10373 * libwget/hsts.c: Likewise 10374 * libwget/ocsp.c: Likewise 10375 * libwget/plugin.c: Likewise 10376 * src/plugin.c: Likewise 10377 103782017-09-06 Tim Rühsen <tim.ruehsen@gmx.de> 10379 10380 * src/plugin.c (plugin_db_finalize): Check if initialized 10381 103822017-09-05 Tim Rühsen <tim.ruehsen@gmx.de> 10383 10384 README.md: document clang not working with 'make check-coverage' 10385 103862017-09-04 Akash Rawal <argmlac@gmail.com> 10387 10388 Fix clang warnings 10389 10390 * include/wget/wget.h: Add new function wget_ssl_set_config_object() 10391 * libwget/ssl_gnutls.c: Likewise 10392 * src/options.c: Use wget_ssl_set_config_object() 10393 * libwget/ssl_gnutls.c (wget_ssl_write_timeout): Remove unreachable code 10394 not required by syntax 10395 * src/wget.c (downloader_thread): Likewise 10396 * unit-tests/test.c (test_hpkp): Remove unnecessary variable with same 10397 name 10398 103992017-09-03 Akash Rawal <argmlac@gmail.com> 10400 10401 Add plugin APIs for custom HSTS, HPKP and OCSP databases 10402 10403 * include/wget/wget.h: Add plugin API declarations 10404 * libwget/hsts.c: Make wget_hsts_db_t extensible 10405 * libwget/hpkp.c: Make wget_hpkp_db_t extensible 10406 * libwget/ocsp.c: Make wget_ocsp_db_t extensible 10407 * libwget/plugin.c: Add virtual functions for plugin API 10408 * src/plugin.c: Add plugin API implementation 10409 * src/wget_plugin.h: Likewise 10410 * src/option.c: Add code to call the implementation 10411 * src/wget.c: Likewise 10412 * tests/test-plugin-dummy: Add plugin with dummy databases for testing 10413 * tests/Makefile.am: Add target libplugindb.la 10414 * tests/test-plugin.c: Add tests for custom databases 10415 * unit-tests/test.c: Add tests for new HPKP functions 10416 * libwget/ssl_gnutls.c: Adapt to API changes 10417 * examples/http_multi_get.c: Likewise 10418 * fuzz/libwget_hpkp_fuzzer.c: Likewise 10419 * fuzz/libwget_hsts_fuzzer.c: Likewise 10420 * fuzz/libwget_ocsp_fuzzer.c: Likewise 10421 10422 This feature implemented as per the specifications at 10423 https://gitlab.com/akash_rawal/wget2/wikis/Stage-4 10424 104252017-09-03 Akash Rawal <argmlac@gmail.com> 10426 10427 Add plugin APIs for intercepting downloaded files. 10428 10429 Merged from branch tmp-akash_rawal-stage3 into master 10430 10431 * include/wget/wget.h: Add function declarations 10432 * libwget/plugin.c: Add virtual functions 10433 * configure.ac: Check for fmemopen() 10434 * src/plugin.c: Add implementation 10435 * src/wget_plugin.h: Likewise 10436 * src/wget.c: Add code that calls the implementation 10437 * tests/test-plugin-dummy.c (pluginapi): Add option 'parse-rot13', 10438 'only-rot13' and 'test-pp' 10439 * tests/test-plugin.c: Add tests 10440 * tests/libtest.c: Load files into dynamic memory instead of stack, 10441 enables testing with large files. 10442 10443 Plugin APIs for intercepting downloaded files have been implemented 10444 as per specifications at https://gitlab.com/akash_rawal/wget2/wikis/Stage-3 10445 104462017-09-02 Tim Rühsen <tim.ruehsen@gmx.de> 10447 10448 Include sys/uio.h for writev() 10449 10450 * configure.ac: Check for sys/uio.h 10451 * libwget/ssl_gnutls.c: Include sys/uio.h for writev() if available 10452 10453 Fixes #260 10454 Reported-by: Gabriele Balducci 10455 10456 104572017-09-01 Darshit Shah <darnir@gnu.org> 10458 10459 Call strerror() to get Error message with some errors 10460 104612017-09-01 Darshit Shah <darnir@gnu.org> 10462 10463 * libwget/http.c(wget_http_send_request): Fix printf specifier data type 10464 104652017-08-31 Darshit Shah <darnir@gnu.org> 10466 10467 Remove magic numbers when setting exit codes 10468 10469 Magic numbers are bad, they make the code very difficult to read since 10470 one must guess their intention. Which is why this commit eliminates all 10471 the magic numbers for setting the exit status and replaces them with a 10472 structured enum. This has the added benefit of having these values as 10473 symbols in the final symbol table. 10474 10475 * src/wget.c: Replace all calls to set_exit_status() to use the 10476 exit_status_t enum values 10477 * src/wget_main.h: Define the exit_status_t enum and associated exit 10478 error code values 10479 104802017-08-31 Darshit Shah <darnir@gnu.org> 10481 10482 * src/wget.c(process_response_header): Remove dead (commented) code 10483 104842017-08-31 Darshit Shah <darnir@gnu.org> 10485 10486 * src/wget.c(process_response_header): Clean up auth failure pathways 10487 104882017-08-31 Avinash Sonawane <rootkea@gmail.com> 10489 10490 Do not print request body if it's an OCSP request 10491 10492 * include/wget/wget.h: Add 'debug_skip_body' flag to 'wget_http_request_t' 10493 * libwget/ssl_gnutls.c: set 'debug_skip_body' flag if it's an OCSP request 10494 * libwget/http.c: Do not print 'req->body' if 'debug_skip_body' is set 10495 104962017-08-31 Avinash Sonawane <rootkea@gmail.com> 10497 10498 * src/wget.c: Fix regression of having extra '/' in filename 10499 105002017-08-23 Darshit Shah <darnir@gnu.org> 10501 10502 New benchmark script 10503 105042017-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 10505 10506 Improve fuzzer helper scripts 10507 10508 * fuzz/get_ossfuzz_corpora: Improve description text 10509 * fuzz/view-coverage.sh: Allow 1+ arguments 10510 105112017-08-23 Darshit Shah <darnir@gnu.org> 10512 10513 Add counter for authorization failures 10514 10515 Fixes #259 10516 105172017-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 10518 10519 Use clang for sanitizer CI 10520 10521 * .gitlab-ci.yml: Use clang for sanitizing, remove memory sanitizer 10522 * configure.ac: Do not combine leak and memory sanitizer for clang 10523 10524 Memory sanitizer only makes sense when all dependencies are built with it. 10525 Else we will see many false positives. 10526 105272017-08-23 Tim Rühsen <tim.ruehsen@gmx.de> 10528 10529 Update gnulib 10530 105312017-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 10532 10533 * libwget/ssl_gnutls.c (_do_handshake): Entangle code 10534 105352017-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 10536 10537 * fuzz/README.md: Add sanitizer options 10538 105392017-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 10540 10541 Update gnulib 10542 105432017-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 10544 10545 Fix 'deflate' content-type decompression 10546 10547 * ../libwget/decompressor.c (deflate_init): Fix deflate initialization 10548 10549 Fixes: #251 10550 Reported-by: Avinash Sonawane 10551 105522017-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 10553 10554 Fix possible buffer read underflow 10555 10556 * libwget/cookie.c (_path_match): Fix underflow 10557 10558 Fixes: #253 10559 Reported-by: Avinash Sonawane 10560 105612017-08-15 Avinash Sonawane <rootkea@gmail.com> 10562 10563 * Suppress clang warning about integer overflow 10564 10565 * src/host.c: Likewise 10566 * src/blacklist.c: Likewise 10567 * src/wget.c: Likewise 10568 105692017-08-15 Avinash Sonawane <rootkea@gmail.com> 10570 10571 * libwget/utils.c: Fix unsigned integer overflow 10572 10573 * wget_match_tail: Likewise 10574 * wget_match_tail_nocase: Likewise 10575 10576 Found by fuzzing 10577 105782017-08-13 Didik Setiawan <ds@didiksetiawan.com> 10579 10580 Fix DATADIR clash when build using MinGW64 10581 10582 * tests/Makefile.am: Change DATADIR to WGET_DATADIR 10583 * unit-tests/Makefile.am: Likewise 10584 * unit-tests/test.c: Likewise 10585 10586 When build Wget2 using MinGW64 and embedding other library 10587 (Libmicrohttpd), DATADIR macro from Wget2 clash with MinGW64 10588 header. 10589 105902017-08-09 Darshit Shah <darnir@gnu.org> 10591 10592 * libwget/bar.c(_bar_set_progress): Fix progress bar printf command to fit total size 10593 105942017-08-08 Darshit Shah <darnir@gnu.org> 10595 10596 * libwget/bar.c: Deduplicate code in function wget_bar_set_slots 10597 105982017-08-07 Darshit Shah <darnir@gnu.org> 10599 10600 * libwget/bar.c: SPlit duplicated code into function 10601 106022017-08-07 Darshit Shah <darnir@gnu.org> 10603 10604 * libwget/bar.c: Replace redundant variable bar->max_slots with bar->nslots 10605 106062017-08-07 Darshit Shah <darnir@gnu.org> 10607 10608 * libwget/bar.c(wget_bar_init): Fix progress bar memory allocation value 10609 106102017-08-01 Darshit Shah <darnir@gnu.org> 10611 10612 Add documentation for wget_bar_set_slots 10613 106142017-08-04 Tim Rühsen <tim.ruehsen@gmx.de> 10615 10616 Hide internal structures/defines from doxygen 10617 10618 * include/wget/wget.h: Use _ as prefix for internals 10619 * libwget/atom_url.c: Likewise 10620 * libwget/cookie.c: Likewise 10621 * libwget/iri.c: Likewise 10622 * libwget/net.c: Likewise 10623 * libwget/rss_url.c: Likewise 10624 * libwget/sitemap_url.c: Likewise 10625 * src/wget_options.h: Likewise 10626 106272017-08-04 Tim Rühsen <tim.ruehsen@gmx.de> 10628 10629 Add docs for xml parsing functions. 10630 10631 * libwget/xml.c: Rename XML_CONTEXT to _xml_context, 10632 document wget_* functions 10633 * docs/Makefile.am: Sort order of man page files, 10634 add building of libwget-xml.3 10635 106362017-08-03 Tim Rühsen <tim.ruehsen@gmx.de> 10637 10638 * configure.ac: Improve ASAN and UBSAN sanitizer flags 10639 106402017-08-03 Tim Rühsen <tim.ruehsen@gmx.de> 10641 10642 * fuzz/*.in/*: Update fuzzer corpora 10643 106442017-08-03 Tim Rühsen <tim.ruehsen@gmx.de> 10645 10646 * libwget/utils.c: Fix param name in docs of wget_strnglob() 10647 106482017-08-03 Tim Rühsen <tim.ruehsen@gmx.de> 10649 10650 * libwget/net.c: Document 'protocol' for wget_tcp_set_protocol() 10651 106522017-08-03 Tim Rühsen <tim.ruehsen@gmx.de> 10653 10654 * libwget/buffer_printf.c: Do not document internal defines 10655 106562017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10657 10658 * fuzz/Makefile.am: Add -Wno-pedantic to suppress dlsym() warnings 10659 106602017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10661 10662 Add new fuzzer libwget_tlssess_fuzzer 10663 10664 * fuzz/Makefile.am: Add libwget_tlssess_fuzzer 10665 * fuzz/libwget_tlssess_fuzzer.c: New file 10666 * fuzz/libwget_tlssess_fuzzer.in/*: Add fuzz corpora 10667 * libwget/tls_session.c: Prepare code for fuzzing 10668 106692017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10670 10671 Add new fuzzer libwget_ocsp_fuzzer 10672 10673 * fuzz/Makefile.am: Add libwget_ocsp_fuzzer 10674 * fuzz/libwget_ocsp_fuzzer.c: New file 10675 * fuzz/libwget_ocsp_fuzzer.in/*: Add fuzz corpora 10676 106772017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10678 10679 * libwget/ocsp.c: Suppress clang warning about integer overflow 10680 106812017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10682 10683 Add new fuzzer libwget_netrc_fuzzer 10684 10685 * fuzz/Makefile.am: Add libwget_netrc_fuzzer 10686 * fuzz/libwget_netrc_fuzzer.c: New file 10687 * fuzz/libwget_netrc_fuzzer.dict: Add dictionary 10688 * fuzz/libwget_netrc_fuzzer.in/*: Add fuzz corpora 10689 106902017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10691 10692 * libwget/netrc.c: Suppress clang warning about integer overflow in hash function 10693 106942017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10695 10696 * libwget/http_parse.c (wget_http_parse_response_header): Fix 1-byte-read-buffer-overflow 10697 10698 Fix oss-fuzz issue #2823. 10699 107002017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10701 10702 * libwget/http_parse.c (wget_http_parse_content_disposition): Fix NULL ptr dereference 10703 10704 Fix oss-fuzz issue #2822. 10705 107062017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10707 10708 Add new fuzzer libwget_hsts_fuzzer 10709 10710 * fuzz/Makefile.am: Add libwget_hsts_fuzzer 10711 * fuzz/libwget_hsts_fuzzer.c: New file 10712 * fuzz/libwget_hsts_fuzzer.in/*: Add fuzz corpora 10713 107142017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10715 10716 Fix HSTS code regarding time_t overflow 10717 10718 * libwget/hsts.c: Use int64_t instead of time_t, 10719 (wget_hsts_new): Fix possible integer overflow, 10720 (_hsts_db_load): Likewise. 10721 10722 Same type of changes that we applied to HPKP code, 10723 to fix the integer over/underflows found by fuzzing. 10724 107252017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10726 10727 * libwget/strlcpy.c: Add documentation 10728 107292017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10730 10731 * libwget/stringmap.c: Suppress clang warning about integer overflow 10732 10733 The hash functions rely on integer overflow. 10734 Found by fuzzing. 10735 107362017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10737 10738 * fuzz/*.in/*: Update fuzzer corpora from oss-fuzz 10739 107402017-08-01 Tim Rühsen <tim.ruehsen@gmx.de> 10741 10742 * libwget/hpkp.c (wget_hpkp_set_maxage): Fix integer overflow 10743 10744 Fix oss-fuzz issue #2820. 10745 107462017-07-31 Tim Rühsen <tim.ruehsen@gmx.de> 10747 10748 * libwget/strlcpy.c: Always include string.h 10749 107502017-07-31 Tim Rühsen <tim.ruehsen@gmx.de> 10751 10752 Check for fmemopen() 10753 10754 * configure.ac: Check for fmemopen() 10755 * fuzz/libwget_hpkp_fuzzer.c: Skip corpora load if fmemopen is missing 10756 107572017-07-31 Tim Rühsen <tim.ruehsen@gmx.de> 10758 10759 * fuzz/libwget_hpkp_fuzzer.c: Skip corpora loading on WIN32 10760 107612017-07-31 Tim Rühsen <tim.ruehsen@gmx.de> 10762 10763 * fuzz/libwget_hpkp_fuzzer.c: Remove assert.h (syntax-check) 10764 107652017-07-31 Tim Rühsen <tim.ruehsen@gmx.de> 10766 10767 Fix DNS caching (regression) 10768 10769 * libwget/net.c: Add 'port' param where needed 10770 10771 Reported-by: Avinash Sonawane 10772 107732017-07-31 Tim Rühsen <tim.ruehsen@gmx.de> 10774 10775 * fuzz/README.md: Add undefined sanitizer to CFLAGS 10776 107772017-07-31 Tim Rühsen <tim.ruehsen@gmx.de> 10778 10779 New fuzzer 'libwget_hpkp_fuzzer' 10780 10781 * fuzz/Makefile.am: Add libwget_hpkp_fuzzer 10782 * fuzz/libwget_hpkp_fuzzer.c: New fuzzer 10783 * fuzz/libwget_hpkp_fuzzer.in/*: Add fuzzer corpora 10784 107852017-07-31 Tim Rühsen <tim.ruehsen@gmx.de> 10786 10787 Fix integer overflows in HPKP code 10788 10789 * libwget/hpkp.c: Use int64_t instead of time_t, 10790 (_hash_hpkp): Suppress integer overflow detection, 10791 (_hpkp_db_load): Avoid integer overflow 10792 10793 Two integer overflows detected by fuzzing. One is wanted and thus 10794 we suppress detection by clang. The other was unwanted and is fixed. 10795 107962017-07-31 Tim Rühsen <tim.ruehsen@gmx.de> 10797 10798 * fuzz/Makefile.am: Build all fuzzers with 'make oss-fuzz' 10799 108002017-07-30 Tim Rühsen <tim.ruehsen@gmx.de> 10801 10802 Add new fuzzer libwget_http_parse_fuzzer 10803 10804 * fuzz/Makefile.am: Add libwget_http_parse_fuzzer 10805 * fuzz/libwget_http_parse_fuzzer.c: New fuzzer 10806 * fuzz/libwget_http_parse_fuzzer.dict: New dictionary 10807 * fuzz/libwget_http_parse_fuzzer.in/*: Fuzzer corpora 10808 * fuzz/run-clang.sh: Use all availbale cores for fuzzing 10809 108102017-07-30 Tim Rühsen <tim.ruehsen@gmx.de> 10811 10812 * libwget/http_parse.c (wget_http_parse_content_disposition): Fix memleak (found by fuzzing) 10813 108142017-07-30 Tim Rühsen <tim.ruehsen@gmx.de> 10815 10816 * libwget/http_parse.c (wget_http_parse_link): Fix memleak (found by fuzzing) 10817 108182017-07-30 Tim Rühsen <tim.ruehsen@gmx.de> 10819 10820 Fix off-by-one buffer read in wget_hpkp_pin_add() 10821 10822 * libwget/hpkp.c (wget_hpkp_pin_add): Fix pinsize value, 10823 (_compare_pin): Fix off-by-one buffer read 10824 * libwget/base64.c (wget_base64_decode_alloc): Add outlen param 10825 * include/wget/wget.h (wget_base64_decode_alloc): Likewise 10826 * fuzz/libwget_base64_fuzzer.c: Add 3rd param to wget_base64_decode_alloc() 10827 10828 Found by local fuzzing 10829 108302017-07-30 Darshit Shah <darnir@gnu.org> 10831 10832 * libwget/http.c: Disable some very verbose debug statements 10833 108342017-07-30 Darshit Shah <darnir@gnu.org> 10835 10836 Support Wget 1.x `--auth-no-challenge` switch 10837 10838 * src/job.c(job_init): Set default value of challenges_alloc to false 10839 (job_free): Free the challenges vector only if the current job owns it 10840 * src/options.c: Add new switch "--auth-no-challenge" 10841 * src/wget.c(add_url_to_queue): Honor the config.auth_no_challenge 10842 option 10843 (add_url): Same 10844 (process_response_header): Fallback to Server provided challenges if 10845 defaults don't work 10846 108472017-07-29 Darshit Shah <darnir@gnu.org> 10848 10849 Fix make syntax-check due to double word 10850 108512017-07-29 Tim Rühsen <tim.ruehsen@gmx.de> 10852 10853 * libwget/http_parse.c (wget_http_parse_header_line): Fix memleaks (found by fuzzing) 10854 108552017-07-29 Darshit Shah <darnir@gnu.org> 10856 10857 * fuzz/libwget_utils_fuzzer.c: Fix call to wget_strglob 10858 108592017-07-29 Darshit Shah <darnir@gnu.org> 10860 10861 Revert "Prevent tiny memory allocations when globbing" 10862 10863 This reverts commit 2d7fe3e9fc364f282c945f372b9a78f8a56d66ff. 10864 10865 With the above patch, a tiny memory allocation is avoided, but in doing 10866 so, it misses an important case. The glob(3) call never returns a 10867 success when the file does not already exist at the given path. 10868 108692017-07-28 Tim Rühsen <tim.ruehsen@gmx.de> 10870 10871 * .gitlab-ci.yml: Move to Fedora 26 10872 108732017-07-28 Tim Rühsen <tim.ruehsen@gmx.de> 10874 10875 Fix more clang warnings 10876 10877 * libwget/css_tokenizer.lex: Skip -Wsuggest-attribute=pure for clang 10878 * libwget/iri.c (wget_iri_parse): Explicit cast 10879 * src/dl.c: Use 'int' for vector size 10880 * src/options.c: Likewise, 10881 remove 'return' after call to exit(), 10882 (set_long_option): Rename local var 'env' to 'path' 10883 * src/plugin.c: Make global vars 'static', 10884 Use 'int' for vector size 10885 108862017-07-28 Tim Rühsen <tim.ruehsen@gmx.de> 10887 10888 Fix more clang warnings 10889 10890 * tests/test-plugin-dummy.c: Make options 'static' 10891 * unit-tests/test-dl.c (string_vector_check): Make vector index 'int' 10892 * unit-tests/test-dl.c (string_vector_dump): Likewise 10893 * unit-tests/test.c (test_hsts): Make port 'uint16_t' 10894 108952017-07-28 Tim Rühsen <tim.ruehsen@gmx.de> 10896 10897 Fix clang warnings 10898 10899 * libwget/css.c (wget_css_parse_buffer): Explicitly typecast 'len' to int 10900 * libwget/http.c (_send_callback): Change datatype of 'rc' from int to ssize_t 10901 * libwget/http.c (_recv_callback): Likewise 10902 * libwget/http.c (wget_http_match_no_proxy): Change param name from 'no_proxies' to 10903 'no_proxies_vec' to not to shadow global 'no_proxies' 10904 * libwget/ssl_gnutls.c (check_ocsp_response): Explicit type conversion to unsigned int 10905 * libwget/net.c (wget_tcp_listen): Move declaration of variable 'on' in #ifdef block where it's used 10906 * src/options.c (set_long_option): Change datatypes of 'invert', 'char_insensitive' 10907 and 'value_present' from int to char 10908 10909 Reported-by: Avinash Sonawane 10910 109112017-07-26 Tim Rühsen <tim.ruehsen@gmx.de> 10912 10913 * fuzz/libwget_cookie_fuzzer.in/*: Update corpora 10914 109152017-07-26 Tim Rühsen <tim.ruehsen@gmx.de> 10916 10917 * fuzz/get_all_corpora: Work even if fuzzers are not pre-built 10918 109192017-07-26 Akash Rawal <argmlac@gmail.com> 10920 10921 * libwget/iri.c (iri_ports): Remove redundant const 10922 109232017-07-25 Akash Rawal <argmlac@gmail.com> 10924 10925 Make wget_iri_set_scheme() update iri->uri 10926 10927 * include/wget/wget.h (wget_iri_st): Add field uri_allocated 10928 * libwget/iri.c: Make wget_iri_set_scheme() update iri->uri 10929 109302017-07-24 Akash Rawal <argmlac@gmail.com> 10931 10932 Fix recursive downloads with '-c' option 10933 10934 * bootstrap.conf: Enable safe-read module 10935 * src/wget.c: Fix recursive downloads with '-c' option 10936 * tests/Makefile.am: Add test-c-r.c 10937 * tests/test-c-r.c: Add tests 10938 109392017-07-25 Tim Rühsen <tim.ruehsen@gmx.de> 10940 10941 Fix fuzzer regression test for WIN32 10942 10943 * fuzz/main.c: Adjust slash and remove .exe 10944 10945 Reported-by: Gisle Vanem 10946 109472017-07-25 Tim Rühsen <tim.ruehsen@gmx.de> 10948 10949 Fail fuzzers if *.in directory does not exist 10950 10951 * fuzz/Makefile.am: Set -DTEST_RUN 10952 * fuzz/main.c: Error on non-exisiting *.in directory 10953 109542017-07-23 Darshit Shah <darnir@gnu.org> 10955 10956 Use C99 bool instead of unsigned int in IRI struct and code 10957 109582017-07-23 Tim Rühsen <tim.ruehsen@gmx.de> 10959 10960 Change type of port numbers from char * to uint16_t 10961 10962 * include/wget/wget.h: Amend struct wget_iri_st, 10963 amend API 10964 * libwget/hsts.c: Amend port type to uint16_t 10965 * libwget/http.c: Likewise 10966 * libwget/http.h: Likewise 10967 * libwget/http_highlevel.c: Likewise 10968 * libwget/http_parse.c: Likewise 10969 * libwget/iri.c: Likewise 10970 * libwget/net.c: Likewise 10971 * src/blacklist.c: Likewise 10972 * src/host.c: Likewise 10973 * src/wget.c: Likewise 10974 * src/wget_host.h: Likewise 10975 * tests/libtest.c: Likewise 10976 * unit-tests/test.c: Likewise 10977 109782017-07-23 Darshit Shah <darnir@gnu.org> 10979 10980 Clean up .gitignore and sort 10981 109822017-07-23 Darshit Shah <darnir@gnu.org> 10983 10984 Remove unneeded make variables 10985 109862017-07-22 Darshit Shah <darnir@gnu.org> 10987 10988 * libwget/{hsts,hpkp}.c: Don't print all contents as debug messages 10989 109902017-07-23 Darshit Shah <darnir@gnu.org> 10991 10992 Fix heap-use-after-free bug in IRI 10993 10994 * libwget/http.c(wget_http_open): iri maybe free'd far before the 10995 connection is closed. Hence, strdup the relevant fields 10996 * libwget/http.c(wget_http_close): Free the newly strdup'ed memory 10997 109982017-07-22 Darshit Shah <darnir@gnu.org> 10999 11000 * include/wget/wget.h: Improve setting of noreturn attribute to functions 11001 110022017-07-22 Tim Rühsen <tim.ruehsen@gmx.de> 11003 11004 * libwget/cookie.c (_compare_cookie2): Fix code to avoid warnings and undefined behavior 11005 110062017-05-02 Avinash Sonawane <rootkea@gmail.com> 11007 11008 Implement --force-progress to force show progress bar 11009 11010 * bootstrap.conf: Add isatty module 11011 * src/options.c: Add force_progress option 11012 * src/wget.c: if force_progress then turn on progress bar 11013 * src/wget_options.h: Add force_progress in config struct 11014 110152017-05-02 Avinash Sonawane <rootkea@gmail.com> 11016 11017 * src/wget.c: Disable progress bar if output isn't TTY 11018 11019 Reported By: Darshit Shah 11020 110212017-07-22 Didik Setiawan <ds@didiksetiawan.com> 11022 11023 Fix digest authentication when server omits algorithm value 11024 11025 * libwget/http.c: Fix digest authentication mechanism when 11026 server omits algorithm value 11027 110282017-07-20 Avinash Sonawane <rootkea@gmail.com> 11029 11030 Fix doxygen warnings 11031 11032 * README.md: Mark options 11033 * docs/wget2_manual.md: Mark URLs 11034 * libwget/bar.c: Fix doc for bar_init(), 11035 add doxygen close tag at end of file 11036 * libwget/buffer.c: Fix doc for wget_buffer_reset() 11037 * libwget/console.c: Move unused tag to front of param 11038 * libwget/hpkp.c: Fix documented params for wget_hpkp_new() 11039 * libwget/iri.c: Fix doc for wget_iri_supported() 11040 * libwget/net.c: Fix doc for wget_tcp_printf() 11041 * libwget/utils.c: Move unused tag to front of param 11042 110432017-07-21 Tim Rühsen <tim.ruehsen@gmx.de> 11044 11045 * src/options.c: Add missing param to print_plugin_help 11046 110472017-07-08 Akash Rawal <argmlac@gmail.com> 11048 11049 Add plugin APIs for intercepting URLs 11050 11051 * include/wget/wget.h: Add function declarations 11052 * libwget/plugin.c: Add virtual functions 11053 * src/plugin.c: Add implementation 11054 * src/wget_plugin.h: Likewise 11055 * src/wget.c: Add code that calls the implementation 11056 * src/wget_job.h: Add a variable to store 'accept' action 11057 * tests/Makefile.am: Add libpluginapi.la 11058 * tests/test-plugin-dummy.c: Add a test plugin for testing 11059 * tests/test-plugin.c: Add tests 11060 11061 Plugin API for intercepting URLs has been implemented as per 11062 specifications at https://gitlab.com/akash_rawal/wget2/wikis/Stage-2 11063 110642017-07-16 Akash Rawal <argmlac@gmail.com> 11065 11066 * src/dl.c: Fix gcc warnings when compiled without plugin support 11067 110682017-07-16 Akash Rawal <argmlac@gmail.com> 11069 11070 * src/dl.c (convert_to_path_if_not): Use wget_aprintf() and simplify the code 11071 110722017-06-12 Akash Rawal <argmlac@gmail.com> 11073 11074 Add command line option forwarding for plugins 11075 11076 * include/wget/wget.h: Add API for command line option forwarding 11077 * libwget/plugin.c: Likewise 11078 * libwget/test_linking: Add function from libwget/plugin.c 11079 * src/dl.c: Remove unused dl_search1() and dl_list1(), 11080 use wget_vector_t for pointer arrays 11081 * src/wget_dl.h: Likewise 11082 * src/plugin.c: Implement option forwarding 11083 * src/wget_plugin.h: Likewise 11084 * src/options.c: Add options '--plugin-opt=', '--plugin-help' 11085 * unit-tests/test-dl.c: Use dl_list() instead of dl_list1() 11086 * tests/Makefile.am: Add test plugins pluginoption, pluginfaulty1, 11087 pluginfaulty2 11088 * tests/test-plugin-dummy.c: Implement additional test plugins 11089 * tests/test-plugin.c: Extend tests for testing option processing 11090 11091 Command line option forwarding has been implemented as per 11092 specifications at https://gitlab.com/akash_rawal/wget2/wikis/Stage-1 11093 11094 Several style issues were also fixed (Thanks Ander Juaristi) 11095 110962017-07-19 Avinash Sonawane <rootkea@gmail.com> 11097 11098 * src/options.c : Push 'invert' handling from set_long_option() to parse functions 11099 * src/options.c : Make --mirror boolean 11100 * src/options.c (parse_proxy): Modify --proxy to have behavior described below: 11101 1) --proxy is boolean which means inherit proxies from environment 11102 2) --no-proxy is boolean which means don't inherit proxies from environment 11103 3) --proxy=<addrs> sets http_proxy and https_proxy to <addrs> (shorthand!) 11104 4) --no-proxy=<addrs> makes wget2 to not use proxy while dealing with <addrs> 11105 111062017-07-19 Avinash Sonawane <rootkea@gmail.com> 11107 11108 * src/options.c (set_long_option): Check if opt->var is NULL before accessing *(opt->var) 11109 111102017-07-19 Avinash Sonawane <rootkea@gmail.com> 11111 11112 * src/options.c (set_long_option): Allow empty string i.e. "" to be argument of an option 11113 * src/options.c (parse_bool): If value is an empty string then set opt->var to 0 11114 111152017-07-19 Avinash Sonawane <rootkea@gmail.com> 11116 11117 * src/options.c (opt_compare_config_linear): Skip '_' or '-' from s1 only if there is '-' or '_' in s2 11118 * src/options.c (set_long_option): Invert value for boolean option with the prefix --no- even if the 11119 option argument is false/0/off e.g. --no-cache=off 11120 * src/options.c (set_long_option): For long options with value separated by '=', treat empty string("") argument 11121 as if no argument was given 11122 111232017-07-17 Avinash Sonawane <rootkea@gmail.com> 11124 11125 * src/options.c (set_long_option): Remove linear search 11126 11127 * src/options.c (_parse_option): Accept command with '_' char e.g. http_proxy 11128 * src/options.c (set_long_option): Remove unnecessary linear search 11129 * src/options.c (set_long_option): options are case-, underscore- and minus-insensitive. 11130 111312017-07-19 Tim Rühsen <tim.ruehsen@gmx.de> 11132 11133 * configure.ac: Add -fdiagnostics-color only for gcc >= 5 11134 111352017-07-16 Tim Rühsen <tim.ruehsen@gmx.de> 11136 11137 Add libwget_html_url_fuzzer 11138 11139 * fuzz/Makefile.am: Add libwget_html_url_fuzzer 11140 * fuzz/libwget_html_url_fuzzer.dict: Add dictionary 11141 * fuzz/libwget_html_url_fuzzer.in/*: New copora 11142 111432017-07-16 Tim Rühsen <tim.ruehsen@gmx.de> 11144 11145 Update gnulib 11146 111472017-07-16 Tim Rühsen <tim.ruehsen@gmx.de> 11148 11149 Revert "* libwget/http_parse.c (wget_http_print_date): Fix gmtime crash with Wine" 11150 11151 This reverts commit 11d413476bac87d9315727f8ce127e7acdde2066. 11152 This has been fixed by gnulib. 11153 111542017-07-16 Tim Rühsen <tim.ruehsen@gmx.de> 11155 11156 Revert "* src/log.c (_write_out): Fix localtime crash with Wine" 11157 11158 This reverts commit e7c78f2e3049c6f862ed06da4424a5b2980ed616. 11159 This has been fixed by gnulib. 11160 111612017-07-14 Tim Rühsen <tim.ruehsen@gmx.de> 11162 11163 * configure.ac: Use pkg-config to find libidn2 config 11164 111652017-07-14 Tim Rühsen <tim.ruehsen@gmx.de> 11166 11167 Use FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION from oss-fuzz 11168 11169 * fuzz/Makefile.am: Remove -DTEST_RUN from AM_CPPFLAGS 11170 * fuzz/main.c: Use #ifndef FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION 11171 111722017-07-14 Tim Rühsen <tim.ruehsen@gmx.de> 11173 11174 Check for fuzzer build in helper scripts 11175 11176 * fuzz/get_ossfuzz_corpora: Check if fuzzer were build for fuzzing 11177 * fuzz/run-clang.sh: Likewise 11178 111792017-07-14 Tim Rühsen <tim.ruehsen@gmx.de> 11180 11181 * fuzz/get_all_corpora: New script to update oss-fuzz corpora 11182 111832017-07-14 Tim Rühsen <tim.ruehsen@gmx.de> 11184 11185 * fuzz/*_fuzzer.in/*: Update fuzzer corpora from oss-fuzz 11186 111872017-07-11 Tim Rühsen <tim.ruehsen@gmx.de> 11188 11189 Fix buffer_printf() when char type is unsigned 11190 11191 Sometimes 'char' is not enough. C doesn't specify if it is 11192 signed or unsigned. Here we had to explicitly use 'signed char' 11193 instead of 'char'. 11194 Gcc can be switched to unsigned char with -funsigned-char. 11195 11196 Reported-by: Michael <michael@cyber-dome.com> 11197 111982017-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 11199 11200 * fuzz/Makefile.am: Fix static build of libwget_cookie_fuzzer 11201 112022017-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 11203 11204 Consolidate response header parsing 11205 11206 * include/wget/wget.h: New prototype wget_http_parse_header_line() 11207 * libwget/http.c (_on_header_callback): Use wget_http_parse_header_line() 11208 * libwget/http_parse.c: New function wget_http_parse_header_line(), 11209 (wget_http_parse_response_header): Use wget_http_parse_header_line() 11210 11211 We had doubled code for HTTP/1.1 and HTTP/2 response header parsing. 11212 Now merged into one function. 11213 11214 Reported-by: Avinash Sonawane 11215 112162017-07-10 Akash Rawal <argmlac@gmail.com> 11217 11218 Fix broken Travis-CI builds 11219 11220 * .travis.sh: Print fuzz/test-suite.log on failure 11221 * .travis.yml: Install libunistring-dev 11222 112232017-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 11224 11225 Add password parsing to wget_iri_parse() 11226 11227 * libwget/iri.c (wget_iri_parse): Find and set password 11228 * unit-tests/test.c: Test password parsing 11229 112302017-07-09 Akash Rawal <argmlac@gmail.com> 11231 11232 Fix const-correctness issue with wget_iri_clone 11233 11234 * include/wget/wget.h: Add 'const' to wget_iri_clone's parameter 11235 * libwget/iri.c: Likewise 11236 112372017-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 11238 11239 * fuzz/run-clang.sh: Needs bash for ctrl-c handling 11240 112412017-07-07 Darshit Shah <darnir@gnu.org> 11242 11243 Fix recursive downloads when using HTTP2 11244 11245 * src/wget.c (downloader_thread): Ensure that the job->original_url 11246 field is always set before attempting a download 11247 112482017-07-07 Tim Rühsen <tim.ruehsen@gmx.de> 11249 11250 * src/options.c: Set default read timeout to 900s 11251 112522017-07-07 Tim Rühsen <tim.ruehsen@gmx.de> 11253 11254 Do not use ALPN when not build with HTTP/2 support 11255 11256 * libwget/ssl_gnutls.c (struct _config): Conditionally set .alpn 11257 * src/options.c (init): Set ALPN string only when build with HTTP/2 support 11258 112592017-07-07 Tim Rühsen <tim.ruehsen@gmx.de> 11260 11261 * src/log.c (_write_out): Fix localtime crash with Wine 11262 112632017-07-07 Tim Rühsen <tim.ruehsen@gmx.de> 11264 11265 * libwget/http_parse.c (wget_http_print_date): Fix gmtime crash with Wine 11266 112672017-07-05 Tim Rühsen <tim.ruehsen@gmx.de> 11268 11269 * fuzz/Makefile.am: Add link libraries to build cookie fuzzer 11270 112712017-07-05 Tim Rühsen <tim.ruehsen@gmx.de> 11272 11273 Disable plugin tests when plugins are disabled 11274 11275 * configure.ac: Add conditional PLUGIN_SUPPORT 11276 * tests/Makefile.am: Enable plugin testing only if PLUGIN_SUPPORT exists 11277 * unit-tests/Makefile.am: Likewise 11278 112792017-07-05 Tim Rühsen <tim.ruehsen@gmx.de> 11280 11281 * configure.ac: Fix static linking with libidn2 11282 112832017-07-04 Tim Rühsen <tim.ruehsen@gmx.de> 11284 11285 Remove h2-14 and h2-16 from ALPN 11286 11287 * libwget/ssl_gnutls.c: Remove h2-14 and h2-16 from default ALPN offer 11288 * examples/http_multi_get.c: Set ALPN to h2,http/1.1 11289 * src/options.c: Likewise 11290 11291 h2-14 and h2-16 were some HTTP/2 drafts, should not be relevant in 11292 the real world now. 11293 112942017-07-03 Tim Rühsen <tim.ruehsen@gmx.de> 11295 11296 * fuzz/Makefile.am: Fix oss-fuzz target for new cookie fuzzer 11297 112982017-07-01 Tim Rühsen <tim.ruehsen@gmx.de> 11299 11300 Enhance memtohex fuzzer, rename it to utils fuzzer 11301 11302 * Makefile.am: Rename libwget_memtohex_fuzzer to libwget_utils_fuzzer 11303 * libwget_memtohex_fuzzer.c: Renamed to libwget_utils_fuzzer.c 11304 * libwget_utils_fuzzer.c: New file 11305 * libwget_utils_fuzzer.in/*: Test corpora 11306 * run-clang.sh: Add merge step in ctrl-c handler 11307 113082017-06-30 Tim Rühsen <tim.ruehsen@gmx.de> 11309 11310 * fuzz/libwget_memtohex_fuzzer.c: Add all functions from utils.c 11311 113122017-06-30 Tim Rühsen <tim.ruehsen@gmx.de> 11313 11314 Update fuzzer with latest corpora from oss-fuzz 11315 11316 * fuzz/get_ossfuzz_corpora: Enhance corpora merging 11317 * fuzz/libwget_*fuzzer.in/*: Updated fuzz corpora 11318 113192017-06-29 Tim Rühsen <tim.ruehsen@gmx.de> 11320 11321 Update gnulib submodule 11322 113232017-06-26 Tim Rühsen <tim.ruehsen@gmx.de> 11324 11325 Update fuzz corpora from OSS-Fuzz 11326 11327 * fuzz/get_ossfuzz_corpora: New file to download and merge corpora 11328 * fuzz/*.in/*: New corpora 11329 * fuzz/run-clang.sh: Updated to work with get_ossfuzz_corpora 11330 113312017-06-25 Tim Rühsen <tim.ruehsen@gmx.de> 11332 11333 * .gitlab-ci.yml: Switch MinGW64 build to Debian 11334 113352017-06-24 Tim Rühsen <tim.ruehsen@gmx.de> 11336 11337 * libwget/ssl_gnutls.c (_cert_verify_hpkp): Fix code for GnuTLS < 3.1.3 11338 113392017-06-24 Tim Rühsen <tim.ruehsen@gmx.de> 11340 11341 Correctly include config.h into css_tokenizer 11342 11343 * libwget/css_tokenizer.lex: Add config.h via %top directive 11344 * libwget/Makefile.am: Remove sed from make target 11345 113462017-06-23 Tim Rühsen <tim.ruehsen@gmx.de> 11347 11348 * libwget/cookie.c (wget_cookie_parse_setcookie): Avoid maxage integer overflow 11349 113502017-06-23 Avinash Sonawane <rootkea@gmail.com> 11351 11352 *configure.ac: Fix warning about 'AM_PROG_MKDIR_P' macro being deprecated 11353 * README.md: Update gettext required version 11354 113552017-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 11356 11357 Increase fuzz coverage for cookie code 11358 11359 * fuzz/libwget_cookie_fuzzer.c: Add more function calls 11360 * fuzz/libwget_cookie_fuzzer.in/*: Add test corpora 11361 113622017-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 11363 11364 * fuzz/libwget_xml_parse_buffer_fuzzer.c: Don't hang when readng from stdin 11365 113662017-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 11367 11368 * libwget/io.c (wget_update_file): Fix leak found by fuzzing 11369 113702017-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 11371 11372 Fix issues in cookie code found by fuzzing 11373 11374 * libwget/cookie.c (wget_cookie_db_load_psl): Fix memleak when 11375 psl_load_file() fails, 11376 (_compare_cookie): Fix strcmp() NULL pointer crash, 11377 (_wget_cookie_normalize_cookie): Fix memleak 11378 113792017-06-22 Avinash Sonawane <rootkea@gmail.com> 11380 11381 * src/options.c: Fix array index out of bounds runtime error 11382 113832017-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 11384 11385 Enhance creating of libwget/css_tokenizer.c 11386 11387 * Makefile.am: Remove '<stdout>' special handling in coverage rules 11388 * libwget/Makefile.am: Generate css_tokenizer.c with proper automake 11389 macros, prepend include line using sed. 11390 113912017-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 11392 11393 * fuzz/libwget_xml_parse_buffer_fuzzer.c: Increase fuzz code coverage 11394 113952017-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 11396 11397 * fuzz/libwget_base64_fuzzer.c: Add coverage for wget.h inline functions 11398 113992017-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 11400 11401 Let CI create online fuzz coverage report 11402 11403 * .gitlab-ci.yml: Place fuzz coverage report in public area 11404 * Makefile.am: Use 'make check' in fuzz directory 11405 * README.md: Add link to fuzz coverage report 11406 * fuzz/Makefile.am: Remove make target 'fuzz-coverage' 11407 * fuzz/coverage.sh: Remove file 11408 114092017-06-18 Tim Rühsen <tim.ruehsen@gmx.de> 11410 11411 * po/POTFILES.in: Add libwget/http_parse.c 11412 114132017-06-18 Tim Rühsen <tim.ruehsen@gmx.de> 11414 11415 * fuzz/libwget_cookie_fuzzer.c: Add forgotten file 11416 114172017-06-18 Tim Rühsen <tim.ruehsen@gmx.de> 11418 11419 Add new fuzzer libwget_cookie_fuzzer 11420 11421 * fuzz/Makefile.am: Add libwget_cookie_fuzzer 11422 * fuzz/libwget_cookie_fuzzer.c: New fuzzer 11423 * fuzz/libwget_cookie_fuzzer.in/*: Corpora 11424 * fuzz/run-clang.sh: Add -lpsl and -lgnutls 11425 114262017-06-18 Tim Rühsen <tim.ruehsen@gmx.de> 11427 11428 Move parsing from libwget/http.c to libwget/http_parse.c 11429 11430 * libwget/Makefile.am: Add http_parse.c 11431 * libwget/http.c: Remove parse functions 11432 * libwget/http_parse.c: New file with parse functions 11433 114342017-06-17 Tim Rühsen <tim.ruehsen@gmx.de> 11435 11436 Add new fuzzer libwget_base64_fuzzer 11437 11438 * fuzz/Makefile.am: Add libwget_base64_fuzzer 11439 * fuzz/libwget_base64_fuzzer.c: New fuzzer 11440 * fuzz/libwget_base64_fuzzer.in/*: Corpora 11441 114422017-06-17 Tim Rühsen <tim.ruehsen@gmx.de> 11443 11444 Increase fuzz coverage for bar.c and rss_url.c 11445 11446 * fuzz/libwget_atom_url_fuzzer.c: Add RSS parsing 11447 * fuzz/libwget_atom_url_fuzzer.dict: Add RSS tags 11448 * fuzz/libwget_bar_fuzzer.c: Increase fuzz code coverage 11449 * fuzz/*.in/*: Update corpora 11450 114512017-06-17 Tim Rühsen <tim.ruehsen@gmx.de> 11452 11453 Check slot value in wget_bar_slot_deregister() 11454 11455 * libwget/bar.c (wget_bar_init): Remove redundant xfree()s, 11456 (wget_bar_slot_downloaded): Check value of 'slot' param 11457 114582017-06-16 Tim Rühsen <tim.ruehsen@gmx.de> 11459 11460 Fix issues in IRI code found by fuzzing 11461 11462 * include/wget/wget.h: Remove nonnull attribute from wget_iri_compare() 11463 * libwget/iri.c (wget_iri_clone): Fix possible crashes due to random free(), 11464 (wget_iri_compare): Check input arguments against NULL, 11465 (wget_iri_set_scheme): Use string compare instead of pointer compare 11466 114672017-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 11468 11469 Add new fuzzer libwget_iri_fuzzer 11470 11471 * fuzz/Makefile.am: Add libwget_sitemap_url_fuzzer 11472 * fuzz/libwget_iri_fuzzer.c: New fuzzer 11473 * fuzz/libwget_iri_fuzzer.dict: New dictionary 11474 * fuzz/libwget_iri_fuzzer.in/*: Corpora 11475 114762017-06-15 Ander Juaristi <a@juaristi.eus> 11477 11478 Fix MinGW compatibility issues 11479 11480 * libwget/buffer_printf.c (_read_precision): do not pass a va_list 11481 as argument. 11482 (_read_field_width): likewise. 11483 114842017-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 11485 11486 Add new fuzzer libwget_sitemap_url_fuzzer 11487 11488 * fuzz/Makefile.am: Add libwget_sitemap_url_fuzzer 11489 * fuzz/libwget_sitemap_url_fuzzer.c: New fuzzer 11490 * fuzz/libwget_sitemap_url_fuzzer.dict: New dictionary 11491 * fuzz/libwget_sitemap_url_fuzzer.in/*: Corpora 11492 * fuzz/run-clang.sh: Add -jobs and -workers 11493 114942017-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 11495 11496 * libwget/utils.c: Fix param in docs of wget_strglob() 11497 114982017-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 11499 11500 Remove nonnnull attribute from functions 11501 11502 * include/wget/wget.h: Remove G_GNUC_WGET_NONNULL from several functions 11503 * libwget/buffer.c: Add explicit checks for NULL 11504 * libwget/buffer_printf.c (wget_buffer_vprintf_append): Likewise 11505 * libwget/net.c: Likewise 11506 11507 The nonnull attribute removes the NULL checks under certain conditions. 11508 That might turn out as a security flaw. 11509 115102017-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 11511 11512 * libwget/metalink.c: Avoid calling memcpy with src=NULL 11513 115142017-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 11515 11516 Add new fuzzer libwget_css_url_fuzzer 11517 11518 * fuzz/Makefile.am: Add libwget_css_url_fuzzer 11519 * fuzz/libwget_css_url_fuzzer.c: New fuzzer 11520 * fuzz/libwget_css_url_fuzzer.dict: New dictionary 11521 * fuzz/libwget_css_url_fuzzer.in/*: Corpora 11522 115232017-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 11524 11525 * libwget/css.c (wget_css_parse_buffer): Fix length calculation 11526 115272017-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 11528 11529 * fuzz/libwget_atom_url_fuzzer.dict: Remove empty line at EOF 11530 115312017-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 11532 11533 Add new fuzzer libwget_atom_url_fuzzer 11534 11535 * fuzz/Makefile.am: Add libwget_atom_url_fuzzer 11536 * fuzz/libwget_atom_url_fuzzer.c: New fuzzer 11537 * fuzz/libwget_atom_url_fuzzer.dict: New dictionary 11538 * fuzz/libwget_atom_url_fuzzer.in/*: Corpora 11539 115402017-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 11541 11542 Add fuzz corpora from OSS-Fuzz 11543 11544 * *.in/*: New copora 11545 * run-clang.sh: Add needed libraries to fuzzer executables (regression) 11546 115472017-06-12 Ander Juaristi <a@juaristi.eus> 11548 11549 docs: Inline docs for net.c 11550 115512017-06-12 Ander Juaristi <a@juaristi.eus> 11552 11553 docs: Inline docs for IRI/URI API 11554 115552017-06-12 Ander Juaristi <a@juaristi.eus> 11556 11557 docs: Inline docs for buffer and buffer_printf 11558 115592017-06-13 Tim Rühsen <tim.ruehsen@gmx.de> 11560 11561 * libwget/http.c (wget_http_add_credentials): Remove debug printf 11562 115632017-06-13 Tim Rühsen <tim.ruehsen@gmx.de> 11564 11565 New options --http-proxy-user and --http-proxy-passord 11566 11567 * docs/wget2_manual.md: Amend description of user/password options 11568 * docs/wget2_manual.md: Add new param to wget_http_add_credentials() 11569 * libwget/http.c (wget_http_parse_response_header): Add Proxy-Authenticate 11570 HTTP header, 11571 (wget_http_add_credentials): Add Proxy-Authorization HTTP header, 11572 (_on_header_callback): Add Proxy-Authenticate HTTP header 11573 * libwget/http_highlevel.c: Amend call to wget_http_add_credentials() 11574 * src/job.c (job_free): Free proxy_challenges 11575 * src/options.c: Add --http-proxy-password and --http-proxy-user 11576 * src/wget.c (process_response_header): Add status code 407, 11577 (_add_authorize_header): New function, 11578 (http_create_request): Use _add_authorize_header() 11579 * src/wget_job.h: Add JOB.proxy_challenges 11580 * src/wget_options.h: Add config.http_proxy_username and config.http_proxy_password 11581 * unit-tests/test.c: Amend call to wget_http_add_credentials() 11582 115832017-06-12 Tim Rühsen <tim.ruehsen@gmx.de> 11584 11585 * libwget/ssl_gnutls.c: Debug output for certain OCSP warnings 11586 115872017-06-11 Akash Rawal <argmlac@gmail.com> 11588 11589 * src/options.c (set_long_option): Fix behavior for options that don't accept arguments and are not boolean 11590 115912017-06-11 Akash Rawal <argmlac@gmail.com> 11592 11593 Allow setting destructor to keys to wget_stringmap_t 11594 11595 * include/wget/wget.h: Add wget_stringmap_key_destructor_t and 11596 wget_stringmap_set_key_destructor() declaration. 11597 * libwget/stringmap.c: Add wget_stringmap_set_key_destructor() 11598 definition. 11599 116002017-06-11 Akash Rawal <argmlac@gmail.com> 11601 11602 Remove rarely used parameter 'off' from wget_hashmap_create() 11603 11604 * libwget/hashmap.c: Remove 'off' parameter, now defaults to -2, 11605 Add wget_hashmap_set_growth_policy() to set 'off' 11606 * include/wget/wget.h: likewise 11607 * libwget/hpkp.c: Adapt to the change in wget_hashmap_create() 11608 * libwget/hsts.c: Likewise 11609 * libwget/netrc.c: Likewise 11610 * libwget/ocsp.c: Likewise 11611 * libwget/stringmap.c: Likewise 11612 * libwget/test_linking.c: Likewise 11613 * libwget/tls_session.c: Likewise 11614 * src/blacklist.c: Likewise 11615 * src/host.c: Likewise 11616 * src/wget.c: Likewise 11617 116182017-06-11 Avinash Sonawane <rootkea@gmail.com> 11619 11620 * unit-tests/test.c: Add tests for wget_http_parse_response_header() 11621 116222017-06-10 Tim Rühsen <tim.ruehsen@gmx.de> 11623 11624 Fix integer overflow in metalink code found by OSS-Fuzz 11625 11626 * fuzz/libwget_metalink_parse_fuzzer.repro/5268760606801920: 11627 Reproducer from oss-fuzz 11628 * libwget/metalink.c (_add_piece): Avoid integer overflow 11629 116302017-06-09 Tim Rühsen <tim.ruehsen@gmx.de> 11631 11632 * fuzz/fuzzer.h: Add extern 'C' 11633 116342017-06-09 Tim Rühsen <tim.ruehsen@gmx.de> 11635 11636 * fuzz/README.md: Add code coverage section 11637 116382017-06-09 Tim Rühsen <tim.ruehsen@gmx.de> 11639 11640 * fuzz/Makefile.am: Add clean-local make target 11641 116422017-06-08 Tim Rühsen <tim.ruehsen@gmx.de> 11643 11644 Increase fuzz coverage of wget_metalink_parse() 11645 11646 * libwget_metalink_parse_fuzzer.c: Call wget_metalink_sort_mirrors() 11647 * libwget_metalink_parse_fuzzer.dict: Add keywords 11648 * libwget_metalink_parse_fuzzer.in/xml1: Add content 11649 116502017-06-08 Tim Rühsen <tim.ruehsen@gmx.de> 11651 11652 Add script for fast checking of fuzz coverage 11653 11654 * fuzz/view-coverage.sh: New helper script 11655 116562017-06-08 Tim Rühsen <tim.ruehsen@gmx.de> 11657 11658 Add more fuzz corpora for wget_xml_parse_buffer() 11659 11660 * fuzz/libwget_xml_parse_buffer_fuzzer.in/*: New files 11661 116622017-06-08 Tim Rühsen <tim.ruehsen@gmx.de> 11663 11664 Increase fuzz coverage for wget_xml_parse_buffer() 11665 11666 * libwget_xml_parse_buffer_fuzzer.c: Increase fuzz coverage 11667 * libwget_xml_parse_buffer_fuzzer.dict: Add 'style' keyword 11668 116692017-06-07 Tim Rühsen <tim.ruehsen@gmx.de> 11670 11671 Use C instead of C++ for fuzzer targets 11672 11673 * configure.ac: Remove AC_PROG_CXX 11674 * fuzz/Makefile.am: Amend *_SOURCES for .c fuzzer targets, 11675 Remove non-portable EXTRA_DIST wildcards, 11676 Add dist-hook to include fuzzer target files, 11677 Add oss-fuzz make target. 11678 * fuzz/fuzzer.h: New file 11679 * fuzz/*cc: Rename to C, include fuzzer.h 11680 * fuzz/main.c: Include fuzzer.h 11681 * fuzz/run-clang.sh: Build and run fuzzer targets made with clang/libFuzzer 11682 116832017-06-07 Akash Rawal <argmlac@gmail.com> 11684 11685 Fix name clashes with gnulib 11686 11687 * tests/test-plugin.c: Rename setenv and unsetenv to setenv_rpl 11688 and unsetenv_rpl respectively. 11689 * unit-tests/test-dl.c: Rename rpl_remove to remove_rpl. 11690 11691 Reported by @gvanem 11692 116932017-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 11694 11695 * tests/Makefile.am: Remove buffer_printf_perf, stringmap_perf from check_PROGRAMS 11696 116972017-06-06 Akash Rawal <argmlac@gmail.com> 11698 11699 Fix sanitizer builds 11700 11701 * .gitlab-ci.yml: Separate sanitizer build from `make distcheck` and 11702 use clang++ as c++ compiler in clang/Fedora build 11703 * .travis.sh: Use clang++ as c++ compiler if CC=clang and 11704 disable sanitizer builds (unstable on Travis-CI) 11705 * .travis.yml: Add lzip package 11706 * .travis_setup.sh: Likewise 11707 117082017-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 11709 11710 Remove obsolete files from tests/ 11711 11712 * tests/buffer_printf_perf.c: Remove 11713 * tests/stringmap_perf.c: Remove 11714 11715 Reported-by: Gisle Vanem 11716 117172017-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 11718 11719 * .gitlab-ci.yml: Parallel 'make check' for Debian image 11720 117212017-05-30 Akash Rawal <argmlac@gmail.com> 11722 11723 Add basic plugin loading mechanism 11724 11725 * configure.ac: Add configure time tests for plugin support backend. 11726 * include/wget/wget.h: Add basic plugin API 11727 * libwget/Makefile.am: Add new file plugin.c to libwget.la 11728 * libwget/plugin.c: Add plugin API functions (new file) 11729 * src/Makefile.am: Add new files dl.c, wget_dl.h, plugin.c, wget_plugin.h 11730 to wget2 11731 * src/dl.c: Add abstraction for dynamic loading of object files 11732 * src/wget_dl.h: Likewise 11733 * src/plugin.c: Add implementation for loading plugins 11734 * src/wget_plugin.h: likewise 11735 * src/options.c: Add options `--plugin=`, `--local-plugin=`, 11736 `--plugin-dirs=`, `--list-plugins` 11737 * src/wget.c: Add function calls to initialize and finalize plugin 11738 support system 11739 * unit-tests/Makefile.am: Add test-dl, libalpha.la, and libbeta.la 11740 * unit-tests/test-dl.c: Add unit test for src/dl.c 11741 * unit-tests/test-dl-dummy.c: Likewise 11742 * tests/Makefile.am: Add test-plugin, libpluginname.la, and 11743 libpluginexit.la 11744 * tests/test-plugin.c: Add tests for plugin support 11745 * tests/test-plugin-dummy.c: Likewise 11746 * libwget/test-linking.c: Fix warnings with --enable-manywarnings 11747 * unit-tests/test.c: likewise 11748 11749 Plugin loading mechanism has been implemented as per specifications 11750 at https://gitlab.com/akash_rawal/wget2/wikis/Stage-0 11751 117522017-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 11753 11754 * fuzz/Makefile.am: Add test corpora to tarball 11755 117562017-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 11757 11758 * configure.ac: Use dist-lzip instead of dist-xz 11759 117602017-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 11761 11762 * configure.ac: Set CXXFLAGS for --enable-sanitize-... options 11763 117642017-06-04 Tim Rühsen <tim.ruehsen@gmx.de> 11765 11766 * .gitlab-ci.yml: YAML doesn't like paths starting with */ 11767 117682017-06-04 Tim Rühsen <tim.ruehsen@gmx.de> 11769 11770 Add OSS-Fuzz reproducers to test suite 11771 11772 * cfg.mk: Exclude *.repro/* from syntax-check 11773 * fuzz/Makefile.am: Add *.repro/* to EXTRA_DIST 11774 * fuzz/libwget_metalink_parse_fuzzer.repro/clusterfuzz-testcase-minimized-5103826937839616: 11775 Reproducer for OSS-Fuzz issue #1915 11776 * fuzz/libwget_xml_parse_buffer_fuzzer.repro/clusterfuzz-testcase-minimized-4704516446355456: 11777 Reproducer for OSS-Fuzz issue #2041 11778 * fuzz/main.c: Scan and test all files from *.repro/ 11779 117802017-06-04 Tim Rühsen <tim.ruehsen@gmx.de> 11781 11782 * libwget/xml.c (getScriptContent): Fix read-off-by-one found by OSS-Fuzz (#2041) 11783 117842017-06-04 Tim Rühsen <tim.ruehsen@gmx.de> 11785 11786 * .gitlab-ci.yml: Add all .log files to artifacts 11787 117882017-06-04 Akash Rawal <argmlac@gmail.com> 11789 11790 * .travis.sh: Fix printing logs when build fails 11791 117922017-06-03 Tim Rühsen <tim.ruehsen@gmx.de> 11793 11794 * fuzz/Makefile.am: Remove lib/ from include directories 11795 117962017-06-02 Tim Rühsen <tim.ruehsen@gmx.de> 11797 11798 Use fuzz targets as unit tests for 'make check' 11799 11800 * Makefile.am: Add 'fuzz' directory to SUBDIRS, 11801 tune 'fuzz-coverage' make target, 11802 print info on how to view coverage report. 11803 * configure.ac: Add AC_PROG_CXX for C++ code in fuzz/ 11804 * fuzz/Makefile: Removed 11805 * fuzz/Makefile.am: New file 11806 * fuzz/libwget_bar_fuzzer.cc: Fix memleak 11807 * fuzz/main.c: Add code for unit testing 11808 11809 The OSS-Fuzz targets will now be taken for unit testing as well. 11810 11811 That is, they will be tested by 'make check' including our sanitizer 11812 and valgrind checks. The fuzz tests will be run once with each test 11813 corpus from the corresponding '.in' directory. 11814 11815 A 'make fuzz-coverage' generates a coverage report for running all 11816 fuzz targets against their corpora. 11817 118182017-06-01 Tim Rühsen <tim.ruehsen@gmx.de> 11819 11820 * fuzz/libwget_memtohex_fuzzer.cc: Add two test vectors 11821 118222017-06-01 Tim Rühsen <tim.ruehsen@gmx.de> 11823 11824 Add make target 'fuzz-coverage' 11825 11826 * Makefile.am: Add target 'fuzz-coverage' to generate a coverage 11827 report for fuzz targets. 11828 * fuzz/Makefile: Add target 'fuzz-coverage' 11829 * fuzz/coverage.sh: New script to call a fuzz target with each corpus. 11830 * fuzz/libwget_metalink_parse_fuzzer.cc: Add check for max input length. 11831 * fuzz/libwget_xml_parse_buffer_fuzzer.cc: Likewise 11832 * fuzz/libwget_robots_parse_fuzzer.cc: Add comment 11833 * fuzz/libwget_robots_parse_fuzzer.in/user_agent2: Amend for max coverage 11834 * fuzz/main.c: selfmade __AFL_LOOP() returns 0 after first invocation. 11835 * fuzz/libwget_memtohex.in: Rename to fuzz/libwget_memtohex_fuzzer.in 11836 * fuzz/libwget_metalink_parse.in: Rename to fuzz/libwget_metalink_parse_fuzzer.in 11837 * fuzz/libwget_robots_parse.in: Rename to fuzz/libwget_robots_parse_fuzzer.in 11838 * fuzz/libwget_xml_parse_buffer.in: Rename to fuzz/libwget_xml_parse_buffer_fuzzer.in 11839 * fuzz/libwget_xml_parse_buffer.options: Removed 11840 * fuzz/libwget_metalink_parse_fuzzer.options: Removed 11841 118422017-06-01 Avinash Sonawane <rootkea@gmail.com> 11843 11844 Use HTTPS url instead of SSH to clone the repo 118452017-05-31 Tim Rühsen <tim.ruehsen@gmx.de> 11846 11847 Move unit tests from tests/ to unit-tests/ 11848 11849 * Makefile.am: Add unit-tests 11850 * cfg.mk: Make syntax-check work with unit-tests/ 11851 * configure.ac: Add unit-tests/Makefile 11852 * tests/Makefile.am: Remove unit tests 11853 * unit-tests/Makefile.am: New file with unit test rules 11854 * unit-tests/*.c: Moved from tests/ 11855 * unit-tests/files: Moved from tests/ 11856 11857 unit-tests/ is for library/API unit testing. 11858 tests/ is for testing wget2 utility. 11859 118602017-05-31 Tim Rühsen <tim.ruehsen@gmx.de> 11861 11862 Enhance fuzz code 11863 11864 * fuzz/*_fuzzer.cc: Use malloc() instead of stack memory to help 11865 sanitizers. 11866 * fuzz/libwget_robots_parse.in/user_agent1: Increase code coverage. 11867 * fuzz/main.c: define __AFL_LOOP() is not already defined. 11868 118692017-05-31 Akash Rawal <argmlac@gmail.com> 11870 11871 Silence GCC warnings 11872 11873 * tests/test.c (test_parser): Replace hardcoded array size with an expression 11874 * tests/test-parse-html.c (test_parse_files): Likewise 11875 118762017-05-30 Tim Rühsen <tim.ruehsen@gmx.de> 11877 11878 * libwget/xml.c (getValue): Fix memleak 11879 11880 Found locally via fuzzing. 11881 118822017-05-30 Tim Rühsen <tim.ruehsen@gmx.de> 11883 11884 * src/wget.c (metalink_parse_localfile): Fix memleak 11885 11886 Found locally via fuzzing. 11887 118882017-05-30 Tim Rühsen <tim.ruehsen@gmx.de> 11889 11890 * libwget/metalink.c (_metalink_parse): Fix memleak 11891 11892 Found locally via fuzzing. 11893 118942017-05-30 Tim Rühsen <tim.ruehsen@gmx.de> 11895 11896 .gitignore: Add generated files from fuzz/ directory 11897 118982017-05-30 Tim Rühsen <tim.ruehsen@gmx.de> 11899 11900 Add two fuzz corpora 11901 11902 * fuzz/libwget_metalink_parse.in/tag1: New file 11903 * fuzz/libwget_xml_parse_buffer.in/tag1: New file 11904 119052017-05-30 Tim Rühsen <tim.ruehsen@gmx.de> 11906 11907 Fix off-by-one read in XML code 11908 11909 * libwget/xml.c (getToken): Fix off-by-one read 11910 11911 Reported-by: oss-fuzz 11912 https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=1915 11913 119142017-05-30 Tim Rühsen <tim.ruehsen@gmx.de> 11915 11916 Fix fuzz minimizing command 11917 11918 * fuzz/README.md: Fix build recipe 11919 * fuzz/run-afl.sh: Fix check whether to call afl-tmin 11920 119212017-05-30 Tim Rühsen <tim.ruehsen@gmx.de> 11922 11923 * fuzz/Makefile: Forgot to add to repo 11924 119252017-05-29 Tim Rühsen <tim.ruehsen@gmx.de> 11926 11927 Add *.options files to fuzzer 11928 11929 * fuzz/libwget_metalink_parse.options: New file 11930 * fuzz/libwget_xml_parse_buffer.options: New file 11931 11932 Max input len is 64 for OSS-Fuzz, we need larger input for 11933 some fuzz targets. 11934 119352017-05-29 Tim Rühsen <tim.ruehsen@gmx.de> 11936 11937 Enhance fuzzing 11938 11939 * cfg.mk: Let make-syntax-check survive 11940 * fuzz/libwget_metalink_parse.dict: New dictionary 11941 * fuzz/libwget_robots_parse.dict: New dictionary 11942 * fuzz/libwget_xml_parse_buffer.dict: New dictionary 11943 * fuzz/libwget_robots_parse.in/disallowed1: Obsolete by afl-cmin 11944 * fuzz/libwget_robots_parse.in/user_agent3: Likewise 11945 * fuzz/libwget_xml_parse_buffer.in/index.html: Removed redundant HTML tags 11946 * fuzz/run-afl.sh: Auto-optimize corpora with afl-tmin 11947 119482017-05-29 Tim Rühsen <tim.ruehsen@gmx.de> 11949 11950 Add initial fuzzing code 11951 11952 * fuzz/*: New files 11953 11954 README.md contains instructions how to build and run single fuzzers. 11955 Currently we have fuzzers and corpora for 11956 - wget_memtohex() 11957 - wget_metalink_parse() 11958 - wget_robots_parse() 11959 - wget_xml_parse_buffer() 11960 11961 * fuzz/libwget_bar.in/123456: New file 11962 119632017-05-28 Tim Rühsen <tim.ruehsen@gmx.de> 11964 11965 * tests/test.c: Add test_strlcpy to cover wget_strlcpy() 11966 119672017-05-28 Tim Rühsen <tim.ruehsen@gmx.de> 11968 11969 * tests/test.c: Add test for wget_strmemcpy for 100% coverage 11970 119712017-05-28 Ander Juaristi <a@juaristi.eus> 11972 11973 Make wget_buffer_realloc static 11974 11975 * libwget/buffer.c (wget_buffer_realloc): rename to _buffer_realloc, 11976 and make it static. 11977 (_buffer_realloc): new function. 11978 11979 This function may overflow if incorrectly used so we make it internal to 11980 'buffer.c'. All the functions there that rely on it know how to use it 11981 correctly. 11982 119832017-05-27 Tim Rühsen <tim.ruehsen@gmx.de> 11984 11985 Fix wget_strmemcpy() and add tests for mem.c 11986 11987 * include/wget/wget.h: Remove nonnull attribute from wget_strmemcpy 11988 * libwget/mem.c: Check input values for wget_strmemcpy 11989 * tests/test.c: Add tests for functions of mem.c 11990 119912017-05-26 Tim Rühsen <tim.ruehsen@gmx.de> 11992 11993 * AUTHORS: Updated contributor list 11994 119952017-05-26 Akash Rawal <argmlac@gmail.com> 11996 11997 * .gitlab-ci.yml: Enable MinGW64 build 11998 119992017-05-25 Tim Rühsen <tim.ruehsen@gmx.de> 12000 12001 Fix clang -Wconversion warnings 12002 12003 * include/wget/wget.h (wget_ready_2_transfer): Change 'mode' to int 12004 * libwget/io.c: Likewise 12005 * libwget/hpkp.c: Explicit int to bit conversion 12006 * libwget/http.c: Likewise 12007 * libwget/net.c: Likewise 12008 * libwget/ocsp.c: Likewise 12009 * libwget/ssl_gnutls.c: Likewise 12010 * tests/libtest.c: Likewise 12011 * libwget/iri.c (wget_iri_unescape_inline): Explicit int to char conversion 12012 * libwget/utils.c: Likewise 12013 120142017-05-24 Tim Rühsen <tim.ruehsen@gmx.de> 12015 12016 * libwget/Makefile.am: Fix CPPFLAGS for libcsstokenizer 12017 120182017-05-24 Tim Rühsen <tim.ruehsen@gmx.de> 12019 12020 * libwget/Makefile.am: Remove macro warnings 12021 120222017-05-22 Didik Setiawan <ds@didiksetiawan.com> 12023 12024 Add support to save file extended attributes 12025 12026 * configure.ac: Check for xattr availability 12027 * src/Makefile.am: Add wget_xattr.h 12028 * src/options.c: Add new config 'xattr', 12029 (parse_execute): Add new option 'xattr 12030 * src/wget.c: (_prepare_file): Add condition for 'xattr', 12031 (http_send_request): Add parameter 'original_url', 12032 (add_url): Assign 'original_url' with proper value, 12033 (download_thread): Add variable 'original_url', 12034 (write_xattr_metadata): New function, 12035 (set_file_metadata): New function 12036 * src/wget_job.h: Add variable 'original_url' 12037 * src/wget_options.h: Add new config 'xattr' 12038 * src/wget_xattr.h: Add new function declaration 'set_file_metadata', 12039 Add fsetxattr() if defined in operating system 12040 12041 These attributes provide a lightweight method of later determining 12042 where a file was downloaded from. Adapted from Wget 1.x. 12043 12044 This patch changes: 12045 * autoconf detects whether extended attributes are available and 12046 enables the code if they are. 12047 * The new flags --xattr and --xattr=no control whether xattr is enabled. 12048 * The original and redirected URLs are recorded as shown below. 12049 * This works for both single fetches and recursive mode. 12050 12051 The attributes that are set are: 12052 user.xdg.origin.url: The URL that the content was fetched from. 12053 user.xdg.referrer.url: The URL that was originally requested. 12054 user.mime_type: Two-part identifier for file formats and format contents. 12055 user.charset: File encoding. 12056 12057 For example, where http://lwn.net redirects to https://lwn.net: 12058 $ wget2 --xattr http://lwn.net 12059 ... 12060 $ getfattr -d index.html 12061 user.charset:"UTF-8" 12062 user.mime_type:="text/html" 12063 user.xdg.origin.url="https://lwn.net/" 12064 user.xdg.referrer.url="http://lwn.net/" 12065 120662017-05-24 Tim Rühsen <tim.ruehsen@gmx.de> 12067 12068 Fix build for non-Windows envs without poll() 12069 12070 * bootstrap.conf: Add module msvc-nothrow 12071 * libwget/io.c: Include msvc-nothrow.h on _WIN32, 12072 (wget_ready_2_transfer): Fix code for non-Windows envs 12073 120742017-05-24 Tim Rühsen <tim.ruehsen@gmx.de> 12075 12076 MinGW cleanup 12077 12078 * include/wget/wget.h: Add const to first param of wget_restrict_file_name 12079 * libwget/utils.c: Likewise 12080 * tests/libtest.c: Remove debug output, start/stop ftp server thread, 12081 popen() without 'b'. 12082 * tests/test-iri.c: Skip test on _WIN32 12083 120842017-05-23 Tim Rühsen <tim.ruehsen@gmx.de> 12085 12086 Fix more tests for MinGW build 12087 12088 * include/wget/wget.h: Fixed indentation 12089 * tests/libtest.h (wget_test_file_t): Add restricted_mode member 12090 * tests/libtest.c (_scan_for_unexpected): Respect 'restricted_mode' value, 12091 (wget_test): Likewise. 12092 * test-iri-percent.c: Set WGET_RESTRICT_NAMES_NOCONTROL to expected filename 12093 * tests/test-iri.c: Likewise 12094 * tests/test-iri-subdir.c: Skip test on Windows build 12095 120962017-05-23 Tim Rühsen <tim.ruehsen@gmx.de> 12097 12098 * contrib/mingw: Fix WINEPATH and ./configure options 12099 121002017-05-23 Tim Rühsen <tim.ruehsen@gmx.de> 12101 12102 * src/wget.c (_get_header): Restrict file name from Content-Disposition 12103 121042017-05-23 Tim Rühsen <tim.ruehsen@gmx.de> 12105 12106 * libwget/utils.c (wget_restrict_file_name): Explicitely use signed char 12107 121082017-05-23 Tim Rühsen <tim.ruehsen@gmx.de> 12109 12110 Fix test-directory-clash on MinGW+Wine 12111 12112 * ../src/wget.c (_prepare_file): Check for EACCESS in unique loop 12113 12114 Reported-by: Akash Rawal 12115 121162017-05-23 Tim Rühsen <tim.ruehsen@gmx.de> 12117 12118 * tests/libtest.c (wget_test): Sync output streams to avoid garbled output 12119 121202017-05-23 Tim Rühsen <tim.ruehsen@gmx.de> 12121 12122 Make _restrict_file_name() public as wget_restrict_file_name() 12123 12124 * include/wget/wget.h: Add wget_restrict_file_name() declaration, 12125 add WGET_RESTRICT_* defines. 12126 * libwget/utils.c: Add wget_restrict_file_name() definition. 12127 * src/options.c (parse_restrict_names): Use WGET_RESTRICT_*, 12128 default to WGET_RESTRICT_NAMES_WINDOWS in _WIN32. 12129 * src/wget.c: Remove restrict_file_name(), 12130 (get_local_filename): Use wget_restrict_file_name(). 12131 * src/wget_options.h: Remove RESTRICT_NAMES_* defines. 12132 * tests/libtest.c (_scan_for_unexpected): Use wget_restrict_file_name(), 12133 (wget_test): Use wget_restrict_file_name(). 12134 12135 Reported-by: Akash Rawal 12136 121372017-05-22 Akash Rawal <argmlac@gmail.com> 12138 12139 Disabled /dev/full test on non-linux systems 12140 12141 * tests/test-wget-1.c: Test full disc only if __linux__ is defined 12142 121432017-05-22 Akash Rawal <argmlac@gmail.com> 12144 12145 Fix detecting directories on _WIN32 12146 12147 * src/wget.c (_prepare_file): Check for dir on errno = EACCES 12148 12149 _WIN32 set errno=EACCES on open(<directory>) while Linux sets 12150 errno=EISDIR. 12151 121522017-05-22 Akash Rawal <argmlac@gmail.com> 12153 12154 Fix test--accept.c and test--follow-tags.c on Win32 12155 12156 * tests/test--accept.c: Use " instead of ' on command line 12157 * tests/test--follow-tags.c: Remove unneeded ' on command line 12158 12159 Win32 doesn't like 'xxx' on the command line, wants "xxx". 12160 121612017-05-20 Akash Rawal <argmlac@gmail.com> 12162 12163 * tests/libtest.c (_write_msg): Flush stderr 12164 12165 Without flushing, the output messages are garbled on 12166 MinGW / Win32 builds. 12167 121682017-05-22 Akash Rawal <argmlac@gmail.com> 12169 12170 Fix select() for _WIN32 12171 12172 * libwget/io.c: Include io.h and winsock2.h, 12173 use OS socket descriptor for select() 12174 121752017-05-20 Tim Rühsen <tim.ruehsen@gmx.de> 12176 12177 * tests/libtest.c: Start/Stop HTTPS server again 12178 121792017-05-20 Tim Rühsen <tim.ruehsen@gmx.de> 12180 12181 Open files in binary mode where needed 12182 12183 * examples/websequencediagram.c: Use binary mode on file open 12184 * libwget/css.c: Likewise 12185 * libwget/hashfile.c: Likewise 12186 * libwget/http_highlevel.c: Likewise 12187 * libwget/io.c: Likewise 12188 * libwget/xml.c: Likewise 12189 * src/job.c: Likewise 12190 * src/options.c: Likewise 12191 * src/wget.c: Likewise 12192 * tests/libtest.c: Likewise 12193 * tests/stringmap_perf.c: Likewise 12194 12195 This fixes discrepancies between stat/st_size and read bytes on 12196 MinGW / Wine / Windows. 12197 121982017-05-19 Tim Rühsen <tim.ruehsen@gmx.de> 12199 12200 Let test suite finish with MinGW + Wine 12201 12202 * contrib/mingw: Amend the MinGW recipe 12203 * libwget/io.c (wget_ready_2_transfer): Wait 10ms before select() on MinGW 12204 * tests/libtest.c: Make 'terminate' a volatile sig_atomic_t. 12205 (_http_server_thread): Add debug output. 12206 (_empty_directory): Fix recursive removing directories. 12207 (wget_test_stop_server): Use wget_thread_cancel() instead of 12208 pthread_kill() + wget_thread_join(). 12209 122102017-05-17 Akash Rawal <argmlac@gmail.com> 12211 12212 * tests/libtest.c: Remove stray EXEEXT from the command line construction 12213 122142017-05-15 Tim Rühsen <tim.ruehsen@gmx.de> 12215 12216 * libwget/Makefile.am: Use convenience library to avoid unused macro warnings 12217 122182017-05-15 Tim Rühsen <tim.ruehsen@gmx.de> 12219 12220 * .gitlab-ci.yml: Remove explicit 'make syntax-check' 12221 122222017-05-15 Tim Rühsen <tim.ruehsen@gmx.de> 12223 12224 * Makefile.am: Add 'make syntax-check' to 'make check' 12225 122262017-05-15 Tim Rühsen <tim.ruehsen@gmx.de> 12227 12228 Remove nonnull attribute from some functions 12229 12230 * include/wget/wget.h: Remove nonnull attribute from some functions 12231 * libwget/encoding.c: Add explicit checks for NULL 12232 * libwget/io.c: Likewise 12233 * libwget/pipe.c: Likewise 12234 * libwget/strlcpy.c: Likewise 12235 * libwget/utils.c: Likewise 12236 122372017-05-12 Gianluca Varisco <gianluca.varisco@gmail.com> 12238 12239 * README: Fixed typo "Accept-Encdoing" -> "Accept-Encoding" 12240 122412017-05-12 Akash Rawal <argmlac@gmail.com> 12242 12243 Remove redundant VGSUPPFILE macro 12244 12245 * tests/Makefile.am: Remove VGSUPPFILE macro from libtest.la 12246 * tests/libtest.c: Replace VGSUPPFILE with an expression using SRCDIR 12247 122482017-05-03 Vijo Cherian <codervijo@gmail.com> 12249 12250 Moved help strings into options structure 12251 12252 * src/options.c: New enum help_section_t, 12253 new functions print_first() and print_next(), 12254 moved all help strings to struct options. 12255 12256 Print help strings such that it fits in 80 columns 12257 122582017-05-03 Avinash Sonawane <rootkea@gmail.com> 12259 12260 * src/options.c (set_long_option): Remove redundant bsearch call 12261 122622017-05-03 Avinash Sonawane <rootkea@gmail.com> 12263 12264 * libwget/http.c: Reorder switch cases in ascending order 12265 122662017-05-03 Avinash Sonawane <rootkea@gmail.com> 12267 12268 * libwget/http.c: Implement Wayback Machine's X-Archive-Orig-last-modified HTTP header 12269 122702017-05-03 Akash Rawal <argmlac@gmail.com> 12271 12272 Make wget_http_connection_t an abstract type 12273 12274 * include/wget/wget.h: Convert wget_http_connection_t to forward declaration 12275 * libwget/http.h: Move wget_http_connection_t definition here 12276 * libwget/Makefile.am: Add new file http.h 12277 * libwget/http.c: Include http.h and add accessor functions 12278 * libwget/http_highlevel.c: Include http.h 12279 * src/wget.c: Replace access to wget_http_connection_t with accessor function calls 12280 122812017-05-03 Tim Rühsen <tim.ruehsen@gmx.de> 12282 12283 * libwget/xalloc.c: Satisfy syntax-check 12284 122852017-05-03 Tim Rühsen <tim.ruehsen@gmx.de> 12286 12287 * examples/getstream.c (header_callback): Use wget_malloc() instead of malloc() 12288 122892017-05-03 Tim Rühsen <tim.ruehsen@gmx.de> 12290 12291 Add new function wget_free() 12292 12293 * examples/check_url_types.c: Use wget_xfree() instead of free() 12294 * examples/http_get2.c: Likewise 12295 * examples/print_css_urls2.c: Likewise 12296 * examples/print_html_urls.c: Likewise 12297 * include/wget/wget.h: Add wget_free() and define wget_xfree() 12298 * libwget/xalloc.c: Implement wget_free() 12299 12300 * include/wget/wget.h: Add declaration for wget_free() 12301 * libwget/xalloc.c: Implement wget_free() 12302 123032017-05-03 Tim Rühsen <tim.ruehsen@gmx.de> 12304 12305 * .gitlab-ci.yml: Enable syntax-check 12306 123072017-05-03 Tim Rühsen <tim.ruehsen@gmx.de> 12308 12309 * src/options.c: Remove trailing whitespace 12310 123112017-05-02 Avinash Sonawane <rootkea@gmail.com> 12312 12313 * tests/libtest.c: Use --no-proxy for wget2 invocation in test-suite 12314 123152017-05-02 Avinash Sonawane <rootkea@gmail.com> 12316 12317 Implement `--proxy` switch to import env proxy vars 12318 12319 * src/wget_options.h: Add `--proxy` option 12320 * src/options.c(init): Import env proxy var by default and only if no 12321 corresponding proxy variable is specified on command-line 12322 123232017-05-03 Akash Rawal <argmlac@gmail.com> 12324 12325 * src/options.c: Implement optional arguments and separate handling for --no-<option> 12326 123272017-05-02 Tim Rühsen <tim.ruehsen@gmx.de> 12328 12329 *libwget/strlcpy.c: Fix wget_strlcpy on OSX / BSD 12330 123312017-05-02 Tim Rühsen <tim.ruehsen@gmx.de> 12332 12333 Rename strlcpy to wget_strlcpy 12334 12335 * include/wget/wget.h: Rename and remove gnulib deps. 12336 * libwget/strlcpy.c: Rename function and fall back to strlcpy. 12337 * */*.c: Rename strlcpy to wget_strlcpy. 12338 123392017-05-02 Tim Rühsen <tim.ruehsen@gmx.de> 12340 12341 Amend --http-proxy and --https-proxy behavior 12342 12343 * libwget/http.c (_parse_proxies): Alloc vector on demand 12344 * src/options.c (wget_http_set_http_proxy): Return # of proxies 12345 successfully parsed, 12346 (wget_http_set_https_proxy). Likewise. 12347 * src/options.c (init): Empty and undefined proxy variable is the same. 12348 * tests/test.c (test_set_proxy): Amend tests. 12349 123502017-05-02 Avinash Sonawane <rootkea@gmail.com> 12351 12352 Remove unnecessary `while` loop & add test 12353 12354 * libwget/http.c (_parse_proxies): Remove non-required `while` 12355 * tests/test.c (test_set_proxy): Add test with leading whitespaces 12356 123572017-05-01 Avinash Sonawane <rootkea@gmail.com> 12358 12359 Add tests for wget_http_set_http(s)_proxy() functions 12360 12361 * tests/test.c (test_set_proxy): Add tests for wget_http_set_http_proxy() 12362 * tests/test.c (test_set_proxy): Add tests for wget_http_set_https_proxy() 12363 * libwget/http.c (wget_http_set_http_proxy): Return length of http_proxies if no parse error 12364 * libwget/http.c (wget_http_set_https_proxy): Return length of https_proxies if no parse error 12365 123662017-05-01 Tim Rühsen <tim.ruehsen@gmx.de> 12367 12368 * src/wget.c: Remove WIN32 futimens() implementation 12369 12370 The newest gnulib brings WIN32 futimens(). 12371 123722017-05-01 Tim Rühsen <tim.ruehsen@gmx.de> 12373 12374 *gnulib: Update 12375 123762017-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 12377 12378 Fix syntax-check 'sc_prohibit_have_config_h' 12379 12380 * cfg.mk: Remove sc_prohibit_have_config_h from local-checks-to-skip 12381 * libwget/*.c: Include <config.h> unconditionally 12382 * src/*.c: Likewise 12383 * tests/*.c: Likewise 12384 123852017-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 12386 12387 Remove useless parens (syntax-check) 12388 12389 * include/wget/wget.h: Remove useless parens 12390 * libwget/ssl_gnutls.c: Likewise 12391 123922017-04-30 Tim Rühsen <tim.ruehsen@gmx.de> 12393 12394 Remove gnulib dependencies from examples and wget.h 12395 12396 * cfg.mk: Add examples as excludes for config.h 12397 * examples/Makefile.am: Remove -I to srcdir and builddir 12398 * examples/*.c: Remove #include <config.h> 12399 * include/wget/wget.h: Remove gettext include and defines 12400 * libwget/private.h: Add gettext include and defines 12401 * src/wget_main.h: Likewise 12402 * tests/libtest.h: Likewise 12403 * src/log.c: Add #include <string.h> 12404 124052017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12406 12407 * cfg.mk: New file with rules for 'make syntax-check' 12408 124092017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12410 12411 Remove useless cpp parens (syntax-check) 12412 124132017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12414 12415 Remove trailing whitespaces (syntax-check) 12416 124172017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12418 12419 Fix space-tab sequences (syntax-check) 12420 124212017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12422 12423 * libwget/net.c: Satisfy sc_require_config_h (syntax-check) 12424 124252017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12426 12427 * bootstrap: Update to version 19 12428 124292017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12430 12431 Remove == from scripts (syntax-check) 12432 12433 * .travis.sh: == -> = 12434 * .travis_setup.sh: Likewise 12435 * contrib/gen_option_diff.sh: Likewise 12436 124372017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12438 12439 Do not include unused strings.h 12440 12441 * examples/getstream.c: Remove #include <strings.h> 12442 * libwget/css.c: Likewise 12443 * libwget/encoding.c: Likewise 12444 * libwget/hashfile.c: Likewise 12445 * libwget/metalink.c: Likewise 12446 * libwget/net.c: Likewise 12447 * libwget/robots.c: Likewise 12448 * src/wget.c: Likewise 12449 124502017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12451 12452 * src/options.c: Use EXIT_SUCCESS (syntax-check) 12453 124542017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12455 12456 * src/wget.c: Remove double colons (syntax-check) 12457 124582017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12459 12460 * po/POTFILES.in: Update (syntax-check) 12461 124622017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12463 12464 Spell filesystem 'file system' (syntax-check) 12465 12466 * docs/wget2_manual.md: filesystem -> file system 12467 * tests/libtest.c: Likewise 12468 * tests/libtest.h: Likewise 12469 * tests/test-wget-1.c: Likewise 12470 124712017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12472 12473 * libwget/net.c: Use O_NONBLOCK instead of O_NDELAY (syntax-check) 12474 124752017-04-29 Tim Rühsen <tim.ruehsen@gmx.de> 12476 12477 * configure.ac: Add --disable-doc / --enable-doc (default) 12478 124792017-04-27 Darshit Shah <darnir@gnu.org> 12480 12481 * libwget/bar.c: Allocate all memory for progress display together 12482 124832017-04-27 Darshit Shah <darnir@gnu.org> 12484 12485 Prevent tiny memory allocations when globbing 12486 12487 * libwget/utils.c: Rename wget_strnglob to wget_strglob. It now calls 12488 glob on the entire string. 12489 * include/wget/wget.h: Change protoype of wget_strnglob 12490 * src/options.c: Change calls to wget_strnglob to use new wget_strglob 12491 calls 12492 124932017-04-27 Darshit Shah <darnir@gnu.org> 12494 12495 Improve the benchmark scripts slightly 12496 124972017-04-26 Tim Rühsen <tim.ruehsen@gmx.de> 12498 12499 Add error handler for decompression 12500 12501 * include/wget/wget.h: New functions wget_decompress_set_error_handler() 12502 and wget_decompress_get_context(). 12503 * libwget/decompressor.c: New functions wget_decompress_set_error_handler() 12504 and wget_decompress_get_context(). 12505 * libwget/http.c: Implement and set decompressor error handler. 12506 125072017-04-13 Michael Heerklotz <michael.heerklotz@check24.de> 12508 12509 Added a more detailed logging for HTTP errors 12510 * src/wget.c: More detailed logging for HTTP response code 12511 125122017-04-25 Tim Rühsen <tim.ruehsen@gmx.de> 12513 12514 * src/options.c: Remove double 'help' option 12515 125162017-04-24 Tim Rühsen <tim.ruehsen@gmx.de> 12517 12518 * libwget/encoding.c (wget_str_to_ascii): Fall back to TR46 transitional on error 12519 125202017-04-24 Tim Rühsen <tim.ruehsen@gmx.de> 12521 12522 * .gitlab-ci.yml: Create coverage report and online docs 12523 125242017-04-24 Darshit Shah <darnir@gnu.org> 12525 12526 Updated CI configuration 12527 125282017-04-25 Tim Rühsen <tim.ruehsen@gmx.de> 12529 12530 * README.md: Update 12531 125322017-04-23 Tim Rühsen <tim.ruehsen@gmx.de> 12533 12534 Add Gitlab CI (Debian) 12535 12536 * .gitlab-ci.yml: New file 12537 125382017-04-22 Darshit Shah <darnir@gnu.org> 12539 12540 * libwget/buffer_printf.c: Cast output of va_arg correctly 12541 125422017-04-21 Avinash Sonawane <rootkea@gmail.com> 12543 12544 * src/options.c (print_version): Remove redundant preprocessor check 125452017-04-21 Tim Rühsen <tim.ruehsen@gmx.de> 12546 12547 Fix --chunk-size with missing Content-Length 12548 12549 * src/wget.c (process_head_response): Don't release job after HEAD 12550 * tests/libtest.c: Implement WGET_TEST_SERVER_SEND_CONTENT_LENGTH 12551 * tests/libtest.h: Define WGET_TEST_SERVER_SEND_CONTENT_LENGTH 12552 * tests/test-wget-1.c: Add new tests for --chunk-size 12553 125542017-04-13 Tim Rühsen <tim.ruehsen@gmx.de> 12555 12556 * src/job.c (job_validate_file): Fix --chunk-size for partially existing files 12557 125582017-04-20 Tim Rühsen <tim.ruehsen@gmx.de> 12559 12560 Hide wget_cookie_st 12561 12562 * include/wget/wget.h: Remove struct wget_cookie_st declaration, 12563 add wget_cookie_to_setcookie(), 12564 add wget_cookie_parse_setcookie(), 12565 change wget_http_parse_setcookie() cookie param. 12566 * libwget/cookie.c: Add struct wget_cookie_st declaration, 12567 (wget_cookie_init): Use xcalloc(), 12568 (wget_cookie_to_setcookie): New function, 12569 (wget_cookie_parse_setcookie): New function, 12570 (_wget_cookie_normalize_cookie): Fix host_only flag. 12571 * libwget/http.c (wget_http_parse_setcookie): Code moved to 12572 wget_http_parse_setcookie(), 12573 (wget_http_parse_response_header): Amend cookie handling. 12574 * tests/test.c (test_cookies): Amend test code and test data. 12575 125762017-04-17 Darshit Shah <darnir@gnu.org> 12577 12578 * src/options.c: Handle --version similar to --help 12579 125802017-04-18 Michael Heerklotz <michael.heerklotz@check24.de> 12581 12582 Ignore all anchor links instead of only '#' 12583 12584 * examples/check_url_types.c: Modified _normalize_uri() to ignore all anchor links 12585 * examples/print_css_urls2.c: Removed check for anchor links 12586 * src/wget.c: Modified _normalize_uri() and _convert_links() to ignore all anchor links 12587 125882017-04-19 Tim Rühsen <tim.ruehsen@gmx.de> 12589 12590 Fix spelling issues found by ka7/misspell_fixer 12591 12592 * include/wget/wget.h: Fix typo 12593 * libwget/base64.c: Likewise 12594 * libwget/http.c: Likewise 12595 * src/wget.c: Likewise 12596 125972017-04-18 Tim Rühsen <tim.ruehsen@gmx.de> 12598 12599 * README.md: Mention 'bash ./bootstrap' 12600 126012017-04-16 Tim Rühsen <tim.ruehsen@gmx.de> 12602 12603 * libwget/encoding.c (wget_memiconv): Treat non-reversible conversion as error 12604 126052017-04-15 Tim Rühsen <tim.ruehsen@gmx.de> 12606 12607 Fix -c --chunk-size if the file is already complete 12608 12609 * src/wget.c (process_response_header): Check for status 416 12610 * tests/test-wget-1.c: Added 2 tests 12611 126122017-04-15 Tim Rühsen <tim.ruehsen@gmx.de> 12613 12614 * src/wget.c (_body_callback_context): Change size of max_memory and length to 64bit 12615 126162017-04-14 Tim Rühsen <tim.ruehsen@gmx.de> 12617 12618 * benchmarks/bench_https_http*: Add warmup runs 12619 126202017-04-13 Michael Heerklotz <michael.heerklotz@check24.de> 12621 12622 Added settings --cut-url-get-vars and --cut-file-get-vars and fixed --backups 12623 12624 * docs/wget2_manual.md: Added doc for the new settings 12625 * src/options.c: Fixed --backups and added --cut-url-get-vars and --cut-file-get-vars 12626 * src/wget.c: Added --cut-url-get-vars and --cut-file-get-vars 12627 * src/wget_options.h: Added --cut-url-get-vars and --cut-file-get-vars 12628 * tests/Makefile.am: Added test-cut-get-vars 12629 * test-cut-get-vars.c: New file 12630 126312017-04-13 Tim Rühsen <tim.ruehsen@gmx.de> 12632 12633 * libwget/html_url.c (_css_parse_uri): Fix -Wunused-param 12634 126352017-04-13 Michael Heerklotz <michael.heerklotz@check24.de> 12636 12637 Fixed crash in convert-links code introduced with html css parsing 12638 12639 * include/wget/wget.h: Removed unused variable. 12640 * libwget/html_url.c: Updated code to always return pointer inside "html" variable. 12641 * test/test-k.c: Updated test to cover the bug. 12642 126432017-04-13 Tim Rühsen <tim.ruehsen@gmx.de> 12644 12645 * libwget/html_url.c: Remove empty _css_parse_encoding() 12646 126472017-04-13 Darshit Shah <darnir@gnu.org> 12648 12649 * libwget/html_url.c: Remove some trailing whitespaces 12650 126512017-04-12 Michael Heerklotz <michael.heerklotz@check24.de> 12652 12653 Added support for html css style tag and attribute 12654 12655 * include/wget/wget.h: Add 'size_t len' to wget_css_parse_buffer() 12656 and wget_css_get_urls() 12657 * libwget/css.c (wget_css_parse_buffer): Use yy_scan_bytes() instead 12658 of yy_scan_string() 12659 * libwget/css_url.c: Add param to wget_css_get_urls() and 12660 wget_css_parse_buffer() 12661 * libwget/html_url.c: Add _css_parse_encoding() and _css_parse_uri(), 12662 (_html_get_url): Deal with 'style' attribute, 12663 (wget_html_free_urls_inline): Free url if needed 12664 * libwget/test_linking.c: Add param to wget_css_parse_buffer() 12665 * libwget/xml.c (parseXML): Add check for 'style' 12666 * src/wget.c (css_parse): Add param 'size_t len' 12667 * tests/Makefile.am: Add new test test-parse-html-css 12668 * tests/test-parse-html-css.c: New file 12669 126702017-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 12671 12672 Follow link:rel=preload with --page-requisites 12673 12674 * libwget/html_url.c (_html_get_url): Add rel=preload to be followed 12675 * tests/test--page-requisites.c: Add a test href for rel=preload 12676 126772017-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 12678 12679 Fix --page-requisites <link> handling 12680 12681 * libwget/html_url.c: Add helper ctx->uri_index, 12682 fix "icon shortcut" -> "shortcut icon", 12683 fix check for "stylesheet" 12684 * src/wget.c (html_parse): Check for !html_url->link_inline 12685 * tests/test--page-requisites.c: Add 'not.txt' URL and reference 12686 126872017-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 12688 12689 Add special HTML to test--page-requisites.c 12690 12691 * tests/test--page-requisites.c: Add HTML code 12692 12693 With --page-requisites shortcut icons should not be followed. 12694 Link with rel="stylesheet" should be followed. 12695 126962017-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 12697 12698 Fix progress bar when off_t is 32 bits 12699 12700 * include/wget/wget.h: Amend prototype of wget_human_readable() 12701 * libwget/bar.c: Use uint64_t instead of off_t 12702 * libwget/utils.c (wget_human_readable): Change 3rd param off_t -> uint64_t 12703 127042017-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 12705 12706 * libwget/init.c (wget_global_init): Init console 12707 127082017-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 12709 12710 * libwget/console.c (_reset_color): Do not reset color if stdout is not a TTY 12711 127122017-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 12713 12714 * tests/test-cut-dirs.c: Add more tests 12715 127162017-04-12 Tim Rühsen <tim.ruehsen@gmx.de> 12717 12718 * src/wget.c (_prepare_file): Case-insensitive check for WIN32 NUL file 12719 127202017-04-12 Michael Heerklotz <michael.heerklotz@check24.de> 12721 12722 Fix --cut-dirs 12723 12724 * src/wget.c (get_local_filename): Add / after host part 12725 * tests/Makefile.am: Add new test 'test-cut-dirs' 12726 * tests/test-cut-dirs.c: New file 12727 127282017-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 12729 12730 * Makefile.am: Add empty line to ChangeLog after date/author 12731 127322017-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 12733 12734 * tests/libtest.c (_ftp_server_thread): Remove redundant check for WIN64 12735 127362017-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 12737 12738 * libwget/pipe.c: Remove redundant check for WIN64 12739 127402017-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 12741 12742 Fix handling of -O /dev/null 12743 12744 * src/wget.c (_prepare_file): Return -2 if filename is /dev/null, 12745 (http_receive_response): Check outfd for >= 0 12746 12747 Reported-by: Gisle Vanem 12748 127492017-04-11 Darshit Shah <darnir@gnu.org> 12750 12751 Prevent wget2rc from being read during tests 12752 12753 * src/options.c: Treat parse_filename{,s} as string vectors and clear 12754 them with a no-* command line option 12755 * tests/libtest.c: Pass --no-config to all the tests by default 12756 127572017-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 12758 12759 * libwget/css_tokenizer.lex: Use clang #pragma for -Wshorten-64-to-32 12760 127612017-04-11 Tim Rühsen <tim.ruehsen@gmx.de> 12762 12763 Fix --chunk-size / -O combination 12764 12765 * src/wget.c (process_head_response): Set metalink->name if 12766 config.output_document is set. 12767 12768 Reported-by: Gisle Vanem 12769 Fixes: #177 12770 127712017-04-10 Tim Rühsen <tim.ruehsen@gmx.de> 12772 12773 Generate prettier ChangeLog from git log 12774 12775 * Makefile.am (gen_changelog): Use git log directly without 12776 additional gnulib tools. 12777 127782017-04-10 Tim Rühsen <tim.ruehsen@gmx.de> 12779 12780 * src/wget.c (_prepare_file): Skip saving to '/dev/null' resp. 'NUL' on WIN32 12781 127822017-04-10 Avinash Sonawane <rootkea@gmail.com> 12783 12784 * configure.ac: Add "-fPIC -pie" for Thread sanitizer 12785 127862017-04-10 Avinash Sonawane <rootkea@gmail.com> 12787 12788 * configure.ac: Fix test for asan and tsan being mutually exclusive 12789 * configure.ac: Fix test for asan and tsan being mutually exclusive 12790 * configure.ac: Make `./configure --help` document about msan and tsan being mutually exclusive 12791 127922017-04-09 Avinash Sonawane <rootkea@gmail.com> 12793 12794 * configure.ac: Check for msan and tsan being mutually exclusive 12795 127962017-04-09 Avinash Sonawane <rootkea@gmail.com> 12797 12798 * configure.ac: Check for Valgrind and Sanitizers being mutually exclusive 12799 12800 * configure.ac: Check for valgrind and Sanitizers being mutually exclusive 12801 * configure.ac: Make ./configure --help document above check 12802 * configure.ac: Make ./configure --help document the existing check of asan and msan 12803 128042017-04-08 Darshit Shah <darnir@gnu.org> 12805 12806 * docs/libwget.doxy: Fix path to libwget header file for documentation 12807 128082017-04-07 Tim Rühsen <tim.ruehsen@gmx.de> 12809 12810 * libwget/css_tokenizer.lex: Suppress clang warnings 12811 128122017-04-07 Tim Rühsen <tim.ruehsen@gmx.de> 12813 12814 * examples/check_url_types.c: Fix clang warning 12815 128162017-04-07 Tim Rühsen <tim.ruehsen@gmx.de> 12817 12818 * libwget/net.c: Support TCP Fast Open on OSX via connectx() 12819 128202017-04-07 Darshit Shah <darnir@gnu.org> 12821 12822 * configure.ac: Don't complain when exhaustive switch has default case 12823 128242017-04-07 Darshit Shah <darnir@gnu.org> 12825 12826 *libwget/http.c: Remove unused attribute from used parameters 12827 128282017-04-07 Darshit Shah <darnir@gnu.org> 12829 12830 *libwget/net.c: Silence clang's -Wconditional-uninitialized 12831 128322017-04-07 Darshit Shah <darnir@gnu.org> 12833 12834 src/options.c (parse_command_line): Silence compiler warning for precision loss 12835 128362017-04-07 Darshit Shah <darnir@gnu.org> 12837 12838 * src/log.c: Silence -Wunused-parameter when not on Windows 12839 128402017-04-07 Darshit Shah <darnir@gnu.org> 12841 12842 * configure.ac: Prevent some useless clang warnings 12843 128442017-04-07 Tim Rühsen <tim.ruehsen@gmx.de> 12845 12846 * libwget/http.c (wget_http_open): Disable push for HTTP/2 connections 12847 128482017-04-07 Tim Rühsen <tim.ruehsen@gmx.de> 12849 12850 Revert "* libwget/net.c: Silence -Wundef if TCP_FASTOPEN is undefined" 12851 12852 This reverts commit 2c17f6fc429f4e1edf78b329ff3509d1ed22f392. 12853 128542017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12855 12856 Fix libraries to use libtool variables 12857 12858 * examples/Makefile.am: Use $(LTLIBICONV) $(LTLIBINTL) $(LTLIBTHREAD) 12859 * src/Makefile.am: Likewise 12860 * tests/Makefile.am: Likewise 12861 128622017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12863 12864 * libwget/utils.c (wget_get_screen_size): Fix return value 12865 128662017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12867 12868 gnulib: Update 12869 128702017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12871 12872 * src/wget.c: Silence -Wbad-function-cast in WIN32 futimens() 12873 128742017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12875 12876 * libwget/ssl_gnutls.c: Silence -Wunused-parameter if GnuTLS isn't available 12877 128782017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12879 12880 * libwget/utils.c: Silence -Wunused-parameter if HAVE_IOCTL is undefined 12881 128822017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12883 12884 * libwget/net.c: Silence -Wundef if TCP_FASTOPEN is undefined 12885 128862017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12887 12888 * libwget/hashfile.c: Silence -Wundef from gnulib's hash code 12889 128902017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12891 12892 * libwget/cookie.c: Silence -Wunused-parameter if without LIBPSL 12893 128942017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12895 12896 * include/wget/wget.h: Silence -Wundef from gnulib's gettext.h 12897 128982017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12899 12900 * configure.ac: Reduce warnings from gnulib code 12901 129022017-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 12903 12904 gnulib: Update 12905 129062017-04-05 Darshit Shah <darnir@gnu.org> 12907 12908 * .gitignore: Update gitignore file with new tests and examples executables 12909 129102017-04-05 Darshit Shah <darnir@gnu.org> 12911 12912 * src/options.c: Ensure --hpkp-file is tilde expanded as well 12913 129142017-04-05 Tim Rühsen <tim.ruehsen@gmx.de> 12915 12916 Fix -Wundef warnings 12917 12918 * libwget/decompressor.c: #if -> #ifdef 12919 * libwget/http.c: Likewise 12920 * libwget/net.c: Likewise 12921 * src/wget.c: Likewise 12922 129232017-04-05 Tim Rühsen <tim.ruehsen@gmx.de> 12924 12925 * m4/wget_manywarnings.m4: Fix for MinGW 12926 129272017-04-05 Tim Rühsen <tim.ruehsen@gmx.de> 12928 12929 * configure.ac: Remove -Wformat when cross-compiling 12930 129312017-04-05 Darshit Shah <darnir@gnu.org> 12932 12933 * src/options.c: Add hpkp-file to output of wget2 --help 12934 129352017-04-05 Tim Rühsen <tim.ruehsen@gmx.de> 12936 12937 * src/options.c: Only read 'no_proxy' if proxy settings exist 12938 129392017-04-05 Tim Rühsen <tim.ruehsen@gmx.de> 12940 12941 Add support for 'no_proxy' environment variable 12942 12943 * include/wget/wget.h: Add flag 'proxied' to wget_http_connection_t, 12944 add arg 'proxied' to wget_http_request_to_buffer(), 12945 add new function declaration wget_http_set_no_proxy(), 12946 add new function declaration wget_http_match_no_proxy(). 12947 * libwget/http.c: Add variable 'no_proxies', 12948 (wget_http_open): Check host against 'no_proxy' and set 'conn->proxied' 12949 when appropriate, 12950 (wget_http_send_request): Call wget_http_request_to_buffer() with 12951 conn->proxied, 12952 (wget_http_request_to_buffer): Check 'proxied', 12953 (_parse_proxies): Cleanup, 12954 (_parse_no_proxies): New function, 12955 (wget_http_set_no_proxy): New function, 12956 (wget_http_match_no_proxy): New function 12957 * libwget/init.c (wget_global_deinit): Free no_proxy memory 12958 * libwget/ip.c: Add code in comments (for later use) 12959 * libwget/utils.c (wget_match_tail, wget_match_tail_nocase): Cleanup 12960 * src/options.c (init): Handle 'no_proxy' env variable 12961 * src/wget_options.h (struct config): Add no_proxy member 12962 129632017-04-04 Didik Setiawan <ds@didiksetiawan.com> 12964 12965 Fix illegal memory access when parsing null-terminated string 12966 129672017-03-28 Didik Setiawan <ds@didiksetiawan.com> 12968 12969 * tests/test.c: Add wget_robots_parse unit test 12970 129712017-04-04 Tim Rühsen <tim.ruehsen@gmx.de> 12972 12973 Add futimens() implementation for _WIN32 12974 12975 * src/wget.c: Add futimens() if _WIN32 is defined 12976 12977 Reported-by: Gisle Vanem 12978 Fixes: #167 12979 129802017-04-04 Avinash Sonawane <rootkea@gmail.com> 12981 12982 Use time_t instead of long 12983 12984 * include/wget/wget.h : Use time_t instead of long 12985 12986 Reported-by : Gisle Vanem 12987 129882017-04-04 Avinash Sonawane <rootkea@gmail.com> 12989 12990 Use https:// instead of git:// 12991 12992 * .gitmodule: Use https:// instead of git:// to support proxy environments 129932017-04-02 Akash Rawal <argmlac@gmail.com> 12994 12995 * libwget/http.c: Avoid signed integer overflow due to intermediate ptrdiff_t by casting operands to uintptr_t 12996 129972017-03-31 Tim Rühsen <tim.ruehsen@gmx.de> 12998 12999 * README.md: Add section about clang's CFI 13000 130012017-03-30 Tim Rühsen <tim.ruehsen@gmx.de> 13002 13003 * tests/test-metalink.c: Free allocated memory to satisfy leak sanitizer 13004 130052017-03-30 Tim Rühsen <tim.ruehsen@gmx.de> 13006 13007 Fix --enable-fsanitize-msan for gcc 13008 13009 * configure.ac: Add -fsanitize=leak if available 13010 130112017-03-30 Tim Rühsen <tim.ruehsen@gmx.de> 13012 13013 Fix logical operator in brotli code 13014 13015 * libwget/decompressor.c (brotli_decompress): Fix expression 13016 13017 Reported-by: Coverity Scan (cid: 165234) 13018 130192017-03-30 Tim Rühsen <tim.ruehsen@gmx.de> 13020 13021 Fix possible NULL ptr dereference 13022 13023 * src/wget.c (process_response): Check value for wget_metalink_parse() 13024 13025 Reported-by: Coverity Scan (cid: 149094) 13026 130272017-03-30 Tim Rühsen <tim.ruehsen@gmx.de> 13028 13029 Fix VPATH issue for test suite 13030 13031 * tests/Makefile.am: Set SRCDIR to $(abs_srcdir) 13032 * tests/libtest.c: Remove '../' in front of SRCDIR 13033 * tests/test-i-https.c: Likewise 13034 13035 Reported-by: Akash Rawal 13036 130372017-03-29 Akash Rawal <argmlac@gmail.com> 13038 13039 Fix path of valgrind-suppressions in vpath build 13040 13041 * tests/Makefile.am: For libtests.la, define VGSUPPFILE to absolute path to 13042 tests/valgrind-suppressions 13043 * tests/libtest.c: Use VGSUPPFILE macro instead of hardcoded path to 13044 valgrind-suppressions 13045 130462017-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 13047 13048 Fix documentation of wget_vaprintf and wget_aprintf 13049 13050 * libwget/printf.c: Both functions do not return NULL. 13051 130522017-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 13053 13054 * configure.ac: Add -Wdeclaration-after-statement to unwanted options (C89 only) 13055 130562017-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 13057 13058 * m4/wget_manywarnings.m4: Call gcc instead of gcc 13059 130602017-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 13061 13062 * m4/wget_manywarings.m4: Add extra options only for gcc >= 6 13063 130642017-03-29 Avinash Sonawane <rootkea@gmail.com> 13065 13066 Use atoll() instead of atol() for time_t variables 13067 13068 * libwget/cookie.c: Use atoll() instead of atol() 13069 * libwget/hsts.c: Likewise 13070 * libwget/http.c: Likewise 13071 * libwget/ocsp.c: Likewise 13072 * libwget/tls_session.c: Likewise 13073 * libwget/hpkp.c: Change type of maxage from long to time_t 13074 130752017-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 13076 13077 * .travis.sh: Set CFLAGS to '-O0 -g' 13078 130792017-03-29 Avinash Sonawane <rootkea@gmail.com> 13080 13081 Read and write time_t as long long int 13082 13083 * libwget/cookie.c: Write time_t as long long int 13084 * libwget/hsts.c: Likewise 13085 * libwget/ocsp.c: Likewise 13086 * libwget/tls_session.c: Likewise 13087 * libwget/hpkp.c: Read and write time_t as long long int 13088 13089 Reported-By: Tim Rühsen 13090 130912017-03-28 Darshit Shah <darnir@gnu.org> 13092 13093 Prevent a NULL pointer dereference 13094 13095 * src/options.c (parse_header): Prevent derefencing a potential NULL 13096 pointer 13097 13098 Reported-by: Coverity Scan (cid: 165230) 13099 131002017-03-28 Darshit Shah <darnir@gnu.org> 13101 13102 Fix resource leak 13103 13104 * libwget/http.c (_on_header_callback): Prevent leaking memory when 13105 there is an empty HTTP2 response 13106 13107 Reported-by: Coverity Scan (cid: 165231) 13108 131092017-03-28 Darshit Shah <darnir@gnu.org> 13110 13111 Prevent resource leak 13112 13113 * src/wget.c (_normalize_uri): Prevent leaking resource urlpart when the 13114 url is just "#". 13115 13116 Reported-by: Coverity Scan (cid: 165232) 13117 131182017-03-27 Tim Rühsen <tim.ruehsen@gmx.de> 13119 13120 Skip IDNA tests when IDNA not available 13121 13122 * libwget/encoding.c: Fix #ifdef 13123 * tests/test-idn-cmd.c: Skip if IDNA not available 13124 * tests/test-idn-meta.c: Likewise 13125 * tests/test-idn-robots.c: Likewise 13126 * tests/test.c (test_iri_parse): Skip IDN test 13127 131282017-03-25 Tim Rühsen <tim.ruehsen@gmx.de> 13129 13130 * examples/check_url_types.c: Move #include <signal.h> further to the top 13131 131322017-03-26 Akash Rawal <argmlac@gmail.com> 13133 13134 Fix vpath builds 13135 13136 * examples/Makefile.am: Add -DWGETVER_FILE to AM_CPPFLAGS 13137 * libwget/Makefile.am: Likewise 13138 * src/Makefile.am: Likewise 13139 * tests/Makefile.am: Likewise 13140 * include/wget/wget.h: Include WGETVER_FILE if defined 13141 131422017-03-25 Tim Rühsen <tim.ruehsen@gmx.de> 13143 13144 Fix regression regarding chunked transfer-encoding 13145 13146 * libwget/http.c (wget_http_get_response_cb): Save data when chunk 13147 complete in one data packet. 13148 * tests/Makefile.am: Add new test 'test-chunked' 13149 * tests/test-chunked.c: New test 13150 131512017-03-23 Tim Rühsen <tim.ruehsen@gmx.de> 13152 13153 * libwget/encoding.c: Include idn-free.h on _WIN32 13154 131552017-03-23 Tim Rühsen <tim.ruehsen@gmx.de> 13156 13157 * libwget/xalloc.c (wget_calloc): Init *p explicit (_noreturn issue) 13158 131592017-03-23 Gisle Vanem <gvanem@yahoo.no> 13160 13161 * libwget/xalloc.c (wget_realloc): Initialize variable p to NULL 13162 131632017-03-23 Avinash Sonawane <rootkea@gmail.com> 13164 13165 Fix misuse of libidn & libidn2 memory 13166 13167 * libwget/encoding.c (wget_str_to_ascii): 13168 Free libidn memory with idn_free(), 13169 Free libidn2 memory with idn2_free() 13170 131712017-03-23 Tim Rühsen <tim.ruehsen@gmx.de> 13172 13173 Fix chunk overflow in HTTP download for 32bit systems 13174 13175 * libwget/http.c: Check for pointer overflow 13176 * tests/test-bad-chunk.c: Adjust tests 13177 13178 Reported-by: Dagobert Michelsen 13179 131802017-03-21 Tim Rühsen <tim.ruehsen@gmx.de> 13181 13182 * libwget/random.c: Include process.h on Windows 13183 131842017-03-21 Tim Rühsen <tim.ruehsen@gmx.de> 13185 13186 Fix console colors for Windows 13187 13188 * include/wget/wget.h: Add WGET_CONSOLE_COLOR_MAGENTA 13189 * libwget/console.c: Add WGET_CONSOLE_COLOR_MAGENTA 13190 * src/log.c: Add colors to write functions 13191 13192 Reported-by: Gisle Vanem 13193 131942017-03-21 Tim Rühsen <tim.ruehsen@gmx.de> 13195 13196 * libwget/html_url.c (_html_get_url): Fix commented info_printf line 13197 131982017-03-21 Tim Rühsen <tim.ruehsen@gmx.de> 13199 13200 * src/options.c: Do not include wordexp.h 13201 132022017-03-21 Tim Rühsen <tim.ruehsen@gmx.de> 13203 13204 * libwget/console.c (wget_console_set_fg_color): Fix WIN32 code 13205 132062017-03-21 Tim Rühsen <tim.ruehsen@gmx.de> 13207 13208 NEWS: Mention brotli compression support 13209 132102017-03-20 Tim Rühsen <tim.ruehsen@gmx.de> 13211 13212 Use IDNA2008 + TR46 non-transitional for IRIs 13213 13214 * libwget/encoding.c: Fix unistring includes, 13215 (wget_str_to_ascii): Check for libidn2 version 13216 * src/options.c (init): Remove "unistring" output for --version 13217 132182017-03-13 Tim Rühsen <tim.ruehsen@gmx.de> 13219 13220 * tests/test-k.c: Test link within <script> tag 13221 132222017-03-10 Tim Rühsen <tim.ruehsen@gmx.de> 13223 13224 Update AUTHORS and NEWS 13225 132262017-03-10 Tim Rühsen <tim.ruehsen@gmx.de> 13227 13228 Check NULL value param in wget_iri_escape() 13229 13230 * include/wget/wget.h: Remove G_GNUC_WGET_NONNULL_ALL from wget_iri_escape 13231 * libwget/iri.c (wget_iri_escape): Check 'src' for NULL value 13232 132332017-03-10 Tim Rühsen <tim.ruehsen@gmx.de> 13234 13235 * examples/check_url_types.c (html_parse): Fix possible crash 13236 132372017-03-09 Tim Rühsen <tim.ruehsen@gmx.de> 13238 13239 Check for invalid 'BASE' URLs in HTML 13240 13241 * src/wget.c (html_parse): Check if 'base' parses correctly 13242 * tests/test-base.c: Add test with invalid base URL 13243 132442017-03-08 Tim Rühsen <tim.ruehsen@gmx.de> 13245 13246 * libwget/tls_session.c: Make code similar to hsts.c 13247 132482017-03-08 Tim Rühsen <tim.ruehsen@gmx.de> 13249 13250 * libwget/hpkp.c: Make code similar to hsts.c 13251 132522017-03-08 Tim Rühsen <tim.ruehsen@gmx.de> 13253 13254 * libwget/hsts.c: Be compliant with Wget 1.x 13255 132562017-03-08 Tim Rühsen <tim.ruehsen@gmx.de> 13257 13258 * libwget/hpkp.c (wget_hpkp_db_add): NULLify freed pointer 13259 132602017-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 13261 13262 * libwget/ssl_gnutls.c: Improve msg for invalid OCSP stapling status 13263 132642017-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 13265 13266 Fix .netrc parsing 13267 13268 * include/wget/wget.h (_wget_netrc_db_st): Rename .key to .host 13269 * libwget/netrc.c (wget_netrc_db_add): Fix parsing 13270 * tests/test.c: New function test_netrc() 13271 132722017-03-07 Tim Rühsen <tim.ruehsen@gmx.de> 13273 13274 * src/wget.c (process_response_header): Fix formatting 13275 132762017-03-05 Tim Rühsen <tim.ruehsen@gmx.de> 13277 13278 * libwget/ssl_gnutls.c (_verify_certificate_callback): Use GnuTLS error printing 13279 132802017-03-05 Tim Rühsen <tim.ruehsen@gmx.de> 13281 13282 * libwget/hpkp.c (_hpkp_db_load): Fix parsing host entries 13283 132842017-03-04 Tim Rühsen <tim.ruehsen@gmx.de> 13285 13286 * tests/test.c: Remove call of wget_hpkp_set_port() 13287 132882017-03-04 Tim Rühsen <tim.ruehsen@gmx.de> 13289 13290 Remove port from HPKP code 13291 13292 * include/wget/wget.h: Remove wget_hpkp_set_port() 13293 * libwget/hpkp.c (struct _wget_hpkp_st): Remove 'port' 13294 * src/wget.c (process_response_header): Do not call wget_hpkp_set_port() 13295 132962017-03-03 Tim Rühsen <tim.ruehsen@gmx.de> 13297 13298 Add examples/check_url_types.c 13299 13300 * examples/Makefile.am: Add rule for examples/check_url_types 13301 * examples/check_url_types.c: New file 13302 13303 This adds a Alexa top-X scanner to find out the numbers of 13304 http:// links in landing pages retrieved via https://. 13305 It follows and counts redirections separated by secure and insecure 13306 ones, flags if the landing page was completely secure, counts links 13307 in the landing HTML page (same-page, external, http, https). 13308 133092017-03-03 Tim Rühsen <tim.ruehsen@gmx.de> 13310 13311 * libwget/http_highlevel.c (wget_http_get): Do not loop forever on repeating 401 responses 13312 133132017-03-03 Tim Rühsen <tim.ruehsen@gmx.de> 13314 13315 * libwget/ssl_gnutls.c (_verify_certificate_callback): Fix #ifdefs 13316 133172017-03-03 Tim Rühsen <tim.ruehsen@gmx.de> 13318 13319 Fix crash on unexpected server PUSH PROMISE (http/2) 13320 13321 * libwget/http.c (_on_frame_recv_callback): Check for valid context, 13322 (_on_header_callback): Likewise 13323 133242017-03-03 Tim Rühsen <tim.ruehsen@gmx.de> 13325 13326 Allow new 'expires' format for Set-Cookie header 13327 13328 * libwget/http.c (wget_http_parse_full_date): Allow format 13329 '1 Mar 2027 09:23:12 GMT' 13330 * tests/test.c (test_cookies): Test new format 13331 133322017-03-01 Tim Rühsen <tim.ruehsen@gmx.de> 13333 13334 * libwget/http.c (_on_frame_recv_callback): Fix decompressor init 13335 133362017-03-01 Tim Rühsen <tim.ruehsen@gmx.de> 13337 13338 * libwget/init.c (wget_global_deinit): Free more resources 13339 133402017-03-01 Tim Rühsen <tim.ruehsen@gmx.de> 13341 13342 * gnulib: Update 13343 133442017-02-28 Tim Rühsen <tim.ruehsen@gmx.de> 13345 13346 Fix URL encoding issue in requests 13347 13348 * libwget/iri.c (wget_iri_escape_query): Do not escape ampersand, 13349 (wget_iri_get_escaped_resource): Use wget_iri_escape_query() 13350 * tests/test-base.c: Add test URLs with spacing in path, query and param 13351 133522017-02-28 Tim Rühsen <tim.ruehsen@gmx.de> 13353 13354 Fix URLs to HTTPS where possible 13355 133562017-02-28 Tim Rühsen <tim.ruehsen@gmx.de> 13357 13358 * examples/websequencediagram_high.c: Use https URLs 13359 133602017-02-28 Tim Rühsen <tim.ruehsen@gmx.de> 13361 13362 * libwget/http.c (wget_http_get_response): Check if body member is set 13363 133642017-02-28 Tim Rühsen <tim.ruehsen@gmx.de> 13365 13366 * examples/websequencediagram.c: Use HTTPS, fix functionality 13367 133682017-02-28 Tim Rühsen <tim.ruehsen@gmx.de> 13369 13370 Fix http to https 13371 13372 * README.md: Fix http to https 13373 * bootstrap.conf: Likewise 13374 * configure.ac: Likewise 13375 133762017-02-28 Tim Rühsen <tim.ruehsen@gmx.de> 13377 13378 Exclude some URLs from recursive download with -p. 13379 13380 * include/wget/wget.h: Add link_inline to WGET_HTML_PARSED_URL 13381 * libwget/html_url.c: Add link_inline to _html_context_t, 13382 (_html_get_url): Set ctx->link_inline 13383 * src/wget.c (html_parse): Exclude action and formaction URLs, 13384 exclude certain 'link' URLs from download 13385 133862017-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 13387 13388 * README.md: Mention HPKP and brotli support 13389 133902017-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 13391 13392 Add brotli decompression, Accept-Encoding 'br' 13393 133942017-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 13395 13396 * wget.c (_css_parse_uri): Fix previous commit 13397 133982017-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 13399 13400 * wget.c (_css_parse_uri): Fix charset conversion for CSS parsing 13401 134022017-02-23 Tim Rühsen <tim.ruehsen@gmx.de> 13403 13404 Fix charset conversion for HTML parsing 13405 13406 * src/wget.c: New helper function _normalize_uri() 13407 (sitemap_parse_text): Check size before allocating stack space 13408 * tests/Makefile.am: Add test-iri-subdir 13409 * tests/test-iri-subdir.c: Add new test 13410 134112017-02-23 Tim Rühsen <tim.ruehsen@gmx.de> 13412 13413 *src/wget.c,src/wget_main.h: Include wget.h in wget_main.h 13414 134152017-02-22 Tim Rühsen <tim.ruehsen@gmx.de> 13416 13417 Restore --config being a list of filenames 13418 13419 * src/options.c: New functions parse_stringlist_expand(), 13420 parse_filenames(). 13421 Use parse_filenames() for --config / --config-file. 13422 134232016-12-03 Darshit Shah <darnir@gnu.org> 13424 13425 Support tilde expansion of filenames from wgetrc 13426 13427 * src/options.c (_shell_expand): New function to perform tilde 13428 expansion on a given string 13429 (parse_filename): Parse the input string and perform shell expansion on 13430 it using _shell_expand 13431 (parse_string): Set the parser functions for various options that may be 13432 set from a config file to use parse_filename 13433 (wget_strnglob): Run glob() on only the specified part of the string 13434 134352017-02-22 Tim Rühsen <tim.ruehsen@gmx.de> 13436 13437 * src/wget.c: Print headers before body with -S 13438 134392017-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 13440 13441 HPKP: Drop/reject expired entries on save/load 13442 13443 * libwget/hpkp.c: Check max_age on load/save. 13444 134452017-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 13446 13447 Remove HPKP entries with zero PINs 13448 13449 * libwget/hpkp.c (wget_hpkp_db_check_pubkey): Fix subdomain default, 13450 (wget_hpkp_db_add): Remove entry on zero PINs 13451 * src/wget.c: Remove comment line 13452 * tests/test.c (test_hpkp): Test this feature 13453 134542017-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 13455 13456 * libwget/ssl_gnutls.c (_cert_verify_hpkp): Fix for GnuTLS < 3.1.3 13457 134582017-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 13459 13460 Fix memleak in HPKP 13461 13462 * include/wget/wget.h: Changed declaration of wget_hpkp_free() 13463 * libwget/hpkp.c: Use wget_hpkp_free() as destructor function 13464 * libwget/http.c (wget_http_free_hpkp_entries): Amended 13465 * tests/test.c: Fix typos 13466 134672017-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 13468 13469 * tests/certs/README: Add info on how to create pin-sha256 values for HPKP 13470 134712017-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 13472 13473 * tests/test.c: Add unit testing for HPKP 13474 134752017-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 13476 13477 * libwget/hpkp.c (_wget_hpkp_compare_pins): Implement 'includeSubDomains' 13478 134792017-02-19 Tim Rühsen <tim.ruehsen@gmx.de> 13480 13481 * libwget/hpkp.c (wget_hpkp_set_host): Free host before assignment 13482 134832017-02-19 Tim Rühsen <tim.ruehsen@gmx.de> 13484 13485 Implement HPKP checking 13486 13487 * include/wget/wget.h: Add function wget_hpkp_db_check_pubkey(), 13488 add WGET_SSL_HPKP_CACHE 13489 * libwget/hpkp.c: Implement wget_hpkp_db_check_pubkey() 13490 * libwget/ssl_gnutls.c: Implement and use _cert_verify_hpkp() 13491 * src/options.c (init): Set WGET_SSL_HPKP_CACHE 13492 134932017-02-17 Tim Rühsen <tim.ruehsen@gmx.de> 13494 13495 HPKP API redesign 13496 134972017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13498 13499 * src/options.c: Fix position of hpkp options in array 13500 135012017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13502 13503 * libwget/hpkp.c (wget_hpkp_db_load): Fix -Wformat= 13504 135052017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13506 13507 * src/options.c (init): Fix calls of wget_str_asprintf() 13508 135092017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13510 13511 libwget/hpkp.c (wget_hpkp_db_load): Fix -Wsign-compare 13512 135132017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13514 13515 * include/wget/wget.h: Fix HPKP function declarations 13516 135172017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13518 13519 * libwget/hpkp.c: Fix signed/unsigned format warnings 13520 135212017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13522 13523 * libwget/hpkp.c (wget_hpkp_new): Fix -Wold-style-definition 13524 135252017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13526 13527 * include/wget/wget.h: Use WGETAPI for hpkp function declarations 13528 135292017-01-09 Ander Juaristi <ajuaristi@gmx.es> 13530 13531 HPKP: version the HPKP file, for the future 13532 13533 * libwget/hpkp.c (wget_hpkp_db_save): write the version number 13534 (wget_hpkp_db_load): check the version number == 1 135352017-01-09 Ander Juaristi <ajuaristi@gmx.es> 13536 13537 HPKP: Check file is regular or symlink 13538 13539 * libwget/hpkp.c (wget_hpkp_db_save): check if file is regular or symlink. If symlink, resolve it since unlink(2) does not follow symlinks. 135402017-01-08 Ander Juaristi <ajuaristi@gmx.es> 13541 13542 HPKP: Fix memory leaks 13543 13544 * src/wget.c (main, process_response_header): add debug traces 13545 * libwget/http.c (wget_http_parse_public_key_pins): fix memory leaks 13546 * libwget/hpkp.c (__wget_hpkp_free, wget_hpkp_db_load): fix memory 13547 leaks 13548 135492017-01-05 Ander Juaristi <ajuaristi@gmx.es> 13550 13551 HPKP: inline docs 13552 135532017-01-05 Ander Juaristi <ajuaristi@gmx.es> 13554 13555 HPKP: return WGET_HPKP_ENTRY_EXISTS 13556 135572017-01-04 Ander Juaristi <ajuaristi@gmx.es> 13558 13559 HPKP: Define return values. 13560 135612017-01-04 Ander Juaristi <ajuaristi@gmx.es> 13562 13563 HPKP: use vector rather than linked list for b64 pins 13564 135652016-12-28 Ander Juaristi <ajuaristi@gmx.es> 13566 13567 HPKP: add log traces + fix bugs 13568 13569 135702016-12-27 Ander Juaristi <ajuaristi@gmx.es> 13571 13572 HPKP: hpkp complete 13573 13574 135752016-12-13 Ander Juaristi <ajuaristi@gmx.es> 13576 13577 HPKP: fix wget_hpkp_db_save() 13578 13579 * src/wget.c (main): call wget_hpkp_db_save() at the end (params 13580 inverted). 13581 * include/wget/wget.h: export wget_hpkp_db_save(). 13582 * libwget/hpkp.c (__hashtable_browse_cb): fix fprintf format string. 13583 (wget_hpkp_db_save): check filename is not an empty string. 13584 (wget_hpkp_db_load): likewise. 13585 135862016-12-13 Ander Juaristi <ajuaristi@gmx.es> 13587 13588 HPKP: new option --hpkp-file 13589 13590 * src/wget_options.h (struct config): new string hpkp_file. 13591 * src/options.c (struct options): new command-line option --hpkp-file. 13592 (init): set --hpkp-file default to $HOME/.wget-hpkp. 13593 (deinit): free config.hpkp_file. 13594 * src/wget.c (main): call wget_hpkp_db_save() at the end. 13595 135962016-12-13 Ander Juaristi <ajuaristi@gmx.es> 13597 13598 HPKP: new option --hpkp-file 13599 13600 136012016-12-09 Ander Juaristi <ajuaristi@gmx.es> 13602 13603 HPKP: functions to load & save the database file 13604 13605 * libwget/hpkp.c (__wget_hpkp_new): new function. NO-OP, NEEDS 13606 IMPLEMENTATION. 13607 (__vector_browse_cb): new callback function to browse the vector. 13608 (__hashtable_browse_cb): new callback function to browse the hash 13609 table. 13610 (wget_hpkp_db_save): new function. 13611 (__wget_hpkp_parse_host_line): new function. 13612 (__wget_hpkp_parse_pin_line): new function. 13613 (wget_hpkp_db_load): new function. 13614 136152016-11-12 Ander Juaristi <ajuaristi@gmx.es> 13616 13617 Prototype HPKP functions 13618 13619 * include/wget/wget.h: new public function wget_hpkp_db_init(). 13620 * libwget/hpkp.c: new file. 13621 136222016-11-12 Ander Juaristi <ajuaristi@gmx.es> 13623 13624 Merge from upstream/master 13625 13626 Conflicts: libwget/Makefile.am 13627 136282016-11-12 Ander Juaristi <ajuaristi@gmx.es> 13629 13630 HPKP boilerplate 13631 13632 * libwget/Makefile.am: new file hpkp.c. 13633 * src/options.c (print_help, struct options): new command line option '--hpkp'. 13634 (init): initialize HPKP database with wget_hpkp_db_init(). 13635 * src/wget_options.h (struct config): new handle 'hpkp' to the HPKP 13636 database. 13637 136382016-10-25 Ander Juaristi <ajuaristi@gmx.es> 13639 13640 HPKP: initial commit 13641 136422017-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 13643 13644 * tests/test.c (test_buffer): Add more tests 13645 136462017-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 13647 13648 * libwget/buffer.c (wget_buffer_free_data): Set size to 0 13649 136502017-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 13651 13652 * Makefile.am: Fix 'make check-coverage' 13653 136542017-02-21 Tim Rühsen <tim.ruehsen@gmx.de> 13655 13656 Fix memleak in cookie code 13657 13658 * examples/http_get2.c: Use wget_cookie_db_free() to free cookie db 13659 * libwget/cookie.c (wget_cookie_db_init): Fix #ifdef around PSL code 13660 * libwget/init.c (wget_global_deinit): Use wget_cookie_db_free() 13661 136622017-02-03 Tim Rühsen <tim.ruehsen@gmx.de> 13663 13664 * contrib/release: Update 13665 136662017-02-03 Tim Rühsen <tim.ruehsen@gmx.de> 13667 13668 * configure.ac: Add xz archive format 13669 136702017-02-01 Tim Rühsen <tim.ruehsen@gmx.de> 13671 13672 Fix possible file name truncation 13673 13674 * tests/test-parse-html.c: Increase buffer for filename 13675 * tests/test.c: Likewise 13676 136772017-01-23 Tim Rühsen <tim.ruehsen@gmx.de> 13678 13679 New function wget_logger_is_active() 13680 13681 * include/wget/wget.h: Add prototype 13682 * libwget/Makefile.am: Add logger.h 13683 * libwget/log.c: Include logger.h 13684 * libwget/logger.c: Include logger.h, add wget_logger_is_active() 13685 * libwget/net.c: Use wget_logger_is_active() 13686 * libwget/private.h: Remove struct _wget_logger_st 13687 136882017-01-23 Tim Rühsen <tim.ruehsen@gmx.de> 13689 13690 Use typedefs for function pointer arguments 13691 13692 * include/wget/wget.h: Add typedefs 13693 * libwget/cookie.c: Use typedefs 13694 * libwget/css.c: Likewise 13695 * libwget/css_url.c: Likewise 13696 * libwget/decompressor.c: Likewise 13697 * libwget/hashmap.c: Likewise 13698 * libwget/hsts.c: Likewise 13699 * libwget/http.c: Likewise 13700 * libwget/init.c: Likewise 13701 * libwget/io.c: Likewise 13702 * libwget/list.c: Likewise 13703 * libwget/logger.c: Likewise 13704 * libwget/metalink.c: Likewise 13705 * libwget/net.c: Likewise 13706 * libwget/netrc.c: Likewise 13707 * libwget/ocsp.c: Likewise 13708 * libwget/private.h: Likewise 13709 * libwget/robots.c: Likewise 13710 * libwget/stringmap.c: Likewise 13711 * libwget/tls_session.c: Likewise 13712 * libwget/vector.c: Likewise 13713 * libwget/xml.c: Likewise 13714 * src/blacklist.c: Likewise 13715 * src/host.c: Likewise 13716 * src/options.c: Likewise 13717 * src/wget.c: Likewise 13718 * tests/stringmap_perf.c: Likewise 13719 * tests/test.c: Likewise 13720 137212017-01-20 Tim Rühsen <tim.ruehsen@gmx.de> 13722 13723 base46: Use size_t as string length param 13724 137252017-01-20 Tim Rühsen <tim.ruehsen@gmx.de> 13726 13727 * src/wget.c (restrict_file_name): Fix clang warning 13728 137292017-01-19 Tim Rühsen <tim.ruehsen@gmx.de> 13730 13731 Add typedef for wget_vector_browse() callback function 13732 13733 * include/wget/wget.h: Add typedef wget_list_browse_cb_t 13734 * libwget/list.c: Use wget_list_browse_cb_t 13735 137362017-01-19 Tim Rühsen <tim.ruehsen@gmx.de> 13737 13738 Add typedef for wget_update() callback functions 13739 13740 * include/wget/wget.h: Add typedef wget_update_cb_t 13741 * libwget/cookie.c: Use wget_update_cb_t 13742 * libwget/hsts.c: Likewise 13743 * libwget/tls_session.c: Likewise 13744 137452017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13746 13747 gnulib: Update 13748 137492017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13750 13751 * libwget/io.c (wget_ready_2_transfer): Fix INF timeout for select() 13752 137532017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13754 13755 Revert "* .travis_setup.sh: Install graphiz and libidn2 on OSX" 13756 13757 This reverts commit 31c287f2ce90593e493c25581b351d7b944b8ff4. 13758 137592017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13760 13761 * .travis_setup.sh: Install graphiz and libidn2 on OSX 13762 137632017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13764 13765 Fix search for '-e' commands 13766 13767 * src/options.c (set_long_option): Fallback to unsharp linear search, 13768 (opt_compare_execute): Amend params. 13769 137702017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13771 13772 Fallback to select() if poll() not available 13773 13774 * bootstrap.conf: Add module select 13775 * libwget/io.c (wget_ready_2_transfer): Add fallback code 13776 137772017-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 13778 13779 * src/wget.c (http_receive_response): Small code cleanup 13780 137812017-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 13782 13783 Add console color abstraction 13784 13785 * docs/Makefile.am: Add libwget-console.3 13786 * include/wget/wget.h: Add console routines 13787 * libwget/Makefile.am: Add libwget/console.c 13788 * libwget/console.c: New file with console functions 13789 * src/log.c: Add WIN32 and console function calls 13790 137912017-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 13792 13793 Add function wget_truncate() 13794 13795 * bootstrap.conf: Ad gnulib module ftruncate 13796 * include/wget/wget.h: Add wget_truncate 13797 * libwget/io.c: Add wget_truncate 13798 * src/job.c: Call wget_truncate instead of truncate 13799 13800 The truncate() function is not covered by gnulib. 13801 wget_truncate() is a wrapper around ftruncate(), which is covered 13802 by gnulib. 13803 138042017-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 13805 13806 * tests/stringmap_perf.c: Use wget_malloc if mmap not available 13807 138082017-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 13809 13810 * libwget/cookie.c (_cookie_db_save): Fix ferror() check 13811 13812 Reported-by: Gisle Vanem 13813 138142017-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 13815 13816 * gnulib: Update 13817 138182017-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 13819 13820 * libwget/Makefile.am: Add more libs to LDADD 13821 138222017-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 13823 13824 * src/wget.c (downloader_thread): Slow down on connection errors 13825 138262017-01-05 Tim Rühsen <tim.ruehsen@gmx.de> 13827 13828 Let MSVC compile libwget/init.c (wget_global_init) 13829 13830 Reported-by: Gisle Vanem 13831 138322017-01-05 Tim Rühsen <tim.ruehsen@gmx.de> 13833 13834 * src/options.c (init): Call wget_net_init() 13835 138362017-01-05 Tim Rühsen <tim.ruehsen@gmx.de> 13837 13838 Remove redundant check for _WIN64 13839 138402017-01-05 Tim Rühsen <tim.ruehsen@gmx.de> 13841 13842 Add network init function to bootstrap sockets on Windows 13843 13844 * bootstrap.conf: Add gnulib module 'sockets' 13845 * examples/http_get2.c: Call wget_net_init() 13846 * include/wget/wget.h: Add wget_net_init() and wget_net_deinit() 13847 * libwget/init.c (wget_global_init): Call wget_net_init(), 13848 (wget_global_deinit): Call wget_net_deinit() 13849 * libwget/net.c: Add wget_net_init() and wget_net_deinit() 13850 138512016-12-17 Tim Rühsen <tim.ruehsen@gmx.de> 13852 13853 Rename wget_str_[v]asprintf to wget_[v]aprintf 13854 13855 * include/wget/wget.h: Rename wget_str_[v]asprintf to wget_[v]aprintf 13856 * libwget/http.c: Likewise 13857 * libwget/iri.c: Likewise 13858 * libwget/printf.c: Likewise 13859 * libwget/vector.c: Likewise 13860 * src/options.c: Likewise 13861 * tests/test-metalink.c: Likewise 13862 138632016-12-21 Tim Rühsen <tim.ruehsen@gmx.de> 13864 13865 Adjust GnuTLS priorities 13866 13867 * libwget/ssl_gnutls.c (wget_ssl_init): Use system defaults for 13868 secure_protocol "auto". 13869 (wget_ssl_open): Set compatibility mode for "auto". 13870 138712016-12-21 Tim Rühsen <tim.ruehsen@gmx.de> 13872 13873 * libwget/cookie.c (wget_cookie_db_init): Fix argument of psl_latest() 13874 138752016-12-16 Tim Rühsen <tim.ruehsen@gmx.de> 13876 13877 Reduce redundant code by using strchrnul() 13878 13879 * bootstrap.conf: Add gnulib module strchrnul 13880 * libwget/http.c (_parse_proxies): Remove redundant code 13881 * libwget/ssl_gnutls.c (wget_ssl_open): Likewise 13882 138832016-12-16 Tim Rühsen <tim.ruehsen@gmx.de> 13884 13885 * docs/wget2_manual.md: Add description for --config-file 13886 138872016-12-16 Tim Rühsen <tim.ruehsen@gmx.de> 13888 13889 Read $SYSTEM_WGET2RC, $WGET2RC and ~/.wget2rc as config file 13890 13891 This patch separates the config files of wget and wget2. 13892 Config files are read in the order they appear on the command line. 13893 13894 Fixes #132 13895 138962016-12-16 Tim Rühsen <tim.ruehsen@gmx.de> 13897 13898 * src/options.c (parse_stringlist): Remove redundant code 13899 139002016-12-15 Tim Rühsen <tim.ruehsen@gmx.de> 13901 13902 * libwget/thread.c: Return a value in dummy wget_thread_cancel() 13903 13904 Reported-by: Gisle Vanem 13905 139062016-12-14 Tim Rühsen <tim.ruehsen@gmx.de> 13907 13908 * src/wget.c (process_response): Ignore sitemaps with --page-requisites 13909 139102016-12-13 Tim Rühsen <tim.ruehsen@gmx.de> 13911 13912 * src/options.c (selftest_options): Clear config.headers before and after use 13913 139142016-12-13 Tim Rühsen <tim.ruehsen@gmx.de> 13915 13916 * src/options.c: Relax CLI and config file parser regarding spaces 13917 139182016-12-12 Tim Rühsen <tim.ruehsen@gmx.de> 13919 13920 Take advantage of libpsl 0.16+ 13921 13922 * libwget/cookie.c (wget_cookie_db_init): Use psl_latest() if possible 13923 139242016-12-03 Tim Rühsen <tim.ruehsen@gmx.de> 13925 13926 * tests/test-bad-chunk.c: Remove WGET_TEST_KEEP_TMPFILES 13927 139282016-12-02 Tim Rühsen <tim.ruehsen@gmx.de> 13929 13930 Add tests with bad chunk size in the HTTP response 13931 13932 * tests/Makefile.am: Add test-bad-chunk.c 13933 * tests/test-bad-chunk.c: New file with 3 tests 13934 139352016-12-02 Tim Rühsen <tim.ruehsen@gmx.de> 13936 13937 * libwget/http.c: Ignore Transfer-Encoding for HTTP/2 requests 13938 139392016-11-30 Tim Rühsen <tim.ruehsen@gmx.de> 13940 13941 * src/options.c: Respect WGETRC and SYSTEM_WGETRC env vars 13942 139432016-11-16 Darshit Shah <darnir@gnu.org> 13944 13945 * docs/wget2_manual.md: Update description of -O 13946 139472016-11-16 Darshit Shah <darnir@gnu.org> 13948 13949 Make the visibiity attribute preceed the prototype 13950 13951 * include/wget/wget.h: Make library symbol export attribute 13952 ((__visibility__)) preceed the function prototype to work on Windows 13953 with MSVC 13954 * tests/libtest.h: Same 13955 13956 Reported-by: Gisle Vanem 13957 139582016-11-16 Darshit Shah <darnir@gnu.org> 13959 13960 Rename LIBWGET_EXPORT to WGETAPI 13961 13962 * include/wget/wget.t: Rename LIBWGET_EXPORT to WGETAPI 13963 * tests/libtest.h: Same 13964 139652016-11-16 Darshit Shah <darnir@gnu.org> 13966 13967 Export symbols only from wget.h 13968 13969 * libwget/iri.c: Move exporting of symbol wget_iri_schemes from here 13970 * include/wget/wget.h: To here 13971 139722016-11-16 Darshit Shah <darnir@gnu.org> 13973 13974 Replace x{malloc,calloc} calls with wget_* methods 13975 13976 * include/wget/wget.h: Remove preprocessor definitions for xmalloc, 13977 xcalloc and xrealloc 13978 * src/job.c(job_validate_file): Replace xcalloc with wget_calloc 13979 * src/wget.c(main): Same 13980 (process_head_response): Same 13981 (_prepare_file): Same 13982 (http_send_request): Same 13983 139842016-11-15 Tim Rühsen <tim.ruehsen@gmx.de> 13985 13986 Fix integer overflow in ratio calculation 13987 13988 * libwget/bar.c (_bar_update_slot): Fix integer overflow in ratio calculation 13989 13990 Fixes #125 13991 139922016-11-15 Darshit Shah <darnir@gnu.org> 13993 13994 Fix timestamp in If-Modified-Since header 13995 13996 * src/wget.c(http_create_request): Use mtime instead of mtime+1 13997 13998 According to RFC 7238, section 3.3, an origin server may use exact 13999 timestamp matching to implement conditional requests. Hence, Wget must 14000 not modify the mtime at all 14001 140022016-11-15 Tim Rühsen <tim.ruehsen@gmx.de> 14003 14004 * src/bar.c (bar_init): Disable progress bar if threading is not available 14005 140062016-11-12 Darshit Shah <darnir@gnu.org> 14007 14008 * libwget/bar.c (_bar_set_progress): Typecast the right value to integer 14009 140102016-11-11 Tim Rühsen <tim.ruehsen@gmx.de> 14011 14012 Move m4 wget_ functions to m4/ 14013 14014 * .gitignore: Add exception m4/wget_manywarnings.m4 14015 * configure.ac: Remove manywarnings m4 code 14016 * m4/wget_manywarnings.m4: Add manywarnings m4 code here 14017 140182016-11-11 Tim Rühsen <tim.ruehsen@gmx.de> 14019 14020 * configure.ac: Define and use wget_MANYWARNINGS m4 macro 14021 140222016-11-11 Tim Rühsen <tim.ruehsen@gmx.de> 14023 14024 * libwget/ip.c: Fix syntax error in docs 14025 140262016-11-09 Tim Rühsen <tim.ruehsen@gmx.de> 14027 14028 * libwget/io.c: Use open() instead of creat() 14029 14030 open() is covered by gnulib, creat() not. 14031 140322016-11-09 Tim Rühsen <tim.ruehsen@gmx.de> 14033 14034 Use defines for O_CREAT perms instead 14035 14036 * src/log.c (_write_out): Use S_* defines instead of 0644 for perms 14037 * src/wget.c (_prepare_file): Likewise 14038 140392016-11-08 Tim Rühsen <tim.ruehsen@gmx.de> 14040 14041 * libwget/thread.c: Add missing wget_thread_cancel() dummy 14042 140432016-11-08 Tim Rühsen <tim.ruehsen@gmx.de> 14044 14045 * AUTHORS: Update 14046 140472016-11-08 Tim Rühsen <tim.ruehsen@gmx.de> 14048 14049 * libwget/net.c: Include sys/ioctl.h on Windows. 14050 (_set_async): Use ioctl() instead of ioctlsocket(). 14051 14052 Reported-by: Gisle Vanem 14053 140542016-11-08 Tim Rühsen <tim.ruehsen@gmx.de> 14055 14056 * include/wget.h: G_GNUC_WGET_PRINT_FORMAT -> G_GNUC_WGET_PRINTF_FORMAT 14057 140582016-11-06 Ander Juaristi <ajuaristi@gmx.es> 14059 14060 New function to check for literal IPv4/IPv6 addresses 14061 14062 * bootstrap.conf: Add gnulib module 'inet_pton' 14063 * docs/Makefile.am: Add man page libwget-ip.3 14064 * wget/wget.h (struct wget_iri_st): New field 'is_ip_address'. 14065 * libwget/Makefile.am: Add new file ip.c 14066 * libwget/ip.c: New file defining new function wget_ip_is_family() 14067 * libwget/iri.c (wget_iri_parse): Check for literal IP address 14068 * src/wget.c: Do not add to HSTS if literal IP address 14069 140702016-11-08 Tim Rühsen <tim.ruehsen@gmx.de> 14071 14072 * configure.ac: Remove old commented code 14073 140742016-11-08 Tim Rühsen <tim.ruehsen@gmx.de> 14075 14076 Update gnulib 14077 140782016-11-08 Tim Rühsen <tim.ruehsen@gmx.de> 14079 14080 Use gnulib basename 14081 14082 * bootstrap.conf: Add gnulib module 'dirname' 14083 * libwget/io.c: Use base_name() to get file basename 14084 140852016-11-07 Tim Rühsen <tim.ruehsen@gmx.de> 14086 14087 * include/wget/wget.h: Fix compilation error 14088 140892016-11-07 Tim Rühsen <tim.ruehsen@gmx.de> 14090 14091 * include/wget/wget.h: Define empty LIBWGET_EXPORT if LIBWGET_STATIC is defined 14092 140932016-11-07 Tim Rühsen <tim.ruehsen@gmx.de> 14094 14095 * libwget/xalloc.c (wget_realloc): Call OOM callback if size is 0 14096 140972016-11-07 Tim Rühsen <tim.ruehsen@gmx.de> 14098 14099 * configure.ac: Do not check for munmap() 14100 141012016-11-06 Ander Juaristi <ajuaristi@gmx.es> 14102 14103 New inline function to compute length of base64 decoded string 14104 14105 * include/wget/wget.h (wget_base64_get_decoded_length): new function 14106 * libwget/tls_session.c (_tls_session_db_load): use 14107 wget_base64_get_decoded_length(). 14108 141092016-11-03 Tim Rühsen <tim.ruehsen@gmx.de> 14110 14111 Use wget_strdup() instead of strdup() 14112 14113 * examples/getstream.c: Use wget_strdup(). 14114 * libwget/cookie.c: Likewise. 14115 * libwget/encoding.c: Likewise. 14116 * libwget/http.c: Likewise. 14117 * libwget/iri.c: Likewise. 14118 * libwget/netrc.c: Likewise. 14119 * src/options.c: Likewise. 14120 * src/wget.c: Likewise. 14121 * tests/test.c: Likewise. 14122 141232016-11-03 Tim Rühsen <tim.ruehsen@gmx.de> 14124 14125 * libwget/hashmap_old.c: Removed 14126 141272016-11-03 Tim Rühsen <tim.ruehsen@gmx.de> 14128 14129 * configure.ac: Understand 'Debian clang' and 'gcc-x.y' for 14130 141312016-11-02 Tim Rühsen <tim.ruehsen@gmx.de> 14132 14133 Fix gcc/clang warnings from the new manywarnings feature 14134 14135 * configure.ac: Suppress -Wunsuffixed-float-constants for gcc, 14136 add -Wno-disabled-macro-expansion to clang flags 14137 * examples/getstream.c: Removed 'unused' attributes from argc and argv. 14138 * include/wget/wget.h: Fix slot to int for bar functions. 14139 * libwget/bar.c: Fix slot to int for bar functions, cast to int, 14140 change type of bitfield 'redraw' to unsigned. 14141 * libwget/hsts.c (wget_hsts_new): Fix int to bitfield conversion. 14142 * libwget/http.c: Cast printf %p format to (void *). 14143 * src/wget.c: Cast printf %p format to (void *), 14144 remove ACTION_DONE from enum actions. 14145 * src/wget_host.h: Fix double declaration of host_remove_job() 14146 * tests/libtest.c (wget_test_check_filesystem): Fix type of 'rc'. 14147 * tests/test-idn-robots.c: Fix comma to semikolon at end of line. 14148 * tests/test.c (test_strcasecmp_ascii): Fix type of loop variable. 14149 141502016-11-02 Tim Rühsen <tim.ruehsen@gmx.de> 14151 14152 Replace manywarnings module by our own code 14153 14154 * bootstrap.conf: Remove manywarnings, add warnings 14155 * configure.ac: Replace manywarnings code 14156 14157 The new code works on recent gcc and clang with an overhead of 14158 ~100ms instead of ~9s here (without -C). 14159 This is meant for developers only and is activated by either 14160 --enable-gcc-warnings or by touching .manywarnings. 14161 141622016-10-23 Darshit Shah <darnir@gnu.org> 14163 14164 Fix parse_header() and add more tests 14165 14166 * src/options.c (parse_header): Fix bug where empty string did not clear 14167 out all the headers 14168 * src/options.c (selftest_options): Fix testcase for parse_header and 14169 add more tests 14170 141712016-10-16 Tim Rühsen <tim.ruehsen@gmx.de> 14172 14173 Check for gcrypt.h 14174 14175 * configure.ac: Check for gcrypt.h 14176 * libwget/hashfile.c: Check for HAVE_GCRYPT_H 14177 141782016-10-14 Tim Rühsen <tim.ruehsen@gmx.de> 14179 14180 Add gnulib digest hash fallback 14181 14182 * bootstrap.conf: Add md2, md5, sha1, sha256, sha512 modules 14183 * libwget/hashfile.c: Add code branch to use gnulib functions if needed 14184 141852016-10-16 Tim Rühsen <tim.ruehsen@gmx.de> 14186 14187 * tests/test-iri.c: Fix test to work on non-UTF-8 locales 14188 141892016-10-16 Tim Rühsen <tim.ruehsen@gmx.de> 14190 14191 * tests/test-iri-percent.c: Fix test to work on non-UTF-8 locales 14192 141932016-10-16 Tim Rühsen <tim.ruehsen@gmx.de> 14194 14195 * tests/test-restrict-ascii.c: Fix test to work on non-UTF-8 locales 14196 141972016-10-14 Tim Rühsen <tim.ruehsen@gmx.de> 14198 14199 * libwget/hashfile.c: Remove unused defines 14200 142012016-10-14 Tim Rühsen <tim.ruehsen@gmx.de> 14202 14203 Make --header replace wget's standard headers 14204 14205 * libwget/http.c (wget_http_create_request): Add 'Host' header 14206 to header list, 14207 (wget_http_send_request): Do not add 'Host' hard-coded, 14208 (wget_http_request_to_buffer): Special treatment of 'Content-Length' 14209 * src/wget.c (http_create_request): Replace wget's HTTP headers by 14210 user-provided headers, except 'Cookie' headers which will be appended. 14211 142122016-10-13 Darshit Shah <darnir@gnu.org> 14213 14214 Support --header option 14215 14216 * src/options.c: Add new --header option and a new parse_header() 14217 parsing function for that. Also add self tests for parsing 14218 * src/wget_options.h: Same 14219 * src/wget.c (http_create_request): Add the user added headers to the 14220 HTTP Request 14221 * src/test-wget-1.c: Add a test for the --header option 14222 142232016-10-13 Tim Rühsen <tim.ruehsen@gmx.de> 14224 14225 Separate progress bar logic into library and application parts 14226 14227 * include/wget/wget.h: Remove struct _wget_bar_ctx and wget_bat_ctx_t, 14228 amend prototypes of wget_bar functions, 14229 new function wget_bar_write_line() 14230 * libwget/bar.c: Bunch of cleanups and code rearrangements 14231 * src/wget_progress.h: Remove 14232 * src/Makefile.am: Remove src/wget_progress.h 14233 * src/bar.c: Remove mutex usage, 14234 (_error_write) call wget_bar_write_line 14235 * src/log.c: Remove redundant param checks 14236 * src/wget.c: Add struct _body_callback_context, 14237 always call bar_slot_begin() on begin of new download, 14238 use downloader->id as progress slot 14239 * src/wget_bar.h: Amend prototypes of bar_ functions 14240 142412016-10-12 Tim Rühsen <tim.ruehsen@gmx.de> 14242 14243 Call bar_update_slot() from main thread 14244 14245 * libwget/bar.c: Fix typos, 14246 (wget_bar_set_slots): Init mem only when needed, 14247 add comments describing Escape sequences 14248 * src/bar.c (bar_update_slots): Take nslots as param 14249 * src/wget.c: Move call to bar_update_slots() from worker to main thread 14250 * src/wget_bar.h: Amend prototype of bar_update_slots() 14251 142522016-10-12 Tim Rühsen <tim.ruehsen@gmx.de> 14253 14254 * docs/wget2_manual.md: More changes for Wget2 14255 142562016-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 14257 14258 * src/wget.c: Fix quota calculation 14259 142602016-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 14261 14262 * libwget/http.c: Fix update of downloaded bytes for progress bar 14263 142642016-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 14265 14266 * libwget/ssl_gnutls.c: Print debug messages using debug_printf() 14267 142682016-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 14269 14270 * src/options.c: Amend Wget2 banner string 14271 142722016-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 14273 14274 * src/host.c (host_queue_free): Also free robots.txt job 14275 142762016-10-07 Darshit Shah <darnir@gnu.org> 14277 14278 * configure.ac: Ignore the Stack Protector warnings 14279 142802016-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 14281 14282 * libwget/http.c: Support compression for HTTP/2 14283 142842016-10-07 Darshit Shah <darnir@gnu.org> 14285 14286 Improved testing on Travis 14287 14288 * .travis.sh: Rewrite for clarity and speed. Testing shows that valgrind 14289 and ASan don't play well together. So split their executions. 14290 * .travis.yml: Enable ccache for builds. Also try to update libc6 if 14291 possible 14292 * .travis.yml: Do not run on OSX with CC=gcc. This is because on OSX, 14293 gcc is only a wrapper around clang. 14294 * .travis_setup.sh: Cosmetic changes only 14295 * Makefile.am (check-coverage): Explcicitly disable valgrind tests when 14296 checking coverage 14297 * tests/libtest.c (wget_test): Use a provided suppressions file for 14298 valgrind. This is required since the valgrind version on Travis seems to 14299 be out of sync with libc. This causes valgrind to report a memory bug 14300 falsely. 14301 * tests/valgrind-suppressions: New file with some valgrind suppressions. 14302 Included suppressions are for a valgrind/libc bug and to suppress 14303 reachable memory messages from within gnutls 14304 143052016-10-06 Darshit Shah <darnir@gnu.org> 14306 14307 Revert "Test travis" 14308 14309 This reverts commit 75daf2cdc791d1d01892aae9138c103e405a1f53. 14310 I screwed up. Pushed to the wrong repository 14311 143122016-10-06 Darshit Shah <darnir@gnu.org> 14313 14314 Test travis 14315 143162016-10-05 Darshit Shah <darnir@gnu.org> 14317 14318 * src/wget.c (nop): Add ifddef guards around SIGWINCH 14319 143202016-10-05 Tim Rühsen <tim.ruehsen@gmx.de> 14321 14322 Fix progress bar details 14323 14324 * include/wget/wget.h: wget_get_screen_size() must not be pure 14325 * libwget/bar.c (_bar_get_width): Leave one space right to avoid linebreaks 14326 * libwget/utils.c (wget_human_readable): Align # of bytes 14327 * src/bar.c (wget_get_screen_size): Fix return value 14328 143292016-10-04 Darshit Shah <darnir@gnu.org> 14330 14331 Export a new function for determining screen dimensions 14332 14333 * include/wget/wget.h: New function wget_get_screen_size. Replaces old 14334 wget_determine_screen_width 14335 * libwget/bar.c (_bar_get_width): Use wget_get_screen_size instead of 14336 wget_determine_screen_width 14337 * libwget/utils.c: Replace function wget_determine_screen_width with a 14338 more generic wget_get_screen_size. 14339 143402016-10-04 Darshit Shah <darnir@gnu.org> 14341 14342 Handle SIGWINCH for progress bar 14343 14344 * include/wget/wget.h: New function wget_bar_screen_resized, as an 14345 indicator that the screen size may have changed. Also, wget_bar_update 14346 may now modify the *bar parameter 14347 * libwget/bar.c: Move detection code for screen size into a separate 14348 function called _bar_get_width. 14349 (wget_bar_update): Check if the screen width has changed and 14350 reallocate more space if necessary 14351 * src/wget.c: Install a new signal handler for SIGWINCH 14352 143532016-10-03 Darshit Shah <darnir@gnu.org> 14354 14355 Move window size detection to libwget 14356 14357 * include/wget/wget.h: Declare new function wget_determine_screen_width. 14358 Also change prototype of wget_bar_init to not accept a max_width 14359 parameter 14360 * src/utils.c: Move function determine_screen_width from here ... 14361 * libwget/utils.c: ... to here. 14362 * src/utils.{c,h}: Delete empty file 14363 * src/Makefile.am: Remove source files utils.{c,h} 14364 * src/bar.c: Move detection of screen width from here ... 14365 * libwget/bar.c: ... to here 14366 * tests/test.c: wget_bar_init() no longer takes a max_width parameter 14367 143682016-10-03 Darshit Shah <darnir@gnu.org> 14369 14370 Cleaner API for progress bar within libwget 14371 14372 * include/wget/wget.h (_wget_bar_ctx): Remove unused elements 14373 expected_size and filename. Rename slotpos to _slotpos to reflect that 14374 it should not be used by the client. 14375 * include/wget/wget.h: Remove declaration for old API functions 14376 bar_register and bar_deregister. Define declarations for 14377 wget_bar_slot_begin, wget_bar_slot_register and 14378 wget_bar_slot_deregister. 14379 * libwget/bar.c (_bar_slot_status_t): Define new enum to store the 14380 status of a slot of the progress bar. 14381 (_bar_slot_t): Remove unused element last_ctx. Also, define slot 14382 elements: filename, file_size, bytes_downloaded and status. 14383 (wget_bar_slot_begin): Define new function. Invoked when the client 14384 has started downloading a new file. 14385 (wget_bar_slot_register): New function to register a new file in a 14386 progress bar slot. 14387 (wget_bar_deregister): Remove old function 14388 (wget_bar_slot_deregister): New function, called after file has 14389 finished downloading. Successfully or unsuccessfully. 14390 (wget_bar_deregister): Remove old function 14391 (_bar_set_progress): last_ctx is no longer used. Instead data is 14392 stored within the slot struct itself 14393 (_bar_update_slot): Same 14394 (_bar_print_final): Same 14395 (wget_bar_deinit): Free the filename stored in the slot 14396 * src/bar.c: Edit to reflect the changes in libwget API 14397 * src/wget.c (_get_header): Same 14398 (http_send_request): Same 14399 (http_receive_response): Same 14400 * src/wget_bar.h: Same 14401 144022016-10-03 Darshit Shah <darnir@gnu.org> 14403 14404 * libwget/bar.c: Remove mutex stdout 14405 14406 The mutex stdout was meant to ensure that multiple routines don't write 14407 to the screen concurrently. However, this mutex is not required, since 14408 such syncronization should be handled by the client using the library. 14409 All other library methods for the progress avoid any form 14410 synchronization primitives for this reason 14411 144122016-10-03 Darshit Shah <darnir@gnu.org> 14413 14414 * src/bar.c: Rename mutex to bar_mutex 14415 144162016-10-02 Darshit Shah <darnir@gnu.org> 14417 14418 * libwget/http.c: Remove unused struct _body_callback_context 14419 144202016-09-30 Tim Rühsen <tim.ruehsen@gmx.de> 14421 14422 Better code coverage for HTML parsing 14423 14424 * tests/test-idn-meta.c: Add HTML5 meta charset 14425 * tests/test-meta-robots.c: Add meta robots 'all' and 'follow' 14426 144272016-09-30 Darshit Shah <darnir@gnu.org> 14428 14429 Mark some functions as pure 14430 14431 * libwget/cookie.c (_compare_cookie{,2}): Mark as pure 14432 * libwget/encoding.c (_utf8_is_valid): Same 14433 * libwget/hsts.c (_compare_hsts): Same 14434 * libwget/ocsp.c (_compare_ocsp): Same 14435 * libwget/netrc.c (_compare_netrc): Same 14436 * libwget/tls_session.c (_compare_tls_session): Same 14437 144382016-09-30 Tim Rühsen <tim.ruehsen@gmx.de> 14439 14440 Rename include/libwget.h to include/wget/wget.h+wgetver.h 14441 144422016-09-29 Tim Rühsen <tim.ruehsen@gmx.de> 14443 14444 Add tests for CSS parsing 14445 14446 * tests/Makefile.am: Add test-parse-css.c 14447 * tests/test-parse-css.c: New file including two tests 14448 144492016-09-29 Tim Rühsen <tim.ruehsen@gmx.de> 14450 14451 * Makefile.am: Remove unwanted stuff from coverage report 14452 144532016-09-29 Tim Rühsen <tim.ruehsen@gmx.de> 14454 14455 Reduce Travis OSX build overhead 14456 144572016-09-29 Tim Rühsen <tim.ruehsen@gmx.de> 14458 14459 * tests/test.c: Add wget_bar_* unit test 14460 144612016-09-29 Tim Rühsen <tim.ruehsen@gmx.de> 14462 14463 * libwget/bar.c: Cosmetics, param check for wget_bar_init() 14464 144652016-09-28 Darshit Shah <darnir@gnu.org> 14466 14467 Don't dereference NULL pointer when copying filename 14468 14469 * libwget/bar.c (wget_bar_deregister): Make sure filename is not NULL 14470 before calling strdup() on it 14471 144722016-09-28 Tim Rühsen <tim.ruehsen@gmx.de> 14473 14474 * libwget/http.c: Check WITH_LIBNGHTTP2 before referring nghttp2_nv 14475 144762016-09-28 Tim Rühsen <tim.ruehsen@gmx.de> 14477 14478 Fix race condition when pipelining reuqests 14479 14480 * libwget/http.c: Add streamid to debug messages 14481 * src/progress.h: Remove member downloader and head from _body_callback_context 14482 * src/wget.c (downloader_thread): Keep track of locking, 14483 (_get_header): Remove reference to downloader, 14484 (http_send_request): Replace downloader with job 14485 14486 With HTTP/2 pipelining (parallel streams per connection) is active. 14487 The bug fixed stops file descriptor leaks and writing data into wrong 14488 files. 14489 144902016-09-27 Dagobert Michelsen <dam@opencsw.org> 14491 14492 * README.md: Add build status for OpenCSW 14493 144942016-09-28 Tim Rühsen <tim.ruehsen@gmx.de> 14495 14496 * src/net.c (wget_tcp_resolve): Fix mutex unlocking 14497 144982016-09-28 Tim Rühsen <tim.ruehsen@gmx.de> 14499 14500 * src/wget.c (add_url, process_response): Fix NULL pointer dereference 14501 14502 Reported-by: Coverity 14503 145042016-09-26 Darshit Shah <darnir@gnu.org> 14505 14506 * src/wget.c (establish_coneection): Don't crash when no mirrors defined for metalink file 14507 145082016-09-26 Darshit Shah <darnir@gnu.org> 14509 14510 * libwget/net.c (wget_tcp_resolve): Don't sleep with an acquired mutex 14511 145122016-09-27 Darshit Shah <darnir@gnu.org> 14513 14514 Support dynamic thread allocation in progress bar 14515 14516 * include/libwget.h.in: Add new function wget_bar_set_slots 14517 * libwget/bar.c (_wget_bar_st): Add new struct member max_slots to store 14518 the maximum number of slots of the progress bar 14519 (wget_bar_init): Initialize the memory for all the possible 14520 progress_bars 14521 (wget_bar_set_slots): New function to modify the number of progress 14522 slots at runtime. Can only increase the number of slots, not decrease. 14523 * src/bar.c(bar_init): Remove unneeded variable lf. bar_init() is 14524 invoked before any threads are created, hence do not print the required 14525 newlines. This is now done by wget_bar_set_slots 14526 (bar_update_slots): Wget2 interface for libwget's wget_bar_set_slots 14527 (_error_write): Fix the position of acquiring the mutex. Don't make 14528 changes to the screen while some other progress bar function may be 14529 running 14530 * src/bar.h: New function bar_update_slots 14531 * src/wget.c: Create global variable nthreads. 14532 (downloader_thread): Update the number of progress bar slots 14533 * src/wget.h: New global variable nthreads 14534 145352016-09-27 Tim Rühsen <tim.ruehsen@gmx.de> 14536 14537 Simplify dynamic creation of threads 14538 14539 * src/bar.c (bar_init): Use config.max_threads 14540 * src/options.c: Remove config.num_threads 14541 * src/options.h: Remove config.num_threads 14542 * src/wget.c (main): Use config.max_threads, remove async_urls 14543 145442016-09-27 Tim Rühsen <tim.ruehsen@gmx.de> 14545 14546 * src/wget.c (main): Fix main loop 14547 14548 This also fixes the CTRL-C issue. 14549 145502016-09-27 Tim Rühsen <tim.ruehsen@gmx.de> 14551 14552 Fix unescaping of '+' in URL 14553 14554 * libwget/iri.c (wget_iri_unescape_inline): Do not touch '+'. 14555 (wget_iri_parse): Replace '+' by ' ' in query part. 14556 * tests/test.c (test_iri_parse): Add more tests 14557 145582016-09-26 Darshit Shah <darnir@gnu.org> 14559 14560 Make wget_bar_update update the entire progress 14561 145622016-09-26 Darshit Shah <darnir@gnu.org> 14563 14564 Draw animated progress bar for unknown file size 14565 14566 * libwget/bar.c: Use a tick counter and draw an animated progress bar, 14567 similar to Wget 1.x when the file size is unknown 14568 145692016-09-26 Darshit Shah <darnir@gnu.org> 14570 14571 * configure.ac: Enable support for disabling assertions 14572 145732016-09-26 Tim Rühsen <tim.ruehsen@gmx.de> 14574 14575 * README.md: Add Coverity and Coveralls.io badge 14576 145772016-09-25 Darshit Shah <darnir@gnu.org> 14578 14579 Update travis script to psuh to coveralls.io 14580 14581 * .travis.sh: Run `make check-coverage` when running with CC=gcc 14582 * .travis.yml: Use the newer trusty tahir builds for updated packages 14583 * .travis.yml: Refactor. Split out the OSX installation steps into 14584 .travis_setup.sh 14585 . .travis_setup.sh: Same 14586 * Makefile.am: Add --ignore-errors source to lcov command line 14587 145882016-09-25 Darshit Shah <darnir@gnu.org> 14589 14590 * Makefile.am: Add targets for checking code coverage 14591 145922016-09-23 Darshit Shah <darnir@gnu.org> 14593 14594 Improved progress bar functioning 14595 145962016-09-23 Darshit Shah <darnir@gnu.org> 14597 14598 Clean up progress bar code 14599 14600 Saving the max, cols and ratio to the slot structure was an optimization 14601 used to prevent unneeded refreshes of the progress bar on slow 14602 connections. However, with the asynchronous progress bar, this 14603 optimization is no longer necessary as we refresh the progress bar far 14604 fewer times already. Also, as other elements like the ETA are added to 14605 the progress bar, these checks will only cause more problems as the ETA 14606 would need to be updated even if no data has been downloaded in some 14607 time. 14608 14609 * libwget/bar.c: Remove ratio, max, cur, cols and first from the 14610 _wget_slot_st struct and eliminate all references to them. 14611 146122016-09-23 Darshit Shah <darnir@gnu.org> 14613 14614 Print a different progress bar when size is unknown 14615 14616 * libwget/bar.c(_wget_bar_st): New struct members known_size and 14617 unknown_size to denote the character with which the progress bar is 14618 filled in these two cases 14619 (wget_bar_init): Initialize the new struct members with the right data 14620 (wget_bar_update): Choose and print the right character based on 14621 whether the size is known or not. 14622 (_bar_print_final): Same 14623 (wget_bar_deinit): Free the struct members 14624 146252016-09-23 Darshit Shah <darnir@gnu.org> 14626 14627 * src/bar.c(_error_write): Fix another potential race condition 14628 14629 In this case, the lock acquisition and release is done around the for 14630 loop instead instead of around the call to wget_bar_update. By creating 14631 such a coarse critical section, we ensure that the entire screen has 14632 been redrawn for the user before any further changes to it are made. 14633 This is especially the case when error_printf() is invoked in quick 14634 succession which would cause multiple instances of wget_update_thread() 14635 to run simultaneously, greatly increasing the lock contention. 14636 146372016-09-23 Darshit Shah <darnir@gnu.org> 14638 14639 * src/bar.c (_bar_update_thread): Fix a potential race condition 14640 146412016-09-20 Darshit Shah <darnir@gnu.org> 14642 14643 Improve accuracy of wget_human_readable 14644 14645 * libwget/utils.c (wget_human_readable): Set acc to 1000 and print to 2 14646 decimal places. This is similar to the settings used in Wget 1.x. 14647 * libwget/bar.c (_bar_slot_t): Define element human_size to be as large 14648 as _BAR_DOWNBYTES_SIZE. 14649 146502016-09-20 Tim Rühsen <tim.ruehsen@gmx.de> 14651 14652 Use wget_human_readable() from Wget1.x 14653 14654 * bootstrap.conf: Remove module 'human' 14655 * include/libwget.h.in: Amend wget_human_readable() 14656 * libwget/bar.c: Use new function wget_human_readable() 14657 * libwget/utils.c: Replace wget_human_readable() 14658 * src/wget.c: Use new function wget_human_readable() 14659 146602016-09-20 Tim Rühsen <tim.ruehsen@gmx.de> 14661 14662 Print human readable total number of bytes in progress bar 14663 14664 * include/libwget.h.in: Add wget_human_readable_alloc() 14665 * libwget/utils.c: Add wget_human_readable_alloc() 14666 * libwget/bar.c: Use wget_human_readable_alloc() 14667 * src/bar.c (bar_init): Set log function after call to wget_bar_init() 14668 * src/wget.c (main): Use wget_human_readable() to print bottom line of bar 14669 146702016-09-20 Tim Rühsen <tim.ruehsen@gmx.de> 14671 14672 Use libwget alloc functions in CSS parser 14673 14674 * libwget/css.c: New wrapper functions yyalloc() and yyrealloc() 14675 * libwget/css_tokenizer.lex: Set options noyyalloc and noyyrealloc 14676 146772016-09-19 Tim Rühsen <tim.ruehsen@gmx.de> 14678 14679 * src/wget.c: Fix calculation of quota 14680 146812016-09-19 Darshit Shah <darnir@gmail.com> 14682 14683 * README.md: Fix typo 14684 146852016-09-17 Darshit Shah <darnir@gmail.com> 14686 14687 Retain a copy of download context for progress bar 14688 14689 When the screen is scrolled or resized, we often need to redraw the 14690 entire progress bar. However, when some threads have completed 14691 downloading, this can cause those progress slots to become blank since 14692 their contexts were deleted and hence lost. To prevent this, keep a copy 14693 of the last context in the slot, so we can access it later for printing 14694 the progress bar. 14695 14696 * libwget/bar.c (_bar_slot_t): New member, last_ctx 14697 (wget_bar_deregister): Create a copy of the context just before we 14698 deregister. This ensures that we copy the latest data for printing 14699 (wget_bar_update): If no context is registered, then try printing the 14700 last context available 14701 (_bar_print_final): Make the method signature similar to 14702 wget_bar_update for consistency 14703 147042016-09-19 Tim Rühsen <tim.ruehsen@gmx.de> 14705 14706 Fix Robots Exclusion Standard 14707 14708 * include/libwget.h.in: Add function wget_list_getnext(). 14709 * libwget/list.c: Add function wget_list_getnext(). 14710 * libwget/robots.c: Fix memory leak. 14711 * src/host.c (host_remove_job): Cleanup queue after downloading and 14712 scanning robots.txt. 14713 * src/job.h (struct JOB): Add flag 'requested_by_user'. 14714 * src/wget.c (add_url_to_queue): Set 'requested_by_user', 14715 (add_url): Fix checking for disallowed paths. 14716 * tests/Makefile.am: Add test 'test-robots'. 14717 * tests/test-robots.c: New test to prove robots functionality. 14718 14719 Special handling for automatic robots.txt jobs 14720 ============================================== 14721 What can happen with --recursive and --span-hosts is that a document from hostA 14722 has links to hostB. All these links might go into the hostB queue before robots.txt 14723 is downloaded and parsed. To avoid downloading of 'disallowed' documents, the queue 14724 for hostB has to be cleaned up right after downloading and parsing robots.txt. 14725 Any links links that have been explicitly requested by the user are still downloaded. 14726 147272016-09-14 Darshit Shah <darnir@gmail.com> 14728 14729 Use gnulib human module for human readable file size 14730 14731 * bootstrap.conf: Add the human module and its dependency unistd 14732 * include/libwget.h.in: Replace method signature for wget_human_readable 14733 * libwget/bar.c (_bar_slot_t): Add a new struct memeber for storing the 14734 human readable string that it printed on the progress bar 14735 (wget_bar_init): Initialise the new human_size struct member 14736 (wget_bar_update): Use the new wget_human_readable() method 14737 (_bar_print_final): Same 14738 (wget_bar_deinit): Free the allocated memory for each slot 14739 * libwget/utils.c (wget_human_readable): Replace the old function with a 14740 call to the human_readable() method from gnulib 14741 147422016-09-14 Darshit Shah <darnir@gmail.com> 14743 14744 Handle return value of wget_thread_create 14745 14746 * src/bar.c: Handle return value of wget_bar_create and fallback to 14747 quiet mode if progress bar cannot be drawn 14748 147492016-09-14 Darshit Shah <darnir@gmail.com> 14750 14751 Update gnulib module 14752 147532016-09-11 Darshit Shah <darnir@gmail.com> 14754 14755 Split logging functions into stdout and stderr 14756 147572016-09-10 Darshit Shah <darnir@gmail.com> 14758 14759 Fix location of error messages in progress bar 14760 14761 * src/bar.c (_error_write): New logging callback function for error 14762 messages which will print the messages to the top of the progress bars 14763 * src/log.c (log_write_error): New public method for writing error 14764 messages to screen 14765 * src/log.h: Same 14766 147672016-09-14 Darshit Shah <darnir@gmail.com> 14768 14769 Separate out the last update of the progress bar 14770 14771 When the progress bar is updated for the last time (whether exiting due 14772 to error, or download complete), we may want to print information not 14773 normally presented in the progress bar, like total time taken. To allow 14774 this, separate out the printing of the last update of the progress bar 14775 14776 * include/libwget.h.in (_wget_bar_ctx): Remove unused members, final and 14777 cond 14778 * libwget/bar.c: Separate out the bar slot selection for reuse. 14779 (_bar_print_final): New method for printing the last update of a 14780 progress bar 14781 147822016-08-08 Tim Rühsen <tim.ruehsen@gmx.de> 14783 14784 Fix wget_bar_free() 14785 147862016-08-07 Darshit Shah <darnir@gmail.com> 14787 14788 Better naming of functions 14789 147902016-08-08 Darshit Shah <darnir@gmail.com> 14791 14792 Separate out bar_deinit and free'ing memory 14793 14794 * libwget/bar.c (wget_bar_init): Invoke wget_bar_deinit() to free up the 14795 memory allocated to the various progress bar sections. Similarly, invoke 14796 wget_bar_free() to free up the memory allocated to the progress bar 14797 itself 14798 147992016-09-14 Darshit Shah <darnir@gmail.com> 14800 14801 Use xfree, xmalloc and xcalloc 14802 14803 * libwget/bar.c: Replace all calls to free(), malloc() and calloc() with 14804 xfree(), xmalloc() and xcalloc() respectively 14805 * libwget/hashmap.c: Same 14806 148072016-08-07 Darshit Shah <darnir@gmail.com> 14808 14809 Change tabs to spaces when not at Beginning of line 14810 148112016-08-07 Tim Ruehsen <tim.ruehsen@gmx.de> 14812 14813 Fix compiler warnings when IOCTL is not avaialble 14814 14815 * src/utils.c (determine_screen_width): Make sure there are no unused 14816 variables when IOCTL is not available 14817 148182016-08-07 Darshit Shah <darnir@gmail.com> 14819 14820 Make compiler happy about strict-prototypes and shadowed variables 14821 148222016-08-06 Darshit Shah <darnir@gmail.com> 14823 14824 Use pthread_cancel to kill a thread 14825 14826 * include/libwget.h.in: Export new function wget_thread_cancel to kill 14827 an existing thread nicely. 14828 * libwget/thread.c: Same 14829 * src/bar.c: Use the cleaner wget_thread_cancel() method to kill the 14830 progress bar instead of polling for a variable. 14831 148322016-07-28 Darshit Shah <darnir@gmail.com> 14833 14834 Create an asynchronous progress bar 14835 14836 Give the progress bar its own thread and let it update the entire 14837 display asynchronously at a specified time interval. The existing 14838 implementation refreshes the progress bar for each network packet 14839 downloaded. Over multiple downloader threads and a high speed network 14840 connection this can lead to far too many redrawings of the screen. Also, 14841 each of the downloader threads will block while trying to acquire the 14842 thread mutex because another thread just retrieved a packet. While I 14843 haven't profiled it, it seems like there would be extremely high lock 14844 contention in the existing implementation. Instead, with a separate 14845 thread, we can update all the progress bar slots simultaneously at 14846 regular intervals. 14847 14848 * bootstrap.conf: Include Gnulib module "ioctl" 14849 * include/libwget.h.in: Define always_inline, flatten and deprecated 14850 compiler attributes for both GCC and Clang 14851 * include/libwget.h.in: Export new functions wget_human_readable, 14852 wget_bar_register and wget_bar_deregister. 14853 * include/libwget.h.in (wget_http_response_t): Add new element 14854 "cur_downloaded" to struct. This element keeps a track of the raw 14855 number of bytes downloaded in the response 14856 (_wget_bar_ctx): Define new struct for storing the progress bar 14857 context. 14858 * libwget/bar.c: Fix display of downloaded ratio. Ensure it does not 14859 exceed 100% by comparing the raw downloaded bytes instead of 14860 uncompressed downloaded bytes. 14861 Add support for printing the filename and downloaded bytes to the 14862 progress bar. 14863 * libwget/http.c (wget_http_response_cb): Store the raw number of 14864 bytes downloaded from the network in the response data 14865 * libwget/utils.c (wget_human_readable): New function to convert an 14866 int value to a human readable string 14867 * src/bar.c: Use the entire screen width instead of just 70 columns 14868 for the progress bar. 14869 Provide methods to register and deregister a progress bar 14870 context 14871 (wget_bar_update_thread): New method for the progress bar thread 14872 to update the output periodically. 14873 * src/bar.h: Same 14874 * src/job.h: Move definition of struct DOWNLOADER from main.c to 14875 here 14876 * src/progress.h: Move definition of _body_callback_context from 14877 main.c to here 14878 * src/utils.c: Add new method to determine the width of the screen 14879 * src/utils.h: Same 14880 * src/wget.c: Use the new progress bar contexts. Update, register 14881 and deregister them 14882 (main): Deinit the progress bar only if it was enabled 14883 * src/Makefile.am: Add new files, progress.h, utils.c, utils.h 14884 148852016-09-16 Tim Rühsen <tim.ruehsen@gmx.de> 14886 14887 Fix following relative links 14888 14889 * src/wget.c (html_parse): Add only absolute URLs to 'known_urls'. 14890 * tests/test-base.c: Test that the changes work. 14891 14892 Reported-by: Jaaap@github.com 14893 148942016-09-16 Tim Rühsen <tim.ruehsen@gmx.de> 14895 14896 * libwget/iri.c (_normalize_path): Fix return value 14897 148982016-09-16 Tim Rühsen <tim.ruehsen@gmx.de> 14899 14900 * src/wget.c (add_url): Check pattern list for URL domain earlier. 14901 14902 This avoids wrong messages about why not following an URL. 14903 149042016-09-15 Tim Rühsen <tim.ruehsen@gmx.de> 14905 14906 * libwget/io.c (wget_update_file): Add comment regarding owner 14907 flags when using mkstemp(). 14908 149092016-09-14 Tim Rühsen <tim.ruehsen@gmx.de> 14910 14911 * src/wget.c (add_url): Avoid theoretical NULL ptr dereference 14912 14913 Reported-by: Coverity 14914 149152016-09-14 Tim Rühsen <tim.ruehsen@gmx.de> 14916 14917 * src/wget.c (process_response_part): Fix NULL pointer dereference 14918 14919 Reported-by: Coverity 14920 149212016-09-14 Tim Rühsen <tim.ruehsen@gmx.de> 14922 14923 * libwget/encoding.c (wget_memiconv): Fix memleak 14924 14925 Reported-by: Coverity 14926 149272016-09-14 Tim Rühsen <tim.ruehsen@gmx.de> 14928 14929 * examples/print_html_urls.c (html_parse_localfile): Fix memleak 14930 14931 Reported-by: Coverity 14932 149332016-09-10 Darshit Shah <darnir@gmail.com> 14934 14935 Fix segfault in case of Basic Authentication 14936 14937 * src/wget.c(downloader_thread): Remove job only if it exists 14938 149392016-09-09 Tim Rühsen <tim.ruehsen@gmx.de> 14940 14941 * libwget/encoding.c: Use ICONV_CONST to avoid cc warning 14942 149432016-09-09 Tim Rühsen <tim.ruehsen@gmx.de> 14944 14945 Fix fnmatch() on Solaris 14946 14947 * bootstrap.conf: Add fnmatch-gnu 14948 * configure.ac: Add AC_GNU_SOURCE 14949 149502016-09-09 Tim Rühsen <tim.ruehsen@gmx.de> 14951 14952 * examples/websequencediagram.c (main): Fix printf format string 14953 149542016-09-04 Tim Rühsen <tim.ruehsen@gmx.de> 14955 14956 * configure.ac: Enable color diagnostics with ccache 14957 149582016-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 14959 14960 * tests/test.c: Fix cookie tests for when libpsl not available 14961 149622016-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 14963 14964 Support cookie prefixes __Secure- and __Host- 14965 14966 * libwget/cookie.c (_wget_cookie_normalize_cookie): Add checks 14967 for __Secure- and __Host- prefixes. 14968 * tests/test.c (test_cookies): Add test cases for the prefixes. 14969 14970 This change implements proposals from draft-ietf-httpbis-cookie-prefixes-00 14971 as modern browsers do. 14972 149732016-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 14974 14975 * include/libwget.h.in: Add attribute const to wget_hash_get_len() 14976 149772016-08-07 Tim Rühsen <tim.ruehsen@gmx.de> 14978 14979 Update gnulib submodule 14980 149812016-07-28 Darshit Shah <darnir@gmail.com> 14982 14983 Make compiler happy about using uninitialized vars 14984 14985 * tests/test.c(test_striconv): Explcitly initialize variables to 14986 NULL to make compiler happy. 14987 149882016-07-28 Darshit Shah <darnir@gmail.com> 14989 14990 Add configure support for Thread Sanitizer as well 14991 14992 * configure.ac: Add configure option to enable Thread Sanitizer. 14993 Enabling this option currently causes all tests to fil due to 14994 various data races. Some of the races can be fixed, while others are 14995 false positives. 14996 [ci skip] 14997 149982016-07-28 Darshit Shah <darnir@gmail.com> 14999 15000 Reduce the depth of git clone on travis to 3 15001 15002 * .travis.yml: We don't need any git operations on Travis. Reduce 15003 the clone depth. 15004 [ci skip] 15005 150062016-07-28 Darshit Shah <darnir@gmail.com> 15007 15008 Make ASan happy about dangling memory in test-metalink 15009 15010 * tests/test-metalink.c: Explicitly free() heap allocated memory 15011 150122016-07-23 Darshit Shah <darnir@gmail.com> 15013 15014 Enable fsanitize options when running on Travis 15015 15016 * .travis.sh: Enable UBSan and ASan sanitizers on Travis 15017 150182016-06-16 Darshit Shah <darnir@gmail.com> 15019 15020 Fix a ubsan warning due to zero length VLA 15021 150222016-07-23 Darshit Shah <darnir@gmail.com> 15023 15024 Add configure option to enable compiler sanitizers 15025 15026 Compiler sanitization options allow the code to be instrumented at 15027 compile time. They add instructions to detect various behaviours at 15028 runtime. These options usually cause significant slowdown and will 15029 often cause the program to crash. Hence, do *not* use them on 15030 production systems. They are meant to be used only by the developers 15031 for testing purposes. Three sanitization options are added to the 15032 configure options in this commit: 15033 1. Undefined Behaviour Sanitizer (ubsan): Detect issues in the code 15034 that are undefined behaviour according to the C specification 15035 2. Address Sanitizer (ASan): Detect addressing issues and memory 15036 leaks at runtime. Slightly similar to Valgrind, but using compiler 15037 added instrumentation 15038 3. Memory Sanitizer (MSan): Detect uninitialized memory reads. 15039 15040 The Address Sanitizer and Memory Sanitizer options are mutually 15041 exclusive, and hence a check for the same is added to the configure 15042 script. 15043 15044 MSan additionally requires that all libraries be instrumented too. 15045 Without that, MSan is nearly unusable due to various false 15046 positives. 15047 15048 * configure.ac: Add option to enable UBSan, ASan and MSan fsanitize 15049 options at compile time. 15050 150512016-07-22 Darshit Shah <darnir@gmail.com> 15052 15053 Update documentation to reflect actual behavior of -O 15054 15055 * docs/wget2_manual.md: Update documentation. -O does not act as a 15056 shell redirection. 15057 150582016-07-22 Darshit Shah <darnir@gmail.com> 15059 15060 Fix issue when -C and -O are passed together 15061 15062 * src/wget.c(http_create_request): When -c and -O are passed 15063 togther, ensure that the output_document is continued and not the 15064 local_filename 15065 150662016-07-22 Darshit Shah <darnir@gmail.com> 15067 15068 Add new test to check for continue and -O options 15069 15070 * tests/test-wget-1.c: Add new test. If the remote filename 15071 exists locally, then Wget2 tries to conitnue downloading that file 15072 despite being given an explicit output document. This test checks 15073 for such behaviour. 15074 150752016-07-22 Tim Rühsen <tim.ruehsen@gmx.de> 15076 15077 * src/options.c: Add linefeeds to --help output 15078 150792016-07-22 Tim Rühsen <tim.ruehsen@gmx.de> 15080 15081 Update README 15082 15083 * README.md: Update text to current status. 15084 * docs/mainpage.md: Likewise. 15085 150862016-07-21 Tim Rühsen <tim.ruehsen@gmx.de> 15087 15088 Add TLS Session Resumption with persistent cache 15089 15090 * include/libwget.h.in: Add wget_tls_session_* set of functions, 15091 add new define WGET_SSL_SESSION_CACHE 15092 * libwget/Makefile.am: Add tls_session.c to libwget 15093 * libwget/ssl_gnutls.c (struct _config): Add tls_session_cache. 15094 (struct _session_context): Add delayed_session_data. 15095 (wget_ssl_set_config_string): Handle WGET_SSL_SESSION_CACHE. 15096 (wget_ssl_open): Add False Start debug messages, 15097 Set session data if cound in cache, 15098 Write session data into cache. 15099 (wget_ssl_read_timeout): Write session data into cache when 15100 False Start has been activated. 15101 * libwget/tls_session.c: New file with session data cache implementation 15102 * src/options.c (print_help): Add new options --tls-resume and 15103 --tls-session-file. 15104 (config): Set tls_resume activated by default. 15105 (options[]): Add tls-resume and tls-session-file. 15106 (init): Create session file name, init session data cache, 15107 Set session cache for TLS layer. 15108 (deinit): Free session cache and session cache filename. 15109 * src/options.h (struct config): Add tls_session_db, tls_session_file, 15110 tls_resume. 15111 * src/wget.c (main): Save session cache if changed. 15112 15113 TLS Session Resumption speeds up TLS handshake by 1xRTT. 15114 Together with TCP Fast Open (enabled by default), we are down at 15115 1xRTT TLS handshake overhead. To get down to 0 RTT we need a different 15116 TLS protocol - TLS v1.3 will hopefully soon be standardized. 15117 151182016-07-20 Tim Rühsen <tim.ruehsen@gmx.de> 15119 15120 Beautify TLS certificate information printout 15121 15122 * libwget/ssl_gnutls.c (_print_x509_certificate_info): 15123 Beautify printout. 15124 (_print_info): Likewise. 15125 151262016-07-20 Tim Rühsen <tim.ruehsen@gmx.de> 15127 15128 * libwget/ssl_gnutls.c (wget_ssl_open): Fix #elif 15129 151302016-07-20 Tim Rühsen <tim.ruehsen@gmx.de> 15131 15132 Add support for TLS False Start 15133 15134 * include/libwget.h.in: New declaration wget_tcp_set_tls_false_start() 15135 and wget_tcp_get_tls_false_start() 15136 * libwget/net.c: New functions wget_tcp_set_tls_false_start() and 15137 wget_tcp_get_tls_false_start() 15138 * libwget/net.h: Add tls_false_start to struct wget_tcp_st 15139 * libwget/ssl_gnutls.c (_do_handshake): Print TLS False Start status. 15140 (wget_ssl_open): Enable TLS False Start for GnuTLS >= 3.5.0. 15141 * src/options.c: Add description for --tls-false-start. 15142 (struct option options): Add option --tls-false-start. 15143 (init): Call wget_tcp_set_tls_false_start(). 15144 * src/options.h (struct config): Add member tls_false_start. 15145 151462016-07-14 Tim Rühsen <tim.ruehsen@gmx.de> 15147 15148 * libwget/ssl_gnutls.c (wget_ssl_close): Retry TLS shutdown on error 15149 151502016-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 15151 15152 * po/Makevars: Removed from git repo 15153 151542016-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 15155 15156 Generate distributed ChangeLog from git log 15157 15158 * bootstrap.conf: Add gnulib module 'gitlog-to-changelog'. 15159 * Makefile.am: Generate the ChangeLog file from the git log. 15160 151612016-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 15162 15163 * configure.ac: Add parallel-tests and gnu to automake options 15164 151652016-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 15166 15167 Declare wget_iri_is(un)reserved() as const 15168 15169 * include/libwget.h.in: Declare wget_iri_isunreserved() and 15170 wget_iri_isunreserved_path() with attribute 'const' 15171 151722016-07-05 Tim Rühsen <tim.ruehsen@gmx.de> 15173 15174 Refactor downloader main loop into a state machine 15175 15176 * src/host.c (_search_queue_for_free_job): Set 'used_by' for 15177 allocated job. 15178 (_search_host_for_free_job): Same. 15179 New function _release_job(). 15180 New function host_release_jobs(). 15181 New function host_add_robotstxt_job(). 15182 (host_remove_job): Fix queue size calculation. 15183 Added several debug lines. 15184 * src/host.h: Declare host_add_robotstxt_job() and host_release_jobs() 15185 * src/job.h: New variable 'used_by' for PART, 15186 New variables 'used_by','downloader' for JOB. 15187 * src/options.c: Default config.http2_request_window to 30. 15188 Default config.http1_request_window to 10. 15189 (deinit): Free config.accept_patterns and config.reject_patterns. 15190 * src/options.h: Add 'http2_request_window' and 'http1_request_window' 15191 to 'struct config'. 15192 * src/wget.c (add_url_to_queue): Call host_add_robotstxt_job(). 15193 (add_url): Call host_add_robotstxt_job(). 15194 Refactor downloader_thread(), process_response_header(), 15195 process_head_response(), process_response_part(), process_response(). 15196 151972016-07-04 Tim Rühsen <tim.ruehsen@gmx.de> 15198 15199 Use TCP Fast Open (TFO) within TLS handshake 15200 15201 * include/libwget.h.in: New declaration wget_tcp_get_tcp_fastopen() 15202 * libwget/net.c: New function wget_tcp_get_tcp_fastopen(), 15203 (wget_tcp_connect): Support TFO 15204 * libwget/ssl_gnutls.c: New function _ssl_writev() 15205 (wget_ssl_open): Set _ssl_writev() as push function in TFO requested 15206 152072016-06-29 Tim Rühsen <tim.ruehsen@gmx.de> 15208 15209 Add benchmark scripts 15210 15211 * benchmarks/README: New file, short description 15212 * benchmarks/bench_https_http1: New file, HTTPS via HTTP/1.1 15213 * benchmarks/bench_https_http2: New file, HTTPS via HTTP/2.0 15214 152152016-06-24 Tim Rühsen <tim.ruehsen@gmx.de> 15216 15217 Refactoring to separate send, receive and HTTP state machine 15218 15219 * examples/websequencediagram.c (main): Call wget_http_request_set_body() 15220 and wget_http_send_request() instead of wget_http_send_request_with_body(). 15221 * include/libwget.h.in: Add WGET_HTTP_USER_DATA, wget_get_timemillis(), 15222 extend wget_thread_cond_wait(), add body, user_data, body_length to 15223 wget_http_request_t, remove wget_http_send_request_with_body(), 15224 add wget_http_request_get_int(), wget_http_request_set_ptr(), 15225 wget_http_request_get_ptr(), wget_http_request_set_body(). 15226 * libwget/http.c: Add wget_http_request_get_int(), 15227 wget_http_request_set_ptr(), wget_http_request_get_ptr(), 15228 wget_http_request_set_body(), 15229 remove wget_http_send_request_with_body(), 15230 (wget_http_request_to_buffer): add body to request buffer. 15231 * libwget/http_highlevel.c (wget_http_get): Replace 15232 wget_http_send_request_with_body() 15233 * libwget/iri.c: Use c-ctype.h instead of ctype.h 15234 (wget_iri_parse): Allow any numbers of / after scheme: 15235 (wget_iri_parse): Catch URIs without /after scheme: 15236 * libwget/list.c (wget_list_browse): Small code rearrangement 15237 * libwget/metalink.c (_add_mirror): Check mirror.iri for NULL 15238 * libwget/ssl_gnutls.c (send_ocsp_request): Replace 15239 wget_http_send_request_with_body() 15240 * libwget/thread.c (wget_thread_cond_signal): Add timeout param 15241 * libwget/utils.c: New function wget_get_timemillis() 15242 * src/blacklist.c: Include wget.h instead of log.h 15243 * src/blacklist.h: Fix indentation 15244 * src/host.c: Add queueing stuff 15245 * src/host.h: Reflect changes in host.c 15246 * src/job.c: Remove queueing stuff 15247 * src/job.h: Reflect changes in job.c 15248 * src/log.c: Sync stdout/stderr to correct output order 15249 * src/log.h: Remove shortcuts of print functions 15250 * src/wget.c: Remove download_part() and http_get(). 15251 Add http_send_request(), http_receive_response(), try_connection(), 15252 establish_connection(), add_statistics(), process_response_header(). 15253 Amend downloader_thread() to reflect the changes. 15254 * src/wget.h: Add shortcut defines for print functions. 15255 * tests/libtest.c (_http_server_thread): Fix compiler warning, 15256 fix debug message. 15257 New function _write_msg() to print server messages yellow. 15258 (wget_test) Add -d to wget command line. 15259 * tests/test-metalink.c (main): Add tests for V3 and V4 metalink 15260 files read from command line (-i, --force-metalink) 15261 * tests/test.c (test_iri_parse): Add test for slash-less mailto: URI 15262 152632016-06-22 Tim Rühsen <tim.ruehsen@gmx.de> 15264 15265 Use bitfields in structs where possible 15266 152672016-07-11 Tim Rühsen <tim.ruehsen@gmx.de> 15268 15269 * Makefile.am: Fix EXTRA_DIST config.rpath to build-aux/config.rpath 15270 152712016-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 15272 15273 * README.md: Fix repo cloning instructions 15274 152752016-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 15276 15277 Move ignored po/ files to po/.gitignore 15278 15279 * .gitignore: Remove po/ files 15280 * po/.gitignore: New file 15281 152822016-07-07 Ander Juaristi <ajuaristi@gmx.es> 15283 15284 Remove references to autogen.sh. Set SKIP_PO. 15285 15286 * bootstrap.conf: add SKIP_PO=1, so that we don't have to pass 15287 the --skip-po flag every time we run bootstrap. 15288 * README.md, contrib/check-hard, contrib/mingw, docs/mainpage.md, 15289 .travis.sh: amend references to autogen.sh. 15290 * autogen.sh: Removed 15291 * .gitignore: Add build-aux 15292 * po/Makevars: Remove auto-generated file from repo 15293 152942016-06-26 Ander Juaristi <ajuaristi@gmx.es> 15295 15296 Use bootstrap 15297 15298 * bootstrap, bootstrap.conf: new files. 15299 * configure.ac: invoke AC_CONFIG_MACRO_DIR and 15300 AC_CONFIG_AUX_DIR. 15301 153022016-06-16 Tim Rühsen <tim.ruehsen@gmx.de> 15303 15304 API changes for HTTP/2 parallel/async requests 15305 15306 * .gitignore: New entry examples/http_multi_get 15307 * Makefile.am: Move subdir 'examples' before 'src' 15308 * examples/Makefile.am: Add http_multi_get 15309 * examples/http_get2.c (main): Adapt code to API changes 15310 * examples/http_multi_get.c: New example to show async/parallel 15311 requests 15312 * examples/websequencediagram.c (main): Adapt code to API changes 15313 * include/libwget.h.in: New functions wget_http_request_set_header_cb, 15314 wget_http_request_set_body_cb, wget_http_request_set_int. 15315 Changed params of wget_http_get_response. 15316 Removed wget_http_get_response_func, wget_http_get_response_stream, 15317 wget_http_get_response_fd. 15318 Changed params of wget_http_get_response_cb. 15319 Add members pending_requests, received_http2_responses, 15320 pending_http2_requests to wget_http_connection_t. 15321 Add member req to wget_http_response_t. 15322 Add members header_callback, body_callback, header_user_data, 15323 body_user_data, response_keepheader to wget_http_request_t. 15324 New function types wget_http_header_callback_t and 15325 wget_http_body_callback_t. 15326 * libwget/http.c: Implementation of the changes in include/libwget.h.in. 15327 * libwget/http_highlevel.c: Adapt code to API changes 15328 * libwget/ssl_gnutls.c: Fixed defaults for SNI, OCSP and 15329 ALPN. 15330 Adapt code to API changes. 15331 * src/options.c: Reflect changes in libwget/ssl_gnutls.c 15332 * src/wget.c: Adapt code to API changes 15333 153342016-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 15335 15336 Fix compiler warnings 15337 15338 * include/libwget.h.in: Declare wget_tcp_get_protocol as PURE 15339 * libwget/base64.c (wget_base64_decode, wget_base64_encode): Fix 15340 warning about missing switch() default 15341 * libwget/http.c (_on_header_callback): Add switch default case, 15342 (wget_http_request_to_buffer): Remove unused variable 'use_proxy' 15343 * libwget/ssl_gnutls.c (_print_info): Add switch default case 15344 * libwget/vector.c (wget_vector_findext): Replace switch by if/else 15345 * src/wget.c (_get_body): Check EAGAIN != EWOULDBLOCK via cpp 15346 153472016-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 15348 15349 Fix compiler warnings in test suite 15350 15351 * tests/libtest.c (wget_test_check_filesystem): Fix printf 15352 format warning 15353 * tests/test-parse-html.c (html_dump): Add switch default case, 15354 fix printf format warning 15355 153562016-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 15357 15358 Suppress warning in auto-generated css_tokenizer.c 15359 15360 * libwget/css_tokenizer.lex: Add GCC pragmas to suppress warnings 15361 when compiling libwget/css_tokenizer.c 15362 153632016-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 15364 15365 Use gnutls_transport_[gs]et_int() in GnuTLS code if possible 15366 15367 * configure.ac: Check for gnutls_transport_get_int() 15368 * libwget/ssl_gnutls.c: Use gnutls_transport_[gs]et_int() if possible 15369 15370 Avoid int <-> pointer conversions 15371 153722016-06-15 Tim Rühsen <tim.ruehsen@gmx.de> 15373 15374 Use less warnings for gnulib compilation on --enable-gcc-warnings 15375 15376 * autogen.sh: Add option --makefile-name=gnulib.mk to gnulib-tool 15377 * configure.ac: Add 'subdir-objects' to AM_INIT_AUTOMAKE, 15378 add macro 'gl_GCC_VERSION_IFELSE' from coreutils project, 15379 enable extra gcc warnings by default when running from Git repo, 15380 disable several warnings for gnulib compilation 15381 * lib/Makefile.am: New file 15382 153832016-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 15384 15385 Add colors to debug and error output 15386 15387 * src/log.c: Add _write_out(), _write_error(), _write_info() 15388 15389 Errors are printed red (if output is a TTY), debug output is 15390 magenta/purple. Not configurable yet. 15391 153922016-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 15393 15394 Print blacklist entries as debug messages 15395 15396 * src/blacklist.c (_blacklist_print): Call debug_printf() 15397 153982016-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 15399 15400 Fix printf format warnings 15401 15402 * tests/buffer_printf_perf.c: Fix printf format warnings 15403 * tests/stringmap_perf.c: Likewise 15404 * tests/test.c: Likewise 15405 154062016-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 15407 15408 Fix printf format warnings 15409 15410 * libwget/http.c: Fix printf format warnings 15411 * libwget/http_highlevel.c: Likewise 15412 * libwget/io.c: Likewise 15413 * libwget/ssl_gnutls.c: Likewise 15414 * src/wget.c: Likewise 15415 154162016-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 15417 15418 Remove warnings from gcc's -Wjump-misses-init 15419 15420 * libwget/ssl_gnutls.c (_verify_certificate_callback): 15421 Move variable initialization upwards 15422 * src/wget.c (html_parse): Move variable initialization upwards 15423 154242016-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 15425 15426 Do not inline _no_memory() 15427 15428 * libwget/xalloc.c: Remove _GL_INLINE from _no_memory() 15429 154302016-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 15431 15432 Add gnulib module 'manywarnings' 15433 15434 * autogen.sh: Add module manywarnings 15435 * configure.ac: Add ./configure option --enable-gcc-warnings to enable 15436 many warnings 15437 * examples/Makefile.am: Add WERROR_CFLAGS and WARN_CFLAGS to AM_CFLAGS 15438 * libwget/Makefile.am: Likewise 15439 * src/Makefile.am: Likewise 15440 * tests/Makefile.am: Likewise 15441 154422016-06-14 Tim Rühsen <tim.ruehsen@gmx.de> 15443 15444 Update gnulib to latest master 15445 15446 * gnulib: Sync with upstream 15447 154482016-06-06 Tim Rühsen <tim.ruehsen@gmx.de> 15449 15450 Fix FTP PASV listening in test suite 15451 15452 * tests/libtest.c: Fix FTP PASV listening 15453 154542016-05-20 Tim Rühsen <tim.ruehsen@gmx.de> 15455 15456 Revert "Support direct mpeg streams in examples/getstream.c" 15457 15458 This reverts commit d3d7342e8063be0428549ebe4df66560b030074c. 15459 154602016-05-20 Tim Rühsen <tim.ruehsen@gmx.de> 15461 15462 Support direct mpeg streams in examples/getstream.c 15463 15464 * examples/getstream.c: Support direct mpeg streams 15465 154662016-05-20 Tim Rühsen <tim.ruehsen@gmx.de> 15467 15468 Support audio/x-scpls in examples/getstream.c 15469 15470 * examples/getstream.c (main): Fix m3u parsing, Add audio/x-scpls 15471 154722016-05-19 Tim Rühsen <tim.ruehsen@gmx.de> 15473 15474 Let test suite survive on case-sensitive filesystems (e.g. HFS+) 15475 15476 * tests/libtest.c: New function wget_test_check_filesystem(), 15477 (_empty_directory, _remove_directory): Do not print 15478 errors about missing directories. 15479 * tests/libtest.h: Add define WGET_TEST_FS_CASEMATTERS 15480 * tests/test--accept.c: Renamed files to avoid casing problems 15481 * tests/test-wget-1.c: Skip casing tests on case-sensitive filesystem 15482 154832016-05-16 Tim Rühsen <tim.ruehsen@gmx.de> 15484 15485 Check value of --progress option 15486 15487 * src/options.c: New parse function parse_progress_type 15488 * src/options.h: Change type of config.progress to char 15489 154902016-05-15 Tim Rühsen <tim.ruehsen@gmx.de> 15491 15492 Fix and enhance examples/getstream.c 15493 15494 * autogen.sh: Add gnulib module 'c-strcasestr' 15495 * examples/getstream.c: Support several playlist formats 15496 15497 getstream.c now supports .m3u, .wax, .asx, .pls, xspf playlist formats. 15498 Playlist parsing is partly hackish - it is just an example after all. 15499 15500 The title information is displayed on stderr, the audio goes to stdout to 15501 allow piping to mpg321, sox, etc. For details see the top comment in the 15502 source file. 15503 155042016-05-15 Tim Rühsen <tim.ruehsen@gmx.de> 15505 15506 Support ICY response (shoutcast response) 15507 15508 * libwget/http.c(wget_http_parse_response_header): Check for ICY response 15509 155102016-05-14 Tim Rühsen <tim.ruehsen@gmx.de> 15511 15512 Add option --content-on-error 15513 15514 * src/options.c: Add --content-on-error to options and to help text 15515 * src/options.h: Add content_on_error to struct config 15516 * src/wget.c: Check config.content_on_error 15517 * tests/test-wget-1.c: Add tests for --content-on-error 15518 155192016-05-10 Tim Rühsen <tim.ruehsen@gmx.de> 15520 15521 Do not use Proxy-Connection header field (RFC 7230 A.1.2.) 15522 15523 * libwget/http.c (wget_http_request_to_buffer): Do not create 15524 Proxy-Connection header field for requests. 15525 155262016-05-10 Tim Rühsen <tim.ruehsen@gmx.de> 15527 15528 Add MinGW recipe/script 15529 15530 * contrib/mingw: New file 15531 155322016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15533 15534 Adjust LDFLAGS for MinGW 15535 15536 * examples/Makefile.am: Add -no-install to AM_LDFLAGS 15537 * libwget/Makefile.am: Add -no-undefined to libwget_la_LDFLAGS 15538 155392016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15540 15541 Add EXEEXT to test suite executables 15542 15543 * tests/Makefile.am: Add $(EXEEXT) to executable names 15544 * tests/libtest.c: Add EXEEXT to wget2 executable, 15545 add support for EMULATOR env variable - e.g. to test with wine 15546 155472016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15548 15549 Determine home directory in a portable fashion 15550 15551 * src/options.c (get_home_dir): Use gnulib glob() to get home directory 15552 155532016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15554 15555 Fix includes for Windows 15556 15557 * libwget/http.c: Include ws2tcpip.h instead of sys/socket.h when available 15558 * libwget/net.c: Likewise 15559 155602016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15561 15562 Fix warnings 15563 15564 * libwget/http.c (wget_http_add_credentials): Fix snprintf format warnings 15565 155662016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15567 15568 Fix async sockets for Windows 15569 15570 * libwget/net.c (_set_async): Add code path for Windows 15571 155722016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15573 15574 Add missing gnulib modules 15575 15576 * autogen.sh: Add gnulib modules fsync, recv, recvfrom, socklen 15577 155782016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15579 15580 Fix build for systems without getuid() 15581 15582 * configure.ac: Check for getuid() 15583 * libwget/io.c (wget_update_file): Check for HAVE_GETUID 15584 155852016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15586 15587 Fix --enable-bzip2/--disable-bzip2 15588 15589 * configure.ac: Fix AC_ARG_WITH for bzip2 15590 155912016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15592 15593 Fix wget_srandom() 15594 15595 * include/libwget.h.in: Adjust wget_random() return type to int 15596 * libwget/random.c (libwget/random.c): Use initstate_r() for seeding, 15597 (wget_srandom): Use initstate_r() for seeding 15598 155992016-05-08 Tim Rühsen <tim.ruehsen@gmx.de> 15600 15601 gnulib: sync from upstream 15602 156032016-05-03 Tim Rühsen <tim.ruehsen@gmx.de> 15604 15605 Add --force-* options to docs 15606 15607 * wget2_manual.md: Add description for --force-css, --force-sitemap, 15608 --force-atom, --force-rss, -force-metalink 15609 156102016-04-25 Tim Rühsen <tim.ruehsen@gmx.de> 15611 15612 Fix docs for wget_srandom() 15613 15614 * libwget/random.c: Fix param description 15615 156162016-04-25 Tim Rühsen <tim.ruehsen@gmx.de> 15617 15618 Add docs for robots.txt parsing 15619 15620 * libwget/robots.c: Add Doxygen docs 15621 * docs/Makefile.am: Add rule for man page 15622 156232016-04-18 Tim Rühsen <tim.ruehsen@gmx.de> 15624 15625 Support parsing of UTF-16 HTML documents 15626 15627 * examples/print_html_urls.c (html_parse_localfile): 15628 Detect BOM (Byte Order Mark), convert UTF-16 to UTF-8 before parsing. 15629 * include/libwget.h.in: Remove wget_charset_transcode(), 15630 add wget_memiconv() and wget_striconv(). 15631 * libwget/encoding.c: Implement wget_memiconv() and wget_striconv() 15632 * src/wget.c (html_parse): Add 'html_len' param, 15633 convert UTF-16 to UTF-8 before parsing. 15634 * tests/test.c: New test for wget_memiconv(). 15635 156362016-04-09 Tim Rühsen <tim.ruehsen@gmx.de> 15637 15638 Add docs for Atom and Sitemap URL extraction functions 15639 15640 * libwget/atom_url.c: Add Doxygen docs 15641 * libwget/sitemap_url.c: Add Doxygen docs 15642 * docs/Makefile.am: Add rules for man pages 15643 * libwget/base64.c: Fix text 15644 156452016-04-09 Tim Rühsen <tim.ruehsen@gmx.de> 15646 15647 Add docs for base64 functions 15648 15649 * libwget/base64.c: Add Doxygen docs 15650 * docs/Makefile.am: Add rule for man page 15651 156522016-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 15653 15654 Add option --fsync-policy 15655 15656 * src/options.c (print_help): Add short description, 15657 (options): Add new option 15658 * src/options.h (struct config): Add fsync_policy 15659 * src/wget.c (http_get): Check config.fsync_policy before fsync() 15660 15661 --fsync-policy calls fsync() for each downloaded file. 15662 This effectively waits until all data is written to hard disk before 15663 wget continues. It slows down the performance - depending on the 15664 speed of your storage. 15665 156662016-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 15667 15668 Write files non-blocking 15669 15670 * src/wget.c (_prepare_file): Use O_NONBLOCK for open(), 15671 (_get_header): Likewise, 15672 (_get_body): Wait for writability if needed, 15673 (http_get): Code cleanups 15674 156752016-04-06 Tim Rühsen <tim.ruehsen@gmx.de> 15676 15677 * libwget/net.c (wget_tcp_read): Use recvfrom() instead of write() 15678 156792016-04-04 Tim Rühsen <tim.ruehsen@gmx.de> 15680 15681 Fix error message 15682 15683 * libwget/http.c (wget_http_get_response_cb): Fix message about 15684 incomplete download 15685 156862016-04-03 Tim Rühsen <tim.ruehsen@gmx.de> 15687 15688 * po/POTFILES.in: libwget/compat.c -> libwget/strlcpy.c 15689 156902016-04-03 Tim Rühsen <tim.ruehsen@gmx.de> 15691 15692 Renamed libwget/compat.c -> libwget/strlcpy.c 15693 156942016-04-02 Tim Rühsen <tim.ruehsen@gmx.de> 15695 15696 Rename --follow-metalink to --metalink 15697 15698 * src/options.c, src/options.h, src/wget.c, tests/test-metalink.c: 15699 Rename --follow-metalink to --metalink, rename config.follow_metalink 15700 to config.metalink. 15701 157022016-04-02 Tim Rühsen <tim.ruehsen@gmx.de> 15703 15704 Share code for Metalink V3 and V4 15705 15706 * libwget/metalink.c: New functions _add_piece, _add_file_hash, 15707 _add_mirror called by metalink V3 + V4 code in _metalink_parse(). 15708 157092016-04-02 Tim Rühsen <tim.ruehsen@gmx.de> 15710 15711 Add tests for RFC 6249 Metalink in HTTP headers 15712 15713 * tests/test-metalink.c: Add tests 15714 157152016-04-02 Tim Rühsen <tim.ruehsen@gmx.de> 15716 15717 Fix memory leak in Metalink V3 parser 15718 15719 * libwget/metalink.c (_metalink_parse): Set destructor for mirror vector 15720 157212016-03-31 Tim Rühsen <tim.ruehsen@gmx.de> 15722 15723 Add tests for Metalink 15724 15725 * tests/Makefile.am: Add test-metalink 15726 * tests/test-metalink.c: New file 15727 15728 This new tests are 15729 - --no-follow-metalink 15730 - metalink v3, without pieces 15731 - metalink v4, without pieces 15732 - metalink v4, with pieces 15733 157342016-03-30 Tim Rühsen <tim.ruehsen@gmx.de> 15735 15736 Add --force-metalink 15737 15738 * libwget.h.in: Remove metalink3_parse, metalink4_parse, 15739 Add wget_metalink_parse 15740 * libwget/metalink.c: Merge metalink4_parse() and metalink3_parse() 15741 into wget_metalink_parse(), 15742 Only store HTTP and HTTPS mirrors 15743 * src/job.c: Adjust debug message 15744 * src/options.c: Add --force-options 15745 * src/options.h: Add config.force_metalink 15746 * src/wget.c: New function metalink_parse_localfile() 15747 157482016-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 15749 15750 *src/options.c: Allow --(no-)follow-metalink without argument 15751 157522016-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 15753 15754 Do not use hard-coded user-agent in robots library code 15755 15756 * include/libwget.h.in: Update prototype of wget_robots_parse() 15757 * libwget/robots.c (wget_robots_parse): Add client name as parameter 15758 * src/wget.c: Call wget_robots_parse() with PACKAGE_NAME as client name 15759 157602016-03-29 Tim Rühsen <tim.ruehsen@gmx.de> 15761 15762 Set default User-Agent: to PACKAGE_NAME"/"PACKAGE_VERSION 15763 15764 * src/options.c: Set PACKAGE_NAME"/"PACKAGE_VERSION as user agent 15765 157662016-03-28 Giuseppe Scrivano <gscrivan@redhat.com> 15767 15768 wget: new option --follow-metalink 15769 15770 * src/options.h (struct config): Define follow_metalink. 15771 * src/options.c (print_help): Add description for follow-metalink. 15772 (struct options): Add follow-metalink. 15773 (struct config): Initialize follow_metalink. 15774 (init): do not follow metalink files when --mirror is used. 15775 * src/wget.c (downloader_thread): Skip following metalinks when follow_metalink is off. 15776 (_get_header): Store the metalink file when follow_metalink is false. 15777 157782016-03-28 Tim Rühsen <tim.ruehsen@gmx.de> 15779 15780 Allow options and URLs in any order 15781 15782 * src/options.c (parse_command_line): Reorder URLs after options 15783 * src/options.h: Fix init() prototype 15784 * src/wget.c, tests/test-parse-html.c, tests/test.c: 15785 Fix main() argv argument 15786 157872016-03-27 Tim Rühsen <tim.ruehsen@gmx.de> 15788 15789 Fix metalink code to download files without pieces 15790 15791 * src/job.c (job_validate_file): Accept files without pieces 15792 157932016-03-27 Tim Rühsen <tim.ruehsen@gmx.de> 15794 15795 * libwget/metalink.c (_metalink4_parse): Set piece.position to 0 15796 157972016-03-26 Giuseppe Scrivano <gscrivan@redhat.com> 15798 15799 src/wget.c (_get_header): Do not store metalink files 15800 15801 Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com> 15802 158032016-03-22 Tim Rühsen <tim.ruehsen@gmx.de> 15804 15805 Add docs in markdown format 15806 15807 * DoxygenLayout.xml: Set mainpage title 15808 * libwget.doxy.in: Scan docs, add mainpage.md as main page 15809 * mainpage.md: New file 15810 * faq.md: New file 15811 * wget2_manual.md: New file, derived from wget.texi 15812 158132016-03-24 Giuseppe Scrivano <gscrivan@redhat.com> 15814 15815 http: fix multi part files 15816 15817 * src/wget.c (download_part): Read temp part for the temporary files. 15818 (_get_header): Write part file to the destination file. 15819 (http_get): Keep a temporary fd for the file parts. 15820 (struct _body_callback_context): Add 'head' and 'part'. 15821 15822 Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com> 15823 158242016-03-20 Giuseppe Scrivano <gscrivan@redhat.com> 15825 15826 Fix compiler warnings 15827 15828 * include/libwget.h.in (wget_bar_init, wget_bar_update, 15829 wget_bar_print): Change types to int and off_t. 15830 * libwget/bar.c (wget_bar_init, wget_bar_update, 15831 wget_bar_print): Change types to int and off_t. 15832 * src/wget.c (_prepare_file): Add prototype. Change type of rc to 15833 size_t. 15834 15835 Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com> 15836 158372016-03-19 Giuseppe Scrivano <gscrivan@redhat.com> 15838 15839 wget: use safe_write module from gnulib 15840 15841 * autogen.sh: Add module 'safe-write'. 15842 * src/wget.c (_prepare_file): Use safe_write instead of write. 15843 (_get_body): Likewise. 15844 15845 Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com> 15846 158472016-03-18 Giuseppe Scrivano <gscrivan@redhat.com> 15848 15849 Do not parse more than 10M and stream directly to disk 15850 15851 * src/wget.c (downloader_thread): drop code to save/append to file. 15852 (_prepare_file): New function based on _save_file. Open the fd and 15853 don't write any payload. 15854 (_body_callback_context): Add outfd, max_memory, length. 15855 (_get_header): Set the destination file here and keep the fd. 15856 (_get_body): Keep in memory only what was specified, stream the rest 15857 directly to file. 15858 (http_get): Always use wget_http_get_response_cb instead of 15859 wget_http_get_response. Close the file fd that was opened in 15860 _prepare_file. 15861 15862 Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com> 15863 158642016-03-16 Giuseppe Scrivano <gscrivan@redhat.com> 15865 15866 Use size_t instead of int for progress bar status 15867 15868 * include/libwget.h.in (wget_bar_init): Change type of nslots and max 15869 from int to size_t. 15870 (wget_bar_deinit): Likewise. 15871 (wget_bar_update): Likewise. 15872 (wget_bar_print): Likewise. 15873 (wget_bar_vprintf): Likewise. 15874 (wget_bar_printf): Likewise. 15875 * libwget/bar.c (wget_bar_init): Change type of nslots and max 15876 from int to size_t. 15877 (wget_bar_deinit): Likewise. 15878 (wget_bar_update): Likewise. 15879 (wget_bar_print): Likewise. 15880 (wget_bar_vprintf): Likewise. 15881 (wget_bar_printf): Likewise. 15882 * src/bar.h (bar_init): Change type of nslots and max from 15883 int to size_t. 15884 (bar_deinit): Likewise. 15885 (bar_update): Likewise. 15886 (bar_print): Likewise. 15887 (bar_vprintf): Likewise. 15888 (bar_printf): Likewise. 15889 * src/bar.c (bar_init): Change type of nslots and max from 15890 int to size_t. 15891 (bar_deinit): Likewise. 15892 (bar_update): Likewise. 15893 (bar_print): Likewise. 15894 (bar_vprintf): Likewise. 15895 (bar_printf): Likewise. 15896 15897 Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com> 15898 158992016-03-16 Giuseppe Scrivano <gscrivan@redhat.com> 15900 15901 libwget/http.c: Rename HTTP_CTYPE_SEPERATOR to HTTP_CTYPE_SEPARATOR 15902 15903 Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com> 15904 159052016-03-18 Tim Rühsen <tim.ruehsen@gmx.de> 15906 15907 Tune cookie parsing 15908 15909 * include/libwget.h.in: Add sort_age to struct wget_cookie_st 15910 * libwget/cookie.c: Many code enhancements 15911 * libwget/http.c (wget_http_parse_setcookie): Relax parsing, 15912 (wget_http_parse_setcookie): Allow \n and \r\n as EOL 15913 * libwget/private.h: Add missing #includes 15914 * tests/Makefile.am: Add disabled test-cookies-http_state 15915 * tests/test-cookies-http_state.c: New cookie test 15916 15917 There still 30 out of 220 tests failing, before this commit we had 15918 135 failing. The left over tests are mainly corner cases, but feel free 15919 to work on it. When all tests pass, we enable test-cookies-http_state 15920 in Makefile.am. 15921 159222016-03-14 Tim Rühsen <tim.ruehsen@gmx.de> 15923 15924 Update .gitignore 15925 15926 * .gitignore: Updated 15927 159282016-03-14 Tim Rühsen <tim.ruehsen@gmx.de> 15929 15930 Add new examples websequencediagram[_highlevel] 15931 15932 * examples/websequencediagram.c: New file 15933 * examples/websequencediagram_high.c: New file 15934 * examples/Makefile.am: Add websequencediagram[_highlevel] 15935 15936 These examples show how to do two requests on one connection, 15937 a POST and a GET, saving the downloaded body to disk. 15938 websequencediagram uses the low-level API. 15939 websequencediagram_highlevel uses the high-level API. 15940 159412016-03-14 Tim Rühsen <tim.ruehsen@gmx.de> 15942 15943 Add WGET_HTTP_SCHEME, WGET_HTTP_BODY, WGET_HTTP_BODY_SAVEAS for wget_http_get() 15944 15945 * include/libwget.h.in: Add WGET_HTTP_SCHEME, WGET_HTTP_BODY, 15946 WGET_HTTP_BODY_SAVEAS 15947 * src/http_highlevel.c: Implement WGET_HTTP_SCHEME, WGET_HTTP_BODY, 15948 WGET_HTTP_BODY_SAVEAS 15949 159502016-03-14 Tim Rühsen <tim.ruehsen@gmx.de> 15951 15952 Use wget2 without libtool wrapper for test suite 15953 15954 * src/Makefile.am: Add wget2_noinstall target 15955 * src/wget.c: On exit, only release memory within test suite 15956 * tests/libtest.c: Call wget2_noinstall for testing 15957 159582016-03-14 Tim Rühsen <tim.ruehsen@gmx.de> 15959 15960 Fix superfluous 'Unsupported cookie-av' message 15961 15962 * libwget/http.c (wget_http_parse_setcookie): Break on end-of-string 15963 159642016-03-12 Tim Rühsen <tim.ruehsen@gmx.de> 15965 15966 Add 'make check-valgrind' to simplify valgrind checking 15967 15968 * Makefile.am: Add rule 'check-valgrind' 15969 159702016-03-01 Tim Rühsen <tim.ruehsen@gmx.de> 15971 15972 Add support for HTML5 srcset attribute in IMG tags 15973 15974 * libwget/html_url.c: Add scanning srcset values 15975 * tests/test--page-requisites.c: Add srcset test cases 15976 159772016-03-01 Tim Rühsen <tim.ruehsen@gmx.de> 15978 15979 Use TravisCI .svg image 15980 15981 * README.md: Use TravisCI .svg image 15982 159832016-02-12 Tim Rühsen <tim.ruehsen@gmx.de> 15984 15985 Remove unneeded variable in options.c 15986 15987 * src/options.c (main): Remove varibale 'truncated' 15988 159892016-02-07 Tim Rühsen <tim.ruehsen@gmx.de> 15990 15991 Add docs for memory functions 15992 15993 * libwget/mem.c: Add docs 15994 * docs/Makefile.am: Add rule for man pages 15995 * include/libwget.h.in: Cosmetic adjustments 15996 159972016-02-07 Tim Rühsen <tim.ruehsen@gmx.de> 15998 15999 Add docs for MD5 convenience functions 16000 16001 * libwget/md5.c: Add docs 16002 * docs/Makefile.am: Add rule for man pages 16003 160042016-02-07 Tim Rühsen <tim.ruehsen@gmx.de> 16005 16006 * libwget/http.c: Remove #define MD5_DIGEST_LENGTH 16007 160082016-02-07 Tim Rühsen <tim.ruehsen@gmx.de> 16009 16010 * libwget/io.c: Remove bug introduced when fixing Coverity 16011 160122016-02-07 Tim Rühsen <tim.ruehsen@gmx.de> 16013 16014 Fix / silence some Coverity findings 16015 160162016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16017 16018 * docs/Makefile.am: Remove redundant list of man files 16019 160202016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16021 16022 * libwget/iri.c: Make iri_ports static 16023 160242016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16025 16026 * docs/Makefile.am: Add rule for xalloc man page 16027 160282016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16029 16030 * libwget/ssl_gnutls.c: Fix the fix of 48c1f0961ee566287543f4149dbfc7fb5ba2b7bf 16031 160322016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16033 16034 * libwget/io.c: Fix some small glitches in Doxygen area 16035 160362016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16037 16038 * libwget/encoding.c: Use '#ifdef WITH_LIBIDN' instead of '#if' 16039 160402016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16041 16042 Add docs for printf functions 16043 16044 * libwget/printf.c: Convert GTK-DOC format into Doxygen 16045 * docs/Makefile.am: Add rule for man pages 16046 160472016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16048 16049 * libwget/ssl_gnutls.c: Fix clang warnings 16050 160512016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16052 16053 Add docs for memory functions 16054 16055 * libwget/xalloc.c: Convert GTK-DOC format into Doxygen 16056 * docs/Makefile.am: Add rule for man pages 16057 160582016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16059 16060 * libwget/utils.c: Fix docs, rename function param 16061 160622016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16063 16064 Fix argument to srandom() 16065 16066 * libwget/random.c: Cast argument to unsigned int 16067 160682016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16069 16070 Fix docs for wget_srandom() 16071 16072 * libwget/random.c: Fix docs for wget_srandom() 16073 160742016-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 16075 16076 Fix check for BUILDING_LIBWGET 16077 16078 * include/libwget.h.in: Use '#if defined' instead of '#if' 16079 160802016-02-03 Tim Rühsen <tim.ruehsen@gmx.de> 16081 16082 Cosmetics 16083 16084 * libwget/thread.c: Fix comment after #endif 16085 * src/wget.c (main): Remove unwanted spaces in if clause 16086 160872016-02-03 Giuseppe Scrivano <gscrivan@redhat.com> 16088 16089 autogen.sh: add 'stdbool' to libwget_gnulib_modules 16090 160912016-02-02 Giuseppe Scrivano <gscrivan@redhat.com> 16092 16093 Honor --disable-threads and support no threading 16094 16095 * include/libwget.h.in: Include <stdbool.h>. 16096 Define wget_thread_support(). 16097 * libwget/thread.c: Change condition to check 16098 for threading support. 16099 (wget_thread_support): New function. 16100 (wget_thread_start): [!USE_POSIX_THREADS && !USE_PTH_THREADS] 16101 Invoke directly the function. 16102 * src/wget.c (main): If no threads are supported then set the 16103 number of max threads to 1. If the queue is not empty, call again the 16104 downloader function. It will be used only in case no threads are 16105 available. 16106 (downloader_thread): Do not wait for new elements in the queue 16107 when there are no threads. 16108 * tests/libtest.c (wget_test_start_server): Skip the test if 16109 there is no threads support. 16110 161112016-02-02 Giuseppe Scrivano <gscrivan@redhat.com> 16112 16113 Fix build when threads are not used 16114 16115 * include/libwget.h.in: [! PTHREAD_MUTEX_INITIALIZER]: Define 16116 WGET_THREAD_COND_INITIALIZER. 16117 161182016-01-25 Tim Rühsen <tim.ruehsen@gmx.de> 16119 16120 Set default for --waitretry to 10s 16121 16122 src/options.c (struct config config): Init .waitretry with 16123 10000 milliseconds. 16124 161252016-01-25 Tim Rühsen <tim.ruehsen@gmx.de> 16126 16127 Check for CTRL-C in timer loops 16128 16129 * src/wget.c (downloader_thread, download_part): Abort on CTRL-C 16130 161312016-01-25 Tim Rühsen <tim.ruehsen@gmx.de> 16132 16133 Updated copyright year for all relevant files 16134 161352016-01-25 Tim Rühsen <tim.ruehsen@gmx.de> 16136 16137 Add helper script for updating copyright dates 16138 16139 contrib/update_copyright: New file 16140 161412016-01-24 Tim Rühsen <tim.ruehsen@gmx.de> 16142 16143 Disable valgrind tests on Travis CI for Mac OS X 16144 16145 * .travis.yml: Do not install valgrind in OS X 16146 * .travis.sh: Disable valgrind tests for OS X 16147 161482016-01-24 Tim Rühsen <tim.ruehsen@gmx.de> 16149 16150 Install valgrind for Travis CI on linux 16151 16152 * .travis.yml: Add valgrind to packages 16153 * .travis.sh: Output tests/test.log 16154 161552016-01-24 Tim Rühsen <tim.ruehsen@gmx.de> 16156 16157 Switch to Travis-CI valgrind testing 16158 16159 * .travis.sh: Add --enable-valgrind-tests to ./configure, 16160 export VALGRIND_TESTS=1 16161 * .travis.yml: Install valgridn on Mac OS X 16162 * tests/libtest.c: Add --max-threads=1 --prefer-family=ipv4 to wget2 command 16163 161642016-01-23 Tim Rühsen <tim.ruehsen@gmx.de> 16165 16166 Add $(LIB_PTHREAD) to makefiles 16167 16168 * examples/Makefile.am, libwget/Makefile.am, src/Makefile.am, 16169 tests/Makefile.am: Add $(LIB_PTHREAD) 16170 161712016-01-22 Tim Rühsen <tim.ruehsen@gmx.de> 16172 16173 Add thread-safe wrapper code around random() and srandom() 16174 16175 * docs/Makefile.am: Add wget_random and wget_srandom to docs 16176 * include/libwget.h.in: Add wget_random and wget_srandom 16177 * libwget/Makefile.am: Add random.c 16178 * libwget/http.c: Use wget_random() 16179 * libwget/random.c: New file with wget_random and wget_srandom 16180 * src/options.c: Remove call to srandom() 16181 161822016-01-22 Tim Rühsen <tim.ruehsen@gmx.de> 16183 16184 Fix Travis-CI files for Mac OS X 16185 16186 * .travis.sh: Remove 'allow_failures', Call .travis.sh script, 16187 Install needed packages via 'brew', Remove configuration, building, testing 16188 * .travis.sh: Add configuration, building, testing 16189 161902016-01-22 Tim Rühsen <tim.ruehsen@gmx.de> 16191 16192 Fix test suite for Mac OSX 16193 16194 * tests/libtest.c (_http_server_thread): Accept HTTP msg when chunked, 16195 Fix removing test directories, 16196 (wget_test_start_server) Remove stale test directories, 16197 Prefer IPv4, Start FTPS thread only if needed, 16198 (_scan_for_unexpected) Skip test for unexpected files on Mac OSX, 16199 (wget_test) Add --max-threads and --prefer-family=ipv4 to wget2 command, 16200 Remove MSWindows conditional code 16201 162022016-01-22 Tim Rühsen <tim.ruehsen@gmx.de> 16203 16204 Remove libpthread check from configure.ac 16205 16206 * configure.ac: Remove check for libpthread 16207 * include/libwget.h.in: Check HAVE_PTHREAD_H, 16208 Do not undefine gettext 16209 * libwget/thread.c: Check HAVE_PTHREAD_H 16210 16211 This check is done by gnulib 'pthread' module. 16212 162132016-01-22 Tim Rühsen <tim.ruehsen@gmx.de> 16214 16215 Use gnulib gettime() instead of clock_gettime() 16216 16217 * src/log.c, src/wget.c: Call gettime() instead of clock_gettime() 16218 162192016-01-22 Tim Rühsen <tim.ruehsen@gmx.de> 16220 16221 Add gnulib modules gettime and pthread 16222 16223 * autogen.sh: Add gnulib modules gettime and pthread 16224 162252016-01-22 Tim Rühsen <tim.ruehsen@gmx.de> 16226 16227 Fix DNS caching 16228 16229 * libwget/net.c (_wget_dns_cache_add): Allow NULL for port 16230 Remove several #ifdefs (should be handled by gnulib now) 16231 162322016-01-19 Tim Rühsen <tim.ruehsen@gmx.de> 16233 16234 Let mkdir() be handled by gnulib 16235 16236 * src/wget.c: Remove conditional around mkdir() 16237 162382016-01-19 Giuseppe Scrivano <gscrivan@redhat.com> 16239 16240 Fix the /dev/null usage 16241 16242 * src/wget.c: do not create the directory if the first character is 16243 '/'. 16244 * tests/test-base.c: add test for -O/dev/null. 16245 162462016-01-19 Giuseppe Scrivano <gscrivan@redhat.com> 16247 16248 Fix compiler warning 16249 16250 * libwget/ssl_gnutls.c (_print_info): Always consider GNUTLS_CRD_SRP 16251 162522016-01-19 Tim Rühsen <tim.ruehsen@gmx.de> 16253 16254 Add docs for utils functions 16255 16256 * docs/Makefile.am: Add libwget-utils.3 16257 * libwget/utils.c: Add utils function docs 16258 162592016-01-19 Tim Rühsen <tim.ruehsen@gmx.de> 16260 16261 Add docs for I/O functions 16262 16263 * docs/Makefile.am: Add libwget-io.3 16264 * libwget/io.c: Add I/O function docs 16265 162662016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16267 16268 Add docs for hash functions 16269 16270 * docs/Makefile.am: Add libwget-hash.3 16271 * include/libwget.h.in: Remove hash function docs 16272 * libwget/hashfile.c: Add hash function docs 16273 162742016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16275 16276 Do not sort document members 16277 16278 * docs/libwget.doxy.in: Set SORT_MEMBER_DOCS = NO 16279 162802016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16281 16282 Enable reading of config.h for docs generation 16283 16284 * docs/libwget.doxy.in: Enable reading of config.h 16285 162862016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16287 16288 Extend .gitignore 16289 16290 * .gitignore: Add entries and sort with LC_ALL=C 16291 162922016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16293 16294 Add contrib/check-hard for maintainer testing 16295 16296 * contrib/check-hard: New file 16297 162982016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16299 16300 Add logo to HTML docs 16301 16302 * docs/gen-logo.sh: Add script to generate logo via imagemagick 16303 * docs/wget-logo.png: Add generated logo 16304 * docs/libwget.doxy.in: Include logo into docs 16305 163062016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16307 16308 Add wget_list_t to docs group libwget-list 16309 16310 * include/libwget.h.in: Add wget_list_t to docs group libwget-list 16311 163122016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16313 16314 Remove NULL check in list code 16315 16316 * libwget/list.c (wget_list_remove): Remove checking first param for NULL. 16317 16318 Since the function is declared with the first param being NON-NULL, gcc will 16319 silently remove the check anyways. Clang even complains about the check. 16320 163212016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16322 16323 Remove NULL check in vector code 16324 16325 * libwget/vector.c (wget_vector_add_str): Remove checking second param 16326 for NULL. 16327 16328 Since the function is declared with 's' being NON-NULL, gcc will silently 16329 remove the check anyways. Clang even complains about the check. 16330 163312016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16332 16333 Fix resource leak when using libgcrypt for hashing 16334 16335 * libwget/hashfile.c (wget_hash_deinit): Close gcrypt context 16336 163372016-01-18 Tim Rühsen <tim.ruehsen@gmx.de> 16338 16339 Add error code WGET_E_TLS_DISABLED 16340 16341 * include/libwget.h.in: New define for WGET_E_TLS_DISABLED 16342 163432016-01-17 Tim Rühsen <tim.ruehsen@gmx.de> 16344 16345 Tweak doxygen config file 16346 16347 * docs/libwget.doxy.in: Hide absolute path in file names, 16348 exclude private header files of libwget/, 16349 exclude symbols starting with _ 16350 163512016-01-17 Tim Rühsen <tim.ruehsen@gmx.de> 16352 16353 Use README.md as main page in HTML docs 16354 16355 * docs/libwget.doxy.in: Add README.md to INPUT 16356 163572016-01-17 Tim Rühsen <tim.ruehsen@gmx.de> 16358 16359 Use doxygen instead of gtk-doc 16360 16361 * Removed all gtk-doc files and references. 16362 * Added doxygen files and rules 16363 163642016-01-15 Tim Rühsen <tim.ruehsen@gmx.de> 16365 16366 Fix double linkage finally 16367 16368 * tests/Makefile.am: Link only once per lib to avoid warnings 16369 163702016-01-15 Tim Rühsen <tim.ruehsen@gmx.de> 16371 16372 Fix wrong argument fetch in buffer printf routine 16373 16374 * libwget/buffer_printf.c (wget_buffer_vprintf_append): Fetch long value 16375 from stack instead of long long with %l[dioux] 16376 16377 Found on Solaris 11 SPARC 16378 163792016-01-15 Tim Rühsen <tim.ruehsen@gmx.de> 16380 16381 Fix Solaris issue in tests/Makefile.am 16382 16383 * tests/Makefile.am: Link only once per lib to avoid warnings 16384 163852016-01-15 Tim Rühsen <tim.ruehsen@gmx.de> 16386 16387 Fix libwget/Makefilea. for Solaris 16388 16389 * libwget:/Makefile.am: Add ../lib/libgnu.la to test_linking_LDADD 16390 163912016-01-15 Tim Rühsen <tim.ruehsen@gmx.de> 16392 16393 Add gnulib module progname 16394 16395 Without this we have an unresolved symbol 'program_name' on 16396 Solaris. 16397 163982016-01-15 Tim Rühsen <tim.ruehsen@gmx.de> 16399 16400 Include config.h on top in css_tokenizer.c 16401 16402 * libwget/Makefile.am: Add #include "config.h" to top of css_tokenizer.c 16403 16404 Found this issue on OpenSolaris 16405 164062016-01-15 Tim Rühsen <tim.ruehsen@gmx.de> 16407 16408 Add symbol visibility control to libwget 16409 16410 * autogen.sh: Add gnulib module lib-symbol-visibility, 16411 add --libtool to gnulib-tool invokation 16412 * examples/Makefile.am: Remove -static from AM_LDFLAGS, 16413 remove ../lib/libgnu.a from LDADD 16414 * include/libwget.h.in: Define LIBWGET_EXPORT, 16415 add LIBWGET_EXPORT to all visible symbols 16416 * libwget/Makefile.am: Remove AM_CPPFLAGS and LDADD, 16417 add libwget_la_CPPFLAGS and libwget_la_LIBADD, 16418 add test_linking_CPPFLAGS, fix test_linking_LDADD 16419 * libwget/iri.c: Add LIBWGET_EXPORT to wget_iri_schemes and iri_ports 16420 * src/Makefile.am: Remove wget2_LDFLAGS, fix wget2_LDADD 16421 * tests/Makefile.am: Fix LDADD, fix test_LDADD, fix test_parse_html_LDADD, 16422 fix libtest_la_CPPFLAGS and libtest_la_LIBADD 16423 * tests/libtest.h: Add LIBWGET_EXPORT to symbols 16424 164252016-01-15 Tim Rühsen <tim.ruehsen@gmx.de> 16426 16427 Exclude private header files from gtk-doc parsing 16428 16429 * docs/libwget/Makefile.am: Add net.h and css_tokenizer.h to IGNORE_HFILES 16430 164312016-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 16432 16433 Remove vasprintf from configure.ac 16434 16435 * configure.ac: Remove vasprintf from AC_CHECK_FUNCS 16436 164372016-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 16438 16439 Add more gnulib modules 16440 16441 * autogen.sh: Add more gnulib modules 16442 16443 What came across my eyes: 16444 closedir 16445 fclose 16446 fdopen 16447 fflush 16448 fopen 16449 fstat 16450 mkstemp 16451 opendir 16452 spawn-pipe 16453 popen 16454 random_r (change random() -> random_r() in a follow-up commit) 16455 readdir 16456 rename 16457 stat 16458 strtoll 16459 time_r 16460 unlink 16461 164622016-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 16463 16464 Removed redundant buffer print routines 16465 16466 Removed wget_buffer_vprintf_append, wget_buffer_printf_append, 16467 wget_buffer_vprintf2 and wget_buffer_printf2. 16468 Renamed their *2 counterpart to their old names. 16469 164702016-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 16471 16472 Added printf-style functions 16473 16474 * libwget/printf.c: New file 16475 * docs/libwget/Makefile.am: Add libwget-printf.3 16476 * docs/libwget/libwget-docs.sgml: Add printf section 16477 * docs/libwget/libwget-sections.txt: Add SECTION libwget-printf 16478 * include/libwget.h.in: Add function prototypes 16479 * libwget/Makefile.am: Add printf.c 16480 * libwget/base64.c, libwget/http.c, libwget/iri.c, libwget/logger.c, 16481 libwget/md5.c, libwget/vector.c, src/options.c: 16482 Use new functions instead of asprintf/vasprintf 16483 164842016-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 16485 16486 Use c-ctype.h where possible 16487 16488 * libwget/atom_url.c, libwget/buffer_printf.c, libwget/css.c, 16489 libwget/html_url.c, libwget/http.c, libwget/rss_url.c, libwget/sitemap_url.c, 16490 src/options.c src/wget.c, tests/libtest.c, tests/test-parse-html.c: 16491 Use c-ctype.h where possible 16492 164932016-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 16494 16495 Use ASCII comparison fpr file extensions 16496 16497 * src/wget.c (_save_file): Use wget_strcasecmp_ascii 16498 164992016-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 16500 16501 Replace strcpy 16502 16503 * libwget/net.c (_wget_dns_cache_add, wget_tcp_set_bind_address): 16504 Replace strcpy by memcpy 16505 * src/wget.c (_convert_links): Replace strcpy by snprintf 16506 * src/wget.c (downloader_thread): Replace strcpy by C99 initializer 16507 * src/wget.c (_save_file): Replace strcpy by memcpy 16508 * tests/test-wget-1.c (main): Replace strcpy by memcpy 16509 165102016-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 16511 16512 Remove redundant code from buufer routines 16513 16514 * libwget/buffer.c (wget_buffer_strcpy, wget_buffer_strcat): 16515 Remove redundant code 16516 165172016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16518 16519 Add gnulib modules memchr, memmove, memset, mkdir, strerror, strstr 16520 16521 * autogen.sh: Add gnulib modules memchr, memmove, memset, mkdir, strerror, strstr 16522 * configure.ac: Remove checks for memchr, memmove, memset, mkdir, 16523 strerror, strstr, strchr, strrchr 16524 165252016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16526 16527 Add gnulib module inline 16528 16529 * autogen.sh: Add gnulib module inline 16530 * configure.ac: Remove AC_C_INLINE 16531 * libwget/hashmap.c: Replace inline by _GL_INLINE 16532 * libwget/iri.c: Replace inline by _GL_INLINE 16533 * libwget/ssl_gnutls.c: Replace inline by _GL_INLINE 16534 * libwget/utils.c: Replace inline by _GL_INLINE 16535 * libwget/xalloc.c: Replace inline by _GL_INLINE, 16536 include config.h 16537 165382016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16539 16540 Add gnulib modules calloc-posix, malloc-posix, realloc-posix 16541 16542 * autogen.sh: Add gnulib modules calloc-posix, malloc-posix, realloc-posix 16543 * configure.ac: Remove two comment lines 16544 * libwget/xalloc.c: Remove rpl_malloc() and rpl_realloc() 16545 165462016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16547 16548 Add gnulib module setlocale 16549 16550 * autogen.sh: Add gnulib module setlocale 16551 * configure.ac: Remove searching for setlocale, locale.h, sys/time.h 16552 * src/wget.c: Move #include <locale.h> (cosmetics) 16553 * examples/Makefile.am: Add $(LIBTHREAD), @INTL_MACOSX_LIBS@ to LDADD 16554 * libwget/Makefile.am: Add to $(LTLIBTHREAD), @INTL_MACOSX_LIBS@ LDADD 16555 * src/Makefile.am: Add $(LIBTHREAD), @INTL_MACOSX_LIBS@ to LDADD 16556 * tests/Makefile.am: Add $(LIBTHREAD), @INTL_MACOSX_LIBS@ to LDADD 16557 165582016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16559 16560 Add gnulib module sys_stat, sys_file, sys_types 16561 16562 * autogen.sh: Add gnulib module sys_stat, sys_file, sys_types 16563 165642016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16565 16566 Add gnulib module sys_time 16567 16568 * autogen.sh: Add gnulib module sys_time 16569 165702016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16571 16572 Add gnulib module errno 16573 16574 * autogen.sh: Add gnulib module errno 16575 165762016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16577 16578 Add gnulib module langinfo 16579 16580 * autogen.sh: Add gnulib module langinfo 16581 * configure.ac: Remove searching for stringprep.h and langinfo.h 16582 * libwget/encoding.c: Remove conditional code 16583 165842016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16585 16586 Add gnulib module iconv 16587 16588 * autogen.sh: Add gnulib module iconv 16589 * configure.ac: Remove AM_ICONV 16590 * examples/Makefile.am: Add $(LIBICONV) to LDADD 16591 * libwget/Makefile.am: Add $(LIBICONV) to LDADD 16592 * src/Makefile.am: Add $(LIBICONV) to LDADD 16593 * tests/Makefile.am: Add $(LIBICONV) to LDADD 16594 165952016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16596 16597 Cover network functions with gnulib modules 16598 16599 * autogen.sh: Add accept, bind, close, connect, getaddrinfo, getsockname, 16600 listen, netdb, netinet_in, open, read, send, sendto, servent, setsockopt, 16601 socket, sys_socket, write to modules 16602 * configure.ac: Remove searching for ws2_32, libsocket, libnsl, netdb.h, 16603 netinet/in.h, sys/socket.h, socket. 16604 Add searching for netinet/tcp.h. 16605 * examples/Makefile.am: Add additional libs. 16606 * libwget/Makefile.am: Add additional libs. 16607 * src/Makefile.am: Add additional libs. 16608 * tests/Makefile.am: Add additional libs. 16609 * libwget/http.c: Remove conditional code. 16610 * libwget/net.c: Remove conditional code. Use c_isdgit(). 16611 * libwget/net.h: Remove conditional code. 16612 166132016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16614 16615 Use gnulib module strdup-posix instead of strdup 16616 16617 * autogen.sh: Replace strdup by strdup-posix 16618 166192016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16620 16621 Check if gcc __sync_fetch_and_add works with long long 16622 16623 * configure.ac: Check if gcc __sync_fetch_and_add works with long long 16624 * src/wget.c (_fetch_and_add_longlong): Check WITH_SYNC_FETCH_AND_ADD_LONGLONG 16625 166262016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16627 16628 Add compatibility to older flex versions 16629 16630 * libwget/Makefile.am: Remove space between -o and filename 16631 166322016-01-12 Tim Rühsen <tim.ruehsen@gmx.de> 16633 16634 Add header file gnulib modules 16635 16636 * autogen.sh: Add gettext-h, inttypes, stdarg, stddef, stdint 16637 * configure.ac: Remove check for inttypes.h, libintl.h, stddef.h 16638 Remove AC_TYPE_* checks 16639 * include/libwget.h.in: Include gnulib gettext.h 16640 * examples/Makefile.am: Fix AM_CPPFLAGS, add $(LIBINTL) to LDADD 16641 * libwget/Makefile.am: Fix LDADD and test_linking_LDADD 16642 * src/Makefile.am: Add $(LIBINTL) to wget2_LDADD 16643 * tests/Makefile.am: Fix AM_CPPFLAGS and libtest_la_CPPFLAGS, 16644 add $(LIBINTL) to LDADD, test_LDADD and test_parse_html_LDADD 16645 166462016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16647 16648 Remove detection of select() from configure.ac 16649 16650 * configure.ac: Remove detection of select() 16651 * src/wget.c: Remove commented include directives 16652 166532016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16654 16655 Add $(LIB_POLL) to *_LDADD 16656 16657 * examples/Makefile.am, src/Makefile.am, tests/Makefile.am: 16658 Add $(LIB_POLL) to *_LDADD 16659 166602016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16661 16662 Add gnulib module poll 16663 16664 * autogen.sh: Add gnulib module poll 16665 * configure.ac: Remove searching for poll.h and sys/poll.h 16666 * libwget/io.c: Remove conditional code 16667 166682016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16669 16670 Use stdlib.h and unistd.h from gnulib 16671 16672 * configure.ac: Remove searching for stdlib.h and unistd.h 16673 166742016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16675 16676 Add gnulib module pwrite 16677 16678 * autogen.sh: Add gnulib module pwrite 16679 * configure.ac: Remove searching for pwrite 16680 * src/wget.c(download_part): Remove conditional code 16681 166822016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16683 16684 Add gnulib module qsort_r 16685 16686 * autogen.sh: Add gnulib module qsort_r 16687 * configure.ac: Remove qsort_r code 16688 * libwget/vector.c: Remove conditional code 16689 166902016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16691 16692 Call wget_str(n)casecmp_ascii instead of str(n)casecmp 16693 16694 * examples/print_css_urls2.c, libwget/cookie.c, libwget/css.c, 16695 libwget/html_url.c, libwget/http.c, libwget/iri.c, libwget/metalink.c, 16696 libwget/net.c, libwget/robots.c, libwget/ssl_gnutls.c, 16697 libwget/stringmap.c, libwget/xml.c, src/job.c, src/options.c, src/wget.c: 16698 Call wget_str(n)casecmp_ascii instead of str(n)casecmp 16699 167002016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16701 16702 Add gnulib modules c-strcase, c-ctype and strcase 16703 16704 * autogen.sh: Add gnulib modules c-strcase, c-ctype and strcase 16705 * configure.ac: Remove search for string.h, strings.h, str(n)casecmp 16706 * libwget/utils.c: Use c_str(n)casecmp, c_isupper, c_isxdigit 16707 167082016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16709 16710 Update po/POTFILES.in 16711 16712 * po/POTFILES.in: Add src/bar.c and src/options.c, 16713 Remove libwget/printf.c 16714 167152016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16716 16717 Remove libwget/printf.c 16718 16719 * configure.ac: Remove check for dprintf 16720 * include/libwget.h.in: Remove dprintf/vdprintf prototypes 16721 * libwget/Makefile.am: Remove printf.c from file list 16722 * libwget/printf.c: Remove file 16723 167242016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16725 16726 Use dup2 gnulib fallback 16727 16728 * autogen.sh: Add dup2 to gnulib modules 16729 * configure.ac: Remove searching for dup2() 16730 * libwget/pipe.c: Remove commented includes 16731 167322016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16733 16734 Remove usage of alloca() 16735 16736 * libwget/hashfile.c (wget_hash_file_fd): Do not use alloca() 16737 * src/options.c (_read_config): Do not use alloca() 16738 167392016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16740 16741 Add $(LIB_CLOCK_GETTIME) to tests 16742 16743 * tests/Makefile.am: Add $(LIB_CLOCK_GETTIME) 16744 167452016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16746 16747 Use strndup gnulib fallback 16748 16749 * autogen.sh: Add strndup to gnulib modules 16750 * configure.ac: Remove searching for strndup() 16751 * include/libwget.h.in: Remove conditional code 16752 167532016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16754 16755 Remove searching for usleep 16756 16757 * configure.ac: Remove searching for usleep() 16758 167592016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16760 16761 Use glob gnulib fallback 16762 16763 * autogen.sh: Add glob to gnulib modules 16764 * configure.ac: Remove searching for glob() and glob.h 16765 * src/options.c: Remove conditional code 16766 167672016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16768 16769 Use nanosleep gnulib fallback 16770 16771 * autogen.sh: Add nanosleep to gnulib modules 16772 * configure.ac: Remove searching for nanosleep 16773 * examples/Makefile.am: Add $(LIBSOCKET) $(LIB_NANOSLEEP) $(LIB_SELECT) 16774 * src/Makefile.am: Add $(LIBSOCKET) $(LIB_NANOSLEEP) $(LIB_SELECT) 16775 * tests/Makefile.am: Add $(LIBSOCKET) $(LIB_NANOSLEEP) $(LIB_SELECT) 16776 * libwget/utils.c: Remove conditional code 16777 167782016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16779 16780 Use futimens gnulib fallback 16781 16782 * autogen.sh: Add futimens to gnulib modules 16783 * configure.ac: Remove searching for futimens 16784 * src/wget.c: Remove conditional code 16785 167862016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16787 16788 Replace gettimeofday by clock_gettime 16789 16790 * configure.ac: Remove check for gettimeofday 16791 * src/log.c: Remove #include <sys/time> 16792 * src/log.c (_write_debug): Use clock_gettime instead of gettimeofday 16793 167942016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16795 16796 Remove searching for fnmatch and fnmatch.h 16797 16798 * configure.ac: Remove searching for fnmatch and fnmatch.h 16799 168002016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16801 16802 Remove searching for fcntl.h 16803 16804 * configure.ac: Remove searching for fcntl.h 16805 168062016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16807 16808 Fix AC_CHECK_FUNCS for gnutls functions 16809 16810 * configure.ac: Remove 'gnutls' from AC_CHECK_FUNCS() 16811 168122016-01-11 Tim Rühsen <tim.ruehsen@gmx.de> 16813 16814 Add gnulib module clock-time 16815 16816 * autogen.sh: Add clock-time to gnulib modules 16817 * configure.ac: Remove searching for clock_gettime 16818 * src/Makefile.am: Add $(LIB_CLOCK_GETTIME) to wget2_LDADD 16819 * src/wget.c (set_file_mtime): Remove conditional code 16820 168212016-01-10 Tim Rühsen <tim.ruehsen@gmx.de> 16822 16823 Replace fnmatch function by gnulib fnmatch 16824 16825 * src/compat_fnmatch.c: Removed 16826 * src/compat_fnmatch.h: Removed 16827 * autogen.sh: Add fnmatch to gnulib modules 16828 * src/Makefile.am: Remove compat_fnmatch.[ch] 16829 * src/wget.c: Unconditionally include fnmatch.h 16830 168312016-01-10 Tim Rühsen <tim.ruehsen@gmx.de> 16832 16833 Remove test-ftp and test-ftps from test suite 16834 16835 * tests/Makefile.am: Remove test-ftp and test-ftps 16836 16837 These test were introduced as helpers for FTPS integration for 16838 Wget1.x (GSOC 2015 project). We disable them until Wget2 supports 16839 FTP/FTPS. 16840 168412016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16842 16843 Fix branch name in Travis-CI icon URL 16844 168452016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16846 16847 Add Travis-CI status icon 16848 168492016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16850 16851 Really fix return code of wget_str(n)casecmp_ascii 16852 16853 * libwget/utils.c (wget_strcasecmp_ascii, wget_strncasecmp_ascii): 16854 Fix return value 16855 168562016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16857 16858 Fix return code of wget_strcasecmp_ascii and wget_strncasecmp_ascii 16859 16860 * libwget/utils.c (wget_strcasecmp_ascii, wget_strncasecmp_ascii): 16861 Fix return value 16862 168632016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16864 16865 Link with libgnu.a 16866 16867 * libwget/Makefile.am: Add lib/ to AM_CPPFLAGS, add libgnu.a to *_LDADD 16868 * src/Makefile.am: Same 16869 * tests/Makefile.am: Same, rename object files to link with tests 16870 168712016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16872 16873 Fix OpenSolaris libidn include directory 16874 16875 * configure.ac: Search for idn/idna.h 16876 * libwget/encoding.c: Include idn/idna.h if found 16877 168782016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16879 16880 Add flock to gnulib modules 16881 16882 * autogen.sh: Add flock to gnulib modules 16883 168842016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16885 16886 Fix Name + Libs in pkg-config file 16887 16888 * libwget.pc.in: Set Name to libwget and change -llibwget to -lwget 16889 168902016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16891 16892 Remove API version from library file names 16893 16894 * .gitignore, Makefile.am, configure.ac, examples/Makefile.am, 16895 libwget.pc.in, libwget/Makefile.am, src/Makefile.am, tests/Makefile.am: 16896 Remove API version from library file names 16897 168982016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16899 16900 Build shared library by default 16901 16902 * configure.ac: Add disable-static to LT_INIT 16903 169042016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16905 16906 Hook gnulib into project 16907 169082016-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 16909 16910 Add gnulib as submodule 16911 169122016-01-08 Tim Rühsen <tim.ruehsen@gmx.de> 16913 16914 Add error code WGET_E_TLS_DISABLED 16915 16916 * libwget/ssl_gnutls.c: If TLS is not compiled in, return WGET_E_TLS_DISABLED 16917 from wget_ssl_open() and wget_ssl_server_open(). 16918 169192016-01-05 Darshit Shah <darnir@gmail.com> 16920 16921 Fix .travis.yml file for Linux systems 16922 16923 * .travis.yml: Container based architecture doesn't allow Sudo. 16924 Migrate file to support container based architecture 16925 * .travis.sh: Same 16926 169272016-01-08 Darshit Shah <darnir@gmail.com> 16928 16929 Do not print responses in tests 16930 16931 * tests/test.c (check_cookie): Do not print responses. We want 16932 this test to expose memory leaks, but printing the response of 16933 wget_cookie_create_request_header isn't needed. 16934 169352016-01-08 Darshit Shah <darnir@gmail.com> 16936 16937 Use absolute pathanme to find PSL data 16938 16939 * tests/Makefile.am: PSL DATADIR should be the absolute pathname to 16940 allow us to run tests from any directory. 16941 169422016-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 16943 16944 Add support for gcrypt hashing 16945 16946 * configure.ac: Add libgcrypt as fallback for libnettle 16947 * libwget/hashfile.c: Add gcrypt hash functions 16948 169492016-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 16950 16951 Fix hash functions for use with libnettle 16952 16953 * src/hashfile: Add needed includes (nettle 3.1.1 and 2.7) 16954 * src/hashfile (wget_hash_deinit): Call digest() instead of update() 16955 169562016-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 16957 16958 Skip HTTPS tests when no HTTPS available 16959 16960 * tests/test-i-https.c: Skip test when no HTTPS available 16961 169622016-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 16963 16964 Do not descend into subdomains if not requested 16965 16966 * src/wget.c (in_host_pattern_list): Fix arguments of wget_match_tail() 16967 169682016-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 16969 16970 Auto-generate version defines for library header file 16971 16972 * include/libwget.h: Removed 16973 * include/libwget.h.in: Added 16974 169752016-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 16976 16977 Rename LIBWGET_API_VERSION to LIBWGET_VERSION 16978 169792016-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 16980 16981 Fix hashing with GnuTLS 16982 16983 * libwget/ssl_gnutls.c: Do not skip automatic library initialization 16984 * libwget/md5.c (wget_md5_printf_hex): Print message on error 16985 * tests/test.c: New tests for hashing functionality 16986 16987 GnuTLS automatically initializes on application startup. 16988 This wastes CPU cycles when not using HTTPS. 16989 Switched off, simple functionality like MD5 fail (needed for metalink and 16990 authorization). We have to implement hashing via -lcrypt or -lgcrypt 16991 later - and than we could use GNUTLS_SKIP_GLOBAL_INIT again. 16992 169932016-01-06 Tim Rühsen <tim.ruehsen@gmx.de> 16994 16995 Do not percent unescape query 16996 16997 * tests/test.c: Test query parsing in API 16998 * tests/test-base.c: Test query handling of wget2 16999 * include/libwget.h: Add flags to wget_iri_t, 17000 add function wget_iri_unescape_inline() 17001 * libwget/iri.c: Add function wget_iri_unescape_inline(), 17002 fix wget_iri_parse() to not percent unescape query 17003 do not add fragment in wget_iri_get_escaped_resource() 17004 170052015-12-15 Tim Rühsen <tim.ruehsen@gmx.de> 17006 17007 Fix iconv conversion 17008 17009 * libwget/encoding.c: Kick out the last character from iconv(). 17010 17011 Thanks to Eli Zaretskii <eliz@gnu.org> for suggesting the fix. 17012 Reported-by: "Andries E. Brouwer" <Andries.Brouwer@cwi.nl> 17013 170142015-12-13 Ander Juaristi <ajuaristi@gmx.es> 17015 17016 Fix error return value in I/O code 17017 17018 * libwget/io.c: fix return value of __read() in case of error 17019 170202015-11-30 Ander Juaristi <ajuaristi@gmx.es> 17021 17022 Complete docs in I/O API 17023 17024 * libwget/http.c (wget_http_get_response_cb): removed unused variable 17025 (compiler warning). 17026 * libwget/io.c 17027 (wget_fdgetline,wget_getline,wget_read_file,wget_update_file): added 17028 source docs. Refactor: avoid code repetition. 17029 (wget_ready_2_transfer): fixed HTML rendering of docs. 17030 * docs/libwget/libwget-sections.txt: added new functions to GTK-Doc. 17031 170322015-11-23 Tim Rühsen <tim.ruehsen@gmx.de> 17033 17034 Fix return value in HSTS and Cookie code 17035 17036 * libwget/cookie.c (wget_cookie_db_load, wget_cookie_db_save): 17037 Fix return value 17038 * libwget/hsts.c (wget_hsts_db_load, wget_hsts_db_save): 17039 Fix return value 17040 170412015-11-23 Tim Rühsen <tim.ruehsen@gmx.de> 17042 17043 Disable 'unused' warnings in CSS tokenizer 17044 17045 * libwget/css_tokenizer.lex: Disable GCC warnings unused-parameter 17046 and unused-function 17047 170482015-11-23 Tim Rühsen <tim.ruehsen@gmx.de> 17049 17050 Cleanup cookie code 17051 17052 Cookies: Add new function wget_cookie_set_keep_session_cookies() 17053 and adjust wget_cookie_db_load() and wget_cookie_db_save(). 17054 Use wget_update_file() for loading and saving. 17055 170562015-11-23 Tim Rühsen <tim.ruehsen@gmx.de> 17057 17058 Add check for function param in HSTS code 17059 17060 * libwget/hsts.c (wget_hsts_db_save): Check hsts_db for NULL, 17061 (wget_hsts_db_load): Fix typo in error message 17062 170632015-11-23 Tim Rühsen <tim.ruehsen@gmx.de> 17064 17065 Fix typo in OSCP error message 17066 17067 * libwget/ocsp.c (wget_ocsp_db_load): Fix typo 17068 170692015-11-23 Tim Rühsen <tim.ruehsen@gmx.de> 17070 17071 Adjust OCSP and HSTS default file names 17072 17073 * src/options.c (print_help, init): Use ~/.wget-hsts and ~/.wget-ocsp 17074 170752015-11-23 Tim Rühsen <tim.ruehsen@gmx.de> 17076 17077 Fix test suite for CygWin 17078 17079 * New CygWin specific code for removing/cleaning test directory 17080 170812015-11-23 Tim Rühsen <tim.ruehsen@gmx.de> 17082 17083 Fix tests/test.c for non-GNU standard C libraries 17084 17085 * tests/test.c (test_buffer_printf): Skip left zero padding strings 17086 170872015-11-18 Tim Rühsen <tim.ruehsen@gmx.de> 17088 17089 Avoid implicit gloabl initialization by GnuTLS 17090 17091 * libwget/ssl_gnutls.c: Use GNUTLS_SKIP_GLOBAL_INIT if defined 17092 170932015-11-18 Tim Rühsen <tim.ruehsen@gmx.de> 17094 17095 Fix basename in wget_update_file() 17096 17097 * libwget/io.c (wget_update_file): Fxi basename 17098 170992015-11-11 Tim Rühsen <tim.ruehsen@gmx.de> 17100 17101 Add new options --netrc and --netrc-file 17102 17103 * libwget/netrc.c: New file for netrc API functions 17104 * include/libwget.h: Add netrc function prototypes 17105 * libwget/Makefile.am: Add netrc.c 17106 * libwget/test_linking.c (main): Add call to wget_netrc_deinit() 17107 * src/options.c: Add --netrc and --netrc-file 17108 * src/options.h: Add config.netrc and config.netrc_file 17109 * src/wget.c (http_get): Get login/password from netrc data 17110 * tests/test-auth-basic.c (main): Add test for --netrc-file 17111 171122015-11-10 Tim Rühsen <tim.ruehsen@gmx.de> 17113 17114 Add define ascii_isspace and ascii_isalpha 17115 17116 * libwget/xml.c: Add and use defines here 17117 171182015-11-10 Tim Rühsen <tim.ruehsen@gmx.de> 17119 17120 Replace strlpcy by memcpy 17121 17122 * src/options.c (set_long_option): Replace strlcpy() 17123 171242015-11-10 Tim Rühsen <tim.ruehsen@gmx.de> 17125 17126 Replace calls to strncpy by memcpy 17127 17128 * libwget/xml.c (parseXML): Replace calls to strncpy() 17129 171302015-11-10 Tim Rühsen <tim.ruehsen@gmx.de> 17131 17132 Replaced strndup() by wget_strmemdup() 17133 17134 * examples/getstream.c, examples/print_css_urls2.c, libwget/cookie.c, 17135 libwget/css_url.c, libwget/encoding.c, libwget/hsts.c, 17136 libwget/html_url.c, libwget/http.c, libwget/metalink.c, 17137 libwget/ocsp.c, libwget/robots.c, libwget/ssl_gnutls.c, 17138 src/options.c, src/wget.c, tests/stringmap_perf.c, 17139 tests/test-wget-1.c 17140 17141 Strndup() calls an additional strlen() on the input string. 17142 This is normally not needed, and thus just consumes CPU cycles. 17143 All calls to strndup() could be replaced by wget_strmemdup, which 17144 basically allocates len+1 memory, calls memcpy and terminates with 17145 a 0 byte. 17146 171472015-11-09 Tim Rühsen <tim.ruehsen@gmx.de> 17148 17149 Add type of file for OCSP databases 17150 17151 * libwget/ocsp.c (_ocsp_db_save_hosts): Add type of file to database 17152 files. 17153 171542015-11-09 Tim Rühsen <tim.ruehsen@gmx.de> 17155 17156 Amend message when successfully updating files 17157 17158 * libwget/io.c (wget_update_file): Amend message 17159 171602015-11-09 Ander Juaristi <ajuaristi@gmx.es> 17161 17162 Avoid potential race condition in HSTS code 17163 17164 * libwget/hsts.c (_hsts_db_load): new internal function. 17165 (wget_hsts_db_load): moved code to _hsts_db_load() internal function. 17166 (wget_hsts_db_save): lock the file before saving, to avoid race 17167 conditions. Reload only if access time greater than last read time. 17168 171692015-11-06 Tim Rühsen <tim.ruehsen@gmx.de> 17170 17171 Use wget_update_file() in OCSP code 17172 17173 * libwget/ocsp.c: Rearranged code to use wget_update_file 17174 171752015-11-06 Tim Rühsen <tim.ruehsen@gmx.de> 17176 17177 Add new function wget_update_file() 17178 17179 * include/libwget.h: Add wget_update_file() 17180 * libwget/io.c: Add wget_update_file() 17181 17182 Add a function to care about locking, temp files, opening 17183 overhead for our typical updating file database procedure. 17184 Useful for HSTS, OCSP and Cookie file databases. 17185 171862015-11-08 Tim Rühsen <tim.ruehsen@gmx.de> 17187 17188 Close va_start() with va_end() 17189 17190 * tests/libtest.c (wget_test): Add va_end() 17191 171922015-11-06 Tim Rühsen <tim.ruehsen@gmx.de> 17193 17194 Adjust HSTS unit tests to latest changes 17195 17196 * tests/test.c (test_hsts): Adjust port 80 test, new port 8080 test 17197 171982015-11-04 Tim Rühsen <tim.ruehsen@gmx.de> 17199 17200 Use logic or instead of binary or 17201 17202 * libwget/atom_url.c (_atom_get_url): Use logic or 17203 172042015-10-31 Ander Juaristi <ajuaristi@gmx.es> 17205 17206 Fixed HSTS policy application 17207 17208 * include/libwget.h (wget_iri_set_scheme): new function. 17209 * libwget/hsts.c (wget_hsts_host_match): HTTP default port -> HTTPS 17210 default port 17211 * libwget/iri.c (wget_iri_set_scheme): new function. 17212 * src/wget.c (http_get): when changing the scheme, change the port 17213 number as well (call wget_iri_set_scheme). 17214 172152015-10-30 Tim Rühsen <tim.ruehsen@gmx.de> 17216 17217 Add function wget_global_get_func() 17218 17219 * include/libwget.h: Add wget_global_get_func() prototype 17220 * libwget/init.c: Add function wget_global_get_func() 17221 172222015-10-30 Tim Rühsen <tim.ruehsen@gmx.de> 17223 17224 Adjust package name and version 17225 17226 * configure.ac: Adjust package name and version 17227 172282015-10-28 Tim Rühsen <tim.ruehsen@gmx.de> 17229 17230 Fix HTTP2 in combination with --progress=bar 17231 17232 * src/wget.c (http_get): Call wget_http_get_response_cb() 17233 172342015-10-28 Tim Rühsen <tim.ruehsen@gmx.de> 17235 17236 Some cleanups 17237 17238 * src/options.c (read_config): Check for config.config_file 17239 (get_home_dir): Cleanup 17240 (init): Check for readability of config.config_file 17241 172422015-10-28 Tim Rühsen <tim.ruehsen@gmx.de> 17243 17244 Free config.config_file on exit 17245 17246 * src/options.c (deinit): Free config.config_file 17247 172482015-10-28 Tim Rühsen <tim.ruehsen@gmx.de> 17249 17250 Exit early on 0 URLs to process 17251 17252 * src/wget.c (main): Exit if no URLs found to process 17253 172542015-10-28 Tim Rühsen <tim.ruehsen@gmx.de> 17255 17256 Print error for unsupported CLI URLs 17257 17258 * src/wget.c (add_url_to_queue): Print error message 17259 172602015-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 17261 17262 Rename --dns-cache to --dns-caching 17263 17264 * src/options.c: Rename --dns-cache to --dns-caching, 17265 Remove double entry from help text 17266 172672015-10-27 Tim Rühsen <tim.ruehsen@gmx.de> 17268 17269 Rename --continue-download to --continue 17270 17271 * src/options.c: Rename --continue-download to --continue 17272 172732015-10-26 Tim Rühsen <tim.ruehsen@gmx.de> 17274 17275 Fix default_page to default-page in help text 17276 17277 * src/options.c: Fix default_page to default-page 17278 172792015-10-25 Tim Rühsen <tim.ruehsen@gmx.de> 17280 17281 Fix HTML BASE with absolute path 17282 17283 * src/wget.c (html_parse): Fix HTML BASE with absolute path 17284 * tests/Makefile.am: Add test-base.c 17285 * tests/test-base.c: New file 17286 172872015-10-25 Tim Rühsen <tim.ruehsen@gmx.de> 17288 17289 Add download statistics 17290 17291 * src/wget.c: New structure _statistics_t, 17292 New function _fetch_and_add_longlong(), 17293 (main): Print stats 17294 (http_get): Collect stats 17295 172962015-10-24 Tim Rühsen <tim.ruehsen@gmx.de> 17297 17298 Remove unused argument argv from option parser functions 17299 17300 * src/options.c (struct option): Remove argv from parser() 17301 (print_help, parse_integer, parse_numbytes, parse_string, 17302 parse_stringset, parse_stringlist, parse_taglist, parse_bool, 17303 parse_mirror, parse_timeout, parse_cert_type, parse_restrict_names, 17304 parse_n_option, parse_prefer_family, parse_execute, parse_execute): 17305 Remove unused argument argv. 17306 * src/options.c (set_long_option): Adjust calls to parser functions. 17307 173082015-10-24 Tim Rühsen <tim.ruehsen@gmx.de> 17309 17310 Comment code for broken servers and document it 17311 17312 * libwget/src (wget_http_parse_response_header): Comment code 17313 for broken servers and document it. 17314 173152015-10-24 Darshit Shah <darnir@gmail.com> 17316 17317 Ensure atleast one downloader thread 17318 17319 * src/options.c (init): The check for the minimum number of downloader 17320 threads should be done on config.max_threads not config.num_threads. 17321 173222015-10-23 Tim Rühsen <tim.ruehsen@gmx.de> 17323 17324 Abort on SIGTERM or on 2x CTRL-C 17325 17326 i* src/wget.c (nop): Abort on SIGTERM or on 2x CTRL-C 17327 173282015-10-23 Tim Rühsen <tim.ruehsen@gmx.de> 17329 17330 Add --config option for backwards compatibility 17331 17332 * src/options.c: Add --config 17333 173342015-10-23 Giuseppe Scrivano <gscrivan@redhat.com> 17335 17336 Exit immediately if cannot set http_proxy or https_proxy 17337 17338 * src/wget.c (main): Exit immediately on an "init()" error. 17339 * src/options.c (init): Raise an error if cannot set http_proxy or https_proxy. 17340 * tests/test-parse-html.c (main): Check for "init" errors. 17341 * tests/test.c (main): Check for "init" errors. 17342 173432015-10-23 Giuseppe Scrivano <gscrivan@redhat.com> 17344 17345 set_http_proxy and set_https_proxy can return an error 17346 17347 * libwget/http.h (wget_http_set_http_proxy): change the return type to int. 17348 (wget_http_set_https_proxy): change the return type to int 17349 * libwget/http.c (wget_http_set_http_proxy): return an error if cannot set the proxies. 17350 (wget_http_set_https_proxy): return an error if cannot set the proxies. 17351 173522015-10-22 Giuseppe Scrivano <gscrivan@redhat.com> 17353 17354 Indent C preprocessor directives 17355 17356 * libwget/css_tokenizer.h: Indent C preprocessor directives. 17357 * libwget/net.h: Likewise. 17358 * libwget/private.h: Likewise. 17359 * src/bar.h: Likewise. 17360 * src/blacklist.h: Likewise. 17361 * src/compat_fnmatch.h: Likewise. 17362 * src/host.h: Likewise. 17363 * src/job.h: Likewise. 17364 * src/log.h: Likewise. 17365 * src/options.h: Likewise. 17366 * src/wget.h: Likewise. 17367 173682015-10-22 Giuseppe Scrivano <gscrivan@redhat.com> 17369 17370 .dir-locals.el: new file 17371 173722015-10-22 Giuseppe Scrivano <gscrivan@redhat.com> 17373 17374 Fix function prototypes 17375 17376 * src/bar.h: Fix function prototype. 17377 * src/blacklist.h: Fix function prototype. 17378 * src/compat_fnmatch.h: Fix function prototype. 17379 * src/host.h: Fix function prototype. 17380 * src/job.h: Fix function prototype. 17381 * src/log.h: Fix function prototype. 17382 * src/options.h: Fix function prototype. 17383 173842015-10-22 Giuseppe Scrivano <gscrivan@redhat.com> 17385 17386 Check for wget_iri_parse errors 17387 17388 * libwget/ssl_gnutls.c (send_ocsp_request): Check for wget_iri_parse errors. 17389 * libwget/http_highlevel.c (wget_http_get): Check for wget_iri_parse errors. 17390 * libwget/http.c (_parse_proxies): Check for wget_iri_parse errors. 17391 173922015-10-21 Ander Juaristi <ajuaristi@gmx.es> 17393 17394 Documented hashing functions 17395 173962015-10-20 Ander Juaristi <ajuaristi@gmx.es> 17397 17398 Typos 17399 174002015-10-20 Ander Juaristi <ajuaristi@gmx.es> 17401 17402 Initial docs for hashing utils 17403 174042015-10-23 Darshit Shah <darnir@gmail.com> 17405 17406 Fix compiler warning about unused return code 17407 17408 * src/options.c (init): Clang complains about an unused return code from 17409 asprintf. So, we check to ensure that asprintf() actually managed to 17410 allocate the said string. 17411 174122015-10-23 Darshit Shah <darnir@gmail.com> 17413 17414 Allow recursive downloads to be more concurrent 17415 17416 * src/wget.c (main): In cases where the queue_size is not known at 17417 startup, spawn the maximum allowed threads for the thread pool. 17418 174192015-10-22 Darshit Shah <darnir@gmail.com> 17420 17421 Add option to explicitly set path to wgetrc file 17422 17423 * src/options.c (print_help, config, init, options): Add new option 17424 --config-file. 17425 * src/options.h: Same as above 17426 * src.options.c (read_config): Use the config.config_file value instead 17427 of a hard coded file path for wgetrc 17428 174292015-10-22 Darshit Shah <darnir@gmail.com> 17430 17431 OCSP and HSTS files should be stored to ~/.wget_* 17432 17433 * src/options.c: Include the pwd.h file to get access to pwd routines 17434 * src/options.c (get_home_dir): New function that return strdup'ed 17435 string containing location of directory for storing config files 17436 * src/options.c (init): Initialize the config.hsts_file and 17437 config.ocsp_file values correctly using get_home_dir() 17438 * src/options.h (config): Move hsts_file and ocsp_file from being const 17439 char* storages to only char* storages. 17440 174412015-10-20 Tim Rühsen <tim.ruehsen@gmx.de> 17442 17443 Add new test tests/test-redirection.c 17444 174452015-10-20 Tim Rühsen <tim.ruehsen@gmx.de> 17446 17447 Add extension of test suite port variables in response headers 17448 17449 * tests/libtest.h: Add member header_alloc to wget_test_url_t 17450 * tests/libtest.c: Extend port vars in response headers 17451 174522015-10-19 Tim Rühsen <tim.ruehsen@gmx.de> 17453 17454 On CTRL-C save downloaded data before exit 17455 17456 * include/libwget.h: New declaration wget_http_abort_connection() 17457 * libwget/http.c: New function wget_http_abort_connection() 17458 * src/wget.c: Call wget_http_abort_connection() on CTRL-C 17459 174602015-10-19 Giuseppe Scrivano <gscrivan@redhat.com> 17461 17462 gnutls: check for gnutls_srp_server_get_username function 17463 174642015-10-18 Darshit Shah <darnir@gmail.com> 17465 17466 Rename option --num-threads to --max-threads 17467 17468 * src/options.[ch]: Rename the option --num-threads to --max-threads 17469 which better describes the content it holds. We keep the --num-threads 17470 config option around, since it will be used to hold the *actual* number 17471 of threads in this instance of Wget2. 17472 * src/wget.c (main): Initialize the value of config.num_threads. This 17473 value is atmost config.max_threads or the number of URLs to download, 17474 whichever is lower. 17475 174762015-10-18 Tim Rühsen <tim.ruehsen@gmx.de> 17477 17478 Change 'git branch' into 'git checkout' 17479 17480 * README.md: Change 'git branch' into 'git checkout' 17481 174822015-10-17 Tim Rühsen <tim.ruehsen@gmx.de> 17483 17484 Fix TLS read timeout issue with GnuTLS 3.4.x 17485 17486 * libwget/ssl_gnutls.c (wget_ssl_read_timeout): Comment out 17487 code path for GnuTLS >= 0x030107 17488 174892015-10-16 Darshit Shah <darnir@gmail.com> 17490 17491 Fix issue when --debug is the first option passed 17492 17493 * src/options.c (init): set_long_option() expects only a the option 17494 name without the preceeding --. Ensure that we remove those when 17495 passing the --debug flag. 17496 174972015-10-15 Tim Rühsen <tim.ruehsen@gmx.de> 17498 17499 Fix shell in contrib/gen_option_diff.sh 17500 17501 * contrib/gen_option_diff.sh: Explicitly use /bin/bash 17502 175032015-10-15 Tim Rühsen <tim.ruehsen@gmx.de> 17504 17505 Add contrib/gen_option_diff.sh 17506 17507 * contrib/gen_option_diff.sh: New script file 17508 175092015-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 17510 17511 Remove mentioning SPDY from README.md 17512 17513 * README.md: Remove mentioning SPDY 17514 175152015-10-11 Tim Rühsen <tim.ruehsen@gmx.de> 17516 17517 Rename wget executable to wget2 17518 17519 * .gitignore: Replace src/wget by src/wget2 17520 * src/Makefile.am: Rename wget to wget2 17521 * tests/Makefile.am: Link wget2-* objects 17522 * tests/libtest.c: Execute wget2 instead of wget 17523 175242015-09-28 Tim Rühsen <tim.ruehsen@gmx.de> 17525 17526 Handle TLS rehandshakes 17527 17528 * libwget/ssl_gnutls.c: New static function _do_handshake() 17529 * libwget/ssl_gnutls.c (wget_ssl_open): Move handshake code to _do_handshake() 17530 * libwget/ssl_gnutls.c (wget_ssl_server_open): Use _do_handshake() 17531 * libwget/ssl_gnutls.c (wget_ssl_read_timeout): Handle rehandshake 17532 * libwget/ssl_gnutls.c (wget_ssl_write_timeout): Handle rehandshake 17533 175342015-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 17535 17536 Add copyright notes to NEWS 17537 17538 * NEWS: Add copyright notes 17539 175402015-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 17541 17542 Print number of downloaded files and bytes when done 17543 17544 * src/wget.c: Print number of downloaded files and bytes when done 17545 175462015-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 17547 17548 Fix SPDY incompatibility 17549 17550 * src/options.c: Remove spdy/3.1 from ALPN offer 17551 175522015-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 17553 17554 Fix README 17555 17556 * README: Fix to wget2 17557 175582015-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 17559 17560 Fix README.md 17561 17562 * README.md: Fix to wget2 17563 175642015-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 17565 17566 Transfer copyright to Free Software Foundation, Inc. 17567 175682015-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 17569 17570 Fix mailing list and website in configure.ac 17571 17572 * configure.ac: Fix mailing list and website in AC_INIT 17573 175742015-09-21 Tim Rühsen <tim.ruehsen@gmx.de> 17575 17576 Add ABOUT-NLS and INSTALL to .gitignore 17577 17578 * .gitignore: Add ABOUT-NLS and INSTALL 17579 175802015-09-21 Tim Rühsen <tim.ruehsen@gmx.de> 17581 17582 Add fallback code for GnuTLS < 3.1.7 17583 17584 * libwget/ssl_gnutls.c (wget_ssl_read_timeout): Add fallback code for 17585 older versions of GnuTLS. 17586 175872015-09-19 Tim Rühsen <tim.ruehsen@gmx.de> 17588 17589 Transform Mget into Wget 17590 175912015-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 17592 17593 More HTTP/2 related code, new option --(no-)http2 17594 175952015-09-13 Tim Rühsen <tim.ruehsen@gmx.de> 17596 17597 Fixed cookie test routine 17598 175992015-09-12 Tim Rühsen <tim.ruehsen@gmx.de> 17600 17601 Added missing breaks to switch in libtest.c 17602 176032015-09-12 Tim Rühsen <tim.ruehsen@gmx.de> 17604 17605 Fix resource leak in new HTTP/2 code 17606 176072015-09-11 Tim Rühsen <tim.ruehsen@gmx.de> 17608 17609 Fix compilation without nghttp2 17610 176112015-09-11 Tim Rühsen <tim.ruehsen@gmx.de> 17612 17613 HTTP/2 download is working 17614 176152015-09-11 Tim Rühsen <tim.ruehsen@gmx.de> 17616 17617 Fix res->encoding in _html_get_url() 17618 176192015-09-10 Tim Rühsen <tim.ruehsen@gmx.de> 17620 17621 Fix typo in comment 17622 176232015-09-07 Tim Rühsen <tim.ruehsen@gmx.de> 17624 17625 Rearrange HTTP header API 17626 176272015-09-07 Tim Rühsen <tim.ruehsen@gmx.de> 17628 17629 Request HTTP/1.1 and HTTP/2.0 via ALPN 17630 176312015-08-31 Tim Rühsen <tim.ruehsen@gmx.de> 17632 17633 Fix Content-Disposition filename retrieval 17634 176352015-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 17636 17637 Fix filename when charset conversion fails #2 17638 176392015-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 17640 17641 Remove iconv() from AC_CHECK_FUNCS 17642 176432015-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 17644 17645 Include sys/types.h in libmget.h 17646 176472015-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 17648 17649 Fix filename when charset conversion fails 17650 176512015-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 17652 17653 Unify mget_ssl_server_open and mget_ssl_open 17654 176552015-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 17656 17657 Add AUTH command to FTP test suite 17658 176592015-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 17660 17661 Add tests/test-ftps.c 17662 176632015-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 17664 17665 Remove INSTALL from git 17666 176672015-07-12 Tim Rühsen <tim.ruehsen@gmx.de> 17668 17669 Add mget_tcp_tls_start() and mget_tcp_tls_stop() 17670 176712015-07-11 Tim Rühsen <tim.ruehsen@gmx.de> 17672 17673 Add FTPS server to test suite 17674 176752015-07-11 Tim Rühsen <tim.ruehsen@gmx.de> 17676 17677 Allow ordered and unordered I/O for FTP test suite 17678 176792015-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 17680 17681 Add --inet4-only to tests/test-ftp.c 17682 176832015-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 17684 17685 Added EPSV to tests/libtest.c 17686 176872015-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 17688 17689 Include sys/wait.h for WIF* macros 17690 176912015-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 17692 17693 Remove tests/.test_* with make clean 17694 176952015-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 17696 17697 Remove unused variables from tests/libtest.c 17698 176992015-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 17700 17701 Remove GNU make specialty from libmget/Makefile.am 17702 177032015-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 17704 17705 Remove qsort_r trampoline version 17706 177072015-07-10 Tim Rühsen <tim.ruehsen@gmx.de> 17708 17709 Fix tests/libtest.c crash recently introduced 17710 177112015-07-09 Tim Rühsen <tim.ruehsen@gmx.de> 17712 17713 Add first FTP testing capability 17714 177152015-07-09 Tim Rühsen <tim.ruehsen@gmx.de> 17716 17717 Don't cache automatic server ports 17718 177192015-07-09 Tim Rühsen <tim.ruehsen@gmx.de> 17720 17721 Add Travis CI OSX check 17722 177232015-07-09 Tim Rühsen <tim.ruehsen@gmx.de> 17724 17725 Fix typo in libmget/encoding.c 17726 177272015-07-06 Tim Rühsen <tim.ruehsen@gmx.de> 17728 17729 Work around a libidn 1.30 vulnerability 17730 177312015-07-06 Tim Rühsen <tim.ruehsen@gmx.de> 17732 17733 Revert "Bump GETTEXT to version 0.19.1" 17734 17735 This reverts commit 469e501859c33ab0cb842eb5928698ed29ed9f9e. 17736 177372015-07-06 Tim Rühsen <tim.ruehsen@gmx.de> 17738 17739 Allow NULL for mget_http_parse_setcookie()i cookie param 17740 177412015-07-06 Tim Rühsen <tim.ruehsen@gmx.de> 17742 17743 Bump GETTEXT to version 0.19.1 17744 177452015-07-06 Tim Rühsen <tim.ruehsen@gmx.de> 17746 17747 Avoid tautological check warnings from clang 17748 177492015-07-06 Tim Rühsen <tim.ruehsen@gmx.de> 17750 17751 Avoid printf format warning 17752 177532015-07-06 Tim Rühsen <tim.ruehsen@gmx.de> 17754 17755 Add check and support for BSD style qsort_r 17756 177572015-07-05 Tim Rühsen <tim.ruehsen@gmx.de> 17758 17759 Fix check for HAVE_DPRINTF in libmget/printf.c 17760 177612015-05-04 Tim Ruehsen <tim.ruehsen@gmx.de> 17762 17763 Fix waste of CPU cycles in mget_iri_relative_to_abs() 17764 177652015-05-02 Tim Ruehsen <tim.ruehsen@gmx.de> 17766 17767 prepared for release 17768 177692015-03-22 Tim Ruehsen <tim.ruehsen@gmx.de> 17770 17771 use random/srandom instead weak functions rand/srand 17772 177732015-03-18 Tim Rühsen <tim.ruehsen@gmx.de> 17774 17775 added .gitignore 17776 177772015-03-13 Tim Rühsen <tim.ruehsen@gmx.de> 17778 17779 fixing two more Coverity issues 17780 177812015-03-12 Tim Ruehsen <tim.ruehsen@gmx.de> 17782 17783 fixing more Coverity issues 17784 177852015-03-09 Tim Rühsen <tim.ruehsen@gmx.de> 17786 17787 fixing more Coverity issues 17788 177892015-03-08 Tim Ruehsen <tim.ruehsen@gmx.de> 17790 17791 changes in hashmap handling 17792 177932015-03-07 Tim Ruehsen <tim.ruehsen@gmx.de> 17794 17795 fix several issues found by Coverity 17796 177972015-03-03 Tim Ruehsen <tim.ruehsen@gmx.de> 17798 17799 comment unused mget_bsprintf functions 17800 178012015-03-03 Tim Ruehsen <tim.ruehsen@gmx.de> 17802 17803 fix possible buffer overflow 17804 178052015-03-01 Tim Ruehsen <tim.ruehsen@gmx.de> 17806 17807 added Coverity Scan Status 17808 178092015-02-28 Tim Ruehsen <tim.ruehsen@gmx.de> 17810 17811 fix strndup() compat function 17812 178132015-02-27 Tim Rühsen <tim.ruehsen@gmx.de> 17814 17815 replace logger vsprintf, Solaris *printf can't handle NULL arguments 17816 178172015-02-27 Tim Rühsen <tim.ruehsen@gmx.de> 17818 17819 add comment regarding Solaris sprintf behaviour 17820 178212015-02-27 Tim Rühsen <tim.ruehsen@gmx.de> 17822 17823 add link to Solaris (p)thread docs 17824 178252015-02-26 Tim Rühsen <tim.ruehsen@gmx.de> 17826 17827 check for Solaris encoding '646' 17828 178292015-02-26 Tim Rühsen <tim.ruehsen@gmx.de> 17830 17831 convert pid to int 17832 178332015-02-26 Tim Rühsen <tim.ruehsen@gmx.de> 17834 17835 added %c to mget_buffer_vprintf_append2 17836 178372015-02-26 Tim Rühsen <tim.ruehsen@gmx.de> 17838 17839 add -pthread compiler option 17840 178412015-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 17842 17843 use mget_buffer_vprintf2 in logger 17844 178452015-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 17846 17847 make %s work with NULL values in mget_buffer_printf2()-like functions 17848 178492015-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 17850 17851 added libmget/net.h to Makefile.am 17852 178532015-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 17854 17855 fixed memleak in mget_http_parse_challenge() 17856 178572015-02-24 Tim Rühsen <tim.ruehsen@gmx.de> 17858 17859 fixed Mget return value for HTTPS failures 17860 178612015-02-22 Tim Ruehsen <tim.ruehsen@gmx.de> 17862 17863 fixed tests/Makefile.am 17864 178652015-02-22 Tim Ruehsen <tim.ruehsen@gmx.de> 17866 17867 updated tests/certs (old certs expired on 11.2.2015) 17868 178692015-02-20 Tim Rühsen <tim.ruehsen@gmx.de> 17870 17871 added ALPN for TLS sessions 17872 178732015-02-06 Tim Ruehsen <tim.ruehsen@gmx.de> 17874 17875 added missing files to src/Makefile.am 17876 178772015-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 17878 17879 add @LTLIBICONV@ @LTLIBINTL@ to tests 17880 178812015-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 17882 17883 add LTLIBICONV and LTLIBINTL to LDADD for Solaris 17884 178852015-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 17886 17887 check for FNM_CASEFOLD 17888 178892015-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 17890 17891 include alloca.h 17892 178932015-02-06 Tim Rühsen <tim.ruehsen@gmx.de> 17894 17895 check for libsocket and libnsl on Solaris 17896 178972015-02-05 Tim Rühsen <tim.ruehsen@gmx.de> 17898 17899 fix compiler warnings 17900 179012015-02-05 Tim Rühsen <tim.ruehsen@gmx.de> 17902 17903 include libintl.h before redefinition of gettext functions 17904 179052015-02-04 Tim Rühsen <tim.ruehsen@gmx.de> 17906 17907 added Resources in comment of libmget/ssl_gnutls.c 17908 179092015-01-30 Tim Rühsen <tim.ruehsen@gmx.de> 17910 17911 include alloca.h if available 17912 179132015-01-30 Tim Rühsen <tim.ruehsen@gmx.de> 17914 17915 require gettext 0.18.1 17916 179172015-01-30 Tim Rühsen <tim.ruehsen@gmx.de> 17918 17919 fix configure and doc stuff 17920 179212015-01-30 Tim Rühsen <tim.ruehsen@gmx.de> 17922 17923 fix compilation with older GnuTLS (without OCSP) 17924 179252015-01-29 Tim Rühsen <tim.ruehsen@gmx.de> 17926 17927 added full cert chain OCSP check 17928 179292015-01-22 Tim Rühsen <tim.ruehsen@gmx.de> 17930 17931 fix building man pages 17932 179332015-01-22 Tim Rühsen <tim.ruehsen@gmx.de> 17934 17935 fix conditional HAVE_PO in configure.ac 17936 179372015-01-16 Tim Rühsen <tim.ruehsen@gmx.de> 17938 17939 OCSP check the whole certificate chain 17940 179412015-01-15 Tim Rühsen <tim.ruehsen@gmx.de> 17942 17943 added OCSP fix from GnuTLS project 17944 17945 https://gitorious.org/gnutls/gnutls/commit/11eebe14b232ec198d1446a3720e6ed78d118c4b 17946 179472015-01-14 Tim Ruehsen <tim.ruehsen@gmx.de> 17948 17949 fix mget_tcp_set_tcp_fastopen() 17950 179512015-01-14 Tim Ruehsen <tim.ruehsen@gmx.de> 17952 17953 removed two indentations in configure.ac 17954 179552015-01-13 Tim Rühsen <tim.ruehsen@gmx.de> 17956 17957 added --tcp-fastopen to enable/disable TCP Fast Open 17958 179592015-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 17960 17961 removed --load-hsts and --save-hsts 17962 179632015-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 17964 17965 add creation time to OCSP entries 17966 179672015-01-09 Tim Rühsen <tim.ruehsen@gmx.de> 17968 17969 fix loading/saving OCSP cache 17970 179712015-01-08 Tim Rühsen <tim.ruehsen@gmx.de> 17972 17973 added OCSP response caching 17974 179752015-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 17976 17977 OCSP code tuning 17978 179792015-01-07 Tim Rühsen <tim.ruehsen@gmx.de> 17980 17981 added linefeed in --help text 17982 179832015-01-05 Tim Rühsen <tim.ruehsen@gmx.de> 17984 17985 mention autopoint as build requisite 17986 179872015-01-05 Tim Rühsen <tim.ruehsen@gmx.de> 17988 17989 Skip po directory if GETTEXT not available 17990 179912014-12-31 Tim Rühsen <tim.ruehsen@gmx.de> 17992 17993 fix OCSP request condition 17994 179952014-12-30 Tim Rühsen <tim.ruehsen@gmx.de> 17996 17997 list pkg-config in Requirements 17998 179992014-12-21 Tim Ruehsen <tim.ruehsen@gmx.de> 18000 18001 added more tests for parsing challenges 18002 180032014-12-18 Tim Ruehsen <tim.ruehsen@gmx.de> 18004 18005 added test for WWW-Authenticate challenge parsing 18006 180072014-12-18 Tim Rühsen <tim.ruehsen@gmx.de> 18008 18009 new options --ocsp and --ocsp-stapling 18010 180112014-12-15 Tim Ruehsen <tim.ruehsen@gmx.de> 18012 18013 Check for gnutls/ocsp.h to compile on older systems 18014 180152014-12-14 Tim Ruehsen <tim.ruehsen@gmx.de> 18016 18017 First experimental commit with OCSP (stapling+non-stapling) support 18018 180192014-12-09 Tim Rühsen <tim.ruehsen@gmx.de> 18020 18021 tail match instead of full match for --domains/-D/--exclude-domains 18022 180232014-12-09 Tim Rühsen <tim.ruehsen@gmx.de> 18024 18025 Add wildcard support for -D/--domains and --exclude-domains 18026 180272014-11-23 Tim Ruehsen <tim.ruehsen@gmx.de> 18028 18029 add +/-psl to --version output 18030 180312014-11-14 Tim Rühsen <tim.ruehsen@gmx.de> 18032 18033 added pkg-config support for library searching 18034 180352014-11-12 Tim Rühsen <tim.ruehsen@gmx.de> 18036 18037 check GnuTLS version for OCSP functionality 18038 180392014-11-12 Tim Rühsen <tim.ruehsen@gmx.de> 18040 18041 fix compilation error, add OCSP stapling status report 18042 180432014-11-12 Tim Rühsen <tim.ruehsen@gmx.de> 18044 18045 Cert messages slightly amended 18046 180472014-11-10 Tim Rühsen <tim.ruehsen@gmx.de> 18048 18049 check for qsort_r() 18050 180512014-10-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18052 18053 added --crl-file for CRL certificates 18054 180552014-10-18 Tim Ruehsen <tim.ruehsen@gmx.de> 18056 18057 moved basic progress bar routines to libmget 18058 180592014-10-13 Tim Rühsen <tim.ruehsen@gmx.de> 18060 18061 fixed race condition when having a filename / directory clash 18062 180632014-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 18064 18065 added port to Referer: HTTP header 18066 180672014-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 18068 18069 changed valgrind testing 18070 180712014-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 18072 18073 better result code compatibility with Wget 18074 180752014-10-10 Tim Rühsen <tim.ruehsen@gmx.de> 18076 18077 more pedantic scheme check for IRI parsing 18078 180792014-10-07 Tim Ruehsen <tim.ruehsen@gmx.de> 18080 18081 removed css_tokenizer.c from po/POTFILES.in 18082 180832014-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 18084 18085 use AM_TESTS_ENVIRONMENT for valgrind testing 18086 180872014-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 18088 18089 suppress lex warnings 18090 180912014-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 18092 18093 building docs with distcheck -jn randomly failed, now using .NOTPARALLEL 18094 180952014-10-07 Tim Rühsen <tim.ruehsen@gmx.de> 18096 18097 tuned makefiles 18098 180992014-09-23 Tim Rühsen <tim.ruehsen@gmx.de> 18100 18101 use ASCII-only string case comparison 18102 181032014-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 18104 18105 added --post-data and --post-file 18106 181072014-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 18108 18109 added --backups 18110 181112014-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 18112 18113 removed debugging from test-directory-clash 18114 181152014-09-22 Tim Rühsen <tim.ruehsen@gmx.de> 18116 18117 fixed directory clashes and added test-directory-clash 18118 181192014-09-18 Tim Rühsen <tim.ruehsen@gmx.de> 18120 18121 added threaded progress bar (--progress=bar) 18122 181232014-09-12 Tim Rühsen <tim.ruehsen@gmx.de> 18124 18125 added progrsss bar code 18126 181272014-09-09 Tim Rühsen <tim.ruehsen@gmx.de> 18128 18129 added --follow-tags and --ignore-tags 18130 181312014-09-07 Tim Ruehsen <tim.ruehsen@gmx.de> 18132 18133 fix for 'unsigned char' architectures 18134 181352014-09-05 Tim Rühsen <tim.ruehsen@gmx.de> 18136 18137 fixed clang detections 18138 181392014-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 18140 18141 rearranged some metalink code 18142 181432014-08-29 Tim Rühsen <tim.ruehsen@gmx.de> 18144 18145 make cross-compilable with MinGW on Linux 18146 181472014-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 18148 18149 added some first MinGW compatibility code 18150 181512014-08-28 Tim Rühsen <tim.ruehsen@gmx.de> 18152 18153 updated PSL file effective_tld_names.dat 18154 181552014-08-27 Tim Rühsen <tim.ruehsen@gmx.de> 18156 18157 amended NEWS 18158 181592014-08-27 Tim Rühsen <tim.ruehsen@gmx.de> 18160 18161 prepared for release 18162 181632014-08-27 Tim Rühsen <tim.ruehsen@gmx.de> 18164 18165 added -m / --mirror 18166 181672014-08-27 Tim Rühsen <tim.ruehsen@gmx.de> 18168 18169 added link to Wikipedia comparison of file systems 18170 181712014-08-27 Tim Rühsen <tim.ruehsen@gmx.de> 18172 18173 added descriptions of options to --help 18174 181752014-08-20 Tim Rühsen <tim.ruehsen@gmx.de> 18176 18177 removed unneeded libpsl reference in src/options.h 18178 181792014-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 18180 18181 increased version number in configure.ac 18182 181832014-08-15 Tim Rühsen <tim.ruehsen@gmx.de> 18184 18185 prepared for release 18186 181872014-07-17 Tim Rühsen <tim.ruehsen@gmx.de> 18188 18189 added -k/--convert-links and -K/--backup-converted 18190 181912014-07-14 Tim Rühsen <tim.ruehsen@gmx.de> 18192 18193 fix tests for when not having libpsl support 18194 181952014-07-11 Tim Rühsen <tim.ruehsen@gmx.de> 18196 18197 use libpsl for cookie checking against Public Suffix List 18198 181992014-07-09 Tim Rühsen <tim.ruehsen@gmx.de> 18200 18201 update PSL data 18202 182032014-07-09 Tim Rühsen <tim.ruehsen@gmx.de> 18204 18205 more cleanups to configure.ac 18206 182072014-07-08 Tim Rühsen <tim.ruehsen@gmx.de> 18208 18209 cleaned up configure.ac 18210 182112014-07-03 Tim Rühsen <tim.ruehsen@gmx.de> 18212 18213 added new test test--accept.c 18214 182152014-07-03 Tim Rühsen <tim.ruehsen@gmx.de> 18216 18217 added --accept, --reject, --ignore-case 18218 182192014-06-29 Tim Ruehsen <tim.ruehsen@gmx.de> 18220 18221 fixed race-condition on 'known_urls' variable 18222 182232014-06-25 Tim Ruehsen <tim.ruehsen@gmx.de> 18224 18225 make distcheck without -j4 18226 182272014-06-25 Tim Ruehsen <tim.ruehsen@gmx.de> 18228 18229 check for HTTP/HTTPS in add_url() 18230 182312014-06-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18232 18233 removed two lines of debugging output 18234 182352014-06-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18236 18237 fixed HTML parsing of uppercase attributes (fixes issue #21) 18238 182392014-05-27 Tim Ruehsen <tim.ruehsen@gmx.de> 18240 18241 do not keep tmpfiles in test--page-requisites.c 18242 182432014-05-27 Tim Ruehsen <tim.ruehsen@gmx.de> 18244 18245 added test--page-requisites and fixed -p/--page-requisites 18246 182472014-04-04 Tim Ruehsen <tim.ruehsen@gmx.de> 18248 18249 fix some clang warnings 18250 182512014-03-21 Tim Ruehsen <tim.ruehsen@gmx.de> 18252 18253 removed psl.c from examples 18254 182552014-03-20 Tim Ruehsen <tim.ruehsen@gmx.de> 18256 18257 changes on psl.c 18258 182592014-03-19 Tim Ruehsen <tim.ruehsen@gmx.de> 18260 18261 make cookies and tests working with new PSL 18262 182632014-03-19 Tim Ruehsen <tim.ruehsen@gmx.de> 18264 18265 working on example/psl.c 18266 182672014-03-19 Tim Ruehsen <tim.ruehsen@gmx.de> 18268 18269 fixed up cookie code 18270 182712014-03-19 Tim Ruehsen <tim.ruehsen@gmx.de> 18272 18273 added experimental PSL code 18274 182752014-03-19 Tim Ruehsen <tim.ruehsen@gmx.de> 18276 18277 updated public suffixes list 18278 182792014-03-08 Tim Ruehsen <tim.ruehsen@gmx.de> 18280 18281 fix sigfault in cookie functions 18282 182832014-03-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18284 18285 added --gnutls-options to NEWS file 18286 182872014-03-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18288 18289 added --gnutls-otpions to directly set a GnuTLS priority string 18290 182912014-03-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18292 18293 added error messages if requested CAs or keys are not found 18294 182952014-02-26 Tim Ruehsen <tim.ruehsen@gmx.de> 18296 18297 improved cookie API 18298 182992014-02-20 Tim Ruehsen <tim.ruehsen@gmx.de> 18300 18301 tuned HSTS API 18302 183032014-02-19 Tim Ruehsen <tim.ruehsen@gmx.de> 18304 18305 added support for RFC 6266 Content-Disposition 18306 183072014-02-17 Tim Ruehsen <tim.ruehsen@gmx.de> 18308 18309 fixed two threading race-conditions 18310 183112014-02-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18312 18313 make HTTPS tests work with GnuTLS 2.12 18314 183152014-02-14 Tim Ruehsen <tim.ruehsen@gmx.de> 18316 18317 fix system ca_directory for GnuTLS < 3 18318 183192014-02-14 Tim Ruehsen <tim.ruehsen@gmx.de> 18320 18321 fixed HTTPS testing to succeed distcheck 18322 183232014-02-12 Tim Ruehsen <tim.ruehsen@gmx.de> 18324 18325 added HTTPS server code for the test suite 18326 183272014-02-11 Tim Ruehsen <tim.ruehsen@gmx.de> 18328 18329 added i/o helper routines to documentation 18330 183312014-02-08 Tim Ruehsen <tim.ruehsen@gmx.de> 18332 18333 added make distcheck to .travis.yml 18334 183352014-02-08 Tim Ruehsen <tim.ruehsen@gmx.de> 18336 18337 fix docs/libmget/Makefile.am again 18338 183392014-02-07 Tim Ruehsen <tim.ruehsen@gmx.de> 18340 18341 fixed man page generation and make distcheck 18342 183432014-02-06 Tim Ruehsen <tim.ruehsen@gmx.de> 18344 18345 fixed generating man pages, added some documentation 18346 183472014-02-05 Tim Ruehsen <tim.ruehsen@gmx.de> 18348 18349 small docs changes 18350 183512014-02-05 Tim Ruehsen <tim.ruehsen@gmx.de> 18352 18353 added plg-config, brushed up configure.ac and Makefiles 18354 183552014-02-05 Tim Ruehsen <tim.ruehsen@gmx.de> 18356 18357 fix type naming to mget type naming convention 18358 183592014-02-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18360 18361 updated gettext 18362 183632014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18364 18365 fixed minor clang warnings 18366 183672014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18368 18369 fixed some minor warnings 18370 183712014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18372 18373 added Travis CI .travis.yml 18374 183752014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18376 18377 fixed minor clang warnings 18378 183792014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18380 18381 fixed some minor warnings 18382 183832014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18384 18385 extended .travis.yml 18386 183872014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18388 18389 fix .travis.yml, typo in configure.ac string 18390 183912014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18392 18393 fix .travis.yml 18394 183952014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18396 18397 fix .travis.yml 18398 183992014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18400 18401 fix .travis.yml 18402 184032014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18404 18405 added Travis CI .travis.yml 18406 184072014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18408 18409 added test routine for HSTS feature 18410 184112014-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18412 18413 fix possible endless loop in http_parse_param() 18414 184152014-02-01 Tim Ruehsen <tim.ruehsen@gmx.de> 18416 18417 added support for RFC 6797 HTTP Strict Transport Security (HSTS) 18418 184192014-02-01 Tim Ruehsen <tim.ruehsen@gmx.de> 18420 18421 Do not download robots.txt twice for HTTP and HTTPS if --protocol-directories is not given 18422 184232014-01-31 Tim Ruehsen <tim.ruehsen@gmx.de> 18424 18425 working on RFC 6797 HSTS (HTTP Strict Transport Security) 18426 184272014-01-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18428 18429 removed failing tests of not yet implemented options 18430 184312014-01-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18432 18433 compile check for available hash functions in libnettle 18434 184352014-01-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18436 18437 use libnettle for checksumming if libgnutls not available or too old 18438 184392014-01-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18440 18441 fixed check for libunistring 18442 184432014-01-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18444 18445 fixed warning in ssl_gnutls.c 18446 184472014-01-21 Tim Ruehsen <tim.ruehsen@gmx.de> 18448 18449 gtk-doc optional, project tested on Cygwin 18450 184512014-01-17 Tim Ruehsen <tim.ruehsen@gmx.de> 18452 18453 edited AUTHORS and NEWS 18454 184552014-01-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18456 18457 brushed up README.md 18458 184592014-01-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18460 18461 added Contacts to README.md 18462 184632014-01-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18464 18465 put mailing list address into configure.ac 18466 184672014-01-12 Tim Ruehsen <tim.ruehsen@gmx.de> 18468 18469 fixed 'make distcheck' 18470 184712014-01-11 Tim Ruehsen <tim.ruehsen@gmx.de> 18472 18473 fixed some clang 3.4 warnings 18474 184752014-01-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18476 18477 added example print_html_url.c (scanning and printing URLs from HTML files) 18478 184792014-01-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18480 18481 fix mailing list address in README.md 2nd 18482 184832014-01-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18484 18485 fix mailing list address in README.md 18486 184872014-01-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18488 18489 added support for bzip2 Content-Encoding / Accept-Encoding type 18490 184912014-01-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18492 18493 added 'lzma' to Content-Encoding 18494 184952013-12-31 Tim Ruehsen <tim.ruehsen@gmx.de> 18496 18497 added support for XZ Content-Encoding / Accept-Encoding type 18498 184992013-12-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18500 18501 extended --adjust-extension to attach .rss and .atom 18502 185032013-12-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18504 18505 added --waitretry, new function mget_millisleep() 18506 185072013-12-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18508 18509 added --wait/-w, --random-wait and -tries/-t 18510 185112013-12-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18512 18513 allow m, h and d modifiers for timeout options 18514 185152013-12-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18516 18517 disable valgrind testing by default, just needed by developers 18518 185192013-12-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18520 18521 use valgrind to check invocations of mget in test suite (and not the test suite itself, as before) 18522 185232013-12-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18524 18525 add IRI that triggers robots.txt to blacklist, create local filename for --continue-download 18526 185272013-12-28 Tim Ruehsen <tim.ruehsen@gmx.de> 18528 18529 fixed test-iri-forced-remote 18530 185312013-12-27 Tim Ruehsen <tim.ruehsen@gmx.de> 18532 18533 fixed test-iri-list 18534 185352013-12-27 Tim Ruehsen <tim.ruehsen@gmx.de> 18536 18537 added new option --input-encoding 18538 185392013-12-25 Tim Ruehsen <tim.ruehsen@gmx.de> 18540 18541 added RSS parsing test 18542 185432013-12-21 Tim Ruehsen <tim.ruehsen@gmx.de> 18544 18545 added parsing RSS 2.0 feeds 18546 185472013-12-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18548 18549 fixed compilation error in atom_url.c 18550 185512013-12-15 Tim Ruehsen <tim.ruehsen@gmx.de> 18552 18553 added parsing Atom 1.0 feeds 18554 185552013-12-14 Tim Ruehsen <tim.ruehsen@gmx.de> 18556 18557 refactored basic sitemap parsing into libmget 18558 185592013-11-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18560 18561 new option --force-sitemap to scan URLs from a local Sitemap XML file 18562 185632013-11-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18564 18565 fix: added query twice if query contained slash(es) 18566 185672013-11-25 Tim Ruehsen <tim.ruehsen@gmx.de> 18568 18569 fixed two more tests 18570 185712013-11-25 Tim Ruehsen <tim.ruehsen@gmx.de> 18572 18573 fixed proxy destruction 18574 185752013-11-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18576 18577 added sitemap parsing 18578 185792013-11-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18580 18581 fix some compiler warnings 18582 185832013-11-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18584 18585 Support arbitrary number of proxies for parallel downloads 18586 185872013-11-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18588 18589 fixed some warnings of clang 3.4 18590 185912013-10-31 Tim Ruehsen <tim.ruehsen@gmx.de> 18592 18593 added --chunk-size to allow multithreaded downloads of single files 18594 185952013-10-27 Tim Ruehsen <tim.ruehsen@gmx.de> 18596 18597 added --(no-)check-hostname to disable certificate hostname checking only 18598 185992013-10-22 Tim Ruehsen <tim.ruehsen@gmx.de> 18600 18601 fix po/POTFILES.in 18602 186032013-10-22 Tim Ruehsen <tim.ruehsen@gmx.de> 18604 18605 update po/POTFILES.in 18606 186072013-10-22 magemore <alex@magemore.com> 18608 18609 fix help: wget replaced with mget 186102013-10-22 Tim Ruehsen <tim.ruehsen@gmx.de> 18611 18612 mention package name zlibg1 in README.md 18613 186142013-10-07 Tim Ruehsen <tim.ruehsen@gmx.de> 18615 18616 updated README.md 18617 186182013-10-06 Tim Ruehsen <tim.ruehsen@gmx.de> 18619 18620 small cleanups regarding use of buffer functions 18621 186222013-10-05 Tim Ruehsen <tim.ruehsen@gmx.de> 18623 18624 check for NULL iri in add_url() 18625 186262013-10-04 Tim Ruehsen <tim.ruehsen@gmx.de> 18627 18628 pull in libunistring if using libidn2 18629 186302013-10-04 Tim Ruehsen <tim.ruehsen@gmx.de> 18631 18632 removed including stringprep.h from examples 18633 186342013-10-04 Tim Ruehsen <tim.ruehsen@gmx.de> 18635 18636 fixed clang 3.4 analyzer issues 18637 186382013-10-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18639 18640 fixed double free in mget.c 18641 186422013-10-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18643 18644 fixed memory leak in hashmap_put_noalloc 18645 186462013-10-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18647 18648 Use Etag in spider mode to prevent double downloading 18649 186502013-10-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18651 18652 use HEAD request in spider mode 18653 186542013-10-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18655 18656 added support for IDNA2008 using Libidn2 18657 186582013-10-01 Tim Ruehsen <tim.ruehsen@gmx.de> 18659 18660 started libidn2 usage 18661 186622013-10-01 Tim Ruehsen <tim.ruehsen@gmx.de> 18663 18664 convert / to %2F instead of %2f 18665 186662013-10-01 Tim Ruehsen <tim.ruehsen@gmx.de> 18667 18668 check if url needs utf-8 encoding 18669 186702013-10-01 Tim Ruehsen <tim.ruehsen@gmx.de> 18671 18672 fixed --restrict-file-names=uppercase 18673 186742013-09-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18675 18676 added --restrict-file-names 18677 186782013-09-28 Tim Ruehsen <tim.ruehsen@gmx.de> 18679 18680 fixed crash on non-valid cookie names 18681 186822013-09-28 Tim Ruehsen <tim.ruehsen@gmx.de> 18683 18684 new file host.c, refactored from mget.c 18685 186862013-09-28 Tim Ruehsen <tim.ruehsen@gmx.de> 18687 18688 moved robots routines to new file robots.c 18689 186902013-09-27 Tim Ruehsen <tim.ruehsen@gmx.de> 18691 18692 fixed up ROBOTS/META 18693 186942013-09-25 Tim Ruehsen <tim.ruehsen@gmx.de> 18695 18696 fine tuning on some library functions 18697 186982013-09-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18699 18700 added --trust-server-names and --restrict-file-names 18701 187022013-09-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18703 18704 preparing for utf-16 support 18705 187062013-09-23 Tim Ruehsen <tim.ruehsen@gmx.de> 18707 18708 recognize 'none' keyword in META/ROBOTS 18709 187102013-09-23 Tim Ruehsen <tim.ruehsen@gmx.de> 18711 18712 added support for name=ROBOTS in META tag 18713 187142013-09-23 Tim Ruehsen <tim.ruehsen@gmx.de> 18715 18716 added src/host.h 18717 187182013-09-22 Tim Ruehsen <tim.ruehsen@gmx.de> 18719 18720 added support for robots.txt Robots Exclusion Standard 18721 187222013-09-20 Tim Ruehsen <tim.ruehsen@gmx.de> 18723 18724 added option --(no-)parent 18725 187262013-09-19 Tim Ruehsen <tim.ruehsen@gmx.de> 18727 18728 more work on the test suite 18729 187302013-09-17 Tim Ruehsen <tim.ruehsen@gmx.de> 18731 18732 added --https-only 18733 187342013-09-17 Tim Ruehsen <tim.ruehsen@gmx.de> 18735 18736 added -e/--execute 18737 187382013-09-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18739 18740 test code tweaking 18741 187422013-09-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18743 18744 fixed test suite issues 18745 187462013-09-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18747 18748 some tweaks on encoding 18749 187502013-09-15 Tim Ruehsen <tim.ruehsen@gmx.de> 18751 18752 better handling of non-ASCII URLs and filenames 18753 187542013-09-14 Tim Ruehsen <tim.ruehsen@gmx.de> 18755 18756 adjust README/README.md 18757 187582013-09-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18759 18760 prepared for listing/scanning directories 18761 187622013-09-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18763 18764 fix memleak in cookie handling 18765 187662013-09-12 Tim Ruehsen <tim.ruehsen@gmx.de> 18767 18768 more work done 18769 187702013-09-10 Tim Ruehsen <tim.ruehsen@gmx.de> 18771 18772 moved metalink stuff to library code 18773 187742013-09-09 Tim Ruehsen <tim.ruehsen@gmx.de> 18775 18776 reenabled metalink3 18777 187782013-09-09 Tim Ruehsen <tim.ruehsen@gmx.de> 18779 18780 threading model reimplemented to put more load onto the workers 18781 187822013-08-18 Tim Ruehsen <tim.ruehsen@gmx.de> 18783 18784 tune xml.c 18785 187862013-08-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18787 18788 mention --secure-protocol=SECURE in help text 18789 187902013-08-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18791 18792 xml parser rewritten to output len and pos of values 18793 187942013-08-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18795 18796 simplified some calls to mget_buffer_init() using C99 18797 187982013-08-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18799 18800 fix GnuTLS crash on handshake errors 18801 188022013-08-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18803 18804 added --secure-protocol=secure to force Perfect Forward Secrecy 18805 188062013-08-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18807 18808 small corrections in ssl_gnutls.c 18809 188102013-07-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18811 18812 prefer Digest over Basic authentication if server offers both 18813 188142013-07-30 Tim Ruehsen <tim.ruehsen@gmx.de> 18815 18816 add -fPIC to CFLAGS, remove private path from tests 18817 188182013-07-28 Tim Ruehsen <tim.ruehsen@gmx.de> 18819 18820 added test-auth-basic.c 18821 188222013-07-27 Tim Ruehsen <tim.ruehsen@gmx.de> 18823 18824 more tests 18825 188262013-05-12 Tim Ruehsen <tim.ruehsen@gmx.de> 18827 18828 use TCP Fastopen if available 18829 188302013-05-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18831 18832 fix bug in hashmap.c 18833 188342013-04-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18835 18836 highlevel improvement 18837 188382013-04-24 Tim Ruehsen <tim.ruehsen@gmx.de> 18839 18840 escape / in query for filename generation 18841 188422013-04-21 Tim Ruehsen <tim.ruehsen@gmx.de> 18843 18844 tuned getstream.c example 18845 188462013-04-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18847 18848 added support for metalink3 18849 188502013-04-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18851 18852 updated Requirements in README.rst 18853 188542013-04-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18855 18856 handle metalink mirror redirections 18857 188582013-04-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18859 18860 changed part download message 18861 188622013-04-12 Tim Ruehsen <tim.ruehsen@gmx.de> 18863 18864 renamed option --base-url to --base 18865 188662013-04-09 Tim Ruehsen <tim.ruehsen@gmx.de> 18867 18868 fixed stringmap/hashmap usage in mget.c and blacklist.c 18869 188702013-04-07 Tim Ruehsen <tim.ruehsen@gmx.de> 18871 18872 getstream.c example now handles Shoutcast metainfo 18873 188742013-03-29 Tim Ruehsen <tim.ruehsen@gmx.de> 18875 18876 cleaned up hashmap/stringmap API 18877 188782013-03-28 Tim Ruehsen <tim.ruehsen@gmx.de> 18879 18880 let HTTP test server understand Range headers 18881 188822013-03-21 Tim Ruehsen <tim.ruehsen@gmx.de> 18883 18884 converted more tests from Perl to C IV 18885 188862013-03-21 Tim Ruehsen <tim.ruehsen@gmx.de> 18887 18888 converted more tests from Perl to C III 18889 188902013-03-19 Tim Ruehsen <tim.ruehsen@gmx.de> 18891 18892 converted more tests from Perl to C II 18893 188942013-03-17 Tim Ruehsen <tim.ruehsen@gmx.de> 18895 18896 converted more tests from Perl to C 18897 188982013-03-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18899 18900 work on test suite 18901 189022013-03-12 Tim Ruehsen <tim.ruehsen@gmx.de> 18903 18904 added --content-disposition 18905 189062013-03-10 Tim Ruehsen <tim.ruehsen@gmx.de> 18907 18908 new C testsuite 18909 189102013-03-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18911 18912 fixed double free in blacklist.c 18913 189142013-03-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18915 18916 added option -p and -l 18917 189182013-02-06 Benoit Goudreault-Emond <bgoudreault@webmd.net> 18919 18920 Ensure the m4 directory is created, or builds won't work out of the box. 18921 189222013-02-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18923 18924 new example getting a stream from a .m3u URL 18925 189262013-02-02 Tim Ruehsen <tim.ruehsen@gmx.de> 18927 18928 new abstract thread functions 18929 189302013-01-30 Benoit Goudreault-Emond <bgoudreault@webmd.net> 18931 18932 Fix errors in potfile.in that made makefile fail 18933 189342013-01-29 Tim Ruehsen <tim.ruehsen@gmx.de> 18935 18936 put work into high level example http_get.c 18937 189382013-01-21 Tim Ruehsen <tim.ruehsen@gmx.de> 18939 18940 added libmget/md5.c 18941 189422013-01-18 Tim Ruehsen <tim.ruehsen@gmx.de> 18943 18944 Updated README.rst 18945 189462013-01-18 Tim Ruehsen <tim.ruehsen@gmx.de> 18947 18948 moved http.c and md5.c into library 18949 189502013-01-18 rockdaboot <tim.ruehsen@gmx.de> 18951 18952 Update README.rst 189532013-01-18 rockdaboot <tim.ruehsen@gmx.de> 18954 18955 Update README.rst 189562013-01-17 Tim Ruehsen <tim.ruehsen@gmx.de> 18957 18958 fix more stuff 18959 189602013-01-16 Tim Ruehsen <tim.ruehsen@gmx.de> 18961 18962 more basic work on libmget 18963 189642013-01-15 Tim Ruehsen <tim.ruehsen@gmx.de> 18965 18966 new examples folder 18967 189682013-01-13 Tim Ruehsen <tim.ruehsen@gmx.de> 18969 18970 fix make distcheck 18971 189722013-01-11 Tim Ruehsen <tim.ruehsen@gmx.de> 18973 18974 more stuff moved into libmget 18975 189762013-01-10 Tim Ruehsen <tim.ruehsen@gmx.de> 18977 18978 more stuff moved into libmget 18979 189802013-01-07 Tim Ruehsen <tim.ruehsen@gmx.de> 18981 18982 added two doc files to git 18983 189842013-01-04 Tim Ruehsen <tim.ruehsen@gmx.de> 18985 18986 more documentation stuff 18987 189882013-01-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18989 18990 started gtk-docs 18991 189922013-01-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18993 18994 updated README.rst 18995 189962013-01-03 Tim Ruehsen <tim.ruehsen@gmx.de> 18997 18998 more auto* stuff 18999 190002012-12-31 Tim Ruehsen <tim.ruehsen@gmx.de> 19001 19002 use LGPL for libmget 19003 190042012-12-28 Tim Ruehsen <tim.ruehsen@gmx.de> 19005 19006 step forward to libmget 19007 190082012-12-26 Tim Ruehsen <tim.ruehsen@gmx.de> 19009 19010 HTTP basic/digest authentication, send Referer HTTP header 19011 190122012-12-21 Tim Ruehsen <tim.ruehsen@gmx.de> 19013 19014 started Authentication 19015 190162012-12-20 Tim Ruehsen <tim.ruehsen@gmx.de> 19017 19018 use gcc -Wsign-compare option 19019 190202012-12-20 Tim Ruehsen <tim.ruehsen@gmx.de> 19021 19022 fixed --secure-protocol=SSLv2/SSLv3 19023 190242012-12-17 Tim Ruehsen <tim.ruehsen@gmx.de> 19025 19026 added -D/--domains and --exclude-domains 19027 190282012-12-17 Tim Ruehsen <tim.ruehsen@gmx.de> 19029 19030 added 'deflate' Content-Encoding 19031 190322012-12-14 Tim Ruehsen <tim.ruehsen@gmx.de> 19033 19034 IDN / international domain support finished 19035 190362012-12-11 Tim Ruehsen <tim.ruehsen@gmx.de> 19037 19038 removed an URI test that had no host host part 19039 190402012-12-10 Tim Ruehsen <tim.ruehsen@gmx.de> 19041 19042 added IDN conversion for locally provided URIs 19043 190442012-12-09 Tim Ruehsen <tim.ruehsen@gmx.de> 19045 19046 started with IDN / URI encoding 19047 190482012-12-08 Tim Ruehsen <tim.ruehsen@gmx.de> 19049 19050 added --bind-address, fixed parsing IPv6 numeric addresses 19051 190522012-12-03 Tim Ruehsen <tim.ruehsen@gmx.de> 19053 19054 added -Q/--quota, added configure script 19055 190562012-11-30 Tim Ruehsen <tim.ruehsen@gmx.de> 19057 19058 started with -Q 19059 190602012-11-30 Tim Ruehsen <tim.ruehsen@gmx.de> 19061 19062 added -i, -F, -B, --force-css 19063 190642012-11-28 Tim Ruehsen <tim.ruehsen@gmx.de> 19065 19066 added --default_page 19067 190682012-11-28 Tim Ruehsen <tim.ruehsen@gmx.de> 19069 19070 added -E/--adjust-extension/--html-extension, C89 NONNULL macro 19071 190722012-11-28 Tim Ruehsen <tim.ruehsen@gmx.de> 19073 19074 removed variadic macros 19075 190762012-11-24 Tim Ruehsen <tim.ruehsen@gmx.de> 19077 19078 updated README.rst 19079 190802012-11-24 Tim Ruehsen <tim.ruehsen@gmx.de> 19081 19082 using autotools 19083 190842012-11-21 Tim Ruehsen <tim.ruehsen@gmx.de> 19085 19086 directory structure changes 19087 190882012-11-21 Tim Ruehsen <tim.ruehsen@gmx.de> 19089 19090 some hashmap optimization 19091 190922012-11-20 Tim Ruehsen <tim.ruehsen@gmx.de> 19093 19094 fixed double free in hashmap_put_noalloc for key/value identity 19095 190962012-11-19 Tim Ruehsen <tim.ruehsen@gmx.de> 19097 19098 added --referer 19099 191002012-11-18 Tim Ruehsen <tim.ruehsen@gmx.de> 19101 19102 added --save-headers 19103 191042012-11-18 Tim Ruehsen <tim.ruehsen@gmx.de> 19105 19106 check for gnutls version number when using *ECDH* stuff 19107 191082012-11-16 Tim Ruehsen <tim.ruehsen@gmx.de> 19109 19110 updated BSD compatibility 19111 191122012-11-16 Tim Ruehsen <tim.ruehsen@gmx.de> 19113 19114 added --clobber / -nc 19115 191162012-11-16 Tim Ruehsen <tim.ruehsen@gmx.de> 19117 19118 added --cache to enable/disable server cache 19119 191202012-11-16 Tim Ruehsen <tim.ruehsen@gmx.de> 19121 19122 added --inet4-only, --inet6-only, --prefer-family 19123 191242012-11-15 Tim Ruehsen <tim.ruehsen@gmx.de> 19125 19126 added --delete-after 19127 191282012-11-15 Tim Ruehsen <tim.ruehsen@gmx.de> 19129 19130 added -n options 19131 191322012-11-15 Tim Ruehsen <tim.ruehsen@gmx.de> 19133 19134 added --strict-comments as dummy option 19135 191362012-11-15 Tim Ruehsen <tim.ruehsen@gmx.de> 19137 19138 added directory options and timestamping 19139 191402012-11-12 Tim Ruehsen <tim.ruehsen@gmx.de> 19141 19142 added --continue / -c 19143 191442012-11-12 Tim Ruehsen <tim.ruehsen@gmx.de> 19145 19146 added --server-response / -S 19147 191482012-11-12 Tim Ruehsen <tim.ruehsen@gmx.de> 19149 19150 added proxy support 19151 191522012-11-09 Tim Ruehsen <tim.ruehsen@gmx.de> 19153 19154 added blacklist.(c|h) and stringmap.(c|h) 19155 191562012-11-08 Tim Ruehsen <tim.ruehsen@gmx.de> 19157 19158 added hashmap functions 19159 191602012-11-06 Tim Ruehsen <tim.ruehsen@gmx.de> 19161 19162 cookie support incl. Mozilla Public Suffix List 19163 191642012-10-05 Tim Ruehsen <tim.ruehsen@gmx.de> 19165 19166 url escaping 19167 191682012-09-27 Tim Ruehsen <tim.ruehsen@gmx.de> 19169 19170 buffer_printf routines 19171 191722012-09-27 Tim Ruehsen <tim.ruehsen@gmx.de> 19173 19174 more internal work 19175 191762012-08-29 Tim Ruehsen <tim.ruehsen@gmx.de> 19177 19178 more gnutls debugging output 19179 191802012-08-27 Tim Ruehsen <tim.ruehsen@gmx.de> 19181 19182 HTTPS options to achieve wget compatibility 19183 191842012-08-24 Tim Ruehsen <tim.ruehsen@gmx.de> 19185 19186 more test functions 19187 191882012-08-22 Tim Ruehsen <tim.ruehsen@gmx.de> 19189 19190 added buffer functions 19191 191922012-08-21 Tim Ruehsen <tim.ruehsen@gmx.de> 19193 19194 more tests, some fixes 19195 191962012-08-17 Tim Ruehsen <tim.ruehsen@gmx.de> 19197 19198 added options -o and -a 19199 192002012-08-16 Tim Ruehsen <tim.ruehsen@gmx.de> 19201 19202 added option -q 19203 192042012-08-15 Tim Ruehsen <tim.ruehsen@gmx.de> 19205 19206 fix gnutls 19207 192082012-08-14 Tim Ruehsen <tim.ruehsen@gmx.de> 19209 19210 * fixed vector.c / vec_insert_sorted_noalloc() 19211 * added test for vec_insert_sorted_noalloc() 19212 * fixed css.c / css_parse_file() 19213 192142012-08-14 Tim Ruehsen <tim.ruehsen@gmx.de> 19215 19216 implemented DNS lookup cache, --dns-cache=on|off 19217 192182012-08-13 Ant Bryan <anthonybryan@gmail.com> 19219 19220 README.rst typos 19221 192222012-08-10 Tim Ruehsen <tim.ruehsen@gmx.de> 19223 19224 some refactoring, handle relative Location: 19225 192262012-08-10 Tim Ruehsen <tim.ruehsen@gmx.de> 19227 19228 updated README.rst 19229 192302012-08-10 Tim Ruehsen <tim.ruehsen@gmx.de> 19231 19232 updated README.rst 19233 192342012-08-10 Tim Ruehsen <tim.ruehsen@gmx.de> 19235 19236 new README.rst 19237 192382012-08-10 Tim Ruehsen <tim.ruehsen@gmx.de> 19239 19240 * HTTPS via gnutls 19241 * Many bug fixes and enhancements 19242 192432012-08-03 Tim Ruehsen <tim.ruehsen@gmx.de> 19244 19245 Updated README 19246 192472012-08-03 Tim Ruehsen <tim.ruehsen@gmx.de> 19248 19249 Repository created 19250
