12019-02-18 Stuart Caie <kyzer@cabextract.org.uk> 2 3 * chmd_read_headers(): a CHM file name beginning "::" but shorter 4 than 33 bytes will lead to reading past the freshly-allocated name 5 buffer - checks for specific control filenames didn't take length 6 into account. Thanks to ADLab of Venustech for the report and 7 proof of concept. 8 92019-02-18 Stuart Caie <kyzer@cabextract.org.uk> 10 11 * chmd_read_headers(): CHM files can declare their chunks are any 12 size up to 4GB, and libmspack will attempt to allocate that to 13 read the file. 14 15 This is not a security issue; libmspack doesn't promise how much 16 memory it'll use to unpack files. You can set your own limits by 17 returning NULL in a custom mspack_system.alloc() implementation. 18 19 However, it would be good to validate chunk size further. With no 20 offical specification, only empirical data is available. All files 21 created by hhc.exe have a chunk size of 4096 bytes, and this is 22 matched by all the files I've found in the wild, except for one 23 which has a chunk size of 8192 bytes, which was created by someone 24 developing a CHM file creator 15 years ago, and they appear to 25 have abandoned it, so it seems 4096 is a de-facto standard. 26 27 I've changed the "chunk size is not a power of two" warning to 28 "chunk size is not 4096", and now only allow chunk sizes between 29 22 and 8192 bytes. If you have CHM files with a larger chunk size, 30 please send them to me and I'll increase this upper limit. 31 32 Thanks to ADLab of Venustech for the report. 33 342019-02-18 Stuart Caie <kyzer@cabextract.org.uk> 35 36 * oabd.c: replaced one-shot copying of uncompressed blocks (which 37 requires allocating a buffer of the size declared in the header, 38 which can be 4GB) with a fixed-size buffer. The buffer size is 39 user-controllable with the new msoab_decompressor::set_param() 40 method (check you have version 2 of the OAB decompressor), and 41 also controls the input buffer used for OAB's LZX decompression. 42 43 Reminder: compression formats can dictate how much memory is 44 needed to decompress them. If memory usage is a security concern 45 to you, write a custom mspack_system.alloc() that returns NULL 46 if "too much" memory is requested. Do not rely on libmspack adding 47 special heuristics to know not to request "too much". 48 49 Thanks to ADLab of Venustech for the report. 50 512018-11-03 Stuart Caie <kyzer@cabextract.org.uk> 52 53 * configure.ac, doc/Makefile.in, doc/Doxyfile.in: remove these 54 template files and replace with static files. You can still build 55 the documentation with make -C doc 56 572018-11-03 Stuart Caie <kyzer@cabextract.org.uk> 58 59 * Makefile.am, src: move the "useful" programs in src/ to examples/ 60 and don't auto-install them. Even though they're useful, they are 61 intended as examples and aren't productised (no commmand-line 62 options, no man pages, etc.) -- if you disagree, feel free to 63 send in a patch 64 652018-11-01 Stuart Caie <kyzer@cabextract.org.uk> 66 67 * cabd_extract(): would not do decompression for random-access 68 offsets if the folder type was LZX. This is a fairly major bug, 69 and affects any decompression where you skip directly to a file, 70 or decompress data out-of-order. Thanks to austin987 for alerting 71 me to this. 72 73 This bug was introduced by the recent 'salvage mode' patch. Even 74 though I'd reviewed all the differences in clamav's copy of 75 libmspack and said "wtf" to this particular change, I didn't 76 notice it was still in the resulting patch I merged. Mea culpa :) 77 78 * test/cabd_test.c: now has a regression test to cover this 79 802018-10-31 Stuart Caie <kyzer@cabextract.org.uk> 81 82 * Makefile.am, test/*_test.c: use the automake test-suite system 83 with the test-suite programs (cabd_test, chmd_test, kwajd_test). 84 This also fixes a longstanding bugbear that these programs don't 85 access their test files using an absolute path. Now this is passed 86 to them and you can run them from any directory. Thanks to Richard 87 Jones for requesting this. 88 892018-10-31 Stuart Caie <kyzer@cabextract.org.uk> 90 91 * configure.ac: require at least automake 1.11, use AM_SILENT_RULES 92 unconditionally 93 942018-10-30 Stuart Caie <kyzer@cabextract.org.uk> 95 96 * configure.ac: remove obsolescent C library tests. AC_HEADER_STDC is 97 removed, and so are most checks for standard C headers. libmspack now 98 makes these assumptions: 99 - <ctype.h> <limits.h> <stdlib.h> <string.h> exist 100 - <ctype.h> defines tolower() 101 - <string.h> defines memset(), memcmp(), strlen() 102 - if towlower() exists, it's defined in <wctype.h> 103 1042018-10-22 Stuart Caie <kyzer@cabextract.org.uk> 105 106 * cabd.c: remove the only use of assert() 107 1082018-10-20 Stuart Caie <kyzer@cabextract.org.uk> 109 110 * src/chmextract.c: add anti "../" and leading slash protection to 111 chmextract. I'm not pleased about this. All the sample code provided 112 with libmspack is meant to be simple examples of library use, not 113 "productised" binaries. Making the "useful" code samples install 114 as binaries was a mistake. They were never intended to protect you 115 from unpacking archive files with relative/absolute paths, and I 116 would prefer that they never will be. 117 1182018-10-17 Stuart Caie <kyzer@cabextract.org.uk> 119 120 * cab.h: Make the CAB block input buffer one byte larger, to allow 121 a maximum-allowed-size input block and the special extra byte added 122 after the block by cabd_sys_read_block to help Quantum alignment. 123 Thanks to Henri Salo for reporting this. 124 1252018-10-17 Stuart Caie <kyzer@cabextract.org.uk> 126 127 * chmd_read_headers(): again reject files with blank filenames, this 128 time because their 1st or 2nd byte is null, not because their length 129 is zero. Thanks again to Hanno Böck for finding the issue. 130 1312018-10-16 Stuart Caie <kyzer@cabextract.org.uk> 132 133 * Makefile.am: using automake _DEPENDENCIES for chmd_test appears to 134 override the default dependencies (e.g. sources), so libchmd.la was no 135 longer considered a dependency of chmd_test. This breaks parallel 136 builds like "make -j4". Added libchmd.la explicitly to dependencies. 137 Thanks to Thomas Deutschmann for reporting this. 138 1392018-10-16 Stuart Caie <kyzer@cabextract.org.uk> 140 141 * cabd.c: add new parameter, MSCABD_PARAM_SALVAGE, which makes CAB file 142 reading and extraction more lenient, to allow damaged or mangled CABs 143 to be extracted. When enabled: 144 - cabd->open() won't reject cabinets with files that have invalid 145 folder indices or filenames. These files will simply be skipped 146 - cabd->extract() won't reject files with invalid lengths, but will 147 limit them to the maximum possible 148 - block output sizes over 32768 bytes won't be rejected 149 - invalid data block checksums won't be rejected 150 151 It's still possible for corrupted files to fail extraction, but more 152 data can be extracted before they do. 153 154 This new parameter doesn't affect the existing MSCABD_PARAM_FIXMSZIP 155 parameter, which ignores MSZIP decompression failures. You can enable 156 both at once. 157 158 Thanks to Micah Snyder from ClamAV for working with me to get this 159 feature into libmspack. This also helps ClamAV move towards using a 160 vanilla copy of libmspack without needing their own patchset. 161 1622018-08-13 Stuart Caie <kyzer@cabextract.org.uk> 163 164 * mspack.h: clarify that mspack_system.free() should allow NULL. If your 165 mspack_system implementation doesn't, it would already have crashed, as 166 there are several places where libmspack calls sys->free(NULL). This 167 change makes it official, and amends a few "if (x) sys->free(x)" cases 168 to the simpler "sys->free(x)" to make it clearer. 169 1702018-08-09 Stuart Caie <kyzer@cabextract.org.uk> 171 172 * Makefile.am: the test file cve-2015-4467-reset-interval-zero.chm is 173 detected by ClamAV as BC.Legacy.Exploit.CVE_2012_1458-1 "infected". 174 My hosting deletes anything that ClamAV calls "infected", so has been 175 continually deleting the official libmspack 0.7alpha release. 176 177 CVE-2012-1458 is the same issue as CVE-2015-4467: both libmspack, and 178 ClamAV using libmspack, could get a division-by-zero crash when the LZX 179 reset interval was zero. This was fixed years ago, but ClamAV still has 180 it as a signature, which today prevents me from releasing libmspack. 181 182 BC.Legacy.Exploit.CVE_2012_1458-1 is a bytecode signature, so I can't 183 see the exact trigger conditions, but I can see that it looks for the 184 "LZXC" signature of the LZX control file, so I've changed this to 185 "lzxc" and added a step in the Makefile to change it back to LZXC, so 186 I can release libmspack whether or not ClamAV keeps the signature. 187 1882018-04-26 Stuart Caie <kyzer@cabextract.org.uk> 189 190 * read_chunk(): the test that chunk numbers are in bounds was off 191 by one, so read_chunk() returned a pointer taken from outside 192 allocated memory that usually crashes libmspack when accessed. 193 Thanks to Hanno Böck for finding the issue and providing a sample. 194 195 * chmd_read_headers(): reject files with blank filenames. Thanks 196 again to Hanno Böck for finding the issue and providing a sample file. 197 1982018-02-06 Stuart Caie <kyzer@cabextract.org.uk> 199 200 * chmd.c: fixed an off-by-one error in the TOLOWER() macro, reported 201 by Dmitry Glavatskikh. Thanks Dmitry! 202 2032017-11-26 Stuart Caie <kyzer@cabextract.org.uk> 204 205 * kwajd_read_headers(): fix up the logic of reading the filename and 206 extension headers to avoid a one or two byte overwrite. Thanks to 207 Jakub Wilk for finding the issue. 208 209 * test/kwajd_test.c: add tests for KWAJ filename.ext handling 210 2112017-10-16 Stuart Caie <kyzer@cabextract.org.uk> 212 213 * test/cabd_test.c: update the short string tests to expect not only 214 MSPACK_ERR_DATAFORMAT but also MSPACK_ERR_READ, because of the recent 215 change to cabd_read_string(). Thanks to maitreyee43 for spotting this. 216 217 * test/msdecompile_md5: update the setup instructions for this script, 218 and also change the script so it works with current Wine. Again, thanks 219 to maitreyee43 for trying to use it and finding it not working. 220 2212017-08-13 Stuart Caie <kyzer@cabextract.org.uk> 222 223 * src/chmextract.c: support MinGW one-arg mkdir(). Thanks to AntumDeluge 224 for reporting this. 225 2262017-08-13 Stuart Caie <kyzer@cabextract.org.uk> 227 228 * read_spaninfo(): a CHM file can have no ResetTable and have a 229 negative length in SpanInfo, which then feeds a negative output length 230 to lzxd_init(), which then sets frame_size to a value of your choosing, 231 the lower 32 bits of output length, larger than LZX_FRAME_SIZE. If the 232 first LZX block is uncompressed, this writes data beyond the end of the 233 window. This issue was raised by ClamAV as CVE-2017-6419. Thanks to 234 Sebastian Andrzej Siewior for finding this by chance! 235 236 * lzxd_init(), lzxd_set_output_length(), mszipd_init(): due to the issue 237 mentioned above, these functions now reject negative lengths 238 2392017-08-05 Stuart Caie <kyzer@cabextract.org.uk> 240 241 * cabd_read_string(): add missing error check on result of read(). 242 If an mspack_system implementation returns an error, it's interpreted 243 as a huge positive integer, which leads to reading past the end of the 244 stack-based buffer. Thanks to Sebastian Andrzej Siewior for explaining 245 the problem. This issue was raised by ClamAV as CVE-2017-11423 246 2472016-04-20 Stuart Caie <kyzer@cabextract.org.uk> 248 249 * configure.ac: change my email address to kyzer@cabextract.org.uk 250 2512015-05-10 Stuart Caie <kyzer@4u.net> 252 253 * cabd_read_string(): correct rejection of empty strings. Thanks to 254 Hanno Böck for finding the issue and providing a sample file. 255 2562015-05-10 Stuart Caie <kyzer@4u.net> 257 258 * Makefile.am: Add subdir-objects option as suggested by autoreconf. 259 260 * configure.ac: Add AM_PROG_AR as suggested by autoreconf. 261 2622015-01-29 Stuart Caie <kyzer@4u.net> 263 264 * system.h: if C99 inttypes.h exists, use its PRI{d,u}{32,64} macros. 265 Thanks to Johnathan Kollasch for the suggestion. 266 2672015-01-18 Stuart Caie <kyzer@4u.net> 268 269 * lzxd_decompress(): the byte-alignment code for reading uncompressed 270 block headers presumed it could wind i_ptr back 2 bytes, but this 271 hasn't been true since READ_BYTES was allowed to read bytes straddling 272 two blocks, leaving just 1 byte in the read buffer. Thanks to Jakub 273 Wilk for finding the issue and providing a sample file. 274 275 * inflate(): off-by-one error. Distance codes are 0-29, not 0-30. 276 Thanks to Jakub Wilk again. 277 278 * chmd_read_headers(), search_chunk(): another fix for checking pointer 279 is within a chunk, thanks again to Jakub Wilk. 280 2812015-01-17 Stuart Caie <kyzer@4u.net> 282 283 * GET_UTF8_CHAR(): Remove 5/6-byte encoding support and check decoded 284 chars are no more than U+10FFFF. 285 286 * chmd_init_decomp(): A reset interval of 0 is invalid. Thanks to 287 Jakub Wilk for finding the issue and providing a sample and patch. 288 2892015-01-15 Stuart Caie <kyzer@4u.net> 290 291 * chmd_read_headers(): add a bounds check to prevent over-reading data, 292 which caused a segfault on 32-bit architectures. Thanks to Jakub Wilk. 293 294 * search_chunk(): change the order of pointer arithmetic operations to 295 avoid overflow during bounds checks, which lead to segfaults on 32-bit 296 architectures. Again, thanks to Jakub Wilk for finding this issue, 297 providing sample files and a patch. 298 2992015-01-08 Stuart Caie <kyzer@4u.net> 300 301 * cabd_extract(): No longer uses broken state data if extracting from 302 folder 1, 2, 1 and setting up folder 2 fails. This prevents a jump to 303 null and thus segfault. Thanks to Jakub Wilk again. 304 305 * cabd_read_string: reject empty strings. They are not found in any 306 valid CAB files. Thanks to Hanno Böck for sending me an example. 307 3082015-01-05 Stuart Caie <kyzer@4u.net> 309 310 * cabd_can_merge_folders(): disallow folder merging if the combined 311 folder would have more than 65535 data blocks. 312 313 * cabd_decompress(): disallow files if their offset, length or 314 offset+length is more than 65535*32768, the maximum size of any 315 folder. Thanks to Jakub Wilk for identifying the problem and providing 316 a sample file. 317 3182014-04-20 Stuart Caie <kyzer@4u.net> 319 320 * readhuff.h: fixed the table overflow check, which allowed one more 321 code after capacity had been reached, resulting in a read of 322 uninitialized data inside the decoding table. Thanks to Denis Kroshin 323 for identifying the problem and providing a sample file. 324 3252013-05-27 Stuart Caie <kyzer@4u.net> 326 327 * test/oabx.c: added new example command for unpacking OAB files. 328 3292013-05-17 Stuart Caie <kyzer@4u.net> 330 331 * mspack.h: Support for decompressing a new file format, the Exchange 332 Offline Address Book (OAB). Thanks to David Woodhouse for writing 333 the implementation. I've bumped the version to 0.4alpha in celebration. 334 3352012-04-15 Stuart Caie <kyzer@4u.net> 336 337 * chmd_read_headers(): More thorough validation of CHM header values. 338 Thanks to Sergei Trofimovich for finding sample files. 339 340 * read_reset_table(): Better test for overflow. Thanks again to 341 Sergei Trofimovich for generating a good example. 342 343 * test/chminfo.c: this test program reads the reset table by itself 344 and was also susceptible to the same overflow problems. 345 3462012-03-16 Stuart Caie <kyzer@4u.net> 347 348 * Makefile.am, configure.ac: make the GCC warning flags conditional 349 on using the GCC compiler. Thanks to Dagobert Michelsen for letting 350 me know. 351 3522011-11-25 Stuart Caie <kyzer@4u.net> 353 354 * lzxd_decompress(): Prevent matches that go beyond the start 355 of the LZX stream. Thanks to Sergei Trofimovich for testing 356 with valgrind and finding a corrupt sample file that exercises 357 this scenario. 358 3592011-11-23 Stuart Caie <kyzer@4u.net> 360 361 * chmd_fast_find(): add a simple check against infinite PMGL 362 loops. Thanks to Sergei Trofimovich for finding sample files. 363 Multi-step PMGL/PMGI infinite loops remain possible. 364 3652011-06-17 Stuart Caie <kyzer@4u.net> 366 367 * read_reset_table(): wasn't reading the right offset for getting 368 the LZX uncompressed length. Thanks to Sergei Trofimovich for 369 finding the bug. 370 3712011-05-31 Stuart Caie <kyzer@4u.net> 372 373 * kwajd.c, mszipd.c: KWAJ type 4 files (MSZIP) are now supported. 374 Thanks to Clive Turvey for sending me the format details. 375 376 * doc/szdd_kwaj_format.html: Updated documentation to cover 377 KWAJ's MSZIP compression. 378 3792011-05-11 Stuart Caie <kyzer@4u.net> 380 381 * cabd_find(): rethought how large vs small file support is 382 handled, as users were getting "library not compiled to support 383 large files" message on some small files. Now checks for actual 384 off_t overflow, rather than trying to preempt it. 385 3862011-05-10: Stuart Caie <kyzer@4u.net> 387 388 * chmd.c: implemented fast_find() 389 390 * test/chmx.c: removed the multiple extraction orders, now it just 391 extracts in the fastest order 392 393 * test/chmd_order.c: new program added to test that different 394 extraction orders don't affect the results of extraction 395 396 * test/chmd_find.c: new program to test that fast_find() works. 397 Either supply your own filename to find, or it will try finding 398 every file in the CHM. 399 400 * configure.ac: because CHM fast find requires case-insensitive 401 comparisons, tolower() or towlower() are used where possible. 402 These functions and their headers are checked for. 403 404 * mspack.h: exposed struct mschmd_sec_mscompressed's spaninfo 405 and struct mschmd_header's first_pmgl, last_pmgl and chunk_cache 406 to the world. Check that the CHM decoder version is v2 or higher 407 before using them. 408 409 * system.c: set CHM decoder version to v2 410 4112011-04-27: Stuart Caie <kyzer@4u.net> 412 413 * many files: Made C++ compilers much happier with libmspack. 414 Changed char * to const char * where possible. 415 416 * mspack.h: Changed user-supplied char * to const char *. 417 Unless you've written your own mspack_system implementation, 418 you will likely be unaffected. 419 If you have written your own mspack_system implementation: 420 1: change open() so it takes a const char *filename 421 2: change message() so it takes a const char *format 422 If you cast your function into the mspack_system struct, 423 you can change the cast instead of the function. 424 4252011-04-27: Stuart Caie <kyzer@4u.net> 426 427 * Makefile.am: changed CFLAGS from "-Wsign-compare -Wconversion 428 -pedantic" to "-W -Wno-unused". This enables more warnings, and 429 disables these specific warnings which are now a hinderance. 430 4312011-04-27: Stuart Caie <kyzer@4u.net> 432 433 * test/cabrip.c, test/chminfo.c: used macros from system.h for 434 printing offsets and reading 64-bit values, rather than 435 reinvent the wheel. 436 437 * cabd_can_merge_folders(): declare variables at the start of 438 a block so older C compilers won't choke. 439 440 * cabd_find(): avoid compiler complaints about non-initialised 441 variables. We know they'll get initialised before use, but the 442 compiler can't reverse a state machine to draw the same conclusion. 443 4442011-04-26: Stuart Caie <kyzer@4u.net> 445 446 * configure.ac, mspack/system.h: Added a configure test to get 447 the size of off_t. If off_t is 8 bytes or more, we presume this 448 system has large file support. This fixes LFS detection for Fedora 449 x86_64 and Darwin/Mac OS X, neither of which declare FILESIZEBITS in 450 <limits.h>. It's not against the POSIX standard to do this: "A 451 definition of [FILESIZEBITS] shall be omitted from the <limits.h> 452 header on specific implementations where the corresponding value is 453 equal to or greater than the stated minimum, but where the value can 454 vary depending on the file to which it is applied." 455 (http://pubs.opengroup.org/onlinepubs/009695399/basedefs/limits.h.html) 456 Thanks to Edward Sheldrake for the patch. 457 4582011-04-26: Stuart Caie <kyzer@4u.net> 459 460 * chmd.c: all 64-bit integer reads are now consolidated into 461 the read_off64() function 462 463 * chmd_read_headers(): this function has been made resilient 464 against accessing memory past the end of a chunk. Thanks to 465 Sergei Trofimovich for sending me examples and analysis. 466 467 * chmd_init_decomp(): this function now reads the SpanInfo file 468 if the ResetTable file isn't available, it also checks that each 469 system file it needs is large enough before accessing it, and 470 some of its code has been split into several new functions: 471 find_sys_file(), read_reset_table() and read_spaninfo() 472 4732011-04-26: Stuart Caie <kyzer@4u.net> 474 475 * mspack.h, chmd.c: now reads the SpanInfo system file if the 476 ResetTable file isn't available. This adds a new spaninfo pointer 477 into struct mschmd_sec_mscompressed 478 4792011-04-26: Stuart Caie <kyzer@4u.net> 480 481 * test/chminfo.c: more sanity checks for corrupted CHM files where 482 entries go past the end of a PMGL/PMGI chunk, thanks to 483 Sergei Trofimovich for sending me examples and analysis. 484 4852011-04-25: Stuart Caie <kyzer@4u.net> 486 487 * cabd_merge(): Drew D'Addesio showed me spanning cabinets which 488 don't have all the CFFILE entries they should, but otherwise have 489 all necessary data for extraction. Changed the merging folders 490 test to be less strict; if folders don't exactly match, warn which 491 files are missing, but allow merging if at least one necessary 492 file is present. 493 4942010-09-24: Stuart Caie <kyzer@4u.net> 495 496 * readhuff.h: Don't let build_decode_table() allow empty trees. 497 It's meant to be special case just for the LZX length tree, so 498 move that logic out to the LZX code. Thanks to Danny Kroshin for 499 discovering the bug. 500 501 * lzxd.c: Allow empty length trees, but not other trees. If 502 the length tree is empty, fail if asked to decode a length symbol. 503 Again, thanks to Danny Kroshin for discovering the bug. 504 5052010-09-20: Stuart Caie <kyzer@4u.net> 506 507 * Makefile.am: Set EXTRA_DIST so it doesn't include .svn 508 directories in the distribution, but does include docs. 509 5102010-09-20: Stuart Caie <kyzer@4u.net> 511 512 * Makefile.am, configure.ac: Use modern auto* practises; turn on 513 automake silent rules where possible, use "m4" directory for libtool 514 macros, use LT_INIT instead of AC_PROG_LIBTOOL and use AM_CPPFLAGS 515 instead of INCLUDES. Thanks to Sergei Trofimovich for the patch. 516 5172010-09-15: Stuart Caie <kyzer@4u.net> 518 519 * many files: Made the code compile with C++ 520 - Renamed all 'this' variables/parameters to 'self' 521 - Added casts to all memory allocations. 522 - Added extern "C" to header files with extern declarations. 523 - Made system.c include system.h. 524 - Changed the K&R-style headers to ANSI-style headers in md5.c 525 5262010-08-04: Stuart Caie <kyzer@4u.net> 527 528 * many files: removed unnecessary <unistd.h> include 529 5302010-07-19: Stuart Caie <kyzer@4u.net> 531 532 * cabd_md5.c, chmd_md5.c: Replace writing files to disk then 533 MD5summing them, with an MD5summer built into mspack_system. 534 Much, much faster results. 535 536 * qtmd_decompress(): Robert Riebisch pointed out a Quantum 537 data integrity check that could never be tripped, because 538 frame_todo is unsigned, so it will never be decremented 539 below zero. Replaced the check with one that assumes that 540 decrementing past zero wraps frame_todo round to a number 541 more than its maximum value (QTM_FRAME_SIZE). 542 5432010-07-18: Stuart Caie <kyzer@4u.net> 544 545 * cabd.c: Special logic to pass cabd_sys_read() errors back 546 to cabd_extract() wasn't compatible with the decompressor 547 logic of returning the same error repeatedly once unpacking 548 fails. This meant that if decompressing failed because of 549 a read error, then the next file in the same folder would 550 come back as "no error", but the decompressed wouldn't have 551 even attempted to decompress the file. Added a new state 552 variable, read_error, with the same lifespan as a decompressor, 553 to pass the underlying reason for MSPACK_ERR_READ errors back. 554 555 * mszipd.c: improve MS-ZIP recovery by saving all the bytes 556 decoded prior to a block failing. This requires remembering 557 how far we got through the block, so the code has been made 558 slightly slower (about 0.003 seconds slower per gigabyte 559 unpacked) by removing the local variable window_posn 560 and keeping it in the state structure instead. 561 5622010-07-16: Stuart Caie <kyzer@4u.net> 563 564 * Makefile.am: strange interactions. When -std=c99 is used, 565 my Ubuntu's <stdio.h> (libc6-dev 2.11.1-0ubuntu7.2) does NOT 566 define fseeko() unless _LARGEFILE_SOURCE is also defined. But 567 configure always uses -std=gnu99, not -std=c99, so its test 568 determines _LARGEFILE_SOURCE isn't needed but HAVE_FSEEKO is 569 true. The implicit fseeko definition has a 32-bit rather than 570 64-bit offset, which means the mode parameter is interpreted 571 as part of the offset, and the mode is taken from the stack, 572 which is generally 0 (SEEK_SET). This breaks all SEEK_CURs. 573 The code works fine when -std=c99 is not set, so just remove 574 it for the time being. 575 5762010-07-12: Stuart Caie <kyzer@4u.net> 577 578 * system.c: Reject reading/writing a negative number of bytes. 579 580 * chmd.c: allow zero-length files to be seen. Previously they were 581 skipped because they were mistaken for directory entries. 582 5832010-07-08: Stuart Caie <kyzer@4u.net> 584 585 * qtmd.c: Larry Frieson found an important bug in the Quantum 586 decoder. Window wraps flush all unwritten data to disk. 587 However, sometimes less data is needed, which makes 588 out_bytes negative, which is then passed to write(). Some 589 write() implementations treat negative sizes it as a large 590 positive integer and segfault trying to write the buffer. 591 592 * Makefile.am, test/*.c: fixed automake file so that the 593 package passes a "make distcheck". 594 5952010-07-07: Stuart Caie <kyzer@4u.net> 596 597 * doc/szdd_kwaj_format.html: explain SZDD/KWAJ file format. 598 599 * lzssd.c: fixed SZDD decompression bugs. 600 601 * test/chmd_compare: Add scripts for comparing chmd_md5 against 602 Microsoft's own code. 603 604 * test/chmd_md5.c: remove the need to decompress everything 605 twice, as this is already in chmx.c if needed. 606 6072010-07-06: Stuart Caie <kyzer@4u.net> 608 609 * many files: added SZDD and KWAJ decompression support. 610 6112010-06-18: Stuart Caie <kyzer@4u.net> 612 613 * system.h: expanded the test for 64-bit largefile support so 614 it also works on 64-bit native operating systems where you 615 don't have to define _FILE_OFFSET_BITS. 616 6172010-06-17: Stuart Caie <kyzer@4u.net> 618 619 * libmspack.pc.in: Added pkg-config support. Thanks to 620 Patrice Dumas for the patch. 621 6222010-06-14: Stuart Caie <kyzer@4u.net> 623 624 * qtmd.c, lzxd.c, mszipd.c: created new headers, readbits.h and 625 readhuff.h, which bundle up the bit-reading and huffman-reading 626 code found in the MSZIP, LZX and Quantum decoders. 627 6282010-06-11: Stuart Caie <kyzer@4u.net> 629 630 * qtmd_static_init(): Removed function in favour of static const 631 tables, same rationale as for lzxd_static_init(). 632 633 * qtmd_read_input(), zipd_read_input(): After testing against my 634 set of CABs from the wild, I've found both these functions _need_ 635 an extra EOF flag, like lzxd_read_input() has. So I've added 636 it. This means CABs get decoded properly AND there's no reading 637 fictional bytes. 638 6392010-06-03: Stuart Caie <kyzer@4u.net> 640 641 * test/cabd_md5.c: updated this so it has better output and 642 doesn't need to be in the same directory as the files for multi- 643 part sets. 644 6452010-05-20: Stuart Caie <kyzer@4u.net> 646 647 * qtmd_read_input(), zipd_read_input(): Both these functions are 648 essentially copies of lzxd_read_input(), but that has a feature 649 they don't have - an extra EOF flag. So if EOF is 650 encountered (sys->read() returns 0 bytes), these don't pass on the 651 error. Their respective bit-reading functions that called them 652 then go on to access at least one byte of the input buffer, which 653 doesn't exist as sys->read() returned 0. Thanks to Michael 654 Vidrevich for spotting this and providing a test case. 655 6562010-05-20: Stuart Caie <kyzer@4u.net> 657 658 * system.h: It turns out no configure.ac tests are needed to 659 decide between __func__ and __FUNCTION__, so I put the standard 660 one (__func__) back into the D() macro, along with some 661 special-case ifdefs for old versions of GCC. 662 663 * lzxd_static_init(): Removed function in favour of static const 664 tables. Jorge Lodos thinks it causes multithreading problems, I 665 disagree. However, there are speed benefits to declaring the 666 tables as static const. 667 668 * cabd_init_decomp(): Fixed code which never runs but would write 669 to a null pointer if it could. Changed it to an assert() as it 670 will only trip if someone rewrites the internals of cabd.c. Thanks 671 to Jorge Lodos for finding it. 672 673 * inflate(): Fixed an off-by-one error: if the LITERAL table 674 emitted code 286, this would read one byte past the end of 675 lit_extrabits[]. Thanks to Jorge Lodos for finding it. 676 6772010-05-06: Stuart Caie <kyzer@4u.net> 678 679 * test/cabrip.c, test/chminfo.c: add fseeko() support 680 6812009-06-01: Stuart Caie <kyzer@4u.net> 682 683 * README: clarify the extended license terms 684 685 * doc, Makefile.am: make the doxygen makefile work when using 686 an alternate build directory 687 6882006-09-20: Stuart Caie <kyzer@4u.net> 689 690 * system.h: I had a choice of adding more to configure.ac to 691 test for __func__ and __FUNCTION__, or just removing __FUNCTION__ 692 from the D() macro. I chose the latter. 693 694 * Makefile.am: Now the --enable-debug in configure will actually 695 apply -DDEBUG to the sources. 696 6972006-09-20: Stuart Caie <kyzer@4u.net> 698 699 * qtmd_decompress(): Fixed a major bug in the QTM decoder, as 700 reported by Tomasz Kojm last year. Removed the restriction on 701 window sizes as a result. Correctly decodes the XLVIEW cabinets. 702 7032006-08-31: Stuart Caie <kyzer@4u.net> 704 705 * lzxd_decompress(): Two major bugs fixed. Firstly, the R0/R1/R2 706 local variables weren't set to 1 after lzxd_reset_state(). 707 Secondly, the LZX decompression stream can sometimes become 708 odd-aligned (after an uncompressed block) and the next 16 bit 709 fetch needs to be split across two input buffers, ENSURE_BITS() 710 didn't cover this case. Many thanks to Igor Glucksmann for 711 discovering both these bugs. 712 7132005-06-30: Stuart Caie <kyzer@4u.net> 714 715 * cabd_search(): fixed problems with searching files > 4GB for 716 cabinets. 717 7182005-06-23: Stuart Caie <kyzer@4u.net> 719 720 * qtmd_init(): The QTM decoder is broken for QTM streams with a 721 window size less than the frame size. Until this is fixed, fail 722 to initialise QTM window sizes less than 15. Thanks to Tomasz Kojm 723 for finding the bug. 724 7252005-03-22: Stuart Caie <kyzer@4u.net> 726 727 * system.h: now undefs "read", as the latest glibc defines read() 728 as a macro which messes everything up. Thanks to Ville Skyttä for 729 the update. 730 7312005-03-14: Stuart Caie <kyzer@4u.net> 732 733 * test/multifh.c: write an mspack_system implementation that can 734 handle normal disk files, open file handles, open file descriptors 735 and raw memory all at the same time. 736 7372005-02-24: Stuart Caie <kyzer@4u.net> 738 739 * chmd_read_headers(): avoid infinite loop when chmhs1_ChunkSize is 740 zero. Thanks to Serge Semashko for the research and discovery. 741 7422005-02-18: Stuart Caie <kyzer@4u.net> 743 744 * mspack.h: renamed the "interface" parameter of mspack_version() to 745 "entity", as interface is a reserved word in C++. Thanks to Yuriy Z 746 for the discovery. 747 7482004-12-09: Stuart Caie <kyzer@4u.net> 749 750 * lzss.h, szdd.h, szddd.h: more work on the SZDD/LZSS design. 751 7522004-06-12: Stuart Caie <kyzer@4u.net> 753 754 * lzxd_static_init(): removed write to lzxd_extra_bits[52], thanks 755 to Nigel Horne from the ClamAV project. 756 7572004-04-23: Stuart Caie <kyzer@4u.net> 758 759 * mspack.h: changed 'this' parameters to 'self' to allow compiling in 760 C++ compilers, thanks to Michal Cihar for the suggestion. 761 762 * mspack.h, system.h, mspack.def, winbuild.sh: integrated some changes 763 from Petr Blahos to let libmspack build as a Win32 DLL. 764 765 * chmd_fast_find(): added the first part of this code, and comments 766 sufficient to finish it :) 767 7682004-04-08 Stuart Caie <kyzer@4u.net> 769 770 * test/chminfo.c: added a program for dumping useful data from CHM 771 files, e.g. index entries and reset tables. I wrote this a while ago 772 for investigating a corrupt cabinet, but I never committed it. 773 7742004-03-26 Stuart Caie <kyzer@4u.net> 775 776 * test/cabd_memory.c: added a new test example which shows an 777 mspack_system implementation that reads and writes from memory only, 778 no file I/O. Even the source code has a little cab file embedded in it. 779 7802004-03-10 Stuart Caie <kyzer@4u.net> 781 782 * cabd.c: updated the location of the CAB SDK. 783 784 * cabd.c: changed a couple of MSPACK_ERR_READ errors not based on 785 read() failures into MSPACK_ERR_DATAFORMAT errors. 786 787 * mszipd_decompress(): repair mode now aborts after writing a 788 repaired block if the error was a hard error (e.g. read error, out 789 of blocks, etc) 790 7912004-03-08 Stuart Caie <kyzer@4u.net> 792 793 * Makefile.am: now builds and installs a versioned library. 794 795 * mszipd.c: completed a new MS-ZIP and inflate implementation. 796 797 * system.c: added mspack_version() and committed to a versioned 798 ABI for the library. 799 800 * cabd.c: made mszip repair functionality work correctly. 801 802 * cabd.c: now identifies invalid block headers 803 804 * doc/: API documentation is now included with the library, not 805 just on the web. 806 807 * chmd.c: fixed error messages and 64-bit debug output. 808 809 * chmd.c: now also catches NULL files in section 1. 810 811 * test/chmx.c: now acts more like cabextract. 812 8132003-08-29 Stuart Caie <kyzer@4u.net> 814 815 * ChangeLog: started keeping a ChangeLog :) 816