1 2XZ Utils Release Notes 3====================== 4 55.2.4 (2018-04-29) 6 7 * liblzma: 8 9 - Allow 0 as memory usage limit instead of returning 10 LZMA_PROG_ERROR. Now 0 is treated as if 1 byte was specified, 11 which effectively is the same as 0. 12 13 - Use "noexcept" keyword instead of "throw()" in the public 14 headers when a C++11 (or newer standard) compiler is used. 15 16 - Added a portability fix for recent Intel C Compilers. 17 18 - Microsoft Visual Studio build files have been moved under 19 windows/vs2013 and windows/vs2017. 20 21 * xz: 22 23 - Fix "xz --list --robot missing_or_bad_file.xz" which would 24 try to print an unitialized string and thus produce garbage 25 output. Since the exit status is non-zero, most uses of such 26 a command won't try to interpret the garbage output. 27 28 - "xz --list foo.xz" could print "Internal error (bug)" in a 29 corner case where a specific memory usage limit had been set. 30 31 325.2.3 (2016-12-30) 33 34 * xz: 35 36 - Always close a file before trying to delete it to avoid 37 problems on some operating system and file system combinations. 38 39 - Fixed copying of file timestamps on Windows. 40 41 - Added experimental (disabled by default) sandbox support using 42 Capsicum (FreeBSD >= 10). See --enable-sandbox in INSTALL. 43 44 * C99/C11 conformance fixes to liblzma. The issues affected at least 45 some builds using link-time optimizations. 46 47 * Fixed bugs in the rarely-used function lzma_index_dup(). 48 49 * Use of external SHA-256 code is now disabled by default. 50 It can still be enabled by passing --enable-external-sha256 51 to configure. The reasons to disable it by default (see INSTALL 52 for more details): 53 54 - Some OS-specific SHA-256 implementations conflict with 55 OpenSSL and cause problems in programs that link against both 56 liblzma and libcrypto. At least FreeBSD 10 and MINIX 3.3.0 57 are affected. 58 59 - The internal SHA-256 is faster than the SHA-256 code in 60 some operating systems. 61 62 * Changed CPU core count detection to use sched_getaffinity() on 63 GNU/Linux and GNU/kFreeBSD. 64 65 * Fixes to the build-system and xz to make xz buildable even when 66 encoders, decoders, or threading have been disabled from libilzma 67 using configure options. These fixes added two new #defines to 68 config.h: HAVE_ENCODERS and HAVE_DECODERS. 69 70 715.2.2 (2015-09-29) 72 73 * Fixed bugs in QNX-specific code. 74 75 * Omitted the use of pipe2() even if it is available to avoid 76 portability issues with some old Linux and glibc combinations. 77 78 * Updated German translation. 79 80 * Added project files to build static and shared liblzma (not the 81 whole XZ Utils) with Visual Studio 2013 update 2 or later. 82 83 * Documented that threaded decompression hasn't been implemented 84 yet. A 5.2.0 NEWS entry describing multi-threading support had 85 incorrectly said "decompression" when it should have said 86 "compression". 87 88 895.2.1 (2015-02-26) 90 91 * Fixed a compression-ratio regression in fast mode of LZMA1 and 92 LZMA2. The bug is present in 5.1.4beta and 5.2.0 releases. 93 94 * Fixed a portability problem in xz that affected at least OpenBSD. 95 96 * Fixed xzdiff to be compatible with FreeBSD's mktemp which differs 97 from most other mktemp implementations. 98 99 * Changed CPU core count detection to use cpuset_getaffinity() on 100 FreeBSD. 101 102 1035.2.0 (2014-12-21) 104 105 Since 5.1.4beta: 106 107 * All fixes from 5.0.8 108 109 * liblzma: Fixed lzma_stream_encoder_mt_memusage() when a preset 110 was used. 111 112 * xzdiff: If mktemp isn't installed, mkdir will be used as 113 a fallback to create a temporary directory. Installing mktemp 114 is still recommended. 115 116 * Updated French, German, Italian, Polish, and Vietnamese 117 translations. 118 119 Summary of fixes and new features added in the 5.1.x development 120 releases: 121 122 * liblzma: 123 124 - Added support for multi-threaded compression. See the 125 lzma_mt structure, lzma_stream_encoder_mt(), and 126 lzma_stream_encoder_mt_memusage() in <lzma/container.h>, 127 lzma_get_progress() in <lzma/base.h>, and lzma_cputhreads() 128 in <lzma/hardware.h> for details. 129 130 - Made the uses of lzma_allocator const correct. 131 132 - Added lzma_block_uncomp_encode() to create uncompressed 133 .xz Blocks using LZMA2 uncompressed chunks. 134 135 - Added support for LZMA_IGNORE_CHECK. 136 137 - A few speed optimizations were made. 138 139 - Added support for symbol versioning. It is enabled by default 140 on GNU/Linux, other GNU-based systems, and FreeBSD. 141 142 - liblzma (not the whole XZ Utils) should now be buildable 143 with MSVC 2013 update 2 or later using windows/config.h. 144 145 * xz: 146 147 - Fixed a race condition in the signal handling. It was 148 possible that e.g. the first SIGINT didn't make xz exit 149 if reading or writing blocked and one had bad luck. The fix 150 is non-trivial, so as of writing it is unknown if it will be 151 backported to the v5.0 branch. 152 153 - Multi-threaded compression can be enabled with the 154 --threads (-T) option. 155 [Fixed: This originally said "decompression".] 156 157 - New command line options in xz: --single-stream, 158 --block-size=SIZE, --block-list=SIZES, 159 --flush-timeout=TIMEOUT, and --ignore-check. 160 161 - xz -lvv now shows the minimum xz version that is required to 162 decompress the file. Currently it is 5.0.0 for all supported 163 .xz files except files with empty LZMA2 streams require 5.0.2. 164 165 * xzdiff and xzgrep now support .lzo files if lzop is installed. 166 The .tzo suffix is also recognized as a shorthand for .tar.lzo. 167 168 1695.1.4beta (2014-09-14) 170 171 * All fixes from 5.0.6 172 173 * liblzma: Fixed the use of presets in threaded encoder 174 initialization. 175 176 * xz --block-list and --block-size can now be used together 177 in single-threaded mode. Previously the combination only 178 worked in multi-threaded mode. 179 180 * Added support for LZMA_IGNORE_CHECK to liblzma and made it 181 available in xz as --ignore-check. 182 183 * liblzma speed optimizations: 184 185 - Initialization of a new LZMA1 or LZMA2 encoder has been 186 optimized. (The speed of reinitializing an already-allocated 187 encoder isn't affected.) This helps when compressing many 188 small buffers with lzma_stream_buffer_encode() and other 189 similar situations where an already-allocated encoder state 190 isn't reused. This speed-up is visible in xz too if one 191 compresses many small files one at a time instead running xz 192 once and giving all files as command-line arguments. 193 194 - Buffer comparisons are now much faster when unaligned access 195 is allowed (configured with --enable-unaligned-access). This 196 speeds up encoding significantly. There is arch-specific code 197 for 32-bit and 64-bit x86 (32-bit needs SSE2 for the best 198 results and there's no run-time CPU detection for now). 199 For other archs there is only generic code which probably 200 isn't as optimal as arch-specific solutions could be. 201 202 - A few speed optimizations were made to the SHA-256 code. 203 (Note that the builtin SHA-256 code isn't used on all 204 operating systems.) 205 206 * liblzma can now be built with MSVC 2013 update 2 or later 207 using windows/config.h. 208 209 * Vietnamese translation was added. 210 211 2125.1.3alpha (2013-10-26) 213 214 * All fixes from 5.0.5 215 216 * liblzma: 217 218 - Fixed a deadlock in the threaded encoder. 219 220 - Made the uses of lzma_allocator const correct. 221 222 - Added lzma_block_uncomp_encode() to create uncompressed 223 .xz Blocks using LZMA2 uncompressed chunks. 224 225 - Added support for native threads on Windows and the ability 226 to detect the number of CPU cores. 227 228 * xz: 229 230 - Fixed a race condition in the signal handling. It was 231 possible that e.g. the first SIGINT didn't make xz exit 232 if reading or writing blocked and one had bad luck. The fix 233 is non-trivial, so as of writing it is unknown if it will be 234 backported to the v5.0 branch. 235 236 - Made the progress indicator work correctly in threaded mode. 237 238 - Threaded encoder now works together with --block-list=SIZES. 239 240 - Added preliminary support for --flush-timeout=TIMEOUT. 241 It can be useful for (somewhat) real-time streaming. For 242 now the decompression side has to be done with something 243 else than the xz tool due to how xz does buffering, but this 244 should be fixed. 245 246 2475.1.2alpha (2012-07-04) 248 249 * All fixes from 5.0.3 and 5.0.4 250 251 * liblzma: 252 253 - Fixed a deadlock and an invalid free() in the threaded encoder. 254 255 - Added support for symbol versioning. It is enabled by default 256 on GNU/Linux, other GNU-based systems, and FreeBSD. 257 258 - Use SHA-256 implementation from the operating system if one is 259 available in libc, libmd, or libutil. liblzma won't use e.g. 260 OpenSSL or libgcrypt to avoid introducing new dependencies. 261 262 - Fixed liblzma.pc for static linking. 263 264 - Fixed a few portability bugs. 265 266 * xz --decompress --single-stream now fixes the input position after 267 successful decompression. Now the following works: 268 269 echo foo | xz > foo.xz 270 echo bar | xz >> foo.xz 271 ( xz -dc --single-stream ; xz -dc --single-stream ) < foo.xz 272 273 Note that it doesn't work if the input is not seekable 274 or if there is Stream Padding between the concatenated 275 .xz Streams. 276 277 * xz -lvv now shows the minimum xz version that is required to 278 decompress the file. Currently it is 5.0.0 for all supported .xz 279 files except files with empty LZMA2 streams require 5.0.2. 280 281 * Added an *incomplete* implementation of --block-list=SIZES to xz. 282 It only works correctly in single-threaded mode and when 283 --block-size isn't used at the same time. --block-list allows 284 specifying the sizes of Blocks which can be useful e.g. when 285 creating files for random-access reading. 286 287 2885.1.1alpha (2011-04-12) 289 290 * All fixes from 5.0.2 291 292 * liblzma fixes that will also be included in 5.0.3: 293 294 - A memory leak was fixed. 295 296 - lzma_stream_buffer_encode() no longer creates an empty .xz 297 Block if encoding an empty buffer. Such an empty Block with 298 LZMA2 data would trigger a bug in 5.0.1 and older (see the 299 first bullet point in 5.0.2 notes). When releasing 5.0.2, 300 I thought that no encoder creates this kind of files but 301 I was wrong. 302 303 - Validate function arguments better in a few functions. Most 304 importantly, specifying an unsupported integrity check to 305 lzma_stream_buffer_encode() no longer creates a corrupt .xz 306 file. Probably no application tries to do that, so this 307 shouldn't be a big problem in practice. 308 309 - Document that lzma_block_buffer_encode(), 310 lzma_easy_buffer_encode(), lzma_stream_encoder(), and 311 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK. 312 313 - The return values of the _memusage() functions are now 314 documented better. 315 316 * Support for multithreaded compression was added using the simplest 317 method, which splits the input data into blocks and compresses 318 them independently. Other methods will be added in the future. 319 The current method has room for improvement, e.g. it is possible 320 to reduce the memory usage. 321 322 * Added the options --single-stream and --block-size=SIZE to xz. 323 324 * xzdiff and xzgrep now support .lzo files if lzop is installed. 325 The .tzo suffix is also recognized as a shorthand for .tar.lzo. 326 327 * Support for short 8.3 filenames under DOS was added to xz. It is 328 experimental and may change before it gets into a stable release. 329 330 3315.0.8 (2014-12-21) 332 333 * Fixed an old bug in xzgrep that affected OpenBSD and probably 334 a few other operating systems too. 335 336 * Updated French and German translations. 337 338 * Added support for detecting the amount of RAM on AmigaOS/AROS. 339 340 * Minor build system updates. 341 342 3435.0.7 (2014-09-20) 344 345 * Fix regressions introduced in 5.0.6: 346 347 - Fix building with non-GNU make. 348 349 - Fix invalid Libs.private value in liblzma.pc which broke 350 static linking against liblzma if the linker flags were 351 taken from pkg-config. 352 353 3545.0.6 (2014-09-14) 355 356 * xzgrep now exits with status 0 if at least one file matched. 357 358 * A few minor portability and build system fixes 359 360 3615.0.5 (2013-06-30) 362 363 * lzmadec and liblzma's lzma_alone_decoder(): Support decompressing 364 .lzma files that have less common settings in the headers 365 (dictionary size other than 2^n or 2^n + 2^(n-1), or uncompressed 366 size greater than 256 GiB). The limitations existed to avoid false 367 positives when detecting .lzma files. The lc + lp <= 4 limitation 368 still remains since liblzma's LZMA decoder has that limitation. 369 370 NOTE: xz's .lzma support or liblzma's lzma_auto_decoder() are NOT 371 affected by this change. They still consider uncommon .lzma headers 372 as not being in the .lzma format. Changing this would give way too 373 many false positives. 374 375 * xz: 376 377 - Interaction of preset and custom filter chain options was 378 made less illogical. This affects only certain less typical 379 uses cases so few people are expected to notice this change. 380 381 Now when a custom filter chain option (e.g. --lzma2) is 382 specified, all preset options (-0 ... -9, -e) earlier are on 383 the command line are completely forgotten. Similarly, when 384 a preset option is specified, all custom filter chain options 385 earlier on the command line are completely forgotten. 386 387 Example 1: "xz -9 --lzma2=preset=5 -e" is equivalent to "xz -e" 388 which is equivalent to "xz -6e". Earlier -e didn't put xz back 389 into preset mode and thus the example command was equivalent 390 to "xz --lzma2=preset=5". 391 392 Example 2: "xz -9e --lzma2=preset=5 -7" is equivalent to 393 "xz -7". Earlier a custom filter chain option didn't make 394 xz forget the -e option so the example was equivalent to 395 "xz -7e". 396 397 - Fixes and improvements to error handling. 398 399 - Various fixes to the man page. 400 401 * xzless: Fixed to work with "less" versions 448 and later. 402 403 * xzgrep: Made -h an alias for --no-filename. 404 405 * Include the previously missing debug/translation.bash which can 406 be useful for translators. 407 408 * Include a build script for Mac OS X. This has been in the Git 409 repository since 2010 but due to a mistake in Makefile.am the 410 script hasn't been included in a release tarball before. 411 412 4135.0.4 (2012-06-22) 414 415 * liblzma: 416 417 - Fix lzma_index_init(). It could crash if memory allocation 418 failed. 419 420 - Fix the possibility of an incorrect LZMA_BUF_ERROR when a BCJ 421 filter is used and the application only provides exactly as 422 much output space as is the uncompressed size of the file. 423 424 - Fix a bug in doc/examples_old/xz_pipe_decompress.c. It didn't 425 check if the last call to lzma_code() really returned 426 LZMA_STREAM_END, which made the program think that truncated 427 files are valid. 428 429 - New example programs in doc/examples (old programs are now in 430 doc/examples_old). These have more comments and more detailed 431 error handling. 432 433 * Fix "xz -lvv foo.xz". It could crash on some corrupted files. 434 435 * Fix output of "xz --robot -lv" and "xz --robot -lvv" which 436 incorrectly printed the filename also in the "foo (x/x)" format. 437 438 * Fix exit status of "xzdiff foo.xz bar.xz". 439 440 * Fix exit status of "xzgrep foo binary_file". 441 442 * Fix portability to EBCDIC systems. 443 444 * Fix a configure issue on AIX with the XL C compiler. See INSTALL 445 for details. 446 447 * Update French, German, Italian, and Polish translations. 448 449 4505.0.3 (2011-05-21) 451 452 * liblzma fixes: 453 454 - A memory leak was fixed. 455 456 - lzma_stream_buffer_encode() no longer creates an empty .xz 457 Block if encoding an empty buffer. Such an empty Block with 458 LZMA2 data would trigger a bug in 5.0.1 and older (see the 459 first bullet point in 5.0.2 notes). When releasing 5.0.2, 460 I thought that no encoder creates this kind of files but 461 I was wrong. 462 463 - Validate function arguments better in a few functions. Most 464 importantly, specifying an unsupported integrity check to 465 lzma_stream_buffer_encode() no longer creates a corrupt .xz 466 file. Probably no application tries to do that, so this 467 shouldn't be a big problem in practice. 468 469 - Document that lzma_block_buffer_encode(), 470 lzma_easy_buffer_encode(), lzma_stream_encoder(), and 471 lzma_stream_buffer_encode() may return LZMA_UNSUPPORTED_CHECK. 472 473 - The return values of the _memusage() functions are now 474 documented better. 475 476 * Fix command name detection in xzgrep. xzegrep and xzfgrep now 477 correctly use egrep and fgrep instead of grep. 478 479 * French translation was added. 480 481 4825.0.2 (2011-04-01) 483 484 * LZMA2 decompressor now correctly accepts LZMA2 streams with no 485 uncompressed data. Previously it considered them corrupt. The 486 bug can affect applications that use raw LZMA2 streams. It is 487 very unlikely to affect .xz files because no compressor creates 488 .xz files with empty LZMA2 streams. (Empty .xz files are a 489 different thing than empty LZMA2 streams.) 490 491 * "xz --suffix=.foo filename.foo" now refuses to compress the 492 file due to it already having the suffix .foo. It was already 493 documented on the man page, but the code lacked the test. 494 495 * "xzgrep -l foo bar.xz" works now. 496 497 * Polish translation was added. 498 499 5005.0.1 (2011-01-29) 501 502 * xz --force now (de)compresses files that have setuid, setgid, 503 or sticky bit set and files that have multiple hard links. 504 The man page had it documented this way already, but the code 505 had a bug. 506 507 * gzip and bzip2 support in xzdiff was fixed. 508 509 * Portability fixes 510 511 * Minor fix to Czech translation 512 513 5145.0.0 (2010-10-23) 515 516 Only the most important changes compared to 4.999.9beta are listed 517 here. One change is especially important: 518 519 * The memory usage limit is now disabled by default. Some scripts 520 written before this change may have used --memory=max on xz command 521 line or in XZ_OPT. THESE USES OF --memory=max SHOULD BE REMOVED 522 NOW, because they interfere with user's ability to set the memory 523 usage limit himself. If user-specified limit causes problems to 524 your script, blame the user. 525 526 Other significant changes: 527 528 * Added support for XZ_DEFAULTS environment variable. This variable 529 allows users to set default options for xz, e.g. default memory 530 usage limit or default compression level. Scripts that use xz 531 must never set or unset XZ_DEFAULTS. Scripts should use XZ_OPT 532 instead if they need a way to pass options to xz via an 533 environment variable. 534 535 * The compression settings associated with the preset levels 536 -0 ... -9 have been changed. --extreme was changed a little too. 537 It is now less likely to make compression worse, but with some 538 files the new --extreme may compress slightly worse than the old 539 --extreme. 540 541 * If a preset level (-0 ... -9) is specified after a custom filter 542 chain options have been used (e.g. --lzma2), the custom filter 543 chain will be forgotten. Earlier the preset options were 544 completely ignored after custom filter chain options had been 545 seen. 546 547 * xz will create sparse files when decompressing if the uncompressed 548 data contains long sequences of binary zeros. This is done even 549 when writing to standard output that is connected to a regular 550 file and certain additional conditions are met to make it safe. 551 552 * Support for "xz --list" was added. Combine with --verbose or 553 --verbose --verbose (-vv) for detailed output. 554 555 * I had hoped that liblzma API would have been stable after 556 4.999.9beta, but there have been a couple of changes in the 557 advanced features, which don't affect most applications: 558 559 - Index handling code was revised. If you were using the old 560 API, you will get a compiler error (so it's easy to notice). 561 562 - A subtle but important change was made to the Block handling 563 API. lzma_block.version has to be initialized even for 564 lzma_block_header_decode(). Code that doesn't do it will work 565 for now, but might break in the future, which makes this API 566 change easy to miss. 567 568 * The major soname has been bumped to 5.0.0. liblzma API and ABI 569 are now stable, so the need to recompile programs linking against 570 liblzma shouldn't arise soon. 571 572