1## zlib-ng 2*zlib data compression library for the next generation systems* 3 4Maintained by Hans Kristian Rosbach 5 aka Dead2 (zlib-ng àt circlestorm dót org) 6 7|CI|Status| 8|:-|-| 9|GitHub Actions|[![Master Branch Status](https://github.com/zlib-ng/zlib-ng/workflows/CI%20CMake/badge.svg)](https://github.com/zlib-ng/zlib-ng/actions) [![Master Branch Status](https://github.com/zlib-ng/zlib-ng/workflows/CI%20Configure/badge.svg)](https://github.com/zlib-ng/zlib-ng/actions) [![Master Branch Status](https://github.com/zlib-ng/zlib-ng/workflows/CI%20NMake/badge.svg)](https://github.com/zlib-ng/zlib-ng/actions)| 10|Buildkite|[![Build status](https://badge.buildkite.com/7bb1ef84356d3baee26202706cc053ee1de871c0c712b65d26.svg?branch=develop)](https://buildkite.com/circlestorm-productions/zlib-ng)| 11|CodeFactor|[![CodeFactor](https://www.codefactor.io/repository/github/zlib-ng/zlib-ng/badge)](https://www.codefactor.io/repository/github/zlib-ng/zlib-ng)| 12|OSS-Fuzz|[![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/zlib-ng.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:zlib-ng) 13|Codecov|[![codecov.io](https://codecov.io/github/zlib-ng/zlib-ng/coverage.svg?branch=develop)](https://codecov.io/github/zlib-ng/zlib-ng/)| 14 15 16Features 17-------- 18 19* Zlib compatible API with support for dual-linking 20* Modernized native API based on zlib API for ease of porting 21* Modern C99 syntax and a clean code layout 22* Deflate medium and quick algorithms based on Intels zlib fork 23* Support for CPU intrinsics when available 24 * Adler32 implementation using SSSE3, AVX2, Neon & VSX 25 * CRC32-B implementation using PCLMULQDQ & ACLE 26 * Hash table implementation using CRC32-C intrinsics on x86 and ARM 27 * Slide hash implementations using SSE2, AVX2, Neon & VSX 28 * Compare256/258 implementations using SSE4.2 & AVX2 29 * Inflate chunk copying using SSE2, AVX2 & Neon 30 * Support for hardware-accelerated deflate using IBM Z DFLTCC 31* Unaligned memory read/writes and large bit buffer improvements 32* Includes improvements from Cloudflare and Intel forks 33* Configure, CMake, and NMake build system support 34* Comprehensive set of CMake unit tests 35* Code sanitizers, fuzzing, and coverage 36* GitHub Actions continuous integration on Windows, macOS, and Linux 37 * Emulated CI for ARM, AARCH64, PPC, PPC64, SPARC64, S390x using qemu 38 39 40History 41------- 42 43The motivation for this fork came after seeing several 3rd party 44contributions containing new optimizations not getting implemented 45into the official zlib repository. 46 47Mark Adler has been maintaining zlib for a very long time, and he has 48done a great job and hopefully he will continue for a long time yet. 49The idea of zlib-ng is not to replace zlib, but to co-exist as a 50drop-in replacement with a lower threshold for code change. 51 52zlib has a long history and is incredibly portable, even supporting 53lots of systems that predate the Internet. This is great, but it does 54complicate further development and maintainability. 55The zlib code has numerous workarounds for old compilers that do not 56understand ANSI-C or to accommodate systems with limitations such as 57operating in a 16-bit environment. 58 59Many of these workarounds are only maintenance burdens, some of them 60are pretty huge code-wise. For example, the [v]s[n]printf workaround 61code has a whopping 8 different implementations just to cater to 62various old compilers. With this many workarounds cluttered throughout 63the code, new programmers with an idea/interest for zlib will need 64to take some time to figure out why all of these seemingly strange 65things are used, and how to work within those confines. 66 67So I decided to make a fork, merge all the Intel optimizations, merge 68the Cloudflare optimizations that did not conflict, plus a couple 69of other smaller patches. Then I started cleaning out workarounds, 70various dead code, all contrib and example code as there is little 71point in having those in this fork for various reasons. 72 73A lot of improvements have gone into zlib-ng since its start, and 74numerous people and companies have contributed both small and big 75improvements, or valuable testing. 76 77Please read LICENSE.md, it is very simple and very liberal. 78 79 80Build 81----- 82 83There are two ways to build zlib-ng: 84 85### Cmake 86 87To build zlib-ng using the cross-platform makefile generator cmake. 88 89``` 90cmake . 91cmake --build . --config Release 92ctest --verbose -C Release 93``` 94 95Alternatively, you can use the cmake configuration GUI tool ccmake: 96 97``` 98ccmake . 99``` 100 101### Configure 102 103To build zlib-ng using the bash configure script: 104 105``` 106./configure 107make 108make test 109``` 110 111Build Options 112------------- 113 114| CMake | configure | Description | Default | 115|:-------------------------|:-------------------------|:--------------------------------------------------------------------------------------|---------| 116| ZLIB_COMPAT | --zlib-compat | Compile with zlib compatible API | OFF | 117| ZLIB_ENABLE_TESTS | | Build test binaries | ON | 118| WITH_GZFILEOP | --without-gzfileops | Compile with support for gzFile related functions | ON | 119| WITH_OPTIM | --without-optimizations | Build with optimisations | ON | 120| WITH_NEW_STRATEGIES | --without-new-strategies | Use new strategies | ON | 121| WITH_NATIVE_INSTRUCTIONS | --native | Compiles with full instruction set supported on this host (gcc/clang -march=native) | OFF | 122| WITH_SANITIZER | --with-sanitizer | Build with sanitizer (memory, address, undefined) | OFF | 123| WITH_FUZZERS | --with-fuzzers | Build test/fuzz | OFF | 124| WITH_MAINTAINER_WARNINGS | | Build with project maintainer warnings | OFF | 125| WITH_CODE_COVERAGE | | Enable code coverage reporting | OFF | 126 127 128Install 129------- 130 131WARNING: We do not recommend manually installing unless you really 132know what you are doing, because this can potentially override the system 133default zlib library, and any incompatibility or wrong configuration of 134zlib-ng can make the whole system unusable, requiring recovery or reinstall. 135If you still want a manual install, we recommend using the /opt/ path prefix. 136 137For Linux distros, an alternative way to use zlib-ng (if compiled in 138zlib-compat mode) instead of zlib, is through the use of the 139_LD_PRELOAD_ environment variable. If the program is dynamically linked 140with zlib, then zlib-ng will temporarily be used instead by the program, 141without risking system-wide instability. 142 143``` 144LD_PRELOAD=/opt/zlib-ng/libz.so.1.2.11.zlib-ng /usr/bin/program 145``` 146 147### Cmake 148 149To install zlib-ng system-wide using cmake: 150 151``` 152cmake --build . --target install 153``` 154 155### Configure 156 157To install zlib-ng system-wide using the configure script: 158 159``` 160make install 161``` 162 163Contributing 164------------ 165 166Zlib-ng is a aiming to be open to contributions, and we would be delighted to 167receive pull requests on github. 168Just remember that any code you submit must be your own and it must be zlib licensed. 169Help with testing and reviewing of pull requests etc is also very much appreciated. 170 171If you are interested in contributing, please consider joining our 172IRC channel #zlib-ng on the Freenode IRC network. 173 174 175Acknowledgments 176---------------- 177 178Thanks to Servebolt.com for sponsoring my maintainership of zlib-ng. 179 180Thanks go out to all the people and companies who have taken the time to contribute 181code reviews, testing and/or patches. Zlib-ng would not have been nearly as good without you. 182 183The deflate format used by zlib was defined by Phil Katz. 184The deflate and zlib specifications were written by L. Peter Deutsch. 185 186zlib was originally created by Jean-loup Gailly (compression) 187and Mark Adler (decompression). 188 189 190Advanced Build Options 191---------------------- 192 193| CMake | configure | Description | Default | 194|:--------------------------------|:----------------------|:--------------------------------------------------------------------|------------------------| 195| ZLIB_DUAL_LINK | | Dual link tests with system zlib | OFF | 196| UNALIGNED_OK | | Allow unaligned reads | ON (x86, arm) | 197| | --force-sse2 | Skip runtime check for SSE2 instructions (Always on for x86_64) | OFF (x86) | 198| WITH_AVX2 | | Build with AVX2 intrinsics | ON | 199| WITH_SSE2 | | Build with SSE2 intrinsics | ON | 200| WITH_SSE4 | | Build with SSE4 intrinsics | ON | 201| WITH_PCLMULQDQ | | Build with PCLMULQDQ intrinsics | ON | 202| WITH_ACLE | --without-acle | Build with ACLE intrinsics | ON | 203| WITH_NEON | --without-neon | Build with NEON intrinsics | ON | 204| WITH_POWER8 | | Build with POWER8 optimisations | ON | 205| WITH_DFLTCC_DEFLATE | --with-dfltcc-deflate | Build with DFLTCC intrinsics for compression on IBM Z | OFF | 206| WITH_DFLTCC_INFLATE | --with-dfltcc-inflate | Build with DFLTCC intrinsics for decompression on IBM Z | OFF | 207| WITH_UNALIGNED | | Allow optimizations that use unaligned reads if safe on current arch| ON | 208| WITH_INFLATE_STRICT | | Build with strict inflate distance checking | OFF | 209| WITH_INFLATE_ALLOW_INVALID_DIST | | Build with zero fill for inflate invalid distances | OFF | 210| INSTALL_UTILS | | Copy minigzip and minideflate during install | OFF | 211 212 213Related Projects 214---------------- 215 216* Fork of the popular minigzip https://github.com/zlib-ng/minizip-ng 217* Python tool to benchmark minigzip/minideflate https://github.com/zlib-ng/deflatebench 218* Python tool to benchmark pigz https://github.com/zlib-ng/pigzbench 219* 3rd party patches for zlib-ng compatibility https://github.com/zlib-ng/patches 220 221