README.md
1# snappy-erlang-nif
2
3This is an Erlang NIF for Google's snappy fast compressor and decompressor.
4
5Google snappy's official repository and bug tracking system is at:
6
7* http://code.google.com/p/snappy/
8
9Its source is included in this project.
10
11# changelog
12
13- tag `1.1.2` as tests pass against OTP-24
14- tag `otp-24` compatibility
15- switch build chain to [rebar3] and [enc] requiring OTP20 or newer
16- tag `snappy-1.1.8`, also first version compatible with OTP23 (thanks @skaes)
17- tag `snappy-1.1.3`
18- add a changelog
19
20[rebar3]: https://rebar3.org/
21[enc]: https://github.com/davisp/erlang-native-compiler
22
23# site
24
25https://github.com/skunkwerks/snappy-erlang-nif
26
27# credits
28
29Software is built by a few people and maintained by many. Thank-you for
30all your patches!
31
32
33- Arne Ehrlich <Arne.Ehrlich@groknet.de>
34- Bryan Chan <bryanpkc@gmail.com>
35- Dave Cottlehuber <dch@skunkwerks.at>
36- Filipe David Manana <fdmanana@apache.org>
37- Joshua Scott <joshua.scott@gmail.com>
38- Kelly McLaughlin <kelly@basho.com>
39- Louis-Philippe Gauthier <lpgauth@gmail.com>
40- Mikhail Uvarov <arcusfelis@gmail.com>
41- Peter Membrey <peter@membrey.hk>
42- Piotr Nosek <piotr.nosek@erlang-solutions.com>
43- benoitc <bchesneau@gmail.com>
44- @dch314
45- glejeune <gregoire.lejeune@free.fr>
46- Stefan Kaes <stkaes@gmail.com>
47
48# performance tests
49
50Snappy is much faster than zlib's deflate compression, specially for reasonably large amounts of
51data. Here follow a few basic tests.
52
53<pre>
54Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]
55
56Eshell V5.8.3 (abort with ^G)
571> code:add_path("ebin").
58true
592> {ok, J} = file:read_file("../seatoncouch/doc_11k.json").
60{ok,<<"{\n\"data3\":\"ColreUHAtn8iYvHIHw2ohiAuaXNGNX9WhlwANGDBYKYs6YEvHXgFXRkWuYoFyfLVqtjZNPedtukWRDXFz7VUOl4APLS7GB9Nw"...>>}
613>
623> timer:tc(snappy, compress, [J]).
63{4365,
64 {ok,<<"úZðÔ{\n\"data3\":\"ColreUHAtn8iYvHIHw2ohiAuaXNGNX9WhlwANGDBYKYs6YEvHXgFXRkWuYoFyfLVqtjZNPedtukWRDXFz7VUO"...>>}}
654> timer:tc(snappy, compress, [J]).
66{115,
67 {ok,<<"úZðÔ{\n\"data3\":\"ColreUHAtn8iYvHIHw2ohiAuaXNGNX9WhlwANGDBYKYs6YEvHXgFXRkWuYoFyfLVqtjZNPedtukWRDXFz7VUO"...>>}}
685> timer:tc(snappy, compress, [J]).
69{113,
70 {ok,<<"úZðÔ{\n\"data3\":\"ColreUHAtn8iYvHIHw2ohiAuaXNGNX9WhlwANGDBYKYs6YEvHXgFXRkWuYoFyfLVqtjZNPedtukWRDXFz7VUO"...>>}}
716> timer:tc(snappy, compress, [J]).
72{112,
73 {ok,<<"úZðÔ{\n\"data3\":\"ColreUHAtn8iYvHIHw2ohiAuaXNGNX9WhlwANGDBYKYs6YEvHXgFXRkWuYoFyfLVqtjZNPedtukWRDXFz7VUO"...>>}}
747> timer:tc(snappy, compress, [J]).
75{113,
76 {ok,<<"úZðÔ{\n\"data3\":\"ColreUHAtn8iYvHIHw2ohiAuaXNGNX9WhlwANGDBYKYs6YEvHXgFXRkWuYoFyfLVqtjZNPedtukWRDXFz7VUO"...>>}}
778> timer:tc(snappy, compress, [J]).
78{111,
79 {ok,<<"úZðÔ{\n\"data3\":\"ColreUHAtn8iYvHIHw2ohiAuaXNGNX9WhlwANGDBYKYs6YEvHXgFXRkWuYoFyfLVqtjZNPedtukWRDXFz7VUO"...>>}}
809>
819> timer:tc(zlib, zip, [J]).
82{1025,
83 <<237,88,217,142,226,88,18,125,175,175,104,241,108,89,
84 119,95,234,13,39,24,179,153,37,49,182,25,181,...>>}
8510> timer:tc(zlib, zip, [J]).
86{753,
87 <<237,88,217,142,226,88,18,125,175,175,104,241,108,89,
88 119,95,234,13,39,24,179,153,37,49,182,25,181,...>>}
8911> timer:tc(zlib, zip, [J]).
90{974,
91 <<237,88,217,142,226,88,18,125,175,175,104,241,108,89,
92 119,95,234,13,39,24,179,153,37,49,182,25,181,...>>}
9312> timer:tc(zlib, zip, [J]).
94{753,
95 <<237,88,217,142,226,88,18,125,175,175,104,241,108,89,
96 119,95,234,13,39,24,179,153,37,49,182,25,181,...>>}
9713> timer:tc(zlib, zip, [J]).
98{960,
99 <<237,88,217,142,226,88,18,125,175,175,104,241,108,89,
100 119,95,234,13,39,24,179,153,37,49,182,25,181,...>>}
10114> timer:tc(zlib, zip, [J]).
102{769,
103 <<237,88,217,142,226,88,18,125,175,175,104,241,108,89,
104 119,95,234,13,39,24,179,153,37,49,182,25,181,...>>}
10515>
10615> byte_size(element(2, snappy:compress(J))).
1072846
10816> byte_size(zlib:zip(J)).
1091858
11017>
11117> crypto:start().
112ok
11318> Large = base64:encode(crypto:rand_bytes(100 * 1024)).
114<<"72GMvPLuWJLJeYSlNm9oNkYkbjTeSBDgTrqamQ/zgj7hcUge1co6LvldJyitKPZkMPQGnrN98EyqSyWto3k7ShNgiNc05Gt4zdXSbTVSDzeID661DnJX"...>>
11519>
11619> timer:tc(snappy, compress, [Large]).
117{365,
118 {ok,<<216,170,8,244,255,127,55,50,71,77,118,80,76,117,
119 87,74,76,74,101,89,83,108,78,109,57,...>>}}
12020> timer:tc(snappy, compress, [Large]).
121{541,
122 {ok,<<216,170,8,244,255,127,55,50,71,77,118,80,76,117,
123 87,74,76,74,101,89,83,108,78,109,57,...>>}}
12421> timer:tc(snappy, compress, [Large]).
125{455,
126 {ok,<<216,170,8,244,255,127,55,50,71,77,118,80,76,117,
127 87,74,76,74,101,89,83,108,78,109,57,...>>}}
12822> timer:tc(snappy, compress, [Large]).
129{479,
130 {ok,<<216,170,8,244,255,127,55,50,71,77,118,80,76,117,
131 87,74,76,74,101,89,83,108,78,109,57,...>>}}
13223>
13323> timer:tc(zlib, zip, [Large]).
134{14666,
135 <<20,155,181,118,195,64,20,5,63,72,133,152,74,49,147,197,
136 234,196,204,236,175,143,115,210,164,112,188,...>>}
13724> timer:tc(zlib, zip, [Large]).
138{19716,
139 <<20,155,181,118,195,64,20,5,63,72,133,152,74,49,147,197,
140 234,196,204,236,175,143,115,210,164,112,188,...>>}
14125> timer:tc(zlib, zip, [Large]).
142{14561,
143 <<20,155,181,118,195,64,20,5,63,72,133,152,74,49,147,197,
144 234,196,204,236,175,143,115,210,164,112,188,...>>}
14526> timer:tc(zlib, zip, [Large]).
146{21683,
147 <<20,155,181,118,195,64,20,5,63,72,133,152,74,49,147,197,
148 234,196,204,236,175,143,115,210,164,112,188,...>>}
14927> timer:tc(zlib, zip, [Large]).
150{11575,
151 <<20,155,181,118,195,64,20,5,63,72,133,152,74,49,147,197,
152 234,196,204,236,175,143,115,210,164,112,188,...>>}
15328>
15428> byte_size(element(2, snappy:compress(Large))).
155136554
15629> byte_size(zlib:zip(Large)).
157103440
15830>
159</pre>
160