README.md
1# Codec
2
3High Performance and Feature-Rich Idiomatic Go Library providing
4encode/decode support for different serialization formats.
5
6Supported Serialization formats are:
7
8 - msgpack: [https://github.com/msgpack/msgpack]
9 - binc: [http://github.com/ugorji/binc]
10
11To install:
12
13 go get github.com/ugorji/go/codec
14
15Online documentation: [http://godoc.org/github.com/ugorji/go/codec]
16
17The idiomatic Go support is as seen in other encoding packages in
18the standard library (ie json, xml, gob, etc).
19
20Rich Feature Set includes:
21
22 - Simple but extremely powerful and feature-rich API
23 - Very High Performance.
24 Our extensive benchmarks show us outperforming Gob, Json and Bson by 2-4X.
25 This was achieved by taking extreme care on:
26 - managing allocation
27 - function frame size (important due to Go's use of split stacks),
28 - reflection use (and by-passing reflection for common types)
29 - recursion implications
30 - zero-copy mode (encoding/decoding to byte slice without using temp buffers)
31 - Correct.
32 Care was taken to precisely handle corner cases like:
33 overflows, nil maps and slices, nil value in stream, etc.
34 - Efficient zero-copying into temporary byte buffers
35 when encoding into or decoding from a byte slice.
36 - Standard field renaming via tags
37 - Encoding from any value
38 (struct, slice, map, primitives, pointers, interface{}, etc)
39 - Decoding into pointer to any non-nil typed value
40 (struct, slice, map, int, float32, bool, string, reflect.Value, etc)
41 - Supports extension functions to handle the encode/decode of custom types
42 - Support Go 1.2 encoding.BinaryMarshaler/BinaryUnmarshaler
43 - Schema-less decoding
44 (decode into a pointer to a nil interface{} as opposed to a typed non-nil value).
45 Includes Options to configure what specific map or slice type to use
46 when decoding an encoded list or map into a nil interface{}
47 - Provides a RPC Server and Client Codec for net/rpc communication protocol.
48 - Msgpack Specific:
49 - Provides extension functions to handle spec-defined extensions (binary, timestamp)
50 - Options to resolve ambiguities in handling raw bytes (as string or []byte)
51 during schema-less decoding (decoding into a nil interface{})
52 - RPC Server/Client Codec for msgpack-rpc protocol defined at:
53 https://github.com/msgpack-rpc/msgpack-rpc/blob/master/spec.md
54 - Fast Paths for some container types:
55 For some container types, we circumvent reflection and its associated overhead
56 and allocation costs, and encode/decode directly. These types are:
57 []interface{}
58 []int
59 []string
60 map[interface{}]interface{}
61 map[int]interface{}
62 map[string]interface{}
63
64## Extension Support
65
66Users can register a function to handle the encoding or decoding of
67their custom types.
68
69There are no restrictions on what the custom type can be. Some examples:
70
71 type BisSet []int
72 type BitSet64 uint64
73 type UUID string
74 type MyStructWithUnexportedFields struct { a int; b bool; c []int; }
75 type GifImage struct { ... }
76
77As an illustration, MyStructWithUnexportedFields would normally be
78encoded as an empty map because it has no exported fields, while UUID
79would be encoded as a string. However, with extension support, you can
80encode any of these however you like.
81
82## RPC
83
84RPC Client and Server Codecs are implemented, so the codecs can be used
85with the standard net/rpc package.
86
87## Usage
88
89Typical usage model:
90
91 // create and configure Handle
92 var (
93 bh codec.BincHandle
94 mh codec.MsgpackHandle
95 )
96
97 mh.MapType = reflect.TypeOf(map[string]interface{}(nil))
98
99 // configure extensions
100 // e.g. for msgpack, define functions and enable Time support for tag 1
101 // mh.AddExt(reflect.TypeOf(time.Time{}), 1, myMsgpackTimeEncodeExtFn, myMsgpackTimeDecodeExtFn)
102
103 // create and use decoder/encoder
104 var (
105 r io.Reader
106 w io.Writer
107 b []byte
108 h = &bh // or mh to use msgpack
109 )
110
111 dec = codec.NewDecoder(r, h)
112 dec = codec.NewDecoderBytes(b, h)
113 err = dec.Decode(&v)
114
115 enc = codec.NewEncoder(w, h)
116 enc = codec.NewEncoderBytes(&b, h)
117 err = enc.Encode(v)
118
119 //RPC Server
120 go func() {
121 for {
122 conn, err := listener.Accept()
123 rpcCodec := codec.GoRpc.ServerCodec(conn, h)
124 //OR rpcCodec := codec.MsgpackSpecRpc.ServerCodec(conn, h)
125 rpc.ServeCodec(rpcCodec)
126 }
127 }()
128
129 //RPC Communication (client side)
130 conn, err = net.Dial("tcp", "localhost:5555")
131 rpcCodec := codec.GoRpc.ClientCodec(conn, h)
132 //OR rpcCodec := codec.MsgpackSpecRpc.ClientCodec(conn, h)
133 client := rpc.NewClientWithCodec(rpcCodec)
134
135## Representative Benchmark Results
136
137A sample run of benchmark using "go test -bi -bench=. -benchmem":
138
139 /proc/cpuinfo: Intel(R) Core(TM) i7-2630QM CPU @ 2.00GHz (HT)
140
141 ..............................................
142 BENCHMARK INIT: 2013-10-16 11:02:50.345970786 -0400 EDT
143 To run full benchmark comparing encodings (MsgPack, Binc, JSON, GOB, etc), use: "go test -bench=."
144 Benchmark:
145 Struct recursive Depth: 1
146 ApproxDeepSize Of benchmark Struct: 4694 bytes
147 Benchmark One-Pass Run:
148 v-msgpack: len: 1600 bytes
149 bson: len: 3025 bytes
150 msgpack: len: 1560 bytes
151 binc: len: 1187 bytes
152 gob: len: 1972 bytes
153 json: len: 2538 bytes
154 ..............................................
155 PASS
156 Benchmark__Msgpack____Encode 50000 54359 ns/op 14953 B/op 83 allocs/op
157 Benchmark__Msgpack____Decode 10000 106531 ns/op 14990 B/op 410 allocs/op
158 Benchmark__Binc_NoSym_Encode 50000 53956 ns/op 14966 B/op 83 allocs/op
159 Benchmark__Binc_NoSym_Decode 10000 103751 ns/op 14529 B/op 386 allocs/op
160 Benchmark__Binc_Sym___Encode 50000 65961 ns/op 17130 B/op 88 allocs/op
161 Benchmark__Binc_Sym___Decode 10000 106310 ns/op 15857 B/op 287 allocs/op
162 Benchmark__Gob________Encode 10000 135944 ns/op 21189 B/op 237 allocs/op
163 Benchmark__Gob________Decode 5000 405390 ns/op 83460 B/op 1841 allocs/op
164 Benchmark__Json_______Encode 20000 79412 ns/op 13874 B/op 102 allocs/op
165 Benchmark__Json_______Decode 10000 247979 ns/op 14202 B/op 493 allocs/op
166 Benchmark__Bson_______Encode 10000 121762 ns/op 27814 B/op 514 allocs/op
167 Benchmark__Bson_______Decode 10000 162126 ns/op 16514 B/op 789 allocs/op
168 Benchmark__VMsgpack___Encode 50000 69155 ns/op 12370 B/op 344 allocs/op
169 Benchmark__VMsgpack___Decode 10000 151609 ns/op 20307 B/op 571 allocs/op
170 ok ugorji.net/codec 30.827s
171
172To run full benchmark suite (including against vmsgpack and bson),
173see notes in ext\_dep\_test.go
174
175