• Home
  • History
  • Annotate
Name Date Size #Lines LOC

..23-Jun-2021-

0doc.goH A D23-Jun-202110.7 KiB2651

README.mdH A D23-Jun-20217.8 KiB207160

binc.goH A D23-Jun-202127.4 KiB1,169941

cbor.goH A D23-Jun-202117.1 KiB757645

decode.goH A D23-Jun-202164.3 KiB2,6431,932

encode.goH A D23-Jun-202137.8 KiB1,4521,033

fast-path.generated.goH A D23-Jun-2021799.2 KiB34,52333,896

fast-path.go.tmplH A D23-Jun-202115.9 KiB545501

fast-path.not.goH A D23-Jun-20211.6 KiB4825

gen-dec-array.go.tmplH A D23-Jun-20212.8 KiB7978

gen-dec-map.go.tmplH A D23-Jun-20212.1 KiB4342

gen-enc-chan.go.tmplH A D23-Jun-2021647 2827

gen-helper.generated.goH A D23-Jun-202111 KiB336184

gen-helper.go.tmplH A D23-Jun-202111 KiB303282

gen.generated.goH A D23-Jun-20215.9 KiB165151

gen.goH A D23-Jun-202163.7 KiB2,1401,691

goversion_arrayof_gte_go15.goH A D23-Jun-2021338 156

goversion_arrayof_lt_go15.goH A D23-Jun-2021367 156

goversion_makemap_gte_go19.goH A D23-Jun-2021358 168

goversion_makemap_lt_go19.goH A D23-Jun-2021288 135

goversion_unexportedembeddedptr_gte_go110.goH A D23-Jun-2021223 92

goversion_unexportedembeddedptr_lt_go110.goH A D23-Jun-2021223 92

goversion_unsupported_lt_go14.goH A D23-Jun-2021503 184

goversion_vendor_eq_go15.goH A D23-Jun-2021260 113

goversion_vendor_eq_go16.goH A D23-Jun-2021260 113

goversion_vendor_gte_go17.goH A D23-Jun-2021206 92

goversion_vendor_lt_go15.goH A D23-Jun-2021206 92

helper.goH A D23-Jun-202167.3 KiB2,5091,665

helper_internal.goH A D23-Jun-20213.1 KiB12276

helper_not_unsafe.goH A D23-Jun-20216.9 KiB273171

helper_unsafe.goH A D23-Jun-202120.2 KiB640281

json.goH A D23-Jun-202134.1 KiB1,4241,101

mammoth-test.go.tmplH A D23-Jun-20217.3 KiB155138

mammoth2-test.go.tmplH A D23-Jun-20212.8 KiB9577

msgpack.goH A D23-Jun-202127.1 KiB1,081856

rpc.goH A D23-Jun-20215.9 KiB233147

simple.goH A D23-Jun-202115.5 KiB653535

test-cbor-goldens.jsonH A D23-Jun-202110.1 KiB640637

test.pyH A D23-Jun-20213.9 KiB12798

README.md

1# Codec
2
3High Performance, Feature-Rich Idiomatic Go codec/encoding library for
4binc, msgpack, cbor, json.
5
6Supported Serialization formats are:
7
8  - msgpack: https://github.com/msgpack/msgpack
9  - binc:    http://github.com/ugorji/binc
10  - cbor:    http://cbor.io http://tools.ietf.org/html/rfc7049
11  - json:    http://json.org http://tools.ietf.org/html/rfc7159
12  - simple:
13
14To install:
15
16    go get github.com/ugorji/go/codec
17
18This package will carefully use 'unsafe' for performance reasons in specific places.
19You can build without unsafe use by passing the safe or appengine tag
20i.e. 'go install -tags=safe ...'. Note that unsafe is only supported for the last 3
21go sdk versions e.g. current go release is go 1.9, so we support unsafe use only from
22go 1.7+ . This is because supporting unsafe requires knowledge of implementation details.
23
24Online documentation: http://godoc.org/github.com/ugorji/go/codec
25Detailed Usage/How-to Primer: http://ugorji.net/blog/go-codec-primer
26
27The idiomatic Go support is as seen in other encoding packages in
28the standard library (ie json, xml, gob, etc).
29
30Rich Feature Set includes:
31
32  - Simple but extremely powerful and feature-rich API
33  - Support for go1.4 and above, while selectively using newer APIs for later releases
34  - Excellent code coverage ( > 90% )
35  - Very High Performance.
36    Our extensive benchmarks show us outperforming Gob, Json, Bson, etc by 2-4X.
37  - Careful selected use of 'unsafe' for targeted performance gains.
38    100% mode exists where 'unsafe' is not used at all.
39  - Lock-free (sans mutex) concurrency for scaling to 100's of cores
40  - Coerce types where appropriate
41    e.g. decode an int in the stream into a float, decode numbers from formatted strings, etc
42  - Corner Cases:
43    Overflows, nil maps/slices, nil values in streams are handled correctly
44  - Standard field renaming via tags
45  - Support for omitting empty fields during an encoding
46  - Encoding from any value and decoding into pointer to any value
47    (struct, slice, map, primitives, pointers, interface{}, etc)
48  - Extensions to support efficient encoding/decoding of any named types
49  - Support encoding.(Binary|Text)(M|Unm)arshaler interfaces
50  - Support IsZero() bool to determine if a value is a zero value.
51    Analogous to time.Time.IsZero() bool.
52  - Decoding without a schema (into a interface{}).
53    Includes Options to configure what specific map or slice type to use
54    when decoding an encoded list or map into a nil interface{}
55  - Mapping a non-interface type to an interface, so we can decode appropriately
56    into any interface type with a correctly configured non-interface value.
57  - Encode a struct as an array, and decode struct from an array in the data stream
58  - Option to encode struct keys as numbers (instead of strings)
59    (to support structured streams with fields encoded as numeric codes)
60  - Comprehensive support for anonymous fields
61  - Fast (no-reflection) encoding/decoding of common maps and slices
62  - Code-generation for faster performance.
63  - Support binary (e.g. messagepack, cbor) and text (e.g. json) formats
64  - Support indefinite-length formats to enable true streaming
65    (for formats which support it e.g. json, cbor)
66  - Support canonical encoding, where a value is ALWAYS encoded as same sequence of bytes.
67    This mostly applies to maps, where iteration order is non-deterministic.
68  - NIL in data stream decoded as zero value
69  - Never silently skip data when decoding.
70    User decides whether to return an error or silently skip data when keys or indexes
71    in the data stream do not map to fields in the struct.
72  - Encode/Decode from/to chan types (for iterative streaming support)
73  - Drop-in replacement for encoding/json. `json:` key in struct tag supported.
74  - Provides a RPC Server and Client Codec for net/rpc communication protocol.
75  - Handle unique idiosyncrasies of codecs e.g.
76    - For messagepack, configure how ambiguities in handling raw bytes are resolved
77    - For messagepack, provide rpc server/client codec to support
78      msgpack-rpc protocol defined at:
79      https://github.com/msgpack-rpc/msgpack-rpc/blob/master/spec.md
80
81## Extension Support
82
83Users can register a function to handle the encoding or decoding of
84their custom types.
85
86There are no restrictions on what the custom type can be. Some examples:
87
88    type BisSet   []int
89    type BitSet64 uint64
90    type UUID     string
91    type MyStructWithUnexportedFields struct { a int; b bool; c []int; }
92    type GifImage struct { ... }
93
94As an illustration, MyStructWithUnexportedFields would normally be
95encoded as an empty map because it has no exported fields, while UUID
96would be encoded as a string. However, with extension support, you can
97encode any of these however you like.
98
99## Custom Encoding and Decoding
100
101This package maintains symmetry in the encoding and decoding halfs.
102We determine how to encode or decode by walking this decision tree
103
104  - is type a codec.Selfer?
105  - is there an extension registered for the type?
106  - is format binary, and is type a encoding.BinaryMarshaler and BinaryUnmarshaler?
107  - is format specifically json, and is type a encoding/json.Marshaler and Unmarshaler?
108  - is format text-based, and type an encoding.TextMarshaler?
109  - else we use a pair of functions based on the "kind" of the type e.g. map, slice, int64, etc
110
111This symmetry is important to reduce chances of issues happening because the
112encoding and decoding sides are out of sync e.g. decoded via very specific
113encoding.TextUnmarshaler but encoded via kind-specific generalized mode.
114
115Consequently, if a type only defines one-half of the symmetry
116(e.g. it implements UnmarshalJSON() but not MarshalJSON() ),
117then that type doesn't satisfy the check and we will continue walking down the
118decision tree.
119
120## RPC
121
122RPC Client and Server Codecs are implemented, so the codecs can be used
123with the standard net/rpc package.
124
125## Usage
126
127Typical usage model:
128
129    // create and configure Handle
130    var (
131      bh codec.BincHandle
132      mh codec.MsgpackHandle
133      ch codec.CborHandle
134    )
135
136    mh.MapType = reflect.TypeOf(map[string]interface{}(nil))
137
138    // configure extensions
139    // e.g. for msgpack, define functions and enable Time support for tag 1
140    // mh.SetExt(reflect.TypeOf(time.Time{}), 1, myExt)
141
142    // create and use decoder/encoder
143    var (
144      r io.Reader
145      w io.Writer
146      b []byte
147      h = &bh // or mh to use msgpack
148    )
149
150    dec = codec.NewDecoder(r, h)
151    dec = codec.NewDecoderBytes(b, h)
152    err = dec.Decode(&v)
153
154    enc = codec.NewEncoder(w, h)
155    enc = codec.NewEncoderBytes(&b, h)
156    err = enc.Encode(v)
157
158    //RPC Server
159    go func() {
160        for {
161            conn, err := listener.Accept()
162            rpcCodec := codec.GoRpc.ServerCodec(conn, h)
163            //OR rpcCodec := codec.MsgpackSpecRpc.ServerCodec(conn, h)
164            rpc.ServeCodec(rpcCodec)
165        }
166    }()
167
168    //RPC Communication (client side)
169    conn, err = net.Dial("tcp", "localhost:5555")
170    rpcCodec := codec.GoRpc.ClientCodec(conn, h)
171    //OR rpcCodec := codec.MsgpackSpecRpc.ClientCodec(conn, h)
172    client := rpc.NewClientWithCodec(rpcCodec)
173
174## Running Tests
175
176To run tests, use the following:
177
178    go test
179
180To run the full suite of tests, use the following:
181
182    go test -tags alltests -run Suite
183
184You can run the tag 'safe' to run tests or build in safe mode. e.g.
185
186    go test -tags safe -run Json
187    go test -tags "alltests safe" -run Suite
188
189## Running Benchmarks
190
191Please see http://github.com/ugorji/go-codec-bench .
192
193## Caveats
194
195Struct fields matching the following are ignored during encoding and decoding
196
197  - struct tag value set to -
198  - func, complex numbers, unsafe pointers
199  - unexported and not embedded
200  - unexported and embedded and not struct kind
201  - unexported and embedded pointers (from go1.10)
202
203Every other field in a struct will be encoded/decoded.
204
205Embedded fields are encoded as if they exist in the top-level struct,
206with some caveats. See Encode documentation.
207