• Home
  • History
  • Annotate
Name Date Size #Lines LOC

..03-May-2022-

benchmark/H10-Sep-2019-12,71110,518

bootstrap/H10-Sep-2019-194160

buffer/H10-Sep-2019-379285

easyjson/H10-Sep-2019-10990

gen/H10-Sep-2019-1,5811,256

jlexer/H10-Sep-2019-1,5721,299

jwriter/H10-Sep-2019-391302

opt/H10-Sep-2019-1,224812

parser/H10-Sep-2019-271223

tests/H10-Sep-2019-1,6401,296

.gitignoreH A D10-Sep-201938 65

.travis.ymlH A D10-Sep-2019124 139

LICENSEH A D10-Sep-20191 KiB84

MakefileH A D10-Sep-20191.2 KiB5343

README.mdH A D10-Sep-201914.6 KiB337272

go.modH A D10-Sep-201943 42

helpers.goH A D10-Sep-20192.2 KiB7954

raw.goH A D10-Sep-20191.1 KiB4630

README.md

1# easyjson [![Build Status](https://travis-ci.org/mailru/easyjson.svg?branch=master)](https://travis-ci.org/mailru/easyjson) [![Go Report Card](https://goreportcard.com/badge/github.com/mailru/easyjson)](https://goreportcard.com/report/github.com/mailru/easyjson)
2
3Package easyjson provides a fast and easy way to marshal/unmarshal Go structs
4to/from JSON without the use of reflection. In performance tests, easyjson
5outperforms the standard `encoding/json` package by a factor of 4-5x, and other
6JSON encoding packages by a factor of 2-3x.
7
8easyjson aims to keep generated Go code simple enough so that it can be easily
9optimized or fixed. Another goal is to provide users with the ability to
10customize the generated code by providing options not available with the
11standard `encoding/json` package, such as generating "snake_case" names or
12enabling `omitempty` behavior by default.
13
14## Usage
15```sh
16# install
17go get -u github.com/mailru/easyjson/...
18
19# run
20easyjson -all <file>.go
21```
22
23The above will generate `<file>_easyjson.go` containing the appropriate marshaler and
24unmarshaler funcs for all structs contained in `<file>.go`.
25
26Please note that easyjson requires a full Go build environment and the `GOPATH`
27environment variable to be set. This is because easyjson code generation
28invokes `go run` on a temporary file (an approach to code generation borrowed
29from [ffjson](https://github.com/pquerna/ffjson)).
30
31## Options
32```txt
33Usage of easyjson:
34  -all
35    	generate marshaler/unmarshalers for all structs in a file
36  -build_tags string
37    	build tags to add to generated file
38  -leave_temps
39    	do not delete temporary files
40  -no_std_marshalers
41    	don't generate MarshalJSON/UnmarshalJSON funcs
42  -noformat
43    	do not run 'gofmt -w' on output file
44  -omit_empty
45    	omit empty fields by default
46  -output_filename string
47    	specify the filename of the output
48  -pkg
49    	process the whole package instead of just the given file
50  -snake_case
51    	use snake_case names instead of CamelCase by default
52  -lower_camel_case
53        use lowerCamelCase instead of CamelCase by default
54  -stubs
55    	only generate stubs for marshaler/unmarshaler funcs
56  -disallow_unknown_fields
57        return error if some unknown field in json appeared
58```
59
60Using `-all` will generate marshalers/unmarshalers for all Go structs in the
61file. If `-all` is not provided, then only those structs whose preceding
62comment starts with `easyjson:json` will have marshalers/unmarshalers
63generated. For example:
64
65```go
66//easyjson:json
67type A struct {}
68```
69
70Additional option notes:
71
72* `-snake_case` tells easyjson to generate snake\_case field names by default
73  (unless overridden by a field tag). The CamelCase to snake\_case conversion
74  algorithm should work in most cases (ie, HTTPVersion will be converted to
75  "http_version").
76
77* `-build_tags` will add the specified build tags to generated Go sources.
78
79## Generated Marshaler/Unmarshaler Funcs
80
81For Go struct types, easyjson generates the funcs `MarshalEasyJSON` /
82`UnmarshalEasyJSON` for marshaling/unmarshaling JSON. In turn, these satisify
83the `easyjson.Marshaler` and `easyjson.Unmarshaler` interfaces and when used in
84conjunction with `easyjson.Marshal` / `easyjson.Unmarshal` avoid unnecessary
85reflection / type assertions during marshaling/unmarshaling to/from JSON for Go
86structs.
87
88easyjson also generates `MarshalJSON` and `UnmarshalJSON` funcs for Go struct
89types compatible with the standard `json.Marshaler` and `json.Unmarshaler`
90interfaces. Please be aware that using the standard `json.Marshal` /
91`json.Unmarshal` for marshaling/unmarshaling will incur a significant
92performance penalty when compared to using `easyjson.Marshal` /
93`easyjson.Unmarshal`.
94
95Additionally, easyjson exposes utility funcs that use the `MarshalEasyJSON` and
96`UnmarshalEasyJSON` for marshaling/unmarshaling to and from standard readers
97and writers. For example, easyjson provides `easyjson.MarshalToHTTPResponseWriter`
98which marshals to the standard `http.ResponseWriter`. Please see the [GoDoc
99listing](https://godoc.org/github.com/mailru/easyjson) for the full listing of
100utility funcs that are available.
101
102## Controlling easyjson Marshaling and Unmarshaling Behavior
103
104Go types can provide their own `MarshalEasyJSON` and `UnmarshalEasyJSON` funcs
105that satisify the `easyjson.Marshaler` / `easyjson.Unmarshaler` interfaces.
106These will be used by `easyjson.Marshal` and `easyjson.Unmarshal` when defined
107for a Go type.
108
109Go types can also satisify the `easyjson.Optional` interface, which allows the
110type to define its own `omitempty` logic.
111
112## Type Wrappers
113
114easyjson provides additional type wrappers defined in the `easyjson/opt`
115package. These wrap the standard Go primitives and in turn satisify the
116easyjson interfaces.
117
118The `easyjson/opt` type wrappers are useful when needing to distinguish between
119a missing value and/or when needing to specifying a default value. Type
120wrappers allow easyjson to avoid additional pointers and heap allocations and
121can significantly increase performance when used properly.
122
123## Memory Pooling
124
125easyjson uses a buffer pool that allocates data in increasing chunks from 128
126to 32768 bytes. Chunks of 512 bytes and larger will be reused with the help of
127`sync.Pool`. The maximum size of a chunk is bounded to reduce redundant memory
128allocation and to allow larger reusable buffers.
129
130easyjson's custom allocation buffer pool is defined in the `easyjson/buffer`
131package, and the default behavior pool behavior can be modified (if necessary)
132through a call to `buffer.Init()` prior to any marshaling or unmarshaling.
133Please see the [GoDoc listing](https://godoc.org/github.com/mailru/easyjson/buffer)
134for more information.
135
136## Issues, Notes, and Limitations
137
138* easyjson is still early in its development. As such, there are likely to be
139  bugs and missing features when compared to `encoding/json`. In the case of a
140  missing feature or bug, please create a GitHub issue. Pull requests are
141  welcome!
142
143* Unlike `encoding/json`, object keys are case-sensitive. Case-insensitive
144  matching is not currently provided due to the significant performance hit
145  when doing case-insensitive key matching. In the future, case-insensitive
146  object key matching may be provided via an option to the generator.
147
148* easyjson makes use of `unsafe`, which simplifies the code and
149  provides significant performance benefits by allowing no-copy
150  conversion from `[]byte` to `string`. That said, `unsafe` is used
151  only when unmarshaling and parsing JSON, and any `unsafe` operations
152  / memory allocations done will be safely deallocated by
153  easyjson. Set the build tag `easyjson_nounsafe` to compile it
154  without `unsafe`.
155
156* easyjson is compatible with Google App Engine. The `appengine` build
157  tag (set by App Engine's environment) will automatically disable the
158  use of `unsafe`, which is not allowed in App Engine's Standard
159  Environment. Note that the use with App Engine is still experimental.
160
161* Floats are formatted using the default precision from Go's `strconv` package.
162  As such, easyjson will not correctly handle high precision floats when
163  marshaling/unmarshaling JSON. Note, however, that there are very few/limited
164  uses where this behavior is not sufficient for general use. That said, a
165  different package may be needed if precise marshaling/unmarshaling of high
166  precision floats to/from JSON is required.
167
168* While unmarshaling, the JSON parser does the minimal amount of work needed to
169  skip over unmatching parens, and as such full validation is not done for the
170  entire JSON value being unmarshaled/parsed.
171
172* Currently there is no true streaming support for encoding/decoding as
173  typically for many uses/protocols the final, marshaled length of the JSON
174  needs to be known prior to sending the data. Currently this is not possible
175  with easyjson's architecture.
176
177* easyjson parser and codegen based on reflection, so it wont works on `package main`
178  files, because they cant be imported by parser.
179
180## Benchmarks
181
182Most benchmarks were done using the example
183[13kB example JSON](https://dev.twitter.com/rest/reference/get/search/tweets)
184(9k after eliminating whitespace). This example is similar to real-world data,
185is well-structured, and contains a healthy variety of different types, making
186it ideal for JSON serialization benchmarks.
187
188Note:
189
190* For small request benchmarks, an 80 byte portion of the above example was
191  used.
192
193* For large request marshaling benchmarks, a struct containing 50 regular
194  samples was used, making a ~500kB output JSON.
195
196* Benchmarks are showing the results of easyjson's default behaviour,
197  which makes use of `unsafe`.
198
199Benchmarks are available in the repository and can be run by invoking `make`.
200
201### easyjson vs. encoding/json
202
203easyjson is roughly 5-6 times faster than the standard `encoding/json` for
204unmarshaling, and 3-4 times faster for non-concurrent marshaling. Concurrent
205marshaling is 6-7x faster if marshaling to a writer.
206
207### easyjson vs. ffjson
208
209easyjson uses the same approach for JSON marshaling as
210[ffjson](https://github.com/pquerna/ffjson), but takes a significantly
211different approach to lexing and parsing JSON during unmarshaling. This means
212easyjson is roughly 2-3x faster for unmarshaling and 1.5-2x faster for
213non-concurrent unmarshaling.
214
215As of this writing, `ffjson` seems to have issues when used concurrently:
216specifically, large request pooling hurts `ffjson`'s performance and causes
217scalability issues. These issues with `ffjson` can likely be fixed, but as of
218writing remain outstanding/known issues with `ffjson`.
219
220easyjson and `ffjson` have similar performance for small requests, however
221easyjson outperforms `ffjson` by roughly 2-5x times for large requests when
222used with a writer.
223
224### easyjson vs. go/codec
225
226[go/codec](https://github.com/ugorji/go) provides
227compile-time helpers for JSON generation. In this case, helpers do not work
228like marshalers as they are encoding-independent.
229
230easyjson is generally 2x faster than `go/codec` for non-concurrent benchmarks
231and about 3x faster for concurrent encoding (without marshaling to a writer).
232
233In an attempt to measure marshaling performance of `go/codec` (as opposed to
234allocations/memcpy/writer interface invocations), a benchmark was done with
235resetting length of a byte slice rather than resetting the whole slice to nil.
236However, the optimization in this exact form may not be applicable in practice,
237since the memory is not freed between marshaling operations.
238
239### easyjson vs 'ujson' python module
240
241[ujson](https://github.com/esnme/ultrajson) is using C code for parsing, so it
242is interesting to see how plain golang compares to that. It is imporant to note
243that the resulting object for python is slower to access, since the library
244parses JSON object into dictionaries.
245
246easyjson is slightly faster for unmarshaling and 2-3x faster than `ujson` for
247marshaling.
248
249### Benchmark Results
250
251`ffjson` results are from February 4th, 2016, using the latest `ffjson` and go1.6.
252`go/codec` results are from March 4th, 2016, using the latest `go/codec` and go1.6.
253
254#### Unmarshaling
255
256| lib      | json size | MB/s | allocs/op | B/op  |
257|:---------|:----------|-----:|----------:|------:|
258| standard | regular   | 22   | 218       | 10229 |
259| standard | small     | 9.7  | 14        | 720   |
260|          |           |      |           |       |
261| easyjson | regular   | 125  | 128       | 9794  |
262| easyjson | small     | 67   | 3         | 128   |
263|          |           |      |           |       |
264| ffjson   | regular   | 66   | 141       | 9985  |
265| ffjson   | small     | 17.6 | 10        | 488   |
266|          |           |      |           |       |
267| codec    | regular   | 55   | 434       | 19299 |
268| codec    | small     | 29   | 7         | 336   |
269|          |           |      |           |       |
270| ujson    | regular   | 103  | N/A       | N/A   |
271
272#### Marshaling, one goroutine.
273
274| lib       | json size | MB/s | allocs/op | B/op  |
275|:----------|:----------|-----:|----------:|------:|
276| standard  | regular   | 75   | 9         | 23256 |
277| standard  | small     | 32   | 3         | 328   |
278| standard  | large     | 80   | 17        | 1.2M  |
279|           |           |      |           |       |
280| easyjson  | regular   | 213  | 9         | 10260 |
281| easyjson* | regular   | 263  | 8         | 742   |
282| easyjson  | small     | 125  | 1         | 128   |
283| easyjson  | large     | 212  | 33        | 490k  |
284| easyjson* | large     | 262  | 25        | 2879  |
285|           |           |      |           |       |
286| ffjson    | regular   | 122  | 153       | 21340 |
287| ffjson**  | regular   | 146  | 152       | 4897  |
288| ffjson    | small     | 36   | 5         | 384   |
289| ffjson**  | small     | 64   | 4         | 128   |
290| ffjson    | large     | 134  | 7317      | 818k  |
291| ffjson**  | large     | 125  | 7320      | 827k  |
292|           |           |      |           |       |
293| codec     | regular   | 80   | 17        | 33601 |
294| codec***  | regular   | 108  | 9         | 1153  |
295| codec     | small     | 42   | 3         | 304   |
296| codec***  | small     | 56   | 1         | 48    |
297| codec     | large     | 73   | 483       | 2.5M  |
298| codec***  | large     | 103  | 451       | 66007 |
299|           |           |      |           |       |
300| ujson     | regular   | 92   | N/A       | N/A   |
301
302\* marshaling to a writer,
303\*\* using `ffjson.Pool()`,
304\*\*\* reusing output slice instead of resetting it to nil
305
306#### Marshaling, concurrent.
307
308| lib       | json size | MB/s | allocs/op | B/op  |
309|:----------|:----------|-----:|----------:|------:|
310| standard  | regular   | 252  | 9         | 23257 |
311| standard  | small     | 124  | 3         | 328   |
312| standard  | large     | 289  | 17        | 1.2M  |
313|           |           |      |           |       |
314| easyjson  | regular   | 792  | 9         | 10597 |
315| easyjson* | regular   | 1748 | 8         | 779   |
316| easyjson  | small     | 333  | 1         | 128   |
317| easyjson  | large     | 718  | 36        | 548k  |
318| easyjson* | large     | 2134 | 25        | 4957  |
319|           |           |      |           |       |
320| ffjson    | regular   | 301  | 153       | 21629 |
321| ffjson**  | regular   | 707  | 152       | 5148  |
322| ffjson    | small     | 62   | 5         | 384   |
323| ffjson**  | small     | 282  | 4         | 128   |
324| ffjson    | large     | 438  | 7330      | 1.0M  |
325| ffjson**  | large     | 131  | 7319      | 820k  |
326|           |           |      |           |       |
327| codec     | regular   | 183  | 17        | 33603 |
328| codec***  | regular   | 671  | 9         | 1157  |
329| codec     | small     | 147  | 3         | 304   |
330| codec***  | small     | 299  | 1         | 48    |
331| codec     | large     | 190  | 483       | 2.5M  |
332| codec***  | large     | 752  | 451       | 77574 |
333
334\* marshaling to a writer,
335\*\* using `ffjson.Pool()`,
336\*\*\* reusing output slice instead of resetting it to nil
337