• Home
  • History
  • Annotate
Name Date Size #Lines LOC

..03-May-2022-

codec/H17-Jul-2018-14778

conformance/H17-Jul-2018-2,3351,988

gogoproto/H17-Jul-2018-1,5681,167

gogoreplace/H17-Jul-2018-2623

io/H17-Jul-2018-709495

jsonpb/H17-Jul-2018-4,4753,772

plugin/H17-Jul-2018-8,4426,241

proto/H17-Jul-2018-26,75221,187

protobuf/H17-Jul-2018-2,4852,233

protoc-gen-combo/H17-Jul-2018-183144

protoc-gen-gofast/H17-Jul-2018-4915

protoc-gen-gogo/H17-Jul-2018-15,29512,053

protoc-gen-gogofast/H17-Jul-2018-4815

protoc-gen-gogofaster/H17-Jul-2018-5117

protoc-gen-gogoslick/H17-Jul-2018-6024

protoc-gen-gogotypes/H17-Jul-2018-7837

protoc-gen-gostring/H17-Jul-2018-4312

protoc-min-version/H17-Jul-2018-6835

sortkeys/H17-Jul-2018-10252

test/H17-Jul-2018-443,317422,072

types/H17-Jul-2018-14,49712,701

vanity/H17-Jul-2018-4,5523,932

version/H17-Jul-2018-7945

.gitignoreH A D17-Jul-201895 65

.mailmapH A D17-Jul-2018551 88

.travis.ymlH A D17-Jul-2018490 2317

AUTHORSH A D17-Jul-2018562 1612

CONTRIBUTORSH A D17-Jul-2018845 2423

GOLANG_CONTRIBUTORSH A D17-Jul-2018218 54

LICENSEH A D17-Jul-20181.7 KiB3729

MakefileH A D17-Jul-20185.1 KiB168122

READMEH A D17-Jul-201811.7 KiB300237

Readme.mdH A D17-Jul-20189.7 KiB152104

bench.mdH A D17-Jul-201815 KiB190175

custom_types.mdH A D17-Jul-20182.4 KiB6951

extensions.mdH A D17-Jul-201816.3 KiB165126

install-protobuf.shH A D17-Jul-2018643 2923

README

1GoGoProtobuf http://github.com/gogo/protobuf extends
2GoProtobuf http://github.com/golang/protobuf
3
4# Go support for Protocol Buffers
5
6Google's data interchange format.
7Copyright 2010 The Go Authors.
8https://github.com/golang/protobuf
9
10This package and the code it generates requires at least Go 1.6.
11
12This software implements Go bindings for protocol buffers.  For
13information about protocol buffers themselves, see
14	https://developers.google.com/protocol-buffers/
15
16## Installation ##
17
18To use this software, you must:
19- Install the standard C++ implementation of protocol buffers from
20	https://developers.google.com/protocol-buffers/
21- Of course, install the Go compiler and tools from
22	https://golang.org/
23  See
24	https://golang.org/doc/install
25  for details or, if you are using gccgo, follow the instructions at
26	https://golang.org/doc/install/gccgo
27- Grab the code from the repository and install the proto package.
28  The simplest way is to run `go get -u github.com/golang/protobuf/protoc-gen-go`.
29  The compiler plugin, protoc-gen-go, will be installed in $GOBIN,
30  defaulting to $GOPATH/bin.  It must be in your $PATH for the protocol
31  compiler, protoc, to find it.
32
33This software has two parts: a 'protocol compiler plugin' that
34generates Go source files that, once compiled, can access and manage
35protocol buffers; and a library that implements run-time support for
36encoding (marshaling), decoding (unmarshaling), and accessing protocol
37buffers.
38
39There is support for gRPC in Go using protocol buffers.
40See the note at the bottom of this file for details.
41
42There are no insertion points in the plugin.
43
44GoGoProtobuf provides extensions for protocol buffers and GoProtobuf
45see http://github.com/gogo/protobuf/gogoproto/doc.go
46
47## Using protocol buffers with Go ##
48
49Once the software is installed, there are two steps to using it.
50First you must compile the protocol buffer definitions and then import
51them, with the support library, into your program.
52
53To compile the protocol buffer definition, run protoc with the --gogo_out
54parameter set to the directory you want to output the Go code to.
55
56	protoc --gogo_out=. *.proto
57
58The generated files will be suffixed .pb.go.  See the Test code below
59for an example using such a file.
60
61## Packages and input paths ##
62
63The protocol buffer language has a concept of "packages" which does not
64correspond well to the Go notion of packages. In generated Go code,
65each source `.proto` file is associated with a single Go package. The
66name and import path for this package is specified with the `go_package`
67proto option:
68
69	option go_package = "github.com/gogo/protobuf/types";
70
71The protocol buffer compiler will attempt to derive a package name and
72import path if a `go_package` option is not present, but it is
73best to always specify one explicitly.
74
75There is a one-to-one relationship between source `.proto` files and
76generated `.pb.go` files, but any number of `.pb.go` files may be
77contained in the same Go package.
78
79The output name of a generated file is produced by replacing the
80`.proto` suffix with `.pb.go` (e.g., `foo.proto` produces `foo.pb.go`).
81However, the output directory is selected in one of two ways.  Let
82us say we have `inputs/x.proto` with a `go_package` option of
83`github.com/golang/protobuf/p`. The corresponding output file may
84be:
85
86- Relative to the import path:
87
88	protoc --gogo_out=. inputs/x.proto
89	# writes ./github.com/gogo/protobuf/p/x.pb.go
90
91  (This can work well with `--gogo_out=$GOPATH`.)
92
93- Relative to the input file:
94
95	protoc --gogo_out=paths=source_relative:. inputs/x.proto
96	# generate ./inputs/x.pb.go
97
98## Generated code ##
99
100The package comment for the proto library contains text describing
101the interface provided in Go for protocol buffers. Here is an edited
102version.
103
104If you are using any gogo.proto extensions you will need to specify the
105proto_path to include the descriptor.proto and gogo.proto.
106gogo.proto is located in github.com/gogo/protobuf/gogoproto
107This should be fine, since your import is the same.
108descriptor.proto is located in either github.com/gogo/protobuf/protobuf
109or code.google.com/p/protobuf/trunk/src/
110Its import is google/protobuf/descriptor.proto so it might need some help.
111
112	protoc --gogo_out=. -I=.:github.com/gogo/protobuf/protobuf *.proto
113
114==========
115
116The proto package converts data structures to and from the
117wire format of protocol buffers.  It works in concert with the
118Go source code generated for .proto files by the protocol compiler.
119
120A summary of the properties of the protocol buffer interface
121for a protocol buffer variable v:
122
123  - Names are turned from camel_case to CamelCase for export.
124  - There are no methods on v to set fields; just treat
125  	them as structure fields.
126  - There are getters that return a field's value if set,
127	and return the field's default value if unset.
128	The getters work even if the receiver is a nil message.
129  - The zero value for a struct is its correct initialization state.
130	All desired fields must be set before marshaling.
131  - A Reset() method will restore a protobuf struct to its zero state.
132  - Non-repeated fields are pointers to the values; nil means unset.
133	That is, optional or required field int32 f becomes F *int32.
134  - Repeated fields are slices.
135  - Helper functions are available to aid the setting of fields.
136	Helpers for getting values are superseded by the
137	GetFoo methods and their use is deprecated.
138		msg.Foo = proto.String("hello") // set field
139  - Constants are defined to hold the default values of all fields that
140	have them.  They have the form Default_StructName_FieldName.
141	Because the getter methods handle defaulted values,
142	direct use of these constants should be rare.
143  - Enums are given type names and maps from names to values.
144	Enum values are prefixed with the enum's type name. Enum types have
145	a String method, and a Enum method to assist in message construction.
146  - Nested groups and enums have type names prefixed with the name of
147  	the surrounding message type.
148  - Extensions are given descriptor names that start with E_,
149	followed by an underscore-delimited list of the nested messages
150	that contain it (if any) followed by the CamelCased name of the
151	extension field itself.  HasExtension, ClearExtension, GetExtension
152	and SetExtension are functions for manipulating extensions.
153  - Oneof field sets are given a single field in their message,
154	with distinguished wrapper types for each possible field value.
155  - Marshal and Unmarshal are functions to encode and decode the wire format.
156
157When the .proto file specifies `syntax="proto3"`, there are some differences:
158
159  - Non-repeated fields of non-message type are values instead of pointers.
160  - Enum types do not get an Enum method.
161
162Consider file test.proto, containing
163
164```proto
165	syntax = "proto2";
166	package example;
167
168	enum FOO { X = 17; };
169
170	message Test {
171	  required string label = 1;
172	  optional int32 type = 2 [default=77];
173	  repeated int64 reps = 3;
174	  optional group OptionalGroup = 4 {
175	    required string RequiredField = 5;
176	  }
177	}
178```
179
180To create and play with a Test object from the example package,
181
182```go
183	package main
184
185	import (
186		"log"
187
188		"github.com/gogo/protobuf/proto"
189		"path/to/example"
190	)
191
192	func main() {
193		test := &example.Test {
194			Label: proto.String("hello"),
195			Type:  proto.Int32(17),
196			Reps:  []int64{1, 2, 3},
197			Optionalgroup: &example.Test_OptionalGroup {
198				RequiredField: proto.String("good bye"),
199			},
200		}
201		data, err := proto.Marshal(test)
202		if err != nil {
203			log.Fatal("marshaling error: ", err)
204		}
205		newTest := &example.Test{}
206		err = proto.Unmarshal(data, newTest)
207		if err != nil {
208			log.Fatal("unmarshaling error: ", err)
209		}
210		// Now test and newTest contain the same data.
211		if test.GetLabel() != newTest.GetLabel() {
212			log.Fatalf("data mismatch %q != %q", test.GetLabel(), newTest.GetLabel())
213		}
214		// etc.
215	}
216```
217
218
219## Parameters ##
220
221To pass extra parameters to the plugin, use a comma-separated
222parameter list separated from the output directory by a colon:
223
224
225	protoc --gogo_out=plugins=grpc,import_path=mypackage:. *.proto
226
227- `paths=(import | source_relative)` - specifies how the paths of
228  generated files are structured. See the "Packages and imports paths"
229  section above. The default is `import`.
230- `plugins=plugin1+plugin2` - specifies the list of sub-plugins to
231  load. The only plugin in this repo is `grpc`.
232- `Mfoo/bar.proto=quux/shme` - declares that foo/bar.proto is
233  associated with Go package quux/shme.  This is subject to the
234  import_prefix parameter.
235
236The following parameters are deprecated and should not be used:
237
238- `import_prefix=xxx` - a prefix that is added onto the beginning of
239  all imports.
240- `import_path=foo/bar` - used as the package if no input files
241  declare `go_package`. If it contains slashes, everything up to the
242  rightmost slash is ignored.
243
244## gRPC Support ##
245
246If a proto file specifies RPC services, protoc-gen-go can be instructed to
247generate code compatible with gRPC (http://www.grpc.io/). To do this, pass
248the `plugins` parameter to protoc-gen-go; the usual way is to insert it into
249the --go_out argument to protoc:
250
251	protoc --gogo_out=plugins=grpc:. *.proto
252
253## Compatibility ##
254
255The library and the generated code are expected to be stable over time.
256However, we reserve the right to make breaking changes without notice for the
257following reasons:
258
259- Security. A security issue in the specification or implementation may come to
260  light whose resolution requires breaking compatibility. We reserve the right
261  to address such security issues.
262- Unspecified behavior.  There are some aspects of the Protocol Buffers
263  specification that are undefined.  Programs that depend on such unspecified
264  behavior may break in future releases.
265- Specification errors or changes. If it becomes necessary to address an
266  inconsistency, incompleteness, or change in the Protocol Buffers
267  specification, resolving the issue could affect the meaning or legality of
268  existing programs.  We reserve the right to address such issues, including
269  updating the implementations.
270- Bugs.  If the library has a bug that violates the specification, a program
271  that depends on the buggy behavior may break if the bug is fixed.  We reserve
272  the right to fix such bugs.
273- Adding methods or fields to generated structs.  These may conflict with field
274  names that already exist in a schema, causing applications to break.  When the
275  code generator encounters a field in the schema that would collide with a
276  generated field or method name, the code generator will append an underscore
277  to the generated field or method name.
278- Adding, removing, or changing methods or fields in generated structs that
279  start with `XXX`.  These parts of the generated code are exported out of
280  necessity, but should not be considered part of the public API.
281- Adding, removing, or changing unexported symbols in generated code.
282
283Any breaking changes outside of these will be announced 6 months in advance to
284protobuf@googlegroups.com.
285
286You should, whenever possible, use generated code created by the `protoc-gen-go`
287tool built at the same commit as the `proto` package.  The `proto` package
288declares package-level constants in the form `ProtoPackageIsVersionX`.
289Application code and generated code may depend on one of these constants to
290ensure that compilation will fail if the available version of the proto library
291is too old.  Whenever we make a change to the generated code that requires newer
292library support, in the same commit we will increment the version number of the
293generated code and declare a new package-level constant whose name incorporates
294the latest version number.  Removing a compatibility constant is considered a
295breaking change and would be subject to the announcement policy stated above.
296
297The `protoc-gen-go/generator` package exposes a plugin interface,
298which is used by the gRPC code generation. This interface is not
299supported and is subject to incompatible changes without notice.
300

Readme.md

1# Protocol Buffers for Go with Gadgets
2
3[![Build Status](https://travis-ci.org/gogo/protobuf.svg?branch=master)](https://travis-ci.org/gogo/protobuf)
4
5gogoprotobuf is a fork of <a href="https://github.com/golang/protobuf">golang/protobuf</a> with extra code generation features.
6
7This code generation is used to achieve:
8
9  - fast marshalling and unmarshalling
10  - more canonical Go structures
11  - goprotobuf compatibility
12  - less typing by optionally generating extra helper code
13  - peace of mind by optionally generating test and benchmark code
14  - other serialization formats
15
16Keeping track of how up to date gogoprotobuf is relative to golang/protobuf is done in this
17<a href="https://github.com/gogo/protobuf/issues/191">issue</a>
18
19## Users
20
21These projects use gogoprotobuf:
22
23  - <a href="http://godoc.org/github.com/coreos/etcd">etcd</a> - <a href="https://blog.gopheracademy.com/advent-2015/etcd-distributed-key-value-store-with-grpc-http2/">blog</a> - <a href="https://github.com/coreos/etcd/blob/master/etcdserver/etcdserverpb/etcdserver.proto">sample proto file</a>
24  - <a href="https://www.spacemonkey.com/">spacemonkey</a> - <a href="https://www.spacemonkey.com/blog/posts/go-space-monkey">blog</a>
25  - <a href="http://badoo.com">badoo</a> - <a href="https://github.com/badoo/lsd/blob/32061f501c5eca9c76c596d790b450501ba27b2f/proto/lsd.proto">sample proto file</a>
26  - <a href="https://github.com/mesos/mesos-go">mesos-go</a> - <a href="https://github.com/mesos/mesos-go/blob/f9e5fb7c2f50ab5f23299f26b6b07c5d6afdd252/api/v0/mesosproto/authentication.proto">sample proto file</a>
27  - <a href="https://github.com/mozilla-services/heka">heka</a> - <a href="https://github.com/mozilla-services/heka/commit/eb72fbf7d2d28249fbaf8d8dc6607f4eb6f03351">the switch from golang/protobuf to gogo/protobuf when it was still on code.google.com</a>
28  - <a href="https://github.com/cockroachdb/cockroach">cockroachdb</a> - <a href="https://github.com/cockroachdb/cockroach/blob/651d54d393e391a30154e9117ab4b18d9ee6d845/roachpb/metadata.proto">sample proto file</a>
29  - <a href="https://github.com/jbenet/go-ipfs">go-ipfs</a> - <a href="https://github.com/ipfs/go-ipfs/blob/2b6da0c024f28abeb16947fb452787196a6b56a2/merkledag/pb/merkledag.proto">sample proto file</a>
30  - <a href="https://github.com/philhofer/rkive">rkive-go</a> - <a href="https://github.com/philhofer/rkive/blob/e5dd884d3ea07b341321073882ae28aa16dd11be/rpbc/riak_dt.proto">sample proto file</a>
31  - <a href="https://www.dropbox.com">dropbox</a>
32  - <a href="https://srclib.org/">srclib</a> - <a href="https://github.com/sourcegraph/srclib/blob/6538858f0c410cac5c63440317b8d009e889d3fb/graph/def.proto">sample proto file</a>
33  - <a href="http://www.adyoulike.com/">adyoulike</a>
34  - <a href="http://www.cloudfoundry.org/">cloudfoundry</a> - <a href="https://github.com/cloudfoundry/bbs/blob/d673710b8c4211037805129944ee4c5373d6588a/models/events.proto">sample proto file</a>
35  - <a href="http://kubernetes.io/">kubernetes</a> - <a href="https://github.com/kubernetes/kubernetes/tree/88d8628137f94ee816aaa6606ae8cd045dee0bff/cmd/libs/go2idl">go2idl built on top of gogoprotobuf</a>
36  - <a href="https://dgraph.io/">dgraph</a> - <a href="https://github.com/dgraph-io/dgraph/releases/tag/v0.4.3">release notes</a> - <a href="https://discuss.dgraph.io/t/gogoprotobuf-is-extremely-fast/639">benchmarks</a></a>
37  - <a href="https://github.com/centrifugal/centrifugo">centrifugo</a> - <a href="https://forum.golangbridge.org/t/centrifugo-real-time-messaging-websocket-or-sockjs-server-v1-5-0-released/2861">release notes</a> - <a href="https://medium.com/@fzambia/centrifugo-protobuf-inside-json-outside-21d39bdabd68#.o3icmgjqd">blog</a>
38  - <a href="https://github.com/docker/swarmkit">docker swarmkit</a> - <a href="https://github.com/docker/swarmkit/blob/63600e01af3b8da2a0ed1c9fa6e1ae4299d75edb/api/objects.proto">sample proto file</a>
39  - <a href="https://nats.io/">nats.io</a> - <a href="https://github.com/nats-io/go-nats-streaming/blob/master/pb/protocol.proto">go-nats-streaming</a>
40  - <a href="https://github.com/pingcap/tidb">tidb</a> - Communication between <a href="https://github.com/pingcap/tipb/blob/master/generate-go.sh#L4">tidb</a> and <a href="https://github.com/pingcap/kvproto/blob/master/generate_go.sh#L3">tikv</a>
41  - <a href="https://github.com/AsynkronIT/protoactor-go">protoactor-go</a> - <a href="https://github.com/AsynkronIT/protoactor-go/blob/master/protobuf/protoc-gen-protoactor/main.go">vanity command</a> that also generates actors from service definitions
42  - <a href="https://containerd.io/">containerd</a> - <a href="https://github.com/containerd/containerd/tree/master/cmd/protoc-gen-gogoctrd">vanity command with custom field names</a> that conforms to the golang convention.
43  - <a href="https://github.com/heroiclabs/nakama">nakama</a>
44  - <a href="https://github.com/src-d/proteus">proteus</a>
45  - <a href="https://github.com/go-graphite">carbonzipper stack</a>
46  - <a href="https://sendgrid.com/">sendgrid</a>
47  - <a href="https://github.com/zero-os/0-stor">zero-os/0-stor</a>
48  - <a href="https://github.com/spacemeshos/go-spacemesh">go-spacemesh</a>
49
50Please let us know if you are using gogoprotobuf by posting on our <a href="https://groups.google.com/forum/#!topic/gogoprotobuf/Brw76BxmFpQ">GoogleGroup</a>.
51
52### Mentioned
53
54  - <a href="http://www.slideshare.net/albertstrasheim/serialization-in-go">Cloudflare - go serialization talk - Albert Strasheim</a>
55  - <a href="https://youtu.be/4xB46Xl9O9Q?t=557">GopherCon 2014 Writing High Performance Databases in Go by Ben Johnson</a>
56  - <a href="https://github.com/alecthomas/go_serialization_benchmarks">alecthomas' go serialization benchmarks</a>
57  - <a href="http://agniva.me/go/2017/11/18/gogoproto.html">Go faster with gogoproto - Agniva De Sarker</a>
58  - <a href="https://www.youtube.com/watch?v=CY9T020HLP8">Evolution of protobuf (Gource Visualization) - Landon Wilkins</a>
59  - <a href="https://fosdem.org/2018/schedule/event/gopherjs/">Creating GopherJS Apps with gRPC-Web - Johan Brandhorst</a>
60  - <a href="https://jbrandhorst.com/post/gogoproto/">So you want to use GoGo Protobuf - Johan Brandhorst</a>
61  - <a href="https://jbrandhorst.com/post/grpc-errors/">Advanced gRPC Error Usage - Johan Brandhorst</a>
62
63## Getting Started
64
65There are several ways to use gogoprotobuf, but for all you need to install go and protoc.
66After that you can choose:
67
68  - Speed
69  - More Speed and more generated code
70  - Most Speed and most customization
71
72### Installation
73
74To install it, you must first have Go (at least version 1.6.3 or 1.9 if you are using gRPC) installed (see [http://golang.org/doc/install](http://golang.org/doc/install)).
75Latest patch versions of 1.9 and 1.10 are continuously tested.
76
77Next, install the standard protocol buffer implementation from [https://github.com/google/protobuf](https://github.com/google/protobuf).
78Most versions from 2.3.1 should not give any problems, but 2.6.1, 3.0.2 and 3.5.1 are continuously tested.
79
80### Speed
81
82Install the protoc-gen-gofast binary
83
84    go get github.com/gogo/protobuf/protoc-gen-gofast
85
86Use it to generate faster marshaling and unmarshaling go code for your protocol buffers.
87
88    protoc --gofast_out=. myproto.proto
89
90This does not allow you to use any of the other gogoprotobuf [extensions](https://github.com/gogo/protobuf/blob/master/extensions.md).
91
92### More Speed and more generated code
93
94Fields without pointers cause less time in the garbage collector.
95More code generation results in more convenient methods.
96
97Other binaries are also included:
98
99    protoc-gen-gogofast (same as gofast, but imports gogoprotobuf)
100    protoc-gen-gogofaster (same as gogofast, without XXX_unrecognized, less pointer fields)
101    protoc-gen-gogoslick (same as gogofaster, but with generated string, gostring and equal methods)
102
103Installing any of these binaries is easy.  Simply run:
104
105    go get github.com/gogo/protobuf/proto
106    go get github.com/gogo/protobuf/{binary}
107    go get github.com/gogo/protobuf/gogoproto
108
109These binaries allow you to use gogoprotobuf [extensions](https://github.com/gogo/protobuf/blob/master/extensions.md). You can also use your own binary.
110
111To generate the code, you also need to set the include path properly.
112
113    protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=. myproto.proto
114
115To use proto files from "google/protobuf" you need to add additional args to protoc.
116
117    protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=\
118    Mgoogle/protobuf/any.proto=github.com/gogo/protobuf/types,\
119    Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,\
120    Mgoogle/protobuf/struct.proto=github.com/gogo/protobuf/types,\
121    Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,\
122    Mgoogle/protobuf/wrappers.proto=github.com/gogo/protobuf/types:. \
123    myproto.proto
124
125Note that in the protoc command, {binary} does not contain the initial prefix of "protoc-gen".
126
127### Most Speed and most customization
128
129Customizing the fields of the messages to be the fields that you actually want to use removes the need to copy between the structs you use and structs you use to serialize.
130gogoprotobuf also offers more serialization formats and generation of tests and even more methods.
131
132Please visit the [extensions](https://github.com/gogo/protobuf/blob/master/extensions.md) page for more documentation.
133
134Install protoc-gen-gogo:
135
136    go get github.com/gogo/protobuf/proto
137    go get github.com/gogo/protobuf/jsonpb
138    go get github.com/gogo/protobuf/protoc-gen-gogo
139    go get github.com/gogo/protobuf/gogoproto
140
141## GRPC
142
143It works the same as golang/protobuf, simply specify the plugin.
144Here is an example using gofast:
145
146    protoc --gofast_out=plugins=grpc:. my.proto
147
148See [https://github.com/gogo/grpc-example](https://github.com/gogo/grpc-example) for an example of using gRPC with gogoprotobuf and the wider grpc-ecosystem.
149
150
151
152