• Home
  • History
  • Annotate
Name Date Size #Lines LOC

..06-May-2019-

gogoproto/H06-May-2019-1,6041,221

io/H06-May-2019-447285

proto/H06-May-2019-16,50612,875

protobuf/google/protobuf/H06-May-2019-2,4202,181

protoc-gen-gogo/descriptor/H06-May-2019-4,0623,261

sortkeys/H06-May-2019-10252

test/issue270/H06-May-2019-3320

types/H06-May-2019-14,49912,793

LICENSEH A D06-May-20191.6 KiB3628

READMEH A D06-May-201911.6 KiB299233

Readme.mdH A D06-May-201910.4 KiB160111

README

1Protocol Buffers for Go with Gadgets
2
3GoGoProtobuf http://github.com/gogo/protobuf extends
4GoProtobuf http://github.com/golang/protobuf
5
6Copyright (c) 2013, The GoGo Authors. All rights reserved.
7
8
9# Go support for Protocol Buffers
10
11Google's data interchange format.
12Copyright 2010 The Go Authors.
13https://github.com/golang/protobuf
14
15This package and the code it generates requires at least Go 1.6.
16
17This software implements Go bindings for protocol buffers.  For
18information about protocol buffers themselves, see
19	https://developers.google.com/protocol-buffers/
20
21## Installation ##
22
23To use this software, you must:
24- Install the standard C++ implementation of protocol buffers from
25	https://developers.google.com/protocol-buffers/
26- Of course, install the Go compiler and tools from
27	https://golang.org/
28  See
29	https://golang.org/doc/install
30  for details or, if you are using gccgo, follow the instructions at
31	https://golang.org/doc/install/gccgo
32- Grab the code from the repository and install the proto package.
33  The simplest way is to run `go get -u github.com/golang/protobuf/protoc-gen-go`.
34  The compiler plugin, protoc-gen-go, will be installed in $GOBIN,
35  defaulting to $GOPATH/bin.  It must be in your $PATH for the protocol
36  compiler, protoc, to find it.
37
38This software has two parts: a 'protocol compiler plugin' that
39generates Go source files that, once compiled, can access and manage
40protocol buffers; and a library that implements run-time support for
41encoding (marshaling), decoding (unmarshaling), and accessing protocol
42buffers.
43
44There is support for gRPC in Go using protocol buffers.
45See the note at the bottom of this file for details.
46
47There are no insertion points in the plugin.
48
49GoGoProtobuf provides extensions for protocol buffers and GoProtobuf
50see http://github.com/gogo/protobuf/gogoproto/doc.go
51
52## Using protocol buffers with Go ##
53
54Once the software is installed, there are two steps to using it.
55First you must compile the protocol buffer definitions and then import
56them, with the support library, into your program.
57
58To compile the protocol buffer definition, run protoc with the --gogo_out
59parameter set to the directory you want to output the Go code to.
60
61	protoc --gogo_out=. *.proto
62
63The generated files will be suffixed .pb.go.  See the Test code below
64for an example using such a file.
65
66## Packages and input paths ##
67
68The protocol buffer language has a concept of "packages" which does not
69correspond well to the Go notion of packages. In generated Go code,
70each source `.proto` file is associated with a single Go package. The
71name and import path for this package is specified with the `go_package`
72proto option:
73
74	option go_package = "github.com/gogo/protobuf/types";
75
76The protocol buffer compiler will attempt to derive a package name and
77import path if a `go_package` option is not present, but it is
78best to always specify one explicitly.
79
80There is a one-to-one relationship between source `.proto` files and
81generated `.pb.go` files, but any number of `.pb.go` files may be
82contained in the same Go package.
83
84The output name of a generated file is produced by replacing the
85`.proto` suffix with `.pb.go` (e.g., `foo.proto` produces `foo.pb.go`).
86However, the output directory is selected in one of two ways.  Let
87us say we have `inputs/x.proto` with a `go_package` option of
88`github.com/golang/protobuf/p`. The corresponding output file may
89be:
90
91- Relative to the import path:
92
93	protoc --gogo_out=. inputs/x.proto
94	# writes ./github.com/gogo/protobuf/p/x.pb.go
95
96  (This can work well with `--gogo_out=$GOPATH`.)
97
98- Relative to the input file:
99
100	protoc --gogo_out=paths=source_relative:. inputs/x.proto
101	# generate ./inputs/x.pb.go
102
103## Generated code ##
104
105The package comment for the proto library contains text describing
106the interface provided in Go for protocol buffers. Here is an edited
107version.
108
109If you are using any gogo.proto extensions you will need to specify the
110proto_path to include the descriptor.proto and gogo.proto.
111gogo.proto is located in github.com/gogo/protobuf/gogoproto
112This should be fine, since your import is the same.
113descriptor.proto is located in either github.com/gogo/protobuf/protobuf
114or code.google.com/p/protobuf/trunk/src/
115Its import is google/protobuf/descriptor.proto so it might need some help.
116
117	protoc --gogo_out=. -I=.:github.com/gogo/protobuf/protobuf *.proto
118
119==========
120
121The proto package converts data structures to and from the
122wire format of protocol buffers.  It works in concert with the
123Go source code generated for .proto files by the protocol compiler.
124
125A summary of the properties of the protocol buffer interface
126for a protocol buffer variable v:
127
128  - Names are turned from camel_case to CamelCase for export.
129  - There are no methods on v to set fields; just treat
130  	them as structure fields.
131  - There are getters that return a field's value if set,
132	and return the field's default value if unset.
133	The getters work even if the receiver is a nil message.
134  - The zero value for a struct is its correct initialization state.
135	All desired fields must be set before marshaling.
136  - A Reset() method will restore a protobuf struct to its zero state.
137  - Non-repeated fields are pointers to the values; nil means unset.
138	That is, optional or required field int32 f becomes F *int32.
139  - Repeated fields are slices.
140  - Helper functions are available to aid the setting of fields.
141	Helpers for getting values are superseded by the
142	GetFoo methods and their use is deprecated.
143		msg.Foo = proto.String("hello") // set field
144  - Constants are defined to hold the default values of all fields that
145	have them.  They have the form Default_StructName_FieldName.
146	Because the getter methods handle defaulted values,
147	direct use of these constants should be rare.
148  - Enums are given type names and maps from names to values.
149	Enum values are prefixed with the enum's type name. Enum types have
150	a String method, and a Enum method to assist in message construction.
151  - Nested groups and enums have type names prefixed with the name of
152  	the surrounding message type.
153  - Extensions are given descriptor names that start with E_,
154	followed by an underscore-delimited list of the nested messages
155	that contain it (if any) followed by the CamelCased name of the
156	extension field itself.  HasExtension, ClearExtension, GetExtension
157	and SetExtension are functions for manipulating extensions.
158  - Oneof field sets are given a single field in their message,
159	with distinguished wrapper types for each possible field value.
160  - Marshal and Unmarshal are functions to encode and decode the wire format.
161
162When the .proto file specifies `syntax="proto3"`, there are some differences:
163
164  - Non-repeated fields of non-message type are values instead of pointers.
165  - Enum types do not get an Enum method.
166
167Consider file test.proto, containing
168
169```proto
170	syntax = "proto2";
171	package example;
172
173	enum FOO { X = 17; };
174
175	message Test {
176	  required string label = 1;
177	  optional int32 type = 2 [default=77];
178	  repeated int64 reps = 3;
179	}
180```
181
182To create and play with a Test object from the example package,
183
184```go
185	package main
186
187	import (
188		"log"
189
190		"github.com/gogo/protobuf/proto"
191		"path/to/example"
192	)
193
194	func main() {
195		test := &example.Test{
196			Label: proto.String("hello"),
197			Type:  proto.Int32(17),
198			Reps:  []int64{1, 2, 3},
199		}
200		data, err := proto.Marshal(test)
201		if err != nil {
202			log.Fatal("marshaling error: ", err)
203		}
204		newTest := &example.Test{}
205		err = proto.Unmarshal(data, newTest)
206		if err != nil {
207			log.Fatal("unmarshaling error: ", err)
208		}
209		// Now test and newTest contain the same data.
210		if test.GetLabel() != newTest.GetLabel() {
211			log.Fatalf("data mismatch %q != %q", test.GetLabel(), newTest.GetLabel())
212		}
213		// etc.
214	}
215```
216
217
218## Parameters ##
219
220To pass extra parameters to the plugin, use a comma-separated
221parameter list separated from the output directory by a colon:
222
223
224	protoc --gogo_out=plugins=grpc,import_path=mypackage:. *.proto
225
226- `paths=(import | source_relative)` - specifies how the paths of
227  generated files are structured. See the "Packages and imports paths"
228  section above. The default is `import`.
229- `plugins=plugin1+plugin2` - specifies the list of sub-plugins to
230  load. The only plugin in this repo is `grpc`.
231- `Mfoo/bar.proto=quux/shme` - declares that foo/bar.proto is
232  associated with Go package quux/shme.  This is subject to the
233  import_prefix parameter.
234
235The following parameters are deprecated and should not be used:
236
237- `import_prefix=xxx` - a prefix that is added onto the beginning of
238  all imports.
239- `import_path=foo/bar` - used as the package if no input files
240  declare `go_package`. If it contains slashes, everything up to the
241  rightmost slash is ignored.
242
243## gRPC Support ##
244
245If a proto file specifies RPC services, protoc-gen-go can be instructed to
246generate code compatible with gRPC (http://www.grpc.io/). To do this, pass
247the `plugins` parameter to protoc-gen-go; the usual way is to insert it into
248the --go_out argument to protoc:
249
250	protoc --gogo_out=plugins=grpc:. *.proto
251
252## Compatibility ##
253
254The library and the generated code are expected to be stable over time.
255However, we reserve the right to make breaking changes without notice for the
256following reasons:
257
258- Security. A security issue in the specification or implementation may come to
259  light whose resolution requires breaking compatibility. We reserve the right
260  to address such security issues.
261- Unspecified behavior.  There are some aspects of the Protocol Buffers
262  specification that are undefined.  Programs that depend on such unspecified
263  behavior may break in future releases.
264- Specification errors or changes. If it becomes necessary to address an
265  inconsistency, incompleteness, or change in the Protocol Buffers
266  specification, resolving the issue could affect the meaning or legality of
267  existing programs.  We reserve the right to address such issues, including
268  updating the implementations.
269- Bugs.  If the library has a bug that violates the specification, a program
270  that depends on the buggy behavior may break if the bug is fixed.  We reserve
271  the right to fix such bugs.
272- Adding methods or fields to generated structs.  These may conflict with field
273  names that already exist in a schema, causing applications to break.  When the
274  code generator encounters a field in the schema that would collide with a
275  generated field or method name, the code generator will append an underscore
276  to the generated field or method name.
277- Adding, removing, or changing methods or fields in generated structs that
278  start with `XXX`.  These parts of the generated code are exported out of
279  necessity, but should not be considered part of the public API.
280- Adding, removing, or changing unexported symbols in generated code.
281
282Any breaking changes outside of these will be announced 6 months in advance to
283protobuf@googlegroups.com.
284
285You should, whenever possible, use generated code created by the `protoc-gen-go`
286tool built at the same commit as the `proto` package.  The `proto` package
287declares package-level constants in the form `ProtoPackageIsVersionX`.
288Application code and generated code may depend on one of these constants to
289ensure that compilation will fail if the available version of the proto library
290is too old.  Whenever we make a change to the generated code that requires newer
291library support, in the same commit we will increment the version number of the
292generated code and declare a new package-level constant whose name incorporates
293the latest version number.  Removing a compatibility constant is considered a
294breaking change and would be subject to the announcement policy stated above.
295
296The `protoc-gen-go/generator` package exposes a plugin interface,
297which is used by the gRPC code generation. This interface is not
298supported and is subject to incompatible changes without notice.
299

Readme.md

1# Protocol Buffers for Go with Gadgets
2
3[![Build Status](https://travis-ci.org/gogo/protobuf.svg?branch=master)](https://travis-ci.org/gogo/protobuf)
4[![GoDoc](https://godoc.org/github.com/gogo/protobuf?status.svg)](http://godoc.org/github.com/gogo/protobuf)
5
6gogoprotobuf is a fork of <a href="https://github.com/golang/protobuf">golang/protobuf</a> with extra code generation features.
7
8This code generation is used to achieve:
9
10  - fast marshalling and unmarshalling
11  - more canonical Go structures
12  - goprotobuf compatibility
13  - less typing by optionally generating extra helper code
14  - peace of mind by optionally generating test and benchmark code
15  - other serialization formats
16
17Keeping track of how up to date gogoprotobuf is relative to golang/protobuf is done in this
18<a href="https://github.com/gogo/protobuf/issues/191">issue</a>
19
20## Users
21
22These projects use gogoprotobuf:
23
24  - <a href="http://godoc.org/github.com/coreos/etcd">etcd</a> - <a href="https://blog.gopheracademy.com/advent-2015/etcd-distributed-key-value-store-with-grpc-http2/">blog</a> - <a href="https://github.com/coreos/etcd/blob/master/etcdserver/etcdserverpb/etcdserver.proto">sample proto file</a>
25  - <a href="https://www.spacemonkey.com/">spacemonkey</a> - <a href="https://www.spacemonkey.com/blog/posts/go-space-monkey">blog</a>
26  - <a href="http://badoo.com">badoo</a> - <a href="https://github.com/badoo/lsd/blob/32061f501c5eca9c76c596d790b450501ba27b2f/proto/lsd.proto">sample proto file</a>
27  - <a href="https://github.com/mesos/mesos-go">mesos-go</a> - <a href="https://github.com/mesos/mesos-go/blob/f9e5fb7c2f50ab5f23299f26b6b07c5d6afdd252/api/v0/mesosproto/authentication.proto">sample proto file</a>
28  - <a href="https://github.com/mozilla-services/heka">heka</a> - <a href="https://github.com/mozilla-services/heka/commit/eb72fbf7d2d28249fbaf8d8dc6607f4eb6f03351">the switch from golang/protobuf to gogo/protobuf when it was still on code.google.com</a>
29  - <a href="https://github.com/cockroachdb/cockroach">cockroachdb</a> - <a href="https://github.com/cockroachdb/cockroach/blob/651d54d393e391a30154e9117ab4b18d9ee6d845/roachpb/metadata.proto">sample proto file</a>
30  - <a href="https://github.com/jbenet/go-ipfs">go-ipfs</a> - <a href="https://github.com/ipfs/go-ipfs/blob/2b6da0c024f28abeb16947fb452787196a6b56a2/merkledag/pb/merkledag.proto">sample proto file</a>
31  - <a href="https://github.com/philhofer/rkive">rkive-go</a> - <a href="https://github.com/philhofer/rkive/blob/e5dd884d3ea07b341321073882ae28aa16dd11be/rpbc/riak_dt.proto">sample proto file</a>
32  - <a href="https://www.dropbox.com">dropbox</a>
33  - <a href="https://srclib.org/">srclib</a> - <a href="https://github.com/sourcegraph/srclib/blob/6538858f0c410cac5c63440317b8d009e889d3fb/graph/def.proto">sample proto file</a>
34  - <a href="http://www.adyoulike.com/">adyoulike</a>
35  - <a href="http://www.cloudfoundry.org/">cloudfoundry</a> - <a href="https://github.com/cloudfoundry/bbs/blob/d673710b8c4211037805129944ee4c5373d6588a/models/events.proto">sample proto file</a>
36  - <a href="http://kubernetes.io/">kubernetes</a> - <a href="https://github.com/kubernetes/kubernetes/tree/88d8628137f94ee816aaa6606ae8cd045dee0bff/cmd/libs/go2idl">go2idl built on top of gogoprotobuf</a>
37  - <a href="https://dgraph.io/">dgraph</a> - <a href="https://github.com/dgraph-io/dgraph/releases/tag/v0.4.3">release notes</a> - <a href="https://discuss.dgraph.io/t/gogoprotobuf-is-extremely-fast/639">benchmarks</a></a>
38  - <a href="https://github.com/centrifugal/centrifugo">centrifugo</a> - <a href="https://forum.golangbridge.org/t/centrifugo-real-time-messaging-websocket-or-sockjs-server-v1-5-0-released/2861">release notes</a> - <a href="https://medium.com/@fzambia/centrifugo-protobuf-inside-json-outside-21d39bdabd68#.o3icmgjqd">blog</a>
39  - <a href="https://github.com/docker/swarmkit">docker swarmkit</a> - <a href="https://github.com/docker/swarmkit/blob/63600e01af3b8da2a0ed1c9fa6e1ae4299d75edb/api/objects.proto">sample proto file</a>
40  - <a href="https://nats.io/">nats.io</a> - <a href="https://github.com/nats-io/go-nats-streaming/blob/master/pb/protocol.proto">go-nats-streaming</a>
41  - <a href="https://github.com/pingcap/tidb">tidb</a> - Communication between <a href="https://github.com/pingcap/tipb/blob/master/generate-go.sh#L4">tidb</a> and <a href="https://github.com/pingcap/kvproto/blob/master/generate_go.sh#L3">tikv</a>
42  - <a href="https://github.com/AsynkronIT/protoactor-go">protoactor-go</a> - <a href="https://github.com/AsynkronIT/protoactor-go/blob/master/protobuf/protoc-gen-protoactor/main.go">vanity command</a> that also generates actors from service definitions
43  - <a href="https://containerd.io/">containerd</a> - <a href="https://github.com/containerd/containerd/tree/master/cmd/protoc-gen-gogoctrd">vanity command with custom field names</a> that conforms to the golang convention.
44  - <a href="https://github.com/heroiclabs/nakama">nakama</a>
45  - <a href="https://github.com/src-d/proteus">proteus</a>
46  - <a href="https://github.com/go-graphite">carbonzipper stack</a>
47  - <a href="https://sendgrid.com/">sendgrid</a>
48  - <a href="https://github.com/zero-os/0-stor">zero-os/0-stor</a>
49  - <a href="https://github.com/spacemeshos/go-spacemesh">go-spacemesh</a>
50  - <a href="https://github.com/weaveworks/cortex">cortex</a> - <a href="https://github.com/weaveworks/cortex/blob/fee02a59729d3771ef888f7bf0fd050e1197c56e/pkg/ingester/client/cortex.proto">sample proto file</a>
51  - <a href="http://skywalking.apache.org/">Apache SkyWalking APM</a> - Istio telemetry receiver based on Mixer bypass protocol
52
53Please let us know if you are using gogoprotobuf by posting on our <a href="https://groups.google.com/forum/#!topic/gogoprotobuf/Brw76BxmFpQ">GoogleGroup</a>.
54
55### Mentioned
56
57  - <a href="http://www.slideshare.net/albertstrasheim/serialization-in-go">Cloudflare - go serialization talk - Albert Strasheim</a>
58  - <a href="https://youtu.be/4xB46Xl9O9Q?t=557">GopherCon 2014 Writing High Performance Databases in Go by Ben Johnson</a>
59  - <a href="https://github.com/alecthomas/go_serialization_benchmarks">alecthomas' go serialization benchmarks</a>
60  - <a href="http://agniva.me/go/2017/11/18/gogoproto.html">Go faster with gogoproto - Agniva De Sarker</a>
61  - <a href="https://www.youtube.com/watch?v=CY9T020HLP8">Evolution of protobuf (Gource Visualization) - Landon Wilkins</a>
62  - <a href="https://fosdem.org/2018/schedule/event/gopherjs/">Creating GopherJS Apps with gRPC-Web - Johan Brandhorst</a>
63  - <a href="https://jbrandhorst.com/post/gogoproto/">So you want to use GoGo Protobuf - Johan Brandhorst</a>
64  - <a href="https://jbrandhorst.com/post/grpc-errors/">Advanced gRPC Error Usage - Johan Brandhorst</a>
65  - <a href="https://www.udemy.com/grpc-golang/?couponCode=GITHUB10">gRPC Golang Course on Udemy - Stephane Maarek</a>
66
67## Getting Started
68
69There are several ways to use gogoprotobuf, but for all you need to install go and protoc.
70After that you can choose:
71
72  - Speed
73  - More Speed and more generated code
74  - Most Speed and most customization
75
76### Installation
77
78To install it, you must first have Go (at least version 1.6.3 or 1.9 if you are using gRPC) installed (see [http://golang.org/doc/install](http://golang.org/doc/install)).
79Latest patch versions of 1.9 and 1.10 are continuously tested.
80
81Next, install the standard protocol buffer implementation from [https://github.com/google/protobuf](https://github.com/google/protobuf).
82Most versions from 2.3.1 should not give any problems, but 2.6.1, 3.0.2 and 3.5.1 are continuously tested.
83
84### Speed
85
86Install the protoc-gen-gofast binary
87
88    go get github.com/gogo/protobuf/protoc-gen-gofast
89
90Use it to generate faster marshaling and unmarshaling go code for your protocol buffers.
91
92    protoc --gofast_out=. myproto.proto
93
94This does not allow you to use any of the other gogoprotobuf [extensions](https://github.com/gogo/protobuf/blob/master/extensions.md).
95
96### More Speed and more generated code
97
98Fields without pointers cause less time in the garbage collector.
99More code generation results in more convenient methods.
100
101Other binaries are also included:
102
103    protoc-gen-gogofast (same as gofast, but imports gogoprotobuf)
104    protoc-gen-gogofaster (same as gogofast, without XXX_unrecognized, less pointer fields)
105    protoc-gen-gogoslick (same as gogofaster, but with generated string, gostring and equal methods)
106
107Installing any of these binaries is easy.  Simply run:
108
109    go get github.com/gogo/protobuf/proto
110    go get github.com/gogo/protobuf/{binary}
111    go get github.com/gogo/protobuf/gogoproto
112
113These binaries allow you to use gogoprotobuf [extensions](https://github.com/gogo/protobuf/blob/master/extensions.md). You can also use your own binary.
114
115To generate the code, you also need to set the include path properly.
116
117    protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=. myproto.proto
118
119To use proto files from "google/protobuf" you need to add additional args to protoc.
120
121    protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=\
122    Mgoogle/protobuf/any.proto=github.com/gogo/protobuf/types,\
123    Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,\
124    Mgoogle/protobuf/struct.proto=github.com/gogo/protobuf/types,\
125    Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,\
126    Mgoogle/protobuf/wrappers.proto=github.com/gogo/protobuf/types:. \
127    myproto.proto
128
129Note that in the protoc command, {binary} does not contain the initial prefix of "protoc-gen".
130
131### Most Speed and most customization
132
133Customizing the fields of the messages to be the fields that you actually want to use removes the need to copy between the structs you use and structs you use to serialize.
134gogoprotobuf also offers more serialization formats and generation of tests and even more methods.
135
136Please visit the [extensions](https://github.com/gogo/protobuf/blob/master/extensions.md) page for more documentation.
137
138Install protoc-gen-gogo:
139
140    go get github.com/gogo/protobuf/proto
141    go get github.com/gogo/protobuf/jsonpb
142    go get github.com/gogo/protobuf/protoc-gen-gogo
143    go get github.com/gogo/protobuf/gogoproto
144
145## GRPC
146
147It works the same as golang/protobuf, simply specify the plugin.
148Here is an example using gofast:
149
150    protoc --gofast_out=plugins=grpc:. my.proto
151
152See [https://github.com/gogo/grpc-example](https://github.com/gogo/grpc-example) for an example of using gRPC with gogoprotobuf and the wider grpc-ecosystem.
153
154
155## License
156This software is licensed under the 3-Clause BSD License
157("BSD License 2.0", "Revised BSD License", "New BSD License", or "Modified BSD License").
158
159
160