1# Protocol Buffers for Go with Gadgets 2 3[](https://travis-ci.org/gogo/protobuf) 4 5gogoprotobuf is a fork of <a href="https://github.com/golang/protobuf">golang/protobuf</a> with extra code generation features. 6 7This code generation is used to achieve: 8 9 - fast marshalling and unmarshalling 10 - more canonical Go structures 11 - goprotobuf compatibility 12 - less typing by optionally generating extra helper code 13 - peace of mind by optionally generating test and benchmark code 14 - other serialization formats 15 16Keeping track of how up to date gogoprotobuf is relative to golang/protobuf is done in this 17<a href="https://github.com/gogo/protobuf/issues/191">issue</a> 18 19## Users 20 21These projects use gogoprotobuf: 22 23 - <a href="http://godoc.org/github.com/coreos/etcd">etcd</a> - <a href="https://blog.gopheracademy.com/advent-2015/etcd-distributed-key-value-store-with-grpc-http2/">blog</a> - <a href="https://github.com/coreos/etcd/blob/master/etcdserver/etcdserverpb/etcdserver.proto">sample proto file</a> 24 - <a href="https://www.spacemonkey.com/">spacemonkey</a> - <a href="https://www.spacemonkey.com/blog/posts/go-space-monkey">blog</a> 25 - <a href="http://badoo.com">badoo</a> - <a href="https://github.com/badoo/lsd/blob/32061f501c5eca9c76c596d790b450501ba27b2f/proto/lsd.proto">sample proto file</a> 26 - <a href="https://github.com/mesos/mesos-go">mesos-go</a> - <a href="https://github.com/mesos/mesos-go/blob/f9e5fb7c2f50ab5f23299f26b6b07c5d6afdd252/api/v0/mesosproto/authentication.proto">sample proto file</a> 27 - <a href="https://github.com/mozilla-services/heka">heka</a> - <a href="https://github.com/mozilla-services/heka/commit/eb72fbf7d2d28249fbaf8d8dc6607f4eb6f03351">the switch from golang/protobuf to gogo/protobuf when it was still on code.google.com</a> 28 - <a href="https://github.com/cockroachdb/cockroach">cockroachdb</a> - <a href="https://github.com/cockroachdb/cockroach/blob/651d54d393e391a30154e9117ab4b18d9ee6d845/roachpb/metadata.proto">sample proto file</a> 29 - <a href="https://github.com/jbenet/go-ipfs">go-ipfs</a> - <a href="https://github.com/ipfs/go-ipfs/blob/2b6da0c024f28abeb16947fb452787196a6b56a2/merkledag/pb/merkledag.proto">sample proto file</a> 30 - <a href="https://github.com/philhofer/rkive">rkive-go</a> - <a href="https://github.com/philhofer/rkive/blob/e5dd884d3ea07b341321073882ae28aa16dd11be/rpbc/riak_dt.proto">sample proto file</a> 31 - <a href="https://www.dropbox.com">dropbox</a> 32 - <a href="https://srclib.org/">srclib</a> - <a href="https://github.com/sourcegraph/srclib/blob/6538858f0c410cac5c63440317b8d009e889d3fb/graph/def.proto">sample proto file</a> 33 - <a href="http://www.adyoulike.com/">adyoulike</a> 34 - <a href="http://www.cloudfoundry.org/">cloudfoundry</a> - <a href="https://github.com/cloudfoundry/bbs/blob/d673710b8c4211037805129944ee4c5373d6588a/models/events.proto">sample proto file</a> 35 - <a href="http://kubernetes.io/">kubernetes</a> - <a href="https://github.com/kubernetes/kubernetes/tree/88d8628137f94ee816aaa6606ae8cd045dee0bff/cmd/libs/go2idl">go2idl built on top of gogoprotobuf</a> 36 - <a href="https://dgraph.io/">dgraph</a> - <a href="https://github.com/dgraph-io/dgraph/releases/tag/v0.4.3">release notes</a> - <a href="https://discuss.dgraph.io/t/gogoprotobuf-is-extremely-fast/639">benchmarks</a></a> 37 - <a href="https://github.com/centrifugal/centrifugo">centrifugo</a> - <a href="https://forum.golangbridge.org/t/centrifugo-real-time-messaging-websocket-or-sockjs-server-v1-5-0-released/2861">release notes</a> - <a href="https://medium.com/@fzambia/centrifugo-protobuf-inside-json-outside-21d39bdabd68#.o3icmgjqd">blog</a> 38 - <a href="https://github.com/docker/swarmkit">docker swarmkit</a> - <a href="https://github.com/docker/swarmkit/blob/63600e01af3b8da2a0ed1c9fa6e1ae4299d75edb/api/objects.proto">sample proto file</a> 39 - <a href="https://nats.io/">nats.io</a> - <a href="https://github.com/nats-io/go-nats-streaming/blob/master/pb/protocol.proto">go-nats-streaming</a> 40 - <a href="https://github.com/pingcap/tidb">tidb</a> - Communication between <a href="https://github.com/pingcap/tipb/blob/master/generate-go.sh#L4">tidb</a> and <a href="https://github.com/pingcap/kvproto/blob/master/generate_go.sh#L3">tikv</a> 41 - <a href="https://github.com/AsynkronIT/protoactor-go">protoactor-go</a> - <a href="https://github.com/AsynkronIT/protoactor-go/blob/master/protobuf/protoc-gen-protoactor/main.go">vanity command</a> that also generates actors from service definitions 42 - <a href="https://containerd.io/">containerd</a> - <a href="https://github.com/containerd/containerd/tree/master/cmd/protoc-gen-gogoctrd">vanity command with custom field names</a> that conforms to the golang convention. 43 - <a href="https://github.com/heroiclabs/nakama">nakama</a> 44 - <a href="https://github.com/src-d/proteus">proteus</a> 45 - <a href="https://github.com/go-graphite">carbonzipper stack</a> 46 - <a href="https://sendgrid.com/">sendgrid</a> 47 - <a href="https://github.com/zero-os/0-stor">zero-os/0-stor</a> 48 - <a href="https://github.com/spacemeshos/go-spacemesh">go-spacemesh</a> 49 50Please let us know if you are using gogoprotobuf by posting on our <a href="https://groups.google.com/forum/#!topic/gogoprotobuf/Brw76BxmFpQ">GoogleGroup</a>. 51 52### Mentioned 53 54 - <a href="http://www.slideshare.net/albertstrasheim/serialization-in-go">Cloudflare - go serialization talk - Albert Strasheim</a> 55 - <a href="https://youtu.be/4xB46Xl9O9Q?t=557">GopherCon 2014 Writing High Performance Databases in Go by Ben Johnson</a> 56 - <a href="https://github.com/alecthomas/go_serialization_benchmarks">alecthomas' go serialization benchmarks</a> 57 - <a href="http://agniva.me/go/2017/11/18/gogoproto.html">Go faster with gogoproto - Agniva De Sarker</a> 58 - <a href="https://www.youtube.com/watch?v=CY9T020HLP8">Evolution of protobuf (Gource Visualization) - Landon Wilkins</a> 59 - <a href="https://fosdem.org/2018/schedule/event/gopherjs/">Creating GopherJS Apps with gRPC-Web - Johan Brandhorst</a> 60 - <a href="https://jbrandhorst.com/post/gogoproto/">So you want to use GoGo Protobuf - Johan Brandhorst</a> 61 - <a href="https://jbrandhorst.com/post/grpc-errors/">Advanced gRPC Error Usage - Johan Brandhorst</a> 62 63## Getting Started 64 65There are several ways to use gogoprotobuf, but for all you need to install go and protoc. 66After that you can choose: 67 68 - Speed 69 - More Speed and more generated code 70 - Most Speed and most customization 71 72### Installation 73 74To install it, you must first have Go (at least version 1.6.3 or 1.9 if you are using gRPC) installed (see [http://golang.org/doc/install](http://golang.org/doc/install)). 75Latest patch versions of 1.9 and 1.10 are continuously tested. 76 77Next, install the standard protocol buffer implementation from [https://github.com/google/protobuf](https://github.com/google/protobuf). 78Most versions from 2.3.1 should not give any problems, but 2.6.1, 3.0.2 and 3.5.1 are continuously tested. 79 80### Speed 81 82Install the protoc-gen-gofast binary 83 84 go get github.com/gogo/protobuf/protoc-gen-gofast 85 86Use it to generate faster marshaling and unmarshaling go code for your protocol buffers. 87 88 protoc --gofast_out=. myproto.proto 89 90This does not allow you to use any of the other gogoprotobuf [extensions](https://github.com/gogo/protobuf/blob/master/extensions.md). 91 92### More Speed and more generated code 93 94Fields without pointers cause less time in the garbage collector. 95More code generation results in more convenient methods. 96 97Other binaries are also included: 98 99 protoc-gen-gogofast (same as gofast, but imports gogoprotobuf) 100 protoc-gen-gogofaster (same as gogofast, without XXX_unrecognized, less pointer fields) 101 protoc-gen-gogoslick (same as gogofaster, but with generated string, gostring and equal methods) 102 103Installing any of these binaries is easy. Simply run: 104 105 go get github.com/gogo/protobuf/proto 106 go get github.com/gogo/protobuf/{binary} 107 go get github.com/gogo/protobuf/gogoproto 108 109These binaries allow you to use gogoprotobuf [extensions](https://github.com/gogo/protobuf/blob/master/extensions.md). You can also use your own binary. 110 111To generate the code, you also need to set the include path properly. 112 113 protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=. myproto.proto 114 115To use proto files from "google/protobuf" you need to add additional args to protoc. 116 117 protoc -I=. -I=$GOPATH/src -I=$GOPATH/src/github.com/gogo/protobuf/protobuf --{binary}_out=\ 118 Mgoogle/protobuf/any.proto=github.com/gogo/protobuf/types,\ 119 Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,\ 120 Mgoogle/protobuf/struct.proto=github.com/gogo/protobuf/types,\ 121 Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types,\ 122 Mgoogle/protobuf/wrappers.proto=github.com/gogo/protobuf/types:. \ 123 myproto.proto 124 125Note that in the protoc command, {binary} does not contain the initial prefix of "protoc-gen". 126 127### Most Speed and most customization 128 129Customizing the fields of the messages to be the fields that you actually want to use removes the need to copy between the structs you use and structs you use to serialize. 130gogoprotobuf also offers more serialization formats and generation of tests and even more methods. 131 132Please visit the [extensions](https://github.com/gogo/protobuf/blob/master/extensions.md) page for more documentation. 133 134Install protoc-gen-gogo: 135 136 go get github.com/gogo/protobuf/proto 137 go get github.com/gogo/protobuf/jsonpb 138 go get github.com/gogo/protobuf/protoc-gen-gogo 139 go get github.com/gogo/protobuf/gogoproto 140 141## GRPC 142 143It works the same as golang/protobuf, simply specify the plugin. 144Here is an example using gofast: 145 146 protoc --gofast_out=plugins=grpc:. my.proto 147 148See [https://github.com/gogo/grpc-example](https://github.com/gogo/grpc-example) for an example of using gRPC with gogoprotobuf and the wider grpc-ecosystem. 149 150 151 152
){kind=link}