README.md
1# Go Plugin System over RPC
2
3`go-plugin` is a Go (golang) plugin system over RPC. It is the plugin system
4that has been in use by HashiCorp tooling for over 4 years. While initially
5created for [Packer](https://www.packer.io), it is additionally in use by
6[Terraform](https://www.terraform.io), [Nomad](https://www.nomadproject.io), and
7[Vault](https://www.vaultproject.io).
8
9While the plugin system is over RPC, it is currently only designed to work
10over a local [reliable] network. Plugins over a real network are not supported
11and will lead to unexpected behavior.
12
13This plugin system has been used on millions of machines across many different
14projects and has proven to be battle hardened and ready for production use.
15
16## Features
17
18The HashiCorp plugin system supports a number of features:
19
20**Plugins are Go interface implementations.** This makes writing and consuming
21plugins feel very natural. To a plugin author: you just implement an
22interface as if it were going to run in the same process. For a plugin user:
23you just use and call functions on an interface as if it were in the same
24process. This plugin system handles the communication in between.
25
26**Cross-language support.** Plugins can be written (and consumed) by
27almost every major language. This library supports serving plugins via
28[gRPC](http://www.grpc.io). gRPC-based plugins enable plugins to be written
29in any language.
30
31**Complex arguments and return values are supported.** This library
32provides APIs for handling complex arguments and return values such
33as interfaces, `io.Reader/Writer`, etc. We do this by giving you a library
34(`MuxBroker`) for creating new connections between the client/server to
35serve additional interfaces or transfer raw data.
36
37**Bidirectional communication.** Because the plugin system supports
38complex arguments, the host process can send it interface implementations
39and the plugin can call back into the host process.
40
41**Built-in Logging.** Any plugins that use the `log` standard library
42will have log data automatically sent to the host process. The host
43process will mirror this output prefixed with the path to the plugin
44binary. This makes debugging with plugins simple. If the host system
45uses [hclog](https://github.com/hashicorp/go-hclog) then the log data
46will be structured. If the plugin also uses hclog, logs from the plugin
47will be sent to the host hclog and be structured.
48
49**Protocol Versioning.** A very basic "protocol version" is supported that
50can be incremented to invalidate any previous plugins. This is useful when
51interface signatures are changing, protocol level changes are necessary,
52etc. When a protocol version is incompatible, a human friendly error
53message is shown to the end user.
54
55**Stdout/Stderr Syncing.** While plugins are subprocesses, they can continue
56to use stdout/stderr as usual and the output will get mirrored back to
57the host process. The host process can control what `io.Writer` these
58streams go to to prevent this from happening.
59
60**TTY Preservation.** Plugin subprocesses are connected to the identical
61stdin file descriptor as the host process, allowing software that requires
62a TTY to work. For example, a plugin can execute `ssh` and even though there
63are multiple subprocesses and RPC happening, it will look and act perfectly
64to the end user.
65
66**Host upgrade while a plugin is running.** Plugins can be "reattached"
67so that the host process can be upgraded while the plugin is still running.
68This requires the host/plugin to know this is possible and daemonize
69properly. `NewClient` takes a `ReattachConfig` to determine if and how to
70reattach.
71
72**Cryptographically Secure Plugins.** Plugins can be verified with an expected
73checksum and RPC communications can be configured to use TLS. The host process
74must be properly secured to protect this configuration.
75
76## Architecture
77
78The HashiCorp plugin system works by launching subprocesses and communicating
79over RPC (using standard `net/rpc` or [gRPC](http://www.grpc.io)). A single
80connection is made between any plugin and the host process. For net/rpc-based
81plugins, we use a [connection multiplexing](https://github.com/hashicorp/yamux)
82library to multiplex any other connections on top. For gRPC-based plugins,
83the HTTP2 protocol handles multiplexing.
84
85This architecture has a number of benefits:
86
87 * Plugins can't crash your host process: A panic in a plugin doesn't
88 panic the plugin user.
89
90 * Plugins are very easy to write: just write a Go application and `go build`.
91 Or use any other language to write a gRPC server with a tiny amount of
92 boilerplate to support go-plugin.
93
94 * Plugins are very easy to install: just put the binary in a location where
95 the host will find it (depends on the host but this library also provides
96 helpers), and the plugin host handles the rest.
97
98 * Plugins can be relatively secure: The plugin only has access to the
99 interfaces and args given to it, not to the entire memory space of the
100 process. Additionally, go-plugin can communicate with the plugin over
101 TLS.
102
103## Usage
104
105To use the plugin system, you must take the following steps. These are
106high-level steps that must be done. Examples are available in the
107`examples/` directory.
108
109 1. Choose the interface(s) you want to expose for plugins.
110
111 2. For each interface, implement an implementation of that interface
112 that communicates over a `net/rpc` connection or over a
113 [gRPC](http://www.grpc.io) connection or both. You'll have to implement
114 both a client and server implementation.
115
116 3. Create a `Plugin` implementation that knows how to create the RPC
117 client/server for a given plugin type.
118
119 4. Plugin authors call `plugin.Serve` to serve a plugin from the
120 `main` function.
121
122 5. Plugin users use `plugin.Client` to launch a subprocess and request
123 an interface implementation over RPC.
124
125That's it! In practice, step 2 is the most tedious and time consuming step.
126Even so, it isn't very difficult and you can see examples in the `examples/`
127directory as well as throughout our various open source projects.
128
129For complete API documentation, see [GoDoc](https://godoc.org/github.com/hashicorp/go-plugin).
130
131## Roadmap
132
133Our plugin system is constantly evolving. As we use the plugin system for
134new projects or for new features in existing projects, we constantly find
135improvements we can make.
136
137At this point in time, the roadmap for the plugin system is:
138
139**Semantic Versioning.** Plugins will be able to implement a semantic version.
140This plugin system will give host processes a system for constraining
141versions. This is in addition to the protocol versioning already present
142which is more for larger underlying changes.
143
144## What About Shared Libraries?
145
146When we started using plugins (late 2012, early 2013), plugins over RPC
147were the only option since Go didn't support dynamic library loading. Today,
148Go supports the [plugin](https://golang.org/pkg/plugin/) standard library with
149a number of limitations. Since 2012, our plugin system has stabilized
150from tens of millions of users using it, and has many benefits we've come to
151value greatly.
152
153For example, we use this plugin system in
154[Vault](https://www.vaultproject.io) where dynamic library loading is
155not acceptable for security reasons. That is an extreme
156example, but we believe our library system has more upsides than downsides
157over dynamic library loading and since we've had it built and tested for years,
158we'll continue to use it.
159
160Shared libraries have one major advantage over our system which is much
161higher performance. In real world scenarios across our various tools,
162we've never required any more performance out of our plugin system and it
163has seen very high throughput, so this isn't a concern for us at the moment.
164