1# Consul Template 2 3[![CircleCI](https://circleci.com/gh/hashicorp/consul-template.svg?style=svg)](https://circleci.com/gh/hashicorp/consul-template) 4[![Go Documentation](http://img.shields.io/badge/go-documentation-%2300acd7)](https://godoc.org/github.com/hashicorp/consul-template) 5 6This project provides a convenient way to populate values from [Consul][consul] 7into the file system using the `consul-template` daemon. 8 9The daemon `consul-template` queries a [Consul][consul] or [Vault][vault] 10cluster and updates any number of specified templates on the file system. As an 11added bonus, it can optionally run arbitrary commands when the update process 12completes. Please see the [examples folder][examples] for some scenarios where 13this functionality might prove useful. 14 15--- 16 17**The documentation in this README corresponds to the master branch of Consul Template. It may contain unreleased features or different APIs than the most recently released version.** 18 19**Please see the [Git tag](https://github.com/hashicorp/consul-template/releases) that corresponds to your version of Consul Template for the proper documentation.** 20 21--- 22 23## Table of Contents 24 25- [Community Support](#community-support) 26- [Installation](#installation) 27- [Quick Example](#quick-example) 28- [Learn Guides](#learn-guides) 29- [Configuration](docs/configuration.md) 30- [Command Line Flags](docs/configuration.md#command-line-flags) 31- [Configuration File](docs/configuration.md#configuration-file) 32- [Reload Configuration and Templates](#reload-configuration-and-templates) 33- [Templating Language](docs/templating-language.md) 34- [API Functions](docs/templating-language.md#api-functions) 35- [Scratch](docs/templating-language.md#scratch) 36- [Helper Functions](docs/templating-language.md#helper-functions) 37- [Math Functions](docs/templating-language.md#math-functions) 38- [Observability](docs/observability.md) 39- [Logging](docs/observability.md#logging) 40- [Modes](docs/modes.md) 41- [Once Mode](docs/modes.md#once-mode) 42- [De-Duplication Mode](docs/modes.md#de-duplication-mode) 43- [Exec Mode](docs/modes.md#exec-mode) 44- [Plugins](docs/plugins.md) 45- [Caveats](#caveats) 46- [Docker Image Use](#docker-image-use) 47- [Dots in Service Names](#dots-in-service-names) 48- [Termination on Error](#termination-on-error) 49- [Commands](#commands) 50 - [Environment](#environment) 51 - [Multiple Commands](#multiple-commands) 52- [Multi-phase Execution](#multi-phase-execution) 53- [Running and Process Lifecycle](#running-and-process-lifecycle) 54- [Debugging](#debugging) 55- [FAQ](#faq) 56- [Contributing](#contributing) 57 58 59## Community Support 60 61If you have questions about how consul-template works, its capabilities or 62anything other than a bug or feature request (use github's issue tracker for 63those), please see our community support resources. 64 65Community portal: https://discuss.hashicorp.com/c/consul 66 67Other resources: https://www.consul.io/community.html 68 69Additionally, for issues and pull requests, we'll be using the :+1: reactions 70as a rough voting system to help gauge community priorities. So please add :+1: 71to any issue or pull request you'd like to see worked on. Thanks. 72 73 74## Installation 75 761. Download a pre-compiled, released version from the [Consul Template releases page][releases]. 77 781. Extract the binary using `unzip` or `tar`. 79 801. Move the binary into `$PATH`. 81 82To compile from source, please see the instructions in the 83[contributing section](#contributing). 84 85## Quick Example 86 87This short example assumes Consul is installed locally. 88 891. Start a Consul cluster in dev mode: 90 91 ```shell 92 $ consul agent -dev 93 ``` 94 951. Author a template `in.tpl` to query the kv store: 96 97 ```liquid 98 {{ key "foo" }} 99 ``` 100 1011. Start Consul Template: 102 103 ```shell 104 $ consul-template -template "in.tpl:out.txt" -once 105 ``` 106 1071. Write data to the key in Consul: 108 109 ```shell 110 $ consul kv put foo bar 111 ``` 112 1131. Observe Consul Template has written the file `out.txt`: 114 115 ```shell 116 $ cat out.txt 117 bar 118 ``` 119 120For more examples and use cases, please see the [examples folder][examples] in 121this repository. 122 123## Learn Guides 124 125In addition to these [examples][examples], HashiCorp has published guides and 126official documentation to help walk through a few common use cases for Consul 127Template. 128* [Consul KV](https://learn.hashicorp.com/consul/developer-configuration/consul-template#use-case-consul-kv) 129* [Consul Catalog](https://learn.hashicorp.com/consul/developer-configuration/consul-template#use-case-discover-all-services) 130* [Vault Agent Templates](https://learn.hashicorp.com/vault/identity-access-management/agent-templates) 131* [Vault Secrets](https://www.vaultproject.io/docs/agent/template#example-template) 132 133## Configuration 134 135Configuration documentation has been moved to [docs/configuration.md](docs/configuration.md). 136 137## Reload Configuration and Templates 138 139While there are multiple ways to run Consul Template, the most common pattern is 140to run Consul Template as a system service. When Consul Template first starts, 141it reads any configuration files and templates from disk and loads them into 142memory. From that point forward, changes to the files on disk do not propagate 143to running process without a reload. 144 145The reason for this behavior is simple and aligns with other tools like haproxy. 146A user may want to perform pre-flight validation checks on the configuration or 147templates before loading them into the process. Additionally, a user may want to 148update configuration and templates simultaneously. Having Consul Template 149automatically watch and reload those files on changes is both operationally 150dangerous and against some of the paradigms of modern infrastructure. Instead, 151Consul Template listens for the `SIGHUP` syscall to trigger a configuration 152reload. If you update configuration or templates, simply send `HUP` to the 153running Consul Template process and Consul Template will reload all the 154configurations and templates from disk. 155 156## Templating Language 157 158Templating Language documentation has been moved to 159[docs/templating-language.md](docs/templating-language.md). 160 161## Caveats 162 163### Docker Image Use 164 165The Alpine Docker image is configured to support an external volume to render 166shared templates to. If mounted you will need to make sure that the 167consul-template user in the docker image has write permissions to the 168directory. Also if you build your own image using these you need to be sure you 169have the permissions correct. 170 171**The consul-template user in docker has a UID of 100 and a GID of 1000.** 172 173This effects the in image directories /consul-template/config, used to add 174configuration when using this as a parent image, and /consul-template/data, 175exported as a VOLUME as a location to render shared results. 176 177Previously the image initially ran as root in order to ensure the permissions 178allowed it. But this ran against docker best practices and security policies. 179 180If you build your own image based on ours you can override these values with 181`--build-arg` parameters. 182 183### Dots in Service Names 184 185Using dots `.` in service names will conflict with the use of dots for [TAG 186delineation](https://github.com/hashicorp/consul-template#service) in the 187template. Dots already [interfere with using 188DNS](https://www.consul.io/docs/agent/services.html#service-and-tag-names-with-dns) 189for service names, so we recommend avoiding dots wherever possible. 190 191### Termination on Error 192 193By default Consul Template is highly fault-tolerant. If Consul is unreachable or 194a template changes, Consul Template will happily continue running. The only 195exception to this rule is if the optional `command` exits non-zero. In this 196case, Consul Template will also exit non-zero. The reason for this decision is 197so the user can easily configure something like Upstart or God to manage Consul 198Template as a service. 199 200If you want Consul Template to continue watching for changes, even if the 201optional command argument fails, you can append `|| true` to your command. Note 202that `||` is a "shell-ism", not a built-in function. You will also need to run 203your command under a shell: 204 205```shell 206$ consul-template \ 207 -template "in.ctmpl:out.file:/bin/bash -c 'service nginx restart || true'" 208``` 209 210In this example, even if the Nginx restart command returns non-zero, the overall 211function will still return an OK exit code; Consul Template will continue to run 212as a service. Additionally, if you have complex logic for restarting your 213service, you can intelligently choose when you want Consul Template to exit and 214when you want it to continue to watch for changes. For these types of complex 215scripts, we recommend using a custom sh or bash script instead of putting the 216logic directly in the `consul-template` command or configuration file. 217 218### Commands 219 220#### Environment 221 222The current processes environment is used when executing commands with the following additional environment variables: 223 224- `CONSUL_HTTP_ADDR` 225- `CONSUL_HTTP_TOKEN` 226- `CONSUL_HTTP_AUTH` 227- `CONSUL_HTTP_SSL` 228- `CONSUL_HTTP_SSL_VERIFY` 229 230These environment variables are exported with their current values when the 231command executes. Other Consul tooling reads these environment variables, 232providing smooth integration with other Consul tools (like `consul maint` or 233`consul lock`). Additionally, exposing these environment variables gives power 234users the ability to further customize their command script. 235 236#### Multiple Commands 237 238The command configured for running on template rendering must be a single 239command. That is you cannot join multiple commands with `&&`, `;`, `|`, etc. 240This is a restriction of how they are executed. **However** you are able to do 241this by combining the multiple commands in an explicit shell command using `sh 242-c`. This is probably best explained by example. 243 244Say you have a couple scripts you need to run when a template is rendered, 245`/opt/foo` and `/opt/bar`, and you only want `/opt/bar` to run if `/opt/foo` is 246successful. You can do that with the command... 247 248`command = "sh -c '/opt/foo && /opt/bar'"` 249 250As this is a full shell command you can even use conditionals. So accomplishes the same thing. 251 252`command = "sh -c 'if /opt/foo; then /opt/bar ; fi'"` 253 254Using this method you can run as many shell commands as you need with whatever 255logic you need. Though it is suggested that if it gets too long you might want 256to wrap it in a shell script, deploy and run that. 257 258### Multi-phase Execution 259 260Consul Template does an n-pass evaluation of templates, accumulating 261dependencies on each pass. This is required due to nested dependencies, such as: 262 263```liquid 264{{ range services }} 265{{ range service .Name }} 266 {{ .Address }} 267{{ end }}{{ end }} 268``` 269 270During the first pass, Consul Template does not know any of the services in 271Consul, so it has to perform a query. When those results are returned, the 272inner-loop is then evaluated with that result, potentially creating more queries 273and watches. 274 275Because of this implementation, template functions need a default value that is 276an acceptable parameter to a `range` function (or similar), but does not 277actually execute the inner loop (which would cause a panic). This is important 278to mention because complex templates **must** account for the "empty" case. For 279example, the following **will not work**: 280 281```liquid 282{{ with index (service "foo") 0 }} 283# ... 284{{ end }} 285``` 286 287This will raise an error like: 288 289```text 290<index $services 0>: error calling index: index out of range: 0 291``` 292 293That is because, during the _first_ evaluation of the template, the `service` 294key is returning an empty slice. You can account for this in your template like 295so: 296 297```liquid 298{{ with service "foo" }} 299{{ with index . 0 }} 300{{ .Node }}{{ end }}{{ end }} 301``` 302 303This will still add the dependency to the list of watches, but will not 304evaluate the inner-if, avoiding the out-of-index error. 305 306## FAQ 307 308**Q: How is this different than confd?**<br> 309A: The answer is simple: Service Discovery as a first class citizen. You are also encouraged to read [this Pull Request](https://github.com/kelseyhightower/confd/pull/102) on the project for more background information. We think confd is a great project, but Consul Template fills a missing gap. Additionally, Consul Template has first class integration with [Vault][vault], making it easy to incorporate secret material like database credentials or API tokens into configuration files. 310 311**Q: How is this different than Puppet/Chef/Ansible/Salt?**<br> 312A: Configuration management tools are designed to be used in unison with Consul Template. Instead of rendering a stale configuration file, use your configuration management software to render a dynamic template that will be populated by [Consul][consul]. 313 314 315## Contributing 316 317To build and install Consul-Template locally, you will need to [install Go][go]. 318 319Clone the repository: 320 321```shell 322$ git clone https://github.com/hashicorp/consul-template.git 323``` 324 325To compile the `consul-template` binary for your local machine: 326 327```shell 328$ make dev 329``` 330 331This will compile the `consul-template` binary into `bin/consul-template` as 332well as your `$GOPATH` and run the test suite. 333 334If you want to compile a specific binary, set `XC_OS` and `XC_ARCH` or run the 335following to generate all binaries: 336 337```shell 338$ make build 339``` 340 341If you want to run the tests, first install [consul](https://www.consul.io/docs/install/index.html) and [vault](https://www.vaultproject.io/docs/install/) locally, then: 342 343```shell 344$ make test 345``` 346 347Or to run a specific test in the suite: 348 349```shell 350go test ./... -run SomeTestFunction_name 351``` 352 353[consul]: https://www.consul.io "Consul by HashiCorp" 354[connect]: https://www.consul.io/docs/connect/ "Connect" 355[examples]: (https://github.com/hashicorp/consul-template/tree/master/examples) "Consul Template Examples" 356[releases]: https://releases.hashicorp.com/consul-template "Consul Template Releases" 357[vault]: https://www.vaultproject.io "Vault by HashiCorp" 358[go]: https://golang.org "Go programming language" 359