1--- 2layout: "language" 3page_title: "Backend Type: s3" 4sidebar_current: "docs-backends-types-standard-s3" 5description: |- 6 Terraform can store state remotely in S3 and lock that state with DynamoDB. 7--- 8 9# S3 10 11**Kind: Standard (with locking via DynamoDB)** 12 13Stores the state as a given key in a given bucket on 14[Amazon S3](https://aws.amazon.com/s3/). 15This backend also supports state locking and consistency checking via 16[Dynamo DB](https://aws.amazon.com/dynamodb/), which can be enabled by setting 17the `dynamodb_table` field to an existing DynamoDB table name. 18A single DynamoDB table can be used to lock multiple remote state files. Terraform generates key names that include the values of the `bucket` and `key` variables. 19 20~> **Warning!** It is highly recommended that you enable 21[Bucket Versioning](http://docs.aws.amazon.com/AmazonS3/latest/UG/enable-bucket-versioning.html) 22on the S3 bucket to allow for state recovery in the case of accidental deletions and human error. 23 24## Example Configuration 25 26```hcl 27terraform { 28 backend "s3" { 29 bucket = "mybucket" 30 key = "path/to/my/key" 31 region = "us-east-1" 32 } 33} 34``` 35 36This assumes we have a bucket created called `mybucket`. The 37Terraform state is written to the key `path/to/my/key`. 38 39Note that for the access credentials we recommend using a 40[partial configuration](/docs/language/settings/backends/configuration.html#partial-configuration). 41 42### S3 Bucket Permissions 43 44Terraform will need the following AWS IAM permissions on 45the target backend bucket: 46 47* `s3:ListBucket` on `arn:aws:s3:::mybucket` 48* `s3:GetObject` on `arn:aws:s3:::mybucket/path/to/my/key` 49* `s3:PutObject` on `arn:aws:s3:::mybucket/path/to/my/key` 50 51This is seen in the following AWS IAM Statement: 52 53```json 54{ 55 "Version": "2012-10-17", 56 "Statement": [ 57 { 58 "Effect": "Allow", 59 "Action": "s3:ListBucket", 60 "Resource": "arn:aws:s3:::mybucket" 61 }, 62 { 63 "Effect": "Allow", 64 "Action": ["s3:GetObject", "s3:PutObject"], 65 "Resource": "arn:aws:s3:::mybucket/path/to/my/key" 66 } 67 ] 68} 69``` 70 71-> **Note:** AWS can control access to S3 buckets with either IAM policies 72attached to users/groups/roles (like the example above) or resource policies 73attached to bucket objects (which look similar but also require a `Principal` to 74indicate which entity has those permissions). For more details, see Amazon's 75documentation about 76[S3 access control](https://docs.aws.amazon.com/AmazonS3/latest/dev/s3-access-control.html). 77 78### DynamoDB Table Permissions 79 80If you are using state locking, Terraform will need the following AWS IAM 81permissions on the DynamoDB table (`arn:aws:dynamodb:::table/mytable`): 82 83* `dynamodb:GetItem` 84* `dynamodb:PutItem` 85* `dynamodb:DeleteItem` 86 87This is seen in the following AWS IAM Statement: 88 89```json 90{ 91 "Version": "2012-10-17", 92 "Statement": [ 93 { 94 "Effect": "Allow", 95 "Action": [ 96 "dynamodb:GetItem", 97 "dynamodb:PutItem", 98 "dynamodb:DeleteItem" 99 ], 100 "Resource": "arn:aws:dynamodb:*:*:table/mytable" 101 } 102 ] 103} 104``` 105 106## Data Source Configuration 107 108To make use of the S3 remote state in another configuration, use the 109[`terraform_remote_state` data 110source](/docs/language/state/remote-state-data.html). 111 112```hcl 113data "terraform_remote_state" "network" { 114 backend = "s3" 115 config = { 116 bucket = "terraform-state-prod" 117 key = "network/terraform.tfstate" 118 region = "us-east-1" 119 } 120} 121``` 122 123The `terraform_remote_state` data source will return all of the root module 124outputs defined in the referenced remote state (but not any outputs from 125nested modules unless they are explicitly output again in the root). An 126example output might look like: 127 128``` 129data.terraform_remote_state.network: 130 id = 2016-10-29 01:57:59.780010914 +0000 UTC 131 addresses.# = 2 132 addresses.0 = 52.207.220.222 133 addresses.1 = 54.196.78.166 134 backend = s3 135 config.% = 3 136 config.bucket = terraform-state-prod 137 config.key = network/terraform.tfstate 138 config.region = us-east-1 139 elb_address = web-elb-790251200.us-east-1.elb.amazonaws.com 140 public_subnet_id = subnet-1e05dd33 141``` 142 143## Configuration 144 145This backend requires the configuration of the AWS Region and S3 state storage. Other configuration, such as enabling DynamoDB state locking, is optional. 146 147### Credentials and Shared Configuration 148 149The following configuration is required: 150 151* `region` - (Required) AWS Region of the S3 Bucket and DynamoDB Table (if used). This can also be sourced from the `AWS_DEFAULT_REGION` and `AWS_REGION` environment variables. 152 153The following configuration is optional: 154 155* `access_key` - (Optional) AWS access key. If configured, must also configure `secret_key`. This can also be sourced from the `AWS_ACCESS_KEY_ID` environment variable, AWS shared credentials file (e.g. `~/.aws/credentials`), or AWS shared configuration file (e.g. `~/.aws/config`). 156* `secret_key` - (Optional) AWS access key. If configured, must also configure `access_key`. This can also be sourced from the `AWS_SECRET_ACCESS_KEY` environment variable, AWS shared credentials file (e.g. `~/.aws/credentials`), or AWS shared configuration file (e.g. `~/.aws/config`). 157* `iam_endpoint` - (Optional) Custom endpoint for the AWS Identity and Access Management (IAM) API. This can also be sourced from the `AWS_IAM_ENDPOINT` environment variable. 158* `max_retries` - (Optional) The maximum number of times an AWS API request is retried on retryable failure. Defaults to 5. 159* `profile` - (Optional) Name of AWS profile in AWS shared credentials file (e.g. `~/.aws/credentials`) or AWS shared configuration file (e.g. `~/.aws/config`) to use for credentials and/or configuration. This can also be sourced from the `AWS_PROFILE` environment variable. 160* `shared_credentials_file` - (Optional) Path to the AWS shared credentials file. Defaults to `~/.aws/credentials`. 161* `skip_credentials_validation` - (Optional) Skip credentials validation via the STS API. 162* `skip_region_validation` - (Optional) Skip validation of provided region name. 163* `skip_metadata_api_check` - (Optional) Skip usage of EC2 Metadata API. 164* `sts_endpoint` - (Optional) Custom endpoint for the AWS Security Token Service (STS) API. This can also be sourced from the `AWS_STS_ENDPOINT` environment variable. 165* `token` - (Optional) Multi-Factor Authentication (MFA) token. This can also be sourced from the `AWS_SESSION_TOKEN` environment variable. 166 167#### Assume Role Configuration 168 169The following configuration is optional: 170 171* `assume_role_duration_seconds` - (Optional) Number of seconds to restrict the assume role session duration. 172* `assume_role_policy` - (Optional) IAM Policy JSON describing further restricting permissions for the IAM Role being assumed. 173* `assume_role_policy_arns` - (Optional) Set of Amazon Resource Names (ARNs) of IAM Policies describing further restricting permissions for the IAM Role being assumed. 174* `assume_role_tags` - (Optional) Map of assume role session tags. 175* `assume_role_transitive_tag_keys` - (Optional) Set of assume role session tag keys to pass to any subsequent sessions. 176* `external_id` - (Optional) External identifier to use when assuming the role. 177* `role_arn` - (Optional) Amazon Resource Name (ARN) of the IAM Role to assume. 178* `session_name` - (Optional) Session name to use when assuming the role. 179 180### S3 State Storage 181 182The following configuration is required: 183 184* `bucket` - (Required) Name of the S3 Bucket. 185* `key` - (Required) Path to the state file inside the S3 Bucket. When using a non-default [workspace](/docs/language/state/workspaces.html), the state path will be `/workspace_key_prefix/workspace_name/key` (see also the `workspace_key_prefix` configuration). 186 187The following configuration is optional: 188 189* `acl` - (Optional) [Canned ACL](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) to be applied to the state file. 190* `encrypt` - (Optional) Enable [server side encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) of the state file. 191* `endpoint` - (Optional) Custom endpoint for the AWS S3 API. This can also be sourced from the `AWS_S3_ENDPOINT` environment variable. 192* `force_path_style` - (Optional) Enable path-style S3 URLs (`https://<HOST>/<BUCKET>` instead of `https://<BUCKET>.<HOST>`). 193* `kms_key_id` - (Optional) Amazon Resource Name (ARN) of a Key Management Service (KMS) Key to use for encrypting the state. 194* `sse_customer_key` - (Optional) The key to use for encrypting state with [Server-Side Encryption with Customer-Provided Keys (SSE-C)](https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerSideEncryptionCustomerKeys.html). This is the base64-encoded value of the key, which must decode to 256 bits. This can also be sourced from the `AWS_SSE_CUSTOMER_KEY` environment variable, which is recommended due to the sensitivity of the value. Setting it inside a terraform file will cause it to be persisted to disk in `terraform.tfstate`. 195* `workspace_key_prefix` - (Optional) Prefix applied to the state path inside the bucket. This is only relevant when using a non-default workspace. Defaults to `env:`. 196 197### DynamoDB State Locking 198 199The following configuration is optional: 200 201* `dynamodb_endpoint` - (Optional) Custom endpoint for the AWS DynamoDB API. This can also be sourced from the `AWS_DYNAMODB_ENDPOINT` environment variable. 202* `dynamodb_table` - (Optional) Name of DynamoDB Table to use for state locking and consistency. The table must have a primary key named `LockID` with type of `string`. If not configured, state locking will be disabled. 203 204## Multi-account AWS Architecture 205 206A common architectural pattern is for an organization to use a number of 207separate AWS accounts to isolate different teams and environments. For example, 208a "staging" system will often be deployed into a separate AWS account than 209its corresponding "production" system, to minimize the risk of the staging 210environment affecting production infrastructure, whether via rate limiting, 211misconfigured access controls, or other unintended interactions. 212 213The S3 backend can be used in a number of different ways that make different 214tradeoffs between convenience, security, and isolation in such an organization. 215This section describes one such approach that aims to find a good compromise 216between these tradeoffs, allowing use of 217[Terraform's workspaces feature](/docs/language/state/workspaces.html) to switch 218conveniently between multiple isolated deployments of the same configuration. 219 220Use this section as a starting-point for your approach, but note that 221you will probably need to make adjustments for the unique standards and 222regulations that apply to your organization. You will also need to make some 223adjustments to this approach to account for _existing_ practices within your 224organization, if for example other tools have previously been used to manage 225infrastructure. 226 227Terraform is an administrative tool that manages your infrastructure, and so 228ideally the infrastructure that is used by Terraform should exist outside of 229the infrastructure that Terraform manages. This can be achieved by creating a 230separate _administrative_ AWS account which contains the user accounts used by 231human operators and any infrastructure and tools used to manage the other 232accounts. Isolating shared administrative tools from your main environments 233has a number of advantages, such as avoiding accidentally damaging the 234administrative infrastructure while changing the target infrastructure, and 235reducing the risk that an attacker might abuse production infrastructure to 236gain access to the (usually more privileged) administrative infrastructure. 237 238### Administrative Account Setup 239 240Your administrative AWS account will contain at least the following items: 241 242* One or more [IAM user](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_users.html) 243 for system administrators that will log in to maintain infrastructure in 244 the other accounts. 245* Optionally, one or more [IAM groups](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_groups.html) 246 to differentiate between different groups of users that have different 247 levels of access to the other AWS accounts. 248* An [S3 bucket](http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html) 249 that will contain the Terraform state files for each workspace. 250* A [DynamoDB table](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.TablesItemsAttributes) 251 that will be used for locking to prevent concurrent operations on a single 252 workspace. 253 254Provide the S3 bucket name and DynamoDB table name to Terraform within the 255S3 backend configuration using the `bucket` and `dynamodb_table` arguments 256respectively, and configure a suitable `workspace_key_prefix` to contain 257the states of the various workspaces that will subsequently be created for 258this configuration. 259 260### Environment Account Setup 261 262For the sake of this section, the term "environment account" refers to one 263of the accounts whose contents are managed by Terraform, separate from the 264administrative account described above. 265 266Your environment accounts will eventually contain your own product-specific 267infrastructure. Along with this it must contain one or more 268[IAM roles](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) 269that grant sufficient access for Terraform to perform the desired management 270tasks. 271 272### Delegating Access 273 274Each Administrator will run Terraform using credentials for their IAM user 275in the administrative account. 276[IAM Role Delegation](http://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html) 277is used to grant these users access to the roles created in each environment 278account. 279 280Full details on role delegation are covered in the AWS documentation linked 281above. The most important details are: 282 283* Each role's _Assume Role Policy_ must grant access to the administrative AWS 284 account, which creates a trust relationship with the administrative AWS 285 account so that its users may assume the role. 286* The users or groups within the administrative account must also have a 287 policy that creates the converse relationship, allowing these users or groups 288 to assume that role. 289 290Since the purpose of the administrative account is only to host tools for 291managing other accounts, it is useful to give the administrative accounts 292restricted access only to the specific operations needed to assume the 293environment account role and access the Terraform state. By blocking all 294other access, you remove the risk that user error will lead to staging or 295production resources being created in the administrative account by mistake. 296 297When configuring Terraform, use either environment variables or the standard 298credentials file `~/.aws/credentials` to provide the administrator user's 299IAM credentials within the administrative account to both the S3 backend _and_ 300to Terraform's AWS provider. 301 302Use conditional configuration to pass a different `assume_role` value to 303the AWS provider depending on the selected workspace. For example: 304 305```hcl 306variable "workspace_iam_roles" { 307 default = { 308 staging = "arn:aws:iam::STAGING-ACCOUNT-ID:role/Terraform" 309 production = "arn:aws:iam::PRODUCTION-ACCOUNT-ID:role/Terraform" 310 } 311} 312 313provider "aws" { 314 # No credentials explicitly set here because they come from either the 315 # environment or the global credentials file. 316 317 assume_role = "${var.workspace_iam_roles[terraform.workspace]}" 318} 319``` 320 321If workspace IAM roles are centrally managed and shared across many separate 322Terraform configurations, the role ARNs could also be obtained via a data 323source such as [`terraform_remote_state`](/docs/language/state/remote-state-data.html) 324to avoid repeating these values. 325 326### Creating and Selecting Workspaces 327 328With the necessary objects created and the backend configured, run 329`terraform init` to initialize the backend and establish an initial workspace 330called "default". This workspace will not be used, but is created automatically 331by Terraform as a convenience for users who are not using the workspaces 332feature. 333 334Create a workspace corresponding to each key given in the `workspace_iam_roles` 335variable value above: 336 337``` 338$ terraform workspace new staging 339Created and switched to workspace "staging"! 340 341... 342 343$ terraform workspace new production 344Created and switched to workspace "production"! 345 346... 347``` 348 349Due to the `assume_role` setting in the AWS provider configuration, any 350management operations for AWS resources will be performed via the configured 351role in the appropriate environment AWS account. The backend operations, such 352as reading and writing the state from S3, will be performed directly as the 353administrator's own user within the administrative account. 354 355``` 356$ terraform workspace select staging 357$ terraform apply 358... 359``` 360 361### Running Terraform in Amazon EC2 362 363Teams that make extensive use of Terraform for infrastructure management 364often [run Terraform in automation](https://learn.hashicorp.com/tutorials/terraform/automate-terraform?in=terraform/automation&utm_source=WEBSITE&utm_medium=WEB_IO&utm_offer=ARTICLE_PAGE&utm_content=DOCS) 365to ensure a consistent operating environment and to limit access to the 366various secrets and other sensitive information that Terraform configurations 367tend to require. 368 369When running Terraform in an automation tool running on an Amazon EC2 instance, 370consider running this instance in the administrative account and using an 371[instance profile](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) 372in place of the various administrator IAM users suggested above. An IAM 373instance profile can also be granted cross-account delegation access via 374an IAM policy, giving this instance the access it needs to run Terraform. 375 376To isolate access to different environment accounts, use a separate EC2 377instance for each target account so that its access can be limited only to 378the single account. 379 380Similar approaches can be taken with equivalent features in other AWS compute 381services, such as ECS. 382 383### Protecting Access to Workspace State 384 385In a simple implementation of the pattern described in the prior sections, 386all users have access to read and write states for all workspaces. In many 387cases it is desirable to apply more precise access constraints to the 388Terraform state objects in S3, so that for example only trusted administrators 389are allowed to modify the production state, or to control _reading_ of a state 390that contains sensitive information. 391 392Amazon S3 supports fine-grained access control on a per-object-path basis 393using IAM policy. A full description of S3's access control mechanism is 394beyond the scope of this guide, but an example IAM policy granting access 395to only a single state object within an S3 bucket is shown below: 396 397```json 398{ 399 "Version": "2012-10-17", 400 "Statement": [ 401 { 402 "Effect": "Allow", 403 "Action": "s3:ListBucket", 404 "Resource": "arn:aws:s3:::myorg-terraform-states" 405 }, 406 { 407 "Effect": "Allow", 408 "Action": ["s3:GetObject", "s3:PutObject"], 409 "Resource": "arn:aws:s3:::myorg-terraform-states/myapp/production/tfstate" 410 } 411 ] 412} 413``` 414 415It is not possible to apply such fine-grained access control to the DynamoDB 416table used for locking, so it is possible for any user with Terraform access 417to lock any workspace state, even if they do not have access to read or write 418that state. If a malicious user has such access they could block attempts to 419use Terraform against some or all of your workspaces as long as locking is 420enabled in the backend configuration. 421 422### Configuring Custom User-Agent Information 423 424Note this feature is optional and only available in Terraform v0.13.1+. 425 426By default, the underlying AWS client used by the Terraform AWS Provider creates requests with User-Agent headers including information about Terraform and AWS Go SDK versions. To provide additional information in the User-Agent headers, the `TF_APPEND_USER_AGENT` environment variable can be set and its value will be directly added to HTTP requests. e.g. 427 428```sh 429$ export TF_APPEND_USER_AGENT="JenkinsAgent/i-12345678 BuildID/1234 (Optional Extra Information)" 430``` 431