xref: /dports/devel/gitlab-runner/gitlab-runner-8925d9a06fd8e452e2161a768462652a2a13111f/vendor/github.com/hashicorp/vault/vault-api-v1.0.4/website/source/docs/secrets/gcp/index.html.md

---
layout: "docs"
page_title: "Google Cloud - Secrets Engines"
sidebar_title: "Google Cloud"
sidebar_current: "docs-secrets-gcp"
description: |-
  The Google Cloud secrets engine for Vault dynamically generates Google Cloud
  service account keys and OAuth tokens based on IAM policies.
---

# Google Cloud Secrets Engine

The Google Cloud Vault secrets engine dynamically generates Google Cloud service
account keys and OAuth tokens based on IAM policies. This enables users to gain
access to Google Cloud resources without needing to create or manage a dedicated
service account.

The benefits of using this secrets engine to manage Google Cloud IAM service accounts are:

- **Automatic cleanup of GCP IAM service account keys** - each Service Account
  key is associated with a Vault lease. When the lease expires (either during
  normal revocation or through early revocation), the service account key is
  automatically revoked.

- **Quick, short-term access** - users do not need to create new GCP Service
  Accounts for short-term or one-off access (such as batch jobs or quick
  introspection).

- **Multi-cloud and hybrid cloud applications** - users authenticate to Vault
  using a central identity service (such as LDAP) and generate GCP credentials
  without the need to create or manage a new Service Account for that user.

~> **NOTE: Deprecation of `access_token` Leases**: In previous versions of this secrets engine
    (released with Vault <= 0.11.1), a lease was generated with access tokens. If you're using
    an old version of the plugin, please upgrade. Read more in the
    [upgrade guide](#deprecation-of-access-token-leases)


## Setup

Most secrets engines must be configured in advance before they can perform their
functions. These steps are usually completed by an operator or configuration
management tool.

1. Enable the Google Cloud secrets engine:

    ```text
    $ vault secrets enable gcp
    Success! Enabled the gcp secrets engine at: gcp/
    ```

    By default, the secrets engine will mount at the name of the engine. To
    enable the secrets engine at a different path, use the `-path` argument.

1. Configure the secrets engine with account credentials, or leave blank or unwritten
   to use Application Default Credentials.

    ```text
    $ vault write gcp/config credentials=@my-credentials.json
    Success! Data written to: gcp/config
    ```

    If you are running Vault from inside [Google Compute Engine][gce] or [Google
    Kubernetes Engine][gke], the instance or pod service account can be used in
    place or specifying the credentials JSON file.
    For more information on authentication, see the [authentication section](#authentication) below.

1. Configure a roleset. Rolesets determine the permissions that Service Account
credentials generated by Vault will have on GCP resources.

    To configure a roleset that generates OAuth2 access tokens (preferred):

    ```text
    $ vault write gcp/roleset/my-token-roleset \
        project="my-project" \
        secret_type="access_token"  \
        token_scopes="https://www.googleapis.com/auth/cloud-platform" \
        bindings=-<<EOF
          resource "//cloudresourcemanager.googleapis.com/projects/my-project" {
            roles = ["roles/viewer"]
          }
        EOF
    ```

    To configure a roleset that generates GCP Service Account keys:

    ```text
    $ vault write gcp/roleset/my-key-roleset \
        project="my-project" \
        secret_type="service_account_key"  \
        bindings=-<<EOF
          resource "//cloudresourcemanager.googleapis.com/projects/my-project" {
            roles = ["roles/viewer"]
          }
        EOF
    ```

    Alternatively, provide a file for the `bindings` argument like so:

    ```text
    $ vault write gcp/roleset/my-roleset
        bindings=@mybindings.hcl
        ...
    ```

    For more information on role bindings and sample role bindings, please see
    the [roleset bindings](#roleset-bindings) section below.

    For more information on the differences between OAuth2 access tokens and
    Service Account keys, see the [things to note](#things-to-note) section
    below.

## Usage

After the secrets engine is configured and a user/machine has a Vault token with
the proper permission, it can generate credentials. Depending on how the roleset
was configured, you can generate OAuth2 tokens or service account keys.

### Access Tokens

To generate OAuth2 tokens, read from `gcp/token/...`. The roleset must have been
created as type `access_token`:

```text
$ vault read gcp/token/my-token-roleset

Key                Value
---                -----
expires_at_seconds    1537402548
token                 ya29.c.ElodBmNPwHUNY5gcBpnXcE4ywG4w1k...
token_ttl             3599
```

This endpoint generates a non-renewable, non-revocable static OAuth2 access token
with a lifetime of one hour, where `token_ttl` is given in seconds and the
`expires_at_seconds` is the expiry time for the token, given as a Unix timestamp.
The `token` value then can be used as a HTTP Authorization Bearer token in requests
to GCP APIs:

```sh
$ curl -H "Authorization: Bearer ya29.c.ElodBmNPwHUNY5gcBpnXcE4ywG4w1k..."
```

### Service Account Keys

To generate service account keys, read from `gcp/key/...`. The roleset must have
been created as type `service_account_key`:

```text
$ vault read gcp/key/my-key-roleset

Key                 Value
---                 -----
lease_id            gcp/key/my-key-roleset/ce563a99-5e55-389b...
lease_duration      30m
lease_renewable     true
key_algorithm       KEY_ALG_RSA_2048
key_type            TYPE_GOOGLE_CREDENTIALS_FILE
private_key_data    ewogICJ0eXBlIjogInNlcnZpY2VfYWNjb3VudCIsC...
```

This endpoint generates a new [GCP IAM service account key][iam-keys] associated
with the roleset's Service Account. When the lease expires (or is revoked
early), the Service Account key will be deleted.

**There is a default limit of 10 keys per Service Account.** For more
information on this limit and recommended mitigation, please see the [things to
note](#things-to-note) section below.

## Roleset Bindings

Roleset bindings define a list of resources and the associated IAM roles on that
resource. Roleset bindings are used as the `binding` argument when creating or
updating a roleset and are specified in the following format using HCL:

```hcl
resource NAME {
  roles = [ROLE, [ROLE...]]
}
```

For example:

```hcl
resource "buckets/my-bucket" {
  roles = [
    "roles/storage.objectAdmin",
    "roles/storage.legacyBucketReader",
  ]
}
```

The top-level `resource` block defines the resource or resource path for which
IAM policy information will be bound. The resource path may be specified in a
few different formats:

- **Project-level self-link** - a URI with scheme and host, generally
  corresponding to the `self_link` attribute of a resource in GCP. This must
  include the resource nested in the parent project.

    ```text
    # compute alpha zone
    https://www.googleapis.com/compute/alpha/projects/my-project/zones/us-central1-c
    ```

- **Full resource name** - a schema-less URI consisting of a DNS-compatible API
  service name and resource path. See the [full resource name API
  documentation][resource-name-full] for more information.

    ```text
    # Compute snapshot
    //compute.googleapis.com/project/my-project/snapshots/my-compute-snapshot

    # Pubsub snapshot
    //pubsub.googleapis.com/project/my-project/snapshots/my-pubsub-snapshot
    ```

- **Relative resource name** - A path-noscheme URI path, usually as accepted by
  the API. Use this if the version or service are apparent from the resource
  type. Please see the [relative resource name API
  documentation][resource-name-relative] for more information.

    ```text
    # Storage bucket objects
    buckets/my-bucket
    buckets/my-bucket/objects/my-object

    # PubSub topics
    projects/my-project/topics/my-pubsub-topic
    ```

The nested `roles` attribute is an array of strings names of [GCP IAM
roles][iam-roles]. The roles may be specified in the following formats:

- **Global role name** - these are global roles built into Google Cloud. For the
  full list of available roles, please see the [list of predefined GCP
  roles][predefined-roles].

    ```text
    roles/viewer
    roles/bigquery.user
    roles/billing.admin
    ```

- **Organization-level custom role** - these are roles that are created at the
  organization level by organization owners.

    ```text
    organizations/my-organization/roles/my-custom-role
    ```

    For more information, please see the documentation on [GCP custom
    roles][custom-roles].

- **Project-level custom role** - these are roles that are created at a
  per-project level by project owners.

    ```text
    projects/my-project/roles/my-custom-role
    ```

    For more information, please see the documentation on [GCP custom
    roles][custom-roles].


## Authentication

The Google Cloud Vault secrets backend uses the official Google Cloud Golang
SDK. This means it supports the common ways of [providing credentials to Google
Cloud][cloud-creds]. In addition to specifying `credentials` directly via Vault
configuration, you can also get configuration from the following values **on the
Vault server**:

1. The environment variables `GOOGLE_APPLICATION_CREDENTIALS`. This is specified
as the **path** to a Google Cloud credentials file, typically for a service
account. If this environment variable is present, the resulting credentials are
used. If the credentials are invalid, an error is returned.

1. Default instance credentials. When no environment variable is present, the
default service account credentials are used. This is useful when running Vault
on [Google Compute Engine][gce] or [Google Kubernetes Engine][gke]

For more information on service accounts, please see the [Google Cloud Service
Accounts documentation][service-accounts].

To use this storage backend, the service account must have the following
minimum scope(s):

```text
https://www.googleapis.com/auth/cloud-platform
```

### Required Permissions

The credentials given to Vault must have the following permissions:

```text
# Service Account + Key Admin
iam.serviceAccounts.create
iam.serviceAccounts.delete
iam.serviceAccounts.get
iam.serviceAccounts.list
iam.serviceAccounts.update
iam.serviceAccountKeys.create
iam.serviceAccountKeys.delete
iam.serviceAccountKeys.get
iam.serviceAccountKeys.list

# IAM Policy Changes
<service>.<resource>.getIamPolicy
<service>.<resource>.setIamPolicy
```

where `<service>` and `<resource>` correspond to permissions which will be
granted, for example:

```text
# Projects
resourcemanager.projects.getIamPolicy
resourcemanager.projects.setIamPolicy

# All compute
compute.*.getIamPolicy
compute.*.setIamPolicy
```

You can either:

- Create a [custom role][custom-roles] using these permissions, and assign this
  role at a project-level

- Assign the set of roles required to get resource-specific
  `getIamPolicy/setIamPolicy` permissions. At a minimum you will need to assign
  `roles/iam.serviceAccountAdmin` and `roles/iam.serviceAccountKeyAdmin` so
  Vault can manage service accounts and keys.


## Things to Note

### Access Tokens vs. Service Account Keys

Advantages of `access_tokens`:
* Can generate infinite number of tokens per roleset

Disadvantages of `access_tokens`:
* Cannot be used with some client libraries or tools
* Have a static life-time of 1 hr that cannot be modified, revoked, or extended.

Advantages of `service_account_keys`:
* Controllable life-time through Vault, allowing for longer access
* Can be used by all normal GCP tooling

Disadvantages of `service_account_keys`:
* Infinite lifetime in GCP (i.e. if they are not managed properly, leaked keys can live forever)
* Limited to 10 per roleset/service account.

When generating OAuth access tokens, Vault will still
generate a dedicated service account and key. This private key is stored in Vault
and is never accessible to other users, and the underlying key can
be rotated. See the [GCP API documentation][api] for more information on
rotation.

### Service Accounts are Tied to Rolesets

Service Accounts are created when the roleset is created (or updated) rather
than each time a secret is generated. This may be different from how other
secrets engines behave, but it is for good reasons:

- IAM Service Account creation and permission propagation can take up to 60
  seconds to complete. By creating the Service Account in advance, we speed up
  the timeliness of future operations and reduce the flakiness of automated
  workflows.

- Each GCP project has a limit on the number of IAM Service Accounts. You can
  [request additional quota][quotas]. The quota increase is processed by humans,
  so it is best to request this additional quota in advance. This limit is
  currently 100, **including system-managed Service Accounts**. If Service
  Accounts were created per secret, this quota limit would reduce the number of
  secrets that can be generated.

### Service Account Keys Quota Limits

GCP IAM has a hard limit (currently 10) on the number of Service Account keys.
Attempts to generate more keys will result in an error. If you find yourself
running into this limit, consider the following:

- Have shorter TTLs or revoke access earlier. If you are not using past Service
  Account keys, consider rotating and freeing quota earlier.

- Create additional rolesets which share the same set of permissions. Additional
  rolesets can be created with the same set of permissions. This will create a
  new service account and increases the number of keys you can create.

- Where possible, use OAuth2 access tokens instead of Service Account keys.

### Resources Must Exist at Roleset Creation

Because we the bindings for the Service Account are set during roleset creation,
resources that do not exist will fail the `getIamPolicy` API call.

### Roleset Creation May Partially Fail

Every Service Account creation, key creation, and IAM policy change is a GCP API
call per resource. If an API call to one of these resources fails, the roleset
creation fails and Vault will attempt to rollback.

These rollbacks are API calls, so they may also fail. The secrets engine uses a
WAL to ensure that unused bindings are cleaned up. In the case of quota limits,
you may need to clean these up manually.

### Do Not Modify Vault-owned IAM Accounts

While Vault will initially create and assign permissions to IAM service
accounts, it is possible that an external user deletes or modifies this service
account. These changes are difficult to detect, and it is best to prevent this
type of modification through IAM permissions.

Vault roleset Service Accounts will have emails in the format:

```
vault<roleset-prefix>-<creation-unix-timestamp>@...
```

Communicate with your teams (or use IAM permissions) to not modify these
resources.


## Help &amp; Support

The Google Cloud Vault secrets engine is written as an external Vault plugin and
thus exists outside the main Vault repository. It is automatically bundled with
Vault releases, but the code is managed separately.

Please report issues, add feature requests, and submit contributions to the
[vault-plugin-secrets-gcp repo on GitHub][repo].


## API
The GCP secrets engine has a full HTTP API. Please see the [GCP secrets engine API docs][api]
for more details.

[api]: https://vaultproject.io/api/secret/gcp/index.html
[cloud-creds]: https://cloud.google.com/docs/authentication/production#providing_credentials_to_your_application
[custom-roles]: https://cloud.google.com/iam/docs/creating-custom-roles
[gce]: https://cloud.google.com/compute/
[gke]: https://cloud.google.com/kubernetes-engine/
[iam-keys]: https://cloud.google.com/iam/docs/service-accounts#service_account_keys
[iam-roles]: https://cloud.google.com/iam/docs/understanding-roles
[predefined-roles]: https://cloud.google.com/iam/docs/understanding-roles#predefined_roles
[repo]: https://github.com/hashicorp/vault-plugin-secrets-gcp
[resource-name-full]: https://cloud.google.com/apis/design/resource_names#full_resource_name
[resource-name-relative]: https://cloud.google.com/apis/design/resource_names#relative_resource_name
[quotas]: https://cloud.google.com/compute/quotas
[service-accounts]: https://cloud.google.com/compute/docs/access/service-accounts


## Upgrade Guides

### Deprecation of Access Token Leases

~> **NOTE**: This only affects access tokens. There is no change to the `service_account_key` secret type

Previous versions of this secrets engine (Vault <= 0.11.1) created a lease for
each access token secret. We have removed them after discovering that these
tokens, specifically Google OAuth2 tokens for IAM service accounts, are
non-revocable and have a static 60 minute lifetime. To match the current
limitations of the GCP APIs, the secrets engine will no longer allow for
revocation or manage the token TTL - more specifically, **the access_token
response will no longer include `lease_id` or other lease information**. This
change does not reflect any change to the actual underlying OAuth tokens or GCP
service accounts.

To upgrade:

- Remove references from `lease_id`, `lease_duration` or other `lease_*`
  attributes when reading responses for the access tokens secrets endpoint (i.e.
  from `gcp/token/$roleset`). See the [documentation for access
  tokens](#access-tokens) to see the new format for the response.

- Be aware of leftover leases from previous versions. While these old leases
  will still be revocable, they will not actually invalidate their associated
  access token, and that token will still be useable for up to one hour.