1--- 2stage: Configure 3group: Configure 4info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments 5type: concepts, howto 6--- 7 8# Using external secrets in CI **(FREE)** 9 10> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218746) in GitLab 13.4 and GitLab Runner 13.4. 11> - `file` setting [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250695) in GitLab 14.1 and GitLab Runner 14.1. 12 13Secrets represent sensitive information your CI job needs to complete work. This 14sensitive information can be items like API tokens, database credentials, or private keys. 15Secrets are sourced from your secrets provider. 16 17Unlike CI/CD variables, which are always presented to a job, secrets must be explicitly 18required by a job. Read [GitLab CI/CD pipeline configuration reference](../yaml/index.md#secrets) 19for more information about the syntax. 20 21GitLab has selected [Vault by HashiCorp](https://www.vaultproject.io) as the 22first supported provider, and [KV-V2](https://www.vaultproject.io/docs/secrets/kv/kv-v2) 23as the first supported secrets engine. 24 25GitLab authenticates using Vault's 26[JSON Web Token (JWT) authentication method](https://www.vaultproject.io/docs/auth/jwt#jwt-authentication), using 27the [JSON Web Token](https://gitlab.com/gitlab-org/gitlab/-/issues/207125) (`CI_JOB_JWT`) 28introduced in GitLab 12.10. 29 30You must [configure your Vault server](#configure-your-vault-server) before you 31can use [use Vault secrets in a CI job](#use-vault-secrets-in-a-ci-job). 32 33The flow for using GitLab with HashiCorp Vault 34is summarized by this diagram: 35 36![Flow between GitLab and HashiCorp](../img/gitlab_vault_workflow_v13_4.png "How GitLab CI_JOB_JWT works with HashiCorp Vault") 37 381. Configure your vault and secrets. 391. Generate your JWT and provide it to your CI job. 401. Runner contacts HashiCorp Vault and authenticates using the JWT. 411. HashiCorp Vault verifies the JWT. 421. HashiCorp Vault checks the bounded claims and attaches policies. 431. HashiCorp Vault returns the token. 441. Runner reads secrets from the HashiCorp Vault. 45 46NOTE: 47Read the [Authenticating and Reading Secrets With HashiCorp Vault](../examples/authenticating-with-hashicorp-vault/index.md) 48tutorial for a version of this feature. It's available to all 49subscription levels, supports writing secrets to and deleting secrets from Vault, 50and supports multiple secrets engines. 51 52## Configure your Vault server 53 54To configure your Vault server: 55 561. Ensure your Vault server is running on version 1.2.0 or higher. 571. Enable the authentication method by running these commands. They provide your Vault 58 server the [JSON Web Key Set](https://tools.ietf.org/html/rfc7517) (JWKS) endpoint for your GitLab instance, so Vault 59 can fetch the public signing key and verify the JSON Web Token (JWT) when authenticating: 60 61 ```shell 62 $ vault auth enable jwt 63 64 $ vault write auth/jwt/config \ 65 jwks_url="https://gitlab.example.com/-/jwks" \ 66 bound_issuer="gitlab.example.com" 67 ``` 68 691. Configure policies on your Vault server to grant or forbid access to certain 70 paths and operations. This example grants read access to the set of secrets 71 required by your production environment: 72 73 ```shell 74 vault policy write myproject-production - <<EOF 75 # Read-only permission on 'ops/data/production/*' path 76 77 path "ops/data/production/*" { 78 capabilities = [ "read" ] 79 } 80 EOF 81 ``` 82 831. Configure roles on your Vault server, restricting roles to a project or namespace, 84 as described in [Configure Vault server roles](#configure-vault-server-roles) on this page. 851. [Create the following CI/CD variables](../variables/index.md#custom-cicd-variables) 86 to provide details about your Vault server: 87 - `VAULT_SERVER_URL` - The URL of your Vault server, such as `https://vault.example.com:8200`. 88 Required. 89 - `VAULT_AUTH_ROLE` - Optional. The role to use when attempting to authenticate. 90 If no role is specified, Vault uses the [default role](https://www.vaultproject.io/api/auth/jwt#default_role) 91 specified when the authentication method was configured. 92 - `VAULT_AUTH_PATH` - Optional. The path where the authentication method is mounted, default is `jwt`. 93 94 NOTE: 95 Support for providing these values in the user interface [is tracked in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/218677). 96 97## Use Vault secrets in a CI job **(PREMIUM)** 98 99> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/28321) in GitLab 13.4 and GitLab Runner 13.4. 100 101After [configuring your Vault server](#configure-your-vault-server), you can use 102the secrets stored in Vault by defining them with the `vault` keyword: 103 104```yaml 105secrets: 106 DATABASE_PASSWORD: 107 vault: production/db/password@ops # translates to secret `ops/data/production/db`, field `password` 108``` 109 110In this example: 111 112- `production/db` - The secret. 113- `password` The field. 114- `ops` - The path where the secrets engine is mounted. 115 116After GitLab fetches the secret from Vault, the value is saved in a temporary file. 117The path to this file is stored in a CI/CD variable named `DATABASE_PASSWORD`, 118similar to [variables of type `file`](../variables/index.md#cicd-variable-types). 119 120To overwrite the default behavior, set the `file` option explicitly: 121 122```yaml 123secrets: 124 DATABASE_PASSWORD: 125 vault: production/db/password@ops 126 file: false 127``` 128 129In this example, the secret value is put directly in the `DATABASE_PASSWORD` variable 130instead of pointing to a file that holds it. 131 132For more information about the supported syntax, read the 133[`.gitlab-ci.yml` reference](../yaml/index.md#secretsvault). 134 135## Configure Vault server roles 136 137When a CI job attempts to authenticate, it specifies a role. You can use roles to group 138different policies together. If authentication is successful, these policies are 139attached to the resulting Vault token. 140 141[Bound claims](https://www.vaultproject.io/docs/auth/jwt#bound-claims) are predefined 142values that are matched to the JWT's claims. With bounded claims, you can restrict access 143to specific GitLab users, specific projects, or even jobs running for specific Git 144references. You can have as many bounded claims you need, but they must *all* match 145for authentication to be successful. 146 147Combining bounded claims with GitLab features like [user roles](../../user/permissions.md) 148and [protected branches](../../user/project/protected_branches.md), you can tailor 149these rules to fit your specific use case. In this example, authentication is allowed 150only for jobs running for protected tags with names matching the pattern used for 151production releases: 152 153```shell 154$ vault write auth/jwt/role/myproject-production - <<EOF 155{ 156 "role_type": "jwt", 157 "policies": ["myproject-production"], 158 "token_explicit_max_ttl": 60, 159 "user_claim": "user_email", 160 "bound_claims_type": "glob", 161 "bound_claims": { 162 "project_id": "42", 163 "ref_protected": "true", 164 "ref_type": "tag", 165 "ref": "auto-deploy-*" 166 } 167} 168EOF 169``` 170 171WARNING: 172Always restrict your roles to a project or namespace by using one of the provided 173claims like `project_id` or `namespace_id`. Without these restrictions, any JWT 174generated by this GitLab instance may be allowed to authenticate using this role. 175 176For a full list of `CI_JOB_JWT` claims, read the 177[How it works](../examples/authenticating-with-hashicorp-vault/index.md#how-it-works) section of the 178[Authenticating and Reading Secrets With HashiCorp Vault](../examples/authenticating-with-hashicorp-vault/index.md) tutorial. 179 180You can also specify some attributes for the resulting Vault tokens, such as time-to-live, 181IP address range, and number of uses. The full list of options is available in 182[Vault's documentation on creating roles](https://www.vaultproject.io/api/auth/jwt#create-role) 183for the JSON web token method. 184 185## Using a self-signed Vault server 186 187When the Vault server is using a self-signed certificate, you will see the following error in the job logs: 188 189```plaintext 190ERROR: Job failed (system failure): resolving secrets: initializing Vault service: preparing authenticated client: checking Vault server health: Get https://vault.example.com:8000/v1/sys/health?drsecondarycode=299&performancestandbycode=299&sealedcode=299&standbycode=299&uninitcode=299: x509: certificate signed by unknown authority 191``` 192 193You have two options to solve this error: 194 195- Add the self-signed certificate to the GitLab Runner server's CA store. 196 If you deployed GitLab Runner using the [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html), you will have to create your own GitLab Runner image. 197- Use the `VAULT_CACERT` environment variable to configure GitLab Runner to trust the certificate: 198 - If you are using systemd to manage GitLab Runner, see [how to add an environment variable for GitLab Runner](https://docs.gitlab.com/runner/configuration/init.html#setting-custom-environment-variables). 199 - If you deployed GitLab Runner using the [Helm chart](https://docs.gitlab.com/runner/install/kubernetes.html): 200 1. [Provide a custom certificate for accessing GitLab](https://docs.gitlab.com/runner/install/kubernetes.html#providing-a-custom-certificate-for-accessing-gitlab), and make sure to add the certificate for the Vault server instead of the certificate for GitLab. If your GitLab instance is also using a self-signed certificate, you should be able to add both in the same `Secret`. 201 1. Add the following lines in your `values.yaml` file: 202 203 ```yaml 204 ## Replace both the <SECRET_NAME> and the <VAULT_CERTIFICATE> 205 ## with the actual values you used to create the secret 206 207 certsSecretName: <SECRET_NAME> 208 209 envVars: 210 - name: VAULT_CACERT 211 value: "/home/gitlab-runner/.gitlab-runner/certs/<VAULT_CERTIFICATE>" 212 ``` 213