1# The Docker executor 2 3GitLab Runner can use Docker to run jobs on user provided images. This is 4possible with the use of **Docker** executor. 5 6The **Docker** executor when used with GitLab CI, connects to [Docker Engine](https://www.docker.com/products/container-runtime) 7and runs each build in a separate and isolated container using the predefined 8image that is [set up in `.gitlab-ci.yml`](https://docs.gitlab.com/ee/ci/yaml/README.html) and in accordance in 9[`config.toml`](../commands/index.md#configuration-file). 10 11That way you can have a simple and reproducible build environment that can also 12run on your workstation. The added benefit is that you can test all the 13commands that we will explore later from your shell, rather than having to test 14them on a dedicated CI server. 15 16The following configurations are supported: 17 18| Runner is installed on: | Executor is: | Container is running: | 19|--------------------------|------------------|------------------------| 20| Windows | `docker-windows` | Windows | 21| Windows | `docker` | Linux | 22| Linux | `docker` | Linux | 23 24These configurations are **not** supported: 25 26| Runner is installed on: | Executor is: | Container is running: | 27|--------------------------|------------------|------------------------| 28| Linux | `docker-windows` | Linux | 29| Linux | `docker` | Windows | 30| Linux | `docker-windows` | Windows | 31| Windows | `docker` | Windows | 32| Windows | `docker-windows` | Linux | 33 34NOTE: 35GitLab Runner uses Docker Engine API 36[v1.25](https://docs.docker.com/engine/api/v1.25/) to talk to the Docker 37Engine. This means the 38[minimum supported version](https://docs.docker.com/develop/sdk/#api-version-matrix) 39of Docker on a Linux server is `1.13.0`, 40[on Windows Server it needs to be more recent](#supported-docker-versions) 41to identify the Windows Server version. 42 43## Using Windows containers 44 45> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/535) in GitLab Runner 11.11. 46 47To use Windows containers with the Docker executor, note the following 48information about limitations, supported Windows versions, and 49configuring a Windows Docker executor. 50 51### Nanoserver support 52 53> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/2492) in GitLab Runner 13.6. 54 55With the support for Powershell Core introduced in the Windows helper image, it is now possible to leverage 56the `nanoserver` variants for the helper image. 57 58### Limitations 59 60The following are some limitations of using Windows containers with 61Docker executor: 62 63- Docker-in-Docker is not supported, since it's [not 64 supported](https://github.com/docker-library/docker/issues/49) by 65 Docker itself. 66- Interactive web terminals are not supported. 67- Host device mounting not supported. 68- When mounting a volume directory it has to exist, or Docker will fail 69 to start the container, see 70 [#3754](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/3754) for 71 additional detail. 72- `docker-windows` executor can be run only using GitLab Runner running 73 on Windows. 74- [Linux containers on 75 Windows](https://docs.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/linux-containers) 76 are not supported, since they are still experimental. Read [the 77 relevant 78 issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4373) for 79 more details. 80- Because of a [limitation in 81 Docker](https://github.com/MicrosoftDocs/Virtualization-Documentation/issues/334), 82 if the destination path drive letter is not `c:`, paths are not supported for: 83 84 - [`builds_dir`](../configuration/advanced-configuration.md#the-runners-section) 85 - [`cache_dir`](../configuration/advanced-configuration.md#the-runners-section) 86 - [`volumes`](../configuration/advanced-configuration.md#volumes-in-the-runnersdocker-section) 87 88 This means values such as `f:\\cache_dir` are not supported, but `f:` is supported. 89 However, if the destination path is on the `c:` drive, paths are also supported 90 (for example `c:\\cache_dir`). 91 92### Supported Windows versions 93 94GitLab Runner only supports the following versions of Windows which 95follows our [support lifecycle for 96Windows](../install/windows.md#windows-version-support-policy): 97 98- Windows Server 20H2. 99- Windows Server 2004. 100- Windows Server 1809. 101 102For future Windows Server versions, we have a [future version support 103policy](../install/windows.md#windows-version-support-policy). 104 105You can only run containers based on the same OS version that the Docker 106daemon is running on. For example, the following [`Windows Server 107Core`](https://hub.docker.com/_/microsoft-windows-servercore) images can 108be used: 109 110- `mcr.microsoft.com/windows/servercore:20H2` 111- `mcr.microsoft.com/windows/servercore:20H2-amd64` 112- `mcr.microsoft.com/windows/servercore:2004` 113- `mcr.microsoft.com/windows/servercore:2004-amd64` 114- `mcr.microsoft.com/windows/servercore:1809` 115- `mcr.microsoft.com/windows/servercore:1809-amd64` 116- `mcr.microsoft.com/windows/servercore:ltsc2019` 117 118### Supported Docker versions 119 120A Windows Server running GitLab Runner must be running a recent version of Docker 121because GitLab Runner uses Docker to detect what version of Windows Server is running. 122 123A known version of Docker that doesn't work with GitLab Runner is `Docker 17.06` 124since Docker does not identify the version of Windows Server resulting in the 125following error: 126 127```plaintext 128unsupported Windows Version: Windows Server Datacenter 129``` 130 131[Read more about troubleshooting this](../install/windows.md#docker-executor-unsupported-windows-version). 132 133### Configuring a Windows Docker executor 134 135NOTE: 136When a runner is registered with `c:\\cache` 137as a source directory when passing the `--docker-volumes` or 138`DOCKER_VOLUMES` environment variable, there is a 139[known issue](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4312). 140 141Below is an example of the configuration for a simple Docker 142executor running Windows. 143 144```toml 145[[runners]] 146 name = "windows-docker-2019" 147 url = "https://gitlab.com/" 148 token = "xxxxxxx" 149 executor = "docker-windows" 150 [runners.docker] 151 image = "mcr.microsoft.com/windows/servercore:1809_amd64" 152 volumes = ["c:\\cache"] 153``` 154 155For other configuration options for the Docker executor, see the 156[advanced 157configuration](../configuration/advanced-configuration.md#the-runnersdocker-section) 158section. 159 160### Services 161 162You can use [services](https://docs.gitlab.com/ee/ci/services/) by 163enabling [network per-build](#network-per-build) networking mode. 164[Available](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1042) 165since GitLab Runner 12.9. 166 167## Workflow 168 169The Docker executor divides the job into multiple steps: 170 1711. **Prepare**: Create and start the [services](https://docs.gitlab.com/ee/ci/yaml/#services). 1721. **Pre-job**: Clone, restore [cache](https://docs.gitlab.com/ee/ci/yaml/#cache) 173 and download [artifacts](https://docs.gitlab.com/ee/ci/yaml/#artifacts) from previous 174 stages. This is run on a special Docker image. 1751. **Job**: User build. This is run on the user-provided Docker image. 1761. **Post-job**: Create cache, upload artifacts to GitLab. This is run on 177 a special Docker Image. 178 179The special Docker image is based on [Alpine Linux](https://alpinelinux.org/) and contains all the tools 180required to run the prepare, pre-job, and post-job steps, like the Git and the 181GitLab Runner binaries for supporting caching and artifacts. You can find the definition of 182this special image [in the official GitLab Runner repository](https://gitlab.com/gitlab-org/gitlab-runner/-/tree/v13.4.1/dockerfiles/runner-helper). 183 184## The `image` keyword 185 186The `image` keyword is the name of the Docker image that is present in the 187local Docker Engine (list all images with `docker images`) or any image that 188can be found at [Docker Hub](https://hub.docker.com/). For more information about images and Docker 189Hub please read the [Docker Fundamentals](https://docs.docker.com/engine/understanding-docker/) documentation. 190 191In short, with `image` we refer to the Docker image, which will be used to 192create a container on which your build will run. 193 194If you don't specify the namespace, Docker implies `library` which includes all 195[official images](https://hub.docker.com/u/library/). That's why you'll see 196many times the `library` part omitted in `.gitlab-ci.yml` and `config.toml`. 197For example you can define an image like `image: ruby:2.6`, which is a shortcut 198for `image: library/ruby:2.6`. 199 200Then, for each Docker image there are tags, denoting the version of the image. 201These are defined with a colon (`:`) after the image name. For example, for 202Ruby you can see the supported tags at <https://hub.docker.com/_/ruby/>. If you 203don't specify a tag (like `image: ruby`), `latest` is implied. 204 205The image you choose to run your build in via `image` directive must have a 206working shell in its operating system `PATH`. Supported shells are `sh`, 207`bash`, and `pwsh` ([since 13.9](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4021)) 208for Linux, and PowerShell for Windows. 209GitLab Runner cannot execute a command using the underlying OS system calls 210(such as `exec`). 211 212## The `services` keyword 213 214The `services` keyword defines just another Docker image that is run during 215your build and is linked to the Docker image that the `image` keyword defines. 216This allows you to access the service image during build time. 217 218The service image can run any application, but the most common use case is to 219run a database container, e.g., `mysql`. It's easier and faster to use an 220existing image and run it as an additional container than install `mysql` every 221time the project is built. 222 223You can see some widely used services examples in the relevant documentation of 224[CI services examples](https://docs.gitlab.com/ee/ci/services/). 225 226If needed, you can [assign an alias](https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#available-settings-for-services) 227to each service. 228 229## Networking 230 231Networking is required to connect services to the build job and may also be used to run build jobs in user-defined 232networks. Either legacy `network_mode` or `per-build` networking may be used. 233 234### Legacy container links 235 236The default network mode uses [Legacy container links](https://docs.docker.com/network/links/) with 237the default Docker `bridge` mode to link the job container with the services. 238 239`network_mode` can be used to configure how the networking stack is set up for the containers 240using one of the following values: 241 242- One of the standard Docker [networking modes](https://docs.docker.com/engine/reference/run/#network-settings): 243 - `bridge`: use the bridge network (default) 244 - `host`: use the host's network stack inside the container 245 - `none`: no networking (not recommended) 246 - Any other `network_mode` value is taken as the name of an already existing 247 Docker network, which the build container should connect to. 248 249For name resolution to work, Docker will manipulate the `/etc/hosts` file in the build 250job container to include the service container hostname (and alias). However, 251the service container will **not** be able to resolve the build job container 252name. To achieve that, use the `per-build` network mode. 253 254Linked containers share their environment variables. 255 256### Network per-build 257 258> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1042) in GitLab Runner 12.9. 259 260This mode will create and use a new user-defined Docker bridge network per build. 261[User-defined bridge networks](https://docs.docker.com/network/bridge/) are covered in detail in the Docker documentation. 262 263Unlike [legacy container links](#legacy-container-links) used in other network modes, 264Docker environment variables are **not** shared across the containers. 265 266Docker networks may conflict with other networks on the host, including other Docker networks, 267if the CIDR ranges are already in use. The default Docker address pool can be configured 268via `default-address-pool` in [`dockerd`](https://docs.docker.com/engine/reference/commandline/dockerd/). 269 270To enable this mode you need to enable the [`FF_NETWORK_PER_BUILD` 271feature flag](../configuration/feature-flags.md). 272 273When a job starts, a bridge network is created (similarly to `docker 274network create <network>`). Upon creation, the service container(s) and the 275build job container are connected to this network. 276 277Both the build job container, and the service container(s) will be able to 278resolve each other's hostnames (and aliases). This functionality is 279[provided by Docker](https://docs.docker.com/network/bridge/#differences-between-user-defined-bridges-and-the-default-bridge). 280 281The build container is resolvable via the `build` alias as well as it's GitLab assigned hostname. 282 283The network is removed at the end of the build job. 284 285## Define image and services from `.gitlab-ci.yml` 286 287You can simply define an image that will be used for all jobs and a list of 288services that you want to use during build time. 289 290```yaml 291image: ruby:2.6 292 293services: 294 - postgres:9.3 295 296before_script: 297 - bundle install 298 299test: 300 script: 301 - bundle exec rake spec 302``` 303 304It is also possible to define different images and services per job: 305 306```yaml 307before_script: 308 - bundle install 309 310test:2.6: 311 image: ruby:2.6 312 services: 313 - postgres:9.3 314 script: 315 - bundle exec rake spec 316 317test:2.7: 318 image: ruby:2.7 319 services: 320 - postgres:9.4 321 script: 322 - bundle exec rake spec 323``` 324 325## Define image and services in `config.toml` 326 327Look for the `[runners.docker]` section: 328 329```toml 330[runners.docker] 331 image = "ruby:2.6" 332 333[[runners.docker.services]] 334 name = "mysql:latest" 335 alias = "db" 336 337[[runners.docker.services]] 338 name = "redis:latest" 339 alias = "cache" 340``` 341 342The example above uses the [array of tables syntax](https://toml.io/en/v0.4.0#array-of-tables). 343 344The image and services defined this way will be added to all builds run by 345that runner, so even if you don't define an `image` inside `.gitlab-ci.yml`, 346the one defined in `config.toml` will be used. 347 348## Define an image from a private Docker registry 349 350Starting with GitLab Runner 0.6.0, you are able to define images located to 351private registries that could also require authentication. 352 353All you have to do is be explicit on the image definition in `.gitlab-ci.yml`. 354 355```yaml 356image: my.registry.tld:5000/namepace/image:tag 357``` 358 359In the example above, GitLab Runner will look at `my.registry.tld:5000` for the 360image `namespace/image:tag`. 361 362If the repository is private you need to authenticate your GitLab Runner in the 363registry. Read more on [using a private Docker registry](../configuration/advanced-configuration.md#use-a-private-container-registry). 364 365## Accessing the services 366 367Let's say that you need a Wordpress instance to test some API integration with 368your application. 369 370You can then use for example the [tutum/wordpress](https://hub.docker.com/r/tutum/wordpress/) as a service image in your 371`.gitlab-ci.yml`: 372 373```yaml 374services: 375- tutum/wordpress:latest 376``` 377 378When the build is run, `tutum/wordpress` will be started first and you will have 379access to it from your build container under the hostname `tutum__wordpress` 380and `tutum-wordpress`. 381 382The GitLab Runner creates two alias hostnames for the service that you can use 383alternatively. The aliases are taken from the image name following these rules: 384 3851. Everything after `:` is stripped. 3861. For the first alias, the slash (`/`) is replaced with double underscores (`__`). 3871. For the second alias, the slash (`/`) is replaced with a single dash (`-`). 388 389Using a private service image will strip any port given and apply the rules as 390described above. A service `registry.gitlab-wp.com:4999/tutum/wordpress` will 391result in hostname `registry.gitlab-wp.com__tutum__wordpress` and 392`registry.gitlab-wp.com-tutum-wordpress`. 393 394## Configuring services 395 396Many services accept environment variables which allow you to easily change 397database names or set account names depending on the environment. 398 399GitLab Runner 0.5.0 and up passes all YAML-defined variables to the created 400service containers. 401 402For all possible configuration variables check the documentation of each image 403provided in their corresponding Docker Hub page. 404 405All variables are passed to all services containers. It's not designed to 406distinguish which variable should go where. 407Secure variables are only passed to the build container. 408 409## Mounting a directory in RAM 410 411You can mount a path in RAM using tmpfs. This can speed up the time required to test if there is a lot of I/O related work, such as with databases. 412If you use the `tmpfs` and `services_tmpfs` options in the runner configuration, you can specify multiple paths, each with its own options. See the [Docker reference](https://docs.docker.com/engine/reference/commandline/run/#mount-tmpfs-tmpfs) for details. 413This is an example `config.toml` to mount the data directory for the official Mysql container in RAM. 414 415```toml 416[runners.docker] 417 # For the main container 418 [runners.docker.tmpfs] 419 "/var/lib/mysql" = "rw,noexec" 420 421 # For services 422 [runners.docker.services_tmpfs] 423 "/var/lib/mysql" = "rw,noexec" 424``` 425 426## Build directory in service 427 428Since version 1.5 GitLab Runner mounts a `/builds` directory to all shared services. 429 430See an issue: <https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1520>. 431 432### PostgreSQL service example 433 434See the specific documentation for 435[using PostgreSQL as a service](https://docs.gitlab.com/ee/ci/services/postgres.html). 436 437### MySQL service example 438 439See the specific documentation for 440[using MySQL as a service](https://docs.gitlab.com/ee/ci/services/mysql.html). 441 442### The services health check 443 444After the service is started, GitLab Runner waits some time for the service to 445be responsive. Currently, the Docker executor tries to open a TCP connection to 446the first exposed service in the service container. 447 448You can see how it is implemented by checking this [Go command](https://gitlab.com/gitlab-org/gitlab-runner/blob/main/commands/helpers/health_check.go). 449 450## The builds and cache storage 451 452The Docker executor by default stores all builds in 453`/builds/<namespace>/<project-name>` and all caches in `/cache` (inside the 454container). 455You can overwrite the `/builds` and `/cache` directories by defining the 456`builds_dir` and `cache_dir` options under the `[[runners]]` section in 457`config.toml`. This will modify where the data are stored inside the container. 458 459If you modify the `/cache` storage path, you also need to make sure to mark this 460directory as persistent by defining it in `volumes = ["/my/cache/"]` under the 461`[runners.docker]` section in `config.toml`. 462 463### Clearing Docker cache 464 465> Introduced in GitLab Runner 13.9, [all created runner resources cleaned up](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/2310). 466 467GitLab Runner provides the [`clear-docker-cache`](https://gitlab.com/gitlab-org/gitlab-runner/blob/main/packaging/root/usr/share/gitlab-runner/clear-docker-cache) 468script to remove old containers and volumes that can unnecessarily consume disk space. 469 470Run `clear-docker-cache` regularly (using `cron` once per week, for example), 471ensuring a balance is struck between: 472 473- Maintaining some recent containers in the cache for performance. 474- Reclaiming disk space. 475 476`clear-docker-cache` can remove old or unused containers and volumes that are created by the GitLab Runner. For a list of options, run the script with `help` option: 477 478```shell 479clear-docker-cache help 480``` 481 482The default option is `prune-volumes` which the script will remove all unused containers (both dangling and unreferenced) and volumes. 483 484### Clearing old build images 485 486The [`clear-docker-cache`](https://gitlab.com/gitlab-org/gitlab-runner/blob/main/packaging/root/usr/share/gitlab-runner/clear-docker-cache) script will not remove the Docker images as they are not tagged by the GitLab Runner. You can however confirm the space that can be reclaimed by running the script with the `space` option as illustrated below: 487 488```shell 489clear-docker-cache space 490 491Show docker disk usage 492---------------------- 493 494TYPE TOTAL ACTIVE SIZE RECLAIMABLE 495Images 14 9 1.306GB 545.8MB (41%) 496Containers 19 18 115kB 0B (0%) 497Local Volumes 0 0 0B 0B 498Build Cache 0 0 0B 0B 499``` 500 501Once you have confirmed the reclaimable space, run the [`docker system prune`](https://docs.docker.com/engine/reference/commandline/system_prune/) command that will remove all unused containers, networks, images (both dangling and unreferenced), and optionally, volumes that are not tagged by the GitLab Runner. 502 503## The persistent storage 504 505The Docker executor can provide a persistent storage when running the containers. 506All directories defined under `volumes =` will be persistent between builds. 507 508The `volumes` directive supports two types of storage: 509 5101. `<path>` - **the dynamic storage**. The `<path>` is persistent between subsequent 511 runs of the same concurrent job for that project. The data is attached to a 512 custom cache volume: `runner-<short-token>-project-<id>-concurrent-<concurrency-id>-cache-<md5-of-path>`. 5131. `<host-path>:<path>[:<mode>]` - **the host-bound storage**. The `<path>` is 514 bound to `<host-path>` on the host system. The optional `<mode>` can specify 515 that this storage is read-only or read-write (default). 516 517### The persistent storage for builds 518 519If you make the `/builds` directory **a host-bound storage**, your builds will be stored in: 520`/builds/<short-token>/<concurrent-id>/<namespace>/<project-name>`, where: 521 522- `<short-token>` is a shortened version of the Runner's token (first 8 letters) 523- `<concurrent-id>` is a unique number, identifying the local job ID on the 524 particular runner in context of the project 525 526## The privileged mode 527 528The Docker executor supports a number of options that allows fine-tuning of the 529build container. One of these options is the [`privileged` mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities). 530 531### Use Docker-in-Docker with privileged mode 532 533The configured `privileged` flag is passed to the build container and all 534services, thus allowing to easily use the Docker-in-Docker approach. 535 536First, configure your runner (`config.toml`) to run in `privileged` mode: 537 538```toml 539[[runners]] 540 executor = "docker" 541 [runners.docker] 542 privileged = true 543``` 544 545Then, make your build script (`.gitlab-ci.yml`) to use Docker-in-Docker 546container: 547 548```yaml 549image: docker:git 550services: 551- docker:dind 552 553build: 554 script: 555 - docker build -t my-image . 556 - docker push my-image 557``` 558 559## The ENTRYPOINT 560 561The Docker executor doesn't overwrite the [`ENTRYPOINT` of a Docker image](https://docs.docker.com/engine/reference/run/#entrypoint-default-command-to-execute-at-runtime). 562 563That means that if your image defines the `ENTRYPOINT` and doesn't allow running 564scripts with `CMD`, the image will not work with the Docker executor. 565 566With the use of `ENTRYPOINT` it is possible to create special Docker image that 567would run the build script in a custom environment, or in secure mode. 568 569You may think of creating a Docker image that uses an `ENTRYPOINT` that doesn't 570execute the build script, but does execute a predefined set of commands, for 571example to build the Docker image from your directory. In that case, you can 572run the build container in [privileged mode](#the-privileged-mode), and make 573the build environment of the runner secure. 574 575Consider the following example: 576 5771. Create a new Dockerfile: 578 579 ```dockerfile 580 FROM docker:dind 581 ADD / /entrypoint.sh 582 ENTRYPOINT ["/bin/sh", "/entrypoint.sh"] 583 ``` 584 5851. Create a bash script (`entrypoint.sh`) that will be used as the `ENTRYPOINT`: 586 587 ```shell 588 #!/bin/sh 589 590 dind docker daemon 591 --host=unix:///var/run/docker.sock \ 592 --host=tcp://0.0.0.0:2375 \ 593 --storage-driver=vf & 594 595 docker build -t "$BUILD_IMAGE" . 596 docker push "$BUILD_IMAGE" 597 ``` 598 5991. Push the image to the Docker registry. 600 6011. Run Docker executor in `privileged` mode. In `config.toml` define: 602 603 ```toml 604 [[runners]] 605 executor = "docker" 606 [runners.docker] 607 privileged = true 608 ``` 609 6101. In your project use the following `.gitlab-ci.yml`: 611 612 ```yaml 613 variables: 614 BUILD_IMAGE: my.image 615 build: 616 image: my/docker-build:image 617 script: 618 - Dummy Script 619 ``` 620 621This is just one of the examples. With this approach the possibilities are 622limitless. 623 624## How pull policies work 625 626When using the `docker` or `docker+machine` executors, you can set the 627`pull_policy` parameter in the runner `config.toml` file as described in the configuration docs' 628[Docker section](../configuration/advanced-configuration.md#the-runnersdocker-section). 629 630This parameter defines how the runner works when pulling Docker images (for both `image` and `services` keywords). 631You can set it to a single value, or a list of pull policies, which will be attempted in order 632until an image is pulled successfully. 633 634If you don't set any value for the `pull_policy` parameter, then 635the runner will use the `always` pull policy as the default value. 636 637Now let's see how these policies work. 638 639### Using the `never` pull policy 640 641The `never` pull policy disables images pulling completely. If you set the 642`pull_policy` parameter of a runner to `never`, then users will be able 643to use only the images that have been manually pulled on the Docker host 644the runner runs on. 645 646If an image cannot be found locally, then the runner will fail the build 647with an error similar to: 648 649```plaintext 650Pulling docker image local_image:latest ... 651ERROR: Build failed: Error: image local_image:latest not found 652``` 653 654#### When to use the `never` pull policy 655 656The `never` pull policy should be used if you want or need to have a full 657control on which images are used by the runner's users. It is a good choice 658for private runners that are dedicated to a project where only specific images 659can be used (not publicly available on any registries). 660 661#### When not to use the `never` pull policy 662 663The `never` pull policy will not work properly with most of [auto-scaled](../configuration/autoscale.md) 664Docker executor use cases. Because of how auto-scaling works, the `never` 665pull policy may be usable only when using a pre-defined cloud instance 666images for chosen cloud provider. The image needs to contain installed 667Docker Engine and local copy of used images. 668 669### Using the `if-not-present` pull policy 670 671When the `if-not-present` pull policy is used, the runner will first check 672if the image is present locally. If it is, then the local version of 673image will be used. Otherwise, the runner will try to pull the image. 674 675#### When to use the `if-not-present` pull policy 676 677The `if-not-present` pull policy is a good choice if you want to use images pulled from 678remote registries, but you want to reduce time spent on analyzing image 679layers difference when using heavy and rarely updated images. 680In that case, you will need once in a while to manually remove the image 681from the local Docker Engine store to force the update of the image. 682 683It is also the good choice if you need to use images that are built 684and available only locally, but on the other hand, also need to allow to 685pull images from remote registries. 686 687#### When not to use the `if-not-present` pull policy 688 689The `if-not-present` pull policy should not be used if your builds use images that 690are updated frequently and need to be used in most recent versions. 691In such a situation, the network load reduction created by this policy may 692be less worthy than the necessity of the very frequent deletion of local 693copies of images. 694 695This pull policy should also not be used if your runner can be used by 696different users which should not have access to private images used 697by each other. Especially do not use this pull policy for shared runners. 698 699To understand why the `if-not-present` pull policy creates security issues 700when used with private images, read the 701[security considerations documentation](../security/index.md#usage-of-private-docker-images-with-if-not-present-pull-policy). 702 703### Using the `always` pull policy 704 705The `always` pull policy will ensure that the image is **always** pulled. 706When `always` is used, the runner will try to pull the image even if a local 707copy is available. The [caching semantics](https://kubernetes.io/docs/concepts/configuration/overview/#container-images) 708of the underlying image provider make this policy efficient. 709The pull attempt is fast because all image layers are cached. 710 711If the image is not found, then the build will fail with an error similar to: 712 713```plaintext 714Pulling docker image registry.tld/my/image:latest ... 715ERROR: Build failed: Error: image registry.tld/my/image:latest not found 716``` 717 718When using the `always` pull policy in GitLab Runner versions older than `v1.8`, it could 719fall back to the local copy of an image and print a warning: 720 721```plaintext 722Pulling docker image registry.tld/my/image:latest ... 723WARNING: Cannot pull the latest version of image registry.tld/my/image:latest : Error: image registry.tld/my/image:latest not found 724WARNING: Locally found image will be used instead. 725``` 726 727This was [changed in GitLab Runner `v1.8`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1905). 728 729#### When to use the `always` pull policy 730 731The `always` pull policy should be used if your runner is publicly available 732and configured as a shared runner in your GitLab instance. It is the 733only pull policy that can be considered as secure when the runner will 734be used with private images. 735 736This is also a good choice if you want to force users to always use 737the newest images. 738 739Also, this will be the best solution for an [auto-scaled](../configuration/autoscale.md) 740configuration of the runner. 741 742#### When not to use the `always` pull policy 743 744The `always` pull policy will definitely not work if you need to use locally 745stored images. In this case, the runner will skip the local copy of the image 746and try to pull it from the remote registry. If the image was built locally 747and doesn't exist in any public registry (and especially in the default 748Docker registry), the build will fail with: 749 750```plaintext 751Pulling docker image local_image:latest ... 752ERROR: Build failed: Error: image local_image:latest not found 753``` 754 755### Using multiple pull policies 756 757> [Introduced](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26558) in GitLab Runner 13.8. 758 759The `pull_policy` parameter allows you to specify a list of pull policies. 760The policies in the list will be attempted in order from left to right until a pull attempt 761is successful, or the list is exhausted. 762 763This functionality can be useful when the Docker registry is not available 764and you need to increase job resiliency. 765If you use the `always` policy and the registry is not available, the job fails even if the desired image is cached locally. 766 767To overcome that behavior, you can add additional fallback pull policies 768that execute in case of failure. 769By adding a second pull policy value of `if-not-present`, the runner finds any locally-cached Docker image layers: 770 771```toml 772[runners.docker] 773 pull_policy = ["always", "if-not-present"] 774``` 775 776**Any** failure to fetch the Docker image causes the runner to attempt the following pull policy. 777Examples include an `HTTP 403 Forbidden` or an `HTTP 500 Internal Server Error` response from the repository. 778 779Note that the security implications mentioned in the `When not to use this pull policy?` sub-section of the 780[Using the if-not-present pull policy](#using-the-if-not-present-pull-policy) section still apply, 781so you should be aware of the security implications and read the 782[security considerations documentation](../security/index.md#usage-of-private-docker-images-with-if-not-present-pull-policy). 783 784```plaintext 785Using Docker executor with image alpine:latest ... 786Pulling docker image alpine:latest ... 787WARNING: Failed to pull image with policy "always": Error response from daemon: received unexpected HTTP status: 502 Bad Gateway (docker.go:143:0s) 788Attempt #2: Trying "if-not-present" pull policy 789Using locally found image version due to "if-not-present" pull policy 790``` 791 792## Docker vs Docker-SSH (and Docker+Machine vs Docker-SSH+Machine) 793 794WARNING: 795Starting with GitLab Runner 10.0, both Docker-SSH and Docker-SSH+machine executors 796are **deprecated** and will be removed in one of the upcoming releases. 797 798We provided a support for a special type of Docker executor, namely Docker-SSH 799(and the autoscaled version: Docker-SSH+Machine). Docker-SSH uses the same logic 800as the Docker executor, but instead of executing the script directly, it uses an 801SSH client to connect to the build container. 802 803Docker-SSH then connects to the SSH server that is running inside the container 804using its internal IP. 805 806This executor is no longer maintained and will be removed in the near future. 807