1--- 2title: Configuration 3sort_rank: 1 4--- 5 6# Configuration 7 8Prometheus is configured via command-line flags and a configuration file. While 9the command-line flags configure immutable system parameters (such as storage 10locations, amount of data to keep on disk and in memory, etc.), the 11configuration file defines everything related to scraping [jobs and their 12instances](https://prometheus.io/docs/concepts/jobs_instances/), as well as 13which [rule files to load](recording_rules.md#configuring-rules). 14 15To view all available command-line flags, run `./prometheus -h`. 16 17Prometheus can reload its configuration at runtime. If the new configuration 18is not well-formed, the changes will not be applied. 19A configuration reload is triggered by sending a `SIGHUP` to the Prometheus process or 20sending a HTTP POST request to the `/-/reload` endpoint (when the `--web.enable-lifecycle` flag is enabled). 21This will also reload any configured rule files. 22 23## Configuration file 24 25To specify which configuration file to load, use the `--config.file` flag. 26 27The file is written in [YAML format](https://en.wikipedia.org/wiki/YAML), 28defined by the scheme described below. 29Brackets indicate that a parameter is optional. For non-list parameters the 30value is set to the specified default. 31 32Generic placeholders are defined as follows: 33 34* `<boolean>`: a boolean that can take the values `true` or `false` 35* `<duration>`: a duration matching the regular expression `((([0-9]+)y)?(([0-9]+)w)?(([0-9]+)d)?(([0-9]+)h)?(([0-9]+)m)?(([0-9]+)s)?(([0-9]+)ms)?|0)`, e.g. `1d`, `1h30m`, `5m`, `10s` 36* `<filename>`: a valid path in the current working directory 37* `<host>`: a valid string consisting of a hostname or IP followed by an optional port number 38* `<int>`: an integer value 39* `<labelname>`: a string matching the regular expression `[a-zA-Z_][a-zA-Z0-9_]*` 40* `<labelvalue>`: a string of unicode characters 41* `<path>`: a valid URL path 42* `<scheme>`: a string that can take the values `http` or `https` 43* `<secret>`: a regular string that is a secret, such as a password 44* `<string>`: a regular string 45* `<size>`: a size in bytes, e.g. `512MB`. A unit is required. Supported units: B, KB, MB, GB, TB, PB, EB. 46* `<tmpl_string>`: a string which is template-expanded before usage 47 48The other placeholders are specified separately. 49 50A valid example file can be found [here](/config/testdata/conf.good.yml). 51 52The global configuration specifies parameters that are valid in all other configuration 53contexts. They also serve as defaults for other configuration sections. 54 55```yaml 56global: 57 # How frequently to scrape targets by default. 58 [ scrape_interval: <duration> | default = 1m ] 59 60 # How long until a scrape request times out. 61 [ scrape_timeout: <duration> | default = 10s ] 62 63 # How frequently to evaluate rules. 64 [ evaluation_interval: <duration> | default = 1m ] 65 66 # The labels to add to any time series or alerts when communicating with 67 # external systems (federation, remote storage, Alertmanager). 68 external_labels: 69 [ <labelname>: <labelvalue> ... ] 70 71 # File to which PromQL queries are logged. 72 # Reloading the configuration will reopen the file. 73 [ query_log_file: <string> ] 74 75# Rule files specifies a list of globs. Rules and alerts are read from 76# all matching files. 77rule_files: 78 [ - <filepath_glob> ... ] 79 80# A list of scrape configurations. 81scrape_configs: 82 [ - <scrape_config> ... ] 83 84# Alerting specifies settings related to the Alertmanager. 85alerting: 86 alert_relabel_configs: 87 [ - <relabel_config> ... ] 88 alertmanagers: 89 [ - <alertmanager_config> ... ] 90 91# Settings related to the remote write feature. 92remote_write: 93 [ - <remote_write> ... ] 94 95# Settings related to the remote read feature. 96remote_read: 97 [ - <remote_read> ... ] 98``` 99 100### `<scrape_config>` 101 102A `scrape_config` section specifies a set of targets and parameters describing how 103to scrape them. In the general case, one scrape configuration specifies a single 104job. In advanced configurations, this may change. 105 106Targets may be statically configured via the `static_configs` parameter or 107dynamically discovered using one of the supported service-discovery mechanisms. 108 109Additionally, `relabel_configs` allow advanced modifications to any 110target and its labels before scraping. 111 112```yaml 113# The job name assigned to scraped metrics by default. 114job_name: <job_name> 115 116# How frequently to scrape targets from this job. 117[ scrape_interval: <duration> | default = <global_config.scrape_interval> ] 118 119# Per-scrape timeout when scraping this job. 120[ scrape_timeout: <duration> | default = <global_config.scrape_timeout> ] 121 122# The HTTP resource path on which to fetch metrics from targets. 123[ metrics_path: <path> | default = /metrics ] 124 125# honor_labels controls how Prometheus handles conflicts between labels that are 126# already present in scraped data and labels that Prometheus would attach 127# server-side ("job" and "instance" labels, manually configured target 128# labels, and labels generated by service discovery implementations). 129# 130# If honor_labels is set to "true", label conflicts are resolved by keeping label 131# values from the scraped data and ignoring the conflicting server-side labels. 132# 133# If honor_labels is set to "false", label conflicts are resolved by renaming 134# conflicting labels in the scraped data to "exported_<original-label>" (for 135# example "exported_instance", "exported_job") and then attaching server-side 136# labels. 137# 138# Setting honor_labels to "true" is useful for use cases such as federation and 139# scraping the Pushgateway, where all labels specified in the target should be 140# preserved. 141# 142# Note that any globally configured "external_labels" are unaffected by this 143# setting. In communication with external systems, they are always applied only 144# when a time series does not have a given label yet and are ignored otherwise. 145[ honor_labels: <boolean> | default = false ] 146 147# honor_timestamps controls whether Prometheus respects the timestamps present 148# in scraped data. 149# 150# If honor_timestamps is set to "true", the timestamps of the metrics exposed 151# by the target will be used. 152# 153# If honor_timestamps is set to "false", the timestamps of the metrics exposed 154# by the target will be ignored. 155[ honor_timestamps: <boolean> | default = true ] 156 157# Configures the protocol scheme used for requests. 158[ scheme: <scheme> | default = http ] 159 160# Optional HTTP URL parameters. 161params: 162 [ <string>: [<string>, ...] ] 163 164# Sets the `Authorization` header on every scrape request with the 165# configured username and password. 166# password and password_file are mutually exclusive. 167basic_auth: 168 [ username: <string> ] 169 [ password: <secret> ] 170 [ password_file: <string> ] 171 172# Sets the `Authorization` header on every scrape request with 173# the configured credentials. 174authorization: 175 # Sets the authentication type of the request. 176 [ type: <string> | default: Bearer ] 177 # Sets the credentials of the request. It is mutually exclusive with 178 # `credentials_file`. 179 [ credentials: <secret> ] 180 # Sets the credentials of the request with the credentials read from the 181 # configured file. It is mutually exclusive with `credentials`. 182 [ credentials_file: <filename> ] 183 184# Optional OAuth 2.0 configuration. 185# Cannot be used at the same time as basic_auth or authorization. 186oauth2: 187 [ <oauth2> ] 188 189# Configure whether scrape requests follow HTTP 3xx redirects. 190[ follow_redirects: <bool> | default = true ] 191 192# Configures the scrape request's TLS settings. 193tls_config: 194 [ <tls_config> ] 195 196# Optional proxy URL. 197[ proxy_url: <string> ] 198 199# List of Azure service discovery configurations. 200azure_sd_configs: 201 [ - <azure_sd_config> ... ] 202 203# List of Consul service discovery configurations. 204consul_sd_configs: 205 [ - <consul_sd_config> ... ] 206 207# List of DigitalOcean service discovery configurations. 208digitalocean_sd_configs: 209 [ - <digitalocean_sd_config> ... ] 210 211# List of Docker service discovery configurations. 212docker_sd_configs: 213 [ - <docker_sd_config> ... ] 214 215# List of Docker Swarm service discovery configurations. 216dockerswarm_sd_configs: 217 [ - <dockerswarm_sd_config> ... ] 218 219# List of DNS service discovery configurations. 220dns_sd_configs: 221 [ - <dns_sd_config> ... ] 222 223# List of EC2 service discovery configurations. 224ec2_sd_configs: 225 [ - <ec2_sd_config> ... ] 226 227# List of Eureka service discovery configurations. 228eureka_sd_configs: 229 [ - <eureka_sd_config> ... ] 230 231# List of file service discovery configurations. 232file_sd_configs: 233 [ - <file_sd_config> ... ] 234 235# List of GCE service discovery configurations. 236gce_sd_configs: 237 [ - <gce_sd_config> ... ] 238 239# List of Hetzner service discovery configurations. 240hetzner_sd_configs: 241 [ - <hetzner_sd_config> ... ] 242 243# List of HTTP service discovery configurations. 244http_sd_configs: 245 [ - <http_sd_config> ... ] 246 247# List of Kubernetes service discovery configurations. 248kubernetes_sd_configs: 249 [ - <kubernetes_sd_config> ... ] 250 251# List of Kuma service discovery configurations. 252kuma_sd_configs: 253 [ - <kuma_sd_config> ... ] 254 255# List of Lightsail service discovery configurations. 256lightsail_sd_configs: 257 [ - <lightsail_sd_config> ... ] 258 259# List of Linode service discovery configurations. 260linode_sd_configs: 261 [ - <linode_sd_config> ... ] 262 263# List of Marathon service discovery configurations. 264marathon_sd_configs: 265 [ - <marathon_sd_config> ... ] 266 267# List of AirBnB's Nerve service discovery configurations. 268nerve_sd_configs: 269 [ - <nerve_sd_config> ... ] 270 271# List of OpenStack service discovery configurations. 272openstack_sd_configs: 273 [ - <openstack_sd_config> ... ] 274 275# List of Scaleway service discovery configurations. 276scaleway_sd_configs: 277 [ - <scaleway_sd_config> ... ] 278 279# List of Zookeeper Serverset service discovery configurations. 280serverset_sd_configs: 281 [ - <serverset_sd_config> ... ] 282 283# List of Triton service discovery configurations. 284triton_sd_configs: 285 [ - <triton_sd_config> ... ] 286 287# List of labeled statically configured targets for this job. 288static_configs: 289 [ - <static_config> ... ] 290 291# List of target relabel configurations. 292relabel_configs: 293 [ - <relabel_config> ... ] 294 295# List of metric relabel configurations. 296metric_relabel_configs: 297 [ - <relabel_config> ... ] 298 299# An uncompressed response body larger than this many bytes will cause the 300# scrape to fail. 0 means no limit. Example: 100MB. 301# This is an experimental feature, this behaviour could 302# change or be removed in the future. 303[ body_size_limit: <size> | default = 0 ] 304# Per-scrape limit on number of scraped samples that will be accepted. 305# If more than this number of samples are present after metric relabeling 306# the entire scrape will be treated as failed. 0 means no limit. 307[ sample_limit: <int> | default = 0 ] 308 309# Per-scrape limit on number of labels that will be accepted for a sample. If 310# more than this number of labels are present post metric-relabeling, the 311# entire scrape will be treated as failed. 0 means no limit. 312[ label_limit: <int> | default = 0 ] 313 314# Per-scrape limit on length of labels name that will be accepted for a sample. 315# If a label name is longer than this number post metric-relabeling, the entire 316# scrape will be treated as failed. 0 means no limit. 317[ label_name_length_limit: <int> | default = 0 ] 318 319# Per-scrape limit on length of labels value that will be accepted for a sample. 320# If a label value is longer than this number post metric-relabeling, the 321# entire scrape will be treated as failed. 0 means no limit. 322[ label_value_length_limit: <int> | default = 0 ] 323 324# Per-scrape config limit on number of unique targets that will be 325# accepted. If more than this number of targets are present after target 326# relabeling, Prometheus will mark the targets as failed without scraping them. 327# 0 means no limit. This is an experimental feature, this behaviour could 328# change in the future. 329[ target_limit: <int> | default = 0 ] 330``` 331 332Where `<job_name>` must be unique across all scrape configurations. 333 334### `<tls_config>` 335 336A `tls_config` allows configuring TLS connections. 337 338```yaml 339# CA certificate to validate API server certificate with. 340[ ca_file: <filename> ] 341 342# Certificate and key files for client cert authentication to the server. 343[ cert_file: <filename> ] 344[ key_file: <filename> ] 345 346# ServerName extension to indicate the name of the server. 347# https://tools.ietf.org/html/rfc4366#section-3.1 348[ server_name: <string> ] 349 350# Disable validation of the server certificate. 351[ insecure_skip_verify: <boolean> ] 352``` 353 354### `<oauth2>` 355 356OAuth 2.0 authentication using the client credentials grant type. 357Prometheus fetches an access token from the specified endpoint with 358the given client access and secret keys. 359 360```yaml 361client_id: <string> 362[ client_secret: <secret> ] 363 364# Read the client secret from a file. 365# It is mutually exclusive with `client_secret`. 366[ client_secret_file: <filename> ] 367 368# Scopes for the token request. 369scopes: 370 [ - <string> ... ] 371 372# The URL to fetch the token from. 373token_url: <string> 374 375# Optional parameters to append to the token URL. 376endpoint_params: 377 [ <string>: <string> ... ] 378``` 379 380### `<azure_sd_config>` 381 382Azure SD configurations allow retrieving scrape targets from Azure VMs. 383 384The following meta labels are available on targets during [relabeling](#relabel_config): 385 386* `__meta_azure_machine_id`: the machine ID 387* `__meta_azure_machine_location`: the location the machine runs in 388* `__meta_azure_machine_name`: the machine name 389* `__meta_azure_machine_computer_name`: the machine computer name 390* `__meta_azure_machine_os_type`: the machine operating system 391* `__meta_azure_machine_private_ip`: the machine's private IP 392* `__meta_azure_machine_public_ip`: the machine's public IP if it exists 393* `__meta_azure_machine_resource_group`: the machine's resource group 394* `__meta_azure_machine_tag_<tagname>`: each tag value of the machine 395* `__meta_azure_machine_scale_set`: the name of the scale set which the vm is part of (this value is only set if you are using a [scale set](https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/)) 396* `__meta_azure_subscription_id`: the subscription ID 397* `__meta_azure_tenant_id`: the tenant ID 398 399See below for the configuration options for Azure discovery: 400 401```yaml 402# The information to access the Azure API. 403# The Azure environment. 404[ environment: <string> | default = AzurePublicCloud ] 405 406# The authentication method, either OAuth or ManagedIdentity. 407# See https://docs.microsoft.com/en-us/azure/active-directory/managed-identities-azure-resources/overview 408[ authentication_method: <string> | default = OAuth] 409# The subscription ID. Always required. 410subscription_id: <string> 411# Optional tenant ID. Only required with authentication_method OAuth. 412[ tenant_id: <string> ] 413# Optional client ID. Only required with authentication_method OAuth. 414[ client_id: <string> ] 415# Optional client secret. Only required with authentication_method OAuth. 416[ client_secret: <secret> ] 417 418# Refresh interval to re-read the instance list. 419[ refresh_interval: <duration> | default = 300s ] 420 421# The port to scrape metrics from. If using the public IP address, this must 422# instead be specified in the relabeling rule. 423[ port: <int> | default = 80 ] 424``` 425 426### `<consul_sd_config>` 427 428Consul SD configurations allow retrieving scrape targets from [Consul's](https://www.consul.io) 429Catalog API. 430 431The following meta labels are available on targets during [relabeling](#relabel_config): 432 433* `__meta_consul_address`: the address of the target 434* `__meta_consul_dc`: the datacenter name for the target 435* `__meta_consul_health`: the health status of the service 436* `__meta_consul_metadata_<key>`: each node metadata key value of the target 437* `__meta_consul_node`: the node name defined for the target 438* `__meta_consul_service_address`: the service address of the target 439* `__meta_consul_service_id`: the service ID of the target 440* `__meta_consul_service_metadata_<key>`: each service metadata key value of the target 441* `__meta_consul_service_port`: the service port of the target 442* `__meta_consul_service`: the name of the service the target belongs to 443* `__meta_consul_tagged_address_<key>`: each node tagged address key value of the target 444* `__meta_consul_tags`: the list of tags of the target joined by the tag separator 445 446```yaml 447# The information to access the Consul API. It is to be defined 448# as the Consul documentation requires. 449[ server: <host> | default = "localhost:8500" ] 450[ token: <secret> ] 451[ datacenter: <string> ] 452# Namespaces are only supported in Consul Enterprise. 453[ namespace: <string> ] 454[ scheme: <string> | default = "http" ] 455# The username and password fields are deprecated in favor of the basic_auth configuration. 456[ username: <string> ] 457[ password: <secret> ] 458 459# A list of services for which targets are retrieved. If omitted, all services 460# are scraped. 461services: 462 [ - <string> ] 463 464# See https://www.consul.io/api/catalog.html#list-nodes-for-service to know more 465# about the possible filters that can be used. 466 467# An optional list of tags used to filter nodes for a given service. Services must contain all tags in the list. 468tags: 469 [ - <string> ] 470 471# Node metadata key/value pairs to filter nodes for a given service. 472[ node_meta: 473 [ <string>: <string> ... ] ] 474 475# The string by which Consul tags are joined into the tag label. 476[ tag_separator: <string> | default = , ] 477 478# Allow stale Consul results (see https://www.consul.io/api/features/consistency.html). Will reduce load on Consul. 479[ allow_stale: <boolean> | default = true ] 480 481# The time after which the provided names are refreshed. 482# On large setup it might be a good idea to increase this value because the catalog will change all the time. 483[ refresh_interval: <duration> | default = 30s ] 484 485# Authentication information used to authenticate to the consul server. 486# Note that `basic_auth`, `authorization` and `oauth2` options are 487# mutually exclusive. 488# `password` and `password_file` are mutually exclusive. 489 490# Optional HTTP basic authentication information. 491basic_auth: 492 [ username: <string> ] 493 [ password: <secret> ] 494 [ password_file: <string> ] 495 496# Optional `Authorization` header configuration. 497authorization: 498 # Sets the authentication type. 499 [ type: <string> | default: Bearer ] 500 # Sets the credentials. It is mutually exclusive with 501 # `credentials_file`. 502 [ credentials: <secret> ] 503 # Sets the credentials to the credentials read from the configured file. 504 # It is mutually exclusive with `credentials`. 505 [ credentials_file: <filename> ] 506 507# Optional OAuth 2.0 configuration. 508oauth2: 509 [ <oauth2> ] 510 511# Optional proxy URL. 512[ proxy_url: <string> ] 513 514# Configure whether HTTP requests follow HTTP 3xx redirects. 515[ follow_redirects: <bool> | default = true ] 516 517# TLS configuration. 518tls_config: 519 [ <tls_config> ] 520``` 521 522Note that the IP number and port used to scrape the targets is assembled as 523`<__meta_consul_address>:<__meta_consul_service_port>`. However, in some 524Consul setups, the relevant address is in `__meta_consul_service_address`. 525In those cases, you can use the [relabel](#relabel_config) 526feature to replace the special `__address__` label. 527 528The [relabeling phase](#relabel_config) is the preferred and more powerful 529way to filter services or nodes for a service based on arbitrary labels. For 530users with thousands of services it can be more efficient to use the Consul API 531directly which has basic support for filtering nodes (currently by node 532metadata and a single tag). 533 534### `<digitalocean_sd_config>` 535 536DigitalOcean SD configurations allow retrieving scrape targets from [DigitalOcean's](https://www.digitalocean.com/) 537Droplets API. 538This service discovery uses the public IPv4 address by default, by that can be 539changed with relabelling, as demonstrated in [the Prometheus digitalocean-sd 540configuration file](/documentation/examples/prometheus-digitalocean.yml). 541 542The following meta labels are available on targets during [relabeling](#relabel_config): 543 544* `__meta_digitalocean_droplet_id`: the id of the droplet 545* `__meta_digitalocean_droplet_name`: the name of the droplet 546* `__meta_digitalocean_image`: the slug of the droplet's image 547* `__meta_digitalocean_image_name`: the display name of the droplet's image 548* `__meta_digitalocean_private_ipv4`: the private IPv4 of the droplet 549* `__meta_digitalocean_public_ipv4`: the public IPv4 of the droplet 550* `__meta_digitalocean_public_ipv6`: the public IPv6 of the droplet 551* `__meta_digitalocean_region`: the region of the droplet 552* `__meta_digitalocean_size`: the size of the droplet 553* `__meta_digitalocean_status`: the status of the droplet 554* `__meta_digitalocean_features`: the comma-separated list of features of the droplet 555* `__meta_digitalocean_tags`: the comma-separated list of tags of the droplet 556* `__meta_digitalocean_vpc`: the id of the droplet's VPC 557 558```yaml 559# Authentication information used to authenticate to the API server. 560# Note that `basic_auth` and `authorization` options are 561# mutually exclusive. 562# password and password_file are mutually exclusive. 563 564# Optional HTTP basic authentication information, not currently supported by DigitalOcean. 565basic_auth: 566 [ username: <string> ] 567 [ password: <secret> ] 568 [ password_file: <string> ] 569 570# Optional `Authorization` header configuration. 571authorization: 572 # Sets the authentication type. 573 [ type: <string> | default: Bearer ] 574 # Sets the credentials. It is mutually exclusive with 575 # `credentials_file`. 576 [ credentials: <secret> ] 577 # Sets the credentials to the credentials read from the configured file. 578 # It is mutually exclusive with `credentials`. 579 [ credentials_file: <filename> ] 580 581# Optional OAuth 2.0 configuration. 582# Cannot be used at the same time as basic_auth or authorization. 583oauth2: 584 [ <oauth2> ] 585 586# Optional proxy URL. 587[ proxy_url: <string> ] 588 589# Configure whether HTTP requests follow HTTP 3xx redirects. 590[ follow_redirects: <bool> | default = true ] 591 592# TLS configuration. 593tls_config: 594 [ <tls_config> ] 595 596# The port to scrape metrics from. 597[ port: <int> | default = 80 ] 598 599# The time after which the droplets are refreshed. 600[ refresh_interval: <duration> | default = 60s ] 601``` 602 603### `<docker_sd_config>` 604 605Docker SD configurations allow retrieving scrape targets from [Docker Engine](https://docs.docker.com/engine/) hosts. 606 607This SD discovers "containers" and will create a target for each network IP and port the container is configured to expose. 608 609Available meta labels: 610 611* `__meta_docker_container_id`: the id of the container 612* `__meta_docker_container_name`: the name of the container 613* `__meta_docker_container_network_mode`: the network mode of the container 614* `__meta_docker_container_label_<labelname>`: each label of the container 615* `__meta_docker_network_id`: the ID of the network 616* `__meta_docker_network_name`: the name of the network 617* `__meta_docker_network_ingress`: whether the network is ingress 618* `__meta_docker_network_internal`: whether the network is internal 619* `__meta_docker_network_label_<labelname>`: each label of the network 620* `__meta_docker_network_scope`: the scope of the network 621* `__meta_docker_network_ip`: the IP of the container in this network 622* `__meta_docker_port_private`: the port on the container 623* `__meta_docker_port_public`: the external port if a port-mapping exists 624* `__meta_docker_port_public_ip`: the public IP if a port-mapping exists 625 626See below for the configuration options for Docker discovery: 627 628```yaml 629# Address of the Docker daemon. 630host: <string> 631 632# Optional proxy URL. 633[ proxy_url: <string> ] 634 635# TLS configuration. 636tls_config: 637 [ <tls_config> ] 638 639# The port to scrape metrics from, when `role` is nodes, and for discovered 640# tasks and services that don't have published ports. 641[ port: <int> | default = 80 ] 642 643# The host to use if the container is in host networking mode. 644[ host_networking_host: <string> | default = "localhost" ] 645 646# Optional filters to limit the discovery process to a subset of available 647# resources. 648# The available filters are listed in the upstream documentation: 649# Services: https://docs.docker.com/engine/api/v1.40/#operation/ServiceList 650# Tasks: https://docs.docker.com/engine/api/v1.40/#operation/TaskList 651# Nodes: https://docs.docker.com/engine/api/v1.40/#operation/NodeList 652[ filters: 653 [ - name: <string> 654 values: <string>, [...] ] 655 656# The time after which the containers are refreshed. 657[ refresh_interval: <duration> | default = 60s ] 658 659# Authentication information used to authenticate to the Docker daemon. 660# Note that `basic_auth` and `authorization` options are 661# mutually exclusive. 662# password and password_file are mutually exclusive. 663 664# Optional HTTP basic authentication information. 665basic_auth: 666 [ username: <string> ] 667 [ password: <secret> ] 668 [ password_file: <string> ] 669 670# Optional `Authorization` header configuration. 671authorization: 672 # Sets the authentication type. 673 [ type: <string> | default: Bearer ] 674 # Sets the credentials. It is mutually exclusive with 675 # `credentials_file`. 676 [ credentials: <secret> ] 677 # Sets the credentials to the credentials read from the configured file. 678 # It is mutually exclusive with `credentials`. 679 [ credentials_file: <filename> ] 680 681# Optional OAuth 2.0 configuration. 682# Cannot be used at the same time as basic_auth or authorization. 683oauth2: 684 [ <oauth2> ] 685 686# Configure whether HTTP requests follow HTTP 3xx redirects. 687[ follow_redirects: <bool> | default = true ] 688 689``` 690 691The [relabeling phase](#relabel_config) is the preferred and more powerful 692way to filter containers. For users with thousands of containers it 693can be more efficient to use the Docker API directly which has basic support for 694filtering containers (using `filters`). 695 696See [this example Prometheus configuration file](/documentation/examples/prometheus-docker.yml) 697for a detailed example of configuring Prometheus for Docker Engine. 698 699### `<dockerswarm_sd_config>` 700 701Docker Swarm SD configurations allow retrieving scrape targets from [Docker Swarm](https://docs.docker.com/engine/swarm/) 702engine. 703 704One of the following roles can be configured to discover targets: 705 706#### `services` 707 708The `services` role discovers all [Swarm services](https://docs.docker.com/engine/swarm/key-concepts/#services-and-tasks) 709and exposes their ports as targets. For each published port of a service, a 710single target is generated. If a service has no published ports, a target per 711service is created using the `port` parameter defined in the SD configuration. 712 713Available meta labels: 714 715* `__meta_dockerswarm_service_id`: the id of the service 716* `__meta_dockerswarm_service_name`: the name of the service 717* `__meta_dockerswarm_service_mode`: the mode of the service 718* `__meta_dockerswarm_service_endpoint_port_name`: the name of the endpoint port, if available 719* `__meta_dockerswarm_service_endpoint_port_publish_mode`: the publish mode of the endpoint port 720* `__meta_dockerswarm_service_label_<labelname>`: each label of the service 721* `__meta_dockerswarm_service_task_container_hostname`: the container hostname of the target, if available 722* `__meta_dockerswarm_service_task_container_image`: the container image of the target 723* `__meta_dockerswarm_service_updating_status`: the status of the service, if available 724* `__meta_dockerswarm_network_id`: the ID of the network 725* `__meta_dockerswarm_network_name`: the name of the network 726* `__meta_dockerswarm_network_ingress`: whether the network is ingress 727* `__meta_dockerswarm_network_internal`: whether the network is internal 728* `__meta_dockerswarm_network_label_<labelname>`: each label of the network 729* `__meta_dockerswarm_network_scope`: the scope of the network 730 731#### `tasks` 732 733The `tasks` role discovers all [Swarm tasks](https://docs.docker.com/engine/swarm/key-concepts/#services-and-tasks) 734and exposes their ports as targets. For each published port of a task, a single 735target is generated. If a task has no published ports, a target per task is 736created using the `port` parameter defined in the SD configuration. 737 738Available meta labels: 739 740* `__meta_dockerswarm_task_id`: the id of the task 741* `__meta_dockerswarm_task_container_id`: the container id of the task 742* `__meta_dockerswarm_task_desired_state`: the desired state of the task 743* `__meta_dockerswarm_task_label_<labelname>`: each label of the task 744* `__meta_dockerswarm_task_slot`: the slot of the task 745* `__meta_dockerswarm_task_state`: the state of the task 746* `__meta_dockerswarm_task_port_publish_mode`: the publish mode of the task port 747* `__meta_dockerswarm_service_id`: the id of the service 748* `__meta_dockerswarm_service_name`: the name of the service 749* `__meta_dockerswarm_service_mode`: the mode of the service 750* `__meta_dockerswarm_service_label_<labelname>`: each label of the service 751* `__meta_dockerswarm_network_id`: the ID of the network 752* `__meta_dockerswarm_network_name`: the name of the network 753* `__meta_dockerswarm_network_ingress`: whether the network is ingress 754* `__meta_dockerswarm_network_internal`: whether the network is internal 755* `__meta_dockerswarm_network_label_<labelname>`: each label of the network 756* `__meta_dockerswarm_network_label`: each label of the network 757* `__meta_dockerswarm_network_scope`: the scope of the network 758* `__meta_dockerswarm_node_id`: the ID of the node 759* `__meta_dockerswarm_node_hostname`: the hostname of the node 760* `__meta_dockerswarm_node_address`: the address of the node 761* `__meta_dockerswarm_node_availability`: the availability of the node 762* `__meta_dockerswarm_node_label_<labelname>`: each label of the node 763* `__meta_dockerswarm_node_platform_architecture`: the architecture of the node 764* `__meta_dockerswarm_node_platform_os`: the operating system of the node 765* `__meta_dockerswarm_node_role`: the role of the node 766* `__meta_dockerswarm_node_status`: the status of the node 767 768The `__meta_dockerswarm_network_*` meta labels are not populated for ports which 769are published with `mode=host`. 770 771#### `nodes` 772 773The `nodes` role is used to discover [Swarm nodes](https://docs.docker.com/engine/swarm/key-concepts/#nodes). 774 775Available meta labels: 776 777* `__meta_dockerswarm_node_address`: the address of the node 778* `__meta_dockerswarm_node_availability`: the availability of the node 779* `__meta_dockerswarm_node_engine_version`: the version of the node engine 780* `__meta_dockerswarm_node_hostname`: the hostname of the node 781* `__meta_dockerswarm_node_id`: the ID of the node 782* `__meta_dockerswarm_node_label_<labelname>`: each label of the node 783* `__meta_dockerswarm_node_manager_address`: the address of the manager component of the node 784* `__meta_dockerswarm_node_manager_leader`: the leadership status of the manager component of the node (true or false) 785* `__meta_dockerswarm_node_manager_reachability`: the reachability of the manager component of the node 786* `__meta_dockerswarm_node_platform_architecture`: the architecture of the node 787* `__meta_dockerswarm_node_platform_os`: the operating system of the node 788* `__meta_dockerswarm_node_role`: the role of the node 789* `__meta_dockerswarm_node_status`: the status of the node 790 791See below for the configuration options for Docker Swarm discovery: 792 793```yaml 794# Address of the Docker daemon. 795host: <string> 796 797# Optional proxy URL. 798[ proxy_url: <string> ] 799 800# TLS configuration. 801tls_config: 802 [ <tls_config> ] 803 804# Role of the targets to retrieve. Must be `services`, `tasks`, or `nodes`. 805role: <string> 806 807# The port to scrape metrics from, when `role` is nodes, and for discovered 808# tasks and services that don't have published ports. 809[ port: <int> | default = 80 ] 810 811# Optional filters to limit the discovery process to a subset of available 812# resources. 813# The available filters are listed in the upstream documentation: 814# https://docs.docker.com/engine/api/v1.40/#operation/ContainerList 815[ filters: 816 [ - name: <string> 817 values: <string>, [...] ] 818 819# The time after which the service discovery data is refreshed. 820[ refresh_interval: <duration> | default = 60s ] 821 822# Authentication information used to authenticate to the Docker daemon. 823# Note that `basic_auth` and `authorization` options are 824# mutually exclusive. 825# password and password_file are mutually exclusive. 826 827# Optional HTTP basic authentication information. 828basic_auth: 829 [ username: <string> ] 830 [ password: <secret> ] 831 [ password_file: <string> ] 832 833# Optional `Authorization` header configuration. 834authorization: 835 # Sets the authentication type. 836 [ type: <string> | default: Bearer ] 837 # Sets the credentials. It is mutually exclusive with 838 # `credentials_file`. 839 [ credentials: <secret> ] 840 # Sets the credentials to the credentials read from the configured file. 841 # It is mutually exclusive with `credentials`. 842 [ credentials_file: <filename> ] 843 844# Optional OAuth 2.0 configuration. 845# Cannot be used at the same time as basic_auth or authorization. 846oauth2: 847 [ <oauth2> ] 848 849# Configure whether HTTP requests follow HTTP 3xx redirects. 850[ follow_redirects: <bool> | default = true ] 851 852``` 853 854The [relabeling phase](#relabel_config) is the preferred and more powerful 855way to filter tasks, services or nodes. For users with thousands of tasks it 856can be more efficient to use the Swarm API directly which has basic support for 857filtering nodes (using `filters`). 858 859See [this example Prometheus configuration file](/documentation/examples/prometheus-dockerswarm.yml) 860for a detailed example of configuring Prometheus for Docker Swarm. 861 862### `<dns_sd_config>` 863 864A DNS-based service discovery configuration allows specifying a set of DNS 865domain names which are periodically queried to discover a list of targets. The 866DNS servers to be contacted are read from `/etc/resolv.conf`. 867 868This service discovery method only supports basic DNS A, AAAA and SRV record 869queries, but not the advanced DNS-SD approach specified in 870[RFC6763](https://tools.ietf.org/html/rfc6763). 871 872The following meta labels are available on targets during [relabeling](#relabel_config): 873 874* `__meta_dns_name`: the record name that produced the discovered target. 875* `__meta_dns_srv_record_target`: the target field of the SRV record 876* `__meta_dns_srv_record_port`: the port field of the SRV record 877 878```yaml 879# A list of DNS domain names to be queried. 880names: 881 [ - <string> ] 882 883# The type of DNS query to perform. One of SRV, A, or AAAA. 884[ type: <string> | default = 'SRV' ] 885 886# The port number used if the query type is not SRV. 887[ port: <int>] 888 889# The time after which the provided names are refreshed. 890[ refresh_interval: <duration> | default = 30s ] 891``` 892 893### `<ec2_sd_config>` 894 895EC2 SD configurations allow retrieving scrape targets from AWS EC2 896instances. The private IP address is used by default, but may be changed to 897the public IP address with relabeling. 898 899The following meta labels are available on targets during [relabeling](#relabel_config): 900 901* `__meta_ec2_ami`: the EC2 Amazon Machine Image 902* `__meta_ec2_architecture`: the architecture of the instance 903* `__meta_ec2_availability_zone`: the availability zone in which the instance is running 904* `__meta_ec2_availability_zone_id`: the [availability zone ID](https://docs.aws.amazon.com/ram/latest/userguide/working-with-az-ids.html) in which the instance is running (requires `ec2:DescribeAvailabilityZones`) 905* `__meta_ec2_instance_id`: the EC2 instance ID 906* `__meta_ec2_instance_lifecycle`: the lifecycle of the EC2 instance, set only for 'spot' or 'scheduled' instances, absent otherwise 907* `__meta_ec2_instance_state`: the state of the EC2 instance 908* `__meta_ec2_instance_type`: the type of the EC2 instance 909* `__meta_ec2_ipv6_addresses`: comma separated list of IPv6 addresses assigned to the instance's network interfaces, if present 910* `__meta_ec2_owner_id`: the ID of the AWS account that owns the EC2 instance 911* `__meta_ec2_platform`: the Operating System platform, set to 'windows' on Windows servers, absent otherwise 912* `__meta_ec2_primary_subnet_id`: the subnet ID of the primary network interface, if available 913* `__meta_ec2_private_dns_name`: the private DNS name of the instance, if available 914* `__meta_ec2_private_ip`: the private IP address of the instance, if present 915* `__meta_ec2_public_dns_name`: the public DNS name of the instance, if available 916* `__meta_ec2_public_ip`: the public IP address of the instance, if available 917* `__meta_ec2_subnet_id`: comma separated list of subnets IDs in which the instance is running, if available 918* `__meta_ec2_tag_<tagkey>`: each tag value of the instance 919* `__meta_ec2_vpc_id`: the ID of the VPC in which the instance is running, if available 920 921See below for the configuration options for EC2 discovery: 922 923```yaml 924# The information to access the EC2 API. 925 926# The AWS region. If blank, the region from the instance metadata is used. 927[ region: <string> ] 928 929# Custom endpoint to be used. 930[ endpoint: <string> ] 931 932# The AWS API keys. If blank, the environment variables `AWS_ACCESS_KEY_ID` 933# and `AWS_SECRET_ACCESS_KEY` are used. 934[ access_key: <string> ] 935[ secret_key: <secret> ] 936# Named AWS profile used to connect to the API. 937[ profile: <string> ] 938 939# AWS Role ARN, an alternative to using AWS API keys. 940[ role_arn: <string> ] 941 942# Refresh interval to re-read the instance list. 943[ refresh_interval: <duration> | default = 60s ] 944 945# The port to scrape metrics from. If using the public IP address, this must 946# instead be specified in the relabeling rule. 947[ port: <int> | default = 80 ] 948 949# Filters can be used optionally to filter the instance list by other criteria. 950# Available filter criteria can be found here: 951# https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstances.html 952# Filter API documentation: https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_Filter.html 953filters: 954 [ - name: <string> 955 values: <string>, [...] ] 956``` 957 958The [relabeling phase](#relabel_config) is the preferred and more powerful 959way to filter targets based on arbitrary labels. For users with thousands of 960instances it can be more efficient to use the EC2 API directly which has 961support for filtering instances. 962 963### `<openstack_sd_config>` 964 965OpenStack SD configurations allow retrieving scrape targets from OpenStack Nova 966instances. 967 968One of the following `<openstack_role>` types can be configured to discover targets: 969 970#### `hypervisor` 971 972The `hypervisor` role discovers one target per Nova hypervisor node. The target 973address defaults to the `host_ip` attribute of the hypervisor. 974 975The following meta labels are available on targets during [relabeling](#relabel_config): 976 977* `__meta_openstack_hypervisor_host_ip`: the hypervisor node's IP address. 978* `__meta_openstack_hypervisor_id`: the hypervisor node's ID. 979* `__meta_openstack_hypervisor_name`: the hypervisor node's name. 980* `__meta_openstack_hypervisor_state`: the hypervisor node's state. 981* `__meta_openstack_hypervisor_status`: the hypervisor node's status. 982* `__meta_openstack_hypervisor_type`: the hypervisor node's type. 983 984#### `instance` 985 986The `instance` role discovers one target per network interface of Nova 987instance. The target address defaults to the private IP address of the network 988interface. 989 990The following meta labels are available on targets during [relabeling](#relabel_config): 991 992* `__meta_openstack_address_pool`: the pool of the private IP. 993* `__meta_openstack_instance_flavor`: the flavor of the OpenStack instance. 994* `__meta_openstack_instance_id`: the OpenStack instance ID. 995* `__meta_openstack_instance_name`: the OpenStack instance name. 996* `__meta_openstack_instance_status`: the status of the OpenStack instance. 997* `__meta_openstack_private_ip`: the private IP of the OpenStack instance. 998* `__meta_openstack_project_id`: the project (tenant) owning this instance. 999* `__meta_openstack_public_ip`: the public IP of the OpenStack instance. 1000* `__meta_openstack_tag_<tagkey>`: each tag value of the instance. 1001* `__meta_openstack_user_id`: the user account owning the tenant. 1002 1003See below for the configuration options for OpenStack discovery: 1004 1005```yaml 1006# The information to access the OpenStack API. 1007 1008# The OpenStack role of entities that should be discovered. 1009role: <openstack_role> 1010 1011# The OpenStack Region. 1012region: <string> 1013 1014# identity_endpoint specifies the HTTP endpoint that is required to work with 1015# the Identity API of the appropriate version. While it's ultimately needed by 1016# all of the identity services, it will often be populated by a provider-level 1017# function. 1018[ identity_endpoint: <string> ] 1019 1020# username is required if using Identity V2 API. Consult with your provider's 1021# control panel to discover your account's username. In Identity V3, either 1022# userid or a combination of username and domain_id or domain_name are needed. 1023[ username: <string> ] 1024[ userid: <string> ] 1025 1026# password for the Identity V2 and V3 APIs. Consult with your provider's 1027# control panel to discover your account's preferred method of authentication. 1028[ password: <secret> ] 1029 1030# At most one of domain_id and domain_name must be provided if using username 1031# with Identity V3. Otherwise, either are optional. 1032[ domain_name: <string> ] 1033[ domain_id: <string> ] 1034 1035# The project_id and project_name fields are optional for the Identity V2 API. 1036# Some providers allow you to specify a project_name instead of the project_id. 1037# Some require both. Your provider's authentication policies will determine 1038# how these fields influence authentication. 1039[ project_name: <string> ] 1040[ project_id: <string> ] 1041 1042# The application_credential_id or application_credential_name fields are 1043# required if using an application credential to authenticate. Some providers 1044# allow you to create an application credential to authenticate rather than a 1045# password. 1046[ application_credential_name: <string> ] 1047[ application_credential_id: <string> ] 1048 1049# The application_credential_secret field is required if using an application 1050# credential to authenticate. 1051[ application_credential_secret: <secret> ] 1052 1053# Whether the service discovery should list all instances for all projects. 1054# It is only relevant for the 'instance' role and usually requires admin permissions. 1055[ all_tenants: <boolean> | default: false ] 1056 1057# Refresh interval to re-read the instance list. 1058[ refresh_interval: <duration> | default = 60s ] 1059 1060# The port to scrape metrics from. If using the public IP address, this must 1061# instead be specified in the relabeling rule. 1062[ port: <int> | default = 80 ] 1063 1064# The availability of the endpoint to connect to. Must be one of public, admin or internal. 1065[ availability: <string> | default = "public" ] 1066 1067# TLS configuration. 1068tls_config: 1069 [ <tls_config> ] 1070``` 1071 1072### `<file_sd_config>` 1073 1074File-based service discovery provides a more generic way to configure static targets 1075and serves as an interface to plug in custom service discovery mechanisms. 1076 1077It reads a set of files containing a list of zero or more 1078`<static_config>`s. Changes to all defined files are detected via disk watches 1079and applied immediately. Files may be provided in YAML or JSON format. Only 1080changes resulting in well-formed target groups are applied. 1081 1082Files must contain a list of static configs, using these formats: 1083 1084**JSON** 1085```json 1086[ 1087 { 1088 "targets": [ "<host>", ... ], 1089 "labels": { 1090 "<labelname>": "<labelvalue>", ... 1091 } 1092 }, 1093 ... 1094] 1095``` 1096 1097**YAML** 1098```yaml 1099- targets: 1100 [ - '<host>' ] 1101 labels: 1102 [ <labelname>: <labelvalue> ... ] 1103``` 1104 1105As a fallback, the file contents are also re-read periodically at the specified 1106refresh interval. 1107 1108Each target has a meta label `__meta_filepath` during the 1109[relabeling phase](#relabel_config). Its value is set to the 1110filepath from which the target was extracted. 1111 1112There is a list of 1113[integrations](https://prometheus.io/docs/operating/integrations/#file-service-discovery) with this 1114discovery mechanism. 1115 1116```yaml 1117# Patterns for files from which target groups are extracted. 1118files: 1119 [ - <filename_pattern> ... ] 1120 1121# Refresh interval to re-read the files. 1122[ refresh_interval: <duration> | default = 5m ] 1123``` 1124 1125Where `<filename_pattern>` may be a path ending in `.json`, `.yml` or `.yaml`. The last path segment 1126may contain a single `*` that matches any character sequence, e.g. `my/path/tg_*.json`. 1127 1128### `<gce_sd_config>` 1129 1130[GCE](https://cloud.google.com/compute/) SD configurations allow retrieving scrape targets from GCP GCE instances. 1131The private IP address is used by default, but may be changed to the public IP 1132address with relabeling. 1133 1134The following meta labels are available on targets during [relabeling](#relabel_config): 1135 1136* `__meta_gce_instance_id`: the numeric id of the instance 1137* `__meta_gce_instance_name`: the name of the instance 1138* `__meta_gce_label_<labelname>`: each GCE label of the instance 1139* `__meta_gce_machine_type`: full or partial URL of the machine type of the instance 1140* `__meta_gce_metadata_<name>`: each metadata item of the instance 1141* `__meta_gce_network`: the network URL of the instance 1142* `__meta_gce_private_ip`: the private IP address of the instance 1143* `__meta_gce_interface_ipv4_<name>`: IPv4 address of each named interface 1144* `__meta_gce_project`: the GCP project in which the instance is running 1145* `__meta_gce_public_ip`: the public IP address of the instance, if present 1146* `__meta_gce_subnetwork`: the subnetwork URL of the instance 1147* `__meta_gce_tags`: comma separated list of instance tags 1148* `__meta_gce_zone`: the GCE zone URL in which the instance is running 1149 1150See below for the configuration options for GCE discovery: 1151 1152```yaml 1153# The information to access the GCE API. 1154 1155# The GCP Project 1156project: <string> 1157 1158# The zone of the scrape targets. If you need multiple zones use multiple 1159# gce_sd_configs. 1160zone: <string> 1161 1162# Filter can be used optionally to filter the instance list by other criteria 1163# Syntax of this filter string is described here in the filter query parameter section: 1164# https://cloud.google.com/compute/docs/reference/latest/instances/list 1165[ filter: <string> ] 1166 1167# Refresh interval to re-read the instance list 1168[ refresh_interval: <duration> | default = 60s ] 1169 1170# The port to scrape metrics from. If using the public IP address, this must 1171# instead be specified in the relabeling rule. 1172[ port: <int> | default = 80 ] 1173 1174# The tag separator is used to separate the tags on concatenation 1175[ tag_separator: <string> | default = , ] 1176``` 1177 1178Credentials are discovered by the Google Cloud SDK default client by looking 1179in the following places, preferring the first location found: 1180 11811. a JSON file specified by the `GOOGLE_APPLICATION_CREDENTIALS` environment variable 11822. a JSON file in the well-known path `$HOME/.config/gcloud/application_default_credentials.json` 11833. fetched from the GCE metadata server 1184 1185If Prometheus is running within GCE, the service account associated with the 1186instance it is running on should have at least read-only permissions to the 1187compute resources. If running outside of GCE make sure to create an appropriate 1188service account and place the credential file in one of the expected locations. 1189 1190### `<hetzner_sd_config>` 1191 1192Hetzner SD configurations allow retrieving scrape targets from 1193[Hetzner](https://www.hetzner.com/) [Cloud](https://www.hetzner.cloud/) API and 1194[Robot](https://docs.hetzner.com/robot/) API. 1195This service discovery uses the public IPv4 address by default, but that can be 1196changed with relabeling, as demonstrated in [the Prometheus hetzner-sd 1197configuration file](/documentation/examples/prometheus-hetzner.yml). 1198 1199The following meta labels are available on all targets during [relabeling](#relabel_config): 1200 1201* `__meta_hetzner_server_id`: the ID of the server 1202* `__meta_hetzner_server_name`: the name of the server 1203* `__meta_hetzner_server_status`: the status of the server 1204* `__meta_hetzner_public_ipv4`: the public ipv4 address of the server 1205* `__meta_hetzner_public_ipv6_network`: the public ipv6 network (/64) of the server 1206* `__meta_hetzner_datacenter`: the datacenter of the server 1207 1208The labels below are only available for targets with `role` set to `hcloud`: 1209 1210* `__meta_hetzner_hcloud_image_name`: the image name of the server 1211* `__meta_hetzner_hcloud_image_description`: the description of the server image 1212* `__meta_hetzner_hcloud_image_os_flavor`: the OS flavor of the server image 1213* `__meta_hetzner_hcloud_image_os_version`: the OS version of the server image 1214* `__meta_hetzner_hcloud_image_description`: the description of the server image 1215* `__meta_hetzner_hcloud_datacenter_location`: the location of the server 1216* `__meta_hetzner_hcloud_datacenter_location_network_zone`: the network zone of the server 1217* `__meta_hetzner_hcloud_server_type`: the type of the server 1218* `__meta_hetzner_hcloud_cpu_cores`: the CPU cores count of the server 1219* `__meta_hetzner_hcloud_cpu_type`: the CPU type of the server (shared or dedicated) 1220* `__meta_hetzner_hcloud_memory_size_gb`: the amount of memory of the server (in GB) 1221* `__meta_hetzner_hcloud_disk_size_gb`: the disk size of the server (in GB) 1222* `__meta_hetzner_hcloud_private_ipv4_<networkname>`: the private ipv4 address of the server within a given network 1223* `__meta_hetzner_hcloud_label_<labelname>`: each label of the server 1224* `__meta_hetzner_hcloud_labelpresent_<labelname>`: `true` for each label of the server 1225 1226The labels below are only available for targets with `role` set to `robot`: 1227 1228* `__meta_hetzner_robot_product`: the product of the server 1229* `__meta_hetzner_robot_cancelled`: the server cancellation status 1230 1231```yaml 1232# The Hetzner role of entities that should be discovered. 1233# One of robot or hcloud. 1234role: <string> 1235 1236# Authentication information used to authenticate to the API server. 1237# Note that `basic_auth` and `authorization` options are 1238# mutually exclusive. 1239# password and password_file are mutually exclusive. 1240 1241# Optional HTTP basic authentication information, required when role is robot 1242# Role hcloud does not support basic auth. 1243basic_auth: 1244 [ username: <string> ] 1245 [ password: <secret> ] 1246 [ password_file: <string> ] 1247 1248# Optional `Authorization` header configuration, required when role is 1249# hcloud. Role robot does not support bearer token authentication. 1250authorization: 1251 # Sets the authentication type. 1252 [ type: <string> | default: Bearer ] 1253 # Sets the credentials. It is mutually exclusive with 1254 # `credentials_file`. 1255 [ credentials: <secret> ] 1256 # Sets the credentials to the credentials read from the configured file. 1257 # It is mutually exclusive with `credentials`. 1258 [ credentials_file: <filename> ] 1259 1260# Optional OAuth 2.0 configuration. 1261# Cannot be used at the same time as basic_auth or authorization. 1262oauth2: 1263 [ <oauth2> ] 1264 1265# Optional proxy URL. 1266[ proxy_url: <string> ] 1267 1268# Configure whether HTTP requests follow HTTP 3xx redirects. 1269[ follow_redirects: <bool> | default = true ] 1270 1271# TLS configuration. 1272tls_config: 1273 [ <tls_config> ] 1274 1275# The port to scrape metrics from. 1276[ port: <int> | default = 80 ] 1277 1278# The time after which the servers are refreshed. 1279[ refresh_interval: <duration> | default = 60s ] 1280``` 1281 1282### `<http_sd_config>` 1283 1284HTTP-based service discovery provides a more generic way to configure static targets 1285and serves as an interface to plug in custom service discovery mechanisms. 1286 1287It fetches targets from an HTTP endpoint containing a list of zero or more 1288`<static_config>`s. The target must reply with an HTTP 200 response. 1289The HTTP header `Content-Type` must be `application/json`, and the body must be 1290valid JSON. 1291 1292Example response body: 1293 1294```json 1295[ 1296 { 1297 "targets": [ "<host>", ... ], 1298 "labels": { 1299 "<labelname>": "<labelvalue>", ... 1300 } 1301 }, 1302 ... 1303] 1304``` 1305 1306The endpoint is queried periodically at the specified 1307refresh interval. 1308 1309Each target has a meta label `__meta_url` during the 1310[relabeling phase](#relabel_config). Its value is set to the 1311URL from which the target was extracted. 1312 1313```yaml 1314# URL from which the targets are fetched. 1315url: <string> 1316 1317# Refresh interval to re-query the endpoint. 1318[ refresh_interval: <duration> | default = 60s ] 1319 1320# Authentication information used to authenticate to the API server. 1321# Note that `basic_auth`, `authorization` and `oauth2` options are 1322# mutually exclusive. 1323# `password` and `password_file` are mutually exclusive. 1324 1325# Optional HTTP basic authentication information. 1326basic_auth: 1327 [ username: <string> ] 1328 [ password: <secret> ] 1329 [ password_file: <string> ] 1330 1331# Optional `Authorization` header configuration. 1332authorization: 1333 # Sets the authentication type. 1334 [ type: <string> | default: Bearer ] 1335 # Sets the credentials. It is mutually exclusive with 1336 # `credentials_file`. 1337 [ credentials: <secret> ] 1338 # Sets the credentials to the credentials read from the configured file. 1339 # It is mutually exclusive with `credentials`. 1340 [ credentials_file: <filename> ] 1341 1342# Optional OAuth 2.0 configuration. 1343oauth2: 1344 [ <oauth2> ] 1345 1346# Optional proxy URL. 1347[ proxy_url: <string> ] 1348 1349# Configure whether HTTP requests follow HTTP 3xx redirects. 1350[ follow_redirects: <bool> | default = true ] 1351 1352# TLS configuration. 1353tls_config: 1354 [ <tls_config> ] 1355``` 1356 1357### `<kubernetes_sd_config>` 1358 1359Kubernetes SD configurations allow retrieving scrape targets from 1360[Kubernetes'](https://kubernetes.io/) REST API and always staying synchronized with 1361the cluster state. 1362 1363One of the following `role` types can be configured to discover targets: 1364 1365#### `node` 1366 1367The `node` role discovers one target per cluster node with the address defaulting 1368to the Kubelet's HTTP port. 1369The target address defaults to the first existing address of the Kubernetes 1370node object in the address type order of `NodeInternalIP`, `NodeExternalIP`, 1371`NodeLegacyHostIP`, and `NodeHostName`. 1372 1373Available meta labels: 1374 1375* `__meta_kubernetes_node_name`: The name of the node object. 1376* `__meta_kubernetes_node_label_<labelname>`: Each label from the node object. 1377* `__meta_kubernetes_node_labelpresent_<labelname>`: `true` for each label from the node object. 1378* `__meta_kubernetes_node_annotation_<annotationname>`: Each annotation from the node object. 1379* `__meta_kubernetes_node_annotationpresent_<annotationname>`: `true` for each annotation from the node object. 1380* `__meta_kubernetes_node_address_<address_type>`: The first address for each node address type, if it exists. 1381 1382In addition, the `instance` label for the node will be set to the node name 1383as retrieved from the API server. 1384 1385#### `service` 1386 1387The `service` role discovers a target for each service port for each service. 1388This is generally useful for blackbox monitoring of a service. 1389The address will be set to the Kubernetes DNS name of the service and respective 1390service port. 1391 1392Available meta labels: 1393 1394* `__meta_kubernetes_namespace`: The namespace of the service object. 1395* `__meta_kubernetes_service_annotation_<annotationname>`: Each annotation from the service object. 1396* `__meta_kubernetes_service_annotationpresent_<annotationname>`: "true" for each annotation of the service object. 1397* `__meta_kubernetes_service_cluster_ip`: The cluster IP address of the service. (Does not apply to services of type ExternalName) 1398* `__meta_kubernetes_service_external_name`: The DNS name of the service. (Applies to services of type ExternalName) 1399* `__meta_kubernetes_service_label_<labelname>`: Each label from the service object. 1400* `__meta_kubernetes_service_labelpresent_<labelname>`: `true` for each label of the service object. 1401* `__meta_kubernetes_service_name`: The name of the service object. 1402* `__meta_kubernetes_service_port_name`: Name of the service port for the target. 1403* `__meta_kubernetes_service_port_protocol`: Protocol of the service port for the target. 1404* `__meta_kubernetes_service_type`: The type of the service. 1405 1406#### `pod` 1407 1408The `pod` role discovers all pods and exposes their containers as targets. For each declared 1409port of a container, a single target is generated. If a container has no specified ports, 1410a port-free target per container is created for manually adding a port via relabeling. 1411 1412Available meta labels: 1413 1414* `__meta_kubernetes_namespace`: The namespace of the pod object. 1415* `__meta_kubernetes_pod_name`: The name of the pod object. 1416* `__meta_kubernetes_pod_ip`: The pod IP of the pod object. 1417* `__meta_kubernetes_pod_label_<labelname>`: Each label from the pod object. 1418* `__meta_kubernetes_pod_labelpresent_<labelname>`: `true`for each label from the pod object. 1419* `__meta_kubernetes_pod_annotation_<annotationname>`: Each annotation from the pod object. 1420* `__meta_kubernetes_pod_annotationpresent_<annotationname>`: `true` for each annotation from the pod object. 1421* `__meta_kubernetes_pod_container_init`: `true` if the container is an [InitContainer](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) 1422* `__meta_kubernetes_pod_container_name`: Name of the container the target address points to. 1423* `__meta_kubernetes_pod_container_port_name`: Name of the container port. 1424* `__meta_kubernetes_pod_container_port_number`: Number of the container port. 1425* `__meta_kubernetes_pod_container_port_protocol`: Protocol of the container port. 1426* `__meta_kubernetes_pod_ready`: Set to `true` or `false` for the pod's ready state. 1427* `__meta_kubernetes_pod_phase`: Set to `Pending`, `Running`, `Succeeded`, `Failed` or `Unknown` 1428 in the [lifecycle](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-phase). 1429* `__meta_kubernetes_pod_node_name`: The name of the node the pod is scheduled onto. 1430* `__meta_kubernetes_pod_host_ip`: The current host IP of the pod object. 1431* `__meta_kubernetes_pod_uid`: The UID of the pod object. 1432* `__meta_kubernetes_pod_controller_kind`: Object kind of the pod controller. 1433* `__meta_kubernetes_pod_controller_name`: Name of the pod controller. 1434 1435#### `endpoints` 1436 1437The `endpoints` role discovers targets from listed endpoints of a service. For each endpoint 1438address one target is discovered per port. If the endpoint is backed by a pod, all 1439additional container ports of the pod, not bound to an endpoint port, are discovered as targets as well. 1440 1441Available meta labels: 1442 1443* `__meta_kubernetes_namespace`: The namespace of the endpoints object. 1444* `__meta_kubernetes_endpoints_name`: The names of the endpoints object. 1445* For all targets discovered directly from the endpoints list (those not additionally inferred 1446 from underlying pods), the following labels are attached: 1447 * `__meta_kubernetes_endpoint_hostname`: Hostname of the endpoint. 1448 * `__meta_kubernetes_endpoint_node_name`: Name of the node hosting the endpoint. 1449 * `__meta_kubernetes_endpoint_ready`: Set to `true` or `false` for the endpoint's ready state. 1450 * `__meta_kubernetes_endpoint_port_name`: Name of the endpoint port. 1451 * `__meta_kubernetes_endpoint_port_protocol`: Protocol of the endpoint port. 1452 * `__meta_kubernetes_endpoint_address_target_kind`: Kind of the endpoint address target. 1453 * `__meta_kubernetes_endpoint_address_target_name`: Name of the endpoint address target. 1454* If the endpoints belong to a service, all labels of the `role: service` discovery are attached. 1455* For all targets backed by a pod, all labels of the `role: pod` discovery are attached. 1456 1457#### `ingress` 1458 1459The `ingress` role discovers a target for each path of each ingress. 1460This is generally useful for blackbox monitoring of an ingress. 1461The address will be set to the host specified in the ingress spec. 1462 1463Available meta labels: 1464 1465* `__meta_kubernetes_namespace`: The namespace of the ingress object. 1466* `__meta_kubernetes_ingress_name`: The name of the ingress object. 1467* `__meta_kubernetes_ingress_label_<labelname>`: Each label from the ingress object. 1468* `__meta_kubernetes_ingress_labelpresent_<labelname>`: `true` for each label from the ingress object. 1469* `__meta_kubernetes_ingress_annotation_<annotationname>`: Each annotation from the ingress object. 1470* `__meta_kubernetes_ingress_annotationpresent_<annotationname>`: `true` for each annotation from the ingress object. 1471* `__meta_kubernetes_ingress_class_name`: Class name from ingress spec, if present. 1472* `__meta_kubernetes_ingress_scheme`: Protocol scheme of ingress, `https` if TLS 1473 config is set. Defaults to `http`. 1474* `__meta_kubernetes_ingress_path`: Path from ingress spec. Defaults to `/`. 1475 1476See below for the configuration options for Kubernetes discovery: 1477 1478```yaml 1479# The information to access the Kubernetes API. 1480 1481# The API server addresses. If left empty, Prometheus is assumed to run inside 1482# of the cluster and will discover API servers automatically and use the pod's 1483# CA certificate and bearer token file at /var/run/secrets/kubernetes.io/serviceaccount/. 1484[ api_server: <host> ] 1485 1486# The Kubernetes role of entities that should be discovered. 1487# One of endpoints, service, pod, node, or ingress. 1488role: <string> 1489 1490# Optional path to a kubeconfig file. 1491# Note that api_server and kube_config are mutually exclusive. 1492[ kubeconfig_file: <filename> ] 1493 1494# Optional authentication information used to authenticate to the API server. 1495# Note that `basic_auth` and `authorization` options are mutually exclusive. 1496# password and password_file are mutually exclusive. 1497 1498# Optional HTTP basic authentication information. 1499basic_auth: 1500 [ username: <string> ] 1501 [ password: <secret> ] 1502 [ password_file: <string> ] 1503 1504# Optional `Authorization` header configuration. 1505authorization: 1506 # Sets the authentication type. 1507 [ type: <string> | default: Bearer ] 1508 # Sets the credentials. It is mutually exclusive with 1509 # `credentials_file`. 1510 [ credentials: <secret> ] 1511 # Sets the credentials to the credentials read from the configured file. 1512 # It is mutually exclusive with `credentials`. 1513 [ credentials_file: <filename> ] 1514 1515# Optional OAuth 2.0 configuration. 1516# Cannot be used at the same time as basic_auth or authorization. 1517oauth2: 1518 [ <oauth2> ] 1519 1520# Optional proxy URL. 1521[ proxy_url: <string> ] 1522 1523# Configure whether HTTP requests follow HTTP 3xx redirects. 1524[ follow_redirects: <bool> | default = true ] 1525 1526# TLS configuration. 1527tls_config: 1528 [ <tls_config> ] 1529 1530# Optional namespace discovery. If omitted, all namespaces are used. 1531namespaces: 1532 names: 1533 [ - <string> ] 1534 1535# Optional label and field selectors to limit the discovery process to a subset of available resources. 1536# See https://kubernetes.io/docs/concepts/overview/working-with-objects/field-selectors/ 1537# and https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ to learn more about the possible 1538# filters that can be used. Endpoints role supports pod, service and endpoints selectors, other roles 1539# only support selectors matching the role itself (e.g. node role can only contain node selectors). 1540 1541# Note: When making decision about using field/label selector make sure that this 1542# is the best approach - it will prevent Prometheus from reusing single list/watch 1543# for all scrape configs. This might result in a bigger load on the Kubernetes API, 1544# because per each selector combination there will be additional LIST/WATCH. On the other hand, 1545# if you just want to monitor small subset of pods in large cluster it's recommended to use selectors. 1546# Decision, if selectors should be used or not depends on the particular situation. 1547[ selectors: 1548 [ - role: <string> 1549 [ label: <string> ] 1550 [ field: <string> ] ]] 1551``` 1552 1553See [this example Prometheus configuration file](/documentation/examples/prometheus-kubernetes.yml) 1554for a detailed example of configuring Prometheus for Kubernetes. 1555 1556You may wish to check out the 3rd party [Prometheus Operator](https://github.com/coreos/prometheus-operator), 1557which automates the Prometheus setup on top of Kubernetes. 1558 1559### `<kuma_sd_config>` 1560 1561Kuma SD configurations allow retrieving scrape target from the [Kuma](https://kuma.io) control plane. 1562 1563This SD discovers "monitoring assignments" based on Kuma [Dataplane Proxies](https://kuma.io/docs/latest/documentation/dps-and-data-model), 1564via the MADS v1 (Monitoring Assignment Discovery Service) xDS API, and will create a target for each proxy 1565inside a Prometheus-enabled mesh. 1566 1567The following meta labels are available for each target: 1568 1569* `__meta_kuma_mesh`: the name of the proxy's Mesh 1570* `__meta_kuma_dataplane`: the name of the proxy 1571* `__meta_kuma_service`: the name of the proxy's associated Service 1572* `__meta_kuma_label_<tagname>`: each tag of the proxy 1573 1574See below for the configuration options for Kuma MonitoringAssignment discovery: 1575 1576```yaml 1577# Address of the Kuma Control Plane's MADS xDS server. 1578server: <string> 1579 1580# The time to wait between polling update requests. 1581[ refresh_interval: <duration> | default = 30s ] 1582 1583# The time after which the monitoring assignments are refreshed. 1584[ fetch_timeout: <duration> | default = 2m ] 1585 1586# Optional proxy URL. 1587[ proxy_url: <string> ] 1588 1589# TLS configuration. 1590tls_config: 1591 [ <tls_config> ] 1592 1593# Authentication information used to authenticate to the Docker daemon. 1594# Note that `basic_auth` and `authorization` options are 1595# mutually exclusive. 1596# password and password_file are mutually exclusive. 1597 1598# Optional HTTP basic authentication information. 1599basic_auth: 1600 [ username: <string> ] 1601 [ password: <secret> ] 1602 [ password_file: <string> ] 1603 1604# Optional the `Authorization` header configuration. 1605authorization: 1606 # Sets the authentication type. 1607 [ type: <string> | default: Bearer ] 1608 # Sets the credentials. It is mutually exclusive with 1609 # `credentials_file`. 1610 [ credentials: <secret> ] 1611 # Sets the credentials with the credentials read from the configured file. 1612 # It is mutually exclusive with `credentials`. 1613 [ credentials_file: <filename> ] 1614 1615# Optional OAuth 2.0 configuration. 1616# Cannot be used at the same time as basic_auth or authorization. 1617oauth2: 1618 [ <oauth2> ] 1619 1620# Configure whether HTTP requests follow HTTP 3xx redirects. 1621[ follow_redirects: <bool> | default = true ] 1622``` 1623 1624The [relabeling phase](#relabel_config) is the preferred and more powerful way 1625to filter proxies and user-defined tags. 1626 1627### `<lightsail_sd_config>` 1628 1629Lightsail SD configurations allow retrieving scrape targets from [AWS Lightsail](https://aws.amazon.com/lightsail/) 1630instances. The private IP address is used by default, but may be changed to 1631the public IP address with relabeling. 1632 1633The following meta labels are available on targets during [relabeling](#relabel_config): 1634 1635* `__meta_lightsail_availability_zone`: the availability zone in which the instance is running 1636* `__meta_lightsail_blueprint_id`: the Lightsail blueprint ID 1637* `__meta_lightsail_bundle_id`: the Lightsail bundle ID 1638* `__meta_lightsail_instance_name`: the name of the Lightsail instance 1639* `__meta_lightsail_instance_state`: the state of the Lightsail instance 1640* `__meta_lightsail_instance_support_code`: the support code of the Lightsail instance 1641* `__meta_lightsail_ipv6_addresses`: comma separated list of IPv6 addresses assigned to the instance's network interfaces, if present 1642* `__meta_lightsail_private_ip`: the private IP address of the instance 1643* `__meta_lightsail_public_ip`: the public IP address of the instance, if available 1644* `__meta_lightsail_tag_<tagkey>`: each tag value of the instance 1645 1646See below for the configuration options for Lightsail discovery: 1647 1648```yaml 1649# The information to access the Lightsail API. 1650 1651# The AWS region. If blank, the region from the instance metadata is used. 1652[ region: <string> ] 1653 1654# Custom endpoint to be used. 1655[ endpoint: <string> ] 1656 1657# The AWS API keys. If blank, the environment variables `AWS_ACCESS_KEY_ID` 1658# and `AWS_SECRET_ACCESS_KEY` are used. 1659[ access_key: <string> ] 1660[ secret_key: <secret> ] 1661# Named AWS profile used to connect to the API. 1662[ profile: <string> ] 1663 1664# AWS Role ARN, an alternative to using AWS API keys. 1665[ role_arn: <string> ] 1666 1667# Refresh interval to re-read the instance list. 1668[ refresh_interval: <duration> | default = 60s ] 1669 1670# The port to scrape metrics from. If using the public IP address, this must 1671# instead be specified in the relabeling rule. 1672[ port: <int> | default = 80 ] 1673``` 1674 1675### `<linode_sd_config>` 1676 1677Linode SD configurations allow retrieving scrape targets from [Linode's](https://www.linode.com/) 1678Linode APIv4. 1679This service discovery uses the public IPv4 address by default, by that can be 1680changed with relabelling, as demonstrated in [the Prometheus linode-sd 1681configuration file](/documentation/examples/prometheus-linode.yml). 1682 1683The following meta labels are available on targets during [relabeling](#relabel_config): 1684 1685* `__meta_linode_instance_id`: the id of the linode instance 1686* `__meta_linode_instance_label`: the label of the linode instance 1687* `__meta_linode_image`: the slug of the linode instance's image 1688* `__meta_linode_private_ipv4`: the private IPv4 of the linode instance 1689* `__meta_linode_public_ipv4`: the public IPv4 of the linode instance 1690* `__meta_linode_public_ipv6`: the public IPv6 of the linode instance 1691* `__meta_linode_region`: the region of the linode instance 1692* `__meta_linode_type`: the type of the linode instance 1693* `__meta_linode_status`: the status of the linode instance 1694* `__meta_linode_tags`: a list of tags of the linode instance joined by the tag separator 1695* `__meta_linode_group`: the display group a linode instance is a member of 1696* `__meta_linode_hypervisor`: the virtualization software powering the linode instance 1697* `__meta_linode_backups`: the backup service status of the linode instance 1698* `__meta_linode_specs_disk_bytes`: the amount of storage space the linode instance has access to 1699* `__meta_linode_specs_memory_bytes`: the amount of RAM the linode instance has access to 1700* `__meta_linode_specs_vcpus`: the number of VCPUS this linode has access to 1701* `__meta_linode_specs_transfer_bytes`: the amount of network transfer the linode instance is allotted each month 1702* `__meta_linode_extra_ips`: a list of all extra IPv4 addresses assigned to the linode instance joined by the tag separator 1703 1704```yaml 1705# Authentication information used to authenticate to the API server. 1706# Note that `basic_auth` and `authorization` options are 1707# mutually exclusive. 1708# password and password_file are mutually exclusive. 1709# Note: Linode APIv4 Token must be created with scopes: 'linodes:read_only', 'ips:read_only', and 'events:read_only' 1710 1711# Optional HTTP basic authentication information, not currently supported by Linode APIv4. 1712basic_auth: 1713 [ username: <string> ] 1714 [ password: <secret> ] 1715 [ password_file: <string> ] 1716 1717# Optional the `Authorization` header configuration. 1718authorization: 1719 # Sets the authentication type. 1720 [ type: <string> | default: Bearer ] 1721 # Sets the credentials. It is mutually exclusive with 1722 # `credentials_file`. 1723 [ credentials: <secret> ] 1724 # Sets the credentials with the credentials read from the configured file. 1725 # It is mutually exclusive with `credentials`. 1726 [ credentials_file: <filename> ] 1727 1728# Optional OAuth 2.0 configuration. 1729# Cannot be used at the same time as basic_auth or authorization. 1730oauth2: 1731 [ <oauth2> ] 1732 1733# Optional proxy URL. 1734[ proxy_url: <string> ] 1735 1736# Configure whether HTTP requests follow HTTP 3xx redirects. 1737[ follow_redirects: <bool> | default = true ] 1738 1739# TLS configuration. 1740tls_config: 1741 [ <tls_config> ] 1742 1743# The port to scrape metrics from. 1744[ port: <int> | default = 80 ] 1745 1746# The string by which Linode Instance tags are joined into the tag label. 1747[ tag_separator: <string> | default = , ] 1748 1749# The time after which the linode instances are refreshed. 1750[ refresh_interval: <duration> | default = 60s ] 1751``` 1752 1753### `<marathon_sd_config>` 1754 1755Marathon SD configurations allow retrieving scrape targets using the 1756[Marathon](https://mesosphere.github.io/marathon/) REST API. Prometheus 1757will periodically check the REST endpoint for currently running tasks and 1758create a target group for every app that has at least one healthy task. 1759 1760The following meta labels are available on targets during [relabeling](#relabel_config): 1761 1762* `__meta_marathon_app`: the name of the app (with slashes replaced by dashes) 1763* `__meta_marathon_image`: the name of the Docker image used (if available) 1764* `__meta_marathon_task`: the ID of the Mesos task 1765* `__meta_marathon_app_label_<labelname>`: any Marathon labels attached to the app 1766* `__meta_marathon_port_definition_label_<labelname>`: the port definition labels 1767* `__meta_marathon_port_mapping_label_<labelname>`: the port mapping labels 1768* `__meta_marathon_port_index`: the port index number (e.g. `1` for `PORT1`) 1769 1770See below for the configuration options for Marathon discovery: 1771 1772```yaml 1773# List of URLs to be used to contact Marathon servers. 1774# You need to provide at least one server URL. 1775servers: 1776 - <string> 1777 1778# Polling interval 1779[ refresh_interval: <duration> | default = 30s ] 1780 1781# Optional authentication information for token-based authentication 1782# https://docs.mesosphere.com/1.11/security/ent/iam-api/#passing-an-authentication-token 1783# It is mutually exclusive with `auth_token_file` and other authentication mechanisms. 1784[ auth_token: <secret> ] 1785 1786# Optional authentication information for token-based authentication 1787# https://docs.mesosphere.com/1.11/security/ent/iam-api/#passing-an-authentication-token 1788# It is mutually exclusive with `auth_token` and other authentication mechanisms. 1789[ auth_token_file: <filename> ] 1790 1791# Sets the `Authorization` header on every request with the 1792# configured username and password. 1793# This is mutually exclusive with other authentication mechanisms. 1794# password and password_file are mutually exclusive. 1795basic_auth: 1796 [ username: <string> ] 1797 [ password: <secret> ] 1798 [ password_file: <string> ] 1799 1800# Optional `Authorization` header configuration. 1801# NOTE: The current version of DC/OS marathon (v1.11.0) does not support 1802# standard `Authentication` header, use `auth_token` or `auth_token_file` 1803# instead. 1804authorization: 1805 # Sets the authentication type. 1806 [ type: <string> | default: Bearer ] 1807 # Sets the credentials. It is mutually exclusive with 1808 # `credentials_file`. 1809 [ credentials: <secret> ] 1810 # Sets the credentials to the credentials read from the configured file. 1811 # It is mutually exclusive with `credentials`. 1812 [ credentials_file: <filename> ] 1813 1814# Optional OAuth 2.0 configuration. 1815# Cannot be used at the same time as basic_auth or authorization. 1816oauth2: 1817 [ <oauth2> ] 1818 1819# Configure whether HTTP requests follow HTTP 3xx redirects. 1820[ follow_redirects: <bool> | default = true ] 1821 1822# TLS configuration for connecting to marathon servers 1823tls_config: 1824 [ <tls_config> ] 1825 1826# Optional proxy URL. 1827[ proxy_url: <string> ] 1828``` 1829 1830By default every app listed in Marathon will be scraped by Prometheus. If not all 1831of your services provide Prometheus metrics, you can use a Marathon label and 1832Prometheus relabeling to control which instances will actually be scraped. 1833See [the Prometheus marathon-sd configuration file](/documentation/examples/prometheus-marathon.yml) 1834for a practical example on how to set up your Marathon app and your Prometheus 1835configuration. 1836 1837By default, all apps will show up as a single job in Prometheus (the one specified 1838in the configuration file), which can also be changed using relabeling. 1839 1840### `<nerve_sd_config>` 1841 1842Nerve SD configurations allow retrieving scrape targets from [AirBnB's Nerve] 1843(https://github.com/airbnb/nerve) which are stored in 1844[Zookeeper](https://zookeeper.apache.org/). 1845 1846The following meta labels are available on targets during [relabeling](#relabel_config): 1847 1848* `__meta_nerve_path`: the full path to the endpoint node in Zookeeper 1849* `__meta_nerve_endpoint_host`: the host of the endpoint 1850* `__meta_nerve_endpoint_port`: the port of the endpoint 1851* `__meta_nerve_endpoint_name`: the name of the endpoint 1852 1853```yaml 1854# The Zookeeper servers. 1855servers: 1856 - <host> 1857# Paths can point to a single service, or the root of a tree of services. 1858paths: 1859 - <string> 1860[ timeout: <duration> | default = 10s ] 1861``` 1862 1863### `<serverset_sd_config>` 1864 1865Serverset SD configurations allow retrieving scrape targets from [Serversets] 1866(https://github.com/twitter/finagle/tree/master/finagle-serversets) which are 1867stored in [Zookeeper](https://zookeeper.apache.org/). Serversets are commonly 1868used by [Finagle](https://twitter.github.io/finagle/) and 1869[Aurora](https://aurora.apache.org/). 1870 1871The following meta labels are available on targets during [relabeling](#relabel_config): 1872 1873* `__meta_serverset_path`: the full path to the serverset member node in Zookeeper 1874* `__meta_serverset_endpoint_host`: the host of the default endpoint 1875* `__meta_serverset_endpoint_port`: the port of the default endpoint 1876* `__meta_serverset_endpoint_host_<endpoint>`: the host of the given endpoint 1877* `__meta_serverset_endpoint_port_<endpoint>`: the port of the given endpoint 1878* `__meta_serverset_shard`: the shard number of the member 1879* `__meta_serverset_status`: the status of the member 1880 1881```yaml 1882# The Zookeeper servers. 1883servers: 1884 - <host> 1885# Paths can point to a single serverset, or the root of a tree of serversets. 1886paths: 1887 - <string> 1888[ timeout: <duration> | default = 10s ] 1889``` 1890 1891Serverset data must be in the JSON format, the Thrift format is not currently supported. 1892 1893### `<triton_sd_config>` 1894 1895[Triton](https://github.com/joyent/triton) SD configurations allow retrieving 1896scrape targets from [Container Monitor](https://github.com/joyent/rfd/blob/master/rfd/0027/README.md) 1897discovery endpoints. 1898 1899One of the following `<triton_role>` types can be configured to discover targets: 1900 1901#### `container` 1902 1903The `container` role discovers one target per "virtual machine" owned by the `account`. 1904These are SmartOS zones or lx/KVM/bhyve branded zones. 1905 1906The following meta labels are available on targets during [relabeling](#relabel_config): 1907 1908* `__meta_triton_groups`: the list of groups belonging to the target joined by a comma separator 1909* `__meta_triton_machine_alias`: the alias of the target container 1910* `__meta_triton_machine_brand`: the brand of the target container 1911* `__meta_triton_machine_id`: the UUID of the target container 1912* `__meta_triton_machine_image`: the target container's image type 1913* `__meta_triton_server_id`: the server UUID the target container is running on 1914 1915#### `cn` 1916 1917The `cn` role discovers one target for per compute node (also known as "server" or "global zone") making up the Triton infrastructure. 1918The `account` must be a Triton operator and is currently required to own at least one `container`. 1919 1920The following meta labels are available on targets during [relabeling](#relabel_config): 1921 1922* `__meta_triton_machine_alias`: the hostname of the target (requires triton-cmon 1.7.0 or newer) 1923* `__meta_triton_machine_id`: the UUID of the target 1924 1925See below for the configuration options for Triton discovery: 1926 1927```yaml 1928# The information to access the Triton discovery API. 1929 1930# The account to use for discovering new targets. 1931account: <string> 1932 1933# The type of targets to discover, can be set to: 1934# * "container" to discover virtual machines (SmartOS zones, lx/KVM/bhyve branded zones) running on Triton 1935# * "cn" to discover compute nodes (servers/global zones) making up the Triton infrastructure 1936[ role : <string> | default = "container" ] 1937 1938# The DNS suffix which should be applied to target. 1939dns_suffix: <string> 1940 1941# The Triton discovery endpoint (e.g. 'cmon.us-east-3b.triton.zone'). This is 1942# often the same value as dns_suffix. 1943endpoint: <string> 1944 1945# A list of groups for which targets are retrieved, only supported when `role` == `container`. 1946# If omitted all containers owned by the requesting account are scraped. 1947groups: 1948 [ - <string> ... ] 1949 1950# The port to use for discovery and metric scraping. 1951[ port: <int> | default = 9163 ] 1952 1953# The interval which should be used for refreshing targets. 1954[ refresh_interval: <duration> | default = 60s ] 1955 1956# The Triton discovery API version. 1957[ version: <int> | default = 1 ] 1958 1959# TLS configuration. 1960tls_config: 1961 [ <tls_config> ] 1962``` 1963 1964### `<eureka_sd_config>` 1965 1966Eureka SD configurations allow retrieving scrape targets using the 1967[Eureka](https://github.com/Netflix/eureka) REST API. Prometheus 1968will periodically check the REST endpoint and 1969create a target for every app instance. 1970 1971The following meta labels are available on targets during [relabeling](#relabel_config): 1972 1973* `__meta_eureka_app_name`: the name of the app 1974* `__meta_eureka_app_instance_id`: the ID of the app instance 1975* `__meta_eureka_app_instance_hostname`: the hostname of the instance 1976* `__meta_eureka_app_instance_homepage_url`: the homepage url of the app instance 1977* `__meta_eureka_app_instance_statuspage_url`: the status page url of the app instance 1978* `__meta_eureka_app_instance_healthcheck_url`: the health check url of the app instance 1979* `__meta_eureka_app_instance_ip_addr`: the IP address of the app instance 1980* `__meta_eureka_app_instance_vip_address`: the VIP address of the app instance 1981* `__meta_eureka_app_instance_secure_vip_address`: the secure VIP address of the app instance 1982* `__meta_eureka_app_instance_status`: the status of the app instance 1983* `__meta_eureka_app_instance_port`: the port of the app instance 1984* `__meta_eureka_app_instance_port_enabled`: the port enabled of the app instance 1985* `__meta_eureka_app_instance_secure_port`: the secure port address of the app instance 1986* `__meta_eureka_app_instance_secure_port_enabled`: the secure port of the app instance 1987* `__meta_eureka_app_instance_country_id`: the country ID of the app instance 1988* `__meta_eureka_app_instance_metadata_<metadataname>`: app instance metadata 1989* `__meta_eureka_app_instance_datacenterinfo_name`: the datacenter name of the app instance 1990* `__meta_eureka_app_instance_datacenterinfo_<metadataname>`: the datacenter metadata 1991 1992See below for the configuration options for Eureka discovery: 1993 1994```yaml 1995# The URL to connect to the Eureka server. 1996server: <string> 1997 1998# Sets the `Authorization` header on every request with the 1999# configured username and password. 2000# password and password_file are mutually exclusive. 2001basic_auth: 2002 [ username: <string> ] 2003 [ password: <secret> ] 2004 [ password_file: <string> ] 2005 2006# Optional `Authorization` header configuration. 2007authorization: 2008 # Sets the authentication type. 2009 [ type: <string> | default: Bearer ] 2010 # Sets the credentials. It is mutually exclusive with 2011 # `credentials_file`. 2012 [ credentials: <secret> ] 2013 # Sets the credentials to the credentials read from the configured file. 2014 # It is mutually exclusive with `credentials`. 2015 [ credentials_file: <filename> ] 2016 2017# Optional OAuth 2.0 configuration. 2018# Cannot be used at the same time as basic_auth or authorization. 2019oauth2: 2020 [ <oauth2> ] 2021 2022# Configures the scrape request's TLS settings. 2023tls_config: 2024 [ <tls_config> ] 2025 2026# Optional proxy URL. 2027[ proxy_url: <string> ] 2028 2029# Configure whether HTTP requests follow HTTP 3xx redirects. 2030[ follow_redirects: <bool> | default = true ] 2031 2032# Refresh interval to re-read the app instance list. 2033[ refresh_interval: <duration> | default = 30s ] 2034``` 2035 2036See [the Prometheus eureka-sd configuration file](/documentation/examples/prometheus-eureka.yml) 2037for a practical example on how to set up your Eureka app and your Prometheus 2038configuration. 2039 2040### `<scaleway_sd_config>` 2041 2042Scaleway SD configurations allow retrieving scrape targets from [Scaleway instances](https://www.scaleway.com/en/virtual-instances/) and [baremetal services](https://www.scaleway.com/en/bare-metal-servers/). 2043 2044The following meta labels are available on targets during [relabeling](#relabel_config): 2045 2046#### Instance role 2047 2048 2049* `__meta_scaleway_instance_boot_type`: the boot type of the server 2050* `__meta_scaleway_instance_hostname`: the hostname of the server 2051* `__meta_scaleway_instance_id`: the ID of the server 2052* `__meta_scaleway_instance_image_arch`: the arch of the server image 2053* `__meta_scaleway_instance_image_id`: the ID of the server image 2054* `__meta_scaleway_instance_image_name`: the name of the server image 2055* `__meta_scaleway_instance_location_cluster_id`: the cluster ID of the server location 2056* `__meta_scaleway_instance_location_hypervisor_id`: the hypervisor ID of the server location 2057* `__meta_scaleway_instance_location_node_id`: the node ID of the server location 2058* `__meta_scaleway_instance_name`: name of the server 2059* `__meta_scaleway_instance_organization_id`: the organization of the server 2060* `__meta_scaleway_instance_private_ipv4`: the private IPv4 address of the server 2061* `__meta_scaleway_instance_project_id`: project id of the server 2062* `__meta_scaleway_instance_public_ipv4`: the public IPv4 address of the server 2063* `__meta_scaleway_instance_public_ipv6`: the public IPv6 address of the server 2064* `__meta_scaleway_instance_region`: the region of the server 2065* `__meta_scaleway_instance_security_group_id`: the ID of the security group of the server 2066* `__meta_scaleway_instance_security_group_name`: the name of the security group of the server 2067* `__meta_scaleway_instance_status`: status of the server 2068* `__meta_scaleway_instance_tags`: the list of tags of the server joined by the tag separator 2069* `__meta_scaleway_instance_type`: commercial type of the server 2070* `__meta_scaleway_instance_zone`: the zone of the server (ex: `fr-par-1`, complete list [here](https://developers.scaleway.com/en/products/instance/api/#introduction)) 2071 2072This role uses the private IPv4 address by default. This can be 2073changed with relabelling, as demonstrated in [the Prometheus scaleway-sd 2074configuration file](/documentation/examples/prometheus-scaleway.yml). 2075 2076#### Baremetal role 2077 2078* `__meta_scaleway_baremetal_id`: the ID of the server 2079* `__meta_scaleway_baremetal_public_ipv4`: the public IPv4 address of the server 2080* `__meta_scaleway_baremetal_public_ipv6`: the public IPv6 address of the server 2081* `__meta_scaleway_baremetal_name`: the name of the server 2082* `__meta_scaleway_baremetal_os_name`: the name of the operating system of the server 2083* `__meta_scaleway_baremetal_os_version`: the version of the operating system of the server 2084* `__meta_scaleway_baremetal_project_id`: the project ID of the server 2085* `__meta_scaleway_baremetal_status`: the status of the server 2086* `__meta_scaleway_baremetal_tags`: the list of tags of the server joined by the tag separator 2087* `__meta_scaleway_baremetal_type`: the commercial type of the server 2088* `__meta_scaleway_baremetal_zone`: the zone of the server (ex: `fr-par-1`, complete list [here](https://developers.scaleway.com/en/products/instance/api/#introduction)) 2089 2090This role uses the public IPv4 address by default. This can be 2091changed with relabelling, as demonstrated in [the Prometheus scaleway-sd 2092configuration file](/documentation/examples/prometheus-scaleway.yml). 2093 2094See below for the configuration options for Scaleway discovery: 2095 2096```yaml 2097# Access key to use. https://console.scaleway.com/project/credentials 2098access_key: <string> 2099 2100# Secret key to use when listing targets. https://console.scaleway.com/project/credentials 2101# It is mutually exclusive with `secret_key_file`. 2102[ secret_key: <secret> ] 2103 2104# Sets the secret key with the credentials read from the configured file. 2105# It is mutually exclusive with `secret_key`. 2106[ secret_key_file: <filename> ] 2107 2108# Project ID of the targets. 2109project_id: <string> 2110 2111# Role of the targets to retrieve. Must be `instance` or `baremetal`. 2112role: <string> 2113 2114# The port to scrape metrics from. 2115[ port: <int> | default = 80 ] 2116 2117# API URL to use when doing the server listing requests. 2118[ api_url: <string> | default = "https://api.scaleway.com" ] 2119 2120# Zone is the availability zone of your targets (e.g. fr-par-1). 2121[ zone: <string> | default = fr-par-1 ] 2122 2123# NameFilter specify a name filter (works as a LIKE) to apply on the server listing request. 2124[ name_filter: <string> ] 2125 2126# TagsFilter specify a tag filter (a server needs to have all defined tags to be listed) to apply on the server listing request. 2127tags_filter: 2128[ - <string> ] 2129 2130# Refresh interval to re-read the targets list. 2131[ refresh_interval: <duration> | default = 60s ] 2132 2133# Configure whether HTTP requests follow HTTP 3xx redirects. 2134[ follow_redirects: <bool> | default = true ] 2135 2136# Optional proxy URL. 2137[ proxy_url: <string> ] 2138 2139# TLS configuration. 2140tls_config: 2141 [ <tls_config> ] 2142``` 2143 2144### `<static_config>` 2145 2146A `static_config` allows specifying a list of targets and a common label set 2147for them. It is the canonical way to specify static targets in a scrape 2148configuration. 2149 2150```yaml 2151# The targets specified by the static config. 2152targets: 2153 [ - '<host>' ] 2154 2155# Labels assigned to all metrics scraped from the targets. 2156labels: 2157 [ <labelname>: <labelvalue> ... ] 2158``` 2159 2160### `<relabel_config>` 2161 2162Relabeling is a powerful tool to dynamically rewrite the label set of a target before 2163it gets scraped. Multiple relabeling steps can be configured per scrape configuration. 2164They are applied to the label set of each target in order of their appearance 2165in the configuration file. 2166 2167Initially, aside from the configured per-target labels, a target's `job` 2168label is set to the `job_name` value of the respective scrape configuration. 2169The `__address__` label is set to the `<host>:<port>` address of the target. 2170After relabeling, the `instance` label is set to the value of `__address__` by default if 2171it was not set during relabeling. The `__scheme__` and `__metrics_path__` labels 2172are set to the scheme and metrics path of the target respectively. The `__param_<name>` 2173label is set to the value of the first passed URL parameter called `<name>`. 2174 2175The `__scrape_interval__` and `__scrape_timeout__` labels are set to the target's 2176interval and timeout. This is **experimental** and could change in the future. 2177 2178Additional labels prefixed with `__meta_` may be available during the 2179relabeling phase. They are set by the service discovery mechanism that provided 2180the target and vary between mechanisms. 2181 2182Labels starting with `__` will be removed from the label set after target 2183relabeling is completed. 2184 2185If a relabeling step needs to store a label value only temporarily (as the 2186input to a subsequent relabeling step), use the `__tmp` label name prefix. This 2187prefix is guaranteed to never be used by Prometheus itself. 2188 2189```yaml 2190# The source labels select values from existing labels. Their content is concatenated 2191# using the configured separator and matched against the configured regular expression 2192# for the replace, keep, and drop actions. 2193[ source_labels: '[' <labelname> [, ...] ']' ] 2194 2195# Separator placed between concatenated source label values. 2196[ separator: <string> | default = ; ] 2197 2198# Label to which the resulting value is written in a replace action. 2199# It is mandatory for replace actions. Regex capture groups are available. 2200[ target_label: <labelname> ] 2201 2202# Regular expression against which the extracted value is matched. 2203[ regex: <regex> | default = (.*) ] 2204 2205# Modulus to take of the hash of the source label values. 2206[ modulus: <int> ] 2207 2208# Replacement value against which a regex replace is performed if the 2209# regular expression matches. Regex capture groups are available. 2210[ replacement: <string> | default = $1 ] 2211 2212# Action to perform based on regex matching. 2213[ action: <relabel_action> | default = replace ] 2214``` 2215 2216`<regex>` is any valid 2217[RE2 regular expression](https://github.com/google/re2/wiki/Syntax). It is 2218required for the `replace`, `keep`, `drop`, `labelmap`,`labeldrop` and `labelkeep` actions. The regex is 2219anchored on both ends. To un-anchor the regex, use `.*<regex>.*`. 2220 2221`<relabel_action>` determines the relabeling action to take: 2222 2223* `replace`: Match `regex` against the concatenated `source_labels`. Then, set 2224 `target_label` to `replacement`, with match group references 2225 (`${1}`, `${2}`, ...) in `replacement` substituted by their value. If `regex` 2226 does not match, no replacement takes place. 2227* `keep`: Drop targets for which `regex` does not match the concatenated `source_labels`. 2228* `drop`: Drop targets for which `regex` matches the concatenated `source_labels`. 2229* `hashmod`: Set `target_label` to the `modulus` of a hash of the concatenated `source_labels`. 2230* `labelmap`: Match `regex` against all label names. Then copy the values of the matching labels 2231 to label names given by `replacement` with match group references 2232 (`${1}`, `${2}`, ...) in `replacement` substituted by their value. 2233* `labeldrop`: Match `regex` against all label names. Any label that matches will be 2234 removed from the set of labels. 2235* `labelkeep`: Match `regex` against all label names. Any label that does not match will be 2236 removed from the set of labels. 2237 2238Care must be taken with `labeldrop` and `labelkeep` to ensure that metrics are 2239still uniquely labeled once the labels are removed. 2240 2241### `<metric_relabel_configs>` 2242 2243Metric relabeling is applied to samples as the last step before ingestion. It 2244has the same configuration format and actions as target relabeling. Metric 2245relabeling does not apply to automatically generated timeseries such as `up`. 2246 2247One use for this is to exclude time series that are too expensive to ingest. 2248 2249### `<alert_relabel_configs>` 2250 2251Alert relabeling is applied to alerts before they are sent to the Alertmanager. 2252It has the same configuration format and actions as target relabeling. Alert 2253relabeling is applied after external labels. 2254 2255One use for this is ensuring a HA pair of Prometheus servers with different 2256external labels send identical alerts. 2257 2258### `<alertmanager_config>` 2259 2260An `alertmanager_config` section specifies Alertmanager instances the Prometheus 2261server sends alerts to. It also provides parameters to configure how to 2262communicate with these Alertmanagers. 2263 2264Alertmanagers may be statically configured via the `static_configs` parameter or 2265dynamically discovered using one of the supported service-discovery mechanisms. 2266 2267Additionally, `relabel_configs` allow selecting Alertmanagers from discovered 2268entities and provide advanced modifications to the used API path, which is exposed 2269through the `__alerts_path__` label. 2270 2271```yaml 2272# Per-target Alertmanager timeout when pushing alerts. 2273[ timeout: <duration> | default = 10s ] 2274 2275# The api version of Alertmanager. 2276[ api_version: <string> | default = v2 ] 2277 2278# Prefix for the HTTP path alerts are pushed to. 2279[ path_prefix: <path> | default = / ] 2280 2281# Configures the protocol scheme used for requests. 2282[ scheme: <scheme> | default = http ] 2283 2284# Sets the `Authorization` header on every request with the 2285# configured username and password. 2286# password and password_file are mutually exclusive. 2287basic_auth: 2288 [ username: <string> ] 2289 [ password: <secret> ] 2290 [ password_file: <string> ] 2291 2292# Optional `Authorization` header configuration. 2293authorization: 2294 # Sets the authentication type. 2295 [ type: <string> | default: Bearer ] 2296 # Sets the credentials. It is mutually exclusive with 2297 # `credentials_file`. 2298 [ credentials: <secret> ] 2299 # Sets the credentials to the credentials read from the configured file. 2300 # It is mutually exclusive with `credentials`. 2301 [ credentials_file: <filename> ] 2302 2303# Optional OAuth 2.0 configuration. 2304# Cannot be used at the same time as basic_auth or authorization. 2305oauth2: 2306 [ <oauth2> ] 2307 2308# Configures the scrape request's TLS settings. 2309tls_config: 2310 [ <tls_config> ] 2311 2312# Optional proxy URL. 2313[ proxy_url: <string> ] 2314 2315# Configure whether HTTP requests follow HTTP 3xx redirects. 2316[ follow_redirects: <bool> | default = true ] 2317 2318# List of Azure service discovery configurations. 2319azure_sd_configs: 2320 [ - <azure_sd_config> ... ] 2321 2322# List of Consul service discovery configurations. 2323consul_sd_configs: 2324 [ - <consul_sd_config> ... ] 2325 2326# List of DNS service discovery configurations. 2327dns_sd_configs: 2328 [ - <dns_sd_config> ... ] 2329 2330# List of EC2 service discovery configurations. 2331ec2_sd_configs: 2332 [ - <ec2_sd_config> ... ] 2333 2334# List of Eureka service discovery configurations. 2335eureka_sd_configs: 2336 [ - <eureka_sd_config> ... ] 2337 2338# List of file service discovery configurations. 2339file_sd_configs: 2340 [ - <file_sd_config> ... ] 2341 2342# List of DigitalOcean service discovery configurations. 2343digitalocean_sd_configs: 2344 [ - <digitalocean_sd_config> ... ] 2345 2346# List of Docker service discovery configurations. 2347docker_sd_configs: 2348 [ - <docker_sd_config> ... ] 2349 2350# List of Docker Swarm service discovery configurations. 2351dockerswarm_sd_configs: 2352 [ - <dockerswarm_sd_config> ... ] 2353 2354# List of GCE service discovery configurations. 2355gce_sd_configs: 2356 [ - <gce_sd_config> ... ] 2357 2358# List of Hetzner service discovery configurations. 2359hetzner_sd_configs: 2360 [ - <hetzner_sd_config> ... ] 2361 2362# List of HTTP service discovery configurations. 2363http_sd_configs: 2364 [ - <http_sd_config> ... ] 2365 2366# List of Kubernetes service discovery configurations. 2367kubernetes_sd_configs: 2368 [ - <kubernetes_sd_config> ... ] 2369 2370# List of Lightsail service discovery configurations. 2371lightsail_sd_configs: 2372 [ - <lightsail_sd_config> ... ] 2373 2374# List of Linode service discovery configurations. 2375linode_sd_configs: 2376 [ - <linode_sd_config> ... ] 2377 2378# List of Marathon service discovery configurations. 2379marathon_sd_configs: 2380 [ - <marathon_sd_config> ... ] 2381 2382# List of AirBnB's Nerve service discovery configurations. 2383nerve_sd_configs: 2384 [ - <nerve_sd_config> ... ] 2385 2386# List of OpenStack service discovery configurations. 2387openstack_sd_configs: 2388 [ - <openstack_sd_config> ... ] 2389 2390# List of Scaleway service discovery configurations. 2391scaleway_sd_configs: 2392 [ - <scaleway_sd_config> ... ] 2393 2394# List of Zookeeper Serverset service discovery configurations. 2395serverset_sd_configs: 2396 [ - <serverset_sd_config> ... ] 2397 2398# List of Triton service discovery configurations. 2399triton_sd_configs: 2400 [ - <triton_sd_config> ... ] 2401 2402# List of labeled statically configured Alertmanagers. 2403static_configs: 2404 [ - <static_config> ... ] 2405 2406# List of Alertmanager relabel configurations. 2407relabel_configs: 2408 [ - <relabel_config> ... ] 2409``` 2410 2411### `<remote_write>` 2412 2413`write_relabel_configs` is relabeling applied to samples before sending them 2414to the remote endpoint. Write relabeling is applied after external labels. This 2415could be used to limit which samples are sent. 2416 2417There is a [small demo](/documentation/examples/remote_storage) of how to use 2418this functionality. 2419 2420```yaml 2421# The URL of the endpoint to send samples to. 2422url: <string> 2423 2424# Timeout for requests to the remote write endpoint. 2425[ remote_timeout: <duration> | default = 30s ] 2426 2427# Custom HTTP headers to be sent along with each remote write request. 2428# Be aware that headers that are set by Prometheus itself can't be overwritten. 2429headers: 2430 [ <string>: <string> ... ] 2431 2432# List of remote write relabel configurations. 2433write_relabel_configs: 2434 [ - <relabel_config> ... ] 2435 2436# Name of the remote write config, which if specified must be unique among remote write configs. 2437# The name will be used in metrics and logging in place of a generated value to help users distinguish between 2438# remote write configs. 2439[ name: <string> ] 2440 2441# Enables sending of exemplars over remote write. Note that exemplar storage itself must be enabled for exemplars to be scraped in the first place. 2442[ send_exemplars: <boolean> | default = false ] 2443 2444# Sets the `Authorization` header on every remote write request with the 2445# configured username and password. 2446# password and password_file are mutually exclusive. 2447basic_auth: 2448 [ username: <string> ] 2449 [ password: <secret> ] 2450 [ password_file: <string> ] 2451 2452# Optional `Authorization` header configuration. 2453authorization: 2454 # Sets the authentication type. 2455 [ type: <string> | default: Bearer ] 2456 # Sets the credentials. It is mutually exclusive with 2457 # `credentials_file`. 2458 [ credentials: <secret> ] 2459 # Sets the credentials to the credentials read from the configured file. 2460 # It is mutually exclusive with `credentials`. 2461 [ credentials_file: <filename> ] 2462 2463# Optionally configures AWS's Signature Verification 4 signing process to 2464# sign requests. Cannot be set at the same time as basic_auth, authorization, or oauth2. 2465# To use the default credentials from the AWS SDK, use `sigv4: {}`. 2466sigv4: 2467 # The AWS region. If blank, the region from the default credentials chain 2468 # is used. 2469 [ region: <string> ] 2470 2471 # The AWS API keys. If blank, the environment variables `AWS_ACCESS_KEY_ID` 2472 # and `AWS_SECRET_ACCESS_KEY` are used. 2473 [ access_key: <string> ] 2474 [ secret_key: <secret> ] 2475 2476 # Named AWS profile used to authenticate. 2477 [ profile: <string> ] 2478 2479 # AWS Role ARN, an alternative to using AWS API keys. 2480 [ role_arn: <string> ] 2481 2482# Optional OAuth 2.0 configuration. 2483# Cannot be used at the same time as basic_auth, authorization, or sigv4. 2484oauth2: 2485 [ <oauth2> ] 2486 2487# Configures the remote write request's TLS settings. 2488tls_config: 2489 [ <tls_config> ] 2490 2491# Optional proxy URL. 2492[ proxy_url: <string> ] 2493 2494# Configure whether HTTP requests follow HTTP 3xx redirects. 2495[ follow_redirects: <bool> | default = true ] 2496 2497# Configures the queue used to write to remote storage. 2498queue_config: 2499 # Number of samples to buffer per shard before we block reading of more 2500 # samples from the WAL. It is recommended to have enough capacity in each 2501 # shard to buffer several requests to keep throughput up while processing 2502 # occasional slow remote requests. 2503 [ capacity: <int> | default = 2500 ] 2504 # Maximum number of shards, i.e. amount of concurrency. 2505 [ max_shards: <int> | default = 200 ] 2506 # Minimum number of shards, i.e. amount of concurrency. 2507 [ min_shards: <int> | default = 1 ] 2508 # Maximum number of samples per send. 2509 [ max_samples_per_send: <int> | default = 500] 2510 # Maximum time a sample will wait in buffer. 2511 [ batch_send_deadline: <duration> | default = 5s ] 2512 # Initial retry delay. Gets doubled for every retry. 2513 [ min_backoff: <duration> | default = 30ms ] 2514 # Maximum retry delay. 2515 [ max_backoff: <duration> | default = 100ms ] 2516 # Retry upon receiving a 429 status code from the remote-write storage. 2517 # This is experimental and might change in the future. 2518 [ retry_on_http_429: <boolean> | default = false ] 2519 2520# Configures the sending of series metadata to remote storage. 2521# Metadata configuration is subject to change at any point 2522# or be removed in future releases. 2523metadata_config: 2524 # Whether metric metadata is sent to remote storage or not. 2525 [ send: <boolean> | default = true ] 2526 # How frequently metric metadata is sent to remote storage. 2527 [ send_interval: <duration> | default = 1m ] 2528 # Maximum number of samples per send. 2529 [ max_samples_per_send: <int> | default = 500] 2530``` 2531 2532There is a list of 2533[integrations](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage) 2534with this feature. 2535 2536### `<remote_read>` 2537 2538```yaml 2539# The URL of the endpoint to query from. 2540url: <string> 2541 2542# Name of the remote read config, which if specified must be unique among remote read configs. 2543# The name will be used in metrics and logging in place of a generated value to help users distinguish between 2544# remote read configs. 2545[ name: <string> ] 2546 2547# An optional list of equality matchers which have to be 2548# present in a selector to query the remote read endpoint. 2549required_matchers: 2550 [ <labelname>: <labelvalue> ... ] 2551 2552# Timeout for requests to the remote read endpoint. 2553[ remote_timeout: <duration> | default = 1m ] 2554 2555# Custom HTTP headers to be sent along with each remote read request. 2556# Be aware that headers that are set by Prometheus itself can't be overwritten. 2557headers: 2558 [ <string>: <string> ... ] 2559 2560# Whether reads should be made for queries for time ranges that 2561# the local storage should have complete data for. 2562[ read_recent: <boolean> | default = false ] 2563 2564# Sets the `Authorization` header on every remote read request with the 2565# configured username and password. 2566# password and password_file are mutually exclusive. 2567basic_auth: 2568 [ username: <string> ] 2569 [ password: <secret> ] 2570 [ password_file: <string> ] 2571 2572# Optional `Authorization` header configuration. 2573authorization: 2574 # Sets the authentication type. 2575 [ type: <string> | default: Bearer ] 2576 # Sets the credentials. It is mutually exclusive with 2577 # `credentials_file`. 2578 [ credentials: <secret> ] 2579 # Sets the credentials to the credentials read from the configured file. 2580 # It is mutually exclusive with `credentials`. 2581 [ credentials_file: <filename> ] 2582 2583# Optional OAuth 2.0 configuration. 2584# Cannot be used at the same time as basic_auth or authorization. 2585oauth2: 2586 [ <oauth2> ] 2587 2588# Configures the remote read request's TLS settings. 2589tls_config: 2590 [ <tls_config> ] 2591 2592# Optional proxy URL. 2593[ proxy_url: <string> ] 2594 2595# Configure whether HTTP requests follow HTTP 3xx redirects. 2596[ follow_redirects: <bool> | default = true ] 2597``` 2598 2599There is a list of 2600[integrations](https://prometheus.io/docs/operating/integrations/#remote-endpoints-and-storage) 2601with this feature. 2602