[[options]] = Options [partintro] -- Options are settings used by <>. * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> * <> You can use <> in your configuration files. -- [[option_allocation_type]] == allocation_type NOTE: This setting is used only when using the <> [source,yaml] ------------- action: allocation description: "Apply shard allocation filtering rules to the specified indices" options: key: ... value: ... allocation_type: ... filters: - filtertype: ... ------------- The value of this setting must be one of `require`, `include`, or `exclude`. Read more about these settings at {ref}/shard-allocation-filtering.html The default value for this setting is `require`. [[option_allow_ilm]] == allow_ilm_indices This option allows Curator to manage ILM-enabled indices. Exercise caution that ILM policies and Curator actions will not interfere with each other. IMPORTANT: Read more about Curator and Index Lifecycle Management <>. [source,yaml] ------------- action: delete_indices description: "Delete the specified indices" options: allow_ilm_indices: true filters: - filtertype: ... ------------- The value of this setting must be either `true` or `false`. The default value for this setting is `false`. [[option_continue]] == continue_if_exception [IMPORTANT] .Using `ignore_empty_list` rather than `continue_if_exception` ==================================== Curator has two general classifications of exceptions: Empty list exceptions, and everything else. The empty list conditions are `curator.exception.NoIndices` and `curator.exception.NoSnapshots`. The `continue_if_exception` option _only_ catches conditions _other_ than empty list conditions. In most cases, you will want to use `ignore_empty_list` instead of `continue_if_exception`. So why are there two kinds of exceptions? When Curator 4 was released, the ability to continue in the event of any exception was covered by the `continue_if_exception` option. However, an empty list is a _benign_ condition. In fact, it's expected with brand new clusters, or when new index patterns are added. The decision was made to split the exceptions, and have a new option catch the empty lists. See <> for more information. ==================================== NOTE: This setting is available in all actions. [source,yaml] ------------- action: delete_indices description: "Delete selected indices" options: continue_if_exception: False filters: - filtertype: ... ------------- If `continue_if_exception` is set to `True`, Curator will attempt to continue on to the next action, if any, even if an exception is encountered. Curator will log but ignore the exception that was raised. The default value for this setting is `False` [[option_copy_aliases]] == copy_aliases NOTE: This setting is only used by the <> action. [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Reimplement source index aliases on target index after successful shrink. options: shrink_node: DETERMINISTIC copy_aliases: True filters: - filtertype: ... ------------- The default value of this setting is `False`. If `True`, after an index has been successfully shrunk, any aliases associated with the source index will be copied to the target index. [[option_count]] == count NOTE: This setting is required when using the <>. [source,yaml] ------------- action: replicas description: >- Set the number of replicas per shard for selected indices to 'count' options: count: ... filters: - filtertype: ... ------------- The value for this setting is the number of replicas to assign to matching indices. There is no default value. This setting must be set by the user or an exception will be raised, and execution will halt. [[option_delay]] == delay NOTE: This setting is only used by the <>, and is optional. [source,yaml] ------------- action: forcemerge description: >- Perform a forceMerge on selected indices to 'max_num_segments' per shard options: max_num_segments: 2 timeout_override: 21600 delay: 120 filters: - filtertype: ... ------------- The value for this setting is the number of seconds to delay between forceMerging indices, to allow the cluster to quiesce. There is no default value. [[option_delete_after]] == delete_after NOTE: This setting is only used by the <> action. [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Delete source index after successful shrink. options: shrink_node: DETERMINISTIC delete_after: True filters: - filtertype: ... ------------- The default value of this setting is `True`. After an index has been successfully shrunk, the source index will be deleted or preserved based on the value of this setting. [[option_delete_aliases]] == delete_aliases NOTE: This setting is only used by the <>, and is optional. [source,yaml] ------------- action: close description: "Close selected indices" options: delete_aliases: False filters: - filtertype: ... ------------- The value for this setting determines whether aliases will be deleted from indices before closing. The default value is `False`. [[option_skip_flush]] == skip_flush NOTE: This setting is only used by the <>, and is optional. [source,yaml] ------------- action: close description: "Close selected indices" options: skip_flush: False filters: - filtertype: ... ------------- If `skip_flush` is set to `True`, Curator will not flush indices to disk before closing. This may be useful for closing red indices before restoring. The default value is `False`. [[option_disable]] == disable_action NOTE: This setting is available in all actions. [source,yaml] ------------- action: delete_indices description: "Delete selected indices" options: disable_action: False filters: - filtertype: ... ------------- If `disable_action` is set to `True`, Curator will ignore the current action. This may be useful for temporarily disabling actions in a large configuration file. The default value for this setting is `False` [[option_extra_settings]] == extra_settings This setting should be nested YAML. The values beneath `extra_settings` will be used by whichever action uses the option. === <> [source,yaml] ------------- action: alias description: "Add/Remove selected indices to or from the specified alias" options: name: alias_name extra_settings: filter: term: user: kimchy add: filters: - filtertype: ... remove: filters: - filtertype: ... ------------- === <> [source,yaml] ------------- action: create_index description: "Create index as named" options: name: myindex # ... extra_settings: settings: number_of_shards: 1 number_of_replicas: 0 mappings: type1: properties: field1: type: string index: not_analyzed ------------- === <> See the {ref}/snapshots-restore-snapshot.html#change-index-settings-during-restore[official Elasticsearch Documentation]. [source,yaml] ------------- actions: 1: action: restore description: >- Restore all indices in the most recent snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do not skip the repository filesystem access check. Use the other options to define the index/shard settings for the restore. options: repository: # If name is blank, the most recent snapshot by age will be selected name: # If indices is blank, all indices in the snapshot will be restored indices: extra_settings: index_settings: number_of_replicas: 0 wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- === <> [source,yaml] ------------- action: rollover description: >- Rollover the index associated with alias 'name', which should be in the form of prefix-000001 (or similar), or prefix-yyyy.MM.DD-1. options: name: aliasname conditions: max_age: 1d max_docs: 1000000 extra_settings: index.number_of_shards: 3 index.number_of_replicas: 1 timeout_override: continue_if_exception: False disable_action: False ------------- === <> NOTE: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-shrink-index.html#_shrinking_an_index[Only `settings` and `aliases` are acceptable] when used in <>. [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Delete source index after successful shrink, then reroute the shrunk index with the provided parameters. options: shrink_node: DETERMINISTIC extra_settings: settings: index.codec: best_compression aliases: my_alias: {} filters: - filtertype: ... ------------- There is no default value. [[option_ignore_empty]] == ignore_empty_list This setting must be either `True` or `False`. [source,yaml] ------------- action: delete_indices description: "Delete selected indices" options: ignore_empty_list: True filters: - filtertype: ... ------------- Depending on your indices, and how you've filtered them, an empty list may be presented to the action. This results in an error condition. When the ignore_empty_list option is set to `True`, the action will exit with an INFO level log message indicating such. If ignore_empty_list is set to `False`, an ERROR level message will be logged, and Curator will exit with code 1. The default value of this setting is `False` [[option_ignore_sync_failures]] == ignore_sync_failures NOTE: This setting is only used by the <>, and is optional. [source,yaml] ------------- action: close description: "Close selected indices" options: ignore_sync_failures: false filters: - filtertype: ... ------------- If `ignore_sync_failures` is set to `true`, Curator will ignore failures during the synced flush that normally takes place before closing. This may be useful for closing a list of indices which may include active indices. The default value is `false`. [[option_ignore]] == ignore_unavailable NOTE: This setting is used by the <>, <>, and <> actions. This setting must be either `True` or `False`. The default value of this setting is `False` === <> [source,yaml] ------------- actions: 1: action: restore description: Restore my_index from my_snapshot in my_repository options: repository: my_repository name: my_snapshot indices: my_index ignore_unavailable: True wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- When the `ignore_unavailable` option is `False` and an index is missing the restore request will fail. === <> [source,yaml] ------------- action: snapshot description: >- Snapshot selected indices to 'repository' with the snapshot name or name pattern in 'name'. Use all other options as assigned options: repository: my_repository name: my_snapshot ignore_unavailable: False wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: ... ------------- When the `ignore_unavailable` option is `False` and an index is missing, the snapshot request will fail. This is not frequently a concern in Curator, as it should only ever find indices that exist. === <> [source,yaml] ------------- action: index_settings description: "Change settings for selected indices" options: index_settings: index: refresh_interval: 5s ignore_unavailable: False preserve_existing: False filters: - filtertype: ... ------------- When the `ignore_unavailable` option is `False` and an index is missing, or if the request is to apply a https://www.elastic.co/guide/en/elasticsearch/reference/5.4/index-modules.html#_static_index_settings[static] setting and the index is opened, the index setting request will fail. The `ignore_unavailable` option allows these indices to be skipped, when set to `True`. NOTE: https://www.elastic.co/guide/en/elasticsearch/reference/5.4/index-modules.html#dynamic-index-settings[Dynamic] index settings can be applied to either open or closed indices. [[option_include_aliases]] == include_aliases NOTE: This setting is only used by the <> action. [source,yaml] ------------- actions: 1: action: restore description: Restore my_index from my_snapshot in my_repository options: repository: my_repository name: my_snapshot indices: my_index include_aliases: True wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- This setting must be either `True` or `False`. The value of this setting determines whether Elasticsearch should include index aliases when restoring the snapshot. The default value of this setting is `False` [[option_include_gs]] == include_global_state NOTE: This setting is used by the <> and <> actions. This setting must be either `True` or `False`. The value of this setting determines whether Elasticsearch should include the global cluster state with the snapshot or restore. When performing a <>, the default value of this setting is `True`. When performing a <>, the default value of this setting is `False`. === <> [source,yaml] ------------- actions: 1: action: restore description: Restore my_index from my_snapshot in my_repository options: repository: my_repository name: my_snapshot indices: my_index include_global_state: False wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- === <> [source,yaml] ------------- action: snapshot description: >- Snapshot selected indices to 'repository' with the snapshot name or name pattern in 'name'. Use all other options as assigned options: repository: my_repository name: my_snapshot include_global_state: True wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: ... ------------- [[option_indices]] == indices NOTE: This setting is only used by the <> action. === <> [source,yaml] ------------- actions: 1: action: restore description: Restore my_index from my_snapshot in my_repository options: repository: my_repository name: my_snapshot indices: my_index wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- This setting must be a list of indices to restore. Any valid YAML format for lists are acceptable here. If `indices` is left empty, or unset, all indices in the snapshot will be restored. The default value of this setting is an empty setting. [[option_key]] == key NOTE: This setting is required when using the <>. [source,yaml] ------------- action: allocation description: "Apply shard allocation filtering rules to the specified indices" options: key: ... value: ... allocation_type: ... filters: - filtertype: ... ------------- The value of this setting should correspond to a node setting on one or more nodes in your cluster. For example, you might have set [source,sh] ----------- node.tag: myvalue ----------- in your `elasticsearch.yml` file for one or more of your nodes. To match allocation in this case, set key to `tag`. These special attributes are also supported: [cols="2*", options="header"] |=== |attribute |description |`_name` |Match nodes by node name |`_host_ip` |Match nodes by host IP address (IP associated with hostname) |`_publish_ip` |Match nodes by publish IP address |`_ip` |Match either `_host_ip` or `_publish_ip` |`_host` |Match nodes by hostname |=== There is no default value. This setting must be set by the user or an exception will be raised, and execution will halt. [[option_max_age]] == max_age [source,yaml] ------------- action: rollover description: >- Rollover the index associated with alias 'aliasname', which should be in the form of prefix-000001 (or similar), or prefix-yyyy.MM.DD-1. options: name: aliasname conditions: max_age: 1d ------------- NOTE: At least one of <>, <>, <> or any combinations of the three are required as `conditions:` for the <> action. The maximum age that is allowed before triggering a rollover. It _must_ be nested under `conditions:` There is no default value. If this condition is specified, it must have a value, or Curator will generate an error. Ages such as `1d` for one day, or `30s` for 30 seconds can be used. [[option_max_docs]] == max_docs [source,yaml] ------------- action: rollover description: >- Rollover the index associated with alias 'aliasname', which should be in the form of prefix-000001 (or similar), or prefix-yyyy.MM.DD-1. options: name: aliasname conditions: max_docs: 1000000 ------------- NOTE: At least one of <>, <>, <> or any combinations of the three are required as `conditions:` for the <> action. The maximum number of documents allowed in an index before triggering a rollover. It _must_ be nested under `conditions:` There is no default value. If this condition is specified, it must have a value, or Curator will generate an error. [[option_max_size]] == max_size [source,yaml] ------------- action: rollover description: >- Rollover the index associated with alias 'aliasname', which should be in the form of prefix-000001 (or similar), or prefix-yyyy.MM.DD-1. options: name: aliasname conditions: max_size: 5gb ------------- NOTE: At least one of <>, <>, <> or any combinations of the three are required as `conditions:` for the <> action. The maximum approximate size an index is allowed to be before triggering a rollover. Sizes must use Elasticsearch approved {ref}/common-options.html#byte-units[byte units]. It _must_ be nested under `conditions:` There is no default value. If this condition is specified, it must have a value, or Curator will generate an error. [[option_mns]] == max_num_segments NOTE: This setting is required when using the <>. [source,yaml] ------------- action: forcemerge description: >- Perform a forceMerge on selected indices to 'max_num_segments' per shard options: max_num_segments: 2 timeout_override: 21600 filters: - filtertype: ... ------------- The value for this setting is the cutoff number of segments per shard. Indices which have more than this number of segments per shard will remain in the index list. There is no default value. This setting must be set by the user or an exception will be raised, and execution will halt. [[option_max_wait]] == max_wait NOTE: This setting is used by the <>, <>, <>, <>, <>, and <> actions. This setting must be a positive integer, or `-1`. This setting specifies how long in seconds to wait to see if the action has completed before giving up. This option is used in conjunction with <>, which is the number of seconds to wait between checking to see if the given action is complete. The default value for this setting is `-1`, meaning that Curator will wait indefinitely for the action to complete. === <> [source,yaml] ------------- action: allocation description: "Apply shard allocation filtering rules to the specified indices" options: key: ... value: ... allocation_type: ... wait_for_completion: True max_wait: 300 wait_interval: 10 filters: - filtertype: ... ------------- === <> [source,yaml] ------------- action: cluster_routing description: "Apply routing rules to the entire cluster" options: routing_type: value: ... setting: enable wait_for_completion: True max_wait: 300 wait_interval: 10 ------------- === <> [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 request_body: source: index: index1 dest: index: index2 filters: - filtertype: none ------------- === <> [source,yaml] ------------- action: replicas description: >- Set the number of replicas per shard for selected indices to 'count' options: count: ... wait_for_completion: True max_wait: 600 wait_interval: 10 filters: - filtertype: ... ------------- === <> [source,yaml] ------------- actions: 1: action: restore description: Restore my_index from my_snapshot in my_repository options: repository: my_repository name: my_snapshot indices: my_index include_global_state: False wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- === <> [source,yaml] ------------- action: snapshot description: >- Snapshot selected indices to 'repository' with the snapshot name or name pattern in 'name'. Use all other options as assigned options: repository: my_repository name: my_snapshot include_global_state: True wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: ... ------------- [[option_migration_prefix]] == migration_prefix NOTE: This setting is used by the <> action. If the destination index is set to `MIGRATION`, Curator will reindex all selected indices one by one until they have all been reindexed. By configuring `migration_prefix`, a value can prepend each of those index names. For example, if I were reindexing `index1`, `index2`, and `index3`, and `migration_prefix` were set to `new-`, then the reindexed indices would be named `new-index1`, `new-index2`, and `new-index3`: [source,yaml] ------------- actions: 1: description: >- Reindex index1, index2, and index3 with a prefix of new-, resulting in indices named new-index1, new-index2, and new-index3 action: reindex options: wait_interval: 9 max_wait: -1 migration_prefix: new- request_body: source: index: ["index1", "index2", "index3"] dest: index: MIGRATION filters: - filtertype: none ------------- `migration_prefix` can be used in conjunction with <> [[option_migration_suffix]] == migration_suffix NOTE: This setting is used by the <> action. If the destination index is set to `MIGRATION`, Curator will reindex all selected indices one by one until they have all been reindexed. By configuring `migration_suffix`, a value can be appended to each of those index names. For example, if I were reindexing `index1`, `index2`, and `index3`, and `migration_suffix` were set to `-new`, then the reindexed indices would be named `index1-new`, `index2-new`, and `index3-new`: [source,yaml] ------------- actions: 1: description: >- Reindex index1, index2, and index3 with a suffix of -new, resulting in indices named index1-new, index2-new, and index3-new action: reindex options: wait_interval: 9 max_wait: -1 migration_suffix: -new request_body: source: index: ["index1", "index2", "index3"] dest: index: MIGRATION filters: - filtertype: none ------------- `migration_suffix` can be used in conjunction with <> [[option_name]] == name NOTE: This setting is used by the <>, <> and <>, actions. The value of this setting is the name of the alias, snapshot, or index, depending on which action makes use of `name`. === date math include::inc_datemath.asciidoc[] === strftime This setting may alternately contain a valid Python strftime string. Curator will extract the strftime identifiers and replace them with the corresponding values. The identifiers that Curator currently recognizes include: [width="50%", cols="> [source,yaml] ------------- action: alias description: "Add/Remove selected indices to or from the specified alias" options: name: alias_name add: filters: - filtertype: ... remove: filters: - filtertype: ... ------------- This option is required by the <> action, and has no default setting in that context. === <> For the <> action, there is no default setting, but you can use strftime: [source,yaml] ------------- action: create_index description: "Create index as named" options: name: 'myindex-%Y.%m' # ... ------------- or use Elasticsearch {ref}/date-math-index-names.html[date math] [source,yaml] ------------- action: create_index description: "Create index as named" options: name: '' # ... ------------- to name your indices. See more in the <> documenation. === <> [source,yaml] ------------- action: snapshot description: >- Snapshot selected indices to 'repository' with the snapshot name or name pattern in 'name'. Use all other options as assigned options: repository: my_repository name: include_global_state: True wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: ... ------------- For the <> action, the default value of this setting is `curator-%Y%m%d%H%M%S` [[option_new_index]] == new_index NOTE: This optional setting is only used by the <> action. [source,yaml] ------------- description: >- Rollover the index associated with alias 'name'. Specify new index name using date math. options: name: aliasname new_index: "" conditions: max_age: 1d wait_for_active_shards: 1 ------------- IMPORTANT: A new index name for rollover should still end with a dash followed by an incrementable number, e.g. `my_new_index-1`, or if using date math, `` or `` === date_math include::inc_datemath.asciidoc[] There is no default value for `new_index`. [[option_node_filters]] == node_filters NOTE: This setting is only used by the <> action. [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Allow master/data nodes to be potential shrink targets, but exclude 'named_node' from potential selection. options: shrink_node: DETERMINISTIC node_filters: permit_masters: True exclude_nodes: ['named_node'] filters: - filtertype: ... ------------- There is no default value for `node_filters`. The current sub-options are as follows: === permit_masters This option indicates whether the shrink action can select master eligible nodes when using `DETERMINISTIC` as the value for <>. The default value is `False`. Please note that this will exclude the elected master, as well as other master-eligible nodes. [IMPORTANT] ===================================== If you have a small cluster with only master/data nodes, you must set `permit_masters` to `True` in order to select one of those nodes as a potential <>. ===================================== === exclude_nodes This option provides means to exclude nodes from selection when using `DETERMINISTIC` as the value for <>. It should be noted that you _can_ use a named node for <> and then exclude it here, and it will prevent a shrink from occurring. [[option_number_of_replicas]] == number_of_replicas NOTE: This setting is only used by the <> action. [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Set the number of replicas to 0. options: shrink_node: DETERMINISTIC number_of_replicas: 0 filters: - filtertype: ... ------------- The value of this setting determines the number of replica shards per primary shard in the target index. The default value is `1`. [[option_number_of_shards]] == number_of_shards NOTE: This setting is only used by the <> action. [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Set the number of shards to 2. options: shrink_node: DETERMINISTIC number_of_shards: 2 filters: - filtertype: ... ------------- The value of this setting determines the number of primary shards in the target index. The default value is `1`. [IMPORTANT] =========================== The value for `number_of_shards` must meet the following criteria: * It must be lower than the number of primary shards in the source index. * It must be a factor of the number of primary shards in the source index. =========================== [[option_partial]] == partial NOTE: This setting is only used by the <> action. [source,yaml] ------------- action: snapshot description: >- Snapshot selected indices to 'repository' with the snapshot name or name pattern in 'name'. Use all other options as assigned options: repository: my_repository name: ... partial: False wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: ... ------------- This setting must be either `True` or `False`. The entire snapshot will fail if one or more indices being added to the snapshot do not have all primary shards available. This behavior can be changed by setting partial to `True`. The default value of this setting is `False` [[option_post_allocation]] == post_allocation NOTE: This setting is only used by the <> action. [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Apply shard routing allocation to target indices. options: shrink_node: DETERMINISTIC post_allocation: allocation_type: include key: node_tag value: cold filters: - filtertype: ... ------------- The only permitted subkeys for `post_allocation` are the same options used by the <> action: <>, <>, and <>. If present, these values will be use to apply shard routing allocation to the target index after shrinking. There is no default value for `post_allocation`. [[option_preserve_existing]] == preserve_existing [source,yaml] ------------- action: index_settings description: "Change settings for selected indices" options: index_settings: index: refresh_interval: 5s ignore_unavailable: False preserve_existing: False filters: - filtertype: ... ------------- This setting must be either `True` or `False`. If `preserve_existing` is set to `True`, and the configuration attempts to push a setting to an index that already has any value for that setting, the existing setting will be preserved, and remain unchanged. The default value of this setting is `False` [[option_refresh]] == refresh NOTE: This setting is only used by the <> action. [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 refresh: True request_body: source: index: index1 dest: index: index2 filters: - filtertype: none ------------- Setting `refresh` to `True` will cause all re-indexed indexes to be refreshed. This differs from the Index API’s refresh parameter which causes just the _shard_ that received the new data to be refreshed. Read more about this setting at {ref}/docs-reindex.html The default value is `True`. [[option_remote_aws_key]] == remote_aws_key NOTE: This option is only used by the <> when performing a remote reindex operation. WARNING: This setting will not work unless the `requests-aws4auth` Python module has been manually installed first. This should be an AWS IAM access key, or left empty. [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 remote_aws_key: AWS_KEY remote_aws_secret_key: AWS_SECRET_KEY remote_aws_region: us-east-1 request_body: source: remote: host: https://otherhost:9200 index: index1 dest: index: index2 filters: - filtertype: none ------------- IMPORTANT: You must set your <> to the proper hostname _with_ port. It may not work setting <> and <> to only a host name due to the different connection module used. [[option_remote_aws_region]] == remote_aws_region NOTE: This option is only used by the <> when performing a remote reindex operation. WARNING: This setting will not work unless the `requests-aws4auth` Python module has been manually installed first. This should be an AWS region, or left empty. [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 remote_aws_key: AWS_KEY remote_aws_secret_key: AWS_SECRET_KEY remote_aws_region: us-east-1 request_body: source: remote: host: https://otherhost:9200 index: index1 dest: index: index2 filters: - filtertype: none ------------- IMPORTANT: You must set your <> to the proper hostname _with_ port. It may not work setting <> and <> to only a host name due to the different connection module used. [[option_remote_aws_secret_key]] == remote_aws_secret_key NOTE: This option is only used by the <> when performing a remote reindex operation. WARNING: This setting will not work unless the `requests-aws4auth` Python module has been manually installed first. This should be an AWS IAM secret access key, or left empty. [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 remote_aws_key: AWS_KEY remote_aws_secret_key: AWS_SECRET_KEY remote_aws_region: us-east-1 request_body: source: remote: host: https://otherhost:9200 index: index1 dest: index: index2 filters: - filtertype: none ------------- IMPORTANT: You must set your <> to the proper hostname _with_ port. It may not work setting <> and <> to only a host name due to the different connection module used. [[option_remote_certificate]] == remote_certificate This should be a file path to a CA certificate, or left empty. [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 remote_certificate: /path/to/my/ca.cert remote_client_cert: /path/to/my/client.cert remote_client_key: /path/to/my/client.key request_body: source: remote: host: https://otherhost:9200 index: index1 dest: index: index2 filters: - filtertype: none ------------- NOTE: This option is only used by the <> when performing a remote reindex operation. This setting allows the use of a specified CA certificate file to validate the SSL certificate used by Elasticsearch. There is no default. [[option_remote_client_cert]] == remote_client_cert NOTE: This option is only used by the <> when performing a remote reindex operation. This should be a file path to a client certificate (public key), or left empty. [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 remote_certificate: /path/to/my/ca.cert remote_client_cert: /path/to/my/client.cert remote_client_key: /path/to/my/client.key request_body: source: remote: host: https://otherhost:9200 index: index1 dest: index: index2 filters: - filtertype: none ------------- Allows the use of a specified SSL client cert file to authenticate to Elasticsearch. The file may contain both an SSL client certificate and an SSL key, in which case <> is not used. If specifying `client_cert`, and the file specified does not also contain the key, use <> to specify the file containing the SSL key. The file must be in PEM format, and the key part, if used, must be an unencrypted key in PEM format as well. [[option_remote_client_key]] == remote_client_key NOTE: This option is only used by the <> when performing a remote reindex operation. This should be a file path to a client key (private key), or left empty. [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 remote_certificate: /path/to/my/ca.cert remote_client_cert: /path/to/my/client.cert remote_client_key: /path/to/my/client.key request_body: source: remote: host: https://otherhost:9200 index: index1 dest: index: index2 filters: - filtertype: none ------------- Allows the use of a specified SSL client key file to authenticate to Elasticsearch. If using <> and the file specified does not also contain the key, use `client_key` to specify the file containing the SSL key. The key file must be an unencrypted key in PEM format. [[option_remote_filters]] == remote_filters NOTE: This option is only used by the <> when performing a remote reindex operation. This is an array of <>, exactly as with regular index selection: [source,yaml] ------------- actions: 1: description: "Reindex matching indices into index2" action: reindex options: wait_interval: 9 max_wait: -1 request_body: source: remote: host: https://otherhost:9200 index: REINDEX_SELECTION dest: index: index2 remote_filters: - filtertype: *first* setting1: ... ... settingN: ... - filtertype: *second* setting1: ... ... settingN: ... - filtertype: *third* filters: - filtertype: none ------------- This feature will only work when the `source` `index` is set to `REINDEX_SELECTION`. It will select _remote_ indices matching the filters provided, and reindex them to the _local_ cluster as the name provided in the `dest` `index`. In this example, that is `index2`. [[option_remote_ssl_no_validate]] == remote_ssl_no_validate This should be `True`, `False` or left empty. [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 remote_ssl_no_validate: True request_body: source: remote: host: https://otherhost:9200 index: index1 dest: index: index2 filters: - filtertype: none ------------- If access to your Elasticsearch instance is protected by SSL encryption, you may set `ssl_no_validate` to `True` to disable SSL certificate verification. Valid use cases for doing so include the use of self-signed certificates that cannot be otherwise verified and would generate error messages. WARNING: Setting `ssl_no_validate` to `True` will likely result in a warning message that your SSL certificates are not trusted. This is expected behavior. The default value is `False`. [[option_remote_url_prefix]] == remote_url_prefix NOTE: This option is only used by the <> when performing a remote reindex operation. This should be a single value or left empty. [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 remote_url_prefix: my_prefix request_body: source: remote: host: https://otherhost:9200 index: index1 dest: index: index2 filters: - filtertype: none ------------- In some cases you may be obliged to connect to a remote Elasticsearch cluster through a proxy of some kind. There may be a URL prefix before the API URI items, e.g. http://example.com/elasticsearch/ as opposed to http://localhost:9200. In such a case, set the `remote_url_prefix` to the appropriate value, 'elasticsearch' in this example. The default is an empty string. [[option_rename_pattern]] == rename_pattern NOTE: This setting is only used by the <> action. [TIP] .from the Elasticsearch documentation ====================================== The <> and <> options can be also used to rename indices on restore using regular expression that supports referencing the original text as explained http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#appendReplacement(java.lang.StringBuffer,%20java.lang.String)[here]. ====================================== [source,yaml] ------------- actions: 1: action: restore description: >- Restore all indices in the most recent snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do not skip the repository filesystem access check. Use the other options to define the index/shard settings for the restore. options: repository: # If name is blank, the most recent snapshot by age will be selected name: # If indices is blank, all indices in the snapshot will be restored indices: rename_pattern: 'index(.+)' rename_replacement: 'restored_index$1' wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- In this configuration, Elasticsearch will capture whatever appears after `index` and put it after `restored_index`. For example, if I was restoring `index-2017.03.01`, the resulting index would be renamed to `restored_index-2017.03.01`. Read more about this setting at {ref}/snapshots-restore-snapshot.html There is no default value. [[option_rename_replacement]] == rename_replacement NOTE: This setting is only used by the <> action. [TIP] .From the Elasticsearch documentation ====================================== The <> and <> options can be also used to rename indices on restore using regular expression that supports referencing the original text as explained http://docs.oracle.com/javase/6/docs/api/java/util/regex/Matcher.html#appendReplacement(java.lang.StringBuffer,%20java.lang.String)[here]. ====================================== [source,yaml] ------------- actions: 1: action: restore description: >- Restore all indices in the most recent snapshot with state SUCCESS. Wait for the restore to complete before continuing. Do not skip the repository filesystem access check. Use the other options to define the index/shard settings for the restore. options: repository: # If name is blank, the most recent snapshot by age will be selected name: # If indices is blank, all indices in the snapshot will be restored indices: rename_pattern: 'index(.+)' rename_replacement: 'restored_index$1' wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- In this configuration, Elasticsearch will capture whatever appears after `index` and put it after `restored_index`. For example, if I was restoring `index-2017.03.01`, the resulting index would be renamed to `restored_index-2017.03.01`. Read more about this setting at {ref}/snapshots-restore-snapshot.html There is no default value. [[option_repository]] == repository NOTE: This setting is only used by the <>, and <> actions. There is no default value. This setting must be set by the user or an exception will be raised, and execution will halt. === <> [source,yaml] ------------- actions: 1: action: restore description: Restore my_index from my_snapshot in my_repository options: repository: my_repository name: my_snapshot indices: my_index wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- === <> [source,yaml] ------------- action: snapshot description: >- Snapshot selected indices to 'repository' with the snapshot name or name pattern in 'name'. Use all other options as assigned options: repository: my_repository name: my_snapshot wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: ... ------------- [[option_requests_per_second]] == requests_per_second NOTE: This option is only used by the <> [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 requests_per_second: -1 request_body: source: index: index1 dest: index: index2 filters: - filtertype: none ------------- `requests_per_second` can be set to any positive decimal number (1.4, 6, 1000, etc) and throttles the number of requests per second that the reindex issues or it can be set to `-1` to disable throttling. The default value for this is option is `-1`. [[option_request_body]] == request_body NOTE: This setting is only used by the <> action. === Manual index selection The `request_body` option is the heart of the reindex action. In here, using YAML syntax, you recreate the body sent to Elasticsearch as described in {ref}/docs-reindex.html[the official documentation.] You can manually select indices as with this example: [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 request_body: source: index: index1 dest: index: index2 filters: - filtertype: none ------------- You can also select multiple indices to reindex by making a list in acceptable YAML syntax: [source,yaml] ------------- actions: 1: description: "Reindex index1,index2,index3 into new_index" action: reindex options: wait_interval: 9 max_wait: -1 request_body: source: index: ['index1', 'index2', 'index3'] dest: index: new_index filters: - filtertype: none ------------- IMPORTANT: You _must_ set at least a <> filter, or the action will fail. Do not worry. If you've manually specified your indices, those are the only ones which will be reindexed. === Filter-Selected Indices Curator allows you to use all indices found by the `filters` section by setting the `source` index to `REINDEX_SELECTION`, like this: [source,yaml] ------------- actions: 1: description: >- Reindex all daily logstash indices from March 2017 into logstash-2017.03 action: reindex options: wait_interval: 9 max_wait: -1 request_body: source: index: REINDEX_SELECTION dest: index: logstash-2017.03 filters: - filtertype: pattern kind: prefix value: logstash-2017.03. ------------- === Reindex From Remote You can also reindex from remote: [source,yaml] ------------- actions: 1: description: "Reindex remote index1 to local index1" action: reindex options: wait_interval: 9 max_wait: -1 request_body: source: remote: host: http://otherhost:9200 username: myuser password: mypass index: index1 dest: index: index1 filters: - filtertype: none ------------- IMPORTANT: You _must_ set at least a <> filter, or the action will fail. Do not worry. Only the indices you specified in `source` will be reindexed. Curator will create a connection to the host specified as the `host` key in the above example. It will determine which port to connect to, and whether to use SSL by parsing the URL entered there. Because this `host` is specifically used by Elasticsearch, and Curator is making a separate connection, it is important to ensure that both Curator _and_ your Elasticsearch cluster have access to the remote host. If you do not whitelist the remote cluster, you will not be able to reindex. This can be done by adding the following line to your `elasticsearch.yml` file: [source,yaml] ------------- reindex.remote.whitelist: remote_host_or_IP1:9200, remote_host_or_IP2:9200 ------------- or by adding this flag to the command-line when starting Elasticsearch: [source,sh] ------------- bin/elasticsearch -Edefault.reindex.remote.whitelist="remote_host_or_IP:9200" ------------- Of course, be sure to substitute the correct host, IP, or port. Other client connection arguments can also be supplied in the form of action options: * <> * <> * <> * <> * <> * <> * <> === Reindex From Remote With Filter-Selected Indices You can even reindex from remote with filter-selected indices on the remote side: [source,yaml] ------------- actions: 1: description: >- Reindex all remote daily logstash indices from March 2017 into local index logstash-2017.03 action: reindex options: wait_interval: 9 max_wait: -1 request_body: source: remote: host: http://otherhost:9200 username: myuser password: mypass index: REINDEX_SELECTION dest: index: logstash-2017.03 remote_filters: - filtertype: pattern kind: prefix value: logstash-2017.03. filters: - filtertype: none ------------- IMPORTANT: Even though you are reindexing from remote, you _must_ set at least a <> filter, or the action will fail. Do not worry. Only the indices specified in `source` will be reindexed. Curator will create a connection to the host specified as the `host` key in the above example. It will determine which port to connect to, and whether to use SSL by parsing the URL entered there. Because this `host` is specifically used by Elasticsearch, and Curator is making a separate connection, it is important to ensure that both Curator _and_ your Elasticsearch cluster have access to the remote host. If you do not whitelist the remote cluster, you will not be able to reindex. This can be done by adding the following line to your `elasticsearch.yml` file: [source,yaml] ------------- reindex.remote.whitelist: remote_host_or_IP1:9200, remote_host_or_IP2:9200 ------------- or by adding this flag to the command-line when starting Elasticsearch: [source,sh] ------------- bin/elasticsearch -Edefault.reindex.remote.whitelist="remote_host_or_IP:9200" ------------- Of course, be sure to substitute the correct host, IP, or port. Other client connection arguments can also be supplied in the form of action options: * <> * <> * <> * <> * <> * <> * <> === Reindex - Migration Curator allows reindexing, particularly from remote, as a migration path. This can be a very useful feature for migrating an older cluster (1.4+) to a new cluster, on different hardware. It can also be a useful tool for serially reindexing indices into newer mappings in an automatable way. Ordinarily, a reindex operation is from either one, or many indices, to a single, named index. Assigning the `dest` `index` to `MIGRATION` tells Curator to treat this reindex differently. [IMPORTANT] ============================= **If it is a _local_ reindex,** you _must_ set either <>, or <>, or both. This prevents collisions and other bad things from happening. By assigning a prefix or a suffix (or both), you can reindex any local indices to new versions of themselves, but named differently. It is true the Reindex API already has this functionality. Curator includes this same functionality for convenience. ============================= This example will reindex all of the remote indices matching `logstash-2017.03.` into the local cluster, but preserve the original index names, rather than merge all of the contents into a single index. Internal to Curator, this results in multiple reindex actions: one per index. All other available options and settings are available. [source,yaml] ------------- actions: 1: description: >- Reindex all remote daily logstash indices from March 2017 into local versions with the same index names. action: reindex options: wait_interval: 9 max_wait: -1 request_body: source: remote: host: http://otherhost:9200 username: myuser password: mypass index: REINDEX_SELECTION dest: index: MIGRATION remote_filters: - filtertype: pattern kind: prefix value: logstash-2017.03. filters: - filtertype: none ------------- IMPORTANT: Even though you are reindexing from remote, you _must_ set at least a <> filter, or the action will fail. Do not worry. Only the indices specified in `source` will be reindexed. Curator will create a connection to the host specified as the `host` key in the above example. It will determine which port to connect to, and whether to use SSL by parsing the URL entered there. Because this `host` is specifically used by Elasticsearch, and Curator is making a separate connection, it is important to ensure that both Curator _and_ your Elasticsearch cluster have access to the remote host. If you do not whitelist the remote cluster, you will not be able to reindex. This can be done by adding the following line to your `elasticsearch.yml` file: [source,yaml] ------------- reindex.remote.whitelist: remote_host_or_IP1:9200, remote_host_or_IP2:9200 ------------- or by adding this flag to the command-line when starting Elasticsearch: [source,sh] ------------- bin/elasticsearch -Edefault.reindex.remote.whitelist="remote_host_or_IP:9200" ------------- Of course, be sure to substitute the correct host, IP, or port. Other client connection arguments can also be supplied in the form of action options: * <> * <> * <> * <> * <> * <> * <> * <> * <> === Other scenarios and options Nearly all scenarios supported by the reindex API are supported in the request_body, including (but not limited to): * Pipelines * Scripting * Queries * Conflict resolution * Limiting by count * Versioning * Reindexing operation type (for example, create-only) Read more about these, and more, at {ref}/docs-reindex.html Notable exceptions include: * You cannot manually specify slices. Instead, use the <> option for automated sliced reindexing. [[option_retry_count]] == retry_count NOTE: This setting is only used by the <>. [source,yaml] ------------- action: delete_snapshots description: "Delete selected snapshots from 'repository'" options: repository: ... retry_interval: 120 retry_count: 3 filters: - filtertype: ... ------------- The value of this setting is the number of times to retry deleting a snapshot. The default for this setting is `3`. [[option_retry_interval]] == retry_interval NOTE: This setting is only used by the <>. [source,yaml] ------------- action: delete_snapshots description: "Delete selected snapshots from 'repository'" options: repository: ... retry_interval: 120 retry_count: 3 filters: - filtertype: ... ------------- The value of this setting is the number of seconds to delay between retries. The default for this setting is `120`. [[option_routing_type]] == routing_type NOTE: This setting is only used by the <>. [source,yaml] ------------- action: cluster_routing description: "Apply routing rules to the entire cluster" options: routing_type: value: ... setting: enable wait_for_completion: True max_wait: 300 wait_interval: 10 ------------- The value of this setting must be either `allocation` or `rebalance` There is no default value. This setting must be set by the user or an exception will be raised, and execution will halt. [[option_setting]] == setting NOTE: This setting is only used by the <>. [source,yaml] ------------- action: cluster_routing description: "Apply routing rules to the entire cluster" options: routing_type: value: ... setting: enable wait_for_completion: True max_wait: 300 wait_interval: 10 ------------- The value of this must be `enable` at present. It is a placeholder for future expansion. There is no default value. This setting must be set by the user or an exception will be raised, and execution will halt. [[option_shrink_node]] == shrink_node NOTE: This setting is only used by the <> action. [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space, excluding master nodes and the node named 'not_this_node' options: shrink_node: DETERMINISTIC node_filters: permit_masters: False exclude_nodes: ['not_this_node'] shrink_suffix: '-shrink' filters: - filtertype: ... ------------- This setting is required. There is no default value. The value of this setting must be the valid name of a node in your Elasticsearch cluster, or `DETERMINISTIC`. If the value is `DETERMINISTIC`, Curator will automatically select the data node with the most available free space and make that the target node. Curator will repeat this process for each successive index when the value is `DETERMINISTIC`. If <>, such as `exclude_nodes` are defined, those nodes will not be considered as potential target nodes. [[option_shrink_prefix]] == shrink_prefix NOTE: This setting is only used by the <> action. [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Prepend target index names with 'foo-' and append a suffix of '-shrink' options: shrink_node: DETERMINISTIC shrink_prefix: 'foo-' shrink_suffix: '-shrink' filters: - filtertype: ... ------------- There is no default value for this setting. The value of this setting will be prepended to target index names. If the source index were `index`, and the `shrink_prefix` were `foo-`, and the `shrink_suffix` were `-shrink`, the resulting target index name would be `foo-index-shrink`. [[option_shrink_suffix]] == shrink_suffix NOTE: This setting is only used by the <> action. [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Prepend target index names with 'foo-' and append a suffix of '-shrink' options: shrink_node: DETERMINISTIC shrink_prefix: 'foo-' shrink_suffix: '-shrink' filters: - filtertype: ... ------------- The default value for this setting is `-shrink`. The value of this setting will be appended to target index names. If the source index were `index`, and the `shrink_prefix` were `foo-`, and the `shrink_suffix` were `-shrink`, the resulting target index name would be `foo-index-shrink`. [[option_slices]] == slices NOTE: This setting is only used by the <> action. This setting can speed up reindexing operations by using {ref}/search-request-body.html#sliced-scroll[Sliced Scroll] to slice on the \_uid. [source,yaml] ------------- actions: 1: description: "Reindex index1,index2,index3 into new_index" action: reindex options: wait_interval: 9 max_wait: -1 slices: 3 request_body: source: index: ['index1', 'index2', 'index3'] dest: index: new_index filters: - filtertype: none ------------- === Picking the number of slices Here are a few recommendations around the number of `slices` to use: * Don’t use large numbers. `500` creates fairly massive CPU thrash, so Curator will not allow a number larger than this. * It is more efficient from a query performance standpoint to use some multiple of the number of shards in the source index. * Using exactly as many shards as are in the source index is the most efficient from a query performance standpoint. * Indexing performance should scale linearly across available resources with the number of slices. * Whether indexing or query performance dominates that process depends on lots of factors like the documents being reindexed and the cluster doing the reindexing. [[option_skip_fsck]] == skip_repo_fs_check NOTE: This setting is used by the <> and <> actions. This setting must be either `True` or `False`. The default value of this setting is `False` === <> Each master and data node in the cluster _must_ have write access to the shared filesystem used by the repository for a snapshot to be 100% valid. For the purposes of a <>, this is a lightweight attempt to ensure that all nodes are _still_ actively able to write to the repository, in hopes that snapshots were from all nodes. It is not otherwise necessary for the purposes of a restore. Some filesystems may take longer to respond to a check, which results in a false positive for the filesystem access check. For these cases, it is desirable to bypass this verification step, by setting this to `True.` [source,yaml] ------------- actions: 1: action: restore description: Restore my_index from my_snapshot in my_repository options: repository: my_repository name: my_snapshot indices: my_index skip_repo_fs_check: False wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- === <> Each master and data node in the cluster _must_ have write access to the shared filesystem used by the repository for a snapshot to be 100% valid. Some filesystems may take longer to respond to a check, which results in a false positive for the filesystem access check. For these cases, it is desirable to bypass this verification step, by setting this to `True.` [source,yaml] ------------- action: snapshot description: >- Snapshot selected indices to 'repository' with the snapshot name or name pattern in 'name'. Use all other options as assigned options: repository: my_repository name: my_snapshot skip_repo_fs_check: False wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: ... ------------- [[option_timeout]] == timeout NOTE: This setting is only used by the <> action. The `timeout` is the length in seconds each individual bulk request should wait for shards that are unavailable. The default value is `60`, meaning 60 seconds. [source,yaml] ------------- actions: 1: description: "Reindex index1,index2,index3 into new_index" action: reindex options: wait_interval: 9 max_wait: -1 timeout: 90 request_body: source: index: ['index1', 'index2', 'index3'] dest: index: new_index filters: - filtertype: none ------------- [[option_timeout_override]] == timeout_override NOTE: This setting is available in all actions. [source,yaml] ------------- action: forcemerge description: >- Perform a forceMerge on selected indices to 'max_num_segments' per shard options: max_num_segments: 2 timeout_override: 21600 filters: - filtertype: ... ------------- If `timeout_override` is unset in your configuration, some actions will try to set a sane default value. The following table shows these default values: [cols="m,", options="header"] |=== |Action Name |Default `timeout_override` Value |close |180 |delete_snapshots |300 |forcemerge |21600 |restore |21600 |snapshot |21600 |=== All other actions have no default value for `timeout_override`. This setting must be an integer number of seconds, or an error will result. This setting is particularly useful for the <> action, as all other actions have a new polling behavior when using <> that should reduce or prevent client timeouts. [[option_value]] == value NOTE: This setting is optional when using the <> and required when using the <>. === <> For the <>, the value of this setting should correspond to a node setting on one or more nodes in your cluster For example, you might have set [source,sh] ----------- node.tag: myvalue ----------- in your `elasticsearch.yml` file for one or more of your nodes. To match allocation in this case, set value to `myvalue`. Additonally, if you used one of the special attribute names `_ip`, `_name`, `_id`, or `_host` for <>, value can match the one of the node ip addresses, names, ids, or host names, respectively. NOTE: To remove a routing allocation, the value of this setting should be left empty, or the `value` setting not even included as an option. For example, you might have set [source,sh] ----------- PUT test/_settings { "index.routing.allocation.exclude.size": "small" } ----------- to keep index `test` from allocating shards on nodes that have `node.tag: small`. To remove this shard routing allocation setting, you might use an action file similar to this: [source,yaml] ----------- --- actions: 1: action: allocation description: -> Unset 'index.routing.allocation.exclude.size' for index 'test' by passing an empty value. options: key: size value: ... allocation_type: exclude filters: - filtertype: pattern kind: regex value: '^test$' ----------- === <> For the <>, the acceptable values for this setting depend on the value of <>. [source,yaml] ------------- action: cluster_routing description: "Apply routing rules to the entire cluster" options: routing_type: ... value: ... setting: enable wait_for_completion: True max_wait: 300 wait_interval: 10 ------------- Acceptable values when <> is either `allocation` or `rebalance` are `all`, `primaries`, and `none` (string, not `NoneType`). If `routing_type` is `allocation`, this can also be `new_primaries`. If `routing_type` is `rebalance`, then the value can also be `replicas`. There is no default value. This setting must be set by the user or an exception will be raised, and execution will halt. [[option_wait_for_active_shards]] == wait_for_active_shards NOTE: This setting is used by the <>, <>, and <> actions. Each use it similarly. This setting determines the number of shard copies that must be active before the client returns. The default value is 1, which implies only the primary shards. Set to `all` for all shard copies, otherwise set to any non-negative value less than or equal to the total number of copies for the shard (number of replicas + 1) Read {ref}/docs-index_.html#index-wait-for-active-shards[the Elasticsearch documentation] for more information. === Reindex [source,yaml] ------------- actions: 1: description: "Reindex index1,index2,index3 into new_index" action: reindex options: wait_interval: 9 max_wait: -1 wait_for_active_shards: 2 request_body: source: index: ['index1', 'index2', 'index3'] dest: index: new_index filters: - filtertype: none ------------- === Rollover [source,yaml] ------------- action: rollover description: >- Rollover the index associated with alias 'name', which should be in the form of prefix-000001 (or similar), or prefix-yyyy.MM.DD-1. options: name: aliasname conditions: max_age: 1d max_docs: 1000000 wait_for_active_shards: 1 extra_settings: index.number_of_shards: 3 index.number_of_replicas: 1 timeout_override: continue_if_exception: False disable_action: False ------------- === Shrink [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Prepend target index names with 'foo-' and append a suffix of '-shrink' options: shrink_node: DETERMINISTIC wait_for_active_shards: all filters: - filtertype: ... ------------- [[option_wfc]] == wait_for_completion NOTE: This setting is used by the <>, <>, <>, <>, <>, and <> actions. This setting must be either `True` or `False`. This setting specifies whether or not the request should return immediately or wait for the operation to complete before returning. === <> [source,yaml] ------------- action: allocation description: "Apply shard allocation filtering rules to the specified indices" options: key: ... value: ... allocation_type: ... wait_for_completion: False max_wait: 300 wait_interval: 10 filters: - filtertype: ... ------------- The default value for the <> action is `False`. === <> [source,yaml] ------------- action: cluster_routing description: "Apply routing rules to the entire cluster" options: routing_type: value: ... setting: enable wait_for_completion: True max_wait: 300 wait_interval: 10 ------------- The default value for the <> action is `False`. === <> [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 request_body: source: index: index1 dest: index: index2 filters: - filtertype: none ------------- The default value for the <> action is `False`. === <> [source,yaml] ------------- action: replicas description: >- Set the number of replicas per shard for selected indices to 'count' options: count: ... wait_for_completion: True max_wait: 600 wait_interval: 10 filters: - filtertype: ... ------------- The default value for the <> action is `False`. === <> [source,yaml] ------------- actions: 1: action: restore description: Restore my_index from my_snapshot in my_repository options: repository: my_repository name: my_snapshot indices: my_index wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- The default value for the <> action is `True`. === <> [source,yaml] ------------- action: snapshot description: >- Snapshot selected indices to 'repository' with the snapshot name or name pattern in 'name'. Use all other options as assigned options: repository: my_repository name: my_snapshot wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: ... ------------- The default value for the <> action is `True`. TIP: During snapshot initialization, information about all previous snapshots is loaded into the memory, which means that in large repositories it may take several seconds (or even minutes) for this command to return even if the `wait_for_completion` setting is set to `False`. [[option_wait_for_rebalance]] == wait_for_rebalance [source,yaml] ------------- action: shrink description: >- Shrink selected indices on the node with the most available space. Delete source index after successful shrink, then reroute the shrunk index with the provided parameters. options: ignore_empty_list: True shrink_node: DETERMINISTIC node_filters: permit_masters: False exclude_nodes: ['not_this_node'] number_of_shards: 1 number_of_replicas: 1 shrink_prefix: shrink_suffix: '-shrink' delete_after: True post_allocation: allocation_type: include key: node_tag value: cold wait_for_active_shards: 1 extra_settings: settings: index.codec: best_compression wait_for_completion: True wait_for_rebalance: True wait_interval: 9 max_wait: -1 filters: - filtertype: ... ------------- NOTE: This setting is used by the <> action. This setting must be `true` or `false`. Setting this to `false` will result in the <> action only checking that the index being shrunk has finished being relocated, and not continue to wait for the cluster to fully rebalance all shards. The default value for this setting is `false`. [[option_wait_interval]] == wait_interval NOTE: This setting is used by the <>, <>, <>, <>, <>, and <> actions. This setting must be a positive integer between 1 and 30. This setting specifies how long to wait between checks to see if the action has completed or not. This number should not be larger than the client <> or the <>. As the default client <> value for is 30, this should be uncommon. The default value for this setting is `9`, meaning 9 seconds between checks. This option is generally used in conjunction with <>, which is the maximum amount of time in seconds to wait for the given action to complete. === <> [source,yaml] ------------- action: allocation description: "Apply shard allocation filtering rules to the specified indices" options: key: ... value: ... allocation_type: ... wait_for_completion: False max_wait: 300 wait_interval: 10 filters: - filtertype: ... ------------- === <> [source,yaml] ------------- action: cluster_routing description: "Apply routing rules to the entire cluster" options: routing_type: value: ... setting: enable wait_for_completion: True max_wait: 300 wait_interval: 10 ------------- === <> [source,yaml] ------------- actions: 1: description: "Reindex index1 into index2" action: reindex options: wait_interval: 9 max_wait: -1 request_body: source: index: index1 dest: index: index2 filters: - filtertype: none ------------- === <> [source,yaml] ------------- action: replicas description: >- Set the number of replicas per shard for selected indices to 'count' options: count: ... wait_for_completion: True max_wait: 600 wait_interval: 10 filters: - filtertype: ... ------------- === <> [source,yaml] ------------- actions: 1: action: restore description: Restore my_index from my_snapshot in my_repository options: repository: my_repository name: my_snapshot indices: my_index wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: state state: SUCCESS exclude: - filtertype: ... ------------- === <> [source,yaml] ------------- action: snapshot description: >- Snapshot selected indices to 'repository' with the snapshot name or name pattern in 'name'. Use all other options as assigned options: repository: my_repository name: my_snapshot wait_for_completion: True max_wait: 3600 wait_interval: 10 filters: - filtertype: ... ------------- [[option_warn_if_no_indices]] == warn_if_no_indices NOTE: This setting is only used by the <> action. This setting must be either `True` or `False`. The default value for this setting is `False`. [source,yaml] ------------- action: alias description: "Add/Remove selected indices to or from the specified alias" options: name: alias_name warn_if_no_indices: False add: filters: - filtertype: ... remove: filters: - filtertype: ... ------------- This setting specifies whether or not the alias action should continue with a warning or return immediately in the event that the filters in the add or remove section result in an empty index list. [WARNING] .Improper use of this setting can yield undesirable results ===================================================================== *Ideal use case:* For example, you want to add the most recent seven days of time-series indices into a _lastweek_ alias, and remove indices older than seven days from this same alias. If you do not not yet have any indices older than seven days, this will result in an empty index list condition which will prevent the entire alias action from completing successfully. If `warn_if_no_indices` were set to `True`, however, it would avert that potential outcome. *Potentially undesirable outcome:* A _non-beneficial_ case would be where if `warn_if_no_indices` is set to `True`, and a misconfiguration results in indices not being found, and therefore not being disassociated from the alias. As a result, an alias that should only query one week now references multiple weeks of data. If `warn_if_no_indices` were set to `False`, this scenario would have been averted because the empty list condition would have resulted in an error. =====================================================================