Actions

See also

Each action has a do_action() method, which accepts no arguments. This is the means by which all actions are executed.

Alias

class curator.actions.Alias(name=None, extra_settings=None, **kwargs)

Bases: object

Alias Action Class

Parameters:
  • name (str) – The alias name

  • extra_settings (dict) – Extra settings, including filters and routing. For more information see here.

actions

The list of actions to perform. Populated by add() and remove()

add(ilo, warn_if_no_indices=False)

Create add statements for each index in ilo for name, then append them to actions. Add any extra_settings that may be there.

Parameters:

ilo (IndexList) – An IndexList Object

check_actions()
Returns:

actions for use with the update_aliases() API call if actions exist, otherwise an exception is raised.

client

The Elasticsearch client object which will later be set by add() or remove()

do_action()

update_aliases() for name with actions

do_dry_run()

Log what the output would be, but take no action.

extra_settings

Any extra things to add to the alias, like filters, or routing. Gets the value from param extra_settings.

name

The parse_date_pattern() rendered version of what was passed by param name.

remove(ilo, warn_if_no_indices=False)

Create remove statements for each index in ilo for name, then append them to actions.

Parameters:

ilo (IndexList) – An IndexList Object

warn_if_no_indices

Preset default value to False.

Allocation

class curator.actions.Allocation(ilo, key=None, value=None, allocation_type='require', wait_for_completion=False, wait_interval=3, max_wait=-1)

Bases: object

Allocation Action Class

Parameters:
  • ilo (IndexList) – An IndexList Object

  • key (str) – An arbitrary metadata attribute key. Must match the key assigned to at least

  • value (str) – An arbitrary metadata attribute value. Must correspond to values associated

  • allocation_type (str) – Type of allocation to apply. Default is require

  • wait_for_completion (bool) – Wait for completion before returning.

  • wait_interval (int) – Seconds to wait between completion checks.

  • max_wait (int) – Maximum number of seconds to wait_for_completion

Note

See more about shard allocation filtering.

client

The Elasticsearch client object derived from index_list

do_action()

put_settings() to indices in index_list with settings.

do_dry_run()

Log what the output would be, but take no action.

index_list

The IndexList object passed as ilo

max_wait

Object attribute that gets the value of param max_wait

settings

Populated at instance creation time. Value is built from the passed params allocation_type, key, and value, e.g. index.routing.allocation.allocation_type.key.value

wait_interval

Object attribute that gets the value of param wait_interval

wfc

Object attribute that gets the value of param wait_for_completion

Close

class curator.actions.Close(ilo, delete_aliases=False, skip_flush=False)

Bases: object

Close Action Class

Parameters:
  • ilo (IndexList) – An IndexList Object

  • delete_aliases (bool) – Delete any associated aliases before closing indices.

  • skip_flush (bool) – Do not flush indices before closing.

client

The Elasticsearch client object derived from index_list

delete_aliases

The value passed as delete_aliases

do_action()

close() open indices in index_list

do_dry_run()

Log what the output would be, but take no action.

index_list

The IndexList object passed from param ilo

skip_flush

The value passed as skip_flush

Cluster Routing

class curator.actions.ClusterRouting(client, routing_type=None, setting=None, value=None, wait_for_completion=False, wait_interval=9, max_wait=-1)

Bases: object

ClusterRouting Action Class

For now, the cluster routing settings are hardcoded to be transient

Parameters:
  • client (Elasticsearch) – A client connection object

  • routing_type (str) – Type of routing to apply. Either allocation or rebalance

  • setting (str) – Currently, the only acceptable value for setting is enable. This is here in case that changes.

  • value (str) – Used only if setting is enable. Semi-dependent on routing_type. Acceptable values for allocation and rebalance are all, primaries, and none (string, not None). If routing_type is allocation, this can also be new_primaries, and if rebalance, it can be replicas.

  • wait_for_completion (bool) – Wait for completion before returning.

  • wait_interval (int) – Seconds to wait between completion checks.

  • max_wait (int) – Maximum number of seconds to wait_for_completion

client

An Elasticsearch client object

do_action()

put_settings() to the cluster with settings.

do_dry_run()

Log what the output would be, but take no action.

max_wait

Object attribute that gets the value of param max_wait. How long in seconds to wfc before returning with an exception. A value of -1 means wait forever.

settings

Populated at instance creation time. Value is built from the passed values from params routing_type and setting, e.g. cluster.routing.routing_type.setting

wait_interval

Object attribute that gets the value of param wait_interval

wfc

Object attribute that gets the value of param wait_for_completion

Cold2Frozen

class curator.actions.Cold2Frozen(ilo, **kwargs)

Bases: object

Cold to Frozen Tier Searchable Snapshot Action Class

For manually migrating snapshots not associated with ILM from the cold tier to the frozen tier.

Parameters:
  • ilo (IndexList) – An IndexList Object

  • index_settings (dict) – (Optional) Settings that should be added to the index when it is mounted. If not set, set the _tier_preference to the tiers available, coldest first.

  • ignore_index_settings (list) – (Optional, array of strings) Names of settings that should be removed from the index when it is mounted.

  • wait_for_completion (bool) – Wait for completion before returning.

DEFAULTS = {'ignore_index_settings': ['index.refresh_interval'], 'index_settings': None, 'wait_for_completion': True}
action_generator()

Yield a dict for use in do_action() and do_dry_run()

Returns:

A generator object containing the settings necessary to migrate indices from cold to frozen

Return type:

dict

assign_kwargs(**kwargs)

Assign the kwargs to the attribute of the same name with the passed value or the default from DEFAULTS

cleanup(current_idx, newidx)

Call delete() to delete the cold tier index.

client

The Elasticsearch client object derived from index_list

do_action()

Do the actions outlined: Extract values from generated kwargs Mount Verify Update Aliases Cleanup

do_dry_run()

Log what the output would be, but take no action.

ignore_index_settings

Object attribute that gets the value of ignore_index_settings.

index_list

The IndexList object passed from param ilo

index_settings

Object attribute that gets the value of index_settings.

indices

Object attribute that contains the to_csv() output of the indices in index_list.

mount_index(newidx, kwargs)

Call mount() to mount the indices in ilo in the Frozen tier.

update_aliases(current_idx, newidx, aliases)

Call update_aliases() to update each new frozen index with the aliases from the old cold-tier index.

Verify aliases look good.

verify_mount(newidx)

Verify that newidx is a mounted index

wait_for_completion

Object attribute that gets the value of param wait_for_completion.

Create Index

class curator.actions.CreateIndex(client, name=None, extra_settings=None, ignore_existing=False)

Bases: object

Create Index Action Class

Parameters:
  • client (Elasticsearch) – A client connection object

  • name (str) – A name, which can contain time.strftime() strings

  • extra_settings (dict) – The settings and mappings for the index. For more information see the create indices documentation.

  • ignore_existing (bool) – If an index already exists, and this setting is True, ignore the 400 error that results in a resource_already_exists_exception and return that it was successful.

aliases

Gets any aliases from extra_settings or is None

client

An Elasticsearch client object

do_action()

create() index identified by name with values from aliases, mappings, and settings

do_dry_run()

Log what the output would be, but take no action.

extra_settings

Any extra settings for the index, like aliases, mappings, or settings. Gets the value from param extra_settings.

ignore_existing

Extracted from the action definition, it should be a boolean informing whether to ignore the error if the index already exists.

mappings

Gets any mappings from extra_settings or is None

name

The parse_date_pattern() rendered version of what was passed as name.

settings

Gets any settings from extra_settings or is None

Delete Indices

class curator.actions.DeleteIndices(ilo, master_timeout=30)

Bases: object

Delete Indices Action Class

Parameters:
  • ilo (IndexList) – An IndexList Object

  • master_timeout (int) – Number of seconds to wait for master node response

client

The Elasticsearch client object derived from index_list

do_action()

delete() indices in index_list

do_dry_run()

Log what the output would be, but take no action.

index_list

The IndexList object passed from param ilo

master_timeout

String value of param master_timeout + s, for seconds.

Delete Snapshots

class curator.actions.DeleteSnapshots(slo, retry_interval=120, retry_count=3)

Bases: object

Delete Snapshots Action Class

Parameters:
  • slo (SnapshotList) – A SnapshotList object

  • retry_interval (int) – Seconds to delay betwen retries. (Default: 120)

  • retry_count (int) – Number of attempts to make. (Default: 3)

client

The Elasticsearch client object derived from snapshot_list

do_action()

delete() snapshots in snapshot_list. Retry up to retry_count times, pausing retry_interval seconds between retries.

do_dry_run()

Log what the output would be, but take no action.

repository

Object attribute that gets its value from snapshot_list.

retry_count

Object attribute that gets the value of param retry_count.

retry_interval

Object attribute that gets the value of param retry_interval.

snapshot_list

The SnapshotList object passed from param slo

Force Merge

class curator.actions.ForceMerge(ilo, max_num_segments=None, delay=0)

Bases: object

ForceMerge Action Class

Parameters:
  • ilo (IndexList) – An IndexList Object

  • max_num_segments (int) – Number of segments per shard to forceMerge

  • delay (int) – Number of seconds to delay between forceMerge operations

client

The Elasticsearch client object derived from index_list

delay

Object attribute that gets the value of param delay.

do_action()

forcemerge() indices in index_list

do_dry_run()

Log what the output would be, but take no action.

index_list

The IndexList object passed from param ilo

max_num_segments

Object attribute that gets the value of param max_num_segments.

Index Settings

class curator.actions.IndexSettings(ilo, index_settings=None, ignore_unavailable=False, preserve_existing=False)

Bases: object

Index Settings Action Class

Parameters:
  • ilo (IndexList) – An IndexList Object

  • index_settings (dict) – A settings structure with one or more index settings to change.

  • ignore_unavailable (bool) – Whether specified concrete indices should be ignored when unavailable (missing or closed)

  • preserve_existing (bool) – Whether to update existing settings. If set to True, existing settings on an index remain unchanged. The default is False

body

Object attribute that gets the value of param index_settings.

client

The Elasticsearch client object derived from index_list

do_action()

put_settings() in body to indices in index_list

do_dry_run()

Log what the output would be, but take no action.

ignore_unavailable

Object attribute that gets the value of param ignore_unavailable.

index_list

The IndexList object passed from param ilo

preserve_existing

Object attribute that gets the value of param preserve_existing.

Open

class curator.actions.Open(ilo)

Bases: object

Open Action Class

Parameters:

ilo (IndexList) – An IndexList Object

client

The Elasticsearch client object derived from index_list

do_action()

open() indices in index_list

do_dry_run()

Log what the output would be, but take no action.

index_list

The IndexList object passed from param ilo

Reindex

class curator.actions.Reindex(ilo, request_body, refresh=True, requests_per_second=-1, slices=1, timeout=60, wait_for_active_shards=1, wait_for_completion=True, max_wait=-1, wait_interval=9, remote_certificate=None, remote_client_cert=None, remote_client_key=None, remote_filters=None, migration_prefix='', migration_suffix='')

Bases: object

Reindex Action Class

Parameters:
  • ilo (IndexList) – An IndexList Object

  • request_body (dict) – The body to send to reindex(), which must be complete and usable, as Curator will do no vetting of the request_body. If it fails to function, Curator will return an exception.

  • refresh (bool) – Whether to refresh the entire target index after the operation is complete.

  • requests_per_second (int) – The throttle to set on this request in sub-requests per second. -1 means set no throttle as does unlimited which is the only non-float this accepts.

  • slices (int) – The number of slices this task should be divided into. 1 means the task will not be sliced into subtasks. (Default: 1)

  • timeout (int) – The length in seconds each individual bulk request should wait for shards that are unavailable. (default: 60)

  • wait_for_active_shards (int) – Sets the number of shard copies that must be active before proceeding with the reindex operation. (Default: 1) means the primary shard only. Set to all for all shard copies, otherwise set to any non-negative value less than or equal to the total number of copies for the shard (number of replicas + 1)

  • wait_for_completion (bool) – Wait for completion before returning.

  • wait_interval (int) – Seconds to wait between completion checks.

  • max_wait (int) – Maximum number of seconds to wait_for_completion

  • remote_certificate (str) – Path to SSL/TLS certificate

  • remote_client_cert – Path to SSL/TLS client certificate (public key)

  • remote_client_key – Path to SSL/TLS private key

  • migration_prefix (str) – When migrating, prepend this value to the index name.

  • migration_suffix (str) – When migrating, append this value to the index name.

body

Object attribute that gets the value of param request_body.

client

The Elasticsearch client object derived from index_list

do_action()

Execute reindex() operation with the request_body from _get_request_body() and arguments refresh, requests_per_second, slices, timeout, wait_for_active_shards, and wfc.

do_dry_run()

Log what the output would be, but take no action.

get_processed_items(task_id)

This function calls get() with the provided task_id. It will get the value from 'response.total' as the total number of elements processed during reindexing. If the value is not found, it will return -1

Parameters:

task_id – A task_id which ostensibly matches a task searchable in the tasks API.

index_list

The IndexList object passed from param ilo

max_wait

Object attribute that gets the value of param max_wait.

migration

Object attribute that is set False unless body has {'dest': {'index': 'MIGRATION'}}, then it is set True

mpfx

Object attribute that gets the value of param migration_prefix.

msfx

Object attribute that gets the value of param migration_suffix.

refresh

Object attribute that gets the value of param refresh.

remote

Object attribute that is set False unless body has {'source': {'remote': {}}}, then it is set True

requests_per_second

Object attribute that gets the value of param requests_per_second.

show_run_args(source, dest)

Show what will run

slices

Object attribute that gets the value of param slices.

sources()

Generator for Reindexing sources & dests

timeout

Object attribute that gets the value of param timeout, convert to str and add s for seconds.

wait_for_active_shards

Object attribute that gets the value of param wait_for_active_shards.

wait_interval

Object attribute that gets the value of param wait_interval.

wfc

Object attribute that gets the value of param wait_for_completion.

Replicas

class curator.actions.Replicas(ilo, count=None, wait_for_completion=False, wait_interval=9, max_wait=-1)

Bases: object

Replica Action Class

Parameters:
  • ilo (IndexList) – An IndexList Object

  • count (int) – The count of replicas per shard

  • wait_for_completion (bool) – Wait for completion before returning.

  • wait_interval (int) – Seconds to wait between completion checks.

  • max_wait (int) – Maximum number of seconds to wait_for_completion

client

The Elasticsearch client object derived from index_list

count

Object attribute that gets the value of param count.

do_action()

Update number_of_replicas with count and put_settings() to indices in index_list

do_dry_run()

Log what the output would be, but take no action.

index_list

The IndexList object passed from param ilo

max_wait

Object attribute that gets the value of param max_wait.

wait_interval

Object attribute that gets the value of param wait_interval.

wfc

Object attribute that gets the value of param wait_for_completion.

Restore

class curator.actions.Restore(slo, name=None, indices=None, include_aliases=False, ignore_unavailable=False, include_global_state=False, partial=False, rename_pattern=None, rename_replacement=None, extra_settings=None, wait_for_completion=True, wait_interval=9, max_wait=-1, skip_repo_fs_check=True)

Bases: object

Restore Action Class

Read more about identically named settings at: elasticsearch.client.SnapshotClient.restore()

Parameters:
  • slo (SnapshotList) – A SnapshotList object

  • name (str) – Name of the snapshot to restore. If None, use the most recent snapshot.

  • indices (list) – Indices to restore. If None, all in the snapshot will be restored.

  • include_aliases (bool) – Restore aliases with the indices.

  • ignore_unavailable (bool) – Ignore unavailable shards/indices.

  • include_global_state (bool) – Restore cluster global state with snapshot.

  • partial (bool) – Do not fail if primary shard is unavailable.

  • rename_pattern (str) – A regular expression pattern with one or more captures, e.g. index_(.+)

  • rename_replacement (str) – A target index name pattern with $# numbered references to the captures in rename_pattern, e.g. restored_index_$1

  • extra_settings (dict) – Index settings to apply to restored indices.

  • wait_for_completion (bool) – Wait for completion before returning.

  • wait_interval (int) – Seconds to wait between completion checks.

  • max_wait (int) – Maximum number of seconds to wait_for_completion

  • skip_repo_fs_check (bool) – Do not validate write access to repository on all cluster nodes before proceeding. Useful for shared filesystems where intermittent timeouts can affect validation, but won’t likely affect snapshot success. (Default: True)

body

Object attribute that gets populated from other params/attributes. Deprecated, but not removed. Lazy way to keep from updating do_dry_run(). Will fix later.

client

The Elasticsearch client object derived from snapshot_list

do_action()

restore() indices from name with passed params.

do_dry_run()

Log what the output would be, but take no action.

ignore_unavailable

Object attribute that gets the value of param ignore_unavailable.

include_aliases

Object attribute that gets the value of param include_aliases.

include_global_state

Object attribute that gets the value of param include_global_state.

index_settings

Object attribute that gets the value of param extra_settings.

max_wait

Object attribute that gets the value of param max_wait.

name

Object attribute that gets the value of param name if not None, or the output from most_recent().

partial

Object attribute that gets the value of param partial.

py_rename_replacement

Object attribute derived from rename_replacement. but with Java regex group designations of $# converted to Python’s \\# style.

rename_pattern

Object attribute that gets the value of param rename_pattern. Empty str if None

rename_replacement

Object attribute that gets the value of param rename_replacement. Empty str if None

report_state()

Log the state of the restore. This should only be done if wait_for_completion is True, and only after completing the restore.

repository

Object attribute that gets the value of repository from snapshot_list.

skip_repo_fs_check

Object attribute that gets the value of param max_wait.

snapshot_list

Internal reference to slo

wait_interval

Object attribute that gets the value of param wait_interval.

wfc

Object attribute that gets the value of param wait_for_completion.

Rollover

class curator.actions.Rollover(client, name=None, conditions=None, new_index=None, extra_settings=None, wait_for_active_shards=1)

Bases: object

Rollover Action Class

Parameters:
  • client (Elasticsearch) – A client connection object

  • name (str) – The name of the single-index-mapped alias to test for rollover conditions.

  • new_index (str) – A new index name

  • conditions (dict) – Conditions to test

  • extra_settings (dict or None) – Must be either None, or a dictionary of settings to apply to the new index on rollover. This is used in place of settings in the Rollover API, mostly because it’s already existent in other places here in Curator

  • wait_for_active_shards (int) – The number of shards expected to be active before returning.

client

Object attribute that gets the value of param client.

conditions

Object attribute that gets the value of param conditions.

do_action()

rollover() the index referenced by alias name

do_dry_run()

Log what the output would be, but take no action.

doit(dry_run=False)

This exists solely to prevent having to have duplicate code in both do_dry_run() and do_action() because rollover() has its own dry_run flag.

log_result(result)

Log the results based on whether the index rolled over or not

name

Object attribute that gets the value of param name.

new_index

The parse_date_pattern() rendered version of what was passed as new_index, or else None

settings

Object attribute that gets the value of param extra_settings.

wait_for_active_shards

Object attribute that gets the value of param wait_for_active_shards.

Shrink

class curator.actions.Shrink(ilo, shrink_node='DETERMINISTIC', node_filters=None, number_of_shards=1, number_of_replicas=1, shrink_prefix='', shrink_suffix='-shrink', copy_aliases=False, delete_after=True, post_allocation=None, wait_for_active_shards=1, wait_for_rebalance=True, extra_settings=None, wait_for_completion=True, wait_interval=9, max_wait=-1)

Bases: object

Shrink Action Class

Parameters:
  • ilo (IndexList) – An IndexList Object

  • shrink_node (str) – The node name to use as the shrink target, or DETERMINISTIC, which will use the values in node_filters to determine which node will be the shrink node.

  • node_filters (dict) – If the value of shrink_node is DETERMINISTIC, the values in node_filters will be used while determining which node to allocate the shards on before performing the shrink.

  • number_of_shards (int) – The number of shards the shrunk index should have

  • number_of_replicas (int) – The number of replicas for the shrunk index

  • shrink_prefix (str) – Prepend the shrunk index with this value

  • shrink_suffix (str) – Append the value to the shrunk index (Default: -shrink)

  • copy_aliases (bool) – Whether to copy each source index aliases to target index after shrinking. The aliases will be added to target index and deleted from source index at the same time. (Default: False)

  • delete_after (bool) – Whether to delete each index after shrinking. (Default: True)

  • post_allocation (dict) – If populated, the allocation_type, key, and value will be applied to the shrunk index to re-route it.

  • extra_settings (dict) – Permitted root keys are settings and aliases.

  • wait_for_active_shards (int) – Wait for this many active shards before returning.

  • wait_for_rebalance (bool) – Wait for rebalance. (Default: True)

  • wait_for_completion (bool) – Wait for completion before returning.

  • wait_interval (int) – Seconds to wait between completion checks.

  • max_wait (int) – Maximum number of seconds to wait_for_completion

client

The Elasticsearch client object derived from index_list

copy_aliases

Object attribute that gets the value of param copy_aliases.

delete_after

Object attribute that gets the value of param delete_after.

do_action()

shrink() the indices in index_list

do_copy_aliases(source_idx, target_idx)

Copy the aliases to the shrunk index

do_dry_run()

Show what a regular run would do, but don’t actually do it.

index_list

The IndexList object passed from param ilo

max_wait

Object attribute that gets the value of param max_wait.

most_available_node()

Determine which data node name has the most available free space, and meets the other node filters settings.

node_filters

Object attribute that gets the value of param node_filters.

number_of_shards

Object attribute that gets the value of param number_of_shards.

post_allocation

Object attribute that gets the value of param post_allocation.

pre_shrink_check(idx, dry_run=False)

Do a shrink preflight check

qualify_single_node()

Qualify a single node as a shrink target

route_index(idx, allocation_type, key, value)

Apply the indicated shard routing allocation

settings

Object attribute that gets values from params number_of_shards and number_of_replicas.

shrink_node

Object attribute that gets the value of param shrink_node.

shrink_node_avail

Object attribute that represents whether shrink_node_name is available

shrink_node_id

Object attribute that represents the node_id of shrink_node_name

shrink_node_name

Object attribute that represents the target node for shrinking.

shrink_prefix

Object attribute that gets the value of param shrink_prefix.

shrink_suffix

Object attribute that gets the value of param shrink_suffix.

wait_for_active_shards

Object attribute that gets the value of param wait_for_active_shards.

wait_for_rebalance

Object attribute that gets the value of param wait_for_rebalance.

wait_interval

Object attribute that gets the value of param wait_interval.

wfc

Object attribute that gets the value of param wait_for_completion.

Snapshot

class curator.actions.Snapshot(ilo, repository=None, name=None, ignore_unavailable=False, include_global_state=True, partial=False, wait_for_completion=True, wait_interval=9, max_wait=-1, skip_repo_fs_check=True)

Bases: object

Snapshot Action Class

Read more about identically named settings at: elasticsearch.client.SnapshotClient.create()

Parameters:
  • ilo (IndexList) – An IndexList Object

  • repository (str) – Repository name.

  • name (str) – Snapshot name.

  • ignore_unavailable (bool) – Ignore unavailable shards/indices.

  • include_global_state (bool) – Store cluster global state with snapshot.

  • partial (bool) – Do not fail if primary shard is unavailable.

  • wait_for_completion (bool) – Wait for completion before returning.

  • wait_interval (int) – Seconds to wait between completion checks.

  • max_wait (int) – Maximum number of seconds to wait_for_completion

  • skip_repo_fs_check (bool) – Do not validate write access to repository on all cluster nodes before proceeding. Useful for shared filesystems where intermittent timeouts can affect validation, but won’t likely affect snapshot success. (Default: True)

client

The Elasticsearch client object derived from index_list

do_action()

elasticsearch.client.SnapshotClient.create() a snapshot of indices, with passed parameters.

do_dry_run()

Log what the output would be, but take no action.

get_state()

Get the state of the snapshot and set state

ignore_unavailable

Object attribute that gets the value of param ignore_unavailable.

include_global_state

Object attribute that gets the value of param include_global_state.

index_list

The IndexList object passed from param ilo

indices

Object attribute that contains the to_csv() output of the indices in index_list.

max_wait

Object attribute that gets the value of param max_wait.

name

The parse_date_pattern() rendered version of what was passed by param name.

partial

Object attribute that gets the value of param partial.

report_state()

Log the state of the snapshot and raise FailedSnapshot if state is not SUCCESS

repository

Object attribute that gets the value of param repository.

settings

Object attribute dictionary compiled from indices, ignore_unavailable, include_global_state, and partial

skip_repo_fs_check

Object attribute that gets the value of param skip_repo_fs_check.

state

Object attribute that tracks the snapshot state.

wait_for_completion

Object attribute that gets the value of param wait_for_completion.

wait_interval

Object attribute that gets the value of param wait_interval.