Actions¶

Alias¶

class curator.actions.Alias(name=None, extra_settings=None, **kwargs)¶

Bases: object

Alias Action Class

Parameters:	name (str) – The alias name extra_settings (dict) – Extra settings, including filters and routing. For more information see here.

actions = None¶: The list of actions to perform. Populated by add() and remove()

add(ilo, warn_if_no_indices=False)¶

Create add statements for each index in ilo for name, then append them to actions. Add any extra_settings that may be there.

Parameters:	ilo (`IndexList`) – An IndexList Object

check_actions()¶

Returns:	`actions` for use with the `update_aliases()` API call if actions exist, otherwise an exception is raised.

client = None¶: The Elasticsearch client object which will later be set by add() or remove()

do_action()¶: update_aliases() for name with actions

do_dry_run()¶: Log what the output would be, but take no action.

extra_settings = None¶: Any extra things to add to the alias, like filters, or routing. Gets the value from param extra_settings.

name = None¶: The parse_date_pattern() rendered version of what was passed by param name.

remove(ilo, warn_if_no_indices=False)¶

Create remove statements for each index in ilo for name, then append them to actions.

Parameters:	ilo (`IndexList`) – An IndexList Object

warn_if_no_indices = None¶: Preset default value to False.

Allocation¶

class curator.actions.Allocation(ilo, key=None, value=None, allocation_type='require', wait_for_completion=False, wait_interval=3, max_wait=-1)¶

Bases: object

Allocation Action Class

Parameters:

ilo (IndexList) – An IndexList Object
key (str) – An arbitrary metadata attribute key. Must match the key assigned to at least
value (str) – An arbitrary metadata attribute value. Must correspond to values associated
allocation_type (str) – Type of allocation to apply. Default is require
wait_for_completion (bool) – Wait for completion before returning.
wait_interval (int) – Seconds to wait between completion checks.
max_wait (int) – Maximum number of seconds to wait_for_completion

Note

See more about shard allocation filtering.

client = None¶: The Elasticsearch client object derived from index_list

do_action()¶: put_settings() to indices in index_list with settings.

do_dry_run()¶: Log what the output would be, but take no action.

index_list = None¶: The IndexList object passed as ilo

max_wait = None¶: Object attribute that gets the value of param max_wait

settings = None¶: Populated at instance creation time. Value is built from the passed params allocation_type, key, and value, e.g. index.routing.allocation.allocation_type.key.value

wait_interval = None¶: Object attribute that gets the value of param wait_interval

wfc = None¶: Object attribute that gets the value of param wait_for_completion

Close¶

class curator.actions.Close(ilo, delete_aliases=False, skip_flush=False)¶

Bases: object

Close Action Class

Parameters:	ilo (`IndexList`) – An IndexList Object delete_aliases (bool) – Delete any associated aliases before closing indices. skip_flush (bool) – Do not flush indices before closing.

client = None¶: The Elasticsearch client object derived from index_list

delete_aliases = None¶: The value passed as delete_aliases

do_action()¶: close() open indices in index_list

do_dry_run()¶: Log what the output would be, but take no action.

index_list = None¶: The IndexList object passed from param ilo

skip_flush = None¶: The value passed as skip_flush

Cluster Routing¶

class curator.actions.ClusterRouting(client, routing_type=None, setting=None, value=None, wait_for_completion=False, wait_interval=9, max_wait=-1)¶

Bases: object

ClusterRouting Action Class

For now, the cluster routing settings are hardcoded to be transient

Parameters:

client (Elasticsearch) – A client connection object
routing_type (str) – Type of routing to apply. Either allocation or rebalance
setting (str) – Currently, the only acceptable value for setting is enable. This is here in case that changes.
value (str) – Used only if setting is enable. Semi-dependent on routing_type. Acceptable values for allocation and rebalance are all, primaries, and none (string, not None). If routing_type is allocation, this can also be new_primaries, and if rebalance, it can be replicas.
wait_for_completion (bool) – Wait for completion before returning.
wait_interval (int) – Seconds to wait between completion checks.
max_wait (int) – Maximum number of seconds to wait_for_completion

client = None¶: An Elasticsearch client object

do_action()¶: put_settings() to the cluster with settings.

do_dry_run()¶: Log what the output would be, but take no action.

max_wait = None¶: Object attribute that gets the value of param max_wait. How long in seconds to wfc before returning with an exception. A value of -1 means wait forever.

settings = None¶: Populated at instance creation time. Value is built from the passed values from params routing_type and setting, e.g. cluster.routing.routing_type.setting

wait_interval = None¶: Object attribute that gets the value of param wait_interval

wfc = None¶: Object attribute that gets the value of param wait_for_completion

Create Index¶

class curator.actions.CreateIndex(client, name=None, extra_settings=None, ignore_existing=False)¶

Bases: object

Create Index Action Class

Parameters:

client (Elasticsearch) – A client connection object
name (str) – A name, which can contain time.strftime() strings
extra_settings (dict) – The settings and mappings for the index. For more information see the create indices documentation.
ignore_existing (bool) – If an index already exists, and this setting is True, ignore the 400 error that results in a resource_already_exists_exception and return that it was successful.

aliases = None¶: Gets any aliases from extra_settings or is None

client = None¶: An Elasticsearch client object

do_action()¶: create() index identified by name with values from aliases, mappings, and settings

do_dry_run()¶: Log what the output would be, but take no action.

extra_settings = None¶: Any extra settings for the index, like aliases, mappings, or settings. Gets the value from param extra_settings.

ignore_existing = None¶: Extracted from the action definition, it should be a boolean informing whether to ignore the error if the index already exists.

mappings = None¶: Gets any mappings from extra_settings or is None

name = None¶: The parse_date_pattern() rendered version of what was passed as name.

settings = None¶: Gets any settings from extra_settings or is None

Delete Indices¶

class curator.actions.DeleteIndices(ilo, master_timeout=30)¶

Bases: object

Delete Indices Action Class

Parameters:	ilo (`IndexList`) – An IndexList Object master_timeout (int) – Number of seconds to wait for master node response

client = None¶: The Elasticsearch client object derived from index_list

do_action()¶: delete() indices in index_list

do_dry_run()¶: Log what the output would be, but take no action.

index_list = None¶: The IndexList object passed from param ilo

master_timeout = None¶: String value of param master_timeout + s, for seconds.

Delete Snapshots¶

class curator.actions.DeleteSnapshots(slo, retry_interval=120, retry_count=3)¶

Bases: object

Delete Snapshots Action Class

Parameters:	slo (`SnapshotList`) – A SnapshotList object retry_interval (int) – Seconds to delay betwen retries. (Default: `120`) retry_count (int) – Number of attempts to make. (Default: `3`)

client = None¶: The Elasticsearch client object derived from snapshot_list

do_action()¶: delete() snapshots in snapshot_list. Retry up to retry_count times, pausing retry_interval seconds between retries.

do_dry_run()¶: Log what the output would be, but take no action.

repository = None¶: Object attribute that gets its value from snapshot_list.

retry_count = None¶: Object attribute that gets the value of param retry_count.

retry_interval = None¶: Object attribute that gets the value of param retry_interval.

snapshot_list = None¶: The SnapshotList object passed from param slo

Force Merge¶

class curator.actions.ForceMerge(ilo, max_num_segments=None, delay=0)¶

Bases: object

ForceMerge Action Class

Parameters:	ilo (`IndexList`) – An IndexList Object max_num_segments (int) – Number of segments per shard to forceMerge delay (int) – Number of seconds to delay between forceMerge operations

client = None¶: The Elasticsearch client object derived from index_list

delay = None¶: Object attribute that gets the value of param delay.

do_action()¶: forcemerge() indices in index_list

do_dry_run()¶: Log what the output would be, but take no action.

index_list = None¶: The IndexList object passed from param ilo

max_num_segments = None¶: Object attribute that gets the value of param max_num_segments.

Index Settings¶

class curator.actions.IndexSettings(ilo, index_settings=None, ignore_unavailable=False, preserve_existing=False)¶

Bases: object

Index Settings Action Class

Parameters:

ilo (IndexList) – An IndexList Object
index_settings (dict) – A settings structure with one or more index settings to change.
ignore_unavailable (bool) – Whether specified concrete indices should be ignored when unavailable (missing or closed)
preserve_existing (bool) – Whether to update existing settings. If set to True, existing settings on an index remain unchanged. The default is False

body = None¶: Object attribute that gets the value of param index_settings.

client = None¶: The Elasticsearch client object derived from index_list

do_action()¶: put_settings() in body to indices in index_list

do_dry_run()¶: Log what the output would be, but take no action.

ignore_unavailable = None¶: Object attribute that gets the value of param ignore_unavailable.

index_list = None¶: The IndexList object passed from param ilo

preserve_existing = None¶: Object attribute that gets the value of param preserve_existing.

Open¶

class curator.actions.Open(ilo)¶

Bases: object

Open Action Class

Parameters:	ilo (`IndexList`) – An IndexList Object

client = None¶: The Elasticsearch client object derived from index_list

do_action()¶: open() indices in index_list

do_dry_run()¶: Log what the output would be, but take no action.

index_list = None¶: The IndexList object passed from param ilo

Reindex¶

class curator.actions.Reindex(ilo, request_body, refresh=True, requests_per_second=-1, slices=1, timeout=60, wait_for_active_shards=1, wait_for_completion=True, max_wait=-1, wait_interval=9, remote_certificate=None, remote_client_cert=None, remote_client_key=None, remote_filters=None, migration_prefix='', migration_suffix='')¶

Bases: object

Reindex Action Class

Parameters:

ilo (IndexList) – An IndexList Object
request_body (dict) – The body to send to reindex(), which must be complete and usable, as Curator will do no vetting of the request_body. If it fails to function, Curator will return an exception.
refresh (bool) – Whether to refresh the entire target index after the operation is complete.
requests_per_second (int) – The throttle to set on this request in sub-requests per second. -1 means set no throttle as does unlimited which is the only non-float this accepts.
slices (int) – The number of slices this task should be divided into. 1 means the task will not be sliced into subtasks. (Default: 1)
timeout (int) – The length in seconds each individual bulk request should wait for shards that are unavailable. (default: 60)
wait_for_active_shards (int) – Sets the number of shard copies that must be active before proceeding with the reindex operation. (Default: 1) means the primary shard only. Set to all for all shard copies, otherwise set to any non-negative value less than or equal to the total number of copies for the shard (number of replicas + 1)
wait_for_completion (bool) – Wait for completion before returning.
wait_interval (int) – Seconds to wait between completion checks.
max_wait (int) – Maximum number of seconds to wait_for_completion
remote_certificate (str) – Path to SSL/TLS certificate
remote_client_cert – Path to SSL/TLS client certificate (public key)
remote_client_key – Path to SSL/TLS private key
migration_prefix (str) – When migrating, prepend this value to the index name.
migration_suffix (str) – When migrating, append this value to the index name.

body = None¶: Object attribute that gets the value of param request_body.

client = None¶: The Elasticsearch client object derived from index_list

do_action()¶: Execute reindex() operation with the request_body from _get_request_body() and arguments refresh, requests_per_second, slices, timeout, wait_for_active_shards, and wfc.

do_dry_run()¶: Log what the output would be, but take no action.

get_processed_items(task_id)¶

This function calls get() with the provided task_id. It will get the value from 'response.total' as the total number of elements processed during reindexing. If the value is not found, it will return -1

Parameters:	task_id – A task_id which ostensibly matches a task searchable in the tasks API.

index_list = None¶: The IndexList object passed from param ilo

max_wait = None¶: Object attribute that gets the value of param max_wait.

migration = None¶: Object attribute that is set False unless body has {'dest': {'index': 'MIGRATION'}}, then it is set True

mpfx = None¶: Object attribute that gets the value of param migration_prefix.

msfx = None¶: Object attribute that gets the value of param migration_suffix.

refresh = None¶: Object attribute that gets the value of param refresh.

remote = None¶: Object attribute that is set False unless body has {'source': {'remote': {}}}, then it is set True

requests_per_second = None¶: Object attribute that gets the value of param requests_per_second.

show_run_args(source, dest)¶: Show what will run

slices = None¶: Object attribute that gets the value of param slices.

sources()¶: Generator for Reindexing sources & dests

timeout = None¶: Object attribute that gets the value of param timeout, convert to str and add s for seconds.

wait_for_active_shards = None¶: Object attribute that gets the value of param wait_for_active_shards.

wait_interval = None¶: Object attribute that gets the value of param wait_interval.

wfc = None¶: Object attribute that gets the value of param wait_for_completion.

Replicas¶

class curator.actions.Replicas(ilo, count=None, wait_for_completion=False, wait_interval=9, max_wait=-1)¶

Bases: object

Replica Action Class

Parameters:	ilo (`IndexList`) – An IndexList Object count (int) – The count of replicas per shard wait_for_completion (bool) – Wait for completion before returning. wait_interval (int) – Seconds to wait between completion checks. max_wait (int) – Maximum number of seconds to `wait_for_completion`

client = None¶: The Elasticsearch client object derived from index_list

count = None¶: Object attribute that gets the value of param count.

do_action()¶: Update number_of_replicas with count and put_settings() to indices in index_list

do_dry_run()¶: Log what the output would be, but take no action.

index_list = None¶: The IndexList object passed from param ilo

max_wait = None¶: Object attribute that gets the value of param max_wait.

wait_interval = None¶: Object attribute that gets the value of param wait_interval.

wfc = None¶: Object attribute that gets the value of param wait_for_completion.

Restore¶

class curator.actions.Restore(slo, name=None, indices=None, include_aliases=False, ignore_unavailable=False, include_global_state=False, partial=False, rename_pattern=None, rename_replacement=None, extra_settings=None, wait_for_completion=True, wait_interval=9, max_wait=-1, skip_repo_fs_check=True)¶

Bases: object

Restore Action Class

Read more about identically named settings at: elasticsearch.client.SnapshotClient.restore()

Parameters:

slo (SnapshotList) – A SnapshotList object
name (str) – Name of the snapshot to restore. If None, use the most recent snapshot.
indices (list) – Indices to restore. If None, all in the snapshot will be restored.
include_aliases (bool) – Restore aliases with the indices.
ignore_unavailable (bool) – Ignore unavailable shards/indices.
include_global_state (bool) – Restore cluster global state with snapshot.
partial (bool) – Do not fail if primary shard is unavailable.
rename_pattern (str) – A regular expression pattern with one or more captures, e.g. index_(.+)
rename_replacement (str) – A target index name pattern with $# numbered references to the captures in rename_pattern, e.g. restored_index_$1
extra_settings (dict) – Index settings to apply to restored indices.
wait_for_completion (bool) – Wait for completion before returning.
wait_interval (int) – Seconds to wait between completion checks.
max_wait (int) – Maximum number of seconds to wait_for_completion
skip_repo_fs_check (bool) – Do not validate write access to repository on all cluster nodes before proceeding. Useful for shared filesystems where intermittent timeouts can affect validation, but won’t likely affect snapshot success. (Default: True)

body = None¶: Object attribute that gets populated from other params/attributes. Deprecated, but not removed. Lazy way to keep from updating do_dry_run(). Will fix later.

client = None¶: The Elasticsearch client object derived from snapshot_list

do_action()¶: restore() indices from name with passed params.

do_dry_run()¶: Log what the output would be, but take no action.

ignore_unavailable = None¶: Object attribute that gets the value of param ignore_unavailable.

include_aliases = None¶: Object attribute that gets the value of param include_aliases.

include_global_state = None¶: Object attribute that gets the value of param include_global_state.

index_settings = None¶: Object attribute that gets the value of param extra_settings.

max_wait = None¶: Object attribute that gets the value of param max_wait.

name = None¶: Object attribute that gets the value of param name if not None, or the output from most_recent().

partial = None¶: Object attribute that gets the value of param partial.

py_rename_replacement = None¶: Object attribute derived from rename_replacement. but with Java regex group designations of $# converted to Python’s \\# style.

rename_pattern = None¶: Object attribute that gets the value of param rename_pattern. Empty str if None

rename_replacement = None¶: Object attribute that gets the value of param rename_replacement. Empty str if None

report_state()¶: Log the state of the restore. This should only be done if wait_for_completion is True, and only after completing the restore.

repository = None¶: Object attribute that gets the value of repository from snapshot_list.

skip_repo_fs_check = None¶: Object attribute that gets the value of param max_wait.

snapshot_list = None¶: Internal reference to slo

wait_interval = None¶: Object attribute that gets the value of param wait_interval.

wfc = None¶: Object attribute that gets the value of param wait_for_completion.

Rollover¶

class curator.actions.Rollover(client, name=None, conditions=None, new_index=None, extra_settings=None, wait_for_active_shards=1)¶

Bases: object

Rollover Action Class

Parameters:

client (Elasticsearch) – A client connection object
name (str) – The name of the single-index-mapped alias to test for rollover conditions.
new_index (str) – A new index name
conditions (dict) – Conditions to test
extra_settings (dict or None) – Must be either None, or a dictionary of settings to apply to the new index on rollover. This is used in place of settings in the Rollover API, mostly because it’s already existent in other places here in Curator
wait_for_active_shards (int) – The number of shards expected to be active before returning.

client = None¶: Object attribute that gets the value of param client.

conditions = None¶: Object attribute that gets the value of param conditions.

do_action()¶: rollover() the index referenced by alias name

do_dry_run()¶: Log what the output would be, but take no action.

doit(dry_run=False)¶: This exists solely to prevent having to have duplicate code in both do_dry_run() and do_action() because rollover() has its own dry_run flag.

log_result(result)¶: Log the results based on whether the index rolled over or not

name = None¶: Object attribute that gets the value of param name.

new_index = None¶: The parse_date_pattern() rendered version of what was passed as new_index, or else None

settings = None¶: Object attribute that gets the value of param extra_settings.

wait_for_active_shards = None¶: Object attribute that gets the value of param wait_for_active_shards.

Shrink¶

class curator.actions.Shrink(ilo, shrink_node='DETERMINISTIC', node_filters=None, number_of_shards=1, number_of_replicas=1, shrink_prefix='', shrink_suffix='-shrink', copy_aliases=False, delete_after=True, post_allocation=None, wait_for_active_shards=1, wait_for_rebalance=True, extra_settings=None, wait_for_completion=True, wait_interval=9, max_wait=-1)¶

Bases: object

Shrink Action Class

Parameters:

ilo (IndexList) – An IndexList Object
shrink_node (str) – The node name to use as the shrink target, or DETERMINISTIC, which will use the values in node_filters to determine which node will be the shrink node.
node_filters (dict) – If the value of shrink_node is DETERMINISTIC, the values in node_filters will be used while determining which node to allocate the shards on before performing the shrink.
number_of_shards (int) – The number of shards the shrunk index should have
number_of_replicas (int) – The number of replicas for the shrunk index
shrink_prefix (str) – Prepend the shrunk index with this value
shrink_suffix (str) – Append the value to the shrunk index (Default: -shrink)
copy_aliases (bool) – Whether to copy each source index aliases to target index after shrinking. The aliases will be added to target index and deleted from source index at the same time. (Default: False)
delete_after (bool) – Whether to delete each index after shrinking. (Default: True)
post_allocation (dict) – If populated, the allocation_type, key, and value will be applied to the shrunk index to re-route it.
extra_settings (dict) – Permitted root keys are settings and aliases.
wait_for_active_shards (int) – Wait for this many active shards before returning.
wait_for_rebalance (bool) – Wait for rebalance. (Default: True)
wait_for_completion (bool) – Wait for completion before returning.
wait_interval (int) – Seconds to wait between completion checks.
max_wait (int) – Maximum number of seconds to wait_for_completion

client = None¶: The Elasticsearch client object derived from index_list

copy_aliases = None¶: Object attribute that gets the value of param copy_aliases.

delete_after = None¶: Object attribute that gets the value of param delete_after.

do_action()¶: shrink() the indices in index_list

do_copy_aliases(source_idx, target_idx)¶: Copy the aliases to the shrunk index

do_dry_run()¶: Show what a regular run would do, but don’t actually do it.

index_list = None¶: The IndexList object passed from param ilo

max_wait = None¶: Object attribute that gets the value of param max_wait.

most_available_node()¶: Determine which data node name has the most available free space, and meets the other node filters settings.

node_filters = None¶: Object attribute that gets the value of param node_filters.

number_of_shards = None¶: Object attribute that gets the value of param number_of_shards.

post_allocation = None¶: Object attribute that gets the value of param post_allocation.

pre_shrink_check(idx, dry_run=False)¶: Do a shrink preflight check

qualify_single_node()¶: Qualify a single node as a shrink target

route_index(idx, allocation_type, key, value)¶: Apply the indicated shard routing allocation

settings = None¶: Object attribute that gets values from params number_of_shards and number_of_replicas.

shrink_node = None¶: Object attribute that gets the value of param shrink_node.

shrink_node_avail = None¶: Object attribute that represents whether shrink_node_name is available

shrink_node_id = None¶: Object attribute that represents the node_id of shrink_node_name

shrink_node_name = None¶: Object attribute that represents the target node for shrinking.

shrink_prefix = None¶: Object attribute that gets the value of param shrink_prefix.

shrink_suffix = None¶: Object attribute that gets the value of param shrink_suffix.

wait_for_active_shards = None¶: Object attribute that gets the value of param wait_for_active_shards.

wait_for_rebalance = None¶: Object attribute that gets the value of param wait_for_rebalance.

wait_interval = None¶: Object attribute that gets the value of param wait_interval.

wfc = None¶: Object attribute that gets the value of param wait_for_completion.

Snapshot¶

class curator.actions.Snapshot(ilo, repository=None, name=None, ignore_unavailable=False, include_global_state=True, partial=False, wait_for_completion=True, wait_interval=9, max_wait=-1, skip_repo_fs_check=True)¶

Bases: object

Snapshot Action Class

Read more about identically named settings at: elasticsearch.client.SnapshotClient.create()

Parameters:

ilo (IndexList) – An IndexList Object
repository (str) – Repository name.
name (str) – Snapshot name.
ignore_unavailable (bool) – Ignore unavailable shards/indices.
include_global_state (bool) – Store cluster global state with snapshot.
partial (bool) – Do not fail if primary shard is unavailable.
wait_for_completion (bool) – Wait for completion before returning.
wait_interval (int) – Seconds to wait between completion checks.
max_wait (int) – Maximum number of seconds to wait_for_completion
skip_repo_fs_check (bool) – Do not validate write access to repository on all cluster nodes before proceeding. Useful for shared filesystems where intermittent timeouts can affect validation, but won’t likely affect snapshot success. (Default: True)

client = None¶: The Elasticsearch client object derived from index_list

do_action()¶: elasticsearch.client.SnapshotClient.create() a snapshot of indices, with passed parameters.

do_dry_run()¶: Log what the output would be, but take no action.

get_state()¶: Get the state of the snapshot and set state

ignore_unavailable = None¶: Object attribute that gets the value of param ignore_unavailable.

include_global_state = None¶: Object attribute that gets the value of param include_global_state.

index_list = None¶: The IndexList object passed from param ilo

indices = None¶: Object attribute that contains the to_csv() output of the indices in index_list.

max_wait = None¶: Object attribute that gets the value of param max_wait.

name = None¶: The parse_date_pattern() rendered version of what was passed by param name.

partial = None¶: Object attribute that gets the value of param partial.

report_state()¶: Log the state of the snapshot and raise FailedSnapshot if state is not SUCCESS

repository = None¶: Object attribute that gets the value of param repository.

settings = None¶: Object attribute dictionary compiled from indices, ignore_unavailable, include_global_state, and partial

skip_repo_fs_check = None¶: Object attribute that gets the value of param skip_repo_fs_check.

state = None¶: Object attribute that tracks the snapshot state.

wait_for_completion = None¶: Object attribute that gets the value of param wait_for_completion.

wait_interval = None¶: Object attribute that gets the value of param wait_interval.