Utility & Helper Methods

Sections

Get Information

get_alias

api.get_alias(client, alias)

Return information about the specified alias.

Parameters:
  • client – The Elasticsearch client connection
  • alias – Alias name to operate on.
Return type:

list of strings

get_indices

api.get_indices(client)

get_repository

api.get_repository(client, repository='')

Return configuration information for the indicated repository.

Parameters:
  • client – The Elasticsearch client connection
  • repository – The Elasticsearch snapshot repository to use
Return type:

dict

get_segmentcount

api.get_segmentcount(client, index_name)

Return a tuple of (shardcount, segmentcount) from the provided index_name.

Parameters:
  • client – The Elasticsearch client connection
  • index_name – The index name
Return type:

tuple

get_snapshot

api.get_snapshot(client, repository='', snapshot='')

Return information about a snapshot (or a comma-separated list of snapshots) If no snapshot specified, it will return all snapshots. If none exist, an empty dictionary will be returned.

Parameters:
  • client – The Elasticsearch client connection
  • repository – The Elasticsearch snapshot repository to use
  • snapshot – The snapshot name, or a comma-separated list of snapshots
Return type:

dict

get_snapshots

api.get_snapshots(client, repository=None)

Get _all snapshots from repository and return a list.

Parameters:
  • client – The Elasticsearch client connection
  • repository – The Elasticsearch snapshot repository to use
Return type:

list of strings

get_version

api.get_version(client)

Return the ES version number as a tuple. Omits trailing tags like -dev, or Beta

Parameters:client – The Elasticsearch client connection
Return type:tuple

Verification

check_csv

api.check_csv(value)

Some of the curator methods should not operate against multiple indices at once. This method can be used to check if a list or csv has been sent.

Parameters:value – The value to test, if list or csv string
Return type:bool

ensure_list

api.ensure_list(indices)

Return a list, even if indices is a single value

Parameters:indices – A list of indices to act upon
Return type:list

index_closed

api.index_closed(client, index_name)

Return True if the indicated index is closed.

Parameters:
  • client – The Elasticsearch client connection
  • index_name – The index name
Return type:

bool

optimized

api.optimized(client, index_name, max_num_segments=None)

Check if an index is optimized.

Parameters:
  • client – The Elasticsearch client connection
  • index_name – The index name
  • max_num_segments – Merge to this number of segments per shard.
Return type:

bool

is_master_node

api.is_master_node(client)

Return True if the connected client node is the elected master node in the Elasticsearch cluster, otherwise return False.

Parameters:client – The Elasticsearch client connection
Return type:bool

to_csv

api.to_csv(indices)

Return a csv string from a list of indices, or a single value if only one value is present

Parameters:indices – A list of indices to act on, or a single value, which could be in the format of a csv string already.
Return type:str

verify_repository

api.verify_repository(client, repository=None)

Verify the existence of a repository

Args:
client: The Elasticsearch client connection
Kwargs:
repository: The Elasticsearch snapshot repository to use
Returns:
A boolean value indicating success or failure.

Index Pruning

prune_allocated

api.prune_allocated(client, indices, key, value, allocation_type)

Return list of indices that do not have the routing allocation rule of key=value

Parameters:
  • client – The Elasticsearch client connection
  • indices – A list of indices to act on
  • key – The allocation attribute to check for
  • value – The value to check for
  • allocation_type – Type of allocation to apply
Return type:

list

prune_closed

api.prune_closed(client, indices)

Return list of indices that are not closed.

Parameters:
  • client – The Elasticsearch client connection
  • indices – A list of indices to act on
Return type:

list

prune_kibana

api.prune_kibana(indices)

Remove any index named .kibana, kibana-int, or .marvel-kibana

Parameters:indices – A list of indices to act upon.
Return type:list

Other

create_snapshot_body

api.create_snapshot_body(indices, ignore_unavailable=False, include_global_state=True, partial=False)

Create the request body for creating a snapshot from the provided arguments.

Parameters:
  • indices – A single index, or list of indices to snapshot.
  • ignore_unavailable – Boolean. Ignore unavailable shards/indices. (default: False)
  • include_global_state – Boolean. Store cluster global state with snapshot. (default: True)
  • partial – Boolean. Do not fail if primary shard is unavailable. (default: False)
Return type:

dict

create_repo_body

api.create_repo_body(repo_type=None, compress=True, chunk_size=None, max_restore_bytes_per_sec=None, max_snapshot_bytes_per_sec=None, location=None, bucket=None, region=None, base_path=None, access_key=None, secret_key=None, **kwargs)

Build the ‘body’ portion for use in creating a repository.

Parameters:
  • repo_type – The type of repository (presently only fs and s3)
  • compress – Turn on compression of the snapshot files. Compression is applied only to metadata files (index mapping and settings). Data files are not compressed. (Default: True)
  • chunk_size – The chunk size can be specified in bytes or by using size value notation, i.e. 1g, 10m, 5k. Defaults to null (unlimited chunk size).
  • max_restore_bytes_per_sec – Throttles per node restore rate. Defaults to 20mb per second.
  • max_snapshot_bytes_per_sec – Throttles per node snapshot rate. Defaults to 20mb per second.
  • location – Location of the snapshots. Required.
  • bucketS3 only. The name of the bucket to be used for snapshots. Required.
  • regionS3 only. The region where bucket is located. Defaults to US Standard
  • base_pathS3 only. Specifies the path within bucket to repository data. Defaults to value of repositories.s3.base_path or to root directory if not set.
  • access_keyS3 only. The access key to use for authentication. Defaults to value of cloud.aws.access_key.
  • secret_keyS3 only. The secret key to use for authentication. Defaults to value of cloud.aws.secret_key.
Returns:

A dictionary suitable for creating a repository from the provided arguments.

Return type:

dict

create_repository

api.create_repository(client, **kwargs)

Create repository with repository and body settings

Parameters:
  • client – The Elasticsearch client connection
  • repo_type – The type of repository (presently only fs and s3)
  • compress – Turn on compression of the snapshot files. Compression is applied only to metadata files (index mapping and settings). Data files are not compressed. (Default: True)
  • chunk_size – The chunk size can be specified in bytes or by using size value notation, i.e. 1g, 10m, 5k. Defaults to null (unlimited chunk size).
  • max_restore_bytes_per_sec – Throttles per node restore rate. Defaults to 20mb per second.
  • max_snapshot_bytes_per_sec – Throttles per node snapshot rate. Defaults to 20mb per second.
  • location – Location of the snapshots. Required.
  • bucketS3 only. The name of the bucket to be used for snapshots. Required.
  • regionS3 only. The region where bucket is located. Defaults to US Standard
  • base_pathS3 only. Specifies the path within bucket to repository data. Defaults to value of repositories.s3.base_path or to root directory if not set.
  • access_keyS3 only. The access key to use for authentication. Defaults to value of cloud.aws.access_key.
  • secret_keyS3 only. The secret key to use for authentication. Defaults to value of cloud.aws.secret_key.
Returns:

A boolean value indicating success or failure.

Return type:

boolean

chunk_index_list

api.chunk_index_list(indices)

This utility chunks very large index lists into 3KB chunks It measures the size as a csv string, then converts back into a list for the return value.

Parameters:indices – A list of indices to act on.