Object Classes¶
IndexList¶
-
class
curator.indexlist.
IndexList
(client)¶ -
all_indices
= None¶ Instance variable. All indices in the cluster at instance creation time. Type:
list()
-
client
= None¶ An Elasticsearch Client object Also accessible as an instance variable.
-
empty_list_check
()¶ Raise exception if indices is empty
-
filter_allocated
(key=None, value=None, allocation_type='require', exclude=True)¶ Match indices that have the routing allocation rule of key=value from indices
Parameters: - key – The allocation attribute to check for
- value – The value to check for
- allocation_type – Type of allocation to apply
- exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is True
-
filter_by_age
(source='name', direction=None, timestring=None, unit=None, unit_count=None, field=None, stats_result='min_value', epoch=None, exclude=False)¶ Match indices by relative age calculations.
Parameters: - source – Source of index age. Can be one of ‘name’, ‘creation_date’, or ‘field_stats’
- direction – Time to filter, either
older
oryounger
- timestring – An strftime string to match the datestamp in an index
name. Only used for index filtering by
name
. - unit – One of
seconds
,minutes
,hours
,days
,weeks
,months
, oryears
. - unit_count – The number of
unit``s. ``unit_count
*unit
will be calculated out to the relative number of seconds. - field – A timestamp field name. Only used for
field_stats
based calculations. - stats_result – Either min_value or max_value. Only used in conjunction with source`=``field_stats` to choose whether to reference the minimum or maximum result value.
- epoch – An epoch timestamp used in conjunction with
unit
andunit_count
to establish a point of reference for calculations. If not provided, the current time will be used. - exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is False
-
filter_by_alias
(aliases=None, exclude=False)¶ Match indices which are associated with the alias identified by name
Parameters: - aliases (list) – A list of alias names.
- exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is False
-
filter_by_count
(count=None, reverse=True, use_age=False, source='creation_date', timestring=None, field=None, stats_result='min_value', exclude=True)¶ Remove indices from the actionable list beyond the number count, sorted reverse-alphabetically by default. If you set reverse to False, it will be sorted alphabetically.
The default is usually what you will want. If only one kind of index is provided–for example, indices matching
logstash-%Y.%m.%d
–then reverse alphabetical sorting will mean the oldest will remain in the list, because lower numbers in the dates mean older indices.By setting reverse to False, then
index3
will be deleted beforeindex2
, which will be deleted beforeindex1
use_age allows ordering indices by age. Age is determined by the index creation date by default, but you can specify an source of
name
,max_value
, ormin_value
. Thename
source requires the timestring argument.Parameters: - count – Filter indices beyond count.
- reverse – The filtering direction. (default: True).
- use_age – Sort indices by age.
source
is required in this case. - source – Source of index age. Can be one of
name
,creation_date
, orfield_stats
. Default:creation_date
- timestring – An strftime string to match the datestamp in an index
name. Only used if source
name
is selected. - field – A timestamp field name. Only used if source
field_stats
is selected. - stats_result – Either min_value or max_value. Only used if
source
field_stats
is selected. It determines whether to reference the minimum or maximum value of field in each index. - exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is True
-
filter_by_regex
(kind=None, value=None, exclude=False)¶ Match indices by regular expression (pattern).
Parameters: - kind – Can be one of:
suffix
,prefix
,regex
, ortimestring
. This option defines what kind of filter you will be building. - value – Depends on kind. It is the strftime string if kind is
timestring
. It’s used to build the regular expression for other kinds. - exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is False
- kind – Can be one of:
-
filter_by_space
(disk_space=None, reverse=True, use_age=False, source='creation_date', timestring=None, field=None, stats_result='min_value', exclude=False)¶ Remove indices from the actionable list based on space consumed, sorted reverse-alphabetically by default. If you set reverse to False, it will be sorted alphabetically.
The default is usually what you will want. If only one kind of index is provided–for example, indices matching
logstash-%Y.%m.%d
–then reverse alphabetical sorting will mean the oldest will remain in the list, because lower numbers in the dates mean older indices.By setting reverse to False, then
index3
will be deleted beforeindex2
, which will be deleted beforeindex1
use_age allows ordering indices by age. Age is determined by the index creation date by default, but you can specify an source of
name
,max_value
, ormin_value
. Thename
source requires the timestring argument.Parameters: - disk_space – Filter indices over n gigabytes
- reverse – The filtering direction. (default: True). Ignored if use_age is True
- use_age – Sort indices by age.
source
is required in this case. - source – Source of index age. Can be one of
name
,creation_date
, orfield_stats
. Default:creation_date
- timestring – An strftime string to match the datestamp in an index
name. Only used if source
name
is selected. - field – A timestamp field name. Only used if source
field_stats
is selected. - stats_result – Either min_value or max_value. Only used if
source
field_stats
is selected. It determines whether to reference the minimum or maximum value of field in each index. - exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is False
-
filter_closed
(exclude=True)¶ Filter out closed indices from indices
Parameters: exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is True
-
filter_forceMerged
(max_num_segments=None, exclude=True)¶ Match any index which has max_num_segments per shard or fewer in the actionable list.
Parameters: - max_num_segments – Cutoff number of segments per shard.
- exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is True
-
filter_kibana
(exclude=True)¶ Match any index named
.kibana
,kibana-int
,.marvel-kibana
, or.marvel-es-data
in indices.Parameters: exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is True
-
filter_opened
(exclude=True)¶ Filter out opened indices from indices
Parameters: exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is True
-
filter_period
(source='name', range_from=None, range_to=None, timestring=None, unit=None, field=None, stats_result='min_value', week_starts_on='sunday', epoch=None, exclude=False)¶ Match indices within ages within a given period.
Parameters: - source – Source of index age. Can be one of ‘name’, ‘creation_date’, or ‘field_stats’
- range_from – How many ``unit``s in the past/future is the origin?
- range_to – How many ``unit``s in the past/future is the end point?
- timestring – An strftime string to match the datestamp in an index
name. Only used for index filtering by
name
. - unit – One of
hours
,days
,weeks
,months
, oryears
. - unit_count – The number of
unit``s. ``unit_count
*unit
will be calculated out to the relative number of seconds. - field – A timestamp field name. Only used for
field_stats
based calculations. - stats_result – Either min_value or max_value. Only used in conjunction with source`=``field_stats` to choose whether to reference the minimum or maximum result value.
- week_starts_on – Either
sunday
ormonday
. Default issunday
- epoch – An epoch timestamp used to establish a point of reference for calculations. If not provided, the current time will be used.
- exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is False
-
index_info
= None¶ Instance variable. Information extracted from indices, such as segment count, age, etc. Populated at instance creation time, and by other private helper methods, as needed. Type:
dict()
-
indices
= None¶ Instance variable. The running list of indices which will be used by an Action class. Populated at instance creation time. Type:
list()
-
iterate_filters
(filter_dict)¶ Iterate over the filters defined in config and execute them.
Parameters: filter_dict – The configuration dictionary Note
filter_dict should be a dictionary with the following form:
{ 'filters' : [ { 'filtertype': 'the_filter_type', 'key1' : 'value1', ... 'keyN' : 'valueN' } ] }
-
working_list
()¶ Return the current value of indices as copy-by-value to prevent list stomping during iterations
-
SnapshotList¶
-
class
curator.snapshotlist.
SnapshotList
(client, repository=None)¶ -
client
= None¶ An Elasticsearch Client object. Also accessible as an instance variable.
-
empty_list_check
()¶ Raise exception if snapshots is empty
-
filter_by_age
(source='creation_date', direction=None, timestring=None, unit=None, unit_count=None, epoch=None, exclude=False)¶ Remove snapshots from snapshots by relative age calculations.
Parameters: - source – Source of snapshot age. Can be ‘name’, or ‘creation_date’.
- direction – Time to filter, either
older
oryounger
- timestring – An strftime string to match the datestamp in an
snapshot name. Only used for snapshot filtering by
name
. - unit – One of
seconds
,minutes
,hours
,days
,weeks
,months
, oryears
. - unit_count – The number of
unit``s. ``unit_count
*unit
will be calculated out to the relative number of seconds. - epoch – An epoch timestamp used in conjunction with
unit
andunit_count
to establish a point of reference for calculations. If not provided, the current time will be used. - exclude – If exclude is True, this filter will remove matching snapshots from snapshots. If exclude is False, then only matching snapshots will be kept in snapshots. Default is False
-
filter_by_count
(count=None, reverse=True, use_age=False, source='creation_date', timestring=None, exclude=True)¶ Remove snapshots from the actionable list beyond the number count, sorted reverse-alphabetically by default. If you set reverse to False, it will be sorted alphabetically.
The default is usually what you will want. If only one kind of snapshot is provided–for example, snapshots matching
curator-%Y%m%d%H%M%S
– then reverse alphabetical sorting will mean the oldest will remain in the list, because lower numbers in the dates mean older snapshots.By setting reverse to False, then
snapshot3
will be acted on beforesnapshot2
, which will be acted on beforesnapshot1
use_age allows ordering snapshots by age. Age is determined by the snapshot creation date (as identified by
start_time_in_millis
) by default, but you can also specify a source ofname
. Thename
source requires the timestring argument.Parameters: - count – Filter snapshots beyond count.
- reverse – The filtering direction. (default: True).
- use_age – Sort snapshots by age.
source
is required in this case. - source – Source of snapshot age. Can be one of
name
, orcreation_date
. Default:creation_date
- timestring – An strftime string to match the datestamp in a
snapshot name. Only used if source
name
is selected. - exclude – If exclude is True, this filter will remove matching snapshots from snapshots. If exclude is False, then only matching snapshots will be kept in snapshots. Default is True
-
filter_by_regex
(kind=None, value=None, exclude=False)¶ Filter out snapshots not matching the pattern, or in the case of exclude, filter those matching the pattern.
Parameters: - kind – Can be one of:
suffix
,prefix
,regex
, ortimestring
. This option defines what kind of filter you will be building. - value – Depends on kind. It is the strftime string if kind is timestring. It’s used to build the regular expression for other kinds.
- exclude – If exclude is True, this filter will remove matching snapshots from snapshots. If exclude is False, then only matching snapshots will be kept in snapshots. Default is False
- kind – Can be one of:
-
filter_by_state
(state=None, exclude=False)¶ Filter out snapshots not matching
state
, or in the case of exclude, filter those matchingstate
.Parameters: - state – The snapshot state to filter for. Must be one of
SUCCESS
,PARTIAL
,FAILED
, orIN_PROGRESS
. - exclude – If exclude is True, this filter will remove matching snapshots from snapshots. If exclude is False, then only matching snapshots will be kept in snapshots. Default is False
- state – The snapshot state to filter for. Must be one of
-
filter_period
(source='name', range_from=None, range_to=None, timestring=None, unit=None, field=None, stats_result='min_value', week_starts_on='sunday', epoch=None, exclude=False)¶ Match indices within ages within a given period.
Parameters: - source – Source of snapshot age. Can be ‘name’, or ‘creation_date’.
- range_from – How many ``unit``s in the past/future is the origin?
- range_to – How many ``unit``s in the past/future is the end point?
- timestring – An strftime string to match the datestamp in an
snapshot name. Only used for snapshot filtering by
name
. - unit – One of
hours
,days
,weeks
,months
, oryears
. - week_starts_on – Either
sunday
ormonday
. Default issunday
- epoch – An epoch timestamp used to establish a point of reference for calculations. If not provided, the current time will be used.
- exclude – If exclude is True, this filter will remove matching indices from indices. If exclude is False, then only matching indices will be kept in indices. Default is False
-
iterate_filters
(config)¶ Iterate over the filters defined in config and execute them.
Parameters: config – A dictionary of filters, as extracted from the YAML configuration file. Note
config should be a dictionary with the following form:
{ 'filters' : [ { 'filtertype': 'the_filter_type', 'key1' : 'value1', ... 'keyN' : 'valueN' } ] }
-
most_recent
()¶ Return the most recent snapshot based on start_time_in_millis.
-
repository
= None¶ An Elasticsearch repository. Also accessible as an instance variable.
-
snapshot_info
= None¶ Instance variable. Information extracted from snapshots, such as age, etc. Populated by internal method __get_snapshots at instance creation time. Type:
dict()
-
snapshots
= None¶ Instance variable. The running list of snapshots which will be used by an Action class. Populated by internal methods __get_snapshots at instance creation time. Type:
list()
-
working_list
()¶ Return the current value of snapshots as copy-by-value to prevent list stomping during iterations
-