Index List¶
Index filtering is performed using the filter_
methods of curator.IndexList
-
class
curator.
IndexList
(client)¶ Bases:
object
IndexList class
-
client
= None¶ An
Elasticsearch
client object passed from paramclient
-
filter_allocated
(key=None, value=None, allocation_type='require', exclude=True)¶ Match indices that have the routing allocation rule of
key=value
fromindices
Parameters: - key – The allocation attribute to check for
- value – The value to check for
- allocation_type – Type of allocation to apply
- exclude – If
exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default is T`rue
-
filter_by_age
(source='name', direction=None, timestring=None, unit=None, unit_count=None, field=None, stats_result='min_value', epoch=None, exclude=False, unit_count_pattern=False)¶ Match indices by relative age calculations.
Parameters: - source – Source of index age. Can be one of
name
,creation_date
, orfield_stats
- direction – Time to filter, either
older
oryounger
- timestring – An
time.strftime()
string to match the datestamp in an index name. Only used for index filtering byname
. - unit – One of
seconds
,minutes
,hours
,days
,weeks
,months
, oryears
. - unit_count – The count of
unit
.unit_count
*unit
will be calculated out to the relative number of seconds. - unit_count_pattern – A regular expression whose capture group identifies the value
for
unit_count
. - field – A timestamp field name. Only used for
field_stats
based calculations. - stats_result – Either
min_value
ormax_value
. Only used in conjunction withsource=field_stats
to choose whether to reference the minimum or maximum result value. - epoch – An epoch timestamp used in conjunction with
unit
andunit_count
to establish a point of reference for calculations. If not provided, the current time will be used. - exclude – If
exclude=True
, this filter will remove matching indices fromindices
. Ifexclude
is False, then only matching indices will be kept inindices
. Default isFalse
- source – Source of index age. Can be one of
-
filter_by_alias
(aliases=None, exclude=False)¶ Match indices which are associated with the alias or list of aliases identified by
aliases
. Indices must appear in all aliases in listaliases
or a 404 error will result, leading to no indices being matched.Parameters: - aliases (list) – A list of alias names.
- exclude – If
exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isFalse
-
filter_by_count
(count=None, reverse=True, use_age=False, pattern=None, source='creation_date', timestring=None, field=None, stats_result='min_value', exclude=True)¶ Remove indices from the actionable list beyond the number
count
, sorted reverse-alphabetically by default. If you setreverse=False
, it will be sorted alphabetically.The default is usually what you will want. If only one kind of index is provided–for example, indices matching
logstash-%Y.%m.%d
– then reverse alphabetical sorting will mean the oldest will remain in the list, because lower numbers in the dates mean older indices.By setting
reverse=False
, thenindex3
will be deleted beforeindex2
, which will be deleted beforeindex1
use_age
allows ordering indices by age. Age is determined by the index creation date by default, but you can specify ansource
ofname
,max_value
, ormin_value
. Thename
source requires the timestring argument.Parameters: - count – Filter indices beyond
count
. - reverse – The filtering direction. (default:
True
). - use_age – Sort indices by age.
source
is required in this case. - pattern – Select indices to count from a regular expression pattern. This pattern
must have one and only one capture group. This can allow a single
count
filter instance to operate against any number of matching patterns, and keepcount
of each index in that group. For example, given apattern
of'^(.*)-\d{6}$'
, it will match bothrollover-000001
andindex-999990
, but notlogstash-2017.10.12
. Following the same example, if my cluster also hadrollover-000002
throughrollover-000010
andindex-888888
throughindex-999999
, it will process both groups of indices, and include or exclude thecount
of each. - source – Source of index age. Can be one of
name
,creation_date
, orfield_stats
. Default:creation_date
- timestring – An
time.strftime()
string to match the datestamp in an index name. Only used ifsource=name
. - field – A timestamp field name. Only used if
source=field_stats
. - stats_result – Either
min_value
ormax_value
. Only used ifsource=field_stats
. It determines whether to reference the minimum or maximum value offield
in each index. - exclude – If
exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isTrue
- count – Filter indices beyond
-
filter_by_regex
(kind=None, value=None, exclude=False)¶ Match indices by regular expression (pattern).
Parameters: - kind – Can be one of:
suffix
,prefix
,regex
, ortimestring
. This option defines whatkind
of filter you will be building. - value – Depends on
kind
. It is thetime.strftime()
string ifkind
istimestring
. It’s used to build the regular expression for other kinds. - exclude – If
exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isFalse
- kind – Can be one of:
-
filter_by_shards
(number_of_shards=None, shard_filter_behavior='greater_than', exclude=False)¶ Match
indices
with a given shard count.Selects all indices with a shard count
greater_than
number_of_shards
by default. Useshard_filter_behavior
to select indices with shard countgreater_than
,greater_than_or_equal
,less_than
,less_than_or_equal
, orequal
tonumber_of_shards
.Parameters: - number_of_shards – shard threshold
- shard_filter_behavior – Do you want to filter on
greater_than
,greater_than_or_equal
,less_than
,less_than_or_equal
, orequal
? - exclude – If
exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isFalse
-
filter_by_size
(size_threshold=None, threshold_behavior='greater_than', exclude=False, size_behavior='primary')¶ Remove indices from the actionable list based on index size.
threshold_behavior
, when set togreater_than
(default), includes if it the index tests to be larger thansize_threshold
. When set toless_than
, it includes if the index is smaller thansize_threshold
Parameters: - size_threshold – Filter indices over n gigabytes
- threshold_behavior – Size to filter, either
greater_than
orless_than
. Defaults togreater_than
to preserve backwards compatability. - size_behavior – Size that used to filter, either
primary
ortotal
. Defaults toprimary
- exclude – If
exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isFalse
-
filter_by_space
(disk_space=None, reverse=True, use_age=False, source='creation_date', timestring=None, field=None, stats_result='min_value', exclude=False, threshold_behavior='greater_than')¶ Remove indices from the actionable list based on space consumed, sorted reverse-alphabetically by default. If you set
reverse
toFalse
, it will be sorted alphabetically.The default is usually what you will want. If only one kind of index is provided–for example, indices matching
logstash-%Y.%m.%d
–then reverse alphabetical sorting will mean the oldest will remain in the list, because lower numbers in the dates mean older indices.By setting
reverse
toFalse
, thenindex3
will be deleted beforeindex2
, which will be deleted beforeindex1
use_age
allows ordering indices by age. Age is determined by the index creation date by default, but you can specify ansource
ofname
,max_value
, ormin_value
. Thename
source
requires the timestring argument.threshold_behavior
, when set togreater_than
(default), includes if it the index tests to be larger thandisk_space
. When set toless_than
, it includes if the index is smaller thandisk_space
Parameters: - disk_space – Filter indices over n gigabytes
- threshold_behavior – Size to filter, either
greater_than
orless_than
. Defaults togreater_than
to preserve backwards compatability. - reverse – The filtering direction. (default:
True
). Ignored ifuse_age
isTrue
- use_age – Sort indices by age.
source
is required in this case. - source – Source of index age. Can be one of
name
,creation_date
, orfield_stats
. Default:creation_date
- timestring – An
time.strftime()
string to match the datestamp in an index name. Only used ifsource=name
is selected. - field – A timestamp field name. Only used if
source=field_stats
is selected. - stats_result – Either
min_value
ormax_value
. Only used ifsource=field_stats
is selected. It determines whether to reference the minimum or maximum value of field in each index. - exclude – If
exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isFalse
-
filter_closed
(exclude=True)¶ Filter out closed indices from
indices
Parameters: exclude – If exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isTrue
-
filter_empty
(exclude=True)¶ Filter indices with a document count of zero. Indices that are closed are automatically excluded from consideration due to closed indices reporting a document count of zero.
Parameters: exclude – If exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isTrue
-
filter_forceMerged
(max_num_segments=None, exclude=True)¶ Match any index which has
max_num_segments
per shard or fewer in the actionable list.Parameters: - max_num_segments – Cutoff number of segments per shard.
- exclude – If
exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isTrue
-
filter_ilm
(exclude=True)¶ Match indices that have the setting
index.lifecycle.name
Parameters: exclude – If exclude=True
, this filter will remove matchingindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isTrue
-
filter_kibana
(exclude=True)¶ Match any index named
.kibana*
inindices
. Older releases addressed index names that no longer exist.Parameters: exclude – If exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isTrue
-
filter_none
()¶ The legendary NULL filter
-
filter_opened
(exclude=True)¶ Filter out opened indices from
indices
Parameters: exclude – If exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isTrue
-
filter_period
(period_type='relative', source='name', range_from=None, range_to=None, date_from=None, date_to=None, date_from_format=None, date_to_format=None, timestring=None, unit=None, field=None, stats_result='min_value', intersect=False, week_starts_on='sunday', epoch=None, exclude=False)¶ Match
indices
with ages within a given period.Parameters: - period_type – Can be either
absolute
orrelative
. Default isrelative
.date_from
anddate_to
are required when usingperiod_type='absolute'
.range_from
andrange_to
are required withperiod_type='relative'
. - source – Source of index age. Can be
name
,creation_date
, orfield_stats
- range_from – How many
unit
(s) in the past/future is the origin? - range_to – How many
unit
(s) in the past/future is the end point? - date_from – The simplified date for the start of the range
- date_to – The simplified date for the end of the range. If this value is the same as
date_from
, the full value ofunit
will be extrapolated for the range. For example, ifunit=months
, anddate_from
anddate_to
are both2017.01
, then the entire month of January 2017 will be the absolute date range. - date_from_format – The
time.strftime()
string used to parsedate_from
- date_to_format – The
time.strftime()
string used to parsedate_to
- timestring – An
time.strftime()
string to match the datestamp in an index name. Only used for index filtering byname
. - unit – One of
hours
,days
,weeks
,months
, oryears
. - field – A timestamp field name. Only used for
field_stats
based calculations. - stats_result – Either
min_value
ormax_value
. Only used in conjunction withsource='field_stats'
to choose whether to reference the min or max result value. - intersect – Only used when
source='field_stats'
. IfTrue
, only indices where bothmin_value
andmax_value
are within the period will be selected. IfFalse
, it will use whichever you specified. Default isFalse
to preserve expected behavior. - week_starts_on – Either
sunday
ormonday
. Default issunday
- epoch – An epoch timestamp used to establish a point of reference for calculations. If not provided, the current time will be used.
- exclude – If
exclude=True
, this filter will remove matching indices fromindices
. Ifexclude=False
, then only matching indices will be kept inindices
. Default isFalse
- period_type – Can be either
-
index_info
= None¶ Information extracted from indices, such as segment count, age, etc. Populated at instance creation time by private helper methods. Type:
dict
-
indices
= None¶ The running list of indices which will be used by one of the
actions
classes. Populated at instance creation time by private helper methods. Type:list
-
iterate_filters
(filter_dict)¶ Iterate over the filters defined in
config
and execute them.Parameters: filter_dict – The configuration dictionary Note
filter_dict
should be a dictionary with the following form:{ 'filters' : [ { 'filtertype': 'the_filter_type', 'key1' : 'value1', ... 'keyN' : 'valueN' } ] }
-
working_list
()¶ Return the current value of
indices
as copy-by-value to prevent list stomping during iterations
-