.. _testing: Testing ####### Ensuring that code changes work with new Elasticsearch versions, ``elasticsearch-py`` Python module versions, and even new Python versions can be daunting. I've tried to make it easy to verify that changes will work. Setup Testing ************* Since ``nose`` testing basically died somewhere during Curator's early days, a new testing framework has become necessary. This is where ``pytest`` comes in. Install ``pytest`` ================== From where your ``git`` cloned or forked repository is, you need to install Curator and its dependencies, including for testing: .. code-block:: shell pip install -U '.[test]' Manually install testing dependencies ------------------------------------- These are indicated in ``pyproject.toml`` in the ``[project.optional-dependencies]`` subsection. An example is listed below: .. code-block:: [project.optional-dependencies] test = [ "requests", "pytest >=7.2.1", "pytest-cov", ] These may change with time, and this document not be updated, so double check dependencies here before running the following: .. code-block:: shell pip install -U requests pytest pytest-cov It should be simpler to run the regular method, but if you have some reason to do this manually, those are the steps. Elasticsearch as a testing dependency ===================================== .. warning:: Not using a dedicated instance or instances for testing will result in deleted data! The tests perform setup and teardown functions which will delete anything in your cluster between each test. .. important:: Integration tests will at least require Elasticsearch running on ``http://127.0.0.1:9200`` or ``TEST_ES_SERVER`` being set. The few tests that require a remote cluster to be configured will need ``REMOTE_ES_SERVER`` to be set as well. I will not cover how to install Elasticsearch locally in this document. It can be done, but it is much easier to use Docker containers instead. If you host a dedicated instance somewhere else (and it must be unsecured for testing), you can specify this as an environment variable: .. code-block:: shell TEST_ES_SERVER="http://10.0.0.1:9201" \ pytest --cov=curator --cov-report html:cov_html Additionally, four tests will be skipped if no value for ``REMOTE_ES_SERVER`` is provided. .. code-block:: shell TEST_ES_SERVER="http://10.0.0.1:9201" \ REMOTE_ES_SERVER="http://10.0.0.2:9201" \ pytest --cov=curator --cov-report html:cov_html The ``REMOTE_ES_SERVER`` must be a separate instance altogether, and the main instance must whitelist that instance for reindexing operations. If that sounds complicated, you're not wrong. There are remedies for this, and Curator comes with the necessary tools Using Docker ------------ Fortunately, Curator provides an out-of-the-box, ready to go set of scripts for setting up not only one container, but both containers necessary for testing the remote reindex functionality. .. warning:: Do not use anything but ``create.sh`` and ``destroy.sh``, or edit the ``Dockerfile.tmpl`` or ``small.options`` files unless you're actively trying to improve these scripts. These keep the Elasticsearch containers lean. Do examine ``create.sh`` to see which Elasticsearch startup flags are being used. Create Docker containers for testing ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Replace ``X.Y.Z`` with an Elasticsearch version: .. code-block:: shell $ cd /path/to/curator_code/docker_test/scripts $ ./create.sh X.Y.Z Docker image curator_estest:8.6.1 not found. Building from Dockerfile... ... Waiting for Elasticsearch instances to become available... This will create both Docker containers, and will print out the ``REMOTE_ES_SERVER`` line to use: .. code-block:: shell Please select one of these environment variables to prepend your 'pytest' run: REMOTE_ES_SERVER="http://10.0.0.2:9201" Clean up Docker containers used for testing ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. note:: The container names ``curator8-es-local`` and ``curator8-es-remote`` are hard coded in both scripts so that ``destroy.sh`` will clean up exactly what ``create.sh`` made. .. code-block:: shell $ cd /path/to/curator_code/docker_test/scripts $ ./destroy.sh curator8-es-local curator8-es-remote curator8-es-local curator8-es-remote Cleanup complete. The ``repo`` directory ^^^^^^^^^^^^^^^^^^^^^^ ``/path/to/curator_code/docker_test/repo`` will be created by ``create.sh`` and deleted by ``destroy.sh``. This is used for snapshot testing and will only ever contain a few files. Anything snapshotted there temporarily is cleaned by the ``teardown`` between tests. Running Tests ************* Using ``pytest`` ================ Using the value of ``REMOTE_ES_SERVER`` you got from ``create.sh``, or your own "remote" Elasticsearch instance, testing is as simple as running: .. note:: All of these examples presume that you are at the base directory of Curator's code such that the ``tests`` direcory is visible. .. code-block:: shell REMOTE_ES_SERVER="http://10.0.0.2:9201" pytest Generating coverage reports --------------------------- .. code-block:: shell $ REMOTE_ES_SERVER="http://10.0.0.2:9201" pytest --cov=curator ............................................................................ [ 12%] ............................................................................ [ 24%] ............................................................................ [ 36%] ............................................................................ [ 48%] ............................................................................ [ 60%] ............................................................................ [ 72%] ............................................................................ [ 84%] ............................................................................ [ 96%] ........................ [100%] ---------- coverage: platform darwin, python 3.11.1-final-0 ---------- Name Stmts Miss Cover ------------------------------------------------------------ curator/__init__.py 10 0 100% curator/_version.py 1 0 100% curator/actions/__init__.py 14 0 100% ... curator/validators/schemacheck.py 42 0 100% ------------------------------------------------------------ TOTAL 4023 1018 75% 475 passed in 4.92s Generating an HTML coverage report ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: shell $ REMOTE_ES_SERVER="http://10.0.0.2:9201" pytest --cov=curator --cov-reporthtml:cov_html ............................................................................ [ 12%] ............................................................................ [ 24%] ............................................................................ [ 36%] ............................................................................ [ 48%] ............................................................................ [ 60%] ............................................................................ [ 72%] ............................................................................ [ 84%] ............................................................................ [ 96%] ........................ [100%] ---------- coverage: platform darwin, python 3.11.1-final-0 ---------- Coverage HTML written to dir cov_html 475 passed in 5.24s At this point, you can view ``/path/to/curator_code/cov_html/index.html`` in your web browser. On macOS, this is as simple as running: .. code-block:: shell $ open cov_html.index.html It will open in your default browser. Testing only unit tests ----------------------- As unit tests do not require a remote Elasticsearch instance, adding the ``REMOTE_ES_SERVER`` environment variable is unnecessary: .. code-block:: shell $ pytest tests/unit You can also add ``--cov=curator`` and/or ``--cov=curator html:cov_html`` options. Testing only integration tests ------------------------------ Most integration tests do not require a remote Elasticsearch instance, so adding the ``REMOTE_ES_SERVER`` environment variable is unnecessary. Having a functional instance of Elasticsearch at ``http://127.0.0.1:9200`` or the ``TEST_ES_SERVER`` environment variable set is required. .. code-block:: shell $ pytest tests/integration You can also add ``--cov=curator`` and/or ``--cov=curator html:cov_html`` options. This will result in 4 skipped tests: .. code-block:: shell $ pytest tests/integration .......................................................................... [ 47%] ...............................s.s...ss................................... [ 94%] ......... [100%] ============================ short test summary info ============================= SKIPPED [1] tests/integration/test_reindex.py:110: REMOTE_ES_SERVER is not defined SKIPPED [1] tests/integration/test_reindex.py:275: REMOTE_ES_SERVER is not defined SKIPPED [1] tests/integration/test_reindex.py:157: REMOTE_ES_SERVER is not defined SKIPPED [1] tests/integration/test_reindex.py:206: REMOTE_ES_SERVER is not defined 153 passed, 4 skipped, 7 warnings in 217.76s (0:03:37) You can see the ``s`` in the test output. The message for each skipped test also clearly explains that ``REMOTE_ES_SERVER`` is undefined. If you were to run this with ``REMOTE_ES_SERVER``, it would clear up the skipped tests. Running specific tests ---------------------- These examples are all derived from unit tests, but the same formatting applies to integration tests as well. The path for those will just be ``tests/integration/test_file.py``. .. important:: Integration tests will at least require Elasticsearch running on ``http://127.0.0.1:9200`` or ``TEST_ES_SERVER`` being set. The few tests that require a remote cluster to be configured will need ``REMOTE_ES_SERVER`` to be set as well. Testing all tests within a given file ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This will test every method of every class in ``test_file.py`` .. code-block:: shell $ pytest tests/unit/test_file.py ................................................... [100%] 51 passed in 0.32s Testing all tests within a given class ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This will test every method of class ``TestClass`` in ``test_file.py`` .. code-block:: shell $ pytest tests/unit/test_file.py::TestClass .............. [100%] 14 passed in 0.35s Testing one test within a given class ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This will test method ``test_method`` of class ``TestClass`` in ``test_file.py`` .. code-block:: shell $ pytest tests/unit/test_file.py::TestClass::test_method . [100%] 1 passed in 0.31s