A module for testing the logic of states and highstates
William Cannon <william.cannon@gmail.com>
new
Saltcheck provides unittest like functionality requiring only the knowledge of salt module execution and yaml. Saltcheck uses salt modules to return data, then runs an assertion against that return. This allows for testing with all the features included in salt modules.
In order to run state and highstate saltcheck tests, a sub-folder in the state directory
must be created and named saltcheck-tests
. Tests for a state should be created in files
ending in *.tst
and placed in the saltcheck-tests
folder. tst
files are run
through the salt rendering system, enabling tests to be written in yaml (or renderer of choice),
and include jinja, as well as the usual grain and pillar information. Like states, multiple tests can
be specified in a tst
file. Multiple tst
files can be created in the saltcheck-tests
folder, and should be named the same as the associated state. The id
of a test works in the
same manner as in salt state files and should be unique and descriptive.
New in version 3000: The saltcheck-tests
folder can be customized using the saltcheck_test_location
minion
configuration setting. This setting is a relative path from the formula's salt://
path
to the test files.
Example Default file system layout:
/srv/salt/apache/
init.sls
config.sls
saltcheck-tests/
init.tst
config.tst
deployment_validation.tst
Alternative example file system layout with custom saltcheck_test_location:
saltcheck_test_location: tests/integration/saltcheck
/srv/salt/apache/
init.sls
config.sls
tests/integration/saltcheck/
init.tst
config.tst
deployment_validation.tst
Tests can be run for each state by name, for all apache/saltcheck/*.tst
files, or for all states assigned to the minion in top.sls. Tests may also be
created with no associated state. These tests will be run through the use of
saltcheck.run_state_tests
, but will not be automatically run by
saltcheck.run_highstate_tests
.
salt '*' saltcheck.run_state_tests apache,apache.config
salt '*' saltcheck.run_state_tests apache check_all=True
salt '*' saltcheck.run_highstate_tests
salt '*' saltcheck.run_state_tests apache.deployment_validation
(str) This is the salt module which will be run locally,
the same as salt-call --local <module>
. The saltcheck.state_apply
module name is
special as it bypasses the local option in order to resolve state names when run in
a master/minion environment.
(list) Optional arguments passed to the salt module
(dict) Optional keyword arguments to be passed to the salt module
(str) One of the supported assertions and required except for saltcheck.state_apply
Tests which fail the assertion and expected_return, cause saltcheck to exit which a non-zero exit code.
(str) Required except by assertEmpty
, assertNotEmpty
, assertTrue
,
assertFalse
. The return of module_and_function is compared to this value in the assertion.
(str) Optional keyword used to parse the module_and_function return. If a salt module
returns a dictionary as a result, the assertion_section
value is used to lookup a specific value
in that return for the assertion comparison.
(str) Optional delimiter to use when splitting a nested structure. Defaults to ':'
(bool) Optional keyword to show results in the assertEqual
, assertNotEqual
,
assertIn
, and assertNotIn
output. Defaults to True.
(bool) Optional keyword to display module_and_function
, args
, assertion_section
,
and assertion results text in the output. If print_result is False, assertion results will be hidden.
This is a per test setting, but can be set globally for all tests by adding saltcheck_output_details: True
in the minion configuration file.
Defaults to False
(dict) Optional keyword for passing in pillar data. Intended for use in potential test
setup or teardown with the saltcheck.state_apply
function.
(bool) Optional keyword to skip running the individual test
echo_test_hello:
module_and_function: test.echo
args:
- "hello"
kwargs:
assertion: assertEqual
expected_return: 'hello'
{% for package in ["apache2", "openssh"] %}
{# or another example #}
{# for package in salt['pillar.get']("packages") #}
test_{{ package }}_latest:
module_and_function: pkg.upgrade_available
args:
- {{ package }}
assertion: assertFalse
{% endfor %}
setup_test_environment:
module_and_function: saltcheck.state_apply
args:
- common
pillar_data:
data: value
verify_vim:
module_and_function: pkg.version
args:
- vim
assertion: assertNotEmpty
{% for package in ["apache2", "openssh"] %}
{# or another example #}
{# for package in salt['pillar.get']("packages") #}
test_{{ package }}_latest:
module_and_function: pkg.upgrade_available
args:
- {{ package }}
assertion: assertFalse
{% endfor %}
setup_test_environment:
module_and_function: saltcheck.state_apply
args:
- common
pillar-data:
data: value
verify_vim:
module_and_function: pkg.version
args:
- vim
assertion: assertNotEmpty
package_latest:
module_and_function: pkg.upgrade_available
args:
- apache2
assertion: assertFalse
skip: True
validate_shell:
module_and_function: user.info
args:
- root
assertion: assertEqual
expected_return: /bin/bash
assertion_section: shell
validate_smb_signing:
module_and_function: lgpo.get
args:
- 'Machine'
kwargs:
return_full_policy_names: True
assertion: assertEqual
expected_return: Enabled
assertion_section: 'Computer Configuration|Microsoft network client: Digitally sign communications (always)'
assertion_section_delimiter: '|'
validate_env_nameNode:
module_and_function: hadoop.dfs
args:
- text
- /oozie/common/env.properties
expected_return: nameNode = hdfs://nameservice2
assertion: assertNotIn
print_result: False
assertEqual
assertNotEqual
assertTrue
assertFalse
assertIn
assertNotIn
assertGreater
assertGreaterEqual
assertLess
assertLessEqual
assertEmpty
assertNotEmpty
Warning
The saltcheck.state_apply function is an alias for
state.apply
. If using the
ACL system saltcheck.*
might provide more capability
than intended if only saltcheck.run_state_tests
and
saltcheck.run_highstate_tests
are needed.
salt.modules.saltcheck.
report_highstate_tests
(saltenv=None)¶Report on tests for states assigned to the minion through highstate. Quits with the exit code for the number of missing tests.
CLI Example:
salt '*' saltcheck.report_highstate_tests
New in version 3000.
salt.modules.saltcheck.
run_highstate_tests
(saltenv=None)¶Execute all tests for states assigned to the minion through highstate and return results
CLI Example:
salt '*' saltcheck.run_highstate_tests
salt.modules.saltcheck.
run_state_tests
(state, saltenv=None, check_all=False)¶Execute tests for a salt state and return results Nested states will also be tested
CLI Example:
salt '*' saltcheck.run_state_tests postfix,common
salt.modules.saltcheck.
run_state_tests_ssh
(state, saltenv=None, check_all=False)¶This function is an alias of run_state_tests
.
Execute tests for a salt state and return results Nested states will also be tested
- param str state
state name for which to run associated .tst test files
- param str saltenv
optional saltenv. Defaults to base
- param bool check_all
boolean to run all tests in state/saltcheck-tests directory
CLI Example:
salt '*' saltcheck.run_state_tests postfix,common
salt.modules.saltcheck.
run_test
(**kwargs)¶Execute one saltcheck test and return result
arg test (keyword) --
CLI Example:
salt '*' saltcheck.run_test
test='{"module_and_function": "test.echo",
"assertion": "assertEqual",
"expected_return": "This works!",
"args":["This works!"] }'
salt.modules.saltcheck.
state_apply
(state_name, **kwargs)¶Runs state.apply
with given options to set up test data.
Intended to be used for optional test setup or teardown
Reference the state.apply
module documentation for arguments and usage options
CLI Example:
salt '*' saltcheck.state_apply postfix