A Python-based DSL
Jack Kuan <kjkuan@gmail.com>
new
all
The pydsl renderer allows one to author salt formulas (.sls files) in pure Python using a DSL that's easy to write and easy to read. Here's an example:
1#!pydsl
2
3apache = state('apache')
4apache.pkg.installed()
5apache.service.running()
6state('/var/www/index.html') \
7 .file('managed',
8 source='salt://webserver/index.html') \
9 .require(pkg='apache')
Notice that any Python code is allow in the file as it's really a Python
module, so you have the full power of Python at your disposal. In this module,
a few objects are defined for you, including the usual (with __
added)
__salt__
dictionary, __grains__
, __pillar__
, __opts__
,
__env__
, and __sls__
, plus a few more:
__file__
local file system path to the sls module.
__pydsl__
Salt PyDSL object, useful for configuring DSL behavior per sls rendering.
include
Salt PyDSL function for creating Include declaration's.
extend
Salt PyDSL function for creating Extend declaration's.
state
Salt PyDSL function for creating ID declaration's.
A state ID declaration is created with a state(id)
function call.
Subsequent state(id)
call with the same id returns the same object. This
singleton access pattern applies to all declaration objects created with the
DSL.
state('example')
assert state('example') is state('example')
assert state('example').cmd is state('example').cmd
assert state('example').cmd.running is state('example').cmd.running
The id argument is optional. If omitted, an UUID will be generated and used as the id.
state(id)
returns an object under which you can create a
State declaration object by accessing an attribute named after any
state module available in Salt.
state('example').cmd
state('example').file
state('example').pkg
...
Then, a Function declaration object can be created from a State declaration object by one of the following two ways:
by calling a method named after the state function on the State declaration object.
state('example').file.managed(...)
by directly calling the attribute named for the State declaration, and supplying the state function name as the first argument.
state('example').file('managed', ...)
With either way of creating a Function declaration object, any Function arg declaration's can be passed as keyword arguments to the call. Subsequent calls of a Function declaration will update the arg declarations.
state('example').file('managed', source='salt://webserver/index.html')
state('example').file.managed(source='salt://webserver/index.html')
As a shortcut, the special name argument can also be passed as the first or second positional argument depending on the first or second way of calling the State declaration object. In the following two examples ls -la is the name argument.
state('example').cmd.run('ls -la', cwd='/')
state('example').cmd('run', 'ls -la', cwd='/')
Finally, a Requisite declaration object with its Requisite reference's can be created by invoking one of the requisite methods (see State Requisites) on either a Function declaration object or a State declaration object. The return value of a requisite call is also a Function declaration object, so you can chain several requisite calls together.
Arguments to a requisite call can be a list of State declaration objects and/or a set of keyword arguments whose names are state modules and values are IDs of ID declaration's or names of Name declaration's.
apache2 = state('apache2')
apache2.pkg.installed()
state('libapache2-mod-wsgi').pkg.installed()
# you can call requisites on function declaration
apache2.service.running() \
.require(apache2.pkg,
pkg='libapache2-mod-wsgi') \
.watch(file='/etc/apache2/httpd.conf')
# or you can call requisites on state declaration.
# this actually creates an anonymous function declaration object
# to add the requisites.
apache2.service.require(state('libapache2-mod-wsgi').pkg,
pkg='apache2') \
.watch(file='/etc/apache2/httpd.conf')
# we still need to set the name of the function declaration.
apache2.service.running()
Include declaration objects can be created with the include
function,
while Extend declaration objects can be created with the extend
function,
whose arguments are just Function declaration objects.
include('edit.vim', 'http.server')
extend(state('apache2').service.watch(file='/etc/httpd/httpd.conf')
The include
function, by default, causes the included sls file to be rendered
as soon as the include
function is called. It returns a list of rendered module
objects; sls files not rendered with the pydsl renderer return None
's.
This behavior creates no Include declaration's in the resulting high state
data structure.
import types
# including multiple sls returns a list.
_, mod = include('a-non-pydsl-sls', 'a-pydsl-sls')
assert _ is None
assert isinstance(slsmods[1], types.ModuleType)
# including a single sls returns a single object
mod = include('a-pydsl-sls')
# myfunc is a function that calls state(...) to create more states.
mod.myfunc(1, 2, "three")
Notice how you can define a reusable function in your pydsl sls module and then
call it via the module returned by include
.
It's still possible to do late includes by passing the delayed=True
keyword
argument to include
.
include('edit.vim', 'http.server', delayed=True)
Above will just create a Include declaration in the rendered result, and
such call always returns None
.
Taking advantage of rendering a Python module, PyDSL allows you to declare a state that calls a pre-defined Python function when the state is executed.
greeting = "hello world"
def helper(something, *args, **kws):
print greeting # hello world
print something, args, kws # test123 ['a', 'b', 'c'] {'x': 1, 'y': 2}
state().cmd.call(helper, "test123", 'a', 'b', 'c', x=1, y=2)
The cmd.call state function takes care of calling our helper
function
with the arguments we specified in the states, and translates the return value
of our function into a structure expected by the state system.
See salt.states.cmd.call()
for more information.
Salt states are explicitly ordered via Requisite declaration's.
However, with pydsl it's possible to let the renderer track the order
of creation for Function declaration objects, and implicitly add
require
requisites for your states to enforce the ordering. This feature
is enabled by setting the ordered
option on __pydsl__
.
Note
this feature is only available if your minions are using Python >= 2.7.
include('some.sls.file')
A = state('A').cmd.run(cwd='/var/tmp')
extend(A)
__pydsl__.set(ordered=True)
for i in range(10):
i = str(i)
state(i).cmd.run('echo '+i, cwd='/')
state('1').cmd.run('echo one')
state('2').cmd.run(name='echo two')
Notice that the ordered
option needs to be set after any extend
calls.
This is to prevent pydsl from tracking the creation of a state function that's
passed to an extend
call.
Above example should create states from 0
to 9
that will output 0
,
one
, two
, 3
, ... 9
, in that order.
It's important to know that pydsl tracks the creations of
Function declaration objects, and automatically adds a require
requisite
to a Function declaration object that requires the last
Function declaration object created before it in the sls file.
This means later calls (perhaps to update the function's Function arg declaration) to a previously created function declaration will not change the order.
When Salt processes a salt formula file, the file is rendered to salt's high state data representation by a renderer before the states can be executed. In the case of the pydsl renderer, the .sls file is executed as a python module as it is being rendered which makes it easy to execute a state at render time. In pydsl, executing one or more states at render time can be done by calling a configured ID declaration object.
#!pydsl
s = state() # save for later invocation
# configure it
s.cmd.run('echo at render time', cwd='/')
s.file.managed('target.txt', source='salt://source.txt')
s() # execute the two states now
Once an ID declaration is called at render time it is detached from the sls module as if it was never defined.
Note
If implicit ordering is enabled (i.e., via __pydsl__.set(ordered=True)
) then
the first invocation of a ID declaration object must be done before a
new Function declaration is created.
The salt.renderers.stateconf
renderer offers a few interesting features that
can be leveraged by the pydsl renderer. In particular, when using with the pydsl
renderer, we are interested in stateconf's sls namespacing feature (via dot-prefixed
id declarations), as well as, the automatic start and goal states generation.
Now you can use pydsl with stateconf like this:
#!pydsl|stateconf -ps
include('xxx', 'yyy')
# ensure that states in xxx run BEFORE states in this file.
extend(state('.start').stateconf.require(stateconf='xxx::goal'))
# ensure that states in yyy run AFTER states in this file.
extend(state('.goal').stateconf.require_in(stateconf='yyy::start'))
__pydsl__.set(ordered=True)
...
-s
enables the generation of a stateconf start state, and -p
lets us pipe
high state data rendered by pydsl to stateconf. This example shows that by
require
-ing or require_in
-ing the included sls' start or goal states,
it's possible to ensure that the included sls files can be made to execute before
or after a state in the including sls file.
To use a custom Python module inside a PyDSL state, place the module somewhere that it can be loaded by the Salt loader, such as _modules in the /srv/salt directory.
Then, copy it to any minions as necessary by using saltutil.sync_modules.
To import into a PyDSL SLS, one must bypass the Python importer and insert it manually by getting a reference from Python's sys.modules dictionary.
For example:
#!pydsl|stateconf -ps
def main():
my_mod = sys.modules['salt.loaded.ext.module.my_mod']
Used when a renderer needs to raise an explicit error. If a line number and buffer string are passed, get_context will be invoked to get the location of the error.