Returners¶

Returners Are Easy To Write!¶

Writing a Salt returner is straightforward.

A returner is a Python module containing at minimum a returner function. Other optional functions can be included to add support for master_job_cache, Storing Job Results in an External System, and Event Returners.

returner

The returner function must accept a single argument. The argument contains return data from the called minion function. If the minion function test.version is called, the value of the argument will be a dictionary. Run the following command from a Salt master to get a sample of the dictionary:

salt-call --local --metadata test.version --out=pprint

import redis
import salt.utils.json


def returner(ret):
    """
    Return information to a redis server
    """
    # Get a redis connection
    serv = redis.Redis(host="redis-serv.example.com", port=6379, db="0")
    serv.sadd("%(id)s:jobs" % ret, ret["jid"])
    serv.set("%(jid)s:%(id)s" % ret, salt.utils.json.dumps(ret["return"]))
    serv.sadd("jobs", ret["jid"])
    serv.sadd(ret["jid"], ret["id"])

The above example of a returner set to send the data to a Redis server serializes the data as JSON and sets it in redis.

Using Custom Returner Modules¶

Place custom returners in a _returners/ directory within the file_roots specified by the master config file.

Like all custom modules, these must be synced to the relevant master or minion before they can be used. See Modular Systems for details.

Any custom returners which have been synced to a minion that are named the same as one of Salt's default set of returners will take the place of the default returner with the same name.

Naming the Returner¶

Note that a returner's default name is its filename (i.e. foo.py becomes returner foo), but that its name can be overridden by using a __virtual__ function. A good example of this can be found in the redis returner, which is named redis_return.py but is loaded as simply redis:

try:
    import redis

    HAS_REDIS = True
except ImportError:
    HAS_REDIS = False

__virtualname__ = "redis"


def __virtual__():
    if not HAS_REDIS:
        return False
    return __virtualname__

Master Job Cache Support¶

master_job_cache, Storing Job Results in an External System, and Event Returners. Salt's master_job_cache allows returners to be used as a pluggable replacement for the Default Job Cache. In order to do so, a returner must implement the following functions:

prep_jid

Ensures that job ids (jid) don't collide, unless passed_jid is provided.

nocache is an optional boolean that indicates if return data should be cached. passed_jid is a caller provided jid which should be returned unconditionally.

def prep_jid(nocache, passed_jid=None):  # pylint: disable=unused-argument
    """
    Do any work necessary to prepare a JID, including sending a custom id
    """
    return passed_jid if passed_jid is not None else salt.utils.jid.gen_jid()

save_load

Save job information. The jid is generated by prep_jid and should be considered a unique identifier for the job. The jid, for example, could be used as the primary/unique key in a database. The load is what is returned to a Salt master by a minion. minions is a list of minions that the job was run against. The following code example stores the load as a JSON string in the salt.jids table.

import salt.utils.json


def save_load(jid, load, minions=None):
    """
    Save the load to the specified jid id
    """
    query = """INSERT INTO salt.jids (
                 jid, load
               ) VALUES (
                 '{0}', '{1}'
               );""".format(
        jid, salt.utils.json.dumps(load)
    )

    # cassandra_cql.cql_query may raise a CommandExecutionError
    try:
        __salt__["cassandra_cql.cql_query"](query)
    except CommandExecutionError:
        log.critical("Could not save load in jids table.")
        raise
    except Exception as e:
        log.critical("Unexpected error while inserting into jids: {0}".format(e))
        raise

get_load

must accept a job id (jid) and return the job load stored by save_load, or an empty dictionary when not found.

def get_load(jid):
    """
    Return the load data that marks a specified jid
    """
    query = """SELECT load FROM salt.jids WHERE jid = '{0}';""".format(jid)

    ret = {}

    # cassandra_cql.cql_query may raise a CommandExecutionError
    try:
        data = __salt__["cassandra_cql.cql_query"](query)
        if data:
            load = data[0].get("load")
            if load:
                ret = json.loads(load)
    except CommandExecutionError:
        log.critical("Could not get load from jids table.")
        raise
    except Exception as e:
        log.critical(
            """Unexpected error while getting load from
         jids: {0}""".format(
                str(e)
            )
        )
        raise

    return ret

External Job Cache Support¶

Salt's Storing Job Results in an External System extends the master_job_cache. External Job Cache support requires the following functions in addition to what is required for Master Job Cache support:

get_jid

Return a dictionary containing the information (load) returned by each minion when the specified job id was executed.

Sample:

{
    "local": {
        "master_minion": {
            "fun_args": [],
            "jid": "20150330121011408195",
            "return": "2018.3.4",
            "retcode": 0,
            "success": true,
            "cmd": "_return",
            "_stamp": "2015-03-30T12:10:12.708663",
            "fun": "test.version",
            "id": "master_minion"
        }
    }
}

get_fun

Return a dictionary of minions that called a given Salt function as their last function call.

Sample:

{
    "local": {
        "minion1": "test.version",
        "minion3": "test.version",
        "minion2": "test.version"
    }
}

get_jids

Return a list of all job ids.

Sample:

{
    "local": [
        "20150330121011408195",
        "20150330195922139916"
    ]
}

get_minions

Returns a list of minions

Sample:

{
     "local": [
         "minion3",
         "minion2",
         "minion1",
         "master_minion"
     ]
}

Please refer to one or more of the existing returners (i.e. mysql, cassandra_cql) if you need further clarification.

Event Support¶

An event_return function must be added to the returner module to allow events to be logged from a master via the returner. A list of events are passed to the function by the master.

The following example was taken from the MySQL returner. In this example, each event is inserted into the salt_events table keyed on the event tag. The tag contains the jid and therefore is guaranteed to be unique.

import salt.utils.json


def event_return(events):
    """
    Return event to mysql server

    Requires that configuration be enabled via 'event_return'
    option in master config.
    """
    with _get_serv(events, commit=True) as cur:
        for event in events:
            tag = event.get("tag", "")
            data = event.get("data", "")
            sql = """INSERT INTO `salt_events` (`tag`, `data`, `master_id` )
                  VALUES (%s, %s, %s)"""
            cur.execute(sql, (tag, salt.utils.json.dumps(data), __opts__["id"]))

Testing the Returner¶

The returner, prep_jid, save_load, get_load, and event_return functions can be tested by configuring the master_job_cache and Event Returners in the master config file and submitting a job to test.version each minion from the master.

Once you have successfully exercised the Master Job Cache functions, test the External Job Cache functions using the ret execution module.

salt-call ret.get_jids cassandra_cql --output=json
salt-call ret.get_fun cassandra_cql test.version --output=json
salt-call ret.get_minions cassandra_cql --output=json
salt-call ret.get_jid cassandra_cql 20150330121011408195 --output=json