salt.returners.salt_cache#

Job-cache returner backed by salt.cache.Cache.

A drop-in replacement for the default local_cache returner that routes every read and write through the salt.cache.Cache abstraction. This unlocks two operational capabilities the default returner does not provide:

  • Pluggable storage. Whichever cache: driver the operator configures (localfs, mmap_cache, redis, …) is what backs the job cache. master_job_cache: salt_cache plus cache: mmap_cache gives an mmap-backed job cache.

  • Multi-ring sharding. Because every entry flows through one salt.cache.Cache instance, the cluster's salt.cluster.ring_membership.owns_for() gate can shard job state across ring members, and the salt.runners.cluster.shed_unowned() / salt.runners.cluster.collect_from_peers() runners can enumerate and move data.

Bank layout#

Each JID's state is spread across a small set of banks so per-JID operations are atomic and ring ownership maps cleanly to a single key per JID at the routing layer:

  • jobs/loads -- key=jid -> pub load dict

  • jobs/minions -- key=jid -> sorted list of minion ids

  • jobs/returns/<jid> -- key=minion_id -> {return, retcode, success, out}

  • jobs/endtimes -- key=jid -> epoch seconds (only when job_cache_store_endtime is set)

  • jobs/nocache -- key=jid -> True (job-was-fired-with-nocache marker)

The jobs/<jid> granularity is what the ring sites consult: the gate at salt/master.py calls ring_membership.owns_for(opts, "jobs", jid) and only persists when this master owns the JID.

Compatibility#

API-compatible with local_cache for every function Salt calls through the returner interface (prep_jid, returner, save_load, save_minions, get_load, get_jid, get_jids, get_jids_filter, clean_old_jobs, update_endtime, get_endtime). The on-disk layout differs from local_cache — operators migrating need to either drain existing jobs or run the migration runners on the old data.

Configuration#

master_job_cache: salt_cache
cache: mmap_cache         # or localfs, redis_cache, ...

When mmap_cache is used, mmap_cache_max_segment_bytes / mmap_cache_dirs apply as usual.

salt.returners.salt_cache.clean_old_jobs()#

Drop loads (and all their associated banks) older than keep_jobs_seconds opt.

Uses salt.cache.Cache.updated() to age each load; falls back to "keep" when the driver can't report a timestamp (some drivers don't implement updated). Mirrors local_cache.clean_old_jobs semantically; the loop is far simpler because we don't have to walk a two-level directory hierarchy.

salt.returners.salt_cache.get_endtime(jid)#

Return the recorded end-time for jid, or None.

salt.returners.salt_cache.get_jid(jid)#

Return {minion_id: return_dict} for every return recorded against jid.

Each return is a dict with at least return; retcode, success, and out are populated when the minion reported them (same shape as local_cache).

salt.returners.salt_cache.get_jids()#

Return {jid: jid_info} for every load on disk.

jid_info is what salt.utils.jid.format_jid_instance() produces — used by the runner / API surfaces (salt-run jobs.list_jobs etc.). When job_cache_store_endtime is on, the matching EndTime is folded in.

salt.returners.salt_cache.get_jids_filter(count, filter_find_job=True)#

Return the count most-recent JIDs as jid_info_ext dicts, optionally filtering out saltutil.find_job traffic.

JIDs sort lexicographically by their timestamp prefix so the most-recent is the last in sorted order. We accumulate into a bounded list to stay O(N log count) rather than O(N log N).

salt.returners.salt_cache.get_load(jid)#

Return the pub load for jid, plus a sorted Minions list if the matched set was recorded by save_minions().

salt.returners.salt_cache.prep_jid(nocache=False, passed_jid=None, recurse_count=0)#

Return a job id and record any pre-flight state for it.

Mirrors local_cache.prep_jid:

  • If passed_jid is supplied (e.g. by salt-call --jid … or a syndic), use it. Otherwise generate one via salt.utils.jid.gen_jid().

  • On collision (an existing jobs/loads key with the same id) generate a new one, up to 5 retries. Operators who hit the retry cap have a deeper clock-skew problem the returner shouldn't paper over.

  • When nocache is set, write a marker so returner() knows to short-circuit subsequent minion returns.

salt.returners.salt_cache.returner(load)#

Persist one minion return for a JID.

Idempotency: a second return from the same minion for the same jid is treated as a replay attempt and dropped (mirrors local_cache's EEXIST branch). Returning False lets the master's reactor flag the event.

salt.returners.salt_cache.save_load(jid, clear_load, minions=None, recurse_count=0)#

Persist the pub load (tgt, fun, arg, …) for a JID.

If the load carries a tgt we compute the matched minion set here (unless the caller supplied one) and pass it to save_minions(). The minion set is what get_load() later exposes as Minions to the UI.

salt.returners.salt_cache.save_minions(jid, minions, syndic_id=None)#

Store the list of matched minions for a JID.

Merge with any previously-saved list so syndic-master appends don't clobber the main-master record (mirrors local_cache.save_minions). syndic_id is accepted for API compatibility but folded into the same merged list; the distinction is preserved on-disk in local_cache only because its file naming scheme uses it for routing.

salt.returners.salt_cache.update_endtime(jid, the_time)#

Record (or overwrite) the end-time for jid.