salt.cache.etcd3_cache#

Minion data cache plugin for the etcd key/value store, using API v3.

New in version 3009.0.

A from-scratch cache backend built around etcd v3 semantics: a flat keyspace, native byte values, single-PUT atomicity, and lease-based expiry. It is not a port of salt.cache.etcd_cache, which targets the v2 HTTP API; the two use different etcd APIs and storage and do not share data.

Storage model#

Each cache entry is a single etcd key whose value is salt.payload.dumps({"d": <data>, "t": <epoch>}) -- the data and its modification timestamp wrapped together. A single etcd PUT is atomic, so there is no sibling timestamp key to keep consistent. The bank/key hierarchy is mapped to a key path by joining components with /; prefix operations always use a trailing slash so bank foo does not match keys under bank foobar.

Concretely, with the default prefix, bank grains key minion-1 is stored at the etcd key /salt_cache/grains/minion-1 and its value is the msgpack-encoded {"d": <data>, "t": <epoch>} envelope as raw bytes (not base64). To inspect the cache directly:

etcdctl get --prefix /salt_cache/

The v2 salt.cache.etcd_cache driver shares the default /salt_cache prefix, but the etcd v2 and v3 APIs use independent keyspaces, so the two do not collide even on the same cluster. No other Salt etcd integration (the etcd execution module, state, or SDB) writes under this prefix.

Setup#

Install the etcd3-py client:

pip install etcd3-py

Enable as the master's data cache:

cache: etcd3

Optional configuration (defaults shown):

etcd.host: 127.0.0.1
etcd.port: 2379
etcd.username: null
etcd.password: null
etcd.ca: null
etcd.client_cert: null
etcd.client_key: null
etcd.path_prefix: /salt_cache

A profile may be used instead of top-level options by setting etcd.cache_profile: my_etcd_config on the master and placing the etcd.* keys under my_etcd_config.

Behaviour notes#

  • store(bank, key, data, expires=N) with a positive expires attaches the key to an etcd v3 lease with that TTL in seconds; etcd deletes the key when the lease expires. This is used by salt.auth for token expiry, so expired tokens are reaped by etcd rather than persisting until a manual flush.

  • list and contains follow salt.cache.localfs semantics: list(bank) returns the immediate children (direct keys and immediate sub-bank names), which is what callers such as salt.utils.master._get_cached_minion_data() expect from cache.list('minions').

  • flush returns True on success, matching salt.cache.redis_cache.

  • The driver refuses to initialize if etcd.path_prefix resolves to an empty or root path, so a misconfiguration cannot turn a bank flush into a range-delete at the cluster root.

Deploying on a shared etcd cluster#

This cache can run against an etcd cluster shared with other tenants (Patroni, Kubernetes, etc.). Recommended practice:

  • Scope access with etcd RBAC. The etcd.path_prefix check is only a fat-finger guard; the real isolation boundary is a prefix-scoped etcd role:

    etcdctl role add salt-cache
etcdctl role grant-permission salt-cache --prefix=true readwrite /salt_cache/
etcdctl user add salt-master
etcdctl user grant-role salt-master salt-cache

  • Use TLS in production. Set etcd.ca, etcd.client_cert and etcd.client_key so cache traffic and credentials are not in the clear.

  • Point at the cluster, not a single node (a TCP load balancer or DNS round-robin), so the master survives a single etcd node failing.

  • Confirm auto-compaction is enabled and monitor the DB quota. Each store adds a revision; without compaction the steady-state write rate (one revision per minion check-in) grows the DB without bound.

  • Multi-master Salt benefits from etcd's linearizable reads: multiple masters sharing this cache see a consistent view without extra coordination.

Migrating from the v2 etcd cache#

The v2 API uses a store isolated from v3, so the v3 cache cannot read v2 data (and cannot interfere with it). Because the master cache is ephemeral and repopulates as minions check in, migration is just: install etcd3-py, change cache: etcd to cache: etcd3, and restart the master. Old v2 keys are orphaned in the v2 store; remove them with ETCDCTL_API=2 etcdctl rm --recursive /salt_cache if desired.

The one user-visible behaviour change is list: the v2 driver returned recursive leaf names, so cache.list('minions') produced ['data', 'data', ...]; this driver returns the immediate children (['minion-1', 'minion-2', ...]).

Value-size limit#

etcd's default --max-request-bytes is 1.5 MiB per request. Grains, mine returns, tokens and minion keys are well under this, but a large pillar tree may exceed it, in which case store raises SaltCacheError wrapping etcdserver: request is too large. Raise the etcd flag (up to ~10 MiB) or use localfs/redis for very large pillar.

salt.cache.etcd3_cache.contains(bank, key)#

Return whether bank/key exists, or -- when key is None -- whether anything exists under the bank prefix (matching salt.cache.localfs's os.path.isdir(bank) semantic).

Raises:

salt.exceptions.SaltCacheError -- On any etcd error.

salt.cache.etcd3_cache.fetch(bank, key)#

Return the data stored at bank/key, or {} on a miss (the cache contract every Salt backend honours).

Raises:

salt.exceptions.SaltCacheError -- On any etcd or deserialization error.

salt.cache.etcd3_cache.flush(bank, key=None)#

Remove bank/key, or the entire bank (and sub-banks) when key is None, via a single etcd delete.

Returns:

True on success, including the no-op case where nothing was deleted (idempotent; matches salt.cache.redis_cache).

Raises:

salt.exceptions.SaltCacheError -- On any etcd error.

salt.cache.etcd3_cache.init_kwargs(kwargs)#

Cache-plugin hook; no per-instance state is needed, so this always returns an empty dict (parity with salt.cache.redis_cache).

salt.cache.etcd3_cache.ls(bank)#

Return the immediate children of bank: direct keys plus immediate sub-bank names, deduplicated. Matches salt.cache.localfs.

Uses a keys-only range scan, so listing a large bank does not transfer the stored values.

Parameters:

bank -- Bank path. The empty bank "" lists the top-level bank names (the salt.runners.cache.migrate() case).

Returns:

A list of child names; [] for an empty or nonexistent bank.

Raises:

salt.exceptions.SaltCacheError -- On any etcd error.

salt.cache.etcd3_cache.store(bank, key, data, expires=None)#

Store data at bank/key as a single etcd key.

Parameters:

  • bank -- Bank path. May contain / for nested banks.

  • key -- Leaf key name within the bank.

  • data -- Anything serializable by salt.payload.

  • expires -- If a positive integer, the key is attached to an etcd v3 lease with that TTL in seconds and etcd deletes it on expiry. None, 0 or a negative value stores without a lease.

Returns:

None on success.

Raises:

salt.exceptions.SaltCacheError -- On any etcd or serialization error.

salt.cache.etcd3_cache.updated(bank, key)#

Return the Unix-epoch timestamp at which bank/key was last stored, or None on a miss.

Raises:

salt.exceptions.SaltCacheError -- On any etcd error.