salt.cache.mmap_cache#

A memory-mapped cache backend, drop-in for localfs. On large fleets mmap_cache runs orders of magnitude faster than the default file-per-entry layout — see Memory-Mapped Cache Backend for benchmark numbers, sizing guidance, and the migration path.

Cache data in memory-mapped files (index + heap architecture).

New in version 3009.0.

The mmap_cache module is a drop-in replacement for the localfs cache backend. It stores cache data in a pair of memory-mapped files per bank:

  • index file — a fixed-size open-addressing hash table that maps keys to heap pointers.

  • heap file — a flat binary append-log that holds the serialized values.

This layout gives O(1) reads and O(1) appends, which makes it well-suited for high-frequency workloads such as Raft log persistence.

Configuration (all optional, can be set in /etc/salt/master):

cache: mmap_cache

# Number of index slots per bank (default: 1 000 000)
mmap_cache_size: 1000000

# Bytes per index slot; must be >= 1 + mmap_cache_key_size + 20
mmap_cache_slot_size: 96

# Maximum key length in bytes
mmap_cache_key_size: 64

The bank concept maps directly to a sub-directory of cachedir. One MmapCache instance (index + heap pair) is created per (cachedir, bank) and kept alive in a module-level registry for the lifetime of the process.

salt.cache.mmap_cache.contains(bank, key, cachedir, **kwargs)#

Return True if bank contains key (or, if key is None, whether the bank itself exists at all).

salt.cache.mmap_cache.fetch(bank, key, cachedir, **kwargs)#

Return the deserialised value for bank/key, or {} if not found.

salt.cache.mmap_cache.flush_(bank, key=None, cachedir=None, **kwargs)#

Remove key from bank, or clear the entire bank if key is None.

Clearing a bank removes the mmap files from the registry and deletes them from disk, mirroring localfs behaviour where shutil.rmtree removes the bank directory.

salt.cache.mmap_cache.get_storage_id(kwargs)#

Return a unique identifier for this cache driver instance.

salt.cache.mmap_cache.init_kwargs(kwargs)#

Return the canonical keyword arguments for this cache driver.

salt.cache.mmap_cache.list_(bank, cachedir, **kwargs)#

Return a list of all keys stored in bank.

salt.cache.mmap_cache.list_all(bank, cachedir, include_data=False, **kwargs)#

Return {key: data} for every entry in bank in a single pass.

Walks the mmap roster once (O(occupied)) and msgpack-decodes each heap entry inline, avoiding the per-key hash probe that list_(bank) + fetch(bank, k) would do.

With include_data=False the data slot is {} so callers can use this purely to enumerate keys without paying msgpack deserialisation; with include_data=True (default behaviour for contract parity with other backends) each value is the msgpack.unpackb round-trip of what store wrote.

salt.cache.mmap_cache.store(bank, key, data, cachedir, **kwargs)#

Serialise data with msgpack and store it under bank/key.

salt.cache.mmap_cache.updated(bank, key, cachedir, **kwargs)#

Return the Unix timestamp (int seconds) of the last write for bank/key, or None if the key does not exist.

This reads only the index — no heap access required.