(release-3008.2)=

Salt 3008.2 release notes#

Changelog#

Removed#

  • Removed 11 stale .txt files under requirements/static/{pkg,ci}/py*/ that were missed by the .txt -> .lock migration. Three are true orphans from dropped Python 3.8 support; eight shadowed current .lock siblings which are the authoritative artifacts. #69488

Changed#

  • Upgrade the bundled onedir Python from 3.10.20 to 3.11.15 on the 3006.x branch. Python 3.10 reaches end of security support in October 2026, while Salt 3006.x must ship security fixes through July 2027. Users upgrading from a previous 3006.x package will need to reinstall any Salt extensions installed via salt-pip because the onedir extras-3.10 directory is replaced by extras-3.11. #69526

Fixed#

  • Fixed salt-ssh TemplateNotFound when a managed Jinja template imports from another template (e.g. {% from "formula/map.jinja" import x with context %}). SaltCacheLoader now prefers opts["_caller_cachedir"] (the master's cachedir, where the master-side fileclient caches requested files) over opts["cachedir"] (the thin minion's remote path) for its Jinja search path. Backport of the 3007.x/3008.x fix. #31531

  • Fixed the mysql returner ignoring the configured mysql.user from salt-ssh and other contexts where __salt__ lacks config.option. get_returner_options fell back to __opts__ and looked up bare attribute names in it, so the master's top-level user opt (the system user salt runs as, typically root) masked the configured database user and the returner connected as the wrong user. The mysql returner now passes a scoped view of __opts__ containing only mysql.* keys so the lookup cannot collide. #32567

  • Fixed non-deterministic pillar rendering when multiple pillar_roots environments matched the same minion. Pillar.get_tops collected saltenvs into a set and iterated them in hash order, so top-file processing order depended on PYTHONHASHSEED and varied per salt-call invocation. An earlier change made _get_envs return an ordered list, but the caller wrapped the result back into a set. get_tops now uses an insertion-ordered dict so iteration follows pillar_roots config order. #44937

  • Documented the supported approaches for relocating Salt's runtime directories when running rootless: SALT_HOME/SALT_EXTRAS_DIR at install time, root_dir for relative relocation, and the per-key (pki_dir, cachedir, log_file, pidfile, sock_dir) overrides. #55971

  • Rewrote the non-root / unprivileged user configuration page for onedir packaging, consolidating the older overlapping pages and documenting SALT_USER/SALT_HOME/SALT_EXTRAS_DIR, root_dir relocation, and systemd drop-ins. #59955

  • Fixed a race in tests/pytests/integration/cli/test_salt.py::test_interrupt_on_long_running_job that intermittently failed on slow CI hosts (Photon OS 5 Arm64, both tcp(fips) and zeromq(fips)). The test used a fixed time.sleep(2) before sending SIGINT, but on slow hosts the salt CLI had not yet published its job (pub_data["jid"] was still unset), so the signal handler emitted only Exiting gracefully on Ctrl-c without a jid and the This job's jid is assertion failed. The test now waits on the master's salt/job/*/new event via event_listener to guarantee the job has been published before interrupting the CLI. #60963

  • Rewrote the FAQ entry on restarting the minion after upgrade for the onedir packaging era. Removed the broken policy-rc.d/prereq workaround and documented the supported patterns based on KillMode=process in the shipped systemd unit. #61078

  • Updated the packaging docs to explain how to install modules' optional Python dependencies into an onedir install via salt-pip. #64160

  • Documented salt-pip for installing optional Python dependencies into a onedir Salt install, including the extras directory layout, SALT_EXTRAS_DIR relocation, and non-root behavior. #64291

  • Fixed the EC2/cloud metadata grain crashing with KeyError: 'headers' when salt.utils.http.query returns an error response (4xx/5xx with a body, e.g. when the IMDS rejects a recursive sub-path lookup). Since 3006.3 the tornado backend has populated body on HTTPError without also populating headers; the grain now treats the missing headers key as "no Content-Type information" instead of letting the lookup blow up the whole grain load. #65184

  • Updated the non-root user docs for the onedir-era directory layout (/opt/saltstack/salt, extras-3.N, package-managed salt user) and explained how to switch an existing install over to a different account. #65243

  • Expanded the packaging test guide with single-test invocations, environment variables, common failures, and CI parity notes. #65253

  • Fixed master-initiated jobs failing on Python 3.12+ with "There is no current event loop in thread 'Thread-N (_target)'" by installing an asyncio event loop on the SyncWrapper worker thread. #65702

  • Fixed TypeError: a coroutine was expected, got None (Python 3.10) / object NoneType can't be used in 'await' expression raised repeatedly by salt-api and salt-master from salt.transport.tcp.PublishClient.on_recv_handler. The salt-api EventListener._handle_event_socket_recv callback was a plain function returning None and is now an async coroutine, so the TCP IPC publish client can schedule it via asyncio.create_task without errors and events are no longer silently dropped. #66177

  • Fixed master 4505 publish port becoming unresponsive under load: TCP PubServer now broadcasts to subscribers concurrently so a single slow subscriber no longer stalls the event publisher loop, and the ZeroMQ master PUB socket now enables ZMTP heartbeats so dead subscribers are reaped within seconds instead of waiting for the kernel TCP keepalive. #66282

  • Refreshed the "running as a non-root user" page; replaced outdated 0.9.10-era guidance and added the onedir-aware steps for changing the runtime user. #66353

  • Documented how to install Salt Extensions (saltext.<name>) into an onedir install with salt-pip, and pointed the developer extensions doc at the install instructions. #66524

  • Fixed salt.utils.vmware to use the supported token/tokenType arguments instead of the deprecated b64token/mechanism arguments when calling pyVim.connect.SmartConnect. pyvmomi 9 raises an exception when either deprecated argument is truthy, which broke salt-cloud, the vsphere execution module, and other VMware integrations as soon as pyvmomi was upgraded. #68211

  • Fixed state.event (and salt-run state.event) crashing with UnicodeDecodeError when an event payload contains raw binary bytes such as the DER-encoded certificate returned by x509.sign_remote_certificate. Undecodable bytes are now base64-encoded in the JSON output instead of aborting the runner. #68411

  • Fixed salt.utils.url.create so salt:// URLs built from relative paths round-trip correctly on Python 3.13+, where urllib.parse.urlunparse no longer emits a file:/// prefix for relative paths. salt-ssh file.managed source: salt://... references now resolve as expected on newer-Python targets (e.g. Debian trixie). #68421

  • Fix set_locale on Debian 13/14 where systemd-localed is unavailable; fall back to /etc/default/locale update. #68425

  • Fixed a prereq chain bug where a state at the head of a chain (e.g. state1 -prereq-> state2 -prereq-> state3) would always run when an intermediate state in the chain always produced changes in test mode (e.g. test.succeed_with_changes, module.run), even though the tail state of the chain produced no changes. #68438

  • Fixed descriptor leaking in salt.utils.http.query #68456

  • Fixed Debian salt-minion package failing to upgrade from a non-onedir release. The salt-minion.preinst script assigned an unused PY_VER variable by exec'ing /opt/saltstack/salt/bin/python3, which does not exist when upgrading from a pre-onedir Debian package (e.g. 3006.0+ds-1+240.1). Under set -e this aborted the upgrade with subprocess returned error exit status 127. The unused assignment is removed. #68460

  • Fixed master cluster event forwarding when a clustered master has no explicit id configured. apply_master_config appends _master to the auto-detected id, but cluster_peers and the on-the-wire data["peers"] dict are keyed by the bare names. The shared peer pubkey path written by MasterKeys and the lookup in MasterPubServerChannel.handle_pool_publish now strip the suffix so peers can decrypt forwarded events instead of failing with KeyError: '<host>_master'. #68462

  • Fixed salt-master package upgrades resetting state directory ownership and the debconf salt-master/user value when the master was configured to run as a non-root user. #68577

  • Don't insert local paths before standard library paths in LazyLoader, preventing sys.path reordering when loader modules are already importable. #68755

  • Fixed Salt minion package upgrades when the minion is configured to run as a non-root user via user: in /etc/salt/minion or /etc/salt/minion.d/*.conf. The Debian preinst now reads the configured user before falling back to filesystem ownership, and the rpm pre-minion scriptlet no longer relies on rpm macro directives inside its shell body to communicate the chosen user to the post-minion scriptlet. #68793

  • Fixed a file descriptor leak in the Salt minion: when the single-master sign-in path in Minion.eval_master raised any exception other than SaltClientError (for example OSError from the underlying transport), or when transport: detect rejected a candidate transport because it could not authenticate, the AsyncPubChannel that had been created was not closed, leaking its socket. Minions with unstable network connectivity could exhaust the per-process file descriptor limit. The channel is now always closed on failure via a try/finally. #68901

  • Fix docker_container.running destroying the original container on the force=True / skip_comparison path by passing the temp container's dict instead of its name to _replace. docker.rename then failed after docker.rm had already removed the original, leaving the minion with the temp container stranded under its generated name. _replace now receives temp_container_name on both call sites, matching the non-force path. #68959

  • Restore the reclass ext_pillar adapter (salt.pillar.reclass_adapter) that was dropped when community extensions were purged from the 3008 tree. Existing ext_pillar: - reclass: master configurations work again on 3008.x without downgrading. #69018

  • Fixed salt.utils.cache.ContextCache.cache_context writing the serialized pillar context to disk with whatever mode the process umask happened to allow (typically 0o644 on default Linux installs) inside a 0o755 parent directory. Pillar context can carry credentials (passwords, vault tokens, API keys), so any local user could read them; even with the file mode tightened, the directory mode let any local user ls the cache and learn which modules and external-pillar backends were in use. The cache file is now written through tempfile.mkstemp (creates with 0o600 by default) followed by atomic os.replace, and the parent context/ directory is created with stat.S_IRWXU (0o700). #69069

  • Fixed kernelpkg.upgrade on Debian 13 (trixie) and other distros that ship a kernelrelease containing characters outside [\d.-] (for example 6.12.86+deb13-amd64). kernelpkg_linux_apt._kernel_type now parses such releases instead of raising AttributeError: 'NoneType' object has no attribute 'group'. #69131

  • Added a regression test covering the TypeError: string indices must be integers crash in AsyncReqChannel.crypted_transfer_decode_dictentry when the master returns a bare-string error payload for a pillar request. The crash itself was already fixed on master by the layered isinstance(ret, dict) guards in salt/channel/client.py; the test pins that behavior. #69228

  • Fix PAM authentication always returning 401 on relenv/onedir installs by preferring sys.executable over /usr/bin/python3 when launching the PAM helper subprocess. #69303

  • Fixed auth tokens being deleted from the localfs cache driver within one master loop_interval (default 60s) of being minted: Cache.clean_expired's fallback path now consults the cache-level _expires envelope instead of file mtime, and LoadAuth.mk_token passes a relative duration to Cache.store(expires=...) rather than an absolute epoch. #69307

  • Fixed salt -b (sync batch mode) failing with SaltClientError: Some exception handling minion payload when the salt-master runs as a non-root user (e.g. salt). The sync CLI batch driver had been writing batch-state persistence files (.batch.p, batch_active.p) under the master's cachedir from the CLI process — pre-creating the JID directory with root ownership and tripping a PermissionError in local_cache.prep_jid on the master.

    The sync CLI driver no longer writes anything under the master's cachedir itself. Instead it ships every state transition to the master-side BatchManager as salt/batch/<jid>/{new,progress,complete,halted} events; the manager — already running as the master daemon's user — persists .batch.p and maintains the active-batch index on the CLI's behalf. salt-run batch.status <jid>, salt-run batch.list_active, and salt-run batch.stop <jid> now work for sync batches in the same deployment shape (non-root master, root CLI) where the original feature was broken. Event-bus failures degrade gracefully: the batch still completes, just without visibility from the runner commands. #69418

  • Added a new opt-in auth_retries minion option that caps the AsyncAuth._authenticate() outer retry loop, so a minion that keeps getting retry responses from sign_in() can bail out with SaltClientError instead of looping silently forever. The default is 0 (unlimited), which preserves the existing 3006.x LTS behavior on upgrade; operators who want the new safety cap set auth_retries explicitly to a positive integer. #69442

  • Fixed Photon OS Arm64 FIPS CI by re-enabling the OpenSSL default provider after installing openssl-fips-provider, working around the disabled-default-provider bug in openssl-fips-provider <= 3.1.2-3.ph5 on the lagging Photon aarch64 mirror. #69448

  • Fixed AESFuncs._register_resources to fire a minion_data_cache_events notification on the master event bus when resource grains are written to the cache, mirroring the existing notification fired by _pillar for ordinary minion grains. #69451

  • Fixed the towncrier changelog template splitting every multi-line fragment into separate top-level bullets with a duplicate [#NNNN] link on each. Multi-line fragments now render as a single bullet with continuation lines indented under it, and the issue link is appended exactly once. #69454

  • Fixed salt.utils.url.parse so salt:///path (three-slash URLs with an empty authority) resolves the same as salt://path. Restores cp.get_file salt:///path/to/file and similar fileclient calls that previously failed because the surplus leading slash was rejected by the master fileserver's absolute-path guard. #69472

  • Fixed saltutil.runner/saltutil.wheel failing git-backed master functions (e.g. git_pillar.update) with failed to stat '/root/.gitconfig' when the master runs as a non-root user. Dropping to the master user with chugid left HOME/USER/LOGNAME pointing at the invoking (root) user; these are now aligned with the runas user, and pygit2's cached global-config search path is refreshed. #69569

  • Stopped logging a spurious random_master is True but there is only one master specified. Ignoring. warning once per master at startup for an all-hot multi-master minion. The warning now fires only for a genuinely single-master configuration. #69571

  • Fix OpenNebula salt-cloud documentation to clarify that VM attributes (memory, cpu, vcpu, etc.) must be specified in the profile configuration, not as command-line arguments to salt-cloud -p. #69573

  • Removed bundled MD5/SHA-1 references that tripped FIPS-compliance scanners against the Salt onedir. The cryptography sdist's top-level docs/ directory (which contains Java/Rust test-vector sources naming weak algorithms, e.g. VerifyRSAOAEPSHA2.java) is now pruned from the onedir during pre-archive-cleanup, and the unused __fetch_verify helper in the vendored bootstrap-salt.sh now uses sha256sum instead of md5sum. #69575

  • Replace deprecated asyncio.iscoroutinefunction with inspect.iscoroutinefunction in salt/utils/event.py and salt/cluster/consensus/raft/scheduler.py to avoid DeprecationWarning on Python 3.12+ (slated for removal in Python 3.16). #69580

  • Fixed salt-run manage.status/manage.up/manage.down reporting every targeted minion as up because synthesized no_return rows from LocalClient.get_cli_event_returns were being counted as successful returns. #69582

  • Fixed salt.utils.atomicfile.atomic_open to fsync the temp file before the atomic rename so a crash after the rename cannot expose a truncated or partial file. #69583

  • Fixed RPM upgrades leaving a previously-running salt-minion service stopped. The %pre minion scriptlet stops the unit so the ownership-restoration chowns don't race a live minion, but the %post / %posttrans scriptlets only called systemctl try-restart - a no-op for an inactive unit. The scriptlets now record the pre-upgrade active state and start the unit unconditionally in %posttrans when the minion was running at the start of the upgrade transaction. #69605

    • Relenv 0.22.16

      • 0.22.15: apply cpython#104135 workaround to bundled ssl.py on Windows

      • 0.22.15: send relenv runtime debug/warning output to stderr (unblocks maturin/pyo3 subprocess consumers)

      • 0.22.16: pin libffi to cpython-bin-deps on Windows #69612

  • Restore Rocky Linux 9 unit zeromq 4 CI green after the 3006.x→3007.x merge-forward pulled in 3006.x-only regression tests that don't fit the 3007.x runtime APIs. Adapt the test_verify_master_*, test_authenticate_*_69442, test_maintenance_duration, test_minion_manager_stop_unblocks_resolve_dns_69466, and test_event_unpack_with_SaltDeserializationError tests to the 3007.x crypt.write_keys() / MasterKeys.gen_signature / io_loop.create_task / LoadAuth init / debug-log-on-skip contracts; skip the test_gen_signature_signs_clean_key variants because the 3007.x cache-refactored MasterKeys.gen_signature signs pub.public_bytes() and cannot exhibit the #68930 whitespace-drift bug. #69624

Added#

  • Added tools/audit_doc_links.py and a weekly doc-linkcheck workflow that wrap Sphinx linkcheck, strip the catch-all ignore, and emit a CSV report so external URL regressions in the docs can be tracked without gating PR CI. #60720

  • added conditional X functionality to linux_acl #62852

  • Added unmask parameter to pillar.ls, pillar.raw, pillar.ext, pillar.keys, and pillar.obfuscate for API consistency with pillar.get / pillar.items / pillar.item / pillar.data. Default masking behavior is unchanged. #69453

  • Documented the gitcli GitFS provider (added in 3008.0) which shells out to the system git binary, auto-detected after pygit2 and gitpython and used as a silent fallback when neither Python library is installed. Documented the cluster_isolated_filesystem master option (added in 3008.0) which lets master clusters run without a shared filesystem; keys, denied keys, file_roots and pillar_roots are sync'd in-band over the cluster transport, with keys.cache_driver: mmap_key as the recommended companion. #69494