The most basic or typical Salt topology consists of a single Master node controlling a group of Minion nodes. An intermediate node type, called Syndic, when used offers greater structural flexibility and scalability in the construction of Salt topologies than topologies constructed only out of Master and Minion node types.
A Syndic node can be thought of as a special passthrough Minion node. A Syndic
node consists of a
salt-syndic daemon and a
salt-master daemon running
on the same system. The
salt-master daemon running on the Syndic node
controls a group of lower level Minion nodes and the
connects higher level Master node, sometimes called a Master of Masters.
salt-syndic daemon relays publications and events between the Master
node and the local
salt-master daemon. This gives the Master node control
over the Minion nodes attached to the
salt-master daemon running on the
Salt does not officially support Syndic and external auth or publisher_acl. It's possible that it might work under certain circumstances, but comprehensive support is lacking. See issue #62618 on GitHub for more information. Currently Syndic is only expected to work when running Salt as root, though work is scheduled to fix this in Salt 3006 (Sulfur).
To setup a Salt Syndic you need to tell the Syndic node and its Master node
about each other. If your Master node is located at
10.10.0.1, then your
configurations would be:
On the Syndic node:
# /etc/salt/master syndic_master: 10.10.0.1 # may be either an IP address or a hostname
# /etc/salt/minion # id is shared by the salt-syndic daemon and a possible salt-minion daemon # on the Syndic node id: my_syndic
On the Master node:
# /etc/salt/master order_masters: True
syndic_master option tells the Syndic node where to find the
Master node in the same way that the
master option tells a
Minion node where to find a Master node.
id option is used by the
salt-syndic daemon to identify
with the Master node and if unset will default to the hostname or IP address of
the Syndic just as with a Minion.
order_masters option configures the Master node to send
extra information with its publications that is needed by Syndic nodes
connected directly to it.
Each Syndic must provide its own
file_roots directory. Files will not
be automatically transferred from the Master node.
New in version 2015.5.0.
Syndic with Multimaster lets you connect a syndic to multiple masters to provide an additional layer of redundancy in a syndic configuration.
Higher level masters should first be configured in a multimaster configuration. See Multimaster Tutorial.
On the syndic, the
syndic_master option is populated with
a list of the higher level masters.
Since each syndic is connected to each master, jobs sent from any master are
forwarded to minions that are connected to each syndic. If the
is set in the master config on the higher level masters, job results are returned
to the master that originated the request in a best effort fashion. Events/jobs
master_id are returned to any available master.
salt-syndic daemon is a separate process that needs to be started in
addition to the
salt-master daemon running on the Syndic node. Starting
salt-syndic daemon is the same as starting the other Salt daemons.
The Master node in many ways sees the Syndic as an ordinary Minion node. In particular, the Master will need to accept the Syndic's Minion key as it would for any other Minion.
On the Syndic node:
# salt-syndic or # service salt-syndic start
On the Master node:
# salt-key -a my_syndic
The Master node will now be able to control the Minion nodes connected to the Syndic. Only the Syndic key will be listed in the Master node's key registry but this also means that key activity between the Syndic's Minions and the Syndic does not encumber the Master node. In this way, the Syndic's key on the Master node can be thought of as a placeholder for the keys of all the Minion and Syndic nodes beneath it, giving the Master node a clear, high level structural view on the Salt cluster.
On the Master node:
# salt-key -L Accepted Keys: my_syndic Denied Keys: Unaccepted Keys: Rejected Keys: # salt '*' test.version minion_1: 2018.3.4 minion_2: 2018.3.4 minion_4: 2018.3.4 minion_3: 2018.3.4
A Master node (a node which is itself not a Syndic to another higher level
Master node) must run a
salt-master daemon and optionally a
A Syndic node must run
salt-master daemons and
A Minion node must run a
salt-master daemon issues a command, it will be received by the
Syndic and Minion nodes directly connected to it. A Minion node will process
the command in the way it ordinarily would. On a Syndic node, the
salt-syndic daemon will relay the command to the
running on the Syndic node, which then propagates the command to the Minions
and Syndics connected to it.
When events and job return data are generated by
salt-minion daemons, they
are aggregated by the
salt-master daemon they are connected to, which
salt-master daemon then relays the data back through its
daemon until the data reaches the Master or Syndic node that issued the command.
syndic_wait is a master configuration file setting that specifies the number of
seconds the Salt client should wait for additional syndics to check in with their
lists of expected minions before giving up. This value defaults to
syndic_wait setting is necessary because the higher-level master does not
have a way of knowing which minions are below the syndics. The higher-level master
has its own list of expected minions and the masters below them have their own lists
as well, so the Salt client does not know how long to wait for all returns. The
syndic_wait option allows time for all minions to return to the Salt client.
To reduce the amount of time the CLI waits for Minions to respond, install
a Minion on the Syndic or tune the value of the
While it is possible to run a Syndic without a Minion installed on the same
system, it is recommended, for a faster CLI response time, to do so. Without a
Minion installed on the Syndic node, the timeout value of
increases significantly - about three-fold. With a Minion installed on the
Syndic, the CLI timeout resides at the value defined in
If you have a very large infrastructure or many layers of Syndics, you may
find that the CLI doesn't wait long enough for the Syndics to return their
events. If you think this is the case, you can set the
syndic_wait value in the Master configs on the Master or
Syndic nodes from which commands are executed. The default value is
and should work for the majority of deployments.
In order for a Master or Syndic node to return information from Minions that
are below their Syndics, the CLI requires a short wait time in order to allow
the Syndics to gather responses from their Minions. This value is defined in
syndic_wait config option and has a default of five seconds.
These are the options that can be used to configure a Syndic node. Note that
id, Syndic config options are placed in the Master config on the
id: Syndic id (shared by the
salt-syndicdaemon with a potential
salt-miniondaemon on the same system)
syndic_master: Master node IP address or hostname
syndic_master_port: Master node ret_port
syndic_log_file: path to the logfile (absolute or not)
syndic_pidfile: path to the pidfile (absolute or not)
syndic_wait: time in seconds to wait on returns from this syndic
Beginning with Salt 2016.11.0, the Pluggable Minion Data Cache
was introduced. The minion data cache contains the Salt Mine data, minion grains, and minion
pillar information cached on the Salt Master. By default, Salt uses the
module, but other external data stores can be used instead.
Using a pluggable minion cache modules allows for the data stored on a Salt Master about Salt Minions to be replicated on other Salt Masters the Minion is connected to. Please see the Minion Data Cache documentation for more information and configuration examples.