The Salt Fileserver and Client¶
Introduction¶
Salt has a modular fileserver, and multiple client classes which are used to interact with it. This page serves as a developer's reference, to help explain how the fileserver and clients both work.
Fileserver¶
The fileserver is not a daemon, so the fileserver and client are not a true
server and client in the traditional sense. Instead, the fileserver is simply a
class (salt.fileserver.Fileserver), located in
salt/fileserver/__init__.py. This class has access to the configured
fileserver backends via a loader instance, referenced as self.servers. When
a request comes in from the fileclient, it will ultimately result in a
Fileserver class function being run.
The functions in this class will run corresponding functions in the configured fileserver backends to perform the requested action. So, in summary:
A fileclient class makes a request...
which triggers the fileserver to run a function...
which runs a named function in each of the configured backends.
Not all of the functions will always execute on every configured backend. For
instance, the find_file function in the fileserver will stop when it finds
a match, so if it finds a match for the desired path in the first configured
backend, it won't proceed and try to find the file in the next backend in the
list.
Additionally, not all backends implement all functions in the
salt.fileserver.Fileserver class. For instance, there is a function called
update, which exists to update remote fileservers such as the git,
hg, and svn backends. This action has no use however in the roots
backend, so it is simply not implemented there, and thus the roots backend
will be skipped if the update function is run on the fileserver.
Backends for the fileserver are located in salt/fileserver/ (the files not
named __init__.py).
Fileclient¶
There are three fileclient classes:
salt.fileclient.RemoteClient¶
This client is used when file_client is set to remote. This
is how minions request files from the master.
Functions in this client will craft a payload and send it to the master via the
transport channel. This is the same way that the minion asks the minion to do
other things, such as updating and requesting data from the mine. The payload
will be a dictionary with a key called cmd, and other values as needed.
Payloads sent via the transport channel are processed my an MWorker instance on
the master, and the MWorker's _handle_aes() function will execute the
command. The command will be a function attribute of the
salt.master.AESFuncs class. The AESFuncs class' __setup_fileserver()
function instantiates a salt.fileserver.Fileserver instance and maps its
member functions to AESFuncs attributes. This is what makes the fileserver
functions available remotely. The result of the function is returned back
through the transport channel to the minion.
Transporting files is done in chunks, the size of which is decided by the
file_buffer_size config option. If you look at the serve_file()
function in any of the fileserver backends, you can see how the loc value
in the payload determines the offset so that an intermediate chunk of the file
can be served. The RemoteClient's get_file() function will loop until the
end of the file is reached, retrieving one chunk at a time.
salt.fileclient.FSClient¶
This client is used when file_client is set to local. This
is how masterless minions request files.
This class inherits from the RemoteClient, but instead of using a transport
channel (zmq, tcp, etc.), it uses a "fake" transport channel
(salt.fileserver.FSChan), which implements its own send() function.
Thus, when a function that the FSClient inherits from the RemoteClient runs
self.channel.send(), it's actually calling
salt.fileserver.FSChan.send(), which calls corresponding functions in the
salt.fileserver.Fileserver() class. The result is that local file requests
use the same code as remote file requests, they just bypass sending them
through an actual transport channel and instead call them on the FSChan's
Fileserver instance.
salt.fileclient.LocalClient¶
This client is now used exclusively by Pillar. This used to be used when
file_client was set to local, but the FSChan class was
written to allow minions with file_client: local to access the full set of
backends. This class will probably be renamed at some point as it is often
confused with salt.client.LocalClient.
The cp Module¶
Most of the user-facing interaction with the fileclient happens via the
cp module. The functions in this module instantiate a
fileclient instance (if one is not already saved to the __context__
dunder) and run fileclient functions.
Updating the Fileserver¶
The master daemon spawns a process dedicated to routine maintenance tasks upon
startup. This process runs an instance of salt.master.Maintenance, which
loops forever, running a series of functions and then sleeping for a length of
time determined by the loop_interval config option. One of the
maintenance tasks is to update the fileserver, and it essentially runs
salt.fileserver.Fileserver.update(), which as we know from above will run
all configured backends' update() functions, if present. This is now remote
fileservers like git, hg, and svn stay up-to-date.
For the local file_client (FSClient), since it does not interact with the master, upon spawning of its FSChan it will update the fileserver.
