You are viewing docs from the master branch, some of these features are not yet released.

The Salt Fileserver and Client¶

Introduction¶

Salt has a modular fileserver, and multiple client classes which are used to interact with it. This page serves as a developer's reference, to help explain how the fileserver and clients both work.

Fileserver¶

The fileserver is not a daemon, so the fileserver and client are not a true server and client in the traditional sense. Instead, the fileserver is simply a class (salt.fileserver.Fileserver), located in salt/fileserver/__init__.py. This class has access to the configured fileserver backends via a loader instance, referenced as self.servers. When a request comes in from the fileclient, it will ultimately result in a Fileserver class function being run.

The functions in this class will run corresponding functions in the configured fileserver backends to perform the requested action. So, in summary:

A fileclient class makes a request...
which triggers the fileserver to run a function...
which runs a named function in each of the configured backends.

Not all of the functions will always execute on every configured backend. For instance, the find_file function in the fileserver will stop when it finds a match, so if it finds a match for the desired path in the first configured backend, it won't proceed and try to find the file in the next backend in the list.

Additionally, not all backends implement all functions in the salt.fileserver.Fileserver class. For instance, there is a function called update, which exists to update remote fileservers such as the git, hg, and svn backends. This action has no use however in the roots backend, so it is simply not implemented there, and thus the roots backend will be skipped if the update function is run on the fileserver.

Backends for the fileserver are located in salt/fileserver/ (the files not named __init__.py).

Fileclient¶

There are three fileclient classes:

salt.fileclient.RemoteClient¶

This client is used when file_client is set to remote. This is how minions request files from the master.

Functions in this client will craft a payload and send it to the master via the transport channel. This is the same way that the minion asks the minion to do other things, such as updating and requesting data from the mine. The payload will be a dictionary with a key called cmd, and other values as needed.

Payloads sent via the transport channel are processed my an MWorker instance on the master, and the MWorker's _handle_aes() function will execute the command. The command will be a function attribute of the salt.master.AESFuncs class. The AESFuncs class' __setup_fileserver() function instantiates a salt.fileserver.Fileserver instance and maps its member functions to AESFuncs attributes. This is what makes the fileserver functions available remotely. The result of the function is returned back through the transport channel to the minion.

Transporting files is done in chunks, the size of which is decided by the file_buffer_size config option. If you look at the serve_file() function in any of the fileserver backends, you can see how the loc value in the payload determines the offset so that an intermediate chunk of the file can be served. The RemoteClient's get_file() function will loop until the end of the file is reached, retrieving one chunk at a time.

salt.fileclient.FSClient¶

This client is used when file_client is set to local. This is how masterless minions request files.

This class inherits from the RemoteClient, but instead of using a transport channel (zmq, tcp, etc.), it uses a "fake" transport channel (salt.fileserver.FSChan), which implements its own send() function. Thus, when a function that the FSClient inherits from the RemoteClient runs self.channel.send(), it's actually calling salt.fileserver.FSChan.send(), which calls corresponding functions in the salt.fileserver.Fileserver() class. The result is that local file requests use the same code as remote file requests, they just bypass sending them through an actual transport channel and instead call them on the FSChan's Fileserver instance.

salt.fileclient.LocalClient¶

This client is now used exclusively by Pillar. This used to be used when file_client was set to local, but the FSChan class was written to allow minions with file_client: local to access the full set of backends. This class will probably be renamed at some point as it is often confused with salt.client.LocalClient.

The `cp` Module¶

Most of the user-facing interaction with the fileclient happens via the cp module. The functions in this module instantiate a fileclient instance (if one is not already saved to the __context__ dunder) and run fileclient functions.

Updating the Fileserver¶

The master daemon spawns a process dedicated to routine maintenance tasks upon startup. This process runs an instance of salt.master.Maintenance, which loops forever, running a series of functions and then sleeping for a length of time determined by the loop_interval config option. One of the maintenance tasks is to update the fileserver, and it essentially runs salt.fileserver.Fileserver.update(), which as we know from above will run all configured backends' update() functions, if present. This is now remote fileservers like git, hg, and svn stay up-to-date.

For the local file_client (FSClient), since it does not interact with the master, upon spawning of its FSChan it will update the fileserver.