Git Fileserver Backend Walkthrough¶

Installing Dependencies¶

Both pygit2 and GitPython are supported Python interfaces to git. If compatible versions of both are installed, pygit2 will be preferred. In these cases, GitPython can be forced using the gitfs_provider parameter in the master config file.

# yum install python-pygit2

# apt-get install python-pygit2

2017-02-10 09:07:34,892 [salt.utils.gitfs ][ERROR ][11211] Import pygit2 failed: CompileError: command 'gcc' failed with exit status 1
2017-02-10 09:07:34,907 [salt.utils.gitfs ][ERROR ][11211] gitfs is configured but could not be loaded, are pygit2 and libgit2 installed?
2017-02-10 09:07:34,907 [salt.utils.gitfs ][CRITICAL][11211] No suitable gitfs provider module is installed.
2017-02-10 09:07:34,912 [salt.master ][CRITICAL][11211] Master failed pre flight checks, exiting

2017-02-15 18:04:45,211 [salt.utils.gitfs ][ERROR   ][6211] Error occurred fetching gitfs remote 'https://foo.com/bar.git': No Content-Type header in response

# apt-get install libgit2
# salt-pip install pygit2==1.6.1 --no-deps

# yum install GitPython

# apt-get install python-git

GitPython:
  pip.installed:
    - name: 'GitPython < 2.0.9'

Simple Configuration¶

To use the gitfs backend, only two configuration changes are required on the master:

1. Include gitfs in the fileserver_backend list in the master config file:

   fileserver_backend:
     - gitfs
   

   Note

   git also works here. Prior to the 2018.3.0 release, only git would work.

2. Specify one or more git://, https://, file://, or ssh:// URLs in gitfs_remotes to configure which repositories to cache and search for requested files:

   gitfs_remotes:
     - https://github.com/saltstack-formulas/salt-formula.git
   

   SSH remotes can also be configured using scp-like syntax:

   gitfs_remotes:
     - git@github.com:user/repo.git
     - ssh://user@domain.tld/path/to/repo.git
   

   Information on how to authenticate to SSH remotes can be found here.

3. Restart the master to load the new configuration.

Multiple Remotes¶

The gitfs_remotes option accepts an ordered list of git remotes to cache and search, in listed order, for requested files.

A simple scenario illustrates this cascading lookup behavior:

If the gitfs_remotes option specifies three remotes:

gitfs_remotes:
  - git://github.com/example/first.git
  - https://github.com/example/second.git
  - file:///root/third

And each repository contains some files:

first.git:
    top.sls
    edit/vim.sls
    edit/vimrc
    nginx/init.sls
    shell/init.sls

second.git:
    edit/dev_vimrc
    haproxy/init.sls
    shell.sls

third:
    haproxy/haproxy.conf
    edit/dev_vimrc

Salt will attempt to lookup the requested file from each gitfs remote repository in the order in which they are defined in the configuration. The git://github.com/example/first.git remote will be searched first. If the requested file is found, then it is served and no further searching is executed. For example:

• A request for the file salt://haproxy/init.sls will be served from the https://github.com/example/second.git git repo.

• A request for the file salt://haproxy/haproxy.conf will be served from the file:///root/third repo.

Also a requested state file overrules a directory with an init.sls-file. For example:

• A request for state.apply shell will be served from the https://github.com/example/second.git git repo.

Per-remote Configuration Parameters¶

The following master config parameters are global (that is, they apply to all configured gitfs remotes):

• gitfs_base

• gitfs_root

• gitfs_ssl_verify

• gitfs_mountpoint (new in 2014.7.0)

• gitfs_user (pygit2 only, new in 2014.7.0)

• gitfs_password (pygit2 only, new in 2014.7.0)

• gitfs_insecure_auth (pygit2 only, new in 2014.7.0)

• gitfs_pubkey (pygit2 only, new in 2014.7.0)

• gitfs_privkey (pygit2 only, new in 2014.7.0)

• gitfs_passphrase (pygit2 only, new in 2014.7.0)

• gitfs_refspecs (new in 2017.7.0)

• gitfs_disable_saltenv_mapping (new in 2018.3.0)

• gitfs_ref_types (new in 2018.3.0)

• gitfs_update_interval (new in 2018.3.0)

These parameters can now be overridden on a per-remote basis. This allows for a tremendous amount of customization. Here's some example usage:

gitfs_provider: pygit2
gitfs_base: develop

gitfs_remotes:
  - https://foo.com/foo.git
  - https://foo.com/bar.git:
    - root: salt
    - mountpoint: salt://bar
    - base: salt-base
    - ssl_verify: False
    - update_interval: 120
  - https://foo.com/bar.git:
    - name: second_bar_repo
    - root: other/salt
    - mountpoint: salt://other/bar
    - base: salt-base
    - ref_types:
      - branch
  - http://foo.com/baz.git:
    - root: salt/states
    - user: joe
    - password: mysupersecretpassword
    - insecure_auth: True
    - disable_saltenv_mapping: True
    - saltenv:
      - foo:
        - ref: foo
  - http://foo.com/quux.git:
    - all_saltenvs: master

In the example configuration above, the following is true:

1. The first and fourth gitfs remotes will use the develop branch/tag as the base environment, while the second and third will use the salt-base branch/tag as the base environment.

2. The first remote will serve all files in the repository. The second remote will only serve files from the salt directory (and its subdirectories). The third remote will only server files from the other/salt directory (and its subdirectories), while the fourth remote will only serve files from the salt/states directory (and its subdirectories).

3. The third remote will only serve files from branches, and not from tags or SHAs.

4. The fourth remote will only have two saltenvs available: base (pointed at develop), and foo (pointed at foo).

5. The first and fourth remotes will have files located under the root of the Salt fileserver namespace (salt://). The files from the second remote will be located under salt://bar, while the files from the third remote will be located under salt://other/bar.

6. The second and third remotes reference the same repository and unique names need to be declared for duplicate gitfs remotes.

7. The fourth remote overrides the default behavior of not authenticating to insecure (non-HTTPS) remotes.

8. Because all_saltenvs is configured for the fifth remote, files from the branch/tag master will appear in every fileserver environment.

   Note

   The use of http:// (instead of https://) is permitted here only because authentication is not being used. Otherwise, the insecure_auth parameter must be used (as in the fourth remote) to force Salt to authenticate to an http:// remote.

9. The first remote will wait 120 seconds between updates instead of 60.

Per-Saltenv Configuration Parameters¶

For more granular control, Salt allows the following three things to be overridden for individual saltenvs within a given repo:

• The mountpoint

• The root

• The branch/tag to be used for a given saltenv

Here is an example:

gitfs_root: salt

gitfs_saltenv:
  - dev:
    - mountpoint: salt://gitfs-dev
    - ref: develop

gitfs_remotes:
  - https://foo.com/bar.git:
    - saltenv:
      - staging:
        - ref: qa
        - mountpoint: salt://bar-staging
      - dev:
        - ref: development
  - https://foo.com/baz.git:
    - saltenv:
      - staging:
        - mountpoint: salt://baz-staging

Given the above configuration, the following is true:

1. For all gitfs remotes, files for the dev saltenv will be located under salt://gitfs-dev.

2. For the dev saltenv, files from the first remote will be sourced from the development branch, while files from the second remote will be sourced from the develop branch.

3. For the staging saltenv, files from the first remote will be located under salt://bar-staging, while files from the second remote will be located under salt://baz-staging.

4. For all gitfs remotes, and in all saltenvs, files will be served from the salt directory (and its subdirectories).

Custom Refspecs¶

GitFS will by default fetch remote branches and tags. However, sometimes it can be useful to fetch custom refs (such as those created for GitHub pull requests). To change the refspecs GitFS fetches, use the gitfs_refspecs config option:

gitfs_refspecs:
  - '+refs/heads/*:refs/remotes/origin/*'
  - '+refs/tags/*:refs/tags/*'
  - '+refs/pull/*/head:refs/remotes/origin/pr/*'
  - '+refs/pull/*/merge:refs/remotes/origin/merge/*'

In the above example, in addition to fetching remote branches and tags, GitHub's custom refs for pull requests and merged pull requests will also be fetched. These special head refs represent the head of the branch which is requesting to be merged, and the merge refs represent the result of the base branch after the merge.

Refspecs can be configured on a per-remote basis. For example, the below configuration would only alter the default refspecs for the second GitFS remote. The first remote would only fetch branches and tags (the default).

gitfs_remotes:
  - https://domain.tld/foo.git
  - https://domain.tld/bar.git:
    - refspecs:
      - '+refs/heads/*:refs/remotes/origin/*'
      - '+refs/tags/*:refs/tags/*'
      - '+refs/pull/*/head:refs/remotes/origin/pr/*'
      - '+refs/pull/*/merge:refs/remotes/origin/merge/*'

Global Remotes¶

The all_saltenvs per-remote configuration parameter overrides the logic Salt uses to map branches/tags to fileserver environments (i.e. saltenvs). This allows a single branch/tag to appear in all GitFS saltenvs.

The fallback global or per-remote configuration can also be used.

This is very useful in particular when working with salt formulas. Prior to the addition of this feature, it was necessary to push a branch/tag to the remote repo for each saltenv in which that formula was to be used. If the formula needed to be updated, this update would need to be reflected in all of the other branches/tags. This is both inconvenient and not scalable.

With all_saltenvs, it is now possible to define your formula once, in a single branch.

gitfs_remotes:
  - http://foo.com/quux.git:
    - all_saltenvs: anything

If you want to also test working branches of the formula repository, use fallback:

gitfs_remotes:
  - http://foo.com/quux.git:
    - fallback: anything

Update Intervals¶

Prior to the 2018.3.0 release, GitFS would update its fileserver backends as part of a dedicated "maintenance" process, in which various routine maintenance tasks were performed. This tied the update interval to the loop_interval config option, and also forced all fileservers to update at the same interval.

Now it is possible to make GitFS update at its own interval, using gitfs_update_interval:

gitfs_update_interval: 180

gitfs_remotes:
  - https://foo.com/foo.git
  - https://foo.com/bar.git:
    - update_interval: 120

Using the above configuration, the first remote would update every three minutes, while the second remote would update every two minutes.

Configuration Order of Precedence¶

The order of precedence for GitFS configuration is as follows (each level overrides all levels below it):

1. Per-saltenv configuration (defined under a per-remote saltenv param)

   gitfs_remotes:
     - https://foo.com/bar.git:
       - saltenv:
         - dev:
           - mountpoint: salt://bar
   

2. Global per-saltenv configuration (defined in gitfs_saltenv)

   gitfs_saltenv:
     - dev:
       - mountpoint: salt://bar
   

3. Per-remote configuration parameter

   gitfs_remotes:
     - https://foo.com/bar.git:
       - mountpoint: salt://bar
   

4. Global configuration parameter

   gitfs_mountpoint: salt://bar
   

Serving from a Subdirectory¶

The gitfs_root parameter allows files to be served from a subdirectory within the repository. This allows for only part of a repository to be exposed to the Salt fileserver.

Assume the below layout:

.gitignore
README.txt
foo/
foo/bar/
foo/bar/one.txt
foo/bar/two.txt
foo/bar/three.txt
foo/baz/
foo/baz/top.sls
foo/baz/edit/vim.sls
foo/baz/edit/vimrc
foo/baz/nginx/init.sls

The below configuration would serve only the files under foo/baz, ignoring the other files in the repository:

gitfs_remotes:
  - git://mydomain.com/stuff.git

gitfs_root: foo/baz

The root can also be configured on a per-remote basis.

Mountpoints¶

The gitfs_mountpoint parameter will prepend the specified path to the files served from gitfs. This allows an existing repository to be used, rather than needing to reorganize a repository or design it around the layout of the Salt fileserver.

Before the addition of this feature, if a file being served up via gitfs was deeply nested within the root directory (for example, salt://webapps/foo/files/foo.conf, it would be necessary to ensure that the file was properly located in the remote repository, and that all of the parent directories were present (for example, the directories webapps/foo/files/ would need to exist at the root of the repository).

The below example would allow for a file foo.conf at the root of the repository to be served up from the Salt fileserver path salt://webapps/foo/files/foo.conf.

gitfs_remotes:
  - https://mydomain.com/stuff.git

gitfs_mountpoint: salt://webapps/foo/files

Mountpoints can also be configured on a per-remote basis.

Using gitfs in Masterless Mode¶

Since 2014.7.0, gitfs can be used in masterless mode. To do so, simply add the gitfs configuration parameters (and set fileserver_backend) in the _minion_ config file instead of the master config file.

Using gitfs Alongside Other Backends¶

Sometimes it may make sense to use multiple backends; for instance, if sls files are stored in git but larger files are stored directly on the master.

The cascading lookup logic used for multiple remotes is also used with multiple backends. If the fileserver_backend option contains multiple backends:

fileserver_backend:
  - roots
  - git

Then the roots backend (the default backend of files in /srv/salt) will be searched first for the requested file; then, if it is not found on the master, each configured git remote will be searched.

file_roots:
  base:
    - /srv/salt
  __env__:
    - /srv/salt

Branches, Environments, and Top Files¶

When using the GitFS backend, branches, and tags will be mapped to environments using the branch/tag name as an identifier.

There is one exception to this rule: the master branch is implicitly mapped to the base environment.

So, for a typical base, qa, dev setup, the following branches could be used:

master
qa
dev

To map a branch other than master as the base environment, use the gitfs_base parameter.

gitfs_base: salt-base

The base can also be configured on a per-remote basis.

{{ saltenv }}:
  '*':
    - mystuff

manage_mystuff:
  pkg.installed:
    - name: mystuff
  file.managed:
    - name: /etc/mystuff.conf
    - source: salt://mystuff/files/mystuff.conf
  service.running:
    - name: mystuffd
    - enable: True
    - watch:
      - file: /etc/mystuff.conf

myminion:
    Data failed to compile:
----------
    Detected conflicting IDs, SLS IDs need to be globally unique.
    The conflicting ID is 'manage_mystuff' and is found in SLS 'base:mystuff' and SLS 'dev:mystuff'
----------
    Detected conflicting IDs, SLS IDs need to be globally unique.
    The conflicting ID is 'manage_mystuff' and is found in SLS 'dev:mystuff' and SLS 'qa:mystuff'

Environment Whitelist/Blacklist¶

The gitfs_saltenv_whitelist and gitfs_saltenv_blacklist parameters allow for greater control over which branches/tags are exposed as fileserver environments. Exact matches, globs, and regular expressions are supported, and are evaluated in that order. If using a regular expression, ^ and $ must be omitted, and the expression must match the entire branch/tag.

gitfs_saltenv_whitelist:
  - base
  - v1.*
  - 'mybranch\d+'

The behavior of the blacklist/whitelist will differ depending on which combination of the two options is used:

• If only gitfs_saltenv_whitelist is used, then only branches/tags which match the whitelist will be available as environments

• If only gitfs_saltenv_blacklist is used, then the branches/tags which match the blacklist will not be available as environments

• If both are used, then the branches/tags which match the whitelist, but do not match the blacklist, will be available as environments.

Authentication¶

gitfs_remotes:
  - https://domain.tld/myrepo.git:
    - user: git
    - password: mypassword

gitfs_remotes:
  - http://domain.tld/insecure_repo.git:
    - user: git
    - password: mypassword
    - insecure_auth: True

gitfs_remotes:
  - git@github.com:user/repo.git:
    - pubkey: /root/.ssh/id_rsa.pub
    - privkey: /root/.ssh/id_rsa
    - passphrase: myawesomepassphrase

gitfs_remotes:
  - https://git:mypassword@domain.tld/myrepo.git

machine domain.tld
login git
password mypassword

gitfs_remotes:
  - http://git:mypassword@domain.tld/insecure_repo.git:
    - insecure_auth: True

gitfs_remotes:
  - ssh://git@github.com/example/salt-states.git

Host github.com
    IdentityFile /root/.ssh/id_rsa_gitfs

Host github.com
    IdentityFile /root/.ssh/id_rsa_gitfs
    StrictHostKeyChecking no

# salt mymaster ssh.set_known_host user=root hostname=github.com
mymaster:
    ----------
    new:
        ----------
        enc:
            ssh-rsa
        fingerprint:
            16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48
        hostname:
            |1|OiefWWqOD4kwO3BhoIGa0loR5AA=|BIXVtmcTbPER+68HvXmceodDcfI=
        key:
            AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==
    old:
        None
    status:
        updated

$ su -
Password:
# ssh github.com
The authenticity of host 'github.com (192.30.252.128)' can't be established.
RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,192.30.252.128' (RSA) to the list of known hosts.
Permission denied (publickey).

$ nmap -p 22 github.com --script ssh-hostkey

Starting Nmap 5.51 ( http://nmap.org ) at 2014-08-18 17:47 CDT
Nmap scan report for github.com (192.30.252.129)
Host is up (0.17s latency).
Not shown: 996 filtered ports
PORT     STATE SERVICE
22/tcp   open  ssh
| ssh-hostkey: 1024 ad:1c:08:a4:40:e3:6f:9c:f5:66:26:5d:4b:33:5d:8c (DSA)
|_2048 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48 (RSA)
80/tcp   open  http
443/tcp  open  https
9418/tcp open  git

Nmap done: 1 IP address (1 host up) scanned in 28.78 seconds

$ ssh-keygen -l -f /dev/stdin <<<`ssh-keyscan github.com 2>/dev/null` | awk '{print $2}'
16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48

Refreshing gitfs Upon Push¶

By default, Salt updates the remote fileserver backends every 60 seconds. However, if it is desirable to refresh quicker than that, the Reactor System can be used to signal the master to update the fileserver on each push, provided that the git server is also a Salt minion. There are three steps to this process:

1. On the master, create a file /srv/reactor/update_fileserver.sls, with the following contents:

   update_fileserver:
     runner.fileserver.update
   

2. Add the following reactor configuration to the master config file:

   reactor:
     - 'salt/fileserver/gitfs/update':
       - /srv/reactor/update_fileserver.sls
   

3. On the git server, add a post-receive hook

   1. If the user executing git push is the same as the minion user, use the following hook:

   #!/usr/bin/env sh
   salt-call event.fire_master update salt/fileserver/gitfs/update
   

   2. To enable other git users to run the hook after a push, use sudo in the hook script:

   #!/usr/bin/env sh
   sudo -u root salt-call event.fire_master update salt/fileserver/gitfs/update
   

4. If using sudo in the git hook (above), the policy must be changed to permit all users to fire the event. Add the following policy to the sudoers file on the git server.

   Cmnd_Alias SALT_GIT_HOOK = /bin/salt-call event.fire_master update salt/fileserver/gitfs/update
   Defaults!SALT_GIT_HOOK !requiretty
   ALL ALL=(root) NOPASSWD: SALT_GIT_HOOK
   

The update argument right after event.fire_master in this example can really be anything, as it represents the data being passed in the event, and the passed data is ignored by this reactor.

Similarly, the tag name salt/fileserver/gitfs/update can be replaced by anything, so long as the usage is consistent.

The root user name in the hook script and sudo policy should be changed to match the user under which the minion is running.

Using Git as an External Pillar Source¶

The git external pillar (a.k.a. git_pillar) has been rewritten for the 2015.8.0 release. This rewrite brings with it pygit2 support (allowing for access to authenticated repositories), as well as more granular support for per-remote configuration. This configuration schema is detailed here.

Why aren't my custom modules/states/etc. syncing to my Minions?¶

In versions 0.16.3 and older, when using the git fileserver backend, certain versions of GitPython may generate errors when fetching, which Salt fails to catch. While not fatal to the fetch process, these interrupt the fileserver update that takes place before custom types are synced, and thus interrupt the sync itself. Try disabling the git fileserver backend in the master config, restarting the master, and attempting the sync again.

This issue is worked around in Salt 0.16.4 and newer.