salt.fileserver.s3fs¶

Amazon S3 Fileserver Backend

New in version 0.16.0.

This backend exposes directories in S3 buckets as Salt environments. To enable this backend, add s3fs to the fileserver_backend option in the Master config file.

fileserver_backend:
  - s3fs

S3 credentials must also be set in the master config file:

s3.keyid: GKTADJGHEIQSXMKKRBJ08H
s3.key: askdjghsdfjkghWupUjasdflkdfklgjsdfjajkghs

Alternatively, if on EC2 these credentials can be automatically loaded from instance metadata.

This fileserver supports two modes of operation for the buckets:

A single bucket per environment

s3.buckets:
  production:
    - bucket1
    - bucket2
  staging:
    - bucket3
    - bucket4

Multiple environments per bucket

s3.buckets:
  - bucket1
  - bucket2
  - bucket3
  - bucket4

Note that bucket names must be all lowercase both in the AWS console and in Salt, otherwise you may encounter SignatureDoesNotMatch errors.

A multiple-environment bucket must adhere to the following root directory structure:

s3://<bucket name>/<environment>/<files>

Note

This fileserver back-end requires the use of the MD5 hashing algorithm. MD5 may not be compliant with all security policies.

Note

This fileserver back-end is only compatible with MD5 ETag hashes in the S3 metadata. This means that you must use SSE-S3 or plaintext for bucket encryption, and that you must not use multipart upload when uploading to your bucket. More information here: https://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonResponseHeaders.html

Objects without an MD5 ETag will be fetched on every fileserver update.

If you deal with objects greater than 8MB, then you should use the following AWS CLI config to avoid mutipart upload:

s3 =
  multipart_threshold = 1024MB

More info here: https://docs.aws.amazon.com/cli/latest/topic/s3-config.html

Note

This fileserver back-end will by default sync all buckets on every fileserver update.

If you want files to be only populated in the cache when requested, you can disable this in the master config:

s3.s3_sync_on_update: False