HTTP Modules¶

HTTPS Request Methods¶

A basic query can be performed by calling this function with no more than a single URL:

salt.utils.http.query("http://example.com")

By default the query will be performed with a GET method. The method can be overridden with the method argument:

salt.utils.http.query("http://example.com/delete/url", "DELETE")

When using the POST method (and others, such as PUT), extra data is usually sent as well. This data can be sent directly (would be URL encoded when necessary), or in whatever format is required by the remote server (XML, JSON, plain text, etc).

salt.utils.http.query(
    "http://example.com/post/url", method="POST", data=json.dumps(mydict)
)

Data Formatting and Templating¶

Bear in mind that the data must be sent pre-formatted; this function will not format it for you. However, a templated file stored on the local system may be passed through, along with variables to populate it with. To pass through only the file (untemplated):

salt.utils.http.query(
    "http://example.com/post/url", method="POST", data_file="/srv/salt/somefile.xml"
)

To pass through a file that contains jinja + yaml templating (the default):

salt.utils.http.query(
    "http://example.com/post/url",
    method="POST",
    data_file="/srv/salt/somefile.jinja",
    data_render=True,
    template_dict={"key1": "value1", "key2": "value2"},
)

To pass through a file that contains mako templating:

salt.utils.http.query(
    "http://example.com/post/url",
    method="POST",
    data_file="/srv/salt/somefile.mako",
    data_render=True,
    data_renderer="mako",
    template_dict={"key1": "value1", "key2": "value2"},
)

Because this function uses Salt's own rendering system, any Salt renderer can be used. Because Salt's renderer requires __opts__ to be set, an opts dictionary should be passed in. If it is not, then the default __opts__ values for the node type (master or minion) will be used. Because this library is intended primarily for use by minions, the default node type is minion. However, this can be changed to master if necessary.

salt.utils.http.query(
    "http://example.com/post/url",
    method="POST",
    data_file="/srv/salt/somefile.jinja",
    data_render=True,
    template_dict={"key1": "value1", "key2": "value2"},
    opts=__opts__,
)

salt.utils.http.query(
    "http://example.com/post/url",
    method="POST",
    data_file="/srv/salt/somefile.jinja",
    data_render=True,
    template_dict={"key1": "value1", "key2": "value2"},
    node="master",
)

Headers¶

Headers may also be passed through, either as a header_list, a header_dict, or as a header_file. As with the data_file, the header_file may also be templated. Take note that because HTTP headers are normally syntactically-correct YAML, they will automatically be imported as an a Python dict.

salt.utils.http.query(
    "http://example.com/delete/url",
    method="POST",
    header_file="/srv/salt/headers.jinja",
    header_render=True,
    header_renderer="jinja",
    template_dict={"key1": "value1", "key2": "value2"},
)

Because much of the data that would be templated between headers and data may be the same, the template_dict is the same for both. Correcting possible variable name collisions is up to the user.

Authentication¶

The query() function supports basic HTTP authentication. A username and password may be passed in as username and password, respectively.

salt.utils.http.query("http://example.com", username="larry", password="5700g3543v4r")

Cookies and Sessions¶

Cookies are also supported, using Python's built-in cookielib. However, they are turned off by default. To turn cookies on, set cookies to True.

salt.utils.http.query("http://example.com", cookies=True)

By default cookies are stored in Salt's cache directory, normally /var/cache/salt, as a file called cookies.txt. However, this location may be changed with the cookie_jar argument:

salt.utils.http.query(
    "http://example.com", cookies=True, cookie_jar="/path/to/cookie_jar.txt"
)

By default, the format of the cookie jar is LWP (aka, lib-www-perl). This default was chosen because it is a human-readable text file. If desired, the format of the cookie jar can be set to Mozilla:

salt.utils.http.query(
    "http://example.com",
    cookies=True,
    cookie_jar="/path/to/cookie_jar.txt",
    cookie_format="mozilla",
)

Because Salt commands are normally one-off commands that are piped together, this library cannot normally behave as a normal browser, with session cookies that persist across multiple HTTP requests. However, the session can be persisted in a separate cookie jar. The default filename for this file, inside Salt's cache directory, is cookies.session.p. This can also be changed.

salt.utils.http.query(
    "http://example.com", persist_session=True, session_cookie_jar="/path/to/jar.p"
)

The format of this file is msgpack, which is consistent with much of the rest of Salt's internal structure. Historically, the extension for this file is .p. There are no current plans to make this configurable.

Proxy¶

If the tornado backend is used (tornado is the default), proxy information configured in proxy_host, proxy_port, proxy_username, proxy_password and no_proxy from the __opts__ dictionary will be used. Normally these are set in the minion configuration file.

proxy_host: proxy.my-domain
proxy_port: 31337
proxy_username: charon
proxy_password: obolus
no_proxy: ['127.0.0.1', 'localhost']

salt.utils.http.query("http://example.com", opts=__opts__, backend="tornado")

Updating CA Bundles¶

The update_ca_bundle() function can be used to update the bundle file at a specified location. If the target location is not specified, then it will attempt to auto-detect the location of the bundle file. If the URL to download the bundle from does not exist, a bundle will be downloaded from the cURL website.

CAUTION: The target and the source should always be specified! Failure to specify the target may result in the file being written to the wrong location on the local system. Failure to specify the source may cause the upstream URL to receive excess unnecessary traffic, and may cause a file to be download which is hazardous or does not meet the needs of the user.

salt.utils.http.update_ca_bundle(
    target="/path/to/ca-bundle.crt",
    source="https://example.com/path/to/ca-bundle.crt",
    opts=__opts__,
)

The opts parameter should also always be specified. If it is, then the target and the source may be specified in the relevant configuration file (master or minion) as ca_bundle and ca_bundle_url, respectively.

ca_bundle: /path/to/ca-bundle.crt
ca_bundle_url: https://example.com/path/to/ca-bundle.crt

If Salt is unable to auto-detect the location of the CA bundle, it will raise an error.

The update_ca_bundle() function can also be passed a string or a list of strings which represent files on the local system, which should be appended (in the specified order) to the end of the CA bundle file. This is useful in environments where private certs need to be made available, and are not otherwise reasonable to add to the bundle file.

salt.utils.http.update_ca_bundle(
    opts=__opts__,
    merge_files=[
        "/etc/ssl/private_cert_1.pem",
        "/etc/ssl/private_cert_2.pem",
        "/etc/ssl/private_cert_3.pem",
    ],
)