salt.modules.highstate_doc¶

This module renders highstate configuration into a more human readable format.

How it works:

highstate or lowstate data is parsed with a processor this defaults to highstate_doc.processor_markdown. The processed data is passed to a jinja template that builds up the document content.

configuration: Pillar

# the following defaults can be overridden
highstate_doc.config:

    # list of regex of state names to ignore in `highstate_doc.process_lowstates`
    filter_id_regex:
        - '.*!doc_skip$'

    # list of regex of state functions to ignore in `highstate_doc.process_lowstates`
    filter_state_function_regex:
        - 'file.accumulated'

    # dict of regex to replace text after `highstate_doc.render`. (remove passwords)
    text_replace_regex:
        'password:.*^': '[PASSWORD]'

    # limit size of files that can be included in doc (10000 bytes)
    max_render_file_size: 10000

    # advanced option to set a custom lowstate processor
    processor: highstate_doc.processor_markdown

State example

{{sls}} note:
    highstate_doc.note:
        - name: example
        - order: 0
        - contents: |
            example `highstate_doc.note`
            ------------------
            This state does not do anything to the system! It is only used by a `processor`
            you can use `requisites` and `order` to move your docs around the rendered file.

{{sls}} a file we don't want in the doc !doc_skip:
    file.managed:
        - name: /root/passwords
        - contents: 'password: sadefgq34y45h56q'
        # also could use `highstate_doc.config: text_replace_regex` to replace
        # password string. `password:.*^': '[PASSWORD]`

To create the help document build a State that uses highstate_doc.render. For performance it's advised to not included this state in your top.sls file.

# example `salt://makereadme.sls`
make helpfile:
    file.managed:
        - name: /root/README.md
        - contents: {{salt.highstate_doc.render()|json}}
        - show_diff: {{opts['test']}}
        - mode: '0640'
        - order: last

Run our makereadme.sls state to create /root/README.md.

# first ensure `highstate` return without errors or changes
salt-call state.highstate
salt-call state.apply makereadme
# or if you don't want the extra `make helpfile` state
salt-call --out=newline_values_only salt.highstate_doc.render > /root/README.md ; chmod 0600 /root/README.md

salt '*' state.apply makereadme

#!/bin/python
import os
import salt.client
s = salt.client.LocalClient()
# NOTE: because of issues with `cp.push` use `highstate_doc.read_file`
o = s.cmd('*', 'highstate_doc.read_file', ['/root/README.md'])
for m in o:
    d = o.get(m)
    if d and not d.endswith('is not available.'):
        # mkdir m
        #directory = os.path.dirname(file_path)
        if not os.path.exists(m):
            os.makedirs(m)
        with open(m + '/README.md','wb') as f:
            f.write(d)
        print('ADDED: ' + m + '/README.md')

# process all the readme.md files to readme.html
if which pandoc; then echo "Found pandoc"; else echo "** Missing pandoc"; exit 1; fi
if which gs; then echo "Found gs"; else echo "** Missing gs(ghostscript)"; exit 1; fi
readme_files=$(find $dest -type f -path "*/README.md" -print)
for f in $readme_files ; do
    ff=${f#$dest/}
    minion=${ff%%/*}
    echo "process: $dest/${minion}/$(basename $f)"
    cat $dest/${minion}/$(basename $f) |             pandoc --standalone --from markdown_github --to html             --include-in-header $dest/style.html             > $dest/${minion}/$(basename $f).html
done

# you could also create a new `processor` for perhaps reStructuredText
# highstate_doc.config:
#     processor: doc_custom.processor_rst

# example `salt://makereadme.jinja`
"""
{{opts['id']}}
==========================================

{# lowstates is set from highstate_doc.render() #}
{# if lowstates is missing use salt.highstate_doc.process_lowstates() #}
{% for s in lowstates %}
{{s.id}}
-----------------------------------------------------------------
{{s.function}}

{{s.markdown.requisite}}
{{s.markdown.details}}

{%- endfor %}
"""

# example `salt://makereadme.sls`
{% import_text "makereadme.jinja" as makereadme %}
{{sls}} or:
    file.managed:
        - name: /root/README_other.md
        - contents: {{salt.highstate_doc.render(jinja_template_text=makereadme)|json}}
        - mode: '0640'

CERTS
-----

``'-----BEGIN RSA PRIVATE KEY-----[\r\n\t\f\S]{0,2200}': 'XXXXXXX'``
``'-----BEGIN CERTIFICATE-----[\r\n\t\f\S]{0,2200}': 'XXXXXXX'``
``'-----BEGIN DH PARAMETERS-----[\r\n\t\f\S]{0,2200}': 'XXXXXXX'``
``'-----BEGIN PRIVATE KEY-----[\r\n\t\f\S]{0,2200}': 'XXXXXXX'``
``'-----BEGIN OPENSSH PRIVATE KEY-----[\r\n\t\f\S]{0,2200}': 'XXXXXXX'``
``'ssh-rsa .* ': 'ssh-rsa XXXXXXX '``
``'ssh-dss .* ': 'ssh-dss XXXXXXX '``

DB
--

``'DB_PASS.*': 'DB_PASS = XXXXXXX'``
``'5432:*:*:.*': '5432:*:XXXXXXX'``
``"'PASSWORD': .*": "'PASSWORD': 'XXXXXXX',"``
``" PASSWORD '.*'": " PASSWORD 'XXXXXXX'"``
``'PGPASSWORD=.* ': 'PGPASSWORD=XXXXXXX'``
``"_replication password '.*'":  "_replication password 'XXXXXXX'"``

OTHER
-----

``'EMAIL_HOST_PASSWORD =.*': 'EMAIL_HOST_PASSWORD =XXXXXXX'``
``"net ads join -U '.*@MFCFADS.MATH.EXAMPLE.CA.* ": "net ads join -U '.*@MFCFADS.MATH.EXAMPLE.CA%XXXXXXX "``
``"net ads join -U '.*@NEXUS.EXAMPLE.CA.* ": "net ads join -U '.*@NEXUS.EXAMPLE.CA%XXXXXXX "``
``'install-uptrack .* --autoinstall': 'install-uptrack XXXXXXX --autoinstall'``
``'accesskey = .*': 'accesskey = XXXXXXX'``
``'auth_pass .*': 'auth_pass XXXXXXX'``
``'PSK "0x.*': 'PSK "0xXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'``
``'SECRET_KEY.*': 'SECRET_KEY = XXXXXXX'``
``"password=.*": "password=XXXXXXX"``
``'<password>.*</password>': '<password>XXXXXXX</password>'``
``'<salt>.*</salt>': '<salt>XXXXXXX</salt>'``
``'application.secret = ".*"': 'application.secret = "XXXXXXX"'``
``'url = "postgres://.*"': 'url = "postgres://XXXXXXX"'``
``'PASS_.*_PASS': 'PASS_XXXXXXX_PASS'``

HTACCESS
--------

``':{PLAIN}.*': ':{PLAIN}XXXXXXX'``

salt.modules.highstate_doc.markdown_basic_jinja_template(**kwargs)¶

Return text for a simple markdown jinja template

This function can be used from the highstate_doc.render modules jinja_template_function option.

salt.modules.highstate_doc.markdown_default_jinja_template(**kwargs)¶

Return text for a markdown jinja template that included a header

This function can be used from the highstate_doc.render modules jinja_template_function option.

salt.modules.highstate_doc.markdown_full_jinja_template(**kwargs)¶

Return text for an advanced markdown jinja template

This function can be used from the highstate_doc.render modules jinja_template_function option.

salt.modules.highstate_doc.process_lowstates(**kwargs)¶

return processed lowstate data that was not blacklisted

render_module_function is used to provide your own. defaults to from_lowstate

salt.modules.highstate_doc.processor_markdown(lowstate_item, config, **kwargs)¶

Takes low state data and returns a dict of processed data that is by default used in a jinja template when rendering a markdown highstate_doc.

This lowstate_item_markdown given a lowstate item, returns a dict like:

vars:       # the raw lowstate_item that was processed
id:         # the 'id' of the state.
id_full:    # combo of the state type and id "state: id"
state:      # name of the salt state module
function:   # name of the state function
name:       # value of 'name:' passed to the salt state module
state_function:    # the state name and function name
markdown:          # text data to describe a state
    requisites:    # requisite like [watch_in, require_in]
    details:       # state name, parameters and other details like file contents

salt.modules.highstate_doc.read_file(name)¶

output the contents of a file:

this is a workaround if the cp.push module does not work. https://github.com/saltstack/salt/issues/37133

help the master output the contents of a document that might be saved on the minions filesystem.

#!/bin/python
import os
import salt.client
s = salt.client.LocalClient()
o = s.cmd('*', 'highstate_doc.read_file', ['/root/README.md'])
for m in o:
    d = o.get(m)
    if d and not d.endswith('is not available.'):
        # mkdir m
        #directory = os.path.dirname(file_path)
        if not os.path.exists(m):
            os.makedirs(m)
        with open(m + '/README.md','wb') as fin:
            fin.write(d)
        print('ADDED: ' + m + '/README.md')

salt.modules.highstate_doc.render(jinja_template_text=None, jinja_template_function='highstate_doc.markdown_default_jinja_template', **kwargs)¶

Render highstate to a text format (default Markdown)

if jinja_template_text is not set, jinja_template_function is used.

jinja_template_text: jinja text that the render uses to create the document. jinja_template_function: a salt module call that returns template text.

Options:

highstate_doc.markdown_basic_jinja_template highstate_doc.markdown_default_jinja_template highstate_doc.markdown_full_jinja_template