Butane Translation#

Jinja Templating#

in addition to jinja inside butane files, files referenced with attribute template=jinja in

storage:files[].contents.template
systemd:units[].template
systemd:units[].dropins[].template

will be rendered through jinja with the described Environment and optional includes from searchpath.

Documentation:

Jinja Template Documentation

Environment#

The Environment available in jinja for the butane translation.

environment defaults available in jinja
any default environment can be overwritten to liking
HOSTNAME: the hostname argument will be set from the ButaneTranspiler
some defaults (DNS_RESOLVER, LOCALE) can be set too on a project config (lowercase names) eg.:

projectname:dns_resolver:
    upstream:
        - "10.10.10.1@53"
    upstream_tls: false

os/jinja_defaults.yml:

# Jinja Default Environment
# HOSTNAME: will be set to hostname from Butanetranspiler

# Default locale (lang, keymap, timezone, country_code)
LOCALE:
  LANG: en_US.UTF-8
  KEYMAP: us
  TIMEZONE: UTC
  COUNTRY_CODE: UN

# DNS Upstream Resolver for internal unbound dns server
#   that serves to system, container and nspawn workloads
DNS_RESOLVER:
  # VERBOSITY: 0, 1, 2, 3, 4, > more verbose
  VERBOSITY: "1"
  # UPSTREAM_TLS: boolean, defaults: true
  # UPSTREAM: list, defaults: None
  #   if undefined, will set to tls split upstream using 2x google, 2x adguard, 2xcloudflare
  #   example upstream list entry: - 1.2.3.4@567#dns.domain
  # FORWARD: list, defaults: None
  #   example forward list entry: - name: zone, addr: addr, tls: true/false
  # SRV: multiline-string, custom text appended to unbound.conf,
  #   placed _inside_ the server section, eg. zone entries and public dns overrides
  # EXTRA: multiline-string, custom text appended to unbound.conf,
  #   must start with a new section.

# FRONTEND:ENABLED true/false, if true traefik does tls termination and http routing
FRONTEND:
  ENABLED: true
  # VERBOSITY: uppercase any of TRACE, DEBUG, INFO, WARN, ERROR, FATAL, PANIC
  VERBOSITY: INFO
  # DASHBOARD: optional string, set to unique hostname,
  #   eg. "traefik.host.domain" to enable traefik debug dashboard
  DASHBOARD:
  # NETWORKS: optional list, defines additional networks for the frontend to connect to
  NETWORKS: []

# Dedicated DNS Server (Knot) for .internal, .podman, .nspawn and reverse ptr
LOCAL_DNS_SERVER:
  # if enabled:  unbound will forward .internal requests to knot to answer requests
  # if disabled: unbound will answer for .internal requests itself, but without dnssec
  ENABLED: false
  # VERBOSITY: debug, info, notice. warning, error, critical, < more verbose
  VERBOSITY: info
  # # optional zones:
  #     place dns data into /etc/local/knot/<zonename>.zone
  # # optional entries:
  #     TEMPLATE, POLICY, KEY, ACL, ZONE: multiline | string,
  #       must be formatted as list "-", will be added to defined section

# Dedicated ACME Certificate Server (step-ca) for *.on.internal
LOCAL_ACME_SERVER:
  # if enabled, step-ca will be used for maintaining on.internal TLS
  #   certificates, needs LOCAL_DNS_SERVER to be enabled too.
  ENABLED: false
  # HOSTNAMES: list, under which hostnames acme provision will be available
  HOSTNAMES:
    - acme.internal
  # DOMAINS: list, which domains are available for acme to provision on
  DOMAINS:
    - on.internal
  # - at.internal

# Additional RPM packets to be available
RPM_OSTREE_INSTALL:
  # unbound, recursive dnsresolver for system, container, compose and nspawn workloads
  - unbound
  # knot, authoritative DNS server for .internal, .podman, .nspawn domains
  - knot
  # podman-compose, used by compose.bu for compose container
  - podman-compose
  # systemd-networkd, used by nspawn.bu for nspawn container network setup
  - systemd-networkd
  # firewalld, use by firewall.bu for firewall rules
  - firewalld
  # clevis, use by clevis-luks-slots.py for rekeying luks slots with new secrets
  - clevis
  - clevis-pin-tpm2

# Fedora-CoreOS stream, architecture, platform and format defaults
FCOS:
  STREAM: testing
  ARCHITECTURE: x86_64
  PLATFORM: metal
  FORMAT: raw.xz

# Network CIDR of Internal, Podman and Nspawn Networks
INTERNAL_CIDR: 10.87.240.1/24
NSPAWN_CIDR: 10.87.241.1/24
PODMAN_CIDR: 10.88.0.1/16
PODMAN_POOL_CIDR: 10.89.0.1/16
# POOL is used from 0 to 99, 100 to 254 are free to use for PODMAN_STATIC_NETWORKS
LIBVIRT_CIDR: 192.168.122.1/24

# firewalld configuring Internal, Podman and Nspawn network traffic
FIREWALL:
  # if disabled: firewalld will not be started
  ENABLED: true
  # LOG_DENIED: true/false, if true: firewalld writes rejects/drops into systemd journal
  LOG_DENIED: true

# DEBUG_CONSOLE_AUTOLOGIN: true/false, if true: add autologin on serial console
DEBUG_CONSOLE_AUTOLOGIN: false

# DEBUG_DISABLE_AUTOUPDATE: true/false, if true: disable autoupdate, keep specific version
DEBUG_DISABLE_AUTOUPDATE: false

# API_PROXY_LOG_HTTP: if true: Write HTTP Proxy Access to Log
API_PROXY_LOG_HTTP: false

# PODMAN_LOAD_SECRETS: default (""), string list of /etc/credstore entries
#   to load into podman for container and compose workloads.
#   "" means to load all secrets in /etc/credstore into podman secrets.
#   example: PODMAN_LOAD_SECRETS: "root_ca.crt root_bundle.crt"
PODMAN_LOAD_SECRETS: ""
# PODMAN_API_LOGGING: str, one of trace debug info warn warning error fatal panic
PODMAN_API_LOG_LEVEL: warning
# PODMAN_STATIC_NETWORKS: additional networks for podman to be configured
PODMAN_STATIC_NETWORKS:
  # PODMAN_POOL is used from 0 to 99, 100 to 254 are free to use for PODMAN_STATIC_NETWORKS
  pgmtls: 10.89.128.1/24
  pgpwd: 10.89.129.1/24

# common update parameter defaults
#   connect as user core (uid=1000), use "update-system-config" as the update service,
#   expect files related to update on /run/user/1000, use sudo to become root.
UPDATE_USER: core
# XXX UPDATE_UID must be an integer
UPDATE_UID: 1000
UPDATE_SERVICE: update-system-config
UPDATE_PATH: /run/user/1000
UPDATE_USE_SUDO: true
# UPDATE_CLEVIS_LUKS_SLOTS: true/false, if enabled, update/rekey LUKS Clevis bindings
#   according to a butane configuration of boot_device:luks and storage:luks
UPDATE_CLEVIS_LUKS_SLOTS: false

available custom filter and functions#

Regex#

"text"|regex_escape()
"text"|regex_search(pattern, ignorecase=False, multiline=False)
"text"|regex_match(pattern, ignorecase=False, multiline=False)
"text"|regex_replace(pattern, replacement, ignorecase=False, multiline=False)

IPv4 Address Manipulation#

"192.168.1.0/24"|cidr2ip(index=0)
- Converts a CIDR notation to an IP address.
- Args:
  - cidr (str): The CIDR notation (e.g., “192.168.1.0/24”).
  - index (int, optional): The 0-based index of the usable IP address to return.
"10.87.240.1/24"|cidr2reverse_ptr()"
- Converts an IPv4 or IPv6 CIDR string into its corresponding reverse DNS zone name.
- Args:
  - cidr_string: The network address in CIDR notation, e.g., “10.87.240.1/24”. Host bits in the address are ignored
- Returns:
  - str: The reverse DNS zone name as a string (e.g., “240.87.10.in-addr.arpa.”)
- Raises:
  - ValueError: If the CIDR is invalid

YAML Output#

{"key": "value"}|toyaml(inline=False)
- dump dict structure to yaml string, same as jinja buildin tojson but outputs yaml instead of json
- set inline=True for compact representation, default is multiline

Hashing#

string|sha256sum
- calculates the SHA256 hash of a string and returns the hexadecimal representation

Time Now and File Time#

"os/podman.bu"|created_at()-> datetime.datetime | None
- Return file modification datetime in UTC or None
- Expects a relative path from the template search path
utc_now()-> datetime.datetime
- Return file modification datetime in UTC or None
local_now()-> datetime.datetime
- Return file modification datetime in local Timezone or None

in jinja, you can then call all python methods on the datetime.datetime objects available.

Butane Yaml Creation#

the butane configuration is created from

ButaneTranspiler(butane_input, basedir, environment)

Name	source	searchpath
`base_dict`	butane_input:string	basedir
`security_dict`	generated:string	basedir
`system_dict`	`*.bu` yaml	basedir + `/infra/os`
`target_dict`	`*.bu` yaml	targetdir

Merge Order#

merged_dict = system_dict+ target_dict+ security_dict+ base_dict
- order is earlier gets overwritten by later

for each “*.bu” in basedir+”infra/os”, targetdir#

*.bu recursive read and execute jinja with environment available
parse result as yaml
inline all local references
- for files and trees use source with base64 encode if file type = binary

source	dest
`storage:trees:[name]:local`	`files:[name]:contents:inline/source`
`storage:files:[name]:contents:local`	`[name]:contents:inline/source`
`systemd:units:[name]:contents_local`	`[name]:contents`
`systemd:units:[name]:dropins:[other]:contents_local`	`[other]:contents`

apply additional filter where template != “”
- storage:files:[name]:contents:template
- systemd:units:[name]:template
- systemd:units:[name]:dropins:[name]:template
merge together

Ignition Json Creation#

the ignition spec file is created from the merged final butane yaml.

Saltstack Yaml Creation#

The saltstack file is created from the resulting merged final butane yaml. It meets all restrictions for the saltstack conversion.

this_dir/update-system-config.sls and basedir/*.slsare appended to input

Restrictions:

only storage:directories/links/files and systemd:units[:dropins] are translated
files must be inlined, files:contents must be of type inline or source (base64 encoded)
systemd:units and systemd:units:dropins must be of type contents

Translation:

Files [/etc/hosts, /etc/hostname, /etc/resolv.conf] are translated to /host_etc/*

Changed Services:

execution creates a commented, non uniqe, not sorted list of service base names
- update_dir=/run/user/1000/update-system-config
- {update_dir}/service_changed.list for services with changed configuration
- {update_dir}/service_enabled.list for services to be enabled
- {update_dir}/service_disabled.list for services to be disabled

see update-system-config.service for detailed usage of service_*

Notes:

podman-systemd, compose.yml and nspawn container: share the same namespace for service change recognition and should therefore not share the same name
podman-systemd container config support files (beside .container and .volume), should also start with the servicename as part of the filename, to be recognized