Container-Based Steps¶

Overview¶

The Container Hardware Manager in ironic-python-agent (IPA) allows running OCI-compatible containers as steps on bare metal nodes. This enables operators to package arbitrary tools – firmware updaters, diagnostic suites, compliance scanners – as container images and execute them during any step-based workflow, such as cleaning, deployment, or servicing.

Basics¶

A workflow for implementing container-based steps with runbooks is:

The operator builds an IPA ramdisk with the ironic-python-agent-podman diskimage-builder element, which installs podman and the Container Hardware Manager into the ramdisk.
The Ironic conductor sends [agent_containers] configuration to IPA via the lookup/heartbeat endpoint. This allows conductor-side settings to override any build-time defaults in the ramdisk.
When a runbook triggers a generic_container_step, IPA uses podman (or docker) to pull and run the specified container image on the bare metal node.
The container runs with host networking by default, executes its task, and exits. IPA reports the result back to the conductor.

Prerequisites¶

IPA ramdisk with podman support¶

The IPA ramdisk must be built with the ironic-python-agent-podman diskimage-builder (DIB) element. This element is currently Debian-based only.

export DIB_ALLOW_ARBITRARY_CONTAINERS=true
export DIB_RUNNER=podman

disk-image-create ironic-python-agent-ramdisk \
    ironic-python-agent-podman \
    debian -o ipa-with-podman

Key DIB environment variables:

DIB_ALLOW_ARBITRARY_CONTAINERS: Set to true to allow any container image. Set to false (default) to restrict to a specific allowlist. Environments which permit non-admin roles to create and execute runbooks should not set this to true for security reasons.
DIB_ALLOWED_CONTAINERS: Comma-separated list of allowed container image URLs. Only used when DIB_ALLOW_ARBITRARY_CONTAINERS is false.
DIB_RUNNER: Container runtime: podman (default) or docker.

Container registry access¶

The container registry hosting your images must be accessible from the cleaning network. If using a private registry, ensure credentials and TLS certificates are configured in the ramdisk or passed via pull_options.

Ironic Conductor Configuration¶

The [agent_containers] configuration group controls how the conductor instructs IPA to handle containers. These settings are sent to IPA at lookup time, so changes take effect without rebuilding the ramdisk.

[agent_containers]
# Allow any container image (default: false)
allow_arbitrary_containers = false

# Allowlist of container images (used when above is false)
allowed_containers = docker://registry.example.com/firmware-tool:latest,docker://registry.example.com/diag-suite:v2

# Container runtime (default: podman)
runner = podman

# Options passed to the pull command (comma separated list, empty
# by default)
pull_options =

# Options passed to the run command (comma separated list)
run_options = --rm,--network=host

Important

pull_options and run_options are comma separated lists, not space separated strings. --rm --network=host is a single option containing spaces, which the container runtime will reject.

Warning

Setting allow_arbitrary_containers = true allows any container image to be pulled and executed with host-level network access on the bare metal node. Only enable this in trusted environments. Prefer using allowed_containers to maintain an explicit allowlist.

Example Container-based Runbooks¶

The built-in step¶

The Container Hardware Manager exposes a built-in step called generic_container_step on the deploy interface. This step has a default priority of 0, meaning it only runs when explicitly invoked via manual cleaning, servicing, or a runbook.

Note

This step was previously called container_clean_step. That name is deprecated but still works, so existing runbooks keep functioning; it will be removed in a future release. The step serves cleaning, deployment and servicing alike, which is why the name no longer says “clean”.

The step accepts the following arguments:

container_url (required): The full container image URL, e.g. docker://registry.example.com/firmware-tool:latest. Unless allow_arbitrary_containers is enabled, this must appear in allowed_containers.
pull_options (optional): Override the default pull options for this specific container. Ignored when allow_arbitrary_containers is false. See Security Considerations.
run_options (optional): Override the default run options for this specific container. Ignored when allow_arbitrary_containers is false. See Security Considerations.

Single-container runbook¶

This example creates a runbook that runs a single firmware update container:

baremetal runbook create \
    --name CUSTOM_CONTAINER_FW_UPDATE \
    --steps '[
        {
            "interface": "deploy",
            "step": "generic_container_step",
            "args": {
                "container_url": "docker://registry.example.com/firmware-tool:latest"
            },
            "order": 1
        }
    ]'

Multi-container runbook¶

Runbooks can combine multiple container steps with traditional steps. This example runs a diagnostic container, then a firmware updater, and finishes with a standard disk metadata erase:

baremetal runbook create \
    --name CUSTOM_CONTAINER_CLEAN \
    --steps '[
        {
            "interface": "deploy",
            "step": "generic_container_step",
            "args": {
                "container_url": "docker://registry.example.com/diag-suite:v2"
            },
            "order": 1
        },
        {
            "interface": "deploy",
            "step": "generic_container_step",
            "args": {
                "container_url": "docker://registry.example.com/firmware-tool:latest",
                "run_options": ["--rm", "--network=host", "--privileged"]
            },
            "order": 2
        },
        {
            "interface": "deploy",
            "step": "erase_devices_metadata",
            "args": {},
            "order": 3
        }
    ]'

Note

The run_options in the second step above only take effect when allow_arbitrary_containers is true. In the default locked down configuration the conductor’s configured run_options are used instead, because options such as --privileged would otherwise let a runbook escape the allowed_containers allowlist entirely.

Adding traits to nodes¶

Runbooks are matched to nodes via traits. Add the matching trait to all nodes that should use the runbook:

baremetal node add trait <node> CUSTOM_CONTAINER_CLEAN

Using the Runbook¶

Manual cleaning¶

Trigger the runbook on a node in manageable state:

baremetal node clean <node> --runbook CUSTOM_CONTAINER_CLEAN

Automated cleaning¶

To use container-based steps for automated cleaning, configure the conductor to use runbook-based or hybrid cleaning and assign the runbook. See Configuring automated cleaning with runbooks for full details on the available configuration levels (per-node, per-resource-class, global).

A minimal example using the global default:

[conductor]
automated_clean = true
automated_cleaning_step_source = runbook
automated_cleaning_runbook = CUSTOM_CONTAINER_CLEAN

All nodes must have the matching trait (CUSTOM_CONTAINER_CLEAN) unless trait validation is disabled via conductor.automated_cleaning_runbook_validate_traits.

Servicing¶

Container steps also work with Node servicing. Trigger a container runbook on an active node:

baremetal node service <node> --runbook CUSTOM_CONTAINER_CLEAN

Alternative Methods¶

Operators may utilize container-based steps that are hardcoded via configuration in-ramdisk.

Ironic-python-agent can be configured to expose arbitrary steps using containers for use in workflows, including automated cleaning, via a yaml configuration file.

For example:

steps:
  - name: manage_container_cleanup
    image: docker://172.24.4.1:5000/cleaning-image:latest
    interface: deploy
    reboot_requested: true
    pull_options:
      - --tls-verify=false
    run_options:
      - --rm
      - --network=host
      - --tls-verify=false
    abortable: true
    priority: 20
  - name: manage_container_cleanup2
    image: docker://172.24.4.1:5000/cleaning-image2:latest
    interface: deploy
    reboot_requested: true
    pull_options:
      - --tls-verify=false
    run_options:
      - --rm
      - --network=host
      - --tls-verify=false
    abortable: true
    priority: 10

By placing a file in your IPA ramdisk with these contents in the path indicated by agent_containers.container_steps_file, cleaning steps manage_container_cleanup and manage_container_cleanup2 will be reported as available cleaning steps at the indicated priority.

Note

priority applies to automated cleaning only. The same steps are offered for deployment and servicing at priority 0, meaning they can be invoked explicitly from a runbook or deploy template but never run on their own. A non-zero priority would otherwise make a cleaning container run on every deployment, because a deploy step above priority zero runs automatically.

Note

A step name must not match a step IPA already provides. erase_devices_metadata or write_image would shadow the real one; IPA reports a configuration error rather than running your container in its place.

This is useful for high-security environments which would prefer the hassle of rebuilding a ramdisk to the risk of permitting runtime decisions around what containers to clean with.

Security Considerations¶

Two kinds of container step, two levels of trust¶

Container steps reach IPA by two different routes, and they are not trusted equally:

Steps baked into the ramdisk via container_steps_file are authored by the operator at image build time and cannot be changed at run time. They are trusted and are not checked against allowed_containers.

Steps driven by step arguments, such as from a runbook, deploy template, manual cleaning or servicing, carry an image chosen by whoever made the API call. These are untrusted and are checked against allowed_containers unless allow_arbitrary_containers is true.

This matters because creating and using a runbook does not require full administrative rights: baremetal:runbook:create is available to role:manager at project scope. Anyone who can run a runbook on a node can therefore choose what executes as root on that node, bounded only by allowed_containers.

General guidance¶

Prefer allowlisting over allow_arbitrary_containers = true. The allowlist (allowed_containers) restricts which images IPA will accept, reducing the risk of running untrusted code. The transport prefix is ignored when matching, so docker://example.com/tool:1 and example.com/tool:1 are the same entry.
Allowlist digests, not tags. An entry naming a tag permits whatever that tag points at, so anyone able to push to the registry chooses what runs as root on your nodes. Only a pinned digest, such as example.com/tool@sha256:..., actually constrains the content.
Options are part of the boundary. When allow_arbitrary_containers is false, pull_options and run_options supplied as step arguments are ignored and the conductor’s configured values are used. Flags alone are enough to defeat an image allowlist, since --privileged, -v /:/host or --entrypoint turn any permitted image into arbitrary host access, so the caller who may not choose the image may not choose the flags either.
Step names must not collide with existing hardware manager steps. A container step named after a real one, such as erase_devices_metadata, would shadow it. IPA refuses such names, reporting a configuration error rather than silently substituting a container for the real step.
TLS verification – the registry certificate is verified by default. Adding --tls-verify=false to pull_options or run_options turns that off, which undoes the allowlist: anything able to answer for the registry can serve its own image under a permitted name. Keep it to test registries and make the registry certificate available in the ramdisk instead.
Container privileges – by default, containers run with --network=host, giving them full access to the node’s network stack. Review run_options and consider adding --read-only or dropping capabilities where possible.

Troubleshooting¶

Container pull failures: Check that the container registry is accessible from the cleaning network. Verify the image URL in the runbook step. If using TLS, ensure certificates are configured correctly in the ramdisk or add --tls-verify=false to pull_options for testing.
Step not found: generic_container_step: The IPA ramdisk was not built with the ironic-python-agent-podman element. Rebuild the ramdisk with podman support as described in Prerequisites.
Container image not permitted: The container URL does not match any entry in allowed_containers and allow_arbitrary_containers is false. Either add the image to the allowlist or set allow_arbitrary_containers = true in [agent_containers]. Note that steps baked into the ramdisk are not subject to the allowlist, so this only applies to images named in step arguments.
Container runtime is not available in this ramdisk: [agent_containers]runner names a runtime the ramdisk does not contain. Either install it or change the option to match what was built into the image.
Container step collides with an existing hardware manager method: A step in container_steps_file is named after a step IPA already provides and would shadow it. Rename the step in the YAML file.
Unknown flag, or the runtime rejects its arguments: pull_options or run_options were written space separated. They are comma separated lists; see Ironic Conductor Configuration.
Trait mismatch: The node does not have a trait matching the runbook name. Add the trait with baremetal node add trait <node> <RUNBOOK_NAME>.

Container-Based Steps

Container-Based Steps¶

Overview¶

Basics¶

Prerequisites¶

IPA ramdisk with podman support¶

Container registry access¶

Ironic Conductor Configuration¶

Example Container-based Runbooks¶

The built-in step¶

Single-container runbook¶

Multi-container runbook¶

Adding traits to nodes¶

Using the Runbook¶

Manual cleaning¶

Automated cleaning¶

Servicing¶

Alternative Methods¶

Security Considerations¶

Two kinds of container step, two levels of trust¶

General guidance¶

Troubleshooting¶

ironic 38.0.1.dev102

Page Contents