diskimage-builder Release Notes¶
Element block-device-efi-lvm has been added which is like block-device-efi but defines an LVM logical group in the root partition. Four logical volumes are defined in that group, mounted to /, /tmp, /var, and /home.
This volume layout will not meet all requirements, but this is more of an example demonstrating the capability to encourage more usage of this existing feature.
Base installs now mount
/sysread-only in chroot environemnts. This is a good indication to various tools and scripts that that they are running in a unprivileged/containerised environment.
Removed support for
grub(as opposed to
grub2) support from the bootloader element.
Removes the long deprecated ironic-agent element. Use ironic-python-agent-ramdisk from the ironic-python-agent-builder project.
centoselement supports CentOS 8 Stream builds when run with
Add support to run dhclient on vlan interfaces created on top of an Ethernet interface. The vlan interfaces in this case will use the default naming convention of <interface>.<vlan>. See also <https://storyboard.openstack.org/#!/story/2008298>`__ for further details.
New environment variable enables the ability to select specific module streams prior to RPM installation on RHEL8 and later.
cloud-init-disable-resizefswill no longer leave
growparton / partition to enabled in cloud.cfg.
DIB_ADD_APT_KEYSargument now copies keys into
/etc/apt/trusted.gpg.d, rather than using
apt-keyto add them.
rhel element fixes trying to install python3 before registering subscriptions.
centos-minimalsupports CentOS 8 Stream builds when run with
bootloaderelement now co-installs a BIOS bootloader for x86-64 when
DIB_BLOCK_DEVICE=efiis set. This makes images that are EFI and BIOS cross-compatiable.
The default for elements building Fedora is updated to Fedora 32
dib-pythonelement is deprecated. In chroot Python tools should use
#!/usrbin/env python3on all distributions now.
Fix cases of ‘mkfs’ failing because the partitions never showed up. Partition mappings will now be updated instead of just adding them with ‘kpartx’. That means that ‘kpartx’ will also remove devmappings for deleted partitions.
Fixes support for the
dhcp-all-interfaceselement when the system networking configuration is managed via
NetworkManagerconfiguration with the
dhcp-all-interfaceselement to account for Centos8/RHEL8’s default change to the
NetworkManagerinternal DHCP client, which can introduce different behavior for infrastucture operators such as those using LACP.
This removes automatic detection and activation of virtualenv environments, which is no longer necessary. This fixes installation under Python 3’s inbuilt
package-installselement can now take a list value for the
ubuntu-minimalelement now supports the Focal release.
pip-and-virtualenvelement does not support Fedora 31 (and greater) or Tumbleweed distributions.
Trusty testing has been removed; although we have not attempted to disable support in elements, the distribution can be effectively considered deprecated from this release.
Added CentOS 8 support.
The ensure-venv module is added to ensure that python3 -m venv is available on the host. This can be useful if you wish to install non-distribution-packaged Python software on a host during build, but not mix any non-distro installation. This works on all platforms with Python 3 available.
Adds efibootmgr and efivar packages to ironic-agent, so when building DIB images they will be present.
centos7element is deprecated and is left only for backward compatibility. Use the
centoselement instead. Note that you should set
7when using the
dhcp-all-interfaceselement could not configure network interfaces properly when
rdisc6is present on the system
dib-init-systemelement installs a utility
/usr/local/bin/dib-init-systemwhich would try to guess the init system (systemd, etc.). This was called from
environment.dfiles, which means that in phases outside the chroot it was attempting to query the init system of the build host. This completely fails in a situation such as running inside a container without a full init system. To avoid this issue, each OS element will set DIB_INIT_SYSTEM directly. The
dib-init-systemscript was not really intended to be called directly, but will now just report the value of
A new variable
DIB_DEBIAN_SECURITY_SUBPATHis added to override the suite subpath of security repositories. Previously this was hard-coded to the upstream repository layout, which may not always match the mirror’s layout.
The “ironic-agent” element can now be found at ironic-python-agent-builder to build the ramdisk. This element is deprecated from diskimage-builder.
pip-and-virtualenvelement also installs
python3-venvon Python-3 Debian-like distributions to ensure
$DIB_PYTHON_VIRTUALENVworks as expected.
The sysprep element has been fixed to truncate
/etc/machine-idwas already being truncated). This ensures a machine-id is not packaged in the image and systemd will be forced to generate a new one upon first boot.
The sysprep element is added as a dependency to the
debootstrapelement to ensure that it runs on all Debian builds, including debian-minimal based images.
DIB_SIMPLE_INIT_NETWORKMANAGER_IPV6_DELAYvariable is removed as it turned out that increasing this delay increased the odds that NetworkManager would fail to configure IPv6 on an interface. Instead, glean has been updated to not up interfaces and relies on NetworkManager to do this. This results in reliable IPv4 and IPv6 config.
centos-minimalsupports CentOS 8 builds when run with
A new variable
DIB_DEBIAN_SECURITY_MIRRORis added to override the path to the security repositories. Previously this was hard-coded to the upstream repositories, which could get out of sync with mirrors specified with
pip-and-virtualenvelement has added variables
DIB_PYTHON_VIRTUALENVto make it more straight forward for other elements to install packages or create virtualenv environment
simple-initelement will now default to using NetworkManager for CentOS and Fedora platforms. For CentOS 8 and Fedora, the
DIB_SIMPLE_INIT_NETWORKMANAGER=0option is no longer supported.
Source install for
pip-and-virtualenvis deprecated for RHEL 8/CentOS 8
The DIB_GZIP_BIN environment variable enables builders to change the path to gzip, such as when an alternate gzip is to be used (ie. pigz).
The yum-minimal element now removes the linux-firmware and linux-firmware-whence packages to save space in cloud images.
journal-to-consoleelement is added to allow forwarding systemd output to the host console. This is useful for debugging early boot issues.
When using the
block-device-efielement to create an EFI based system (most notably, with ARM64), the final disk size is now expanded for the 512MiB system EFI partition introduced with 2.24.0
DIB_SIMPLE_INIT_NETWORKMANAGER_IPV6_DELAYvariable is added for the
simple-initelement to set the
net.ipv6.conf.default.router_solicition_delaysysctl setting (it defaults to 30s). If you are seeing an extra “blank” interface in
nmcli con showit is likely that NM has not correctly configured itself in the presence of an interface already configured with IPV6 by the kernel. Increasing this value may help work-around the problem.
Added an environment variable, DIB_UBUNTU_KERNEL, that allows you to specify the kernel meta package to install in the image. It currently supports “linux-image-generic” (The default), “linux-image-kvm”, and “linux-image-virtual”.
Adds a new element
rhelto handle all supported RHEL releases, which are currently ‘7’ and ‘8’. As of now, users of diskimage-builder will still be able to use the ‘rhel7’ element, or migrate to ‘rhel’ and specify their respective DIB_RELEASE value.
The opensuse and opensuse-minimal element are now defaulting to build 15.1. Building for 42.3 and 15.0 is still supported, however requires setting DIB_RELEASE to ‘42.3’ resp. ‘15.0’ explicitly.
Support in opensuse and opensuse-minimal for openSUSE 42.2 has been removed as these are end of life.
The rhel7 element is deprecated and is left only for backwards compatibility. Use the rhel element instead. Note that you should set DIB_RELEASE to 7 to indicate which release you are using. Please read the notes.
The size of the ESP partition when building EFI based images is increased considerably from 12MiB to 550MiB, in line with modern guidelines. The existing size was often not enough to update packages.
--image-extra-sizeoption is provided to override the default 60% padding growth of the image size with a fixed gigabyte value.
--mkfs-journal-sizeoption is added to override the default journal size for basic ext4 root partitions.
--image-extra-sizeargument has changed from gigabytes to megabytes to make it more practical for use on smaller images.
The opensuse and opensuse-minimal element are now defaulting to build 15.0. Building for 42.2 and 42.3 is still supported, however requires setting DIB_RELEASE to ‘42.3’ resp. ‘42.2’ explicitly.
Support in opensuse and opensuse-minimal for openSUSE 13.* has been removed as these are end of life.
a new post-install script was added in openssh-server element to ensure KexAlgorithms, Ciphers and MACs for sshd_config will be configured following good pratices on https://infosec.mozilla.org/guidelines/openssh. This option is activated by default, users can set DIB_OPENSSH_SERVER_HARDENING to 0 to disable this sshd configuration
Adds a new element
init-ibft-interfacesto initialize network interfaces with configuration provided via iBFT. The new element is now a dependency of the
The simple-init element can now use NetworkManager instead of legacy scripts on Red Hat platforms.
The package-installs element now supports skipping installation of packages based on an environment variable specified in the config file. See the package-installs element documentation for full details.
Setting values with the
selinux-permissiveelement has been moved from
pre-install.dphase to avoid selinux related packages being installed before the settings are applied.
--logfilewill now imply
DIB_QUIET=1(i.e. logs will not appear on stdout). You can override this with an explicit
DIB_QUIET=0if you want stdout and file logging.
post-rootstage is added that runs outside the chroot and before the image size calculation. For example, this may be useful for running things that copy much data into the image and need to be outside of the chroot and after install stages which run inside the chroot.
pre-finalisestage is added that runs outside the chroot and before the
finalisestage. For example, this may be useful for mounting external resources that are used inside the chroot during the
finalisestage, but do not need to be distributed in the final image such as build-time caches.
It has been clarified that the
DIB_BOOTLOADER_DEFAULT_CMDLINEvariable appends its values to grubs
GRUB_CMDLINE_LINUX_DEFAULT, which is used during all normal boots but not rescue boots; as opposed to applying to
Add new modprobe element. This element will replace modprobe-blacklist element. It wil still have the blacklist functionality, but it also adds the feature of passing a complete file with settings to the modprobe.d directory. Adding this functionality, that will allow elements that depends on this module, to just copy the specified files to the final directory.
You would need to modify your
SHA256SUMSvariables if you were using them to build Ubuntu with elements/ubuntu:
DIB_CLOUD_IMAGESwould need to contain URL with path, and
SHA256SUMSwould not now neccesarily point to
GPT support is added to the bootloader; see documentation for configuration examples. This should be considered a technology preview; there may be minor behaviour modifications as we enable UEFI and support across more architectures.
This fixes bug 1742170 where the grub root label is different than the file system label when booting from a whole disk image.
This fixes bug 1744403 where 60-ironic-agent-install fails to run when DIB_INSTALLTYPE_ironic_agent=source is set. pip installs constraints with the -c argument and this argument is a relatively newer addion. The currently installed pip may not support that option and must be upgraded before proceeding.
Adds sysprep element included by all systemd distros
--logfileoption is added to save output to a given file.
By default, all
stdoutwill log with timestamps (this used to be inconsistent; python tools logged with a timestamp, and bash parts did not). If you set
1the timestamp prefix will be suppressed (this would be appropriate if you are running diskimage-builder and capturing its output, and adding your own timestamp). Note that output to
--logfilewill always be timestamped.
You can set
1to suppress all output. This is likely only useful when used with the
Adds default sysprep element clearing /etc/machine-id which prevents duplicated /etc/machine-id by forcing systemd to generate a new id for each booted system.
Adds lvm support, allowing to create volumes.
fedoraelements have been updated to default to Fedora 26. Note if you to pin to specific versions, be sure to specify
The opensuse and opensuse-minimal element are now defaulting to 42.3, which is the latest stable openSUSE release. Building for 42.2 is still supported, however requires setting DIB_RELEASE to ‘42.2’ explicitly.
rhelelements have been removed. These were building version 6 which is no longer supported (mostly due to a lack of python 2.7). Version 7 support is available via the
rhel7elements (which downloads and modifies the upstream cloud images) or via
centos-minimal(which builds the image from a empty chroot; only available for CentOS).
centos-minimalis suggested as this is what OpenStack Infrastructure uses for its elements.
DISTRO=centos7, despite building the same thing as far as upper levels are concerned. We plan to rectify this in the version 8 time-frame.
DIB_[DISTRO]_DISTRIBUTION_MIRRORvariables have been removed. These were undocumented ways to set
DIB_DISTRIBUTION_MIRRORfor some elements. It was not implemented consistently and causing some confusion. If you need to setup mirrors for testing purposes, the
openstack-ci-mirrorselement is a good example that is used by OpenStack gate testing.
dracut-networkelement has been removed. It has not been functioning for some time due to incorrect paths.
dib-run-partselement is no longer required as
disk-image-createwill directly source the internal version for running scripts within the chroot. This element was unintentionally leaving
/usr/local/bin/dib-run-partsin the built image. From code search we do not believe anyone was relying on the presence of this script. If you do require it, you should source the
dib-utilspackage to install.
dib no longer exports
dib-run-parts. Adding this was an oversight made during v2 development, since
dib-utilsalready provides this. The
dib-run-partsused internally (
diskimage_builder/lib/dib-run-parts) is not intended to be used by external tools. If you require
dib-run-parts, you should install the
The packages specified with the -p command-line operation are now installed after the install.d phase, not before. This is to give elements priority when installing packages. The flag documentation has been updated to describe this.
dib-init-system did not correctly find the init system for Debian Jessie and Debian Stretch. This version also looks for /bin/systemctl as as hint for systemd and fixes the problem.
Version 2.0.0 of diskimage-builder incorporates recent work from the feature/v2 branch. This includes incorporating some largely internal changes to the way it finds and calls elements, enhancements to partitioning and removal of some long-deprecated elements. If you use dib exclusively via the command-line disk-image-create installed from a package or via pypi you are unlikely to notice any difference (if you run it directly from a git-tree checkout, you may be affected).
2.0.0 includes a new framework for partitioning contributed by Andreas Florath. This should allow for creating multiple partitions, images with encryption, LVM support and flexibility for multiple-devices, all of which are currently not supported. Please check the v2 documentation, specs and reach out if these features interest you (some parts still in review).
Element override is now supported. If you have an element of the same name earlier in the
ELEMENTS_PATH, it will override later instances (previously, the behaviour was undefined).
For purposes of both users and development we want dib to be as “pythonic” as possible and behave like all other projects. Two major visible changes are:
command-line scripts are entry points (i.e. need to be installed)
elements have moved under diskimage_create module
The result of the first is that
./bin/disk-image-createfrom the source tree is no longer there. Like all other projects, you should install dib into a virtualenv (if you’re developing, use pip -e) and
disk-image-createwill “just work”.
The second change, moving the inbuilt elements under the
diskimage_createmodule, is a simplification so we always have a canonical path to our elements. Since we now always know where elements are relative to the imported diskimage_builder module we can drop all the path guessing complexity. This has other good flow-on effects such as
testrbeing able to find unit-tests for elements in the normal fashion and having imports work as usual.
We are aware there are a number of tools that like to take dib elements and do things with them. Reading some of the dib source you may find there is a canonical way to find out the included dib elements path – ask dib itself, something like
DIB_ELEMENTS=$(python -c ' import diskimage_builder.paths; diskimage_builder.paths.show_path("elements")')
Note you probably do not want this. As mentioned, another feature of v2 is override elements – an element that appears first in the element path-list will override any built-in one (just like $PATH). There is a function,
diskimage_builder.get_elements(), which will correctly process the element path, calculate overrides and return a canonical list of elements, their dependencies and correct paths.
That said, you probably do not want this either! There are a number of elements that do things on behalf of other elements – they look for a file in the included elements, say, and use that as a manifest for something. Previously, these would just have to make up their own element processing via inspection of the command-line arguments. dib now exports pre-computed variables that an element can walk for all the current build elements – a YAML list for easy python decoding and a function that builds an array for Bash elements.
A number of long-deprecated elements have been removed in v2, which are to the best of our knowledge unused.
We have removed and deprecated the
dib-utilspackage. This was intended to be a more generic repository of tools that might be useful outside dib, but that did not eventuate and it has been folded back into dib for simplicity.