Xena Series Release Notes


New Features

  • Adds support for custom Multipathd configuration.

  • Since Kolla containers can built with user provided repos.yaml Kayobe can override the file with their own content. The override files can be ${KAYOBE_CONFIG_PATH}/kolla/repos.yaml (default Kolla filename) or ${KAYOBE_CONFIG_PATH}/kolla/repos.yml. Multiple Environments supported.

Bug Fixes

  • Improves performance of Bifrost operations by preventing unnecessary requests to the Ironic API.

  • Fixes download of roles from Ansible Galaxy following the renaming of the mrlesmithjr.manage_lvm role. LP#2023502

  • Fixes an issue where generation of passwords.yml for Kolla Ansible could fail if the directory containing the file does not exist. This is typical in a multiple environment setup, when creating a new environment. See story 2010293 for details.

  • Fixed issue of seed containers being unable to use password protected registry by adding docker login function to kayobe deploy-containers role.


New Features

  • Adds support for configuring arbitrarily named VLAN interfaces using systemd-networkd. See story 2010266 for details.

Bug Fixes

  • Fixes an issue with systemd-networkd configuration on Ubuntu with multiple VLAN interfaces. See story 2009013 for details.


Bug Fixes

  • public-openrc.sh is now only generated if the admin-openrc.sh file generated by Kolla Ansible exists. This fixes an issue where the task would fail, when running in a clean environment, with a set of Kolla Ansible tags that did not include the generation of admin-openrc.sh. See story 2009323.


Bug Fixes

  • Fixes an error when generating passwords.yml if an unencrypted file exists but a password has been supplied.


New Features

  • Updates base CentOS Stream 8 cloud image to CentOS-Stream-GenericCloud-8-20220913.0.x86_64.


Bug Fixes

  • Fixes an issue with nclu-switch command ordering, when description was applied first to a non-existent (virtual) interface. See story 2010279 for details.


Upgrade Notes

  • Updates base Rocky Linux 8 cloud image to Rocky-8-GenericCloud.latest.x86_64.qcow2

Bug Fixes

  • Fixes an issue where a custom playbook using become_user could fail when setting permissions on temporary files. The acl package is now installed on all systems by default.


Bug Fixes

  • Fixes an issue with undefined kolla_enable_hacluster variable.


New Features

  • Adds support for custom Placement configuration.

  • Adds support for global configuration options for Apt in files in /etc/apt/apt.conf.d/ on Ubuntu systems. See story 2009655 for details.

  • Adds support for configuring Apt repositories on Ubuntu hosts. See story 2009655 for details.

  • Add the bonding 802.3ad aggregation selection option.

  • Enables hardware clock (RTC) synchronisation by default when applying the chrony role. This setting is configurable with the new variable chrony_rtcsync_enabled.

  • Adds support for inspection of L3-routed Ironic networks via DHCP-relay.

  • The new filter net_no_ip adds the attribute no_ip which can be set to true to skip IP address allocation and configuration for specific networks.

  • Adds a new variable seed_hypervisor_enable_snat that allows users to enable SNAT service on the seed hypervisor. The default value is false.

  • Adds support for Rocky Linux 8 as Host OS.

  • Adds support for running package updates on Ubuntu hosts via the following existing commands:

    • kayobe seed host package update --packages <packages>

    • kayobe seed hypervisor host package update --packages <packages>

    • kayobe infra vm host package update --packages <packages>

    • kayobe overcloud host package update --packages <packages>

Security Issues

  • Fixes an issue where any passwords in kolla_ansible_custom_passwords were exposed in Ansible logs. When using verbosity level 3 (-vvv), they were also exposed in Ansible output.

Bug Fixes

  • Ironic inspection through Bifrost now work even if DHCP-relay is used. The dhcp-range in dnsmasq.conf corrctly configured with network mask.

  • In production environments, the provision network may be separated from the other networks, so in this case, if you want Bifrost’s DHCP service provides the correct gateway for the clients the inspection_gateway should be used instead of the gateway attribute for the provision network. This also avoids configuring the multiple IP gateways on a single host which leads to unpredictable results.

  • Fixes an issue where the Neutron SR-IOV agent image is not built when the service is enabled.

  • Fixes an issue with idempotence of local Kolla Ansible configuration generation.

  • Fixes an issue with the seed’s configdrive when the admin network is a VLAN. See story 2008089 for details.

  • Enables deployment of Grafana when Monasca is enabled, as a replacement for the retired monasca-grafana image. See story 2009717 for details.

  • Fixes Ansible inventory generation with some custom group mappings using the same group names for Kayobe and Kolla Ansible. See story 2009927 for details.

  • The set of commands starting with kayobe overcloud database now generate the kolla configuration necessary to login to the nodes running the database.

  • Fixes an issue with config drive generation for infrastructure and seed VMs when using untagged interfaces. The symptom of this issue is that kayobe cannot login to the instance. If you check the libvirt console log, you will see KeyError: 'vlan_link'. See story 2009910 for details.

  • Fixes an issue where hacluster images are not built when the service is enabled.

  • Fixes an issue with IPA image builds which used the master branch of ironic-python-agent, even on stable releases of Kayobe, or when explicitly setting ipa_build_source_version.

  • Fixes an issue seen when using Jinja2 3.1.0.

  • Fixes an issue where any passwords in kolla_ansible_custom_passwords were exposed in Ansible logs. When using verbosity level 3 (-vvv), they were also exposed in Ansible output.

  • Fixes an issue where patch links could be erroneously created on hosts not in the overcloud group. See Story 2009911 for details.

  • Fixes an issue where the MTU defined in Kayobe was not applied to Ironic provisioning and cleaning networks in Neutron.

  • Deployment image (IPA) build no longer uses master version of upper-constraints. Instead, it defaults to using the constraints for the OpenStack release associated with the version of Kayobe being used. See story 2009810 for details.

  • Fixes failures to run kayobe overcloud bios raid configure by upgrading the stackhpc.drac role to version 1.1.6.

  • Fixes an issue with masking NTP services which are not found. See story 2009821 for details.


Bug Fixes

  • Fixes a failure to detect the Kayobe installation prefix when lib is present multiple times in the installation path. See story 2009721 for details.


New Features

  • Adds support for configuring apt’s proxy setting for Ubuntu hosts. See story 2009035 for details.

  • Adds support for deploying infrastructure VMs on the seed hypervisor. These can be used to provide supplementary services that do not run well within a containerised environment or are dependencies of the control plane. See story 2008741 for details.

  • Adds Arista switch support for the Neutron ML2 genericswitch driver.

  • Adds a new kolla_bifrost_deploy_image_filename variable used to define the name of the root disk image to provision. This may be used to deploy different images on different hosts.

  • Adds a new kolla_bifrost_use_firewalld variable used to define whether Bifrost uses firewalld, which is now disabled by default.

  • Adds support for CentOS Stream 8 as a host Operating System and base container image. This is the only distribution of CentOS supported from the Wallaby release. The Victoria release will support both CentOS Linux 8 and CentOS Stream 8 hosts and images, and provides a route for migration.

  • Adds support for installing Ansible collections. See story 2008391 for details.

  • Adds a --diff argument to kayobe CLI commands. This is passed through to ansible-playbook for Kayobe and Kolla Ansible playbooks, and can be used with the --check argument to see changes that would be made to files.

  • Adds a new variable kolla_docker_registry_insecure to configure whether Docker should use an insecure registry for Kolla images.

  • Adds a new flag, docker_registry_network_mode, which defaults to host. This may be used to set the network mode of the Docker registry container.

  • Adds support for passing through additional host variables from Kayobe to Kolla Ansible. This is done via the following variables:

    • kolla_seed_inventory_pass_through_host_vars_extra

    • kolla_seed_inventory_pass_through_host_vars_map_extra

    • kolla_overcloud_inventory_pass_through_host_vars_extra

    • kolla_overcloud_inventory_pass_through_host_vars_map_extra

    See story 2008797 for details.

  • Adds support for configuring a firewall via firewalld on CentOS. See story 2008991 for details.

  • Adds support for merging the following configuration files from the environment-specific directory (etc/kayobe/environments/<environment>) and the base directory (etc/kayobe).

    • kolla/config/bifrost/bifrost.yml

    • kolla/config/bifrost/dib.yml

    • kolla/config/bifrost/servers.yml

    • kolla/globals.yml

    • kolla/kolla-build.conf

    See story 2002009 for details.

  • Adds a new kayobe overcloud service prechecks command to run Kolla Ansible prechecks without deploying services.

  • Adds a new variable seed_enable_snat that allows users to enable SNAT service on the seed. The default value is false.

  • Adds support for configuring the filter and gather_subset arguments for the setup module via kayobe_ansible_setup_filter and kayobe_ansible_setup_gather_subset respectively. These can be used to reduce the number of facts, which can have a significant effect on performance of Ansible.

  • Adds a new command, kayobe overcloud facts gather, to gather Ansible facts for overcloud hosts. This may be useful for populating a fact cache.

  • Adds support for configuring active built-in tuned profile by using the giovtorres.tuned Ansible role. This is only supported on CentOS.

  • Adds support for Ubuntu Focal 20.04 as a host and container Operating System for seed, seed hypervisor and overcloud hosts.

  • Adds support for the metalink option in custom DNF repositories configured with dnf_custom_repos in dnf.yml.

Known Issues

  • Switching an existing deployment from binary to source images can break Horizon, which can be resolved by flushing contents of memcached with docker restart memcached. See Kolla Ansible bug 1886549 for details.

Upgrade Notes

  • Updates all references to Ansible facts within Kayobe from using individual fact variables to using the items in the ansible_facts dictionary. This allows users to disable fact variable injection in their Ansible configuration, which may provide some performance improvement. Check for facts referenced in local configuration files, and update to use ansible_facts before disabling fact variable injection.

  • Updates the maximum supported version of Ansible from 2.9 to 4.x (ansible-core 2.11). The minimum supported version is updated from 2.9 to 2.10. This is true for both Kayobe and Kolla Ansible.

  • Upgrading directly from Ansible 2.9 to Ansible 2.10 or from Ansible 2.10 to Ansible 4 is known to cause problems. You should uninstall Ansible before upgrading your Kayobe virtual environment:

    pip uninstall ansible

    If upgrading from Ansible 2.10 to a newer version, also uninstall ansible-base:

    pip uninstall ansible-base
  • Bifrost is now configured to avoid using firewalld, to prevent conflicts with firewall rules set by Kayobe on the seed host. The existing behaviour can be retained by setting kolla_bifrost_use_firewalld to True in bifrost.yml.

  • CentOS Linux 8 is no longer supported as a host Operating System or base container image. CentOS users should migrate to CentOS Stream 8. The Victoria release will support both CentOS Linux 8 and CentOS Stream 8 hosts and images, and provides a route for migration.

  • Updates the default image type to source. Users wishing to build and deploy binary type images should set kolla_install_type to binary in kolla.yml. This change is to reflect the reality that source images are tested more thoroughly and we (as OpenStack community) have better control over them.

  • Consistent network device naming is now enabled by default in overcloud root disk images, by setting net.ifnames=1 on the kernel command line. This is performed using the DIB_BOOTLOADER_DEFAULT_CMDLINE diskimage-builder environment variable, which is set to nofb nomodeset gfxpayload=text net.ifnames=1 to preserve diskimage-builder defaults. To restore existing behaviour, set DIB_BOOTLOADER_DEFAULT_CMDLINE to nofb nomodeset gfxpayload=text net.ifnames=0 in the kolla_bifrost_dib_env_vars_extra dictionary.

  • The --check argument to kayobe CLI commands is now passed through to Kolla Ansible playbooks.

  • The default configuration of Docker, as set by Kolla Ansible, has changed to stop using an insecure registry for Kolla images. To avoid breaking existing deployments, kolla_docker_registry_insecure is automatically set to true if Kayobe is configured to deploy an insecure registry service. If using an insecure registry not deployed by Kayobe, you will need to set the value of kolla_docker_registry_insecure to true or configure TLS for your registry.

  • Updates the NTP implementation from the chrony container deployed by kolla-ansible to configuring chrony as a host service. Chrony is now installed on all hosts in the ntp group, which defaults to include the seed, overcloud, and seed-hypervisor groups. On existing deployments, you should run kayobe overcloud host configure to migrate from the kolla-ansible deployed container. This can optionally be scoped to just use the ntp tag. You can continue to use the kolla container by setting kolla_enable_chrony to true.

  • Support for deployment of a chrony container managed by Kolla Ansible has been removed.

  • Removes the iscsi interface from kolla_ironic_enabled_deploy_interfaces, and changes kolla_ironic_default_deploy_interface to direct. This is in line with upstream changes in Ironic during the Xena cycle, in which the iscsi deploy driver was removed.

    Existing nodes using the iscsi deploy driver should be updated to an alternative such as direct before upgrading.

  • Kayobe now applies a sensible tuned profile to each host by default. This may need to be customised, for example if the seed node is not a virtual machine. See the documentation and story 2007853 for details.

  • Kolla images Docker namespace used in Kayobe was switched from kolla to openstack.kolla to reflect Kolla project changes.

Deprecation Notes

  • The following variables are deprecated, in favour of using configuration files kolla/globals.yml and kolla/kolla-build.conf respectively.

    • kolla_extra_globals

    • kolla_bifrost_extra_globals

Bug Fixes

  • Prevents Bifrost from using firewalld to avoid conflicts with firewall rules set by Kayobe on the seed host. See story 2009252 for more details.

  • Setting kolla_enable_ovn in kolla.yml did not configure Neutron’s integration with OVN. See story 2009080 for details.

  • Sets proxy option when using dnf during user bootstrapping, before dnf.conf is updated. This allows Kayobe to install Python 3 during host configuration when dnf requires a proxy to operate.

  • Adds missing hook support for the kayobe environment create command.

  • Fixes some issues seen when using the --check argument with kayobe overcloud host configure. See story 2004798.

  • Fixes an issue bug where introspection data save would fail. See story 2009129 for more details.

  • Fixes an issue with systemd-networkd configuration for VLAN interfaces when the interface is untagged.

  • Fixes an issue with configuration validation when no public API network is in use. See story 2009134 for details.

  • Filter out 25 Gigabit Ethernet interface names in the Ironic inspector rule setting node names from interface LLDP switch port descriptions.

  • Fixes an issue with container image builds by using host as the default network_mode for kolla-build. See story 2008942 for details.

  • Fixes an issue with systemd-networkd MTU mismatch in veth pair on Ubuntu. See story 2009072 for details.

  • Fixes an issue where cached seed VM images are unnecessarily owned by root. See story 2009277 for details.