2023.1 Series Release Notes


Bug Fixes

  • Fixed a potential race condition in the member batch update API call, the load balancers might not have been locked properly.

  • Fix a bug that prevented the operating_status of a health-monitor to be set to ONLINE when ipv6 addresses were enclosed within square brackets in controller_ip_port_list.


New Features

  • The new “cpu-pinning” element optimizes the amphora image for better vertical scaling. When an amphora flavor with multiple vCPUs is configured it will configure the kernel to isolate (isolcpus) all vCPUs except the first one. Furthermore, it uninstalls irqbalance and sets the IRQ affinity to the first CPU. That way the other CPUs are free to be used by HAProxy exclusively. A new customized TuneD profile applies some more tweaks for improving network latency. This new feature is disabled by default, but can be enabled by running diskimage-create.sh with the -m option or setting the AMP_ENABLE_CPUPINNING environment variable to 1 before running the script.

  • Amphora agent has been adjusted to complement the vertical scaling optimizations implemented in the new cpu-pinning element. If the flavor uses multiple vCPUs it will configure HAProxy automatically to pin each of its worker threads to an individual CPU that was isolated by the element (all vCPUs starting from the second one).

  • The cpu-pinning element for the amphora image sets the kernel bootarg nohz_full=1-N to enable full dynticks on all CPUs except the first one (on single CPU images this will have no effect). This should reduce kernel noise on those CPUs to a minimum and reduce latency.

Upgrade Notes

  • The Octavia API will now check that the HTTP Accept header, if present, is compatible with the application/json content type. If not the user will get a 406 status code response, Not Acceptable.

  • Amphora vertical scaling optimizations require a new amphora image build with the optional CPU pinning feature enabled in order to become effective.

  • diskimage-create.sh has been updated to build Ubuntu Jammy (22.04) amphora images per default.

  • In order for the full dynticks optimization to become effective a new amphora image needs to be built with the new optional CPU pinning feature enabled.

Deprecation Notes

  • The configuration option user_data_config_drive is deprecated. The nova user_data option is too small to replace the normal file based config_drive provisioning for cloud-init. This option has never been functional in Octavia and will be removed to reduce confusion.

  • Amphora load balancers support single process mode only now. Split listener configuration, which was used up to API version 0.5, has been removed from the codebase.

Security Issues

  • Filter out private information from the taskflow logs when ‘’INFO’’ level messages are enabled and when jobboard is enabled. Logs might have included TLS certificates and private_key. By default, in Octavia only WARNING and above messages are enabled in taskflow and jobboard is disabled.

Bug Fixes

  • The Octavia API will now check that the HTTP Accept header, if present, is compatible with the application/json content type. If not the user will get a 406 status code response, Not Acceptable. This change also ensures that the API responses have a content type of application/json.

  • Fixed the ability to use the ‘text/plain’ mime type with the healthcheck endpoint.

  • Added a filter to hide a bogus ComputeWaitTimeoutException exception when creating an amphora when jobboard is disabled. This exception is part of the flow when creating a load balancer or an amphora and should not be shown to the user.

  • The parameters of a taskflow Flow were logged in ‘’INFO’’ level messages by taskflow, it included TLS-enabled listeners and pools parameters, such as certificates and private_key.

  • Fix amphora haproxy_count to return the number of haproxy processes that are running.

  • Fix an authentication error with Barbican when creating a TERMINATED_HTTPS listener with application credential tokens or trust IDs.

  • Fixed a “corrupted global server state file” error in Centos 9 Stream when reloading the state of the servers after restarting haproxy. It also fixed the recovering of the operational state of the servers in haproxy after its restart.

  • Fix a bug when full graph of load balancer is created without listeners if jobboard_enabled=False

  • Fixed a bug that prevented Octavia from creating listeners with the fully-populated load balancer API in SINGLE topology mode.

  • Fixed backwards compatibility issue with the feature that preserves HAProxy server states between reloads. HAProxy version 1.5 or below do not support this feature, so Octavia will not to activate it on amphorae with those versions.

  • Fixed the policy of the legacy admin role, it is still an admin with sRBAC.

  • Removed system scope policies, all the policies are now project scoped.

  • Modified default Keepalived LVS persistence granularity configuration value so it would be ipv6 compatible.

  • Fix an issue with PING health-monitors on Centos 8 Stream. Changes in Centos and systemd prevent an unprivileged user from sending ping requests from a network namespace.

  • Usage of castellan_cert_manager as cert_manager has been significantly improved. Now you can define configuration options for castellan in octavia.conf and they will be passed properly to castellan beckend. This allows to use allowed castellan backends as for certificate storage.

  • Fixed SQLAlchemy warnings about the relationship between the Tags object and the other Octavia resources.

Other Notes

  • When a HTTPS termination listener gets configured, Octavia will tweak the HAProxy tune.ssl.cachesize setting to use about half of the available memory (free + buffers + cached) on the amphora minus the memory needed for network sockets based on the global max connections setting. This allows to make better reuse of existing SSL sessions and helps to lower the number of computationally expensive SSL handshakes.


Bug Fixes

  • Fix the rescheduling of taskflow tasks that have been resumed after being interrupted.