Xena Series Release Notes

9.0.1-35

Known Issues

  • When using a distribution with a recent SELinux release such as CentOS 8 Stream, PING health-monitor does not work as shell_exec_t calls are denied by SELinux.

  • Fixed configuration issue which allowed authenticated and authorized users to inject code into HAProxy configuration using API requests. Octavia API no longer accepts unencoded whitespace characters in url_path values in update requests for healthmonitors.

Upgrade Notes

  • The fix that updates the Netfilter Conntrack Sysfs variables requires rebuilding the amphora image in order to be effective.

Bug Fixes

  • The generated RSyslog configuration on the amphora supports now RSyslog failover with TCP if multiple RSyslog servers were specified.

  • Ensure that the provided rsyslog configuration file is used by the rsyslog by restarting the service, it fixes the log offloading feature on distributions that start rsyslog before cloud-init.

  • Fixed issues when building amphora image for Centos Stream 9.

  • Fixed issues when building amphora image for RHEL 9.

  • Correctly detect the member operating status “drain” when querying status data from HAProxy.

  • Fix the shutdown of the driver-agent, the process might have been stuck while waiting for threads to finish. Systemd would have killed the process after a timeout, but some children processes might have leaked on the controllers.

  • Enable required SELinux booleans for CentOS or RHEL amphora image.

  • Fix a bug that prevented the provisioning_state of a health-monitor to be set to ERROR when an error occurred while creating, updating or deleting a health-monitor.

  • Fix an issue with IPv6 members that could have been set in operating_status ERROR just after being added.

  • Fix an issue with amphorav2 and persistence, some long tasks executed by a controller might have been released in taskflow and rescheduled on another controller. Octavia now ensures that a task is never released early by using a keepalive mechanism to notify taskflow (and its redis backend) that a job is still running.

  • Fixed an issue with members in ERROR operating status that may have been updated briefly to ONLINE during a Load Balancer configuration change.

  • Netfilter Conntrack Sysfs variables net.netfilter.nf_conntrack_max and nf_conntrack_expect_max get set to sensible values on the amphora now. Previously, kernel default values were used which were much too low for the configured net.netfilter.nf_conntrack_buckets value. As a result packets could get dropped because the conntrack table got filled too quickly. Note that this affects only UDP and SCTP protocol listeners. Connection tracking is disabled for TCP-based connections on the amphora including HTTP(S).

  • Fix a potential race condition when updating a resource in the amphorav2 worker. The worker was not waiting for the resource to be set to PENDING_UPDATE, so the resource may have been updated with old data from the database, resulting in a no-op update.

  • Fixed issue with SELinux and the lvs-masquerade.sh script on the amphora. The script already runs with root permissions, so the use of sudo inside the script is unneeded.

  • Fix an issue when Octavia performs a failover of an ACTIVE-STANDBY load balancer that has both amphorae missing. Some tasks in the controller took too much time to timeout because the timeout value defined in [haproxy_amphora].active_connection_max_retries and [haproxy_amphora].active_connection_rety_interval was not used.

  • Fix a bug that could have triggered a race condition when configuring a member interface in the amphora. Due to a race condition, a network interface might have been deleted from the amphora, leading to a loss of connectivity.

  • Fixed “Could not retrieve certificate” error when updating/deleting the client_ca_tls_container_ref field of a listener after a CA/CRL was deleted.

  • Fixed validations in L7 rule and session cookie APIs in order to prevent authenticated and authorized users to inject code into HAProxy configuration. CR and LF (\r and \n) are no longer allowed in L7 rule keys and values. The session persistence cookie names must follow the rules described in https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie.

  • Validate that the creation of L7 policies is compatible with the protocol of the listener in the Amphora driver. L7 policies are allowed for Terminated HTTPS or HTTP protocol listeners, but not for HTTPS, TCP or UDP protocols listeners.

9.0.1

Bug Fixes

  • Fix an issue with the provisioning status of a load balancer that was set to ERROR too early when an error occurred, making the load balancer mutable while the execution of the tasks for this resources haven’t finished yet.

  • Fix an issue that could set the provisioning status of a load balancer to a PENDING_UPDATE state when an error occurred in the amphora failover flow.

9.0.0

New Features

  • AmphoraV2 provider that was intoduced in earier releases now is default Amphora provider for Octavia. Alias amphorav1 is available for previous version of Amphora provider. Alias amphorav2 is now the same as amphora. By default, jobboard (usage persistence storage) is not enabled, configurable via jobboard_enabled option in task_flow section.

Upgrade Notes

  • Support for new features, such as ALPN on pools, HTTP/2 on pools, gRPC, and SCTP require an updated amphora image.

  • The spare pool feature was removed after being deprecated in the Victoria release. After an upgrade of the controllers, spare amphorae will be automatically deleted by the Octavia health-manager service.

Deprecation Notes

  • The [amphora_agent].agent_server_network_file configuration option is now deprecated, the new Amphora network configuration tool introduced in Xena does not support a single configuration file.

Bug Fixes

  • Fixed amphora driver pool ALPN compatibity with older amphora images.

  • Amphora network configuration for the VIP interface and the pool member interfaces are now applied with the amphora-interface tool. amphora-interface uses pyroute2 low-level functions to configure the interfaces instead of distribution-specific tools such as “network-scripts” or “/etc/network/interfaces” files.

  • Disable conntrack for TCP flows in the Amphora, it reduces memory usage for HAProxy-based listeners and prevents some kernel warnings about dropped packets.

  • Fix an issue with amphorav2 driver, a failover of an amphora created an amphora with an ERROR status.

  • Fixes loadbalancer creation failure when one of the listener port matches with the octavia generated peer ports and the allowed_cidr is explicitly set to 0.0.0.0/0 on the listener. This is due to creation of two security group rules with remote_ip_prefix as None and remote_ip_prefix as 0.0.0.0/0 which neutron rejects the second request with security group rule already exists.

  • Fix an issue with the rsyslog configuration file in the Amphora when the log offloading feature and the local log storage feature are both disabled.

  • Fix a serialization error when using host_routes in VIP subnets when persistence in the amphorav2 driver is enabled.

  • Fixed MAX_TIMEOUT for timeout_client_data, timeout_member_connect, timeout_member_data, timeout_tcp_inspect API listener. The value was reduced from 365 days to 24 days, which now does not exceed the value of the data type in DB.

  • Fixed an issue with the lo interface in the amphora-haproxy network namespace. The lo interface was down and prevented haproxy to communicate with other haproxy processes (for persistent stick tables) on configuration change. It delayed old haproxy worker cleanup and increased the memory consumption usage after reloading the configuration.

  • Increase the limit value for nr_open and file-max in the amphora, the new value is based on what HAProxy 2.x is expecting from the system with the greatest maxconn value that Octavia can set.

  • Fix weighted round-robin for UDP and SCTP listeners with keepalived and lvs. The algorithm must be specified as ‘wrr’ in order for weighted round-robin to work correctly, but was being set to ‘rr’.

  • Fix a bug that allowed a user to create a load balancer on a vip_subnet_id that belongs to another user using the subnet UUID.

Other Notes

  • The diskimage-create.sh default for Ubuntu is now focal.