Zed Series Release Notes¶
When using ML2/OVN, during an upgrade procedure, the OVS system-id stored value can be changed. The ovn-controller service will create the “Chassis” and “Chassis_Private” registers based on this OVS system-id. If the ovn-controller process is not gracefully stopped, that could lead to the existence of duplicated “Chassis” and “Chassis_Private” registers in the OVN Southbound database.
[bug 2022914] Neutron-API supports using relays as the southbound connection in a ML2/OVN setup. Before the maintenance worker of the API required a leader_only connection, which was removed.
Fixed the scenario where the DHCP agent is deployed in conjunction with the OVN metadata agent in order to serve metadata for baremetal nodes. In this scenario, the DHCP agent would not set the route needed for the OVN metadata agent service resulting in baremetal nodes not being able to query the metadata service. For more information see bug 1982569.
For OVN versions v22.09.0 and above, the
mcast_flood_reportsoption is now set to
falseon all ports except “localnet” types. In the past, this option was set to
trueas a workaround for a bug in core OVN multicast implementation.
Now the ML2/OVN trunk driver prevents a trunk creation if the parent port is already bound. In the same way, if a parent port being used in a trunk is bound, the trunk cannot be deleted.
During the port bulk creation, if an IPAM allocation fails (for example, if the IP address is outside of the subnet CIDR), the other IPAM allocations already created are deleted before raising the exception. Fixes bug 2039550.
A new OVN maintenance method
remove_duplicated_chassis_registersis added. This method will periodically check the OVN Southbound “Chassis” and “Chassis_Private” tables looking for duplicated registers. The older ones (based on the “Chassis_Private.nb_cfg_timestamp” value) will be removed when more than one register has the same hostname, that should be unique.
The external_mac entry in the NAT table is used to distribute/centralize the traffic to the FIPs. When there is an external_mac set the traffic is distributed (DVR). When it is empty it is centralized through the gateway port (no DVR). Upon port status transition to down, the external_mac was removed regardless of DVR being enabled or not, leading to centralize the FIP traffic for DVR – though it was for down ports that won’t accept traffic anyway.
Adds a maintenance task that runs once a day and is responsible for cleaning up Hash Ring nodes that haven’t been updated in 5 days or more. See LP #2033281 for more information.
Added the missing extension
uplink-status-propagationto the ML2/OVN mechanism driver. This extension is used by the ML2/SR-IOV mechanism driver, that could be loaded with ML2/OVN. Now it is possible to create ports with the “uplink-status-propagation” flag defined.
The high availability of metadata service on isolated networks is limited or non-existent. IPv4 metadata is redundant when the DHCP agent managing it is redundant, but recovery is tied to the renewal of the DHCP lease, making most recoveries very slow. IPv6 metadata is not redundant at all as the IPv6 metadata address can only be configured in a single place at a time as it is link-local. Multiple agents trying to configure it will generate an IPv6 duplicate address detection failure.
Administrators may observe the IPv6 metadata address in “dadfailed” state in the DHCP namespace for this reason, which is only an indication it is not highly available. Until a redesign is made to the isolated metadata service there is not a better deployment option. See bug 1953165 for information.
The redirect-type=bridged option is only used if all the tenant networks connected to the router are of type VLAN or FLAT. In this case their traffic will be distributed. However, if there is a mix of VLAN/FLAT and geneve networks connected to the same router, the redirect-type option is not set, and therefore the traffic for the VLAN/FLAT networks will also be centralized but not tunneled.
1986003 Fixed an issue with concurrent requests to activate the same port binding where one of the requests returned a 500 Internal Server Error. With the fix one request will return successfully and the other will return a 409 Conflict (Binding already active). This fixes errors in nova live-migrations where those concurrent requests might be sent. Nova handles the 409/Conflict response gracefully.
Fix an issue in the OVN driver where network metadata could become unavailable if the metadata port was ever deleted, even if accidental. To re-create the port, a user can now disable, then enable, DHCP for one of the subnets associated with the network using the Neutron API. This will try and create the port, similar to what happens in the DHCP agent for ML2/OVS. For more information, see bug 2015377.
[bug 2003455] As part of a previous commit (https://review.opendev.org/c/openstack/neutron/+/875644) the redirect-type=bridged option was set in all the router gateway ports (cr-lrp ovn ports). However this was breaking the N/S traffic for geneve tenant networks connected to the provider networks through those routers with the redirect-type option enabled. To fix this we ensure that the redirect-type option is only set if all the networks connected to the router are of VLAN or FLAT type, otherwise we fall back to the default option. This also means that if there is a mix of VLAN and geneve tenant networks connected to the same router, the VLAN traffic will be centralized (but not tunneled). If the traffic for the VLAN/FLAT needs to be distributed, then it should use a different router.
Address scope is now added to all OVN LSP port registers in the northbound. Northd then writes the address scope from the northbound to the southbound so it can be used there by the ovn-bgp-agent.
Until the OVN bug (https://bugzilla.redhat.com/show_bug.cgi?id=2162756) is fixed, setting the “reside-on-redirect-chassis” to true for the logical router port associated to vlan provider network is needed. This workaround makes the traffic centrallized, but not tunneled, through the node with the gateway port, thus avoiding MTU issues.
The default value for the
metadata_workersconfiguration option has changed to 0 for the ML2/OVN driver. Since [OVN] Allow to execute “MetadataProxyHandler” in a local thread, the OVN metadata proxy handler can be spawned in the same process of the OVN metadata agent, in a local thread. That reduces the number of OVN SB database connections to one.
[bug 2003455] It is added an extra checking to ensure the “reside-on-redirect-chassis” is set to true for the logical router port associated to vlan provider network despite having the “ovn_distributed_floating_ip” enabled or not. This is needed as there is an OVN bug (https://bugzilla.redhat.com/show_bug.cgi?id=2162756) making it not work as expected. Until that is fixed, we need these workaround that makes the traffic centrallized, but not tunneled, through the node with the gateway port, thus avoiding MTU issues.
Normalise OVN agent heartbeat timestamp format to match other agent types. This fixes parsing of
GET /v2.0/agentsfor some clients, such as gophercloud.
Neutron can record full connection using log-related feature introduced in OVN 21.12. For more info see bug LP#<https://bugs.launchpad.net/neutron/+bug/2003706>
Since OVN 20.06, the “Chassis” register configuration is stored in the “other_config” field and replicated into “external_ids”. This replication is stopped in OVN 22.09. The ML2/OVN plugin tries to retrieve the “Chassis” configuration from the “other_config” field first; if this field does not exist (in OVN versions before 20.06), the plugin will use “external_ids” field instead. Neutron will be compatible with the different OVN versions (with and without “other_config” field).
Introduce the experimental features framework.
Some Neutron features are not supported due to lack of resources or technical expertise to maintain them. As they arise, those features will be marked as experimental by the Neutron core team. Deployers will be able to continue using experimental features by explicitly enabling them in the ‘experimental’ section of neutron.conf. The ML2 linuxbridge driver is the first feature to be marked as experimental. To continue using it, deployers have to set to True the ‘linuxbridge’ option in the ‘experimental’ section of neutron.conf.
Add support for port ranges in the port forwarding rules. The supported ranges are N:M with N <= M. Also, the ranges of internal and external ports relation must be: internal range = external range or internal range = 1.
After the port is considered as provisioned, the Nova port binding update could have not been received, leaving the port as not bound. Now the port provisioning method has an active wait that will retry several times, waiting for the port binding update. If received, the port status will be set as active if the admin state flag is set.
Support for IPv6 NDP proxy has been added. Read the related specification for more details.
Support for baremetal provisioning using OVN’s built-in DHCP server has been added for IPv4.
Added support for QoS minimum bandwidth rules (egress only) in ML2/OVN. OVN supports setting these rule types in the logical switch ports since release 22.06.0.
OVN mechanism driver refuses to bind a port to a dead agent.
Core OVN now can set the destination host on the logical switch port during a live migration. That allows to prepare the destination host earlier, achieving a quicker live migration and a lower downtime during the switch between hosts. Neutron includes this information in the port options.
Added support for router gateway IP QoS in OVN backend. The L3 OVN router plugin now can apply router QoS policy rules on the router gateway port.
Ovn configuration items “ovn_nb_connection” and “ovn_sb_connection” can set multiple addresses separated by commas. Setting NB/SB “connection” inactivity probe can also work well, if multiple connection be specified.
Added a new configuration variable, in
[OVS]section, to control the OVS OpenFlow rule processing operations when using the OVS native firewall driver (
openflow_processed_per_port: by default “False”. If enabled, all OpenFlow rules associated to a port will be processed at once, in a single transaction. If disabled, the flows will be processed in batches of “AGENT_RES_PROCESSING_STEP=100” number of OpenFlow rules.
uplink-status-propagationextension is enabled, all existing ports before enabling it will have the flag “propagate_uplink_status” enabled by default. This is aligned with the aim of an administrator that enables this extension. Now only new ports can be created with this flag disabled.
Gateway IP QoS network inheritance is now available for OVN L3 plugin QoS extension. If the router external network (gateway network) has a QoS policy associated, the gateway IP port will inherit the network QoS policy.
QoS rule type list accepts two filter flags:
all_supported: if True, the listing call will print all QoS rule types supported by at least one loaded mechanism driver.
all_rules: if True, the listing call will print all QoS rule types supported by the Neutron server.
Both filter flags are exclusive and not required.
DbQuotaDriverNullas production ready database quota driver. This driver does not have access to the database and will return empty values to the request queries. This driver can be used to override the Neutron quota engine.
A new script to remove the duplicated port bindings was added. This script will list all
ml2_port_bindingsrecords in the database, finding those ones with the same port ID. Then the script removes those ones with status=INACTIVE. This script is useful to remove those leftovers that remain in the database after a failed live migration. It is important to remark that this script should not be executed during any live migration process.
use_random_fullysetting to allow an operator to disable the iptables random-fully property on an iptable rules.
use_random_fullysetting is disabled, it will prevent random fully from being used and if there’re 2 guests in different networks using the same source_ip and source_port and they try to reach the same dest_ip and dest_port, packets might be dropped in the kernel do to the racy tuple generation . Disabling this setting should only be done if source_port is really important such as in network firewall ACLs and that the source_ip are never repeating within the platform.
Previously deprecated configuration option
allow_overlapping_ipsis now removed.
Python 3.6 & 3.7 support has been dropped. The minimum version of Python now supported is Python 3.8.
A new configuration option called
[ovn]/disable_ovn_dhcp_for_baremetal_portshas been added to ML2/OVN for IPv4. Since PXE booting nodes can be very sensitive depending on the hardware and some operators may prefer to use a fully-fledged DHCP server instead of OVN’s DHCP server this option allows for disabling OVN’s built-in DHCP server for baremetal ports (vnic type “baremetal”) when set to True. It defaults to False.
live_migration_eventsconfiguration option is removed. Now Neutron assumes this flag is always set. This configuration option depended on the Nova patch only wait for plugtime events in pre-live-migration.
The ML2 linuxbridge agent has been marked as experimental due to lack of resources to maintain it. To continue using it, deployers have to set to True the ‘linuxbridge’ option in the ‘experimental’ section of neutron.conf
1942329 Port binding logic for direct-physical ports has been extended to allow providing the MAC address of the physical device via the binding profile. If it is provided then Neutron overwrites the value of the
device_mac_addressfield of the port object in the database with the value from the active binding profile. If there are ports bound before the nova side of this fix is depolyed then the VM using the port needs to be moved or the port needs to be detached and re-attached to force nova to provide the MAC address of the direct-physical port in the port binding.
Forbid the creation of a duplicate NDP proxy entry on the same router, since the IP address of a router is unique and an IPv6 address only needs one NDP proxy.
Fixes an issue in the ML2/OVN driver where the network segment tag was not being updated in the OVN Northbound database. For more information, see bug 1944708.
The OVN migration performs validation by default. This validation means an instance is spawned and is tested by simple ping after the migration is finished. Also it tries to create new workload post migration. This is useful for very simple scenarios when migration is tested but is not really useful in production since likely the production envrionments already have running workloads. It makes more sense to require the validation explicitly rather than implicitly run it as the migration is mostly intended for production. The VALIDATE_MIGRATION now defaults to False and needs to be changed to True if validation upon request.
From now on, gateway interface will be kept up on all nodes where HA router is hosted, regardless of their state (active or standby). For more information see bug 1952907.
OVN driver reverted to using stateful NAT for floating IP implementation. The previous switch to stateless didn’t materialize the expected performance benefits and instead introduced problems with potential hardware offloading.