Stein Series Release Notes


Deprecation Notes

  • Abstract method plug_new from the neutron.agent.linux.interface.LinuxInterfaceDriver class now accepts an optional parameter link_up. Usage of this method, which takes from 5 to 9 positional arguments, without link_up is now deprecated and will not be possible starting in the W release. Third-party drivers which inherit from this base class should update the implementation of their plug_new method.

Security Issues

  • A change was made to the metadata proxy to not allow a user to override header values, it will now always insert the correct information and remove unnecessary fields before sending requests to the metadata agent. For more information, see bug 1865036.

Bug Fixes

  • Bug described a flooding issue on the neutron-ovs-agent integration bridge. And bug proposed a solution for it. The accepted egress packets will be taken care in the final egress tables (61 when openflow firewall is not enabled, table 94 otherwise) with direct output flows for unicast traffic with a minimum influence on the existing cloud networking. A new config option explicitly_egress_direct, with default value False, was added for the aim of distinguishing clouds which are running the network node mixed with compute services, upstream neutron CI should be an example. In such situation, this explicitly_egress_direct should be set to False, because there are numerous cases from HA routers which can not be covered, particularly when you have centralized floating IPs running in such mixed hosts. Otherwise, set explicitly_egress_direct to True to avoid the flooding. One more note is if your network nodes are for networing services only, we recommand you disable all the security_group to get a higher performance.

  • Fixes an issue that the OVS firewall driver does not configure security group rules using remote group properly when a corresponding remote group has no port on a local hypervisor. For more information see bugs: 1862703 and 1854131.

  • Add a new match rule based on physical VLAN tag for OpenFlow firewall traffic identifying mechanism to the TRANSIENT table. This fixes the distributed router east-west traffic between VLAN type networks. For more information, see bug 1831534.


Upgrade Notes

  • For users affected by bug 1853840 the hypervisor name now can be set per physical network device in config option resource_provider_hypervisors which is located in the [ovs] ini-section for ovs-agent and [sriov_nic] ini-section for sriov-agent. Hypervisor names default to socket.gethostname() which works out of the box with libvirt even when the config option is set to a non-default value.

Bug Fixes

  • Neutron now locates the root resource provider of the resource provider tree it creates by using the hypervisor name instead of the hostname. These are different in rare cases only. The hypervisor name can be set per physical network device in config option resource_provider_hypervisors which is located in the [ovs] ini-section for ovs-agent and [sriov_nic] ini-section for sriov-agent. Hypervisor names default to socket.gethostname() which works out of the box with libvirt even when the config option is set to a non-default value. We believe this change fixes bug 1853840.

  • Owners of security groups now see all security group rules which belong to the security group, even if the rule was created by the admin user. Fixes bug 1824248.

Other Notes

  • A new config option, host_dvr_for_dhcp, was added to neutron.conf for DVR to determine whether to host the DVR local router to the scheduled DHCP node(s).


Bug Fixes

  • [bug 1812168] Remove Floating IP DNS record upon associated port deletion.

Other Notes

  • A new config option, radvd_user, was added to l3_agent.ini for the L3 agent. This option defines the username passed to radvd, used to drop “root” privileges and change user ID to username and group ID to the primary group of the user. If no user specified (by default), the user executing the L3 agent will be passed. If “root” specified, because radvd is spawned as root, no “username” parameter will be passed. (For more information see bug 1844688.)


Security Issues

  • The OVS Firewall blocks traffic that does not have either the IPv4 or IPv6 ethertypes at present. This is a behavior change compared to the iptables_hybrid firewall, which only operates on IP packets and thus does not address other ethertypes. There is now a configuration option in the neutron openvswitch agent configuration file for permitted ethertypes and then ensures that the requested ethertypes are permitted on initialization.

Bug Fixes

  • Fixes an issue where deletion of a provider network could result in ML2 mechanism drivers not being passed information about the network’s provider fields. The consequences of this depend on the mechanism driver in use, but could result in the event being ignored, leading to an incorrectly configured network. See bug 1841967 for details.

  • When updating the fixed-ips of a port residing on a routed provider network the port update would always fail if host was not set. See bug: 1844124.


New Features

  • Added support for custom scripts used to kill external processes managed by neutron agents, such as dnsmasq or keepalived. Such custom scripts, if defined, will be used instead default kill command to kill such external processes.

Upgrade Notes

  • The first address in an IPv6 network is now a valid, usable IP for routers. It had previously been reserved, but now can be assigned to a router so that an IPv6 address ending in “::” could be a valid default route.

Bug Fixes

  • Previously a network’s dns_domain attribute was ignored by the DHCP agent. With this release, OpenStack deployments using Neutron’s DHCP agent will be able to specify a per network dns_domain and have instances configure that domain in their dns resolver configuration files (Linux’s /etc/resolv.conf) to allow for local partial DNS lookups. The per-network dns_domain value will override the DHCP agent’s default dns_domain configuration value. Note that it’s also possible to update a network’s dns_domain, and that new value will be propogated to new instances or when instances renew their DHCP lease. However, existing leases will live on with the old dns_domain value.

Other Notes

  • In order to improve heavy load ovs agent restart success rate, instead a retry or fullsync, the native driver of_connect_timeout and of_request_timeout are now set to 300s. The value does not have side effect for the regular pressure ovs agent.

  • A new option [ovs] of_inactivity_probe has been added to allow changing the inactivity probe interval when using the OVS ML2 agent with the native OpenFlow driver. Operators can increase this if they are experiencing OpenFlow timeouts. The default value is 10 seconds.



Add new tool neutron-status upgrade check.

Added support for network segment range management. This introduces the ability for administrators to control the segment ranges globally or on a per-tenant basis via the Neutron API.

Support alias end points for rules in QoS API.

Existing subnets that were created outside of a subnet pool can know be moved, or “onboarded” into an existing subnet pool. This provides a way for subnets to be brought under the management of a subnet pool and begin participating in an address scope. By enabling onboarding, existing subnets can be used with features that build on subnet pools and address scopes. Subnet onboarding is subject to all the same restrictions as and guarantees currently enforced by subnet pools and address scopes.

New Features

  • New framework for neutron-status upgrade check command is added. This framework allows adding various checks which can be run before a Neutron upgrade to ensure if the upgrade can be performed safely. Stadium and 3rd party projects can register their own checks to this new neutron-status CLI tool using entrypoints in neutron.status.upgrade.checks namespace.

  • Add support for listing floating ip pools (subnets) in L3 plugin. A new API resource floatingip-pools is introduced. This API endpoint can return a list of floating ip pools which are essentially mappings between network UUIDs and subnet CIDRs. Users can use this API to find out the pool to create the floating IPs.

  • Before Stein, network segment ranges were configured as an entry in ML2 config file /etc/neutron/plugins/ml2/ml2_conf.ini that was statically defined for tenant network allocation and therefore had to be managed as part of the host deployment and management. The new network-segment-range API extension has been introduced, which exposes the network segment ranges to be administered via API. This allows users with admin privileges to be able to dynamically manage the shared and/or tenant specific network segment ranges. Standard attributes with tagging support are introduced to the new resource. The feature is controlled by the newly-added service plugin network_segment_range. A set of default network segment ranges will be created out of the ranges that are defined in the host ML2 config file /etc/neutron/plugins/ml2/ml2_conf.ini, such as network_vlan_ranges, vni_ranges for ml2_type_vxlan, tunnel_id_ranges for ml2_type_gre and vni_ranges for ml2_type_geneve.

  • L3 agent supports QoS bandwidth limit functionality for port forwarding floating IPs now. If floating IP has binding QoS policy (with bandwidth limit rules), the traffic bandwidth will be limited.

  • Introduce the attribute propagate_uplink_status to ports. Right now, the SRIOV mechanism driver leverages this attribute to decide if the VF link should follow the state of the PF. For example, if the PF is down, the VF link state is automatically set to down as well. Operators can turn on this feature via the configuration option:

    extension_drivers = uplink_status_propagation

    The API extension uplink_status_propagation is introduced to indicate if this feature is turned on.

  • Add config option rpc_response_max_timeout to configure the maximum time waiting for an RPC response.

  • Security groups are now supported via the network RBAC mechanism. Please refer to the admin guide for further details.

  • New configuration options for neutron-ovs-agent under section [ovs]: resource_provider_bandwidths and resource_provider_inventory_defaults. The former controls the total (available bandwidth) field of the physical network interface resource provider inventories. It defaults to not creating resource providers in Placement. The latter can be used to tune the other fields (allocation_ratio, min_unit, max_unit, reserved, step_size) of resource provider inventories.

  • New configuration options for neutron-sriov-agent under section [sriov_nic]: resource_provider_bandwidths and resource_provider_inventory_defaults. The former controls the total (available bandwidth) field of the physical network interface resource provider inventories. It defaults to not creating resource providers in Placement. The latter can be used to tune the other fields (allocation_ratio, min_unit, max_unit, reserved, step_size) of resource provider inventories.

  • A new config option resync_throttle has been added for Neutron DHCP agent. This new option allows to throttle the number of resync state events between the local DHCP state and Neutron to only once per resync_throttle seconds. Default value for this new option is set to 1 and it should be configured per a user’s specific scenario, i.e. how responsive the user would like his/her system to be for those DHCP resync state events. The option is introduced together with the event driven periodic task for DHCP agents. This enhances the agent with a faster reaction on the resync request but ensuring a minimum interval taken between them to avoid too frequent resyncing. For more information see bug 1780370.

  • The Neutron L3 and DHCP agents now dynamically tune the number of processing greenthreads they run based on the number of objects they are managing, with the current values for this range being between eight and thirty-two threads, which is an increase over the previous static value of eight threads. This should help address some of the scaling problems in the agents. For more information see bug 1813787.

  • A new attribute qos_policy_id is added to the L3 router gateway.

    • It enables users to associate QoS policies to L3 router gateways to control the rate of transmission of the associated SNAT traffic.

    • At the moment, only bandwidth limit rules are supported in the QoS polices.

    • To enable this feature, the qos service plugin has to be configured in the Neutron server and the gateway_ip_qos extension has to be configured in the L3 agents. Please refer to the QoS section of the OpenStack Networking Guide for more specific details.

  • Add get_standard_device_mappings to SriovNicSwitchMechanismDriver and OpenvswitchMechanismDriver so they can return the interface or bridge mappings in a standard way. The common format is a dict like: {‘physnet_name’: [‘device_or_bridge_1’, ‘device_or_bridge_2’]}.

  • The qos-rules-alias API extension was implemented to enable users to perform GET, PUT and DELETE operations on QoS rules as though they are first level resources. In other words, the user doesn’t have to specify the QoS policy ID.

  • Neutron child processes now set their process titles to match their roles (‘api worker’, ‘rpc worker’, ‘periodic worker’, ‘services worker’, or any other defined by workers from out-of-tree plugins.) This behavior can be disabled by setting the setproctitle config option in the [default] section in neutron.conf to off. The original process string is also appended to the end, to help with scripting that is looking for the old strings. There is also an option called brief, which results in much shorter and easier to read process names. The default setting for this option is on, for a combination of backwards compatibility and identifying different processes easily. The recommended setting is brief, once the deployer has verified that none of their tooling depends on the older strings.

  • Existing subnets can now be moved into a subnet pool, and by extension can be moved into address scopes they were not initially participating in.

Upgrade Notes

  • Operator can now use new CLI tool neutron-status upgrade check to check if Neutron deployment can be safely upgraded from N-1 to N release.

  • Adds Floating IP port forwarding table column protocol to the uniq constraints. In one expand script, we drop the original uniq constraints first, then create the new uniq constraints with column protocol.

  • The external_network_bridge config option has been removed. Existing users of this option will now have their router’s gateway interface created in the integration bridge and it will be wired by the L2 agent.

  • The number of api and rpc workers may change on upgrade. It is strongly recommended that all deployers set these values in their neutron configurations, rather than using the defaults.

  • The deprecated ovsdb_interface configuration option has been removed, the default native driver is now always used. In addition, the deprecated ovs_vsctl_timeout option, which was renamed to ovsdb_timeout in Queens, has also been removed.

  • During the dependency resolution procedure, the code that loads service plugins was refactored to not raise an exception if one plugin is configured multiple times, with the last one taking effect. This is a change from the previous behavior.

  • The change to the process title happens by default with the new setproctitle config option. The old string is still part of the new process title, but any scripts looking for exact string matches of the old string may need to be modified.

  • The Neutron API now enforces that ports are a valid option for security group rules based on the protocol given, instead of relying on the backend firewall driver to do this enforcement, typically silently ignoring the port option in the rule. The valid set of whitelisted protocols that support ports are TCP, UDP, UDPLITE, SCTP and DCCP. Ports used with other protocols will now generate an HTTP 400 error. For more information, see bug 1818385.

Deprecation Notes

  • The signature of notifications for resource agent for events after_create and after_update was extended. A new keyword argument was added: status. This is to make the same status information available to notification consumers as it was available already where the notification is sent in class AgentDbMixin. Valid status values are defined in neutron_lib.agent.constants. Consuming notifications by the old signature is deprecated. Unless processing arguments as **kwargs, out-of-tree notification consumers need to adapt.

  • Function get_binding_levels from neutron.plugins.ml2.db module is deprecated and will be removed in the future. New function get_binding_levels_objs should be used instead. This new function returns PortBindingLevel OVO objects.

  • The L2 population agent_boot_time config option is deprecated in favor of the direct RPC agent restart state transfer. It will be removed in the Train release.

Critical Issues

  • The neutron-openvswitch-agent can sometimes spend too much time handling a large number of ports, exceeding its timeout value, agent_boot_time, for L2 population. Because of this, some flow update operations will not be triggerred, resulting in lost flows during agent restart, especially for host-to-host vxlan tunnel flows, causing the original tunnel flows to be treated as stale due to the different cookie IDs. The agent’s first RPC loop will also do a stale flow clean-up procedure and delete them, leading to a loss of connectivity. Please ensure that all neutron-server and neutron-openvswitch-agent binaries are upgraded for the changes to take effect, after which the L2 population agent_boot_time config option will no longer be used.

Bug Fixes

  • Floating IP port forwardings with different protocols could not have the same internal or external port number to the same VM port. After this fix we will allow creating port forwardings with same internal or external port number in different protocols.

  • Fixes bug 1501206. This ensures that DHCP agent instances running dnsmasq as a DNS server can no longer be exploited as DNS amplifiers when the tenant network is using publicly routed IP addresses by adding an option that will allow them to only serve DNS requests from local networks.

  • Add resource_type into log object query to distinguish between security group and firewall group log objects. For more information see bug 1787119.

  • Fixes an issue causing IP allocation on port update to fail when the initial IP allocation was deferred due to lack of binding info. If both the port mac_address and binding info (binding_host_id) were updated in the same request, the fixed_ips field was added to the request internally. The code to complete the deferred allocation failed to execute in that case. (For more information see bug 1811905.)

  • Neutron API workers default to the number of CPU cores. This can lead to high cpu/low memory boxes getting into trouble. The defaults have been tweaked to attempt to put an upper bound on the default of either the number of cores, or half of system memory, whichever is lower. In addition, the default number of RPC workers has been changed from a value of 1, to a value of half the number of API workers.

  • The neutron-openvswitch-agent was changed to notify the neutron-server in its first RPC loop that it has restarted. This signals neutron-server to provide updated L2 population information to correctly program FDB entries, ensuring connectivity to instances is not interrupted. This fixes the following bugs: 1794991, 1799178, 1813703, 1813714, 1813715.

  • Reject QoS minimum bandwidth rule operations on ports, networks without physnet, see bug 1819029.

Other Notes

  • Support fetching specific db column in OVO. A new method get_values is added to neutron object classes. This method can be leveraged to fetch specific field of the object.

  • If an instance port is under a dvr router, and the port already has binding port forwarding(s). Neutron will no longer allow binding a floating IP to that port again, because dvr floating IP traffic rules will break the existing port forwarding functionality.

  • Add new configuration group ovs_driver and new configuration option under it vnic_type_blacklist, to make the previously hardcoded supported_vnic_types parameter of the OpenvswitchMechanismDriver configurable. The vnic_types listed in the blacklist will be removed from the supported_vnic_types list.

  • Add new configuration group sriov_driver and new configuration option under it vnic_type_blacklist, to make the previously hardcoded supported_vnic_types parameter of the SriovNicSwitchMechanismDriver configurable. The vnic_types listed in the blacklist will be removed from the supported_vnic_types list.

  • The metering agent iptables driver can now load its interface driver by using a stevedore alias in the metering_agent.ini file. For example, interface_driver = openvswitch instead of interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver

  • Neutron server now rejects (as NotImplementedError) updates of minimum_bandwidth QoS rules if the rule is already in effect on bound ports. Implementing updates will require updates to Placement allocations and possibly migrating servers where the new minimum_bandwidth can be satisifed.

  • Neutron now supports having service plugins require other plugin(s) as dependencies. For example, the port_forwarding service plugin requires the router service plugin to achieve full functionality. A new list, required_service_plugins, was added to each service plugin so the required dependencies of each service plugin can be initialized. If one service plugin requires another, but the requirement is not set in the config file, neutron will now initialize it to the plugin directory.

  • Use publish for AGENT's AFTER_CREATE and AFTER_UPDATE events with DBEventPayload instead of the deprecated notify callback.