Current Series Release Notes

Current Series Release Notes

New Features

  • API microversion 2.74 adds support for specifying optional host and/or hypervisor_hostname parameters in the request body of POST /servers. These request a specific destination host/node to boot the requested server. These parameters are mutually exclusive with the special availability_zone format of zone:host:node. Unlike zone:host:node, the host and/or hypervisor_hostname parameters still allow scheduler filters to be run. If the requested host/node is unavailable or otherwise unsuitable, earlier failure will be raised. There will be also a new policy named compute:servers:create:requested_destination. By default, it can be specified by administrators only.

  • When enable_dhcp is set on a subnet but there is no DHCP port on neutron then the dhcp_server value in meta hash will contain the subnet gateway IP instead of being absent.

  • Added a new locked_reason option in microversion 2.73 to the POST /servers/{server_id}/action request where the action is lock. It enables the user to specify a reason when locking a server. This information will be exposed through the response of the following APIs:

    • GET servers/{server_id}

    • GET /servers/detail

    • POST /servers/{server_id}/action where the action is rebuild

    • PUT servers/{server_id}

    In addition, locked will be supported as a valid filter/sort parameter for GET /servers/detail and GET /servers so that users can filter servers based on their locked value. Also the instance action versioned notifications for the lock/unlock actions now contain the locked_reason field.

  • In this release support was added for two additional libvirt video models: gop, the UEFI graphic output protocol device model; and the none device model. Existing support for virtio has been extended to all architectures and may now be requested via the hw_video_model image metadata property. Prior to this release the virtio video model was unconditionally enabled for AARCH64. This is unchanged but it can now be explicitly enabled on all supported architectures. The none video model can be used to disable emulated video devices when using pGPU or vGPU passthrough.

  • The scheduler can now use placement to more efficiently query for hosts that support the disk_format of the image used in a given request. The [scheduler]/query_placement_for_image_type_support config option enables this behavior, but must not be turned on until all computes have been upgraded to this version and thus are exposing image type support traits.

  • An option --before has been added to nova-manage db archive_deleted_rows command. This options limits archiving of records to those deleted before the specified date.

  • A mandatory scheduling pre-filter has been added which will exclude disabled compute nodes where the related nova-compute service status is mirrored with a COMPUTE_STATUS_DISABLED trait on the compute node resource provider(s) for that service in Placement. See the admin scheduler configuration docs for details.

  • The Quobyte Nova volume driver now supports identifying Quobyte mounts via the mounts fstype field, which is used by Quobyte 2.x clients. The previous behaviour is deprecated and may be removed from the Quobyte clients in the future.

  • In this release SR-IOV live migration support is added to the libvirt virt driver for Neutron interfaces. Neutron SR-IOV interfaces can be grouped into two categories, direct mode interfaces and indirect. Direct mode SR-IOV interfaces are directly attached to the guest and exposed to the guest OS. Indirect mode SR-IOV interfaces have a software interface such as a macvtap between the guest and the SR-IOV device. This feature enables transparent live migration for instances with indirect mode SR-IOV devices. As there is no generic way to copy hardware state during a live migration, direct mode migration is not transparent to the guest. For direct mode interfaces, we mimic the workflow already in place for suspend and resume. For instance with SR-IOV devices, we detach the direct mode interfaces before migration and re-attach them after the migration. As a result, instances with direct mode SR-IOV port will lose network connectivity during a migration unless a bond with a live migratable interface is created within the guest.

Upgrade Notes

  • The [DEFAULT]/block_device_allocate_retries configuration option now has a minimum required value of 0. Any previous configuration with a value less than zero will now result in an error.

  • The libvirt driver’s RBD imagebackend no longer supports setting force_raw_images to False. Setting force_raw_images = False and images_type = rbd in nova.conf will cause the nova compute service to refuse to start. To fix this, set force_raw_images = True. This change was required to fix the bug 1816686.

    Note that non-raw cache image files will be removed if you set force_raw_images = True and images_type = rbd now.

  • Live migration of an instance with PCI devices is now blocked in the following scenarios:

    1. Instance with non-network related PCI device.

    2. Instance with network related PCI device and either:
      1. Neutron does not support extended port binding API.

      2. Source or destination compute node does not support libvirt-sriov-live-migration.

    Live migration will fail with a user friendly error.


    Previously, the operation would have failed with an obscure error. Ending up with the instance still running on the source node or ending up in an inoperable state.

  • The max_concurrent_live_migrations configuration option has been restricted by the minimum value and now raises a ValueError if the value is less than 0.

  • If you upgraded your OpenStack deployment to Stein without switching to use the now independent placement service, you must do so before upgrading to Train. Instructions for one way to do this are available.

  • It is now possible to count quota usage for cores and ram from the placement service and instances from instance mappings in the API database instead of counting resources from cell databases. This makes quota usage counting resilient in the presence of down or poor-performing cells.

    Quota usage counting from placement is opt-in via the [quota]count_usage_from_placement configuration option.

    There are some things to note when opting in to counting quota usage from placement:

    • Counted usage will not be accurate in an environment where multiple Nova deployments are sharing a placement deployment because currently placement has no way of partitioning resource providers between different Nova deployments. Operators who are running multiple Nova deployments that share a placement deployment should not set the [quota]count_usage_from_placement configuration option to True.

    • Behavior will be different for resizes. During a resize, resource allocations are held on both the source and destination (even on the same host, see until the resize is confirmed or reverted. Quota usage will be inflated for servers in the VERIFY_RESIZE state and operators should weigh the advantages and disadvantages before enabling [quota]count_usage_from_placement.

    • The populate_queued_for_delete and populate_user_id online data migrations must be completed before usage can be counted from placement. Until the data migration is complete, the system will fall back to legacy quota usage counting from cell databases depending on the result of an EXISTS database query during each quota check, if [quota]count_usage_from_placement is set to True. Operators who want to avoid the performance hit from the EXISTS queries should wait to set the [quota]count_usage_from_placement configuration option to True until after they have completed their online data migrations via nova-manage db online_data_migrations.

    • Behavior will be different for unscheduled servers in ERROR state. A server in ERROR state that has never been scheduled to a compute host will not have placement allocations, so it will not consume quota usage for cores and ram.

    • Behavior will be different for servers in SHELVED_OFFLOADED state. A server in SHELVED_OFFLOADED state will not have placement allocations, so it will not consume quota usage for cores and ram. Note that because of this, it will be possible for a request to unshelve a server to be rejected if the user does not have enough quota available to support the cores and ram needed by the server to be unshelved.

  • The --version argument has been removed in the following commands. Use the VERSION positional argument instead.

    • nova-manage db sync

    • nova-manage api_db sync

  • The cells v1 feature has been deprecated since the 16.0.0 Pike release and has now been removed. The nova-cells service and nova-manage cells commands have been removed, while the nova-manage cell_v2 simple_cell_setup command will no longer check if cells v1 is enabled and therefore can no longer exit with 2.

    The cells v1 specific REST APIs have been removed along with their related policy rules. Calling these APIs will now result in a 410 (Gone) error response.

    • GET /os-cells

    • POST /os-cells

    • GET /os-cells/capacities

    • GET /os-cells/detail

    • GET /os-cells/info

    • POST /os-cells/sync_instances

    • GET /os-cells/{cell_id}

    • PUT /os-cells/{cell_id}

    • DELETE /os-cells/{cell_id}

    • GET /os-cells/{cell_id}/capacities

    The cells v1 specific policies have been removed.

    • cells_scheduler_filter:DifferentCellFilter

    • cells_scheduler_filter:TargetCellFilter

    The cells v1 specific configuration options, previously found in cells, have been removed.

    • enabled

    • name

    • capabilities

    • call_timeout

    • reserve_percent

    • cell_type

    • mute_child_interval

    • bandwidth_update_interval

    • instance_update_sync_database_limit

    • mute_weight_multiplier

    • ram_weight_multiplier

    • offset_weight_multiplier

    • instance_updated_at_threshold

    • instance_update_num_instances

    • max_hop_count

    • scheduler

    • rpc_driver_queue_base

    • scheduler_filter_classes

    • scheduler_weight_classes

    • scheduler_retries

    • scheduler_retry_delay

    • db_check_interval

    • cells_config

    In addition, the following cells v1 related RPC configuration options, previously found in upgrade_levels, have been removed.

    • cells

    • intercell

  • The [DEFAULT]/default_flavor option deprecated in 14.0.0 (Newton) has been removed.

  • Config option [ironic]api_endpoint was deprecated in the 17.0.0 Queens release and is now removed. To achieve the same effect, set the [ironic]endpoint_override option. (However, it is preferred to omit this setting and let the endpoint be discovered via the service catalog.)

  • The Libvirt SR-IOV migration feature intoduced in this release requires both the source and destination node to support the feature. As a result it will be automatically disabled until the conductor and compute nodes have been upgraded.

  • The block-storage (cinder) version 3.44 API is now required when working with volume attachments. A check has been added to the nova-status upgrade check command for this requirement.

  • To resolve bug 1805659 the default value of [notifications]/notification_format is changed from both to unversioned. For more information see the documentation of the config option. If you are using versioned notifications, you will need to adjust your config to versioned

Deprecation Notes

  • The RetryFilter is deprecated and will be removed in an upcoming release. Since the 17.0.0 (Queens) release, the scheduler has provided alternate hosts for rescheduling so the scheduler does not need to be called during a reschedule which makes the RetryFilter useless. See the Return Alternate Hosts spec for details.

  • Compatibility code for compute drivers that do not implement the update_provider_tree interface is deprecated and will be removed in a future release.

Bug Fixes

  • Fixes a bug causing mount failures on systemd based systems that are using the systemd-run based mount with the Nova Quobyte driver.

  • The os-volume_attachments update API, commonly referred to as the swap volume API will now return a 400 (BadRequest) error when attempting to swap from a multi attached volume with more than one active read/write attachment resolving bug #1775418.

  • Bug 1811726 is fixed by deleting the resource provider (in placement) associated with each compute node record managed by a nova-compute service when that service is deleted via the DELETE /os-services/{service_id} API. This is particularly important for compute services managing ironic baremetal nodes.

  • Unsetting ‘[DEFAULT] dhcp_domain’ will now correctly result in the metadata service/config drive providing an instance hostname of ‘${hostname}’ instead of ‘${hostname}None’, as was previously seen.

  • Fixes a bug that caused Nova to fail on mounting Quobyte volumes whose volume URL contained multiple registries.

  • Add support for noVNC >= v1.1.0 for VNC consoles. Prior to this fix, VNC console token validation always failed regardless of actual token validity with noVNC >= v1.1.0. See for more details.

  • There had been bug 1777591 that placement filters out the specified target host when deploying an instance by the random limitation. In previous releases the bug has been worked around by unlimiting the results from the Placement service if the target host is specified. From this release, the Nova scheduler uses more optimized path retrieving only the target host information from placement. Note that it still uses the unlimit workaround if a target host is specified without a specific node and multiple nodes are found for the target host. This can happen in some of the virt drivers such as the Ironic driver.

  • Update the way QEMU cache mode is configured for Nova guests: If the file system hosting the directory with Nova instances is capable of Linux’s O_DIRECT, use none; otherwise fallback to writeback cache mode. This improves performance without compromising data integrity. Bug 1818847.

    Context: What makes writethrough so safe against host crashes is that it never keeps data in a “write cache”, but it calls fsync() after every write. This is also what makes it horribly slow. But cache mode none doesn’t do this and therefore doesn’t provide this kind of safety. The guest OS must explicitly flush the cache in the right places to make sure data is safe on the disk; and all modern OSes flush data as needed. So if cache mode none is safe enough for you, then writeback should be safe enough too.

Other Notes

  • A new [libvirt]/rbd_connect_timeout configuration option has been introduced to limit the time spent waiting when connecting to a RBD cluster via the RADOS API. This timeout currently defaults to 5 seconds.

    This aims to address issues reported in bug 1834048 where failures to initially connect to a RBD cluster left the nova-compute service inoperable due to constant RPC timeouts being hit.

  • Numbered request groups can be defined in the flavor extra_spec but they can come from other sources as well (e.g. neutron ports). If there is more than one numbered request group in the allocation candidate query and the flavor does not specify any group policy then the query will fail in placement as group_policy is mandatory in this case. Nova previously printed a warning to the scheduler logs but let the request fail. However the creator of the flavor cannot know if the flavor later on will be used in a boot request that has other numbered request groups. So nova will start defaulting the group_policy to ‘none’ which means that the resource providers fulfilling the numbered request groups can overlap. Nova will only default the group_policy if it is not provided in the flavor extra_spec, and there is more than one numbered request group present in the final request, and the flavor only provided one or zero of such groups.

  • A --dry-run option has been added to the nova-manage placement heal_allocations CLI which allows running the command to get output without committing any changes to placement.

  • An --instance option has been added to the nova-manage placement heal_allocations CLI which allows running the command on a specific instance given its UUID.

  • The dhcp_domain option has been undeprecated and moved to the [api] group. It is used by the metadata service to configure fully-qualified domain names for instances, in addition to its role configuring DHCP services for nova-network. This use case was missed when deprecating the option initially.

Creative Commons Attribution 3.0 License

Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.