Current Series Release Notes

38.0.0-13

New Features

• We add a new argument, --prefix, to tempest cleanup tool that will allow users delete only resources that match the prefix. When this option is used, saved_state.json file is not needed (no need to run with --init-saved-state first). If there is one, it will be ignored and the cleanup will be done based on the given prefix only. Note, that some resources are not named thus they will not be deleted when filtering based on the prefix.

• A new config option in the volume_feature_enabled section, volume_types_for_data_volume, is added to allow the user to specify which volume types can be used for data volumes in a new test test_instances_with_cinder_volumes_on_all_compute_nodes. By default, this option is set to None.

Upgrade Notes

• The following decorators are no longer available in the tempest.test module. Use the tempest.common.utils module instead.

  • services

  • requires_ext

  • is_extension_enabled

• Default value of the [image-feature-enabled] image_import has been changed from False to True, and now the image import feature is tested by default.

• Default value of the [image-feature-enabled] os_glance_reserved has been changed from False to True and now the reservation of os_glance namespace is tested by default.

Deprecation Notes

• The [image-feature-enabled] image_import option has been deprecated. The image import feature works in both standalone mode and WSGI mode since Victoria and the image import feature can be always tested.

• The [image-feature-enabled] os_glance_reserved option has been deprecatd because glance reserves the os_glance namespace since Wallaby.

38.0.0

Prelude

This release is to tag Tempest for OpenStack 2024.1 release. This release marks the start of 2024.1 release support in Tempest. After this release, Tempest will support below OpenStack Releases: * 2024.1 * 2023.2 * 2023.1 * Zed Current development of Tempest is for OpenStack 2024.2 development cycle. Every Tempest commit is also tested against master during the 2024.2 cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a 2024.2 (or future release) cloud. To be on safe side, use this tag to test the OpenStack 2024.1 release.

New Features

• Add a new config options migration_source_host and migration_dest_host in the compute section, which if is set takes source or destination host from options, otherwise a host is chosen automatically.

Upgrade Notes

• The deprecated [network] dns_servers option has been removed.

37.0.0

Prelude

This is an intermediate release during the 2024.1 development cycle to mark the end of support for Yoga release in Tempest. After this release, Tempest will support below OpenStack Releases: * 2023.2 * 2023.1 * Zed Current development of Tempest is for OpenStack 2024.1 development cycle.

New Features

• A new config option in the default section, resource_name_prefix, is added to allow users to customize the name (specifically the prefix) of the resources created by Tempest during a test run. By default it is set to tempest. Tempest cleanup CLI will then use this config option to cleanup only the resources that match the prefix. Make sure this prefix does not match with the resource name you do not want Tempest cleanup CLI to delete.

• Add delete image from specific store API to image V2 client

• A new tempest.conf option called backup_driver was added to tempest to indicate which backup driver is used by Cinder. This option allows us to create a resource properly and avoid cleanup issues when Swift is used as the backup driver.

Upgrade Notes

• The following deprecated options in the [identity-feature-enabled] section have been removed. Project tags API and application credentials API are now always tested if identity v3 API is available.

  • project_tag

  • application_credentials

• Default value of the [identity-feature-enabled] access_rule option has been changed from False to True and now the access rule API is always tested when identity API is available.

• The deprecated [compute-feature-enabled] block_migrate_cinder_iscsi option has been removed. Now the [compute-feature-enabled] block_migration_for_live_migration option is solely used to determine when to run block migration based tests during live migration.

• The deprecated [compute-feature-enabled] nova_cert option has been removed. The nova-cert service was removed from nova in 16.0.0 release. Tests of compute root certificates API have also been removed.

• The deprecated [compute-feature-enabled] rdp_console config option has been removed.

• The deprecated [compute-feature-enabled] vnc_server_header option has been removed.

Deprecation Notes

• The Keystone access_rule is enabled by default since Train release and we no longer need a separate config in Tempest to enable it. Therefore the [identity-feature-enabled] access_rule option has been deprecated and will be removed in a future release.

36.0.0

Prelude

Glance v1 APIs were removed in Rocky release and last supported release for v1 was Queens. Tempest master does not support the Rocky or Queens release so we removed the Glance v1 tests, config option, and its service clients. If you would like to test the v1 APIs then you can use the old Tempest version.

Tempest remove the nova-network tests and service clients. The nova-network was removed from Rocky release and current Tempest master does not support the Rocky release. Below are the service clients have been removed: * floating_ip_pools_client * floating_ips_bulk_client * fixed_ips_client * list_virtual_interfaces

This release is to tag Tempest for OpenStack 2023.2 release. This release marks the start of 2023.2 release support in Tempest. After this release, Tempest will support below OpenStack Releases:

• 2023.2

• 2023.1

• Zed

• Yoga

Current development of Tempest is for OpenStack 2024.1 development cycle. Every Tempest commit is also tested against master during the 2024.1 cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a 2024.1 (or future release) cloud. To be on safe side, use this tag to test the OpenStack 2023.2 release.

New Features

• The following tasks_client tempest client for glance v2 image task API is implemented in this release.

• Adding a new config option [identity]/user_minimum_password_age which allows to specify the number of days that a password must be used before the user can change it. For this option to take effect, identity-feature-enabled.security_compliance must be set to True.

35.0.0

Prelude

This is an intermediate release during the 2023.2 development cycle to mark the end of support for EM Xena release in Tempest. After this release, Tempest will support below OpenStack Releases: * 2023.1 * Zed * Yoga Current development of Tempest is for OpenStack 2023.2 development cycle.

New Features

• Add delete_attachment to the v3 AttachmentsClient and terminate_connection to the v3 VolumesClient.

34.2.0

Prelude

This is an intermediate release during the 2023.2 development cycle to make scenario tests server SSHABLE functionality available to plugins and other consumers.

Upgrade Notes

• Update default value of config option CONF.identity.v3_entrypoint_type from adminURL to public. This was deprecated in Q release, and was missed. The default entrypoint used by tempest should be the public one.

34.0.0

Prelude

Adding placement service for config options enforce_scope so that we can switch the scope and new defaults enforcement for placement service.

This release is to tag Tempest for OpenStack 2023.1 release. This release marks the start of 2023.1 release support in Tempest. After this release, Tempest will support below OpenStack Releases:

• 2023.1

• Zed

• Yoga

• Xena

Current development of Tempest is for OpenStack 2023.2 development cycle. Every Tempest commit is also tested against master during the 2023.2 cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a 2023.2 (or future release) cloud. To be on safe side, use this tag to test the OpenStack 2023.1 release.

New Features

• Adds a ssh_allow_agent parameter to the RemoteClient class wrapper and the direct ssh Client class to allow a caller to explicitly request that the SSH Agent is not consulted for authentication. This is useful if your attempting explicit password based authentication as paramiko, the underlying library used for SSH, defaults to utilizing an ssh-agent process before attempting password authentication.

Bug Fixes

• There was a bug (bug#1964509) in dynamic credentials creation where project credentials with different roles are created with the new projects. Credential of different role of projects must be created within the same project. For exmaple, ‘project_admin’, ‘project_member’, ‘project_reader’, and ‘primary’, credentials will be created in the same projects. ‘alt’, ‘project_alt_admin’, ‘project_alt_member’, ‘project_alt_reader’ will be created within the same project.

  ‘admin’ credenatials is considered and kept as legacy admin and will be created under new project. If any test want to test with admin role in projectA and non-admin/admin in projectB then test can request projectA admin using ‘admin’ or ‘project_alt_admin’ and non-admin in projectB using ‘primary’, ‘project_member’, or ‘project_reader’/admin in projectB using ‘project_admin’. Many existing tests using the ‘admin’ with new project to assert on the resource list so we are keeping ‘admin’ a kind of legacy admin.

33.0.0

Prelude

This is an intermediate release during the 2023.1 development cycle to mark the end of support for EM Wallaby release in Tempest. After this release, Tempest will support below OpenStack Releases:

• Zed

• Yoga

• Xena

Current development of Tempest is for OpenStack 2023.1 development cycle.

New Features

• The server_external_events tempest client for compute Server External Events API is implemented in this release.

• Add a new config option [image_feature_enabled]/manage_locations which enables tests for the show_multiple_locations=True functionality in glance. In order for this to work you must also have a store capable of hosting images with an HTTP URI.

32.0.0

Prelude

This release is to tag Tempest for OpenStack Zed release. This release marks the start of Zed release support in Tempest. After this release, Tempest will support below OpenStack Releases:

• Zed

• Yoga

• Xena

• Wallaby

Current development of Tempest is for OpenStack 2023.1 development cycle. Every Tempest commit is also tested against master during the 2023.1 cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a 2023.1 (or future release) cloud. To be on safe side, use this tag to test the OpenStack Zed release.

New Features

• The following image_cache tempest client for glance v2 image caching API is implemented in this release.

• Add configuration parameter tempurl_digest_hashlib into object-storage-feature-enabled which configures the hashing algorithm to use for the temp_url tests; defaults to ‘sha256’.

Security Issues

• Swift used to support only ‘sha1’ for temp_url hashing but from many years now ‘sha256’ and ‘sha512’ are also available. These are stronger than ‘sha1’ and tempest now allows configuring which one to use.

31.1.0

New Features

• Added new module net_downtime including the fixture NetDowntimeMeter that can be used to measure how long the connectivity with an IP is lost during certain operations like a server live migration. The configuration option allowed_network_downtime has been added with a default value of 5.0 seconds, which would be the maximum time that the connectivity downtime is expected to last.

31.0.0

Prelude

This is an intermediate release during the Zed development cycle to mark the end of support for EM Ussuri release in Tempest. After this release, Tempest will support below OpenStack Releases:

• Yoga

• Xena

• Wallaby

• Victoria

Current development of Tempest is for OpenStack Zed development cycle.

This is an intermediate release during the Zed development cycle to mark the end of support for EM Victoria release in Tempest. After this release, Tempest will support below OpenStack Releases:

• Yoga

• Xena

• Wallaby

Current development of Tempest is for OpenStack Zed development cycle.

Upgrade Notes

• As the version of cirros used in OpenStack CI does not support SHA-2 signatures for ssh, any connection from a FIPS enabled machine will fail in case validation.ssh_key_type is set to rsa (the default until now). Using ecdsa keys helps us avoid the mentioned issue. From now on, the validation.ssh_key_type option will be set to ecdsa by default for testing simplicity. This change shouldn’t have any drastic effect on any tempest consumer, in case rsa ssh type is required in a consumer’s scenario, validation.ssh_key_type can be overridden to rsa.

• Python 3.6 and 3.7 support has been dropped. Last release of Tempest to support python 3.6 and 3.7 is Temepst 30.0.0. The minimum version of Python now supported by Tempest is Python 3.8.

• Remove deprecated config option api_extensions from compute_feature_enabled groups.

30.0.0

Prelude

This release is to tag Tempest for OpenStack Yoga release. This release marks the start of Yoga release support in Tempest. After this release, Tempest will support below OpenStack Releases:

• Yoga

• Xena

• Wallaby

• Victoria

• Ussuri

Current development of Tempest is for OpenStack Zed development cycle. Every Tempest commit is also tested against master during the Zed cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Zed (or future release) cloud. To be on safe side, use this tag to test the OpenStack Yoga release.

New Features

• Add parameter to specify the SSH key type. Current options are ‘rsa’ (which is the default) and ‘ecdsa’. Tempest now supports the importing and generation of both ‘rsa’ and ‘ecdsa’ SSH key types.

• Add a new client to lists, creates, shows information for, updates and deletes neutron floating ips port forwarding resource.

Deprecation Notes

• Old APIMicroversionFixture classes tempest.api.compute.api_microversion_fixture.APIMicroversionFixture and tempest.api.volume.api_microversion_fixture.APIMicroversionFixture has been deprecated for removal in favor of new location tempest.lib.common.api_microversion_fixture.APIMicroversionFixture

Bug Fixes

• [bug 1948935] The default value of account-generator –concurrency parameter is now set to 2 instead of 1.

29.2.0

New Features

• Added QoS minimum packet rate rule client:

  • create_minimum_packet_rate_rule

  • update_minimum_packet_rate_rule

  • show_minimum_packet_rate_rule

  • list_minimum_packet_rate_rules

  • delete_minimum_packet_rate_rule also

29.1.0

Bug Fixes

• Improve cleanup after Swift testing. Ensures containers are empty before deleting to prevent errors due to delayed execution.

29.0.0

Prelude

This release is to tag Tempest for OpenStack Xena release. This release marks the start of Xena release support in Tempest. After this release, Tempest will support below OpenStack Releases:

• Xena

• Wallaby

• Victoria

• Ussuri

Current development of Tempest is for OpenStack Yoga development cycle. Every Tempest commit is also tested against master during the Yoga cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Yoga (or future release) cloud. To be on safe side, use this tag to test the OpenStack Xena release.

28.1.0

New Features

• Add “QoS bandwidth limit rules” APIs to: “tempest/tests/lib/services/network/test_qos_limit_bandwidth_rules_client.py” module.

  • List bandwidth limit rules for QoS policy

  • Create bandwidth limit rule

  • Show bandwidth limit rule details

  • Update bandwidth limit rule

  • Delete bandwidth limit rule

• Add a new client for keystone’s unified limits API to create and update limits.

• Lists neutron’s Loggable resources API service clients are available in tempest/lib/services/network/loggable_resource_client.py module.

Upgrade Notes

• Application credentials are supported by Keystone since Queens. As Tempest currently supports only much newer OpenStack versions (Ussuri or later), the configuration option which enables application credentials testing (CONF.identity-feature-enabled.application_credentials) is now enabled by default.

• Project tags are supported by Keystone since Queens. As Tempest currently supports only much newer OpenStack versions (Ussuri or later), the configuration option which enables project tags testing (CONF.identity-feature-enabled.project_tags) is now enabled by default.

Deprecation Notes

• Application credentials are supported by Keystone since Queens. As Tempest currently supports only much newer OpenStack versions (Ussuri or later), the configuration option which enables application credentials testing (CONF.identity-feature-enabled.application_credentials) is now deprecated.

• Project tags are supported by Keystone since Queens. As Tempest currently supports only much newer OpenStack versions (Ussuri or later), the configuration option which enables project tags testing (CONF.identity-feature-enabled.project_tags) is now deprecated.

28.0.0

Prelude

This is an intermediate release during the Xena development cycle to mark the end of support for EM Train release in Tempest. After this release, Tempest will support below OpenStack Releases:

• Wallaby

• Victoria

• Ussuri

Current development of Tempest is for OpenStack Xena development cycle.

Tempest’s identity service client TokenClientJSON and V3TokenClientJSON has been removed.

Tempest Service clients for volume v1 APIs are removed.

New Features

• Adding new config option for volume tests which allows to specify the size a volume will be extended by (if a test does extend a volume or needs a new bigger volume). The option is beneficial in case such tests are executed on systems where the chunk size (the minimum size a volume can be extended by) is other than 1 (originally hardcoded in the tests).:

  CONF.volume.volume_size_extend

• Add placement API usage method for evaluating resource class utilization of the resource provider. The following API call is available for tempest from now in the resource_providers_client:

  • GET /resource_providers/{uuid}/usages

• Add a TransfersV355Client to the volume v3 transfer_client library supporting create, list, show, delete, and accept operations for the new Volume Transfers API of the Block Storage API v3. The min_microversion of this API is 3.55.

• Add a new client to lists, creates, shows information for, and updates neutron log resource.

Upgrade Notes

• Tempest’s identity service client TokenClientJSON and V3TokenClientJSON were deprecated since long which are not removed. Please use new service clients TokenClient and V3TokenClient instead.

• Cinder removed the volume v1 APIs in queens release and Tempest now support only stable train onwards release so all the Tempest service clients for volume v1 APIs are now removed.

Deprecation Notes

• Tempest service clients for image v1 APIs (tempest.lib.services.image.v1 module) is deprecated and will be removed once Tempest stop supporting stable Ussuri release which is last release Image v1 APIs are present in glance.

• Tempest service clients for volume v2 APIs (tempest.lib.services.volume.v2 module) is deprecated and will be removed once Tempest stop supporting stable wallaby release which is last release volume v2 APIs are present in cinder.

27.0.0

Prelude

Tempest.scenario.manager is now a stable interface for Tempest plugins.

This release is to tag Tempest for OpenStack Wallaby release. This release marks the start of Wallaby release support in Tempest. After this release, Tempest will support below OpenStack Releases:

• Wallaby

• Victoria

• Ussuri

• Train

Current development of Tempest is for OpenStack Xena development cycle. Every Tempest commit is also tested against master during the Xena cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Xena (or future release) cloud. To be on safe side, use this tag to test the OpenStack Wallaby release.

New Features

• In this release, we made tempest/scenario/manager.py a stable interface ready to be consumed by all tempest plugins. The effort was tracked in the following etherpad [1], and the related review can be listed via [2]:

  • [1] https://etherpad.opendev.org/p/tempest-scenario-manager

  • [2] https://review.opendev.org/#/q/topic:bp/tempest-scenario-manager-stable(status:open+OR+status:merged)

  Some time ago, tempest/scenario/manager.py got copied to most of the plugins and therefore, it diverged - every plugin’s copy had slight differences. In this release, we pushed changes to unify the manager’s methods and improved their APIs in order to have them easier consumable:

  • we added implementations of methods that were often used in plugins’ manager.py however were not implemented in our manager

  • we improved APIs by f.e. adding a kwargs argument so that the consumers are more in control of the data that are passed to tempest clients

  • we modified logic of a few methods so that it complies with the plugins’ manager versions in order to prepare for a situation when the plugins can reuse Tempest manager as much as possible rather than keeping their own copy

  • we made methods consistent w.r.t. names and parameters

  • we split the lengthy methods to have more readable code

  • previously private methods which had a potential to be reused were made public so that it’s clear they are expected to be used in tempest plugins

  • missing docstrings have been added

• Add a new client to handle requests to create and delete assisted volume snapshots.

• Adds a method to images_client to get tasks relevant to a given image. Also adds has_version() method to image versions_client to probe for availability of a given API version.

Upgrade Notes

• img_dir scenario option has been deprecated and it’s being removed. Starting Tempest 25.0.0 release, CONF.scenario.img_file needs a full path for the image. Until this release, old behavior was maintained and kept working however a user needs to specify the full path in CONF.scenario.img_file config option from now on.

• The old deprecated data-ultis from tempest.common.utils has been removed. If you are still using this, use the stable version of data-utils from new location tempest.lib.common.utils.

Other Notes

• A new [compute-feature-enabled]ide_bus config option has been introduced to indicate if the environment supports attaching disks to an instance using an IDE bus.

  This currently defaults to True but should be set to False when using the libvirt OpenStack Nova virt driver and the q35 machine type as support for this bus is no longer provided.

26.1.0

Prelude

This is an intermediate release during the Wallaby development cycle to make new functionality available to plugins and other consumers.

The integrated horizon dashboard test is now moved from tempest-horizon plugin into Tempest. You do not need to install tempest-horizon to run the horizon test which can be run using Tempest itself.

Support for RBAC new system scope is added in Tempest.

New Features

• The following tempest clients for keystone v3 OS_FEDERATION API were implemented in this release

  • identity_providers

  • protocols

  • mappings

  • service_providers

• Add project alternate admin, member and reader role for dynamic credentials.

• Added methods to the identity v3 roles client to support:

  • PUT /v3/system/users/{user}/roles/{role}

  • GET /v3/system/users/{user}/roles

  • GET /v3/system/users/{user}/roles/{role}

  • DELETE /v3/system/users/{user}/roles/{role}

  • PUT /v3/system/groups/{group}/roles/{role}

  • GET /v3/system/groups/{group}/roles

  • GET /v3/system/groups/{group}/roles/{role}

  • DELETE /v3/system/groups/{group}/roles/{role}

• Added public interface create_loginable_secgroup_rule(). Since this interface is meant to be used by tempest plugins, It doesn’t neccessarily require to be private api.

• Added public interface create_security_group_rule(). Since this interface is meant to be used by tempest plugins, It doesn’t neccessarily require to be private api.

• Added public interface log_console_output(). It used to be a private method with name _log_console_output(). Since this interface is meant to be used by tempest plugins, It doesn’t neccessarily require to be private api.

• Keystone provides the new scoped token called system which can be used to query the system scoped API operation. Projects are moving towards the policy with new scope types, Keystone, Nova already provide the new policy for RBAC checks. Tempest has added the support to query the system scoped token from keystone to test the new policy. As next step, we will be moving all the Tempest tests on the project’s new policy.

• Dynamic credentials now support the scope type for specific roles too along with admin, member, reader role. Test can specify the scope in the prefix of cls.credentials name. If system is prefix in cls.credentials name then creds will be created with scope as system. If domain is prefix in cls.credentials name then creds will be created with scope as domain otherwise default project scope will be used. For Example:

  credentials = [['my_role', 'role1'], # this will be old style and project scoped
                 ['project_my_role', 'role1'], # this will be project scoped
                 ['domain_my_role', 'role1'], # this will be domain scoped
                 ['system_my_role', 'role1']] # this will be system scoped
  

  And below is how test can access the credential manager of respective credentials type:

  cls.os_my_role.any_client
  cls.os_project_my_role.any_client
  cls.os_domain_my_role.any_client
  cls.os_system_my_role.any_client
  

  For backward compatibility, we set the credentials manager class attribute in old style form too which is prefix with os_roles_*, example cls.os_roles_my_role but we recommend to use the new style attribute as shown above.

• Adds new personas that can be used to test service policies for all default scopes (project, domain, and system) and roles (reader, member, and admin). Both dynamic credentials and pre-provisioned credentials are supported.

Known Issues

• Scope support for specific role is not yet added for pre-provisioned credentials.

Upgrade Notes

• In this release tempest/manager.py is removed after more than 4 years of deprecation.

• The tempest.lib.common.utils.data_utils.random_bytes() helper function will no longer allow a size of more than 1MiB. Tests generally do not need to generate and use large payloads for feature verification and it is easy to lose track of and duplicate large buffers. The sum total of such errors can become problematic in paralllelized and constrained CI environments.

• Deprecated config options to select the Volume API version have been removed. Use CONF.volume.catalog_type to run volume tests under v3 or v2 APIs.

  • CONF.volume-feature-enabled.api_v2

  • CONF.volume-feature-enabled.api_v3

Deprecation Notes

• In this release the following tempest arguments are deprecated and replaced by new ones which are functionally equivalent:

  • –black-regex is replaced by –exclude-regex

  • –blacklist-file is replaced by –exclude-list

  • –whitelist-file is replaced by –include-list

  For now Tempest supports both (new and old ones) in order to make the transition for all consumers smoother. However, that’s just a temporary case and the old options will be removed soon.

Bug Fixes

• [bug 1613819] admin_domain_name and default_credentials_domain_name parameters under [auth] now affect a domain used for creating test users just as they affect it for projects. Previously a domain with an id set to “default” had to be present in order for test user creation to succeed with Keystone v3.

• Fixes the bug# 1917168

26.0.0

Prelude

This is an intermediate release during the Wallaby development cycle to mark the end of support for EM Stein release in Tempest. After this release, Tempest will support below OpenStack Releases:

• Victoria

• Ussuri

• Train

Current development of Tempest is for OpenStack Wallaby development cycle.

New Features

• Add a new config option can_migrate_between_any_hosts in the compute-feature-enabled section, which can be set to False for environment with non homogeneous compute nodes, so that it can select a destination host for migrating automatically, otherwise the testcase may fail unexpectedly. e.g., if source host is with CPU “E5-2699 v4” and the selected target host is with CPU “E5-2670 v3”, the live-migration will fail because of the downgrade issue.

• Add a new config option [compute-feature-enabled] shelve_migrate which enable test for environment that support cold migration of qcow2 unshelved instance.

• A new config option in the validation section, image_alt_ssh_user, to specify the user name used to authenticate to an alternative instance (instance using image_ref_alt) in tests. By default this is set to root.

• A new config option in the validation section, image_alt_ssh_password, to specify the password used to authenticate to an alternative instance (instance using image_ref_alt) in tests. By default this is set to password.

• Added missing client methods for keystone’s OS-ENDPOINT-POLICY and OS-EP-FILTER APIs.

• Added missing clients and tests for keystone’s v3 EC2 API which already existed for keystone v2.

• Add show type API to v3 types_client library.

  • default_volume_type

• Added associate_floating_ip() and dissociate_floating_ip() methods to the scenario manager.

• Add glance image import APIs function to v2 images_client library.

  • stage_image_file

  • info_import

  • info_stores

  • image_import

• [blueprint blueprint adopt-oslo-versioned-objects-for-db] Any reference to “tenant_id” in Network objects is replaced with “project_id”.

• New config option to network-feature-enabled: available_features. This is a list which can contain features that are not discoverable through Neutron API, or it can be the special entry all.

• Add placement API methods for testing Routed Provider Networks feature. The following API calls are available for tempest from now in the new resource_providers_client:

  • GET /resource_providers

  • GET /resource_providers/{uuid}

  • GET /resource_providers/{uuid}/inventories

  • GET /resource_providers/{uuid}/aggregates

Upgrade Notes

• The following deprecated image scenario options are removed after a ~4 year deprecation period.

  • ami_img_file

  • ari_img_file

  • aki_img_file

  Starting Tempest 25.0.0 release, CONF.scenario.img_file need a full path for the image. CONF.scenario.img_dir was deprecated and will be removed in the next release. Till Tempest 25.0.0, old behavior is maintained and keep working but starting Tempest 26.0.0, you need to specify the full path in CONF.scenario.img_file config option.

• The test_reboot_server_soft has been skipped for more than 6 years. Take into account that the minimum scenario test uses soft reboot and the nova functional test also covers reboot.

• Member role has been deprecated and replaced by member. Therefore the default value of config option [object-storage].operator_role is changed to member. (Fixes bug #1330132)

• Remove the deprecated CLI tempest-account-generator in favor of tempest account-generator command. You can use tempest account-generator CLI to generate the accounts yaml file.

• A number of Compute APIs that only worked with the XenAPI virt driver have been removed in the Compute service. As a result, their corresponding tests are now disabled by default. They can be re-enabled using the new [compute_feature_enabled] xenapi_apis config option.

Bug Fixes

• Fixed bug #1890060. tempest subunit_describe_calls –verbose not working with Cliff CLI. The subunit_describe_calls –verbose argument was a boolean and worked in the non Cliff CLI which is now deprecated, but does not work with cliff since –verbase is a standard cliff argument which is an int. Since the tool is in lib directory we cannot change the interface, so we add a new argument -a –all-stdout that will allow cliff CLI to support the feature in subunnit_describe_calls to print request and response headers and bodies to stdout.

• is_resource_deleted method of v3 volumes_client might have returned a KeyError exception due to an incorrect accessing of a volume id in the case the volume was in error_deleting state. incorrect code - volume[‘id’] correct code - volume[‘volume’][‘id’] More details about the issue can be found at https://bugs.launchpad.net/tempest/+bug/1887980

• Concurrency parameter for account-generator command was accepting negative values and zero. The concurrency parameter now accepts only positive numbers. When a negative value or zero is passed to the program then the program ends and help is displayed.

Other Notes

• New configuration options CONF.glance.image_feature_enabled.image_import has been introduced to enable the image import tests. If your glance deployement support image import functionality then you can enable the image import tests via this flag. Default value of this new config option is false.

24.0.0

Prelude

Remove the support of python3.5. Tempest, its plugins dependencies in ussuri cycle are python-requires>=py3.6 which makes distro not having python 3.6 to do hack to install py3.6 etc. It time to drop the py3.5 from Tempest. Last supported version of Tempest for py3.5 is 23.0.0.

This release is to tag the Tempest for OpenStack Ussuri release. This release marks the start of Ussuri release support in Tempest. After this release, Tempest will support below OpenStack Releases: * Ussuri * Train * Stein Current development of Tempest is for OpenStack Victoria development cycle. Every Tempest commit is also tested against master during the Victoria cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ussuri (or future release) cloud. To be on safe side, use this tag to test the OpenStack Ussuri release.

New Features

• A new config option ‘subnet_id’ is added to section ‘network’ to specify subnet which should be used for allocation of IPs for VMs created during testing. It should be used when the tested network contains more than one subnet otherwise test of external connectivity will fail. (Fixes bug

• Add the option –worker-file in tempest run command. This is to give tempest more granularity to manually configure how the different sets of tests can be grouped to run with the different worker. You can configure tests regex to run under workers. You can also mix manual scheduling with standard one by mentioning concurrency. For example, the user can setup tempest to run with different concurrences, to be used with different regexps.

• A new attachments client library has been introduced for the volume service.

  Initially only the show_attachment API is provided. This API requires a minimum volume API microversion of 3.27.

Upgrade Notes

• Default value of config option CONF.placement.region is updated from RegionOne to empty string.

  As per tempest design, if tempest conf is not having any region for a service then identity region should be used. In case of placement the default value is “RegionOne” which is considered as placement region if region missing in tempest conf. In order to have identity region to be used as default we need to change to empty string for placement service. Empty string can be seen being used in other services like volume, image etc.

• Python 2.7 support has been dropped. Last release of Tempest to support python 2.7 is Temepst 23.0.0. The minimum version of Python now supported by Tempest is Python 3.6.

• Remove the deprecated argument os-tenant-name or OS_TENANT_NAME in favour of os-project-name argument.

• Remove the deprecated CLI verify-tempest-config in favour of tempest verify-config command. You can use tempest verify-config CLI to verify the tempest conf file.

Deprecation Notes

• The [compute-feature-enabled]/block_migrate_cinder_iscsi is deprecated ahead of removal in a future release. Once removed the [compute-feature-enabled]/block_migration_for_live_migration configurable will then be used to determine when to run block migration based tests during live migration.

• The config options CONF.compute.spice_console and CONF.compute.rdp_console are deprecated because test cases using them are removed. We can add them back when adding the test cases again.

Bug Fixes

• Fixed bug #1858417. Adding consistency group exceptions ConsistencyGroupException and ConsistencyGroupSnapshotException that didn’t exist before and caused failure in cinder-tempest-plugin.

23.0.0

Prelude

This is an intermediate release during the Ussuri development cycle to mark the end of support for EM Queens in Tempest. After this release, Tempest will support below OpenStack Releases:

• Train

• Stein

• Rocky

Current development of Tempest is for OpenStack Ussuri development cycle.

This is the last release of Tempest to officially support python2.7.

New Features

• tempest cleanup CLI is extended about region deletion. Until now, the regions have been neglected by tempest cleanup. From now on, tempest cleanup is able to delete leftover regions as well.

• Added new tempest subcommand for subunit-describe-calls

Deprecation Notes

• Deprecated command for subunit-describe-calls

22.1.0

Bug Fixes

• Bug#1847749. This provides the workaround of Skip Exception raised instead of skipping the CLI tests. If you are running Tempest with stestr > 2.5.0 then use this fix. Ref- https://github.com/testing-cabal/testtools/issues/272

22.0.0

Prelude

This release is to tag the Tempest for OpenStack Train release. This release marks the start of Train release support in Tempest. After this release, Tempest will support below OpenStack Releases:

• Train

• Stein

• Rocky

• Queens

Current development of Tempest is for OpenStack Ussuri development cycle. Every Tempest commit is also tested against master during the Ussuri cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ussuri (or future release) cloud. To be on safe side, use this tag to test the OpenStack Train release.

New Features

• Add show api version details function to v3 versions_client library for cinder.

  • show_version

• A new kwarg, log_req_body, was added to the tempest.lib.common.rest_client.RestClient method raw_request(). This kwarg takes in a string which will be used in place of the request body, which is logged by default. The intent of this option is to be used for security reasons to avoid logging sensitive information that are part of request bodies.

• Add segments client to Tempest to make possible the testing of the Routed Provider Networks feature. The following API calls are available for tempest from now:

  • POST /segments

  • PUT /segments/{segment_id}

  • GET /segment/{segment_id}

  • DELETE /segments/{segment_id}

  • GET /segments

Deprecation Notes

• The config option CONF.compute.vnc_server_header is deprecated because it has become obsolete with the usage of different response header fields to accomplish the same goal in accordance with RFC7231 Section 6.2.2.

Bug Fixes

• Adequately validates WebSocket upgrade in test_novnc and removes unneeded configuration complexity. Closes bug #1838777 and #1840788.

21.0.0

Prelude

This is an intermediate release during the Train development cycle to mark the end of support for Pike in Tempest. After this release, Tempest will support below OpenStack Releases:

• Stein

• Rocky

• Queens

Current development of Tempest is for OpenStack Train development cycle.

New Features

• Add qos-policies and qos-minimum-bandwidth-rule clients to Tempest to make possible the testing of the placement based bandwidth allocation feature. The following API calls are available for tempest from now:

  QoS policies client:

  • GET /qos/policies

  • POST /qos/policies

  • GET /qos/policies/{policy_id}

  • PUT /qos/policies/{policy_id}

  • DELETE /qos/policies/{policy_id}

  QoS minimum bandwidth rules client:

  • GET qos/policies/{policy_id}/minimum_bandwidth_rules

  • POST /qos/policies/{policy_id}/minimum_bandwidth_rules

  • GET qos/policies/{policy_id}/minimum_bandwidth_rules/{rule_id}

  • PUT qos/policies/{policy_id}/minimum_bandwidth_rules/{rule_id}

  • DELETE /qos/policies/{policy_id}/minimum_bandwidth_rules/{rule_id}

• Add list host API support to the volume v3 client library. This feature enables callers to list all hosts for a given project.

• Add migrate volume API support to the volume v3 client library. This features allows callers to migrate volumes between backends.

• Add show default group type details API to v3 group_types_client library.

  • show_default_group_type

• New decorator unstable_test is added to tempest.lib.decorators. It can be used to mark some test as unstable thus it will be still run by tempest but job will not fail if this test will fail. Such test will be skipped in case of failure. It can be used for example when there is known bug related which cause irregular tests failures. Marking such test as unstable will help other developers to get their job done and still run this test to get additional debug data or to confirm if some potential fix really solved the issue.

• A new parameter, compute/compute_volume_common_az is introduced to specify availability zone where tempest creates instances and volumes for scenario tests, to allow us to run scenario tests in the deployment which has multiple availability zones and cinder/cross_az_attach in nova.conf is set to False.

• New library interface to set the API microversion on Service Clients. APIMicroversionFixture, can be used to set the API microversion on multiple services. This Fixture will take care of reseting the service microversion to None once test is finished.

• Add microversion support for scenario tests. Scenario test calls multiple service API within same test and many services like compute, volume and placement etc support API microversion. With microversion support in scenario test, we can call different service API with different microvesion. Which means we can implement scenario tests for microversion also. Currently Scenario manager support below services microversion:

  • Compute

  • Volume

  • Placement

Upgrade Notes

• Remove deprecated config option endpoint_type from identity group. Use v2_public_endpoint_type from identity group instead. Remove deprecated config option tenant_isolation_domain_name from auth group. Use default_credentials_domain_name from auth group instead.

• Remove deprecated config option admin_username from identity groups. Use admin_username from auth instead. Remove deprecated config option admin_tenant_name from auth and identity groups. Use admin_project_name from auth instead. Remove deprecated config option admin_password from identity groups. Use admin_password from auth instead. Remove deprecated config option admin_domain_name from identity groups. Use admin_domain_name from auth instead.

Deprecation Notes

• The config option CONF.network.dns_servers is no longer used anywhere, so it is deprecated and will be removed in the future.

Other Notes

• New configuration options [compute]/certified_image_ref and [compute]/certified_image_trusted_certs have been introduced. These are required in order to run the ServerShowV263Test test and allow a signed image with the required img_signature_* properties set along with a list of trusted certificates to be used during the test.

20.0.0

Prelude

This release is to tag the Tempest for OpenStack Stein release. This release marks the start of Stein release support in Tempest and the end of support for Ocata in Tempest. After this release, Tempest will support below OpenStack Releases:

• Stein

• Rocky

• Queens

• Pike

Current development of Tempest is for OpenStack Train development cycle. Every Tempest commit is also tested against master during the Train cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Train (or future release) cloud. To be on safe side, use this tag to test the OpenStack Stein release.

New Features

• Add basic read-only Placement client to Tempest to make possible the testing of the placement based bandwidth allocation feature. The following API calls are available for tempest from now:

  • GET /allocation_candidates

  • GET /allocations/{consumer_uuid}

  Add new config group placement, with the config options:

  • endpoint_type to use for communication with placement service.

  • catalog_type of the placement service.

  • region as the placement region name to use.

  • min_microversion and max_microversion as the range between placement API requests are sent.

• Add a new config setting immutable_user_source in the [identity-feature-enabled] group that defaults to false. This setting, combined with the usage of the @testtools.skipIf() decorator, will allow tests that require user creation, deletion, or modification to skip instead of failing in environments that are LDAP-backed. In such environments, the user source is read-only, so this feature flag is needed to allow such tests to gracefully skip without having to blacklist them.

• Add support of OSProfiler library for profiling and distributed tracing of OpenStack. A new config option key in section profiler is added, the option sets the secret key used to enable profiling in OpenStack services. The value needs to correspond to the one specified in [profiler]/hmac_keys option of OpenStack services.

• A new parameter follow_redirects has been added to the class RestClient, which is passed through to ClosingHttp or ClosingProxyHttp respectively. The default value is True which corresponds to the previous behaviour of following up to five redirections before returning a response. Setting follow_redirects = False allows to disable this behaviour, so that any redirect that is received is directly returned to the caller. This allows tests to verify that an API is responding with a redirect.

• Adds the new method to AgentsClient that implements agent deletion according to the API [0]. [0] https://developer.openstack.org/api-ref/network/v2/index.html#delete-agent

• The tempest.lib.decorators.attr decorator now supports a condition kwarg which can be used to conditionally apply the attr to the test function if the condition evaluates to True.

• Add show_version function to the NetworkVersionsClient client. This allows the possibility of getting details for Networking API.

Upgrade Notes

• Changed the default value of ‘api_v1’ config option in the ‘image-feature-enabled’ group to False from True, because glance v1 APIs are deprecated. Please set True explicitly on the option if still testing glance v1 APIs.

• The tempest-full, tempest-full-py3 and tempest-slow zuul v3 job configurations now set ENABLE_VOLUME_MULTIATTACH: true in the devstack_localrc variables section. If you have a plugin job configuration that inherits from one of these jobs and the backend cinder volume driver or nova compute driver do not support volume multiattach then you should override and set this variable to ENABLE_VOLUME_MULTIATTACH: false in your job configuration.

• Below config option was deprecated for removal since juno release. It’s time to remove it as all supported stable branches and Tempest plugins are good to handle it.

  • [volume_feature_enabled].api_v1

  Also Tempest removes the below corresponding service clients alias from client.py which were being set based on above removed config option.

  • self.backups_client

  • self.encryption_types_client

  • self.snapshots_client

  • self.volume_availability_zone_client

  • self.volume_hosts_client

  • self.volume_limits_client

  • self.volume_qos_client

  • self.volume_quotas_client

  • self.volume_services_client

  • self.volume_types_client

  • self.volumes_client

  • self.volumes_extension_client

• tempest.lib.common.utils.misc.find_test_caller was deprecated during Kilo release cycle in favor of tempest.lib.common.utils.test_utils.find_test_caller. The deprecated version of find_test_caller is removed.

• CONF.validation.run_validation default enabled. This option required to be set true in order to run api tests stability when the guest cooperation required. For example when the guest needs react on Volume/Interface detach. The ssh test makes sure the VM is alive and ready when the detach needs to happen. The option was enabled on the gate for a long time.

Deprecation Notes

• The scheduler_available_filters option is being deprecated in favor of scheduler_enabled_filters. The new name is more indicative of what the option means. scheduler_enabled_filters’s default value is set to the default value of Nova’s enabled_filters. scheduler_available_filters’s default was all. There was confusion around this value. Sometimes it was understood to mean the default Nova filters are enabled, other times it was understood to mean all filters are enabled. While all is still allowed for scheduler_enabled_filters for backwards compatibility, it is strongly recommended to provide an explicit list of filters that matches what’s configured in nova.conf.

Bug Fixes

• [bug 1616892] Tempest now allows tests to verify that an API responds with a redirect.

• Fixed bug #1791007. tempest workspace register and tempest workspace rename CLI will error if None or empty string is passed in –name arguments. Earlier both CLI used to accept the None or empty string as name which was confusing.

• Fixed bug #1799883. tempest workspace register and tempest workspace move CLI will now validate the value of the --path CLI argument and raise an exception if it is None or empty string. Earlier both CLI actions were accepting None or empty path which was confusing.

• Fixed bug #1808473. tempest run CLI will error if a non-exist config file is input to parameter –config-file. Earlier non-exist config value was silently getting ignored and the default config file was used instead which used to give false behavior to the user on using the passed config file.

• Patch https://review.opendev.org/#/c/499575/ introduced support creating Neutron port with certain capabilities. Currently capabilities list interpreted as string this change fix it.

  tempest.conf [network] port_profile = capabilities:[switchdev]

  result: {‘capabilities’:’[switchdev]’}

  expected: {‘capabilities’: [‘switchdev’]}

19.0.0

Prelude

This release is to tag the Tempest for OpenStack Rocky release. After this release, Tempest will support below OpenStack Releases:

• Rocky

• Queens

• Pike

• Ocata

Current development of Tempest is for OpenStack Stein development cycle. Every Tempest commit is also tested against master during the Stein cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Stein (or future release) cloud. To be on safe side, use this tag to test the OpenStack Rocky release.

New Features

• Add v3-ext/OS-ENDPOINT-POLICY API calls to support creation, deletion and retrieval of associations between service endpoints and policies. Such associations enable an endpoint to request its policy.

• Add enable_service, disable_service , disable_log_reason, freeze_host and thaw_host API endpoints to volume v3 services_client.

• Add a new parameter called bug_type to tempest.lib.decorators.related_bug and tempest.lib.decorators.skip_because decorators, which accepts 2 values:

  • launchpad

  • storyboard

  This offers the possibility of tracking bugs related to tests using launchpad or storyboard references. The default value is launchpad for backward compatibility.

  Passing in a non-digit bug value to either decorator will raise a InvalidParam exception (previously ValueError).

• Add update flavor API to compute flavors_client library.

• [blueprint application-credentials] Tempest can test keystone’s application credentials interface. A new client library is added for application credentials, and a new config option, [identity-feature-enabled]/application_credentials, can control whether the application credentials feature is tested (defaults to False, indicating the feature is not enabled in the cloud under test).

• Add project_tags_client to the identity v3 library. This feature enables the possibility of invoking the following API actions:

  • update_project_tag

  • list_project_tags

  • update_all_project_tags

  • check_project_tag_existence

  • delete_project_tag

  • delete_all_project_tags

• The update_service API is added to the services_client compute library. This API is introduced in microversion 2.53 and supersedes the following APIs:

  • PUT /os-services/disable (disable_service)

  • PUT /os-services/disable-log-reason (disable_log_reason)

  • PUT /os-services/enable (enable_service)

  • PUT /os-services/force-down (update_forced_down)

• New string configuration option vnc_server_header is added to compute-feature-enabled section. It allows the expected VNC server name in the response header to be specified. For example, obvious at hand names are ‘WebSockify’, ‘nginx’.

• Adds volume service clients for v3 APIs. As v3 base API should be identical to v2 APIs, we just copy all existing v2 service client for v3 API.

Upgrade Notes

• Cinder CLI calls have now been updated to use the --os-endpoint-type option instead of --endpoint-type. The latter had been deprecated in Cinder and has been removed in the Rocky release.

• Remove deprecated config option allow_tenant_isolation from auth and compute groups. Use use_dynamic_credentials directly instead of the removed option.

• The volume config option catalog_type default is changed to volumev3 which is v3 API endpoint configured in devstack. With this change Tempest will be testing v3 API as default. User who want to test v2 API can still test by configuring the catalog_type to v2 endpoint.

Deprecation Notes

• The v2 volume API has been deprecated since Pike release. Volume v3 API is current and Tempest volume tests can be run against v2 or v3 API based on config option CONF.volume.catalog_type. If catalog_type is volumev2, then all the volume tests will run against v2 API. If catalog_type is volumev3 which is default in Tempest, then all the volume tests will run against v3 API. That makes below config options unusable in Tempest which used to select the target volume API for volume tests.

  • CONF.volume-feature-enabled.api_v2

  • CONF.volume-feature-enabled.api_v3

  Tempest deprecate the above two config options in Rocky release and will be removed in future. Alternatively CONF.volume.catalog_type can be used to run the Tempest against volume v2 or v3 API.

• Deprecates the volume service clients for v2 APIs. Volume v2 APIs are deprecated in all supported stable branches, so it’s time to deprecate the tempest service clients for v2 APIs and remove in future release.

Security Issues

• The x-subject-token of a response header is ommitted from log, but clients specify the same token on a request header on Keystone API and that was not omitted. In this release, that has been omitted for a security reason.

Bug Fixes

• Fix show_image_file interface in v2 ImagesClient: Bug#1756264. The expected success code of show_image_file is changed from 200 to [200, 204, 206].

• Fix VNC server response header issue when it is behind reverse proxy

18.0.0

Prelude

This release marks the start of Queens release support in Tempest. This release also marks the end of support for Newton in Tempest.

New Features

• Add group type specs APIs to v3 group_types_client library.

  • create_or_update_group_type_specs

  • list_group_type_specs

  • show_group_type_specs_item

  • update_group_type_specs_item

  • delete_group_type_specs_item

• A new config option ‘port_profile’ is added to the section ‘network’ to specify capabilities of the port. By default this is set to {}. When using OVS HW offload feature we need to create Neutron port with a certain capability. This is done by creating Neutron port with binding profile. To be able to test this we need profile capability support in Tempest as well.

• Add show default quotas API to network quotas_client library. This feature enables the possibility to show default network quotas for a specified project.

• Add show encryption specs item API to v2 encryption_types_client library. This feature enables the possibility to show specific encryption specs for a volume type.

• Add extension API show quota details to network quotas_client library. This feature enables the possibility to show a quota set for a specified project that includes the quota’s used, limit and reserved counts per resource.

• Add update group types API to v3 group_types_client library; min_microversion of this API is 3.11.

• A function has been added to the common library to allow mounting and unmounting of the config drive consistently.

• The Tempest CLI commands have switched from calling testrepository internally to use stestr instead. This means that all of the features and bug fixes from moving to stestr are available to the tempest commands.

• Adds a new CLI arg in tempest run, --black-regex, which is a regex to exclude the tests that match it.

• A new boolean configuration option [compute-feature-enabled]/volume_backed_live_migration has been added. If enabled, tests which validate the behavior of Nova’s volume-backed live migration feature will be executed. The option defaults to False.

Upgrade Notes

• Below config options or feature flags were deprecated for removal. It’s time to remove them as all supported stable branches are good to handle them.

  • [identity-feature-enabled].forbid_global_implied_dsr

  • [image-feature-enabled].deactivate_image

  • [default].resources_prefix

  • config group orchestration

  • [service_available].heat

• Remove the deprecated decorator skip_unless_attr in lib/decorators.py.

• The tox ostestr job (normally invoked with tox -eostestr) has been removed. This was lightly used, and in the near future ostestr will be removed from the tempest requirements file. If you were relying on this functionality you can replicate it by using the venv-tempest tox job. For example, simply running tox -evenv-tempest -- ostestr will do the same thing the old ostestr job did.

• Tempest CLI commands will no long rely on anything from testr. This means any data in existing testr internals that were being exposed are no longer present. For example things like the .testr directories will be silently ignored. There is a potential incompatibility for existing users who are relying on test results being stored by testr. Anything relying on previous testr behavior will need to be updated to handle stestr.

Bug Fixes

• Fixes tempest run CLI args mutually exclusive behavior which should not be the case anymore (Bug#1751201).

Other Notes

• The CLIClient class, when it calls a command line client, uses --os-project-name instead of --os-tenant-name for the project, and passes --os-identity-api-version (default empty). All CLI clients still available in supported releases of OpenStack which are wrapped by the cmd_with_auth() method support those switches.

• A new configuration option [compute-feature-enabled]/volume_multiattach has been added which defaults to False. Set this to True to enable volume multiattach testing. These tests require that compute API version 2.60 is available and block storage API version 3.44 is available.

  Note

  In the Queens release, the only compute driver that supports volume multiattach is the libvirt driver, and only then when qemu<2.10 or libvirt>=3.10. The only volume backend in Queens that supports volume multiattach is lvm.

• OpenStack Releases supported after this release are Queens, Pike, and Ocata. The release under current development of this tag is Rocky, meaning that every Tempest commit is also tested against master during the Rocky cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Rocky (or future release) cloud.

17.2.0

New Features

• Add support of args and kwargs when calling func in call_until_true, also to log the cost time when call_until_true returns True or False for debugging.

• Add list_auth_domains API endpoint to the identity v3 client. This allows the possibility of listing all domains a user has access to via role assignments.

Upgrade Notes

• Replace any call in your code to credentials_factory.DEFAULT_PARAMS with a call to config.service_client_config().

• Remove deprecated APIs (show_pools and show_backend_capabilities) from volume v2 volumes_client, and the deprecated APIs are re-realized in volume v2 scheduler_stats_client (list_pools) and capabilities_client (show_backend_capabilities) accordingly.

Bug Fixes

• The credentials_factory module used to load configuration at import time which caused configuration being loaded at test discovery time. This was fixed by removing the DEFAULT_PARAMS variable. This variable was redundant (and outdated), the same dictionary (but up to date) can be obtained via invoking config.service_client_config() with no service parameter.

17.1.0

Prelude

This is an intermediate release during the Queens development cycle to make new functionality available to plugins and other consumers.

New Features

• Adds a new cli option to tempest run, --load-list <list-file> to specify target tests to run from a list-file. The list-file supports the output format of the tempest run --list-tests command.

• Add reset group snapshot status API to v3 group_snapshots_client library, min_microversion of this API is 3.19. This feature enables the possibility to reset group snapshot status.

• Add reset group status API to v3 groups_client library, min_microversion of this API is 3.20. This feature enables the possibility to reset group status.

• Add the validation_resources module to tempest.lib. The module provides a set of helpers that can be used to provision and cleanup all the resources required to perform ping / ssh tests against a virtual machine: a keypair, a security group with targeted rules and a floating IP.

• Add the proxy_url optional parameter to the get_credentials method in tempest/lib/auth.py so that that helper can be used when going through and HTTP proxy.

• Add a new function called compare_version_header_to_response to tempest.lib.common.api_version_utils, which compares the API microversion in the response header to another microversion using the comparators defined in tempest.lib.common.api_version_request.APIVersionRequest.

  It is now possible to determine how to retrieve an attribute from a response body of an API call, depending on the returned microversion.

  Add a new exception type called InvalidParam to tempest.lib.exceptions, allowing the possibility of raising an exception if an invalid parameter is passed to a library function.

• Adds a new config options, proxy_url. This options is used to configure running tempest through a proxy server.

• The RestClient class in tempest.lib.rest_client has a new kwarg parameters, proxy_url, that is used to set a proxy server.

• A new class was added to tempest.lib.http, ClosingProxyHttp. This behaves identically to ClosingHttp except that it requires a proxy url and will establish a connection through a proxy

• A new boolean config option domain_specific_drivers is added to the section identity-feature-enabled. This option must be enabled when testing an environment that is configured to use domain-specific identity drivers.

• Define below object storage service clients as libraries. Add new service clients to the library interface so the other projects can use these modules as stable libraries without any maintenance changes.

  • account_client

  • container_client

  • object_client

• Two extra modules are now marked as stable for plugins, test.py and clients.py. The former includes the test base class with its automatic credentials provisioning and test resource managing fixtures. The latter is built on top of ServiceClients and it adds aliases and a few custom configurations to it.

Upgrade Notes

• As of the Queens release, tempest no longer tests the identity v2.0 API because the majority of the v2.0 API have been removed from the identity project. Once the Queens release reaches end-of-life, we can remove the v2.0 tempest tests and clean up v2.0 testing cruft.

• Remove deprecated APIs from volume v2 volumes_client, and the deprecated APIs are re-realized in volume v2 transfers_client.

  • create_volume_transfer

  • show_volume_transfer

  • list_volume_transfers

  • delete_volume_transfer

  • accept_volume_transfer

• Remove two deprecated skip decorators in config module: skip_unless_config and skip_if_config.

• Remove deprecated get_ipv6_addr_by_EUI64 method from data_utils. Use the same method from oslo_utils.netutils.

Bug Fixes

• Allow to specify new domain parameters:

  • user_domain_name

  • user_domain_id

  • project_domain_name

  • project_domain_id

  for CLIClient class, whose values will be substituted to --os-user-domain-name, --os-user-domain-id, --os-project-domain-name and --os-project-domain-id respectively during command execution.

  This allows to prevent possible test failures with authentication in Keystone v3. Bug: #1719687

• Fix list_group_snapshots API in v3 group_snapshots_client: Bug#1715786. The url path for list group snapshots with details API is changed from ?detail=True to /detail.

17.0.0

Prelude

This release marks the start of support for the Pike release in Tempest.

New Features

• Add create_group_from_source to groups_client in the volume service library.

• Implement the rest_client method is_resource_deleted in the network security group client.

• The list_backups method of the v2 BackupsClient class now has an additional **params argument that enables passing additional information in the query string of the HTTP request.

• Add show volume image metadata API to v2 volumes_client library. This feature enables the possibility to show volume’s image metadata.

• Add update_group to groups_client in the volume service library.

• Add group_snapshots client for the volume service as library. Add tempest tests for create group snapshot, delete group snapshot, show group snapshot, and list group snapshots volume APIs.

• Add list_group_type and show_group_type in the group_types client for the volume service. Add tests for create/delete/show/list group types.

• The credentials_factory.py module is now marked as stable for Tempest plugins. It provides helpers that can be used by Tempest plugins to obtain test credentials for their test cases in a format that honors the Tempest configuration in use. Credentials may be provisioned on the fly during the test run, or they can be setup in advance and fed to test via a YAML file; they can be setup for identity v2 or identity v3.

• Add the disable_log_reason and the update_forced_down API endpoints to the compute services_client. Add ‘2.11’ compute validation schema for compute services API.

• Enhances the v3 identity client with the check_token_existence endpoint, allowing users to check the existence of tokens

• The tempest module tempest.common.dynamic creds which is used for dynamically allocating credentials has been migrated into tempest lib.

• Define below object storage service clients as libraries. Add new service clients to the library interface so the other projects can use these modules as stable libraries without any maintenance changes.

  • bulk_middleware_client

  • capabilities_client

• The tempest module tempest.common.preprov_creds which is used to provide credentials from a list of preprovisioned resources has been migrated into tempest lib at tempest.lib.common.preprov_creds.

• The InvalidTestResource exception class from tempest.exceptions has been migrated into tempest.lib.exceptions

• The tempest module tempest.common.fixed_network which provided utilities for finding fixed networks by and helpers for picking the network to use when multiple tenant networks are available has been migrated into tempest lib at tempest.lib.common.fixed_network.

• When registering service clients from installed plugins, all registrations are now processed, even if one or more fails. All exceptions encountered during the registration process are recorded. If at least one exception was encountered, the registration process fails and all interim errors are reported.

• The __repr__ method is now implemented for the base tempest.Exception class, its implementation is identical to __str__: it reports the error message merged with input parameters.

• Add a new identity catalog client. At this point, the new client contains a single functionality, “show_catalog”, which returns a catalog object.

Upgrade Notes

• Tempest checks a volume delete by waiting for NotFound(404) on show_volume(). Sometime a volume delete fails and the volume status becomes error_deleting which means the delete is failed. So Tempest doesn’t need to wait anymore. A new release of Tempest raises an exception DeleteErrorException instead of waiting.

Deprecation Notes

• Remove the support of python3.4, because in Ubuntu Xenial only python3.5 is available (python3.4 is restricted to <= Mitaka).

Bug Fixes

• Add more accurate ip version check in addresses schema which will limit the ip version value in [4, 6].

• Add a missing return statement to the retype_volume API in the v2 volumes_client library: Bug#1703997

  This changes the response body from None to an empty dictionary.

• Fix RemoteClient default ssh_shell_prologue: Bug#1707478

  The default ssh_shell_proloque has been modified from specifying erroneous PATH=$$PATH:/sbin to PATH=$PATH:/sbin.

Other Notes

• OpenStack Releases supported after this release are Pike, Ocata, and Newton. The release under current development of this tag is Queens, meaning that every Tempest commit is also tested against master during the Queens cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Queens (or future release) cloud.

16.1.0

Prelude

This is an intermediate release during the Pike development cycle to make new functionality available to plugins and other consumers.

New Features

• Add a new client to handle the OAUTH token feature from the identity API.

• A new boolean config option serial_console is added to the section compute-feature-enabled. If enabled, tests, which validate the behavior of Nova’s serial console feature (an alternative to VNC, RDP, SPICE) can be executed.

• Add a new client to handle the domain configuration feature from the identity v3 API.

• A new config option in the network-feature-enabled section, floating_ips, to specify whether floating ips are available in the cloud under test. By default this is set to True.

• Add force detach volume feature API to v2 volumes_client library. This feature enables the possibility to force a volume to detach, and roll back an unsuccessful detach operation after you disconnect the volume.

• Defines the identity v3 OS-EP-FILTER EndPoint Groups API client. This client manages Create, Get, Update, Check, List, and Delete of EndPoint Group.

• Defines the identity v3 OS-EP-FILTER extension API client. This client manages associations between endpoints, projects along with groups.

• The delete_volume method of the VolumesClient class now has an additional **params argument that enables passing additional information in the query string of the HTTP request.

• Add list volume transfers with details API to v2 transfers_client library. This feature enables the possibility to list volume transfers with details.

• A new config option ‘manage_snapshot_ref’ is added in the volume section, to specify snapshot ref parameter for different storage backend drivers when managing an existing snapshot. By default it is set to fit the LVM driver.

• The list_endpoints method of the v3 EndPointsClient class now has an additional **params argument that enables passing additional information in the query string of the HTTP request.

• Add --save-state option to allow saving state of cloud before tempest run.

• Add validation schema for Nova server diagnostics API

• Add show host API to the volume v2 hosts_client library. This feature enables the possibility to show details for a host.

• Add show snapshot metadata item API to v2 snapshots_client library. This feature enables the possibility to show a snapshot’s metadata for a specific key.

• Add show volume metadata item API to v2 volumes_client library. This feature enables the possibility to show a volume’s metadata for a specific key.

• Define v3 volumes_client for the volume service as a library interface, allowing other projects to use this module as a stable library without maintenance changes. Add show volume summary API to v3 volumes_client library, min_microversion of this API is 3.12.

  • volumes_client(v3)

• Define v3 backups_client for the volume service as a library interface, allowing other projects to use this module as a stable library without maintenance changes. Add update backup API to v3 backups_client library, min_microversion of this API is 3.9.

  • backups_client(v3)

• Add groups and group_types clients for the volume service as library. Add tempest tests for create group, delete group, show group, and list group volume APIs.

• Define v2 quota_classes_client for the volume service as library interfaces, allowing other projects to use this module as stable libraries without maintenance changes.

  • quota_classes_client(v2)

• A new configuration flag api_v2_admin is introduced in the identity feature flag group to allow for enabling/disabling all identity v2 admin tests. The new flag only applies when the existing api_v2 flag is set to True

• Add additional API endpoints to the identity v2 client token API: - list_endpoints_for_token - check_token_existence

• A new attr decorator has been added in the tempest.lib.decorators module. For example, use it to tag specific tests, which could be leveraged by test runners to run only a subset of Tempest tests.

• A new related_bug decorator has been added to tempest.lib.decorators. Use it to decorate and tag a test that was added in relation to a launchpad bug report.

• Move base_client from tempest.lib.services.volume.v3 to tempest.lib.services.volume, so if we want to add new interfaces based on a v2 client, we can make that v2 client inherit from volume.base_client.BaseClient to get microversion support, and then to make the new v3 client inherit from the v2 client, thus to avoid the multiple inheritance.

• Define v2.0 tags_client for the network service as a library interface, allowing other projects to use this module as a stable library without maintenance changes.

  • tags_client(v2.0)

• Pause teardown When pause_teardown flag in tempest.conf is set to True a pdb breakpoint is added to tearDown and tearDownClass methods in test.py. This allows to pause cleaning resources process, so that used resources can be examined. Closer examination of used resources may lead to faster debugging.

• Added tempest workspace remove --name <workspace_name> --rmdir feature to delete the workspace directory as well as entry.

Upgrade Notes

• Switch to use Glance v2 APIs in volume tests, by adding the Glance v2 client images_client.

• The Heat API tests have been removed from tempest, they were unmaintained. The future direction of api test for heat is their in-tree Gabbi tests

• The volume config option ‘api_v3’ default is changed to True because the volume v3 API is CURRENT.

• verify_tempest_config command starts using extension_client of cinder v2 API only, because cinder v3 API is current and v2 and v1 are deprecated and v3 extension API is the same as v2. Then we can reuse the v2 client for v3 API also.

Deprecation Notes

• Image APIs in compute are deprecated, Image native APIs are recommended. And Glance v1 APIs are deprecated and v2 APIs are current. Image client compute_images_client and Glance v1 APIs are removed in volume tests.

• The config option forbid_global_implied_dsr from the IdentityFeature group is now deprecated. This feature flag was introduced to support testing of old OpenStack versions which are not supported anymore.

• Deprecate default value for configuration parameter v3_endpoint_type of identity section in OpenStack Pike and modify the default value to publicURL in OpenStack Q release.

• Deprecate class BaseClient from volume.v3.base_client and move it to volume.base_client. tempest.lib.services.volume.v3.base_client.BaseClient (new tempest.lib.services.volume.base_client.BaseClient)

Bug Fixes

• When receiving nullable list as a response body, tempest.lib rest_client module raised an exception without valid json deserialization. A new release fixes this bug.

16.0.0

Prelude

This release indicates end of support for Mitaka in Tempest.

New Features

• Add a new client to handle the OAUTH consumers feature from the identity API.

• Add missing API call, list all role inference rules, to the roles_client library. This feature enables the possibility of listing all role inference rules in the system.

• Add cascade parameter to volumes_client. This option provides the ability to delete a volume and have Cinder handle deletion of snapshots associated with that volume by passing an additional argument to volume delete, “cascade=True”.

• Define the compute server evacuate client method in the servers_client library.

• Add the list auth projects API to the identity client library. This feature enables the possibility to list projects that are available to be scoped to based on the X-Auth-Token provided in the request.

• Add versions_client module for image service. This new module provides list_versions() method which shows API versions from Image service.

• Add the list security groups by server API to the servers_client library. This feature enables the possibility to list security groups for a server instance.

• Add versions_client module for identity service. This new module provides list_versions() method which shows API versions from Identity service.

• Add versions_client module for volume service. This new module provides list_versions() method which shows API versions from Volume service.

• Interface show_quota_set of compute quotas_client has been extended to include the argument “detail”, which allows for detailed quota set information for a project to be retrieved, if set to True.

• Add remote_client under tempest.lib. This remote_client under tempest.lib is defined as stable interface, and now this module provides the following essential methods.

  • exec_command

  • validate_authentication

  • ping_host

• Adds a new cli option to tempest run, --combine, which is used to indicate you want the subunit stream output combined with the previous run’s in the testr repository

• Add update encryption type API to the v2 encryption_types_client library. This feature enables the possibility to update an encryption type for an existing volume type.

• Add the unmanage volume API service method in v2 volumes_client library. Define v2 volume_manage_client client for the volume service as library interfaces, allowing other projects to use this module as stable libraries without maintenance changes.

  • volume_manage_client(v2)

• Add server tags APIs to the servers_client library. This feature enables the possibility of updating, deleting and checking existence of a tag on a server, as well as updating and deleting all tags on a server.

• Define volume transfers service clients as libraries. The following volume transfers service clients are defined as library interface.

  • transfers_client(v2)

Upgrade Notes

• The JSON_ENC and TXT_ENC option in the _error_checker section have been added with additional content-type which are defined in RFC7231 but missing in the current rest_client.py file. The lack of these additional content-type will cause defcore test to fail for OpenStack public cloud which uses tomcat module in the api gateway. The additions are application/json;charset=utf-8, text/html;charset=utf-8, text/plain;charset=utf-8

• The default value of rand_name()’s prefix argument is changed to ‘tempest’ from None to identify resources are created by Tempest.

• The volume config option ‘api_v1’ default is changed to False because the volume v1 API has been deprecated since Juno release.

• The call_until_true of test module is removed because it was marked as deprecated and Tempest provides it from test_utils as a stable interface instead. Please switch to use test_utils.call_until_true if necessary.

• Remove Cinder v1 API tests. Cinder v1 API has been deprecated since Juno release, and Juno is not supported by current Tempest. Then Cinder v1 API tests are removed from Tempest.

• The deprecated config option allow_port_security_disabled from compute_feature_enabled group has been removed.

• Below deprecated config options from compute group have been removed. Corresponding config options already been available in validation group.

  • compute.image_ssh_user (available as validation.image_ssh_user)

  • compute.ssh_user (available as validation.image_ssh_user)

  • scenario.ssh_user (available as validation.image_ssh_user)

  • compute.network_for_ssh (available as validation.network_for_ssh)

  • compute.ping_timeout (available as validation.ping_timeout)

• The deprecated config option ‘dvr_extra_resources’ from network group has been removed. This option was for extra resources which were provisioned to bind a router to Neutron L3 agent. The extra resources need to be provisioned in Liberty release or older, and are not required since Mitaka release. Current Tempest doesn’t support Liberty, so this option has been removed from Tempest.

• The deprecated config option ‘reseller’ from identity_feature_enabled group has been removed.

• The deprecated volume_services option in the volume_feature_enabled section has now been removed.

• Tempest now defaults to using Keystone v3 API for the authentication, because Keystone v3 API is CURRENT and the v2 API is deprecated.

Deprecation Notes

• The deactivate_image configuration switch from the config module is deprecated. It was added to support the older-than-kilo releases which we don’t support anymore.

• The dvr_extra_resources configuration switch from the config module is deprecated. It was added to support the Liberty Release which we don’t support anymore.

• Glance v1 APIs are deprecated and v2 are current. Tempest should tests only v2 APIs. Below API version selection config options for glance have been deprecated and will be removed in future.

  • CONF.image_feature_enabled.api_v2

  • CONF.image_feature_enabled.api_v1

• The resources_prefix is marked as deprecated because it is enough to set ‘tempest’ as the prefix on rand_name() to identify resources which are created by Tempest and no projects set this option on OpenStack dev community.

• The skip_unless_attr decorator in lib/decorators.py has been deprecated, please use the standard testtools.skipUnless and testtools.skipIf decorators.

• The skip_unless_config and skip_if_config decorators in the config module have been deprecated and will be removed in the Queens dev cycle. Use the testtools.skipUnless (or a variation of) instead.

• Volume v1 API is deprecated and the v3 are CURRENT. Tempest doesn’t need to test the v1 API as the default. The volume config option ‘api_v1’ has been marked as deprecated.

• Deprecate the client_parameters argument in tempest.lib.services.clients.ServiceClients. The parameter is actually not honoured already - see https://bugs.launchpad.net/tempest/+bug/1680915

• Deprecate volume v2 transfers resource methods from volumes_client(v2) libraries. Same methods are available in new transfers service client: transfers_client(v2) The following methods of volume v2 volumes_clients have been deprecated:

  • create_volume_transfer (v2.volumes_client)

  • show_volume_transfer (v2.volumes_client)

  • list_volume_transfers (v2.volumes_client)

  • delete_volume_transfer (v2.volumes_client)

  • accept_volume_transfer (v2.volumes_client)

Bug Fixes

• Fix below volume v2 service clients to make v2 API call: Bug#1667354

  • SchedulerStatsClient

  • CapabilitiesClient

• The ‘sahara’ config option in the ‘service-available’ group has been moved to the sahara plugin (openstack/sahara-tests) along with tests and service client during the Ocata timeframe. A ‘sahara’ config option was left over on Tempest side, and it’s removed now. As long as the sahara plugin is installed, this change as no impact on users of sahara tests.

Other Notes

• OpenStack Releases Supported after this release are Newton and Ocata

  The release under current development as of this tag is Pike, meaning that every Tempest commit is also tested against master branch during the Pike cycle. However, this does not necessarily mean that using Tempest as of this tag will work against Pike (or future releases) cloud.

15.0.0

Prelude

This release is marking the start of Ocata release support in Tempest

New Features

• As in the [doc]: http://developer.openstack.org/api-ref/image/v2/metadefs-index.html, there are some apis are not included, add them.

  • namespace_objects_client(v2)

  • namespace_tags_client(v2)

• Add the implied roles feature API to the roles_client library. This feature enables the possibility to create inferences rules between roles (a role being implied by another role).

• Define v2 snapshot_manage_client client for the volume service as library interfaces, allowing other projects to use this module as stable libraries without maintenance changes.

  • snapshot_manage_client(v2)

• Added customized JSON schema format checker for ‘date-time’ format. Compute response schema will be validated against customized format checker.

Upgrade Notes

• The default value for the allow_port_security_disabled option in the compute-feature-enabled section has been changed from False to True.

• The default value for the reseller option in the identity-feature-enabled section has been changed from False to True.

• The default value for the volume_services option in the volume_feature_enabled section has been changed from False to True.

• The deprecated compute microversion config options from ‘compute-feature-enabled’ group have been removed. Those config options are available under ‘compute’ group to configure the min and max microversion for compute service. * CONF.compute.min_microversion * CONF.compute.max_microversion

• Below deprecated config options from compute group have been removed. Corresponding config options already been available in validation group.

  • compute.use_floatingip_for_ssh (available as validation.connect_method)

  • compute.ssh_auth_method (available as validation.auth_method)

  • compute.image_ssh_password (available as validation.image_ssh_password)

  • compute.ssh_shell_prologue (available as validation.ssh_shell_prologue)

  • compute.ping_size `` (available as ``validation.ping_size)

  • compute.ping_count `` (available as ``validation.ping_count)

  • compute.floating_ip_range `` (available as ``validation.floating_ip_range)

• The deprecated input-scenario config options and group have been removed. The input scenarios functionality already being removed from tempest and from this release, their corresponding config options too.

• Below deprecated network config options have been removed. Those config options already been renamed to below meaningful names.

  • tenant_network_cidr (removed) -> project_network_cidr

  • tenant_network_mask_bits (removed) -> project_network_mask_bits

  • tenant_network_v6_cidr (removed) -> project_network_v6_cidr

  • tenant_network_v6_mask_bits (removed) -> project_network_v6_mask_bits

  • tenant_networks_reachable (removed) -> project_networks_reachable

Deprecation Notes

• The allow_port_security_disabled option in the compute-feature-enabled section is now deprecated.

• The reseller option in the identity-feature-enabled section is now deprecated.

• The volume_services option in the volume_feature_enabled section is now deprecated.

Other Notes

• OpenStack releases supported at this time are Mitaka, Newton, and Ocata.

  The release under current development as of this tag is Pike, meaning that every Tempest commit is also tested against master during the Pike cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Pike (or future releases) cloud.

14.0.0

Prelude

This release is marking the end of Liberty release support in Tempest

New Features

• The cred_provider abstract class which serves as the basis for both of tempest’s cred providers, pre-provisioned credentials and dynamic credentials, is now a library interface. This provides the common signature required for building a credential provider.

• The cred_client module was added to tempest.lib. This module provides a wrapper to the keystone services client which provides a uniform interface that abstracts out the differences between keystone api versions.

• As in the [doc]: http://developer.openstack.org/api-ref/image/v2/metadefs-index.html, there are some apis are not included, add them.

  • namespace_properties_client(v2)

• Define the identity service role_assignments_client as a library. Add role_assignments_client to the library interface so the other projects can use this module as a stable library without any maintenance changes.

• A Neutron Service Providers client is added to deal with resources of the ‘/service-providers’ route.

• A new optional parameter port for ssh client (tempest.lib.common.ssh.Client) to specify destination port for a host. The default value is 22.

• Define the Volume v3 service clients as library interfaces, allowing other projects to use these modules as stable libraries without maintenance changes.

  • messages_client(v3)

• The volume_limits client was added to tempest.lib.

• Define volume service clients as libraries. The following volume service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.

  • volumes_client(v1)

  • volumes_client(v2)

  • capabilities_client(v2)

  • scheduler_stats_client(v2)

Upgrade Notes

• Add an error translation to list_versions() of versions_client of both compute and network. This can affect users who are expecting that these clients return error status code instead of the exception. It is needed to change the code for handling the exception like the other clients code.

• The Stress tests framework and all the stress tests have been removed.

• All tests for the Ironic project have been removed from Tempest. Those exist as a Tempest plugin in the Ironic project.

• The Negative Tests Generator has been removed (it was not used by any Tempest tests).

• All tests for the Sahara project have been removed from Tempest. They now live as a Tempest plugin in the openstack/sahara-tests repository.

Deprecation Notes

• The api_extensions config option in the compute-feature-enabled group is now deprecated. This option will be removed from tempest when all the OpenStack releases supported by tempest no longer support the API extensions mechanism. This was removed from Nova during the Newton cycle, so this will be removed at the Mitaka EOL.

• The bootable config option in the volume_feature_enabled group is removed because the corresponding feature os-set_bootable has been implemented 2.5 years ago and all OpenStack versions which are supported by Tempest should support the feature.

Other Notes

• OpenStack releases supported at this time are Mitaka and Newton.

  The release under current development as of this tag is Ocata, meaning that every Tempest commit is also tested against master during the Ocata cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ocata (or future releases) cloud.

13.0.0

Prelude

This release is marking the start of Newton release support in Tempest

New Features

• Define identity service clients as libraries. Add new service clients to the library interface so the other projects can use these modules as stable libraries without any maintenance changes.

  • identity_client(v2)

  • groups_client(v3)

  • trusts_client(v3)

  • users_client(v3)

  • identity_client(v3)

  • roles_client(v3)

  • inherited_roles_client(v3)

  • credentials_client(v3)

• Define volume service clients as libraries. The following volume service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.

  • backups_client

  • encryption_types_client (v1)

  • encryption_types_client (v2)

  • qos_clients (v1)

  • qos_clients (v2)

  • snapshots_client (v1)

  • snapshots_client (v2)

• Define the v1 and v2 types_client clients for the volume service as library interfaces, allowing other projects to use these modules as stable libraries without maintenance changes.

Upgrade Notes

• the already deprecated tempest-cleanup standalone command has been removed. The corresponding functionalities can be accessed through the unified tempest command (tempest cleanup).

Deprecation Notes

• Oslo.utils provides same method get_ipv6_addr_by_EUI64, so deprecate it in Newton and remove it in Ocata.

• The call_until_true function is moved from the tempest.test module to the tempest.lib.common.utils.test_utils module. Backward compatibility is preserved until Ocata.

Other Notes

• OpenStack releases supported at this time are Liberty, Mitaka, and Newton.

  The release under current development as of this tag is Ocata, meaning that every Tempest commit is also tested against master during the Ocata cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ocata (or future releases) cloud.

12.2.0

New Features

• RestClient now supports setting timeout in urllib3.poolmanager. Clients will use CONF.service_clients.http_timeout for timeout value to wait for http request to response.

• KeystoneAuthProvider will accept http_timeout and will use it in get_credentials.

• Define identity service clients as libraries. The following identity service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.

  • endpoints_client(v3)

  • policies_client (v3)

  • regions_client(v3)

  • services_client(v3)

  • projects_client(v3)

• The Tempest plugin interface contains a new optional method, which allows plugins to declare and automatically register any service client defined in the plugin.

• tempest.lib exposes a new stable interface, the clients module and ServiceClients class, which provides a convenient way for plugin tests to access service clients defined in Tempest as well as service clients defined in all loaded plugins. The new ServiceClients class only exposes for now the service clients which are in tempest.lib, i.e. compute, network and image. The remaining service clients (identity, volume and object-storage) will be added in future updates.

• A new optional interface TempestPlugin.get_service_clients is available to plugins. It allows them to declare any service client they implement. For now this is used by tempest only, for auto-registration of service clients in the new class ServiceClients.

• A new singleton class clients.ClientsRegistry is available. It holds the service clients registration data from all plugins. It is used by ServiceClients for auto-registration of the service clients implemented in plugins.

• A new helper method service_client_config has been added to the stable module config.py that returns extracts from configuration into a dictionary the configuration settings relevant for the initialization of a service client.

• Define volume service clients as libraries The following volume service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.

  • availability_zone_client(v1)

  • availability_zone_client(v2)

  • extensions_client(v1)

  • extensions_client(v2)

  • hosts_client(v1)

  • hosts_client(v2)

  • quotas_client(v1)

  • quotas_client(v2)

  • services_client(v1)

  • services_client(v2)

Upgrade Notes

• The nova_cert option default is changed to False. The nova certification management APIs were a hold over from ec2, and are not used by any other parts of nova. They are deprecated for removal in nova after the newton release. This makes false a more sensible default going forward.

• The previously deprecated Javelin utility has been removed from Tempest. As an alternative Ansible can be used to construct similar yaml workflows to what Javelin used to provide.

Deprecation Notes

• The new clients module provides a stable alternative to tempest classes manager.Manager and clients.Manager. manager.Manager only exists now to smoothen the transition of plugins to the new interface, but it will be removed shortly without further notice.

12.1.0

New Features

• Adds a network version client for querying Neutron’s API version discovery URL (“GET /”).

• Tempest library auth interface now supports scope. Scope allows to control the scope of tokens requested via the identity API. Identity V2 supports unscoped and project scoped tokens, but only the latter are implemented. Identity V3 supports unscoped, project and domain scoped token, all three are available.

• Adds the tempest run command to the unified tempest CLI. This new command is used for running tempest tests.

• Adds tempest workspaces command and WorkspaceManager. This is used to have a centralized repository for managing different tempest configurations.

• Adds subunit-describe-calls. A parser for subunit streams to determine what REST API calls are made inside of a test and in what order they are called.

  • Input can be piped in or a file can be specified

  • Output is shortened for stdout, the output file has more information

• Tempest library auth interface now supports filtering with catalog name. Note that filtering by name is only successful if a known service type is provided.

• Define identity service clients as libraries The following identity service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.

  • endpoints_client(v2)

  • roles_client(v2)

  • services_client(v2)

  • tenants_client(v2)

  • users_client(v2)

• Define image service clients as libraries The following image service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.

  • image_members_client(v1)

  • images_client(v1)

  • image_members_client(v2)

  • images_client(v2)

  • namespaces_client(v2)

  • resource_types_client(v2)

  • schemas_client(v2)

• A new test_utils module has been added to tempest.lib.common.utils. It should hold any common utility functions that help writing Tempest tests.

• A new utility function called call_and_ignore_notfound_exc has been added to the test_utils module. That function call another function passed as parameter and ignore the NotFound exception if it raised.

• Define routers_client as stable library interface. The routers_client module is defined as library interface, so the other projects can use the module as stable library without any maintenance changes.

• The RestClient (in tempest.lib.common.rest_client) now supports POSTing and PUTing data with chunked transfer encoding. Just pass an iterable object as the body argument and set the chunked argument to True.

• A new generator called chunkify is added in tempest.lib.common.utils.data_utils that yields fixed-size chunks (slices) from a Python sequence.

Upgrade Notes

• The input scenarios functionality no longer exists in tempest. This caused a large number of issues for limited benefit and was only used by a single test, test_server_basic_ops. If you were using this functionality you’ll now have to do it manually with a script and/or tempest workspaces

• The integrated dashboard scenario test has been removed and is now in a separate tempest plugin tempest-horizon. The removed test coverage can be used by installing tempest-horizon on the server where you run tempest.

• The deprecated legacy credential provider has been removed. The only way to configure credentials in tempest now is to use the dynamic or preprovisioned credential providers

• All tests for the Trove project have been removed from tempest. They now live as a tempest plugin in the trove project.

• The location on disk that the tempest init command looks for has changed. Previously it would attempt to use python packaging’s data files to guess where setuptools/distutils were installing data files, which was incredibly unreliable and depended on how you installed tempest and which versions of setuptools, distutils, and python you had installed. Instead, now it will use either /etc/tempest, $XDG_CONFIG_PATH/.config/tempest, or ~/.tempest/etc (attempted in that order). If none of these exist it will create an empty ~/.tempest/etc directory. If you were relying on the previous behavior and none of these directories were being used you will need to move the files to live in one of these directories.

Deprecation Notes

• tempest.lib.common.utils.misc.find_test_caller has been moved into the tempest.lib.common.utils.test_utils module. Calling the find_test_caller function with its old location is deprecated.

• All the options in the input-scenario group are now deprecated. These were only used in tree by the now removed input scenarios functionality in test_server_basic_ops. They were only deprecated because there could be external consumers via plugins. They will be removed during the Ocata cycle.

12.0.0

Prelude

This release is marking the end of Kilo release support in Tempest

Other Notes

• OpenStack Releases Supported after this release are Liberty and Mitaka The release under current development as of this tag is Newton, meaning that every Tempest commit is also tested against master during the Newton cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Newton (or future releases) cloud.

11.0.0

Prelude

This release is marking the start of Mitaka release support in tempest

New Features

• Tempest library interface addition(API Microversion testing interfaces).

• Compute Microversion testing support in Service Clients.

Other Notes

• OpenStack Releases Supported at this time are Kilo, Liberty, Mitaka The release under current development as of this tag is Newton, meaning that every Tempest commit is also tested against master during the Newton cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Newton (or future releases) cloud.

10.0.0

Prelude

This release includes the addition of the stable library interface for tempest. This behaves just as tempest-lib did prior to this, but instead it lives directly in the tempest project. For more information refer to the library docs.

New Features

• Tempest library interface

Other Notes

• OpenStack Releases Supported at this time are the same as in the previous release 9, Kilo and Liberty.

  The release under current development as of this tag is Mitaka, meaning that every Tempest commit is also tested against master during the Mitaka cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Mitaka (or future releases) cloud.

• Start using reno for managing release notes.