Current Series Release Notes

28.0.0.0rc1-169

New Features

  • Dell PowerFlex driver: Added NVMe-TCP support.

  • The PowerFlex driver now detects the system’s generation type (genType) to enable EC-specific optimizations. Valid values are EC (Erasure Coding) and Mirroring. PowerFlex v4.x systems do not return a genType field in API responses and are based on Mirroring architecture; the driver defaults to Mirroring for these systems. PowerFlex v5.0+ systems return genType explicitly. Note that in-place upgrades from v4.x to v5.0 are not supported due to architectural differences (Mirroring vs. EC).

  • This patch adds the ability to use compression-only for the capacity saving value, either through the hbsd:capacity_saving extra spec, or using the hitachi_drs_default_csv setting for DRS volumes. These values can be retyped back and forth, and the volumes successfully deleted regardless of which setting is currently active.

  • Hitachi: Add ability to configure Data Reduction Shared (DRS) volumes using configuration elements instead of through typing. As storage requirements for DRS are currently all or none, this simplifies creating volumes for DRS-required storage. This patch also sets DRS management to True by default.

  • HPE/XP: Add ability to configure Data Reduction Shared (DRS) volumes using configuration elements instead of through typing. As storage requirements for DRS are currently all or none, this simplifies creating volumes for DRS-required storage. This patch also sets DRS management to True by default.

  • NEC: Add ability to configure Data Reduction Shared (DRS) volumes using configuration elements instead of through typing. As storage requirements for DRS are currently all or none, this simplifies creating volumes for DRS-required storage. This patch also sets DRS management to True by default.

  • Hitachi: Add ability to create temporarily immutable snapshots (snapshot retention) on an hourly granularity . This feature can be enabled on a per-Volume (extra specs) or per-Snapshot (properties) basis. Snapshots with a non-expired retention period cannot be deleted.

  • Hitachi HBSD driver: Enable support for VSP One Block High End. VSP One Block High End supports ADR functionality that offers up to 4:1 data saving, up to 50M IOPS, 8 9’s of availability, and Thin Image Advanced that supports superior ROW functionality. In addition, the Block High End supports vClone technology that allows for instantaneous cloning and shared data between clones.

  • New Cinder volume driver for HPE Alletra MP. The new driver supports iSCSI, FC and NVMe TCP.

  • LightOS driver: retyping a volume between two volume types that differ only in lightos:qos_policy now applies the new QoS policy to the volume in place. Previously the LightOS driver implemented no retype, so every retype fell back to a host-assisted migration that copied the whole volume to a new one. A retype that changes where a volume lives on the cluster – its project, replica count or compression – still migrates, as does one that removes a QoS policy.

  • NetApp Cinder driver for ASAr2 has been updated to now use an SVM as a pool. Instead of the previous implementation of cluster as pool. This implementation gives flexibility for the customers to map multiple SVMs as Cinder pools.

    Cinder driver for ASAr2 now supports additional capabilities for ASAr2 platform.

    The new capabilities include below:

    • netapp_disaggregated_platform - True means the driver is of type disaggregated. Unified ONTAP platform should report a False value.

    • netapp_storage_availability_zones - Reports all the storage availability zones that are mapped to the SVM.

    Apart from the new capabilities, below capabilities are also fixed:

    • pool_name - Pool name is now suffixed with cluster_name and svm name pool_name: <cluster_name>[:<svm_name>]

    • consistencygroup_support and consistent_group_snapshot_enabled capabilities now reports as False as they are yet to be supported for ASAr2.

    • free_capacity_gb - free capacity is reported as sum of aggregate level free capacity across mapped aggregates.

    • total_capacity_gb - total capacity is reported as sum of aggregate level total capacity across mapped aggregates.

    • provisioned_capacity_gb - provisioned capacity is calculated as sum of sizes all the storage units created on the SVM. Since snapshots in ASAr2 are implemented as storage-unit snapshots (FlexVol Snapshots) there is no need to exclude snapshot volumes from the list of storage units.

    • utilization - Due to performance counter APIs not present today, utilization of pool is fixed as 50 units as of now. This will be fixed in upcoming release where plan is to use ONTAP performance APIs to implement the utilization factor.

    • total_volumes - Total volumes is the sum of all LUNs/Namespaces dpending on protocol, created on a pool. This capability can be used to loadbalance the volume creation using goodness and filter functions if multiple SVMs are added from the same cluster as pools.

  • Added a new configuration option netapp_allow_zapi_fallback for the NetApp ONTAP REST driver. It controls whether the REST client is allowed to silently fall back to the legacy ZAPI client for calls that are not yet implemented over REST.

    Defaults to True to preserve existing behavior. When set to False, any call that would otherwise fall back to ZAPI instead raises an explicit NetAppDriverException, making unimplemented REST calls visible instead of silent, which is useful for tracking and closing remaining ZAPI-to-REST parity gaps.

  • NetApp ONTAP driver: Added support for async single consistency group (CG) replication. When netapp_consistent_replication is enabled, the driver creates a single ONTAP consistency group that includes all FlexVols in the backend, providing crash-consistent replication across all volumes. This ensures all volumes in the backend are replicated together and enables crash consistent disaster recovery. The feature supports both creating new consistency groups and adopting existing single CGs. Falls back to per-FlexVol replication when disabled (default behavior).

  • NetApp ONTAP driver: Added validation during driver initialization to check for pre-existing SnapMirror relationships when replication is enabled. The driver validates that existing SnapMirror relationships follow the NetApp Cinder driver’s naming convention (matching source and destination volume names). If conflicting SnapMirrors are found with different destination names, the driver raises an error with detailed remediation steps, preventing replication conflicts.

  • NetApp ONTAP driver: Improved replication policy configuration with explicit choices validation. The netapp_replication_policy option now supports values: AutomatedFailOver (SnapMirror active sync for automatic failover), AutomatedFailOverDuplex (active sync with bidirectional replication), Asynchronous (async with hourly schedule), and MirrorAllSnapshots (default, async mirroring all snapshots).

  • NetApp ONTAP iSCSI and FC drivers: Added support for the clone_across_pools capability, enabling efficient cross-pool volume cloning.

    When creating a volume from a source that is located in a different storage pool (FlexVol) — such as a cached Glance image or a volume-backed Glance image stored directly in Cinder — the driver can now perform the clone operation directly on the storage backend instead of falling back to the slower “attach and dd” method. This significantly improves volume creation performance when using multiple pools, including deployments where Cinder is configured as a Glance backend with images stored in raw format (see Volume-backed images).

    The driver automatically detects the clone boundary and uses the appropriate ONTAP operation:

    • Same FlexVol: Uses efficient FlexClone (instant clone)

    • Cross-aggregate: Uses ONTAP LUN copy operation

    This feature works with both ZAPI and REST API clients and requires no additional configuration. The capability is automatically reported in the volume stats and used by the image-volume cache when enabled.

    For more details about the clone_across_pools capability, please see bp clone_across_pools

  • NetApp ONTAP driver: Several storage management operations are now handled natively by the driver’s REST API client instead of falling back to the older management API. This improves compatibility for deployments configured to use the REST API client exclusively (netapp_use_legacy_client=False). The following operations are now supported natively over REST:

    • Canceling an in-progress file copy during error cleanup.

    • Assigning a QoS policy to an existing LUN, including when managing an existing LUN into Cinder.

    • Looking up, renaming, and cleaning up stale snapshots as part of routine snapshot housekeeping.

    • Creating crash-consistent snapshots across a group of volumes.

    • Reporting per-file space usage, used to decide which cached images can be safely removed.

    • Reporting clone-split progress and deduplication space savings for a volume.

    Reporting per-file space usage and deduplication space savings relies on the private CLI passthrough endpoints /private/cli/volume/file/ show-disk-usage and /private/cli/volume/efficiency, since no public REST equivalents exist yet.

  • Added support for crash-consistent replication using the active sync policies - AutomatedFailOver and AutomatedFailOverDuplex with NetApp storage backends. By enabling the new netapp_consistent_replication flag, all FlexVol volumes in a backend stanza are protected together in a single Consistency Group (CG), streamlining disaster recovery and ensuring data consistency. The feature automatically manages destination CG and FlexVol creation. Also with AutomatedFailOverDuplex policy, an option to set the host proximity has been added using the netapp_proximal_nodes flag when the OpenStack volumes are attached. Migration from the previous per-FlexVol replication strategy requires cleanup of legacy artifacts, with prechecks enforced.

  • Quobyte driver: adds thin provisioning support for Quobyte volumes.

Upgrade Notes

  • The fix for Bug #2163042 stops the dependent backup counter (num_dependent_backups) from drifting, but does not repair values that have already drifted. A backup that wrongly reports dependent backups, and so cannot be deleted, will continue to do so until its row is corrected in the database.

  • The boto3 library is no longer directly required by cinder. The library and its dependencies should be installed when s3 backup driver is used.

  • PowerFlex 5.0 (EC generation) is not backward compatible with PowerFlex 4.x (Mirroring generation). In-place upgrade from v4.x to v5.0 is not supported.

  • With the fix for Bug #2142083, for os-brick versions later than 7.0.0, users can configure the PowerFlex san_username in the /opt/emc/scaleio/openstack/connector.conf file. This allows volume detach operations to succeed even if the PowerFlex san_username is changed. This issue only affects legacy attached volumes. New volume attachment and detachment operations will not encounter this issue since the mapping happens on the Cinder side.

  • The [oslo_policy] enforce_scope and [oslo_policy] enforce_new_defaults configuration options now default to True. This means the RBAC new defaults are enabled by default. Operators who still rely on the deprecated old defaults should explicitly set enforce_new_defaults = False in the [oslo_policy] section of cinder.conf before upgrading and plan their migration to the new RBAC defaults. See the Policy Personas and Permissions documentation for details.

  • Now the google-api-python-client library is an optional dependency. It should be additionally installed when the GCS volume backup driver is used.

  • Previously, DRS configurations could only be achieved with extra specs. The new hitachi_use_drs_volumes allows for configuring the driver to always use DRS volumes. This is useful for for storage such as the VSP One Block series that always require DRS volumes.

  • The hitachi_manage_drs_volumes configuration value is now defaulted to True. This is an important change as the False setting may cause failures when volume resize and delete operations are requested, and cloning has occurred. Only set this to False if you anticipate not needing to delete or resize volumes.

  • KIOXIA KumoScale driver: the driver no longer calls urllib3.disable_warnings() at import. Deployments running it will now see InsecureRequestWarning messages that were previously hidden; the driver’s requests have always been unverified (cert_reqs=ssl.CERT_NONE), the warnings were just being discarded. Set suppress_requests_ssl_warnings to quiet them.

  • Lightbits driver: the driver no longer calls urllib3.disable_warnings() when its module is imported. That call suppressed all urllib3 warnings for the entire cinder-volume process, affecting every configured backend and not just Lightbits, and it took effect even in deployments with no Lightbits backend configured.

    Deployments that run the Lightbits driver with driver_ssl_cert_verify = False will now see InsecureRequestWarning messages that were previously hidden. To suppress them, set the existing suppress_requests_ssl_warnings option, which is the supported, operator-controlled way to do this. The recommended alternative is to enable driver_ssl_cert_verify and configure a trusted CA bundle.

  • NetApp ONTAP drivers now default to HTTPS with certificate verification enabled. After upgrading, existing backends using self-signed ONTAP certificates must either set netapp_ssl_cert_path to a trusted CA bundle or temporarily set netapp_ssl_cert_verify = false (not recommended for production). When ONTAP certificates expire, provisioning will fail until the trust material is renewed or netapp_ssl_cert_verify = false is explicitly configured as a known security trade-off.

    Backends that set netapp_server_port = 80 while relying on the previous implicit HTTP default must also set netapp_transport_type = http or move to port 443 with HTTPS.

  • The Everpure driver now requires py-pure-client>=1.89.0.

  • Now the python-swiftclient library is an optional dependency. It should be additionally installed when the swift volume backup driver is used.

Deprecation Notes

  • The IBM Flashsystem drivers have been marked as ‘unsupported’ since the OpenStack Train release, thereby indicating that they are subject to removal from the cinder codebase. The vendor no longer supports the Flashsystem drivers, and so they are removed in this release.

Security Issues

  • KIOXIA KumoScale driver: an import-scope urllib3.disable_warnings() call was removed. With no warning category it disabled every urllib3 warning process-wide, including TLS warnings from other backends sharing the cinder-volume process, in deployments with no KumoScale backend configured, and it overrode suppress_requests_ssl_warnings.

  • Lightbits driver: an import-scope urllib3.disable_warnings() call was removed. Because it ran at import time with no warning category argument, it silently disabled every urllib3 warning process-wide – including TLS warnings raised by other backends sharing the same cinder-volume process – and overrode the operator’s suppress_requests_ssl_warnings setting.

  • NetApp ONTAP client defaults now use HTTPS transport by default for both REST and legacy ZAPI paths.

    TLS verification behavior is explicitly controlled through netapp_ssl_cert_verify and netapp_ssl_cert_path with the following precedence:

    • netapp_ssl_cert_verify = false: certificate verification is disabled.

    • netapp_ssl_cert_verify = true and netapp_ssl_cert_path set: verify against the provided CA/certificate bundle path.

    • netapp_ssl_cert_verify = true and no netapp_ssl_cert_path: verify against system default CA roots.

    The help text for netapp_ssl_cert_path is clarified to reflect that it applies to both REST and legacy ZAPI clients. This change addresses transport/verification hardening aligned with LP#2157914.

Bug Fixes

  • Bug #2163042: Fixed incremental backups sharing a parent losing count of each other when created or deleted concurrently, which could leave the parent backup permanently undeletable. The dependent counter is now updated in the database.

  • NetApp driver bug #1883603: Fixed an issue where concurrent iSCSI terminate_connection calls on multiattach volumes could race and skip the final unmap, leaving stale LUN mappings on the array.

  • IBM Spectrum Virtualize family driver: Bug #1992160: Fixed an issue where the driver was collecting all node IP addresses during initialization. The driver now only records IPs for nodes with the host flag set to yes.

  • IBM Spectrum Virtualize family driver: Bug #2004537: Fixed misleading error raised when storwize_portset in cinder.conf is invalid. Cinder now reports the correct error message instead of incorrectly indicating lack of iSCSI support.

  • IBM Storage Virtualize Family driver: Bug #2015722: The internal driver use of opaque error codes resulted in some log messages that did not identify an error correctly. The driver code now uses symbolic constants with meaningful names to correct this issue.

  • NetApp driver bug #2101948: Fixed an issue in the ONTAP REST path where EMS logging could switch the active vserver on a shared connection. With urllib3 connection pooling and concurrent requests, this could route in-flight API calls to the wrong SVM and cause failures like NetAppDriverException: Volume "sharename" not found.

  • NetApp driver bug #2121791: Fixed NetApp NVMe/TCP: Fixed multipath support for volume attachments Previously, the NVMe over TCP driver only returned a single target portal when initializing volume connections, even when multiple paths were available on the NetApp array.This caused volumes to attach using only one path, preventing proper multipath functionality. The initialize_connection method now returns all available target portals, enabling native NVMe multipath to utilize all configured paths for improved redundancy and performance.

  • Bug #2129271: Transport specific NVMe-oF protocol identifiers (for example, NVMe-TCP) are now included when the scheduler determines whether a driver’s storage protocol is an NVMe-oF protocol. Previously, only non-protocol-specific identifier variants were considered which could lead to shared targets not working correctly.

  • Bug #2139068: Fixed an issue where volume.delete.end notifications were not sent when volumes were deleted as part of a group deletion process. This ensures delete notifications are properly emitted for every volume removed during group deletion.

  • Dell PowerFlex Driver Bug #2139336: Fixed PowerFlex 5.0 issues pertaining to driver initialization, volume creation, snapshot operations, source‑based provisioning, backup handling.

  • PowerFlex 5.0 no longer requires volume sizes to be in 8 GB multiples. Previous versions (< 5.0) had this hard requirement, but it has been relaxed in v5.

  • PowerFlex Driver Bug #2142083: Fixed an issue where volume detach operations failed after changing the PowerFlex san_username. Previously, the PowerFlex san_username was stored in the Nova block_device_mapping table and reused during detach operations. With this fix, the driver now reads the san_username from the connector.conf file instead.

  • NetApp ONTAP driver bug #2153545: Fixed excessive 401 Unauthorized noise flooding the ONTAP HTTP access logs. Python’s urllib HTTPBasicAuthHandler was performing a challenge-response round-trip on every ZAPI call, generating a large number of 401 entries per minute. The NaServer HTTP client has been migrated to requests.Session with HTTPBasicAuth to send credentials preemptively.

  • NetApp ONTAP driver: Fixed a bug where SnapMirror quiesce, abort, and break operations incorrectly used the legacy ZAPI client when netapp_consistent_replication is enabled.

    When netapp_consistent_replication is set to True, the driver manages replication through ONTAP Consistency Groups (CGs), which are only supported via the REST API. Other code paths in the driver that operate on CG-based SnapMirror relationships (such as creation, failover, and resync) already enforce REST by passing force_rest=True when obtaining the API client. However, the quiesce_then_abort and break_snapmirror methods were missing this enforcement and fell back to the ZAPI client.

    As a result, customers using netapp_consistent_replication would encounter errors during replication failover or failback, because ONTAP rejects ZAPI-based quiesce, abort, and break calls against CG SnapMirror relationships. The fix passes force_rest based on the value of netapp_consistent_replication, ensuring these operations use the REST client when CG replication is active and ZAPI otherwise, preserving backward compatibility.

  • Fixed bug #2156039: NetApp ONTAP driver: During backend reinitialization, if a consistency group expand is still in progress on the array, SnapMirror repair for the CG relationship can return ONTAP error 6621658 with “Expand operation in progress, wait for the operation to complete”. Cinder now treats this as an expected transient condition and logs an informational message instead of reporting a SnapMirror re-sync failure in the logs.

  • NetApp ONTAP driver: Fixed a pre-existing key mismatch in the create_snapmirror_for_cg SnapMirror repair log message. The debug log format string referenced %(src_cg)s and %(dest_cg)s but the payload used src_volume and dest_volume, which would have caused a KeyError if the repair branch was triggered. The payload keys are now aligned with the format string.

  • Dell PowerFlex Driver Bug #2160063: Fixed NVMe-TCP volume attachment failure when host name exceeds 31 characters. PowerFlex has a 31-character limit for host names. The driver now truncates long host names using an MD5 hash suffix to preserve uniqueness while staying within the limit.

  • Quobyte driver bug #2074377: Fixes a bug that caused failures on some image_util operations due to changed method signatures.

  • Bug #2144550: Fixed a bug that could cause image creation from volumes to fail due to incorrect handling of the volume metadata passed to Glance.

  • Hitachi driver: Fixed cloning for B20 Storage Array when creating a clone that is larger than the source volume. This fix first creates the clone, and then resizes it. The parent LDEV must be resized first if it was created with hitachi_manage_drs_volumes=False configured.

  • Ceph backup driver bug #2155612: Fixed data corruption when restoring a sparse volume (such as a LUKS-encrypted one) to a new volume. The driver skipped all-zero chunks without advancing the destination offset, so every subsequent non-zero chunk was written at the wrong location and the restored volume was silently truncated even though the restore reported success. The destination offset is now advanced when a zero chunk is discarded, so the restored layout matches the source.

  • Bug #2127675: Fixed cloning across pools functionality for image volume cache.

  • Bug #2081283: Fixed unset operation for default volume type when project doesn’t exist.

  • LVM driver with the nvmet target helper: fixed volume export failing with TypeError: Port.setup() got multiple values for argument 'err_func' against recent nvmetcli changes. The nvmet library removed the root argument from Port.setup(); since nvmet is not version-pinned, Cinder now detects the installed library’s signature and only forwards root when it is still accepted, working with both older and newer nvmetcli.

  • Bug #2125312: Fixed an issue where cascade volume deletion failed for non-admin users after the following sequence: create a snapshot, revert to it, transfer the volume to another project, create a new snapshot, then attempt cascade delete. Admin users were unaffected. Cascade delete now works correctly for non-admin users in this scenario.

  • Bug 1936474 Fixed improper cleanup of glance image metadata when rescheduling instance.

  • Hitachi VSP driver: Fix to correctly support auto space reclamation. bug #2153971 <https://bugs.launchpad.net/cinder/+bug/2153971>

  • Hitachi VSP driver: VCP-Volume deletion skipped/left behind if dependent VC-Volume deletion times. bug #2158825 <https://bugs.launchpad.net/cinder/+bug/2158825> This issue may occur in a customer environment, but it has not been observed at scale during normal development or QA testing. It is more prominent in an environment where volumes are brought up and torn down constantly, such as with Tempest CI testing.

  • Fixed performance issues in the Hitachi HBSD driver by adding caching and request batching. Caching is enabled by default, and can be disabled by setting hitachi_rest_use_object_caching to False in cinder.conf. With a 4 port configuration, this patch offers a roughly 70% increase in performance when host groups are managed by the driver, and a 90%+ increase in performance when using custom configured host groups (after the cache has been built).

  • HPE driver Bug #2143385: Fixed: Handled deprecation of QOS parameters and comments for online copy. While passing QOS params use units.k(1000) as multiplier, as backend is expecting values in KB.

  • LightOS driver: retyping a volume between two volume types that name different LightOS projects left the source volume behind on the backend. The driver resolved a volume’s project from the lightos:project_name extra spec of its volume type on every call, but Cinder keeps the destination volume type on the database record it uses to delete the source volume, so the delete was addressed to the destination project and failed. A volume’s project and LightOS UUID are now recorded in its provider_id when it is created, and travel with the record that addresses it. Volumes created before this fix are stamped automatically when the volume service starts; until then they keep resolving their project from their volume type.

  • NetApp ONTAP driver: Fixed SnapMirror creation to always use ‘extended_data_protection’ (XDP) relationship type instead of the deprecated ‘data_protection’ (DP) type. XDP is required for consistency group support and prevents ONTAP error 13001 on newer versions where DP relationships are no longer supported. This change applies to all new SnapMirrors regardless of FlexVol or FlexGroup pool type.

  • NetApp ONTAP driver: Fixed replication policy detection to use explicit string matching instead of substring matching. This prevents false positives where policy names containing certain substrings would be incorrectly identified.

  • NetApp ONTAP block driver bug #2152031: Fixed an inconsistency in the NetApp ONTAP block driver where LUN space-allocation (thin-provisioning SCSI UNMAP support) was always disabled when creating volumes, regardless of the ONTAP version in use.

    Starting with ONTAP 9.15.1, NetApp changed the default LUN space-allocation setting from disabled to enabled. The driver now reflects this: when the netapp:space_allocation extra spec is not set, the driver checks the ONTAP version and enables space-allocation by default for ONTAP 9.15.1 and later, preserving the previous disabled default for older versions.

    The explicit netapp:space_allocation extra spec continues to take precedence over version-based defaults.

    ONTAP defaults and limits

  • NetApp ONTAP driver bug #2163623: Fixed consistency group snapshot clones with the REST API client sourcing from the live file instead of the snapshot, or failing outright when snapshot_directory_access_enabled is False on the FlexVol. The private-CLI fallback used in that case now also preserves the backup-clone, overwrite, space-reservation, and QoS policy settings that were previously dropped when falling back.

  • NetApp ONTAP driver bug #2164909: Fixed canceling an in-progress file copy always failing on the ZAPI client, which prevented the copy job from being properly stopped on the storage controller.

  • Everpure driver: Fixed the backend reporting its provisioned capacity under the provisioned_capacity key instead of the standardized, scheduler-required provisioned_capacity_gb key. As a result the scheduler never saw the value and silently fell back to allocated_capacity_gb for thin-provisioning/over-subscription decisions, and capacity-usage notifications were suppressed for Pure backends because the pool failed the scheduler’s required-keys check. The driver now reports provisioned_capacity_gb, restoring accurate over-subscription accounting and capacity telemetry.

  • Everpure driver bug #2158313: Fixed creating a volume from a snapshot, and setting QoS limits, with newer py-pure-client releases.

  • Bug #2121464: Fixed Pure Storage driver raising an AttributeError when attaching a volume to a host it was already connected to. The fix now covers all supported protocols (iSCSI, Fibre Channel and NVMe).

  • Everpure driver bug #2143833: The Everpure driver now correctly retrieves both LACP bond and VIF network interfaces. Previously, only LACP bond interfaces were returned due to a filter logic issue.

  • [Pure Storage] bug #2138744: Added @coordination.synchronized to initialize and terminate connections to serialize operations, preventing premature LUN reuse.

  • Quobyte driver: Drops the psutil dependency for the Quobyte driver.

  • Quobyte driver bug #2142477: Fixed missing meta data setting when creating a volume from a snapshot.