HPE 3PAR and HPE Primera Driver for OpenStack Cinder¶
HPE3PARISCSIDriver drivers, which are based on
the Block Storage service (Cinder) plug-in architecture, run volume operations
by communicating with the HPE 3PAR and HPE Primera storage system over HTTP,
HTTPS, and SSH connections. The HTTP and HTTPS communications use
python-3parclient, which is part of PyPi.
For information on HPE 3PAR and HPE Primera Driver for OpenStack Cinder, refer to content kit page.
To use the HPE 3PAR and HPE Primera drivers, install the following software and components on the HPE 3PAR storage system:
HPE 3PAR Operating System software version 3.1.3 MU1 or higher.
Deduplication provisioning requires SSD disks and HPE 3PAR Operating System software version 3.2.1 MU1 or higher.
Enabling Flash Cache Policy requires the following:
Array must contain SSD disks.
HPE 3PAR Operating System software version 3.2.1 MU2 or higher.
python-3parclient version 4.2.0 or newer.
Flash Cache must be enabled on the array with the CLI command createflashcache SIZE, where size must be in 16 GB increments. For example, createflashcache 128g will create 128 GB of Flash Cache for each node pair in the array.
The Dynamic Optimization is required to support any feature that results in a volume changing provisioning type or CPG. This may apply to the volume migrate, retype and manage commands.
The Virtual Copy feature supports any operation that involves volume snapshots. This applies to the volume snapshot-* commands.
Enabling Volume Compression requires the following:
Array must contain SSD disks.
HPE 3PAR Operating System software version 3.3.1 MU1 or higher.
HPE 3PAR Storage System with 8k or 20k series
HPE 3PAR Web Services API Server must be enabled and running.
One Common Provisioning Group (CPG).
Additionally, you must install the
python-3parclientversion 4.2.0 or newer from PyPi on the system with the enabled Block Storage service volume drivers.
To use the HPE Primera backend, install the following software and components on the HPE Primera storage system:
HPE Primera Operating System software version 4.0.0.
On HPE Primera storage system, Dedup & Compression is combined as single option ‘deco’. Due to this, only either ‘thin’ volume or ‘deco’ volume can be created.
Also, port number 443 is used instead of 8080. This only affects cinder configuration.
Additionally, you must install the
python-3parclientversion 4.2.11 or newer from PyPi on the system with the enabled Block Storage service volume drivers.
Create, delete, attach, and detach volumes.
Create, list, and delete volume snapshots.
Create a volume from a snapshot.
Copy an image to a volume.
Copy a volume to an image.
Clone a volume.
Extend a volume.
Migrate a volume with back-end assistance.
Retype a volume.
Manage and unmanage a volume.
Manage and unmanage a snapshot.
Replicate host volumes.
Fail-over host volumes.
Fail-back host volumes.
Retype a replicated volume.
Create, delete, update, snapshot, and clone generic volume groups.
Create and delete generic volume group snapshots.
Create a generic volume group from a group snapshot or another group.
Group Replication with More Granularity (Tiramisu).
Volume Revert to Snapshot.
Additional Backend Capabilities.
Report Backend State in Service List.
Attach a volume to multiple servers simultaneously (multiattach).
Volume type support for both HPE 3PAR drivers includes the ability to set the
following capabilities in the OpenStack Block Storage API
cinder.api.contrib.types_extra_specs volume type extra specs extension
To work with the default filter scheduler, the key values are case sensitive
and scoped with
hpe3par:. For information about how to set the key-value
pairs and associate them with a volume type, run the following command:
$ openstack help volume type
Volumes that are cloned only support the extra specs keys cpg, snap_cpg, provisioning and vvs. The others are ignored. In addition the comments section of the cloned volume in the HPE 3PAR / HPE Primera array is not populated.
If volume types are not used or a particular key is not set for a volume type, the following defaults are used:
hpe3par:cpg- Defaults to the
hpe3par_cpgsetting in the
hpe3par:snap_cpg- Defaults to the
hpe3par_snapsetting in the
hpe3par_snapis not set, it defaults to the
hpe3par:provisioning- Defaults to
thinprovisioning, the valid values are
hpe3par:persona- Defaults to the
2 - Generic-ALUApersona. The valid values are:
1 - Generic
2 - Generic-ALUA
3 - Generic-legacy
4 - HPUX-legacy
5 - AIX-legacy
6 - EGENERA
7 - ONTAP-legacy
8 - VMware
9 - OpenVMS
10 - HPUX
11 - WindowsServer
hpe3par:flash_cache- Defaults to
false, the valid values are
QoS support for both HPE 3PAR drivers includes the ability to set the
following capabilities in the OpenStack Block Storage API
cinder.api.contrib.qos_specs_manage qos specs extension module:
The qos keys above no longer require to be scoped but must be created and associated to a volume type. For information about how to set the key-value pairs and associate them with a volume type, run the following commands:
$ openstack help volume qos
The following keys require that the HPE 3PAR / HPE Primera array has a Priority Optimization enabled.
The virtual volume set name that has been predefined by the Administrator with quality of service (QoS) rules associated to it. If you specify extra_specs
hpe3par:vvs, the qos_specs
maxBWSsettings are ignored.
The QoS I/O issue bandwidth minimum goal in MBs. If not set, the I/O issue bandwidth rate has no minimum goal.
The QoS I/O issue bandwidth rate limit in MBs. If not set, the I/O issue bandwidth rate has no limit.
The QoS I/O issue count minimum goal. If not set, the I/O issue count has no minimum goal.
The QoS I/O issue count rate limit. If not set, the I/O issue count rate has no limit.
The latency goal in milliseconds.
The priority of the QoS rule over other rules. If not set, the priority is
normal, valid values are
Since the Icehouse release, minIOPS and maxIOPS must be used together to set I/O limits. Similarly, minBWS and maxBWS must be used together. If only one is set the other will be set to the same value.
The following key requires that the HPE 3PAR / HPE Primera array has an Adaptive Flash Cache enabled.
hpe3par:flash_cache- The flash-cache policy, which can be turned on and off by setting the value to
hpe3par:compression- The volume compression, which can be turned on and off by setting the value to
Other restrictions and considerations for
For a compressed volume, minimum volume size needed is 16 GB; otherwise resulting volume will be created successfully but will not be a compressed volume.
A full provisioned volume cannot be compressed, if a compression is enabled and provisioning type requested is full, the resulting volume defaults to thinly provisioned compressed volume.
While creating volume on HPE Primera storage system, only below two combinations are supported. If any other combination is used, then volume is not created.
thin volume: provisioning =
thinand compression =
deco volume: provisioning =
dedupand compression =
LDAP and AD authentication is now supported in the HPE 3PAR driver.
The 3PAR back end must be properly configured for LDAP and AD authentication prior to configuring the volume driver. For details on setting up LDAP with 3PAR, see the 3PAR user guide.
hpe3par_password parameters in
cinder.conf can be used with LDAP and AD credentials.
Enable the HPE 3PAR Fibre Channel and iSCSI drivers¶
HPE3PARISCSIDriver are installed with the
python-3parclientPython package on the OpenStack Block Storage system.
$ pip install 'python-3parclient>=4.0,<5.0'
Verify that the HPE 3PAR Web Services API server is enabled and running on the HPE 3PAR / HPE Primera storage system.
Log onto the HPE 3PAR / HPE Primera storage system with administrator access.
$ ssh 3paradm@<HPE storage system IP Address>
View the current state of the Web Services API Server.
$ showwsapi -Service- -State- -HTTP_State- HTTP_Port -HTTPS_State- HTTPS_Port -Version- Enabled Active Enabled 8008 Enabled 8080 1.1
If the Web Services API Server is disabled, start it.
If the HTTP or HTTPS state is disabled, enable one of them.
$ setwsapi -http enable
$ setwsapi -https enable
To stop the Web Services API Server, use the stopwsapi command. For other options run the setwsapi -h command.
If you are not using an existing CPG, create a CPG on the HPE 3PAR / HPE Primera storage system to be used as the default location for creating volumes.
Make the following changes in the
# WSAPI Server URL. # This setting applies to both 3PAR and Primera. # Example 1: for 3PAR, URL is: https://<3par ip>:8080/api/v1 # Example 2: for Primera, URL is: https://<primera ip>:443/api/v1 # 3PAR / Primera username with the 'edit' role hpe3par_username=edit3par # 3PAR / Primera password for the user specified in hpe3par_username hpe3par_password=3parpass # 3PAR / Primera CPG to use for volume creation hpe3par_cpg=OpenStackCPG_RAID5_NL # IP address of SAN controller for SSH access to the array san_ip=10.10.22.241 # Username for SAN controller for SSH access to the array san_login=3paradm # Password for SAN controller for SSH access to the array san_password=3parpass # FIBRE CHANNEL DRIVER # (uncomment the next line to enable the FC driver) #volume_driver=cinder.volume.drivers.hpe.hpe_3par_fc.HPE3PARFCDriver # iSCSI DRIVER # If you enable the iSCSI driver, you must also set values # for hpe3par_iscsi_ips or iscsi_ip_address in this file. # Note: The iSCSI driver is supported with 3PAR (all versions) # and Primera (version 4.2 or higher). If you configure iSCSI # with Primera 4.0 or 4.1, the driver will fail to start. # (uncomment the next line to enable the iSCSI driver) #volume_driver=cinder.volume.drivers.hpe.hpe_3par_iscsi.HPE3PARISCSIDriver # iSCSI multiple port configuration # hpe3par_iscsi_ips=10.10.220.253:3261,10.10.222.234 # Still available for single port iSCSI configuration #iscsi_ip_address=10.10.220.253 # Enable HTTP debugging to 3PAR / Primera hpe3par_debug=False # Enable CHAP authentication for iSCSI connections. hpe3par_iscsi_chap_enabled=false # The CPG to use for Snapshots for volumes. If empty hpe3par_cpg will be # used. hpe3par_cpg_snap=OpenStackSNAP_CPG # Time in hours to retain a snapshot. You can't delete it before this # expires. hpe3par_snapshot_retention=48 # Time in hours when a snapshot expires and is deleted. This must be # larger than retention. hpe3par_snapshot_expiration=72 # The ratio of oversubscription when thin provisioned volumes are # involved. Default ratio is 20.0, this means that a provisioned # capacity can be 20 times of the total physical capacity. max_over_subscription_ratio=20.0 # This flag represents the percentage of reserved back-end capacity. reserved_percentage=15
You can enable only one driver on each cinder instance unless you enable multiple back-end support. See the Cinder multiple back-end support instructions to enable this feature.
You can configure one or more iSCSI addresses by using the
hpe3par_iscsi_ipsoption. Separate multiple IP addresses with a comma (
,). When you configure multiple addresses, the driver selects the iSCSI port with the fewest active volumes at attach time. The 3PAR array does not allow the default port 3260 to be changed, so IP ports need not be specified.
Save the changes to the
cinder.conffile and restart the cinder-volume service.
The HPE 3PAR Fibre Channel and iSCSI drivers are now enabled on your OpenStack system. If you experience problems, review the Block Storage service log files for errors.
The following table contains all the configuration options supported by the HPE 3PAR Fibre Channel and iSCSI drivers.
Configuration option = Default value
(List of String) List of the 3PAR / Primera CPG(s) to use for volume creation
(String) The 3PAR / Primera CPG to use for snapshots of volumes. If empty the userCPG will be used.
(Boolean) Enable HTTP debugging to 3PAR / Primera
(Boolean) Enable CHAP authentication for iSCSI connections.
(List of String) List of target iSCSI addresses to use.
(String) 3PAR / Primera password for the user specified in hpe3par_username
(String) The time in hours when a snapshot expires and is deleted. This must be larger than expiration
(String) The time in hours to retain a snapshot. You can’t delete it before this expires.
(String) The nsp of 3PAR backend to be used when: (1) multipath is not enabled in cinder.conf. (2) Fiber Channel Zone Manager is not used. (3) the 3PAR backend is prezoned with this specific nsp only. For example if nsp is 2 1 2, the format of the option’s value is 2:1:2
(String) 3PAR / Primera username with the ‘edit’ role
Specify NSP for FC Bootable Volume¶
Given a system connected to HPE 3PAR via FC and multipath setting is NOT used in cinder.conf. When the user tries to create a bootable volume, it fails intermittently with the following error: Fibre Channel volume device not found
This happens when a zone is created using second or later target from 3PAR backend. In this case, HPE 3PAR client code picks up first target to form initiator target map. This can be illustrated with below example.
Sample output of showport command:
$ showport -sortcol 6
N:S:P Mode State ----Node_WWN---- -Port_WWN/HW_Addr- Type Protocol Partner FailoverState 0:1:1 target ready 2FF70002AC002DB6 20110002AC002DB6 host FC - - 0:1:2 target ready 2FF70002AC002DB6 20120002AC002DB6 host FC 1:1:2 none 1:1:1 initiator ready 2FF70002AC002DB6 21110002AC002DB6 rcfc FC - - 1:1:2 target ready 2FF70002AC002DB6 21120002AC002DB6 host FC 0:1:2 none 2:1:1 initiator ready 2FF70002AC002DB6 22110002AC002DB6 rcfc FC - - 2:1:2 target ready 2FF70002AC002DB6 22120002AC002DB6 host FC 3:1:2 none 3:1:1 target ready 2FF70002AC002DB6 23110002AC002DB6 host FC - - 3:1:2 target ready 2FF70002AC002DB6 23120002AC002DB6 host FC 2:1:2 none
Suppose zone is created using targets “2:1:2” and “3:1:2” from above output. Then initiator target map is created using target “0:1:1” only. In such a case, the path is not found, and bootable volume creation fails.
To avoid above mentioned failure, the user can specify the target in 3PAR backend section of cinder.conf as follows:
hpe3par_target_nsp = 3:1:2
Using above mentioned nsp, respective wwn information is fetched. Later initiator target map is created using wwn information and bootable volume is created successfully.
Note: If above mentioned option (nsp) is not specified in cinder.conf, then the original flow is executed i.e first target is picked and bootable volume creation may fail.
Peer Persistence support¶
Given 3PAR/Primera backend configured with replication setup, currently only Active/Passive replication is supported by 3PAR/Primera in OpenStack. When failover happens, nova does not support volume force-detach (from dead primary backend) / re-attach to secondary backend. Storage engineer’s manual intervention is required.
To overcome above scenario, support for Peer Persistence is added. Given a system with Peer Persistence configured and replicated volume is created. When this volume is attached to an instance, vlun is created automatically in secondary backend, in addition to primary backend. So that when a failover happens, it is seamless.
For Peer Persistence support, perform following steps: 1] enable multipath 2] set replication mode as “sync” 3] configure a quorum witness server
Specify ip address of quorum witness server in
[within backend section] as given below:
[3pariscsirep] hpe3par_api_url = http://10.50.3.7:8008/api/v1 hpe3par_username = <user_name> hpe3par_password = <password> ... <other parameters> ... replication_device = backend_id:CSIM-EOS12_1611702, replication_mode:sync, quorum_witness_ip:10.50.3.192, hpe3par_api_url:http://10.50.3.22:8008/api/v1, ... <other parameters> ...
Support duplicated FQDN in network¶
The 3PAR driver uses the FQDN of the node that is doing the attach as an unique identifier to map the volume.
The problem is that the FQDN is not always unique, there are environments where the same FQDN can be found in different systems, and in those cases if both try to attach volumes the second system will fail.
One example of this happening would be on a QA environment where you are creating VMs and they all have names like controller-0.localdomain and compute-0.localdomain.
To support these kind of environments, the user can specify below flag in backend_defaults section or the specific cinder driver section of cinder.conf as follows:
unique_fqdn_network = False
When this flag is used, then during attach volume to instance, iscsi initiator name is used instead of FQDN.
If above mentioned flag is not specified in cinder.conf, then its value is considered as True (by default) and FQDN is used (existing behavior).