Manage volumes¶
Depending on the setup of your cloud provider, they may give you an endpoint to
use to manage volumes. You can use the openstack CLI to manage volumes.
For the purposes of the compute service, attaching, detaching and creating a server from a volume are of primary interest.
Refer to the CLI documentation for more information.
Volume multi-attach¶
Nova added support for multiattach volumes in the 17.0.0 Queens release.
This document covers the nova-specific aspects of this feature. Refer to the block storage admin guide for more details about creating multiattach-capable volumes.
Boot from volume and attaching a volume to a server that is not SHELVED_OFFLOADED is supported. Ultimately the ability to perform these actions depends on the compute host and hypervisor driver that is being used.
There is also a recorded overview and demo for volume multi-attach.
Requirements¶
The minimum required compute API microversion for attaching a multiattach-capable volume to more than one server is 2.60.
Cinder 12.0.0 (Queens) or newer is required.
The
nova-computeservice must be running at least Queens release level code (17.0.0) and the hypervisor driver must support attaching block storage devices to more than one guest. Refer to Feature Support Matrix for details on which compute drivers support volume multiattach.When using the libvirt compute driver, the following native package versions determine multiattach support:
libvirt must be greater than or equal to 3.10, or
qemu must be less than 2.10
Swapping an in-use multiattach volume is not supported (this is actually controlled via the block storage volume retype API).
Known issues¶
Creating multiple servers in a single request with a multiattach-capable volume as the root disk is not yet supported: https://bugs.launchpad.net/nova/+bug/1747985
Subsequent attachments to the same volume are all attached in read/write mode by default in the block storage service. A future change either in nova or cinder may address this so that subsequent attachments are made in read-only mode, or such that the mode can be specified by the user when attaching the volume to the server.
Testing¶
Continuous integration testing of the volume multiattach feature is done
via the tempest-full and tempest-slow jobs, which, along with the
tests themselves, are defined in the tempest repository.
Managing volume attachments¶
During the lifecycle of an instance admins may need to check various aspects of how a given volume is mapped both to an instance and the underlying compute hosting the instance. This could even include refreshing different elements of the attachment to ensure the latest configuration changes within the environment have been applied.
Note
If you encounter any dangling volume attachments in either the Nova or
Cinder databases, a hard reboot of the affected instance can help
resolve the issue. During the instance reboot process, Nova performs
a synchronization mechanism that verifies the availability of volume
attachments in the Cinder database. Any missing or dangling/stale
attachments are detected and deleted from both Nova and Cinder during
hard reboot process.
Checking an existing attachment¶
Existing volume attachments can be checked using the following OpenStack Client commands:
List all volume attachments for a given instance:
$ openstack server volume list 216f9481-4c9d-4530-b865-51cedfa4b8e7
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
| ID | Device | Server ID | Volume ID |
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
| 8b9b3491-f083-4485-8374-258372f3db35 | /dev/vdb | 216f9481-4c9d-4530-b865-51cedfa4b8e7 | 8b9b3491-f083-4485-8374-258372f3db35 |
+--------------------------------------+----------+--------------------------------------+--------------------------------------+
List all volume attachments for a given instance with the Cinder volume attachment and Block Device Mapping UUIDs also listed with microversion >=2.89:
$ openstack --os-compute-api-version 2.89 server volume list 216f9481-4c9d-4530-b865-51cedfa4b8e7
+----------+--------------------------------------+--------------------------------------+------+------------------------+--------------------------------------+--------------------------------------+
| Device | Server ID | Volume ID | Tag | Delete On Termination? | Attachment ID | BlockDeviceMapping UUID |
+----------+--------------------------------------+--------------------------------------+------+------------------------+--------------------------------------+--------------------------------------+
| /dev/vdb | 216f9481-4c9d-4530-b865-51cedfa4b8e7 | 8b9b3491-f083-4485-8374-258372f3db35 | None | False | d338fb38-cfd5-461f-8753-145dcbdb6c78 | 4e957e6d-52f2-44da-8cf8-3f1ab755e26d |
+----------+--------------------------------------+--------------------------------------+------+------------------------+--------------------------------------+--------------------------------------+
List all Cinder volume attachments for a given volume from microversion >= 3.27:
$ openstack --os-volume-api-version 3.27 volume attachment list --volume-id 8b9b3491-f083-4485-8374-258372f3db35
+--------------------------------------+--------------------------------------+--------------------------------------+----------+
| ID | Volume ID | Server ID | Status |
+--------------------------------------+--------------------------------------+--------------------------------------+----------+
| d338fb38-cfd5-461f-8753-145dcbdb6c78 | 8b9b3491-f083-4485-8374-258372f3db35 | 216f9481-4c9d-4530-b865-51cedfa4b8e7 | attached |
+--------------------------------------+--------------------------------------+--------------------------------------+----------+
Show the details of a Cinder volume attachment from microversion >= 3.27:
$ openstack --os-volume-api-version 3.27 volume attachment show d338fb38-cfd5-461f-8753-145dcbdb6c78
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| ID | d338fb38-cfd5-461f-8753-145dcbdb6c78 |
| Volume ID | 8b9b3491-f083-4485-8374-258372f3db35 |
| Instance ID | 216f9481-4c9d-4530-b865-51cedfa4b8e7 |
| Status | attached |
| Attach Mode | rw |
| Attached At | 2021-09-14T13:03:38.000000 |
| Detached At | |
| Properties | access_mode='rw', attachment_id='d338fb38-cfd5-461f-8753-145dcbdb6c78', auth_method='CHAP', auth_password='4XyNNFV2TLPhKXoP', auth_username='jsBMQhWZJXupA4eWHLQG', cacheable='False', driver_volume_type='iscsi', encrypted='False', qos_specs=, target_discovered='False', target_iqn='iqn.2010-10.org.openstack:volume-8b9b3491-f083-4485-8374-258372f3db35', target_lun='0', target_portal='192.168.122.99:3260', volume_id='8b9b3491-f083-4485-8374-258372f3db35' |
+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
Refresh a volume attachment with nova-manage¶
New in version 24.0.0: (Xena)
Admins may also refresh an existing volume attachment using the following nova-manage commands.
Note
Users can also refresh volume attachments by shelving and later unshelving their instances. The following is an alternative to that workflow and useful for admins when having to mass refresh attachments across an environment.
Note
Future work will look into introducing an os-refresh admin API that will include orchestrating the shutdown of an instance and refreshing volume attachments among other things.
To begin the admin can use the volume_attachment show subcommand to dump
existing details of the attachment directly from the Nova database. This
includes the stashed connection_info not shared by the API.
$ nova-manage volume_attachment show 216f9481-4c9d-4530-b865-51cedfa4b8e7 8b9b3491-f083-4485-8374-258372f3db35 --json | jq .attachment_id
"d338fb38-cfd5-461f-8753-145dcbdb6c78"
If the stored connection_info or attachment_id are incorrect then the
admin may want to refresh the attachment to the compute host entirely by
recreating the Cinder volume attachment record(s) and pulling down fresh
connection_info. To do this we first need to ensure the instance is stopped:
$ openstack server stop 216f9481-4c9d-4530-b865-51cedfa4b8e7
Once stopped the host connector of the compute hosting the instance has to be
fetched using the volume_attachment get_connector subcommand:
root@compute $ nova-manage volume_attachment get_connector --json > connector.json
Note
Future work will remove this requirement and incorporate the gathering of the host connector into the main refresh command. Unfortunately until then it must remain a separate manual step.
We can then provide this connector to the volume_attachment refresh
subcommand. This command will connect to the compute, disconnect any host
volume connections, delete the existing Cinder volume attachment,
recreate the volume attachment and finally update Nova’s database.
$ nova-manage volume_attachment refresh 216f9481-4c9d-4530-b865-51cedfa4b8e7 8b9b3491-f083-4485-8374-258372f3db35 connector.json
The Cinder volume attachment and connection_info stored in the Nova database should now be updated:
$ nova-manage volume_attachment show 216f9481-4c9d-4530-b865-51cedfa4b8e7 8b9b3491-f083-4485-8374-258372f3db35 --json | jq .attachment_id
"9ce46f49-5cfc-4c6c-b2f0-0287540d3246"
The instance can then be restarted and the event list checked
$ openstack server start $instance
