Nova-volume is the service that allows you to give extra block level storage to your OpenStack Compute instances. You may recognize this as a similar offering from Amazon EC2 known as Elastic Block Storage (EBS). However, nova-volume is not the same implementation that EC2 uses today. Nova-volume is an iSCSI solution that employs the use of Logical Volume Manager (LVM) for Linux. Note that a volume may only be attached to one instance at a time. This is not a ‘shared storage’ solution like a SAN of NFS on which multiple servers can attach to.

Before going any further; let's discuss the nova-volume implementation in OpenStack:

The nova-volumes service uses iSCSI-exposed LVM volumes to the compute nodes which run instances. Thus, there are two components involved:

  1. lvm2, which works with a VG called "nova-volumes" (Refer to http://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux) for further details)

  2. open-iscsi, the iSCSI implementation which manages iSCSI sessions on the compute nodes

Here is what happens from the volume creation to its attachment (we use euca2ools for examples, but the same explanation goes with the API):

  1. The volume is created via $euca-create-volume; which creates an LV into the volume group (VG) "nova-volumes"

  2. The volume is attached to an instance via $euca-attach-volume; which creates a unique iSCSI IQN that will be exposed to the compute node

  3. The compute node which run the concerned instance has now an active ISCSI session; and a new local storage (usually a /dev/sdX disk)

  4. libvirt uses that local storage as a storage for the instance; the instance get a new disk (usually a /dev/vdX disk)

For this particular walkthrough, there is one cloud controller running nova-api, nova-scheduler, nova-objectstore, nova-network and nova-volume services. There are two additional compute nodes running nova-compute. The walkthrough uses a custom partitioning scheme that carves out 60GB of space and labels it as LVM. The network is a /28 .80-.95, and FlatManger is the NetworkManager setting for OpenStack Compute (Nova).

Please note that the network mode doesn't interfere at all with the way nova-volume works, but networking must be set up for for nova-volumes to work. Please refer to Chapter 7, Networking for more details.

To set up Compute to use volumes, ensure that nova-volume is installed along with lvm2. The guide will be split in four parts :

  • A- Installing the nova-volume service on the cloud controller.

  • B- Configuring the "nova-volumes" volume group on the compute nodes.

  • C- Troubleshooting your nova-volume installation.

  • D- Backup your nova volumes.

This is simply done by installing the two components on the cloud controller :

apt-get install lvm2 nova-volume

sudo visudo

Then add an entry for the nova user (here is the default sudoers file with our added nova user) :

# /etc/sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#
# See the man page for details on how to write a sudoers file.
#

Defaults	env_reset

# Host alias specification

# User alias specification

# Cmnd alias specification
nova ALL = (root) NOPASSWD: /bin/dd

# User privilege specification
root	ALL=(ALL) ALL

# Allow members of group sudo to execute any command
# (Note that later entries override this, so you might need to move
# it further down)
%sudo ALL=(ALL) ALL
#
#includedir /etc/sudoers.d

# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL

            

That will allow the nova user to run the "dd" command (which empties a volume before its deletion).

Since you have created the volume group, you will be able to use the following tools for managing your volumes:

euca-create-volume

euca-attach-volume

euca-detach-volume

euca-delete-volume

[Note]Note

If you are using KVM as your hypervisor, then the actual device name in the guest will be different than the one specified in the euca-attach-volume command. You can specify a device name to the KVM hypervisor, but the actual means of attaching to the guest is over a virtual PCI bus. When the guest sees a new device on the PCI bus, it picks the next available name (which in most cases is /dev/vdc) and the disk shows up there on the guest.

If the volume attachment doesn't work, you should be able to perform different checks in order to see where the issue is. The nova-volume.log and nova-compute.log will help you to diagnosis the errors you could encounter :

nova-compute.log / nova-volume.log

While Diablo provides the snapshot functionality (using LVM snapshot), were are going to see here how you can backup your EBS-volumes. The way we will do it offers the advantage to make backup that don't size much, in fact, only existing data will be backed up, not the whole volume. So let's suppose we create a 100 gb nova-volume for an instance, while only 4 gigabytes are used ; we will only backup these 4 giga-bytes, here are the tools we are going to use in order to achieve that :

1- Create a snapshot of a used volume

2- Partition table discovery

3- Use tar in order to create archives

4- Checksum calculation I

5- After work cleaning

6- Automate your backups

You will mainly have more and more volumes allocated to your nova-volume service. It might be interesting then to automate things a bit. This script here will assist you on this task. The script does the operations we just did earlier, but also provides mail report and backup running (based on the " backups_retention_days " setting). It is meant to be launched from the server which runs the nova-volumes component.

Here is how a mail report looks like :

Backup Start Time - 07/10 at 01:00:01
Current retention - 7 days 

The backup volume is mounted. Proceed...
Removing old backups...  : /BACKUPS/EBS-VOL/volume-00000019/volume-00000019_28_09_2011.tar.gz
	 /BACKUPS/EBS-VOL/volume-00000019 - 0 h 1 m and 21 seconds. Size - 3,5G 

The backup volume is mounted. Proceed...
Removing old backups...  : /BACKUPS/EBS-VOL/volume-0000001a/volume-0000001a_28_09_2011.tar.gz
	 /BACKUPS/EBS-VOL/volume-0000001a - 0 h 4 m and 15 seconds. Size - 6,9G 
---------------------------------------
Total backups size - 267G - Used space : 35%
Total execution time - 1 h 75 m and 35 seconds
            

The script also provides the ability to SSH to your instances and run a mysqldump into them. In order to make this to work, make sure the connection via the nova's project keys is possible. If you don't want to run the mysqldumps, then just turn off this functionality by putting enable_mysql_dump=0 into the script (see all settings at the top of the script)



loading table of contents...