Migration enables an administrator to move a virtual machine instance from one compute host to another. A typical scenario is planned maintenance on the source host, but migration can also be useful to redistribute the load when many VM instances are running on a specific physical machine.
Not all Compute service hypervisor drivers support live-migration, or support all live-migration features.
Consult the Hypervisor Support Matrix to determine which hypervisors support live-migration.
See the Hypervisor configuration pages for details on hypervisor-specific configuration settings.
The migration types are:
Non-live migration, also known as cold migration or simply migration.
The instance is shut down, then moved to another hypervisor and restarted. The instance recognizes that it was rebooted, and the application running on the instance is disrupted.
This section does not cover cold migration.
The instance keeps running throughout the migration. This is useful when it is not possible or desirable to stop the application running on the instance.
Live migrations can be classified further by the way they treat instance storage:
Block live migration requires copying disks from the source to the destination host. It takes more time and puts more load on the network. Shared-storage and volume-backed live migration does not copy disks.
In a multi-cell cloud, instances can be live migrated to a different host in the same cell, but not across cells.
The following sections describe how to configure your hosts for live migrations using the KVM and XenServer hypervisors.
To enable any type of live migration, configure the compute hosts according to the instructions below:
Set the following parameters in
nova.conf on all compute hosts:
You must not make the VNC server listen to the IP address of its compute host, since that addresses changes when the instance is migrated.
Since this setting allows VNC clients from any IP address to connect to instance consoles, you must take additional measures like secure networks or firewalls to prevent potential attackers from gaining access to instances.
instances_path must have the same value for all compute hosts.
In this guide, the value
/var/lib/nova/instances is assumed.
Ensure that name resolution on all compute hosts is identical, so that they can connect each other through their hostnames.
If you use
/etc/hosts for name resolution and enable SELinux,
/etc/hosts has the correct SELinux context:
# restorecon /etc/hosts
Enable password-less SSH so that
root on one compute host can log on to any other compute host
without providing a password.
libvirtd daemon, which runs as root,
uses the SSH protocol to copy the instance to the destination
and can’t know the passwords of all compute hosts.
You may, for example, compile root’s public SSH keys on all compute hosts
authorized_keys file and deploy that file to the compute hosts.
Configure the firewalls to allow libvirt to communicate between compute hosts.
By default, libvirt uses the TCP port range from 49152 to 49261 for copying memory and disk contents. Compute hosts must accept connections in this range.
For information about ports used by libvirt, see the libvirt documentation.
Be mindful of the security risks introduced by opening ports.
No additional configuration is required for block migration and volume-backed live migration.
Be aware that block migration adds load to the network and storage subsystems.
Live migration copies the instance’s memory from the source to the destination compute host. After a memory page has been copied, the instance may write to it again, so that it has to be copied again. Instances that frequently write to different memory pages can overwhelm the memory copy process and prevent the live migration from completing.
This section covers configuration settings that can help live migration of memory-intensive instances succeed.
Live migration completion timeout
The Compute service aborts a migration when it has been running for too long. The timeout is calculated based on the instance size, which is the instance’s memory size in GiB. In the case of block migration, the size of ephemeral storage in GiB is added.
The timeout in seconds is the instance size multiplied by the
live_migration_completion_timeout, whose default is 800. For
shared-storage live migration of an instance with 8GiB memory will
time out after 6400 seconds.
Live migration progress timeout
The Compute service also aborts a live migration when it detects that
memory copy is not making progress for a certain time. You can set
this time, in seconds,
through the configurable parameter
the default value of
live_migration_progress_timeout is 0,
which disables progress timeouts. You should not change
this value, since the algorithm that detects memory copy progress
has been determined to be unreliable. It may be re-enabled in
Near the end of the memory copy, the instance is paused for a short time so that the remaining few pages can be copied without interference from instance memory writes. The Compute service initializes this time to a small value that depends on the instance size, typically around 50 milliseconds. When it notices that the memory copy does not make sufficient progress, it increases the time gradually.
You can influence the instance downtime algorithm with the help of three configuration variables on the compute hosts:
live_migration_downtime = 500 live_migration_downtime_steps = 10 live_migration_downtime_delay = 75
live_migration_downtime sets the maximum permitted
downtime for a live migration, in milliseconds.
The default is 500.
live_migration_downtime_steps sets the total number of
adjustment steps until
live_migration_downtime is reached.
The default is 10 steps.
sets the time interval between two
adjustment steps in seconds. The default is 75.
One strategy for a successful live migration of a memory-intensive instance is slowing the instance down. This is called auto-convergence. Both libvirt and QEMU implement this feature by automatically throttling the instance’s CPU when memory copy delays are detected.
Auto-convergence is disabled by default.
You can enable it by setting
Before enabling auto-convergence, make sure that the instance’s application tolerates a slow-down.
Be aware that auto-convergence does not guarantee live migration success.
Live migration of a memory-intensive instance is certain to succeed when you enable post-copy. This feature, implemented by libvirt and QEMU, activates the virtual machine on the destination host before all of its memory has been copied. When the virtual machine accesses a page that is missing on the destination host, the resulting page fault is resolved by copying the page from the source host.
Post-copy is disabled by default. You can enable it by setting
When you enable both auto-convergence and post-copy, auto-convergence remains disabled.
The page faults introduced by post-copy can slow the instance down.
When the network connection between source and destination host is interrupted, page faults cannot be resolved anymore and the instance is rebooted.
The full list of live migration configuration parameters is documented in the OpenStack Configuration Reference Guide
Compatible XenServer hypervisors. The hypervisors must support the Storage XenMotion feature. See your XenServer manual to make sure your edition has this feature.
--block-migrateparameter with the live migration command.