The Sysadmin Wiki

To do a live migration for a domU, the back-storage of your virtual machine must be shared storage, like iSCSI , AoE, etc...  Setting up AoE is explained at ata-over-ethernet and iSCSI at configure-iscsi-target-linux.

To do live migration between two host, these hosts must be on the same subnet, layer 2 network.

To make it as easy as possible on each host :

  • host_1 : the host the vm is running on before migration
  • host_2 : the host the vm is running on after migration
  • storage_host : is the host running the storage area, it doesn't known anything about xen or live migration, it only has data.
  • guest :the virtual machine to migrate from host_1 to host_2

Here the xen vm is shared on an iSCSI device. The procedure to do it on a AoE device, will be very similar. In the sample configuration there are three hosts, between host_1 and host_2 we will perform live migration, and the third host runs the iSCSI storage area.

First, be sure to properly install Xen on your system, and set up some options, Install_xen_debian. In this example it is assumed that there is already a vm running on host_1.

So, here are the steps to do live migration.

Enable xen to do a live migration[]

With a default install of xen on a debian Linux, live migration will not work. Some options must first be set in the main configuration file of xen :


Find the following lines :

# (xend-relocation-hosts-allow )
# (xend-relocation-port 8002)
# (xend-relocation-address )
# (xend-relocation-server yes)

They are commented out, with a # at the beginning of each line. Remove the # from all four lines to uncomment them. This enables xen to do a live migration.  Those four lines become :

(xend-relocation-hosts-allow )
(xend-relocation-port 8002)
(xend-relocation-address )
(xend-relocation-server yes)

Restart the xen daemon with the following command:

xend restart

Create an iSCSI device on the network[]

On the storage_host configure an iSCSI disc to store the vm data.  Here Debian Linux is used, client and server side setup are detailed here. This box doesn't need to known anything about xen and live migration, it will only store the data.

Enable iSCSI storage for the first host running the live migration[]

Let's use this iSCSI device from host_1. Follow this link if you still don't known how to attach an iSCSI device on your host.

On the system, let's say that the iSCSI device got the device block :


I first had to configure this new block device on the system, let's use :


and create one primary partition to be used by LVM, so type number of :


Initialise the partition to be used with LVM :

root@host_1 # pvcreate /dev/sda1

then create the volume group :

root@host_1 # vgcreate vg_xen /dev/sda1

create the logical volume needed for the virutal machine, 3GO will be enought for me, and don't forget to create also a swap device :

root@host_1 # lvcreate -L 3G -n lvm-xen vg_xen
root@host_1 # lvcreate -L 1G -n lvm-xen-swap vg_xen

create the file system and enable the swap device :

root@host_1 # mkfs.ext3 /dev/vg_xen/lvm-xen
root@host_1 # mkswap /dev/vg_xen/lvm-xen-swap
root@host_1 # swapon /dev/vg_xen/lvm-xen-swap

Put the vm data on this iSCSI storage area[]

Work related to block device and file system are finished at this point.

Let's now mount the partition where the data of VM guest was on our local disk :

root@host_1 # mount /dev/disk-1/debian-base-xen /mnt/old-vm

Note that is important that the vm is not actively used. So, shutdown the VM if is currently used or do a snapshot if it is used and on a LVM. We will now mount the iSCSI volume :

root@host_1 # mount /dev/vg_xen/lvm-xen /mnt/new-vm

And now, copy all the the data of vm from one place to another place :

root@host_1 # cp -ax /mnt/old-vm/* /mnt/new-vm/

When this is done, you can launch your virtual machine through your local disk or through your iSCSI device. You only have to change the relevant line of your vm configuration file, so related to disk parameter.

Configure the first host[]

You have to edit a line within the xen configuration file of this vm, the line that looks like that one :

disk   =       ['phy:/dev/disk-1/debian-base-xen,sda1,w', 'phy:/dev/disk-1/debian-base-xen-swap,sda2,w']

As the data of your vm are also place now on (we did a copy at the [last step] ) :


and the swap device on :


change the line to this one :

disk   =       ['phy:/dev/vg_xen/lvm-xen,sda1,w', 'phy:/dev/vg_xen/lvm-xen-swap,sda2,w']

Now, you can launch your vm on host_1, there won't be any difference, except that the vm guest data are on a shared storage, but you won't notice anything.

Configure the second host[]

The configuration xen file for host_virtual is :


Let's now put the configuration file done on __host_not_virtual_1__ on the other host that will run the live migration ( host_not_virtual_2 ) :

root@host_1 # scp /etc/xen/debian-base-xen.cfg root@host_2:/etc/xen/

Let's do some work on this other host ( host_2 ). We first had to add/attach the iSCSI device available for him, you can find [here] how you can do it. I suppose that the new iSCSI device on host_2 is accessible through the block device :


At this point host_not_virtual_2 still don't known that :


has a logical volume on it, 2 step needs to be done :

root@host_2 # vgscan


root@host_2 # vgchange -a y

Now, if you do on host_not_virtual_host_2 a :

root@host_2 # vgdisplay vg_xen

you will see the logical volume that we did configure on host_not_virtual_1.

Do the live migration[]

And finally, you can now enjoy doing a live migration. First, find the vm id of the vm that you want to migrate from :




This can be done with :

# xm list

Let's say that the vm id was :


I did this command to migrate the vm :

root@host_1 # xm migrate --live 2 host_2

And after a moment, you won't see anymore this vm on :


but actually, it will be on :


External links[]