Moving disks from Xen to KVM

Sunday, 8. 04. 2012  –  Category: stash

Overview

  • Moving virtual machines from Xen to KVM
  • The storage for both is under LVM
  • Under Xen the storage for each VM isn’t partitioned as a disk, it’s just a filesystem. GRUB under KVM will need to see a boot sector and a partition table.
  • Under Xen the storage for each VM doesn’t contain a kernel. GRUB under KVM will need one to boot the VM.

Sequence

  • Measure disks on Xen server, create on KVM server. Create the new root disk 10MB larger than its counterpart.1
    
    kyero@xen03 ~ $ sudo lvscan
      ACTIVE            '/dev/vg4/mail-root' [20.00 GB] inherit
      ACTIVE            '/dev/vg4/mail-swap' [1.00 GB] inherit
    

    
    [kyero@vm02 ~]$ sudo lvcreate -L20490M -n mail-root vg4 
    [kyero@vm02 ~]$ sudo lvcreate -L2G -n mail-swap vg4 
    
  • Create a boot disk on the KVM server. We make it 4 cylinders long, roughly 32MB.2
    
    [kyero@vm02 ~]$ sudo dd if=/dev/zero of=/var/boot-images/mail-boot bs=32901120 count=1
    
  • Partition the boot disk with one partition starting at cylinder zero
    
    [kyero@vm02 ~]$ sudo sfdisk /var/boot-images/mail-boot
    last_lba(): I don't know how to handle files with mode 81a4
    Warning: /var/boot-images/mail-boot is not a block device
    Disk /var/boot-images/mail-boot: cannot get geometry
    
    Disk /var/boot-images/mail-boot: 4 cylinders, 255 heads, 63 sectors/track
    
    sfdisk: ERROR: sector 0 does not have an msdos signature
     /var/boot-images/mail-boot: unrecognized partition table type
    Old situation:
    No partitions found
    Input in the following format; absent fields get a default value.
    <start> <size> <type [E,S,L,X,hex]> <bootable [-,*]> <c,h,s> <c,h,s>\nUsually you only need to specify <start> and <size> (and perhaps <type>).
    
    /var/boot-images/mail-boot1 :0
    /var/boot-images/mail-boot1          0+      3       4-     32129+  83  Linux
    /var/boot-images/mail-boot2 :
    /var/boot-images/mail-boot2          0       -       0          0    0  Empty
    /var/boot-images/mail-boot3 :
    /var/boot-images/mail-boot3          0       -       0          0    0  Empty
    /var/boot-images/mail-boot4 :
    /var/boot-images/mail-boot4          0       -       0          0    0  Empty
    New situation:
    Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
    
       Device Boot Start     End   #cyls    #blocks   Id  System
    /var/boot-images/mail-boot1          0+      3       4-     32129+  83  Linux
    /var/boot-images/mail-boot2          0       -       0          0    0  Empty
    /var/boot-images/mail-boot3          0       -       0          0    0  Empty
    /var/boot-images/mail-boot4          0       -       0          0    0  Empty
    Warning: no primary partition is marked bootable (active)
    This does not matter for LILO, but the DOS MBR will not boot this disk.
    Do you want to write this to disk? [ynq] y
    Successfully wrote the new partition table
    
    Re-reading the partition table ...
    BLKRRPART: Inappropriate ioctl for device
    
    If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
    to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
    (See fdisk(8).)
    
    
  • Format and mount the partition
    
    [kyero@vm02 ~]$ sudo kpartx -av /var/boot-images/mail-boot 
    add map loop1p1 : 0 64259 linear /dev/loop1 1
    
    [kyero@vm02 ~]$ sudo mke2fs /dev/mapper/loop1p1 
    mke2fs 1.39 (29-May-2006)
    Filesystem label=
    OS type: Linux
    Block size=1024 (log=0)
    Fragment size=1024 (log=0)
    8032 inodes, 32128 blocks
    1606 blocks (5.00%) reserved for the super user
    First data block=1
    Maximum filesystem blocks=33030144
    4 block groups
    8192 blocks per group, 8192 fragments per group
    2008 inodes per group
    Superblock backups stored on blocks: 
            8193, 24577
    
    Writing inode tables: done                            
    Writing superblocks and filesystem accounting information: done
    
    This filesystem will be automatically checked every 34 mounts or
    180 days, whichever comes first.  Use tune2fs -c or -i to override.
    
    [kyero@vm02 ~]$ sudo mount /dev/mapper/loop1p1 /mnt/boot/
    
    [kyero@vm02 ~]$ df -h /mnt/boot
    Filesystem            Size  Used Avail Use% Mounted on
    /dev/mapper/loop1p1    31M  389K   29M   2% /mnt/boot
    
  • Copy a kernel to it, configure GRUB’s notion of the root device and install the boot sector.
    
    [kyero@vm02 ~]$ sudo rsync -a /boot/ /mnt/boot/
    [kyero@vm02 ~]$ sudo vi /mnt/boot/grub/menu.lst
    [kyero@vm02 ~]$ sudo touch /mnt/boot/kvm-boot-image
    [kyero@vm02 ~]$ sudo grub --device-map=/dev/null
    
        GNU GRUB  version 0.97  (640K lower / 3072K upper memory)
    
     [ Minimal BASH-like line editing is supported.  For the first word, TAB
       lists possible command completions.  Anywhere else TAB lists the possible
       completions of a device/filename.]
    grub> device (hd0) /var/boot-images/mail-boot
    device (hd0) /var/boot-images/mail-boot
    grub> find /kvm-boot-image
    find /kvm-boot-image
     (hd0,0)
    grub> root (hd0,0)
    root (hd0,0)
     Filesystem type is ext2fs, partition type 0x83
    grub> setup (hd0)
    setup (hd0)
     Checking if "/boot/grub/stage1" exists... no
     Checking if "/grub/stage1" exists... yes
     Checking if "/grub/stage2" exists... yes
     Checking if "/grub/e2fs_stage1_5" exists... yes
     Running "embed /grub/e2fs_stage1_5 (hd0)"... failed (this is not fatal)
     Running "embed /grub/e2fs_stage1_5 (hd0,0)"... failed (this is not fatal)
    Done.
    grub> quit
    
    [kyero@vm02 ~]$ sudo umount /mnt/boot
    [kyero@vm02 ~]$ sudo kpartx -d /var/boot-images/mail-boot 
    loop deleted : /dev/loop1
    
    
  • Populate the new root disk with one cylinder of zeros and a copy of the old root disk.
    The first cylinder will contain the MBR and not much else. The padding means it’s easy to align the old root disk at cylinder one.
    Use pv so we can see the copy progress.
    The dd-lvm-out script is mostly just a sudo-blessed wrapper.
    [kyero@vm02 ~]$ ( dd if=/dev/zero bs=8225280 count=1 ; ssh xen03.local -l kyero sudo /data/system/bin/dd-lvm-out.sh mail-root ) | pv -e -r -s 21474836480 | sudo dd bs=1048577 of=/dev/vg4/mail-root
    1+0 records in
    1+0 records out
    8225280 bytes (8.2 MB) copied, 0.03395 seconds, 242 MB/s
    20480+0 records in
    20480+0 records out
    21474836480 bytes (21 GB) copied, 611.215 s, 35.1 MB/s
    0+662630 records in
    0+662630 records out
    21483061795 bytes (21 GB) copied, 615.249 seconds, 34.9 MB/s
    
  • Partition the new root disk with a single partition starting at cylinder one.
    [kyero@vm02 ~]$ sudo sfdisk /dev/vg4/mail-root
    Checking that no-one is using this disk right now ...
    BLKRRPART: Invalid argument
    OK
    
    Disk /dev/vg4/mail-root: 2614 cylinders, 255 heads, 63 sectors/track
    
    sfdisk: ERROR: sector 0 does not have an msdos signature
     /dev/vg4/mail-root: unrecognized partition table type
    Old situation:
    No partitions found
    Input in the following format; absent fields get a default value.
    <start> <size> <type [E,S,L,X,hex]> <bootable [-,*]> <c,h,s> <c,h,s>\nUsually you only need to specify <start> and <size> (and perhaps <type>).
    
    /dev/vg4/mail-root1 :1
    /dev/vg4/mail-root1          1    2613    2613   20988922+  83  Linux
    /dev/vg4/mail-root2 :
    /dev/vg4/mail-root2          0+      0       1-      8032   83  Linux
    /dev/vg4/mail-root3 :
    /dev/vg4/mail-root3          0       -       0          0    0  Empty
    /dev/vg4/mail-root4 :
    /dev/vg4/mail-root4          0       -       0          0    0  Empty
    New situation:
    Units = cylinders of 8225280 bytes, blocks of 1024 bytes, counting from 0
    
       Device Boot Start     End   #cyls    #blocks   Id  System
    /dev/vg4/mail-root1          1    2613    2613   20988922+  83  Linux
    /dev/vg4/mail-root2          0+      0       1-      8032   83  Linux
    /dev/vg4/mail-root3          0       -       0          0    0  Empty
    /dev/vg4/mail-root4          0       -       0          0    0  Empty
    Warning: no primary partition is marked bootable (active)
    This does not matter for LILO, but the DOS MBR will not boot this disk.
    Do you want to write this to disk? [ynq] y
    Successfully wrote the new partition table
    
    Re-reading the partition table ...
    BLKRRPART: Invalid argument
    
    If you created or changed a DOS partition, /dev/foo7, say, then use dd(1)
    to zero the first 512 bytes:  dd if=/dev/zero of=/dev/foo7 bs=512 count=1
    (See fdisk(8).)
    
    
  • Mount the new root filesystem
    [kyero@vm02 ~]$ sudo kpartx -av /dev/vg4/mail-root
    add map mail-root1 : 0 41977845 linear /dev/vg4/mail-root 16065
    add map mail-root2 : 0 16064 linear /dev/vg4/mail-root 1
    [kyero@vm02 ~]$ sudo e2fsck /dev/mapper/mail-root1
    e2fsck 1.39 (29-May-2006)
    /dev/mapper/mail-root1: clean, 435213/1310720 files, 4086961/5242880 blocks
    
    [kyero@vm02 ~]$ sudo mount /dev/mapper/mail-root1 /mnt/tmp
    
  • Add the new kernel’s modules, tweak filesystem mounts, disable swap, check console.
    
    [kyero@vm02 ~]$ sudo rsync -a /lib/modules/ /mnt/tmp/lib/modules/
    
    [kyero@vm02 ~]$ sudo vi /mnt/tmp/etc/fstab
    
    [kyero@vm02 ~]$ sudo vi /mnt/tmp/etc/inittab 
    
  • Create the KVM guest, assigning the boot, root and swap storage
    [kyero@vm02 ~]$ sudo virt-install -n mail -r 1024 --vcpus 1 --os-type linux --os-variant generic26 --accelerate --disk path=/var/boot-images/mail-boot --disk path=/dev/mapper/vg4-mail--root,bus=ide,format=raw --disk path=/dev/mapper/vg4-mail--swap,bus=ide,format=raw --network bridge:br0 --network bridge:br1 --import --noreboot
    Starting install...
    Guest installation complete... you can restart your domain
    by running 'virsh start mail'
    
    [kyero@vm02 ~]$ sudo virsh dumpxml mail
    ...
        <disk type='file' device='disk'>
          <driver name='qemu' type='raw' cache='none'/>
          <source file='/var/boot-images/mail-boot'/>
          <target dev='hda' bus='ide'/>
          <address type='drive' controller='0' bus='0' unit='0'/>
        </disk>
        <disk type='block' device='disk'>
          <driver name='qemu' type='raw' cache='none'/>
          <source dev='/dev/mapper/vg4-mail--root'/>
          <target dev='hdb' bus='ide'/>
          <address type='drive' controller='0' bus='0' unit='1'/>
        </disk>
        <disk type='block' device='disk'>
          <driver name='qemu' type='raw' cache='none'/>
          <source dev='/dev/mapper/vg4-mail--swap'/>
          <target dev='hdd' bus='ide'/>
          <address type='drive' controller='0' bus='1' unit='1'/>
        </disk>
    ...
    
  • Boot the KVM guest
    
    [kyero@vm02 ~]$ sudo virsh start mail
    [kyero@vm02 ~]$ sudo virsh autostart mail
    [kyero@vm02 ~]$ sudo virsh console mail
    
  • Log into KVM guest, partition /dev/hdd and configure it as swap, update /etc/fstab.
  1. Later we align the original root filesystem at cylinder 1 on this new disk. A LVM cylinder is roughly 8MB []
  2. File backed disk is used rather than LVM owing to grief with GRUB and LVM (Error 22: No such partition) []

Comments are closed.