Jagane's Cloud Computing Blog

As a followup to my previous post, I set out to accomplish a (seemingly) simple task: Install CentOS 5.4 with KVM Virtualization on a system and then create a CentOS 5.4 KVM VM with virtio Net and Disk drivers.

It turns out that there is more to this task than meets the eye. So, here’s my step by step procedure.

Step 1: Install Centos (Redhat 5.4) with KVM Virtualization on a Intel VT or AMD Pacifica enabled server (I used a Intel Core 2 Duo E6420/2GB/120GB SATA system)

• Install CentOS 5.4 64 bit with the “Virtualization” option
• While installing, choose the “Customize now” instead of customize later, and select KVM instead of ‘Virtualization’ in the Virtualization customization screen.
• For this install, I chose to disable SELinux. I’m sure its useful in some security contexts, but for most of my use – it is just a source of endless problems. Someday, I might actually spend the time to learn how SELinux works. Right now, it feels to me like the Windowsification of Linux. Moving on…
• When the newly installed system boots up, you need to create a bridge(software switch) called br0, move the IP address of eth0 to br0, and then make eth0 an uplink to the bridge br0. Here’s how to do it:
  • Create a file /etc/sysconfig/network-scripts/ifcfg-br0 with the following contents:
    • DEVICE=br0
      TYPE=Bridge
      BOOTPROTO=static
      IPADDR=192.168.1.213
      NETMASK=255.255.255.0
      GATEWAY=192.168.1.10
      ONBOOT=yes
  • Edit /etc/sysconfig/network-scripts/ifcfg-eth0 and replace its contents with:
    • DEVICE=eth0
      ONBOOT=yes
      BRIDGE=br0
      NM_CONTROLLED=no
  • Reboot your system. Note that this configuration is for a static IP server.
  • Add the following lines to /etc/sysconfig/iptables to allow relevant traffic:
    • -A RH-Firewall-1-INPUT -i br0 -j ACCEPT
    • -A RH-Firewall-1-INPUT -m state –state NEW -m tcp -p tcp –dport 5900:6900 -j ACCEPT

Step 2: Create a CentOS 5.4 KVM VM using the CentOS boot CD

Well, this task got complicated quickly. My intent was to make this VM connect to a bridged network interface, so that I could benchmark it by running nuttcp to another physical machine. CentOS 5.4 (Redhat 5.4) does not come with scripts for a bridged VM network out of the box. This is why we needed to create the br0 bridge in the previous step.

• In order to use the br0 bridge effectively, we need a utility called tunctl (I have a precompiled version here – http://www.thinsy.com/utils/tunctl.gz ). Please this in /usr/sbin on your new CentOS box.
• It turns out that creating a VM by calling qemu directly involves a lot of options. I ended up building a script for this purpose. You can download it here: http://www.thinsy.com/utils/start_a_kvm.sh.gz. Place start_a_kvm.sh in /usr/sbin.
• Create a directory for our VM called /vms/1
• Create two 8GB files vdisk0 and vdisk1 in this directory using the following commands:
  • dd if=/dev/zero of=./vdisk0 count=1 bs=1 seek=8589934591
  • dd if=/dev/zero of=./vdisk1 count=1 bs=1 seek=8589934591
• Create a file called vm.params with the following contents ( a sample is available at http://www.thinsy.com/utils/vm.params):
  • MAC=52:54:00:00:00:01
    DISK0=/vms/1/vdisk0
    DISK1=/vms/1/vdisk1
    MEMORY=512
    VNCDISP=0
• For booting and installing the VM from the CentOS 5.4 CD image, run the following command:
  • /usr/sbin/start_a_kvm.sh /vms/1/vm.params /tmp/CentOS-5.4-i386-bin-DVD.iso boot_from_cd
• This will cause a tap interface called tap0 to be created, connected to the bridge br0, and the VM created by calling kvm-qemu directly
• The start_a_kvm.sh script sets up the VM to publish a graphical console using the VNC protocol at TCP port 5900 + $VNCDISP, where VNCDISP is set in the vm.params file. Use your favorite vncviewer to connect to this graphical console.
• When the VM is started up, you will get the graphical console of the VM. Now go through the process of installing the OS on your newly created VM

Step 3: First boot of the CentOS in your newly created VM

After the OS installation is completed, you can reboot the VM from the virtual hard drive, without the CDROM image attached. Here is the command to do that:

/usr/sbin/start_a_kvm.sh /vms/1/vm.params

There you have it – a KVM VM with paravirtualized drivers (virtio) for network and disk.

All this without the use of libvirt or virt-manager or one of the myriad programs that did not quite work for me.

Step 4: Fixup VNC mouse tracking

One of the most annoying things about the qemu vnc server is the fact that the mouse works like cr**. Here’s a simple fix for that problem. Download the following xorg.conf file and place it in your newly created VM’s /etc/X11 directory. This configures a VNC screen of size 1024×768 with a mouse that actually works – http://www.thinsy.com/utils/xorg.conf.gz

— soapbox mode on —

Truth be told, if you peel away the layers of crud such as libvirt and virt-manager, the KVM architecture is quite elegant. I am convinced that KVM, and not Xen, is the future of Linux virtualization. My preliminary tests indicate that the KVM performance is not as good as Xen, but I fully expect that to be resolved quickly. Besides, if performance is of great import to you, you should be looking at ‘Containers’ technology such as openvz.

— soapbox mode off —