Tuesday, August 04, 2009

Dual Boot Citrix Xenserver 5

I'm very proud of this hack. So proud that it's actually a quad boot, but two of the four are Windows, which we don't care about, so we'll stick with the title of dual boot. Here's the scenario:

A couple weeks ago, I did the VCP-310 cert, with the anticipation of rolling that into the VCP-410 next month. Until then, I'm going to do the Citrix CCA cert. This means we need a Citrix Xenserver... Unfortunately, the only machine I have that is not doing something is my oldest kid's desktop. Now, this is not as bad as it sounds, as he's off doing a summer session at some university somewhere, or something.

Since he's gone, we'll wipe the drive and load Citrix. Not so fast! He's got Windows on that thing and uses it as a game station when he's in town. I'd already partitioned it and loaded it with boot to Fedora as well as Windows, so Citrix should be a non-issue. Except, Citrix Xenserver thinks it should be the only thing on the system... Just like ESX.

So, here we go:
1. Disconnect the hard drive and load a second hard drive, pinned out as master.
2. Load Citrix Xenserver in the default configuration. Reboot and test.
3. Disconnect second hard drive and pin out as slave. Cable it and the original drive to the primary controller.
4. Boot to BIOS and ensure both are visible. This ensures that both are pinned correctly.
5. Boot off the previously existing Linux partition to Single. Use fdisk -l to validate that the second drive is visible.

(This is where it starts to get hairy.)

6. The master appears as /dev/sda and the secondary as /dev/sdb. Mount the second drive:
mount /dev/sdb1 /mnt
ls -l /mnt
Notice anything unusual? No boot partition. Citrix is loading the MBR with a LILO like loader.
7. Inspect /boot:
chain.c32
config-2.6.18-128.1.6.el5.xs5.5.0.496.1012kdump
config-2.6.18-128.1.6.el5.xs5.5.0.496.1012xen
extlinux.conf
extlinux.sys
initrd-2.6.18-128.1.6.el5.xs5.5.0.496.1012kdump.img
initrd-2.6.18-128.1.6.el5.xs5.5.0.496.1012xen.img
initrd-2.6-xen.img
mboot.c32
menu.c32
System.map-2.6.18-128.1.6.el5.xs5.5.0.496.1012kdump
System.map-2.6.18-128.1.6.el5.xs5.5.0.496.1012xen
vmlinuz-2.6.18-128.1.6.el5.xs5.5.0.496.1012kdump
vmlinuz-2.6.18-128.1.6.el5.xs5.5.0.496.1012xen
vmlinuz-2.6-xen
vmlinuz-kdump
xen-3.3.1.gz
xen-3.3.1.map
xen-3.3.gz
xen-3.gz
xen.gz
Some of this looks familiar. We just need to arrange it into our existing grub.conf.
8. I looked at an old FC6 Xen platform's grub.conf:
title Fedora Core (2.6.20-1.3002.fc6xen)
    root (hd0,0)
    kernel /xen.gz-2.6.20-1.3002.fc6
    module /vmlinuz-2.6.20-1.3002.fc6xen ro root=/dev/hda3
    module /initrd-2.6.20-1.3002.fc6xen.img
(Why in the world am I still running Fedora Core 6? Because, it is most similar to RHEL5.)
9. A closer look at /boot will confirm that many of those files are actually symlinks. So lets add the following to grub.conf:
title Citrix Xenserver 5.5.0
    root (hd1,0)
    kernel /xen-3.3.1.gz
    module /vmlinuz-2.6-xen ro root=/dev/sdb1
    module /initrd-2.6-xen.img
Notice the root line-- hd1,0 not hd0,0-- we're on the second drive. Notice the vmlinuz line-- we are using /dev/sdb1-- that's what we mounted.
10. Reboot... and watch it fail!

I'll save you the agony of troubleshooting. First, in the Fedora example, boot is partition 1, so we prefix the files with a slash (/). Citrix did not use a boot partition: root is partition 1. Therefore, we need to prefix the files with the directory: /boot/

Second, we mounted /dev/sdb1, which was the Xenserver's root partition. Yet, when Xenserver boots, it sees the drive as /dev/hdb1. This is because I happen to have Fedora 8 loaded, and effective F8, all drives are sd, even though the drives are actually IDE rather than SCSI. Since Xenserver is based on el5 (look at the vmlinuz links) it sees the drive as hd.

Here's what actually worked:
title Citrix Xenserver 5.5.0
    root (hd1,0)
    kernel /boot/xen-3.3.1.gz
    module /boot/vmlinuz-2.6-xen ro root=/dev/hdb1
    module /boot/initrd-2.6-xen.img

10 comments:

  1. So in order to get this to work with Windows, you had to dual boot Fedora with Windows First? I'm going to try the same thing, but I want to dual boot Server 2008 Hyper-V and XEN. No fedora. Any suggestions?
    http://www.thegenerationv.com

    ReplyDelete
  2. What you need is a boot loader. My box has Fedora so it can also run VMware Server 2.0, and as such, uses GRUB. Any boot loader should do, and will most likely use the same information. You might also investigate whether your BIOS would support booting from multiple hard drives.

    ReplyDelete
  3. Very good Article. Very simple. Has Vista on Drive1, installed Citrix to Drive2. Installed EasyBCD on the Vista OS. Installed the NeoGrub bootloader through EasyBCD and added the config entry above for the Citrix. i.e :

    title Citrix Xenserver 5.5.0
    root (hd1,0)
    kernel /boot/xen-3.3.1.gz
    module /boot/vmlinuz-2.6-xen ro root=/dev/hdb1
    module /boot/initrd-2.6-xen.img

    Rebooted and worked first time.

    If it automatically reboots during the Citrix load the line :

    module /boot/vmlinuz-2.6-xen ro root=/dev/hdb1

    may be wrong. I had to change it to :

    module /boot/vmlinuz-2.6-xen ro root=/dev/sdb1

    Thanks !

    ReplyDelete
  4. how to make this work in Grub2 ? I tryeed that in Ubuntu 9.10 and made basic configuration changes but i just get a message that kernel should be loaded first ..

    ReplyDelete
  5. You've got a syntax error. Notice in the article, I show two Grub stanzas. The first is wrong the second is right. Your vmlinuz and initrd lines needed to include /boot.

    ReplyDelete
  6. Yep.. probably.. but i do have that boot there.. wonder what else might go wrong.. ?

    changing the order of xen and vmlinuz.. does not make any difference..

    Was thinking if it does not find that drive for some reason but when i did update-grub it did find xenserver from that drive and added it into grub config.. of course it does not work either... : /

    menuentry "XenServer (xenenterprise) (on /dev/sda1)" {
    insmod ext2
    set root=(hd0,1)
    search --no-floppy --fs-uuid --set 9e15bb36-a0ab-4435-820a-e554341a962f
    linux /boot/xen-3.3.1.gz
    linux /boot/vmlinuz-2.6.18-128.1.6.el5.xs5.5.0.496.1012xen root=/dev/sda1 ro
    initrd /boot/initrd-2.6-xen.img
    }

    ReplyDelete
  7. one addition.. if i choose grub command line from the grub menu and then type exit it will boot to XEN... hmmn : ) ?

    ReplyDelete
  8. Doug,

    Thanks for taking the time to create this as it's concise information that really does help people out there (including myself)!

    Cheers,

    John

    ReplyDelete
  9. Hi - I am trying to Dual Boot Win7 Pro x64 with XenServer 5.6.0 It all seems to work fine using NeoGrub. I had to boot to the XenServer console and check the directory of the /boot folder as the names of the xen-3.3.1.gz had changed to xen-3.4.2.gz. My box seems to boot XenServer but then almost stops immediately with the error of "(XEN) Not enough memory to relocate the dom0 kernel image"
     
    Do you have any ideas how i can resolve this? My System has 8GB of RAM so i can't think that is not enough.
     
    When i boot straight to the disk XenServer is on from BIOS XenServer boots fine.
     
    It's only a problem after the Windows Boot Loader has appeared then Selecting NeoGrub then into XenServer.
    Thanks
    Mark

    ReplyDelete
  10. Thanks! It worked like a charm on my Machine (Intel P4 2.8 GHz, 4 GB Ram, 1 TB + 500 GB HDD) running Windows 7 and XenServer 5.6.1 SP2. I am using EasyBCD to dual boot.

    :)

    ReplyDelete