The sun has reached just the right angle during my commute to cast this interesting shadow.
Tuesday, September 18, 2007
DC Commute
Thursday, September 13, 2007
Xen: Full Virt Solaris 10, Pt 4
Haha!!!!! I got it! And (unfortunately) it is a bug in Xen.
After significant playing with the video board, I decided that maybe it was the CD-RW. Perhaps, Linux was aquiring the CD-RW as read-write, and the Solaris VM was trying to access the locked device. To test, I pulled the cable on the CD-RW and rebooted. Without the CD-RW, the error returned. Not the the CD-RW, I thought.
After a few more hours, a couple glasses of wine, and a few minutes of falling asleep in my chair, I decided that maybe I could observe the error on the VM if I could get the console to activate at start, rather than post start. I had no luck, but I did notice something I had not seen.
On the GUI Manager, virt-manager, was a Details button. Clicking this allows a user to browse statistics and hardware for a VM. I examined the hardware, and noticed something strange: the shared CD-RW had a source path of /dev. That's not going to work.
This would be easy to fix on RHEL5, because we could edit the config file. On F7, the config is stored in a database that I had not been able to crack. Then I found a blog entry by Jim Klein where he described a command to export and import from the database. A quick XML edit fixed ten days worth of problems.
After significant playing with the video board, I decided that maybe it was the CD-RW. Perhaps, Linux was aquiring the CD-RW as read-write, and the Solaris VM was trying to access the locked device. To test, I pulled the cable on the CD-RW and rebooted. Without the CD-RW, the error returned. Not the the CD-RW, I thought.
After a few more hours, a couple glasses of wine, and a few minutes of falling asleep in my chair, I decided that maybe I could observe the error on the VM if I could get the console to activate at start, rather than post start. I had no luck, but I did notice something I had not seen.
On the GUI Manager, virt-manager, was a Details button. Clicking this allows a user to browse statistics and hardware for a VM. I examined the hardware, and noticed something strange: the shared CD-RW had a source path of /dev. That's not going to work.
This would be easy to fix on RHEL5, because we could edit the config file. On F7, the config is stored in a database that I had not been able to crack. Then I found a blog entry by Jim Klein where he described a command to export and import from the database. A quick XML edit fixed ten days worth of problems.
# virsh dumpxml starbuck01 > /etc/xen/starbuck01.xmlNotice the source dev. Changed it to '/dev/cdrom', and imported per Klein's instructions:
# cat /etc/xen/starbuck01.xml
--- snip ---
<disk type='block' device='cdrom'>
<driver name='phy'/ />
<source dev='/dev/' />
<target dev='hdc' />
<readonly />
</disk>
--- snip ---
virsh define /etc/xen/starbuck01.xmlError gone.
Wednesday, September 12, 2007
Xen: Full Virt Solaris 10, Pt 3
Thus far, every attempt to get S10 to run as a Xen guest has ultimately failed. The problem has been the exact same video driver issue I've been fighting for a month. This means this is, beyond any doubt, a Solaris problem.
In each case, Solaris would install disk 1 in the text mode, then reboot the VM, and die. I tried a logical volume and I tried a real partition. I tried SELinux disabled. Finally (since I had the Cirrus Logic card installed, anyway) I decided to give it a 34th try. The trick was to configure F7 to recognize the Cirrus Logic video board...
Okay, that was easy. Of course my max resolution is now 1024x768.
Launched the VM install and found the graphical installed ran perfectly on the first try. Install completed. But when I went to start the VM:
In other words, it failed again, with the same error. I still think this has something to do with the video.
In each case, Solaris would install disk 1 in the text mode, then reboot the VM, and die. I tried a logical volume and I tried a real partition. I tried SELinux disabled. Finally (since I had the Cirrus Logic card installed, anyway) I decided to give it a 34th try. The trick was to configure F7 to recognize the Cirrus Logic video board...
system-config-display90 seconds later...
Okay, that was easy. Of course my max resolution is now 1024x768.
Launched the VM install and found the graphical installed ran perfectly on the first try. Install completed. But when I went to start the VM:
# xm start starbuck01
Error: Device 5632 (vbd) could not be connected.
Device /dev/ is mounted read-write in the privileged domain,
and so cannot be mounted read-only by a guest.
In other words, it failed again, with the same error. I still think this has something to do with the video.
F7 and S10 Dual Boot, Pt 2
I was surprised how easily I was able to recover the S10 boot that was destroyed by a failed VM install. Solaris takes a primary partition and slices it into virtual partitions. The VM install wiped out the physical partition that ran S10. My concern was that a new install of S10 would kill the MBR or /boot, this wounding Fedora.
Since the S10 /boot and Grub are installed in the slice, I was able to reinstall S10 without effecting Fedora. Furthermore, since the "real" Grub is in the Linux partition, once the S10 install was complete, the system booted to Linux Grub, which handed off the Solaris Grub.
A passing side note: I had to reinstall the Cirrus Logic video card in order to install Solaris.
Since the S10 /boot and Grub are installed in the slice, I was able to reinstall S10 without effecting Fedora. Furthermore, since the "real" Grub is in the Linux partition, once the S10 install was complete, the system booted to Linux Grub, which handed off the Solaris Grub.
A passing side note: I had to reinstall the Cirrus Logic video card in order to install Solaris.
Saturday, September 08, 2007
F7 and S10 Dual Boot
Oops. Why is it that no good news ever starts with "oops"?
As per standard operating procedure, I loaded Solaris 10 in the PC. Turns out, S10 uses GRUB, so BIOS hands off to MBR, which handed off to GRUB in sda2. (I had left sda1 open for a Linux /boot partition. Once I Solaris stable, I loaded Fedora 7.
I loaded F7 with /boot on sda1, / (root) on sda3, and a swap partition on sda5. Unfortunately, Fedora did not recognize Solaris at install time, so it did not add a GRUB stanza. With the install complete, BIOS hands off to MBR, which hands off to GRUB on sda1. Solaris was non-bootable. A quick modification to grub.conf:
Here's the oops. I attempted to build an S10 VM in a dedicated partition: sda7 (sda6 was an LVM partition). What I did not realize was that when linux boots, it sees that sda2 is a type bf Solaris. It then looks inside the partition, and allocated partition numbers to the Solaris slices. This means that when I added a partition, sda7, it actually appear to Linux as sda11.
When I installed the VM, however, I specified sda7. This means the VM installed in the slice that was S10 root. Thus, S10 is no longer bootable.
Oops!
Furthermore, on reboot, the partition that appeared as sda11, now appeared as sda7 (as expected) and sda10 was now the S10 /export/home slice.
1. Recover the S10 boot without loosing F7.
2. Rebuild the VM on the correct partition.
Oops. I accidentally deposited $10,000 to your account.No. Oops is always bad.
Oh well, I guess you can keep the money
As per standard operating procedure, I loaded Solaris 10 in the PC. Turns out, S10 uses GRUB, so BIOS hands off to MBR, which handed off to GRUB in sda2. (I had left sda1 open for a Linux /boot partition. Once I Solaris stable, I loaded Fedora 7.
I loaded F7 with /boot on sda1, / (root) on sda3, and a swap partition on sda5. Unfortunately, Fedora did not recognize Solaris at install time, so it did not add a GRUB stanza. With the install complete, BIOS hands off to MBR, which hands off to GRUB on sda1. Solaris was non-bootable. A quick modification to grub.conf:
title Solaris 10Now, BIOS hands off to MBR, which hands off to GRUB on sda1, which chooses Solaris 10, which hands off to GRUB on sda2 (presenting a second GRUB screen!), which boots S10.
rootnoverify (hd0,1)
makeactive
chainloader +1
Here's the oops. I attempted to build an S10 VM in a dedicated partition: sda7 (sda6 was an LVM partition). What I did not realize was that when linux boots, it sees that sda2 is a type bf Solaris. It then looks inside the partition, and allocated partition numbers to the Solaris slices. This means that when I added a partition, sda7, it actually appear to Linux as sda11.
When I installed the VM, however, I specified sda7. This means the VM installed in the slice that was S10 root. Thus, S10 is no longer bootable.
Oops!
Furthermore, on reboot, the partition that appeared as sda11, now appeared as sda7 (as expected) and sda10 was now the S10 /export/home slice.
# grep "sda.*>" /var/log/dmesgNow I have two problems:
sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 >
sda2: < solaris: [s0] sda8 [s1] sda9 [s2] sda10 [s7] sda11 >
1. Recover the S10 boot without loosing F7.
2. Rebuild the VM on the correct partition.
Friday, September 07, 2007
Xen: Full Virt Solaris 10, Pt 2
As I expected, the Solaris 10 install failed when it attempted to access CD 2. My research seems to indicate there is a known problem with the x86 CD ISO's installing in text mode. Unfortunately, the fix is not a high priority for Sun. But then Sun is a hardware company, so what do you expect?
On subsequent attempts to install, the GUI would not launch, though I was surprised to find that the default selections for the virtual display adapter would render 1024x800. What was truly bizarre was that the virtual display adapter showed as a Cirrus Logic card, which is what I had to use to get the "real" machine running.
And then I stumbled upon this:
Solaris 10 6/06 Installation Guide
Under System Requirements, it qualifies GUI install as requiring 512 MB for x86. The default memory footprint for a Xen VM is 500 MB. Ooops.
I set the memory for the VM to 532 MB, accepted all of Solaris 10's defaults, and...
Eureka! I got the GUI. Now I feel confident the install will work.
On subsequent attempts to install, the GUI would not launch, though I was surprised to find that the default selections for the virtual display adapter would render 1024x800. What was truly bizarre was that the virtual display adapter showed as a Cirrus Logic card, which is what I had to use to get the "real" machine running.
And then I stumbled upon this:
Solaris 10 6/06 Installation Guide
Under System Requirements, it qualifies GUI install as requiring 512 MB for x86. The default memory footprint for a Xen VM is 500 MB. Ooops.
I set the memory for the VM to 532 MB, accepted all of Solaris 10's defaults, and...
Eureka! I got the GUI. Now I feel confident the install will work.
Wednesday, September 05, 2007
Xen: Full Virt Solaris 10
I am astounded at the ease with which I could create a Solaris 10 virtual machine under Fedora 7. The virt-manager recognized the Solaris 10 CDs and launched the image without error or complaint. It also claimed support for Solaris 9, as well as some BSD beasties (who I care not about.)
Of course the Solaris install was still completely borked. I was able to get the installer to display a good 800x600 test pattern by selecting the XF86 VESA compliant driver, but the installer still insisted upon running in the text mode. Of course, we know what that means-- We're going to have trouble with CD 2.
Of course the Solaris install was still completely borked. I was able to get the installer to display a good 800x600 test pattern by selecting the XF86 VESA compliant driver, but the installer still insisted upon running in the text mode. Of course, we know what that means-- We're going to have trouble with CD 2.
L2S: /home vs /export/home
One of my big annoyances with this Linux to Solaris transition has been the difference between the Linux /home directory and the Solaris /export/home directory. From the Solaris point of view, no one should ever have a local account. Since all accounts will be exported, they all go under /export/home. When a network user log's in, the automounter mounts their account to /home.
Turns out, we can trick the automounter to clean up this mess by commenting out the last line of /etc/auto_home, and adding the user name and exported directory:
Now we steal a trick the Linux automounter man page. Replace the last line with the following:
The asterick and ampersand allow wild carding. Next time users login, they find them selves in /home/$USER.
Turns out, we can trick the automounter to clean up this mess by commenting out the last line of /etc/auto_home, and adding the user name and exported directory:
#
# Copyright 2003 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)auto_home 1.6 03/04/28 SMI"
#
# Home directory map for automounter
#
#+auto_home
doug localhost:/export/home/doug
Now we steal a trick the Linux automounter man page. Replace the last line with the following:
* localhost:/export/home/&
The asterick and ampersand allow wild carding. Next time users login, they find them selves in /home/$USER.
Solaris 10 Stats
I few notes on running Solaris 10 on the AMD 64 X2 CPU:
Not what I expected to see some 64's in there.
Still no 64's, but at least it sees two CPU's.
Ah... There's some 64's.
But this is what bothers me:
We have a 64-bit OS, but we're running everything 32 bit. Except for Java. Does that seem just a little unusual?
# uname -a
SunOS starbuck 5.10 Generic_118855-33 i86pc i386 i86pc
Not what I expected to see some 64's in there.
# psrinfo -v
Status of virtual processor 0 as of: 09/06/2007 00:01:51
on-line since 09/05/2007 17:55:46.
The i386 processor operates at 2109 MHz,
and has an i387 compatible floating point processor.
Status of virtual processor 1 as of: 09/06/2007 00:01:51
on-line since 09/05/2007 17:55:51.
The i386 processor operates at 2109 MHz,
and has an i387 compatible floating point processor.
Still no 64's, but at least it sees two CPU's.
# isainfo -v
64-bit amd64 applications
cx16 sse3 sse2 sse fxsr amd_3dnowx amd_3dnow amd_mmx mmx cmov amd_sysc
cx8 tsc fpu
32-bit i386 applications
cx16 sse3 sse2 sse fxsr amd_3dnowx amd_3dnow amd_mmx mmx cmov amd_sysc
cx8 tsc fpu
Ah... There's some 64's.
But this is what bothers me:
# pkginfo | grep "64-bit"
system SUNWj5dmx JDK 5.0 64-bit ...
system SUNWj5dvx JDK 5.0 64-bit ...
system SUNWj5rtx JDK 5.0 64-bit ...
We have a 64-bit OS, but we're running everything 32 bit. Except for Java. Does that seem just a little unusual?
Solaris 10: Locked and Loaded... Finally
I needed to get an instance of Solaris running at the house to get a little extra research time. The job is running 9, but Sun has opened 10 for "free" private use. What ensued was several weeks of agony and tragedy. Here's what it took to get Open Solaris 10 running under x86.
Download: Solaris 10 is distributed on 5 images. Unfortunately, the images are packaged in two formats, the first of which requires a Solaris system with CD burner. Seeing as I did not have a Solaris system available, I had to use the second option, which is 5 Windows executables. Turns out these were self extracting archives that each contained a single ISO image. What? Why didn't they just post the ISO and let me use whatever OS I wanted to burn the CD?
Installation: I dusted off a PIII 550, stuffed 396M of memory, and started the CD install. Luckily, it failed within sixty seconds. "Error 28: Selected item can not fit into memory". The problem stems from Solaris 10 needing 256M of memory. Uhhh... Isn't 396 more than 256? Guess not. I moved the install CD to two other systems both with 1G. Both moved past the error. Okay. I can get a new mobo with a gig of RAM for under $200.
New Hardware: Ordered parts from New Egg and installed in a spare case. Foxconn mobo, AMD-64X2 4200, 1G DDR2 800, integrated ATI Xpress 1250 video, and 1G Realtek ethernet for $184.96. Start the install, get past the error, install crashed. Let’s try again using text mode. Much better, except for one small detail... In text mode, you can only install the base system. The other CD's are then installed from the functioning base system. Guess that’s okay. Besides, now that we’re past the error, we need to grab the 64-bit ISO’s, so we’ll just consider this install a dry run.
Download, part 2: Back to the Sun website, follow the links for x86 64-bit, download the first Windows EXE. Oddly, it seems to have the same name as the 32-bit download; even the file sizes are the same. Let’s go back and examine the installed system. Hey! It’s booting a 64-bit kernel! Obviously, the ISO contains both the 32 and 64 bit kernels, and the OS recognizes which needs to be loaded. That is an innovation over Linux, though I question the value of loaded a 64 kernel and running 32-bit application stack.
Additional CD’s: The system is now booting to a console login. As I expected, the 1G Realtek card is not recognized. Luckily, I had the foresight to install a 100M Intel ethernet card. Inserted disk 2, mount, execute the install command, instant failure. Seems disk 2’s installer is a GUI utility, and we don’t have a window system. (As a matter of fact, I didn’t really care about a windowing system.) Certainly, there was a way to force the install utility to text, but it was easier to restart the install.
Installation, part 2: Selected text install, attempted to manually configure the X window system. Several attempts proved that there was not a compatible driver for the ATI board. There is a driver for an ATI Mach64 board, circa 2002. No go. Tried a Cirrus Logic 5434, circa 1998. Yeah, figures that one worked. After all it’s a decade behind current technology… just like Solaris. Graphical installer launched, just as it should, installed without any problems.
Operation: After the successful install, I logged in to switch the system from GUI to Console mode (runlevel 5 to runlevel 3). Normally this would be a straight forward exercise if it weren’t for Sun’s decision to deprecate the /etc/inittab and /etc/rc?.d scripts. How was this a good idea? Found a few hundred references to others that had to learn this lesson the hard way. Since there is not inittab to provide virtual consoles, you have to get the system to a Desktop Session, login, and issue the command:
Issued a power down, pulled the Cirrus Logic, and booted to a console session.
Finally! System is now running.
Time to dual boot this puppy to Linux.
Download: Solaris 10 is distributed on 5 images. Unfortunately, the images are packaged in two formats, the first of which requires a Solaris system with CD burner. Seeing as I did not have a Solaris system available, I had to use the second option, which is 5 Windows executables. Turns out these were self extracting archives that each contained a single ISO image. What? Why didn't they just post the ISO and let me use whatever OS I wanted to burn the CD?
Installation: I dusted off a PIII 550, stuffed 396M of memory, and started the CD install. Luckily, it failed within sixty seconds. "Error 28: Selected item can not fit into memory". The problem stems from Solaris 10 needing 256M of memory. Uhhh... Isn't 396 more than 256? Guess not. I moved the install CD to two other systems both with 1G. Both moved past the error. Okay. I can get a new mobo with a gig of RAM for under $200.
New Hardware: Ordered parts from New Egg and installed in a spare case. Foxconn mobo, AMD-64X2 4200, 1G DDR2 800, integrated ATI Xpress 1250 video, and 1G Realtek ethernet for $184.96. Start the install, get past the error, install crashed. Let’s try again using text mode. Much better, except for one small detail... In text mode, you can only install the base system. The other CD's are then installed from the functioning base system. Guess that’s okay. Besides, now that we’re past the error, we need to grab the 64-bit ISO’s, so we’ll just consider this install a dry run.
Download, part 2: Back to the Sun website, follow the links for x86 64-bit, download the first Windows EXE. Oddly, it seems to have the same name as the 32-bit download; even the file sizes are the same. Let’s go back and examine the installed system. Hey! It’s booting a 64-bit kernel! Obviously, the ISO contains both the 32 and 64 bit kernels, and the OS recognizes which needs to be loaded. That is an innovation over Linux, though I question the value of loaded a 64 kernel and running 32-bit application stack.
Additional CD’s: The system is now booting to a console login. As I expected, the 1G Realtek card is not recognized. Luckily, I had the foresight to install a 100M Intel ethernet card. Inserted disk 2, mount, execute the install command, instant failure. Seems disk 2’s installer is a GUI utility, and we don’t have a window system. (As a matter of fact, I didn’t really care about a windowing system.) Certainly, there was a way to force the install utility to text, but it was easier to restart the install.
Installation, part 2: Selected text install, attempted to manually configure the X window system. Several attempts proved that there was not a compatible driver for the ATI board. There is a driver for an ATI Mach64 board, circa 2002. No go. Tried a Cirrus Logic 5434, circa 1998. Yeah, figures that one worked. After all it’s a decade behind current technology… just like Solaris. Graphical installer launched, just as it should, installed without any problems.
Operation: After the successful install, I logged in to switch the system from GUI to Console mode (runlevel 5 to runlevel 3). Normally this would be a straight forward exercise if it weren’t for Sun’s decision to deprecate the /etc/inittab and /etc/rc?.d scripts. How was this a good idea? Found a few hundred references to others that had to learn this lesson the hard way. Since there is not inittab to provide virtual consoles, you have to get the system to a Desktop Session, login, and issue the command:
bash-3.00# svcs –a | grep cde
online ... svc:/network/rpc/cde-calendar-manager:default
online ... svc:/application/cde-printinfo:default
online ... svc:/application/graphical-login/cde-login:default
online ... svc:/network/rpc/cde-ttdbserver:tcp
online ... svc:/network/cde-spc:default
bash-3.00# svcadm disable cde-login
Issued a power down, pulled the Cirrus Logic, and booted to a console session.
Finally! System is now running.
Time to dual boot this puppy to Linux.
Subscribe to:
Posts (Atom)