Amazon API Perl Modules

I'm a Perl novice, having only used it for about eight years. As a result, when I needed to add some Amazon modules to my Perl stack for integration with the AWS API, I was at a loss. Once I extracted that zip, here's what worked:

cd /usr/local/src/amazon-*-perl-library
ls
ReadMe.html     src
cd src; ls
Amazon
cp -ar Amazon /usr/local/lib/perl5/site_perl/*
# test it
perl -MAmazon::SimpleDB::Client -e 1

In this case, no output is good.

I tried to get ::VERSION to work, but it gave a blank line. Guess that command only works if you've used Perl for ten years.

More XenServer Paravirt Adventures

While working with someone else about running Linux under Citrix Xenserver 5, I ran across an interesting paradox. Daniel points out that the ability to run Linux as a DomU has been moved into the kernel, no longer requiring the kernel-xen RPM.

Yet, when I tried to install F11, on XenServer as a paravirt VM using the "CentOS 5.3 x64" template, it failed. When I clicked the Logs tab in XenCenter, I got:

Unable to access a required file in the the specified repository:
file:///xxxyyyzzz/images/xen/vmlinuz

The template is expecting the standard install image format.

Tried a few other templates with the same result. The only way I was able to get it to run was by installing using "Other install media", which meant it was running HVM.

The paradoxical part of this, is that Fedora implies that it can be done, yet when attempted on a Fedora Xen Dom0, an F11 paravirt VM fails:

libvir: Xen error : Domain not found:
    xenUnifiedDomainLookupByUUID
Invalid URL location given: [Errno 14] HTTP Error 404

In a way, its the message XenServer threw: I can't find the specially compiled paravirt kernel.

My guess at this point is that there has to be a new installer that understands how to use the newly compile options. Looks like there is some more research to be done. Stay tuned.

I'm Not Allowed to Innovate

If an innovation falls in the forest, and there are no managers around to approve it, does it cause any change?

Kagan's answer:

Only if it crushes small bunnies.

Citrix XenDesktop Express

Citrix finally released the evaluation product for XenDesktop 4, with a free Express edition for ten users. Before downloading, watch the introductory video, then come back here, to find out that for 87.3% of you the product will be unusable.

Forty seconds into the video, the narrator announces that the download "includes everything you need for a basic environment". BRAAAAAM! Wrong: You need at least one Windows Server 2003 licenses, and a functioning Microsoft Active Directory Controller. (Which is a second 2003 license... but who's counting.)

A significant disappointment, as I had hoped this could be a tool used by a small mobile work force. Instead, the Express product is completely useless. If an organization has MS Server and AD, they need that full version, not the express.

I guess that's four hours wasted.

Windows 7 - Product Key Is Not Valid

When reinstalling W7 (after all, it is Christmas) I was advised that my product key was not valid. Me... Use an invalid product key... Never! And besides, this is a retail upgrade pack.

So what's wrong?

Turns out this is a known issue where under a normal reinstall, your product key will be rejected. At issue is the fact that his is an upgrade disk rather than a "full version". It assumes you have an existing OS. The good news is there is a Microsoft work around.

http://windows.microsoft.com/en-US/windows7/Windows-7-activation-error-invalid-product-key

To save you the effort of reading their propaganda:

1. Select Custom Install.
2. Highlight the target disk.
3. Click Advanced.
4. Click Format.

After the second install completes, you will be able to use your product key.

Santa Cristina

When I went to open this bottle of Santa Cristina Tuscan Sangiovese, I became highly distressed. Not only was it a 90% blend... and I avoid blends... but it was 10% Merlot! I'm allergic to Merlot.

No, really, I am allergic to Merlot. May grapes, in fact. It has to do with where the varietal is more related to blackberries or blueberries. Blackberries have lumpy skins with lots of little seeds, blueberries have smooth skins and one small (or no) seed. I'm allergic to blackberries, and merlot grapes are closely related.

(The proof exists in a book by Oz Clarke called "Introducing Wine", 2000, Webster's International.)

I tried it anyway, and luckily, no reaction, so the 10% was okay. As for the wine, the merlot added some body and richness, but nothing of interest. I've got another bottle, but will avoid this in the future. I found it satisfactory, but other might like it.

I'll got 4 of 10.

Miyone Granache

My second granache was also a Spanish wine, though I think it was from the Spanish riviera, judging by the unusual spelling on the label. (The southern coast speaks a different dialect of Spanish, called Catalan, which is closer to Italian.) Unfortunately, I was not as impressed with this one.

In my research on granache, it is most often used in blends, which I try to avoid. The problem, they say, is that the grapes tend to color toward rose rather than red. Special techniques allow the unblended granache to maintain its rich color, but are difficult to master. Miyone didn't quite get it.

I give it a 4 of 10. Not bad, but not good.

Perlat Granache

I decided to try a couple Spanish granache reds. This one was reasonably priced and fuller bodied than the Italians I've been drinking recently. I give it good marks a strong 6 of 10.

Santa Margherita

Unlike most of my selections that are mostly from wandering around the wine store looking at labels, I sought this one out based upon an advertisement in Wine & Food magazine. I got the mag as one of those "try it free" subscriptions, and ended up canceling. Instead I subscribed to Wine Spectator-- much better!

The wine sounded perfect: northern Italy, not a blend, safe white, and I like pinot grigio. But.. I knew it had to be expensive. Turns out not as much as I expected. I paid about $27 for the bottle, and, according to WS, $40 is considered expensive.

This was one of those moments when any novice wine drinker thinks, "is this really worth twice the price?" Yes, this one was. It was excellent. The only problem was the fact that many other norther Italian whites are excellent, too.

Factoring in the cost, I have to say a high 7 or 10.

AMD Athlon 64 Family CPUs with SVM

In my search to tell which AMD CPUs support the HVM virtualization extension, SVM, I found a page at XenSource site. Not only does it cover AMD, but also the totally convoluted Intel VMX issue. One would imagine that by now all chips would be virtualization optimized, but not yet. For AMD:

older Athlon™ 64 processors above 4000+
Athlon™ 64 X2 processors above 4800+
all AMD Athlon™ X2 processors (I guess they're dropping the "64")
all AMD Athlon™ X2 BE, LE, EE processors
all AMD Phenom™ X3 and X4 processors

They also mention Turion, but I'm satisfied with my laptop, as is.

"CompUSA" Extreme 908 Liquid Cooled PC

I got an e-mail from CompUSA.com about their new system, the Extreme 908, with liquid cooling system. Now I think liquid cooling is a great idea, but does anyone else see a problem with this design:I'll give you a hint: Think about gravity and leaking pipes.

Tera Bites fdisk

For half a dozen years, I've been showing people how to use fdisk in varying capacities including positions as a technical trainer for Red Hat and VMware. I had a standard joke that:

...you can specify a partition size K's, M's, or G's, but I've not had a chance to test if T for terabyte will work.

Today, I had the chance:

Command (m for help): n
Command action
e   extended
p   primary partition (1-4) p
Partition number (1-4): 1
First cylinder (1-182401, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-182401, default 182401): +1T
Unsupported suffix: 'T'.
Supported: 10^N: KB (KiloByte), MB (MegaByte), GB (GigaByte)
2^N: K (KibiByte), M (MebiByte), G (GibiByte)
Last cylinder, +cylinders or +size{K,M,G} (1-182401, default 182401):

So, fdisk does not support terabytes. This was under Fedora 11, so its a somewhat recent distro. I'm sure thee is probably a replacement somewhere that does work, but not using fdisk is like not using a screwdriver, just because a Ryobi 18v power driver is available.

The good news is that fdisk isn't broken, per se: +1024G works just fine.

P2V Happiness

It worked. I was able to use the Citrix Xenconvert to P2V a Windows XP Home system to a VM on a XenServer 5. There were a few lessons learned that should have been obvious... or maybe documented.

Citrix XenConvert works differently than some other P2V utilities in the respect that most of its operations are offline. It creates the image on the local hard drive, and does not contact the virtual infrastructure until the image is finished and ready for import. This is good because it prevents network issues from preventing image creation. The downside is that it requires the source machine to have 60% free space. (Preferably on a second disk.)

On the server side, you have to have a default landing zone (Storage Repository) for the imported VMs. I forgot to define the default SR, so my import failed. The good news is that the image was retained on the source machine, so I was able to transfer the image, and import directly.

It took about an hour to create on the source side, but about four hours to import the image on the server side. This is because the server is "governed" to prevent impact on production VMs. (In the real world, I'd have a staging server just for imports, deployments, and installs.)

Last detail is that once on the server, I had to strip the hardware specific software. The drivers were already loaded.

P2V, Windows XP, and SP3

In the continuing saga of P2V, my last plan was to wipe out the Linux instance (gasp!) on the target machine and attempt the process as strictly Windows XP. This box is an HP, so I fired up the recovery partition, and put it back to factory defaults. Of course that means there was 5 gig of garbage-ware on the C: drive, so I invested some time uninstalling stuff. Since it will once again be a dual boot system, I left things like multimedia and DVD burning software in the instance.

Once I had C: down to 10Gb (still bloated) I decided I would install SP3 on the physical machine. The plan was that the physical drive would handle the service pack better than the virtual drive. Much to my dismay, the machine booted, crashed, and rebooted. I assumed my server based copy of SP3 had been corrupted.

I was able to boot to safe mode, uninstall SP3, and recover the system to a normal desktop. That confirmed SP3 was the culprit. This time, I used Windows update. And... It crashed again. Same problem: perpetual reboot.

After loosing another hour monkeying with Microsoft's OS, I found that there is a known bug with SP3 effecting some HPs with AMD processors. Oh, thanks. I found a workaround for the problem on Jesper Johansson's blog. He's written a VBS utility that will prep the system for SP3. In my case, it worked retroactively.

At this point, SP3 is loaded, and P2V is running. Let's see what happens next.

Windows P2V Failures

First, the bad news: I have made several attempts at doing a Windows P2V, and they have all failed. Last night's attempt with the Citrix XenConvert software was no better... Except it lead me to realize why my attempts were all dismal failures.

Before we get to the the why, let me point out that Citrix XenConvert is one seriously cool tool. More on the later.

Now, the good news: All my P2V attempts have grabbed the drive from the Windows system (all have been XP), and written the content to the VMs drive. I can mount the drive and see the NTFS partition. When I boot the VM, they uniformly refuse to load.

And the problem is... All my Windows systems are dual boot. This means that when the P2V software packages the physical machine, it grabs the MBR and the NTFS, but misses the /boot partition. Without /boot, it can't get the grub.conf to realize that it doesn't need /boot.

The solution should be fdisk /mbr, but in order for that to work, you needed the Administrator password to the physical system. (Notice we are talking about the Administrative account, not an account with admin access.) Unfortunately, Compaq and HP obscure the account on their pre-installed systems. Oops.

In order to test that the problem is grub related, I plan to recover one of the systems to factory defaults, try the converter, then reinstall Linux.

Fedora On Citrix XenServer 5

I've been having problems getting Fedora to run on Citrix XenServer 5.5.0 and have been disappointed with the resources available to complete the task. But, once again, persistence has rewarded us with victory. The problem is "an issue" with the paravirt console driver.

On the Citrix XenServer, the console is meant to redirect to /dev/xvc0 rather than tty0. With Red Hat (and Fedora's) implementation of xen, the paravirt driver intercepts tty0 and seamlessly handles the reroute. No so with Citrix, meaning we need to help the VM find the location of the console. Three changes to the VM are needed:

vi /boot/grub/grub.conf
add a kernel argument:
...xen ro console=xvc0 root=...

vi /etc/inittab
change the tty1 line:
.../sbin/mingetty xvc0

to allow root login at console:
echo xvc0 >> /etc/securetty

What might make this difficult is the fact that without these changes, you can't get to the console. Unfortunately, without the console, you can't make the changes.

The interesting thing is that you can boot to single user mode, and avoid the problem. To do this:
  1. Using XenCenter, right click the newly created VM and select Properties.
  2. Click Startup Options.
  3. For OS Boot Parameters add single.
  4. Click OK and reboot the VM.
  5. Make the changes listed above.
  6. Use the same steps to remove single from the boot params.
  7. Reboot.
This should allow you to modify the VMs configuration.

Heaven and Hell

Just needed to document this for posterity:

Heaven is where:
the chefs are Italian,
the police are British,
the lovers are French,
the mechanics are German,
and things are run by the Swiss.

Hell is where:
the chefs are British,
the police are German,
the lovers are Swiss,
the mechanics are French,
and things are run by the Itilians.

Automated YUM Update

Most Linux machines use a derivative of the YUM update utility to install software and patch systems. I've been working with virtual appliances lately and have several VMs with very low memory and disk foot prints that can't handle a full update. I'll kickstart the box, and when I try to run an up2date, the process crashes for lack of space.

Here's a little script to solve the problem:

for J in `echo {a..z}`; do echo "Doing $J..."; \
yum update -y "$J*"; yum clean packages; done

It cycles through letters a to z, updates that letter and its dependencies, then removes the downloads. Once its finished, its best to run the full update command to grab packages that start with numbers... and that notorious piece of crap NetworkManager, which starts with a cap.

Oh, and I don't care that this technique is slow.

Happy Almost Halloween

US Capitol, just before sunrise.

Got My Windows 7: First Impressions

Of course, this was built in a virtual machine, so I didn't bother with sound on the first install. I'm more interested in the desktop. Further, this is Home Premium, so many features are missing.

Screen Resolution: Yes! We are one click away from the change resolution screen. Wonderful. Even better than XP.
Taskbar: Still have the stupid clock, but at least they've included the date. Whoa! You can turnoff the clock! Fianlly... after a more than a decade.
The name of minimized windows is hidden; only the icon shows on the taskbar. Its configurable.
We can not change the "Stop" function. In Vista, a laptop could only suspend directly from the desktop (even though it indicated it was an "OFF" feature.) Now we can customize it.
Control Panel: Where's my classic view? Gone. Sad.
Of course everything has moved, so it will take a year to figure out how to change anything.
Networking: What's a homegroup? Or a library?
Six clicks to get to the screen o set a static IP: Its eight in Vista.
Gadgets: Gadgets can GoGo anywhere on the desktop. Better, but they still suck.
Accessories: Dramatically improved Paint.
Better WordPad-- very Office-ish, but no spell check. :(
Windows PowerShell (finally, ls works. How very Unix of them.)

All in all, I have to say... Very good.

Free Companion Certificate

For years, I've gotten inserts, ads, and e-mails from American Express and its partners touting "Try our service, and we'll give you a voucher good for a free companion airline certificate." I've always wondered whether it was a scam. I finally took the leap, jumped through the hoops, got my voucher, and can now report:

Yes, it is a scam.

First, let me point out that this is regarding companion airline certificates for things like buying a magazine subscription. I don't know if this is the same thing Amex offers with their Delta Skymiles card, because I will never again willingly fly on Delta. Delta sucks, so I refuse to degrade myself to research the point. Amex offers other similar services, but I have never used those services, so again, I can not speak for them.

Here's how the companion ticket for magazine subscription deal works. They divide the country into zones, and charge a flat fee for travel between zones. Thus, Baltimore to San Francisco is Zone 1 to Zone 6. The website says the flat rate for you and your companion is $675. The question now is whether that is better than buying two tickets.

Okay... Reality check: The assumption is that we'll fly two for one, but if we save 10%, don't we still come out ahead? Of course! So, will we be satisfied with just saving money, or do we demand two for one? You and me... we're reasonable people-- if we save $50, we'll be happy.

I checked against two airlines, Southwest and US Airways, whose cheapest flights were one dollar apart. Let's just say they were $370 each. For you and your companion, that's $740... So we save money, right?

Not so fast: There are $80 in taxes and fees: that's $755. Hey: That's more!

But wait: They'll kick in $350 in savings if we'll accept their itinerary. At $405, we win. Yeah!

What? There's another catch? Yes. We can't actually go where we want. In order to save money, we have to go where they say:
In other words, we have to leave from our home airport, fly to a different destination, leave from somewhere other than where we went, and return to somewhere we don't live. Maybe there just aren't any round trip flights from Baltimore to San Francisco.
Okay, so maybe there are.

You, know... It's almost as if they don't want you to use these tickets.

FlashForward Fubar

It's kind of a bummer when a TV show's plot is so complex that the writers confuse themselves by the second episode.

The premise of the show is that everyone on the planet blacks-out, and has a shared two minute vision of the future. This means that in one character's vision, he is in a meeting in London with a known co-worker. The co-worker's vision is that the other guy is in London, and they are in a meeting. In other words, the visions correlate.

Ok... So lets take two families of characters:

Family 1
Dad = Mark
Mother = Olivia
Daughter = Charlie

Family 2
Dad = Greg
Son = Brandon
Mother = dead

Mark's vision is that he's at work. Olivia's vision is that she is in her home, with a man other than Mark. It seems that the visions can carry feelings back from the future, so she feels that she loves the other man. Mark feels fear. The important thing is that Olivia gets a look at the man's face, but he does not see her.

Shortly after the blackout and the vision, a child, Brandon, arrives at Olivia's hospital. She tells the child he'll be fine, and he says: "I know, Olivia." This means that in Brandon's vision, he knows Olivia... yet he is not in Olivia's vision. No correlation. Brandon's father, Greg, arrives at the hospital, but does not recognize Olivia. That correlates, since Olivia doesn't think the man saw her.

Then there's the daughter: Charlie. She visits the hospital, and recognizes Brandon. This would be okay if she and Brandon are together in the future.

Here's the problem. Brandon "knows" Olivia, but Olivia does not know Brandon. Olivia "knows" Greg, but Greg does not know Olivia. Charlie "knows" Brandon, but Brandon does not seem to know Charlie (but he hasn't seen her yet, so there maybe some flexibility on this one.)

Charlie, who is holding out on information, also knows a potential bad guy named DGibbons. Let's imagine that Charlie and Brandon were kidnapped by DGibbons. Lets say that DGibbons holds up a picture of Olivia and announces "Brandon, do you know this woman?" At this point, Charlie says, "That's my mother, Olivia." This works.

Except... Why isn't Brandon concerned with DGibbons, like Charlie? Why isn't Olivia concerned about her daughter? Did she send her-- and Brandon-- off to a sleep-over so she could bonk Greg? And why would DGibbon kidnap the children and then interrogate Brandon about Olivia? He already knows who they are because Mark is chasing him.

The writter's are stuck on a Mobius strip and are depending on you to forget these details. The good news is that ABC will cancel the show in a few episodes. Their executives have less of an attention span then you do.

SAGE is "On Guard!"

I had need to research some historical information about IBM's SAGE system and stumbled on the video. Mundo retro! Unfortunately, I remember taking stuff like this out of service.

ESXi Clone Without vCenter

Tossed together a little script to clone VMs on an ESXi that is not vCenter managed. Rather than paste the whole thing into the blog, I've posted it over on the site:

http://dougbunger.com/scripts/esxiClone

Once the script executes, access the ESXi via vClient, and choose the Configuration tab. Select Storage, browse the share, navigate to the vmx file and Add to Inventory.

No, Mrs Frederick is not Lena

The SyFy channel reads my blog. Ok, not the channel itself, but surely the producers, director, writers, and starts of Warehouse 13. I know this because last night, Lena and Mrs Frederick were in the same room. This blows away my theory that they are the same person.

I'm guessing they read my blog entry and changed the story arch, just to throw me off track.

...And I don't want to hear anything about last night's show being recorded three months ago.

Clip Comments From httpd.conf

Just a little regex to clip comments from httpd.conf, but it would work with most config files.

grep "^[\s \t]*[A-Za-z\<]" /etc/httpd/conf/httpd.conf

Its An Automobile, Not A Laptop

Let me see if understand this: Buick thinks I'll choose their car over a Lexus on the basis of the amount of RAM allocated to the navigation system? They think I don't care about milage or performance or ride quality? No wonder they're backrupt.

Is Mrs Frederick Lena?

Its taken nearly a decade for the network formally known as SciFi to reach its full potential. The change started with their take on The Invisible Man, but it was Battlestar Galactica that put them over the top and Eureka that proved BSG wasn't a fluke. Now they've got Warehouse 13, which is to the Secret Service, what the X-files was to the FBI.

But with less shark jumping...

Let me qualify: W13 is not a serious show. Its just for fun, and succeeds. But below the surface, there is a story arch.

Part of the story line is the enigmatic character, Mrs Frederick, a stately black women who cruises around DC in a limo driven by an Odd Job like chauffeur, ordering senior government men to do her bidding. She was in last night's episode, but I missed the significance of the dialog, until I heard it the second time tonight, while my son was time shifting the episode.

Arty was talking to Lena, the young black inn keeper, and stepped into the other room for a doughnut. When he returns, Lena is gone, and Mrs Frederick is waiting for him. Arty jerks back in surprise and says: "Mrs Frederick! What did you do with Lena?"

You know-- We've never seen Lena and Mrs Frederick in the same room.

Finally: VMware for Executives

Executive: I'm not satisfied with performance.
IT Puppet: With VMware, computers work better, and save money.

Too bad most of the information is only marginally accurate, but what do you expect when your primary data source is a sock puppet.

Chinatown Arch-- All Done!

They got the scaffold down. All cleaned up, new paint. Your stimulus dollars at work.

HowTo stunnel, Pt 2

In my example config, I specified:

syslog=yes
debug=7
output=/opt/stunnel/stunnel.log

First, the entry syslog=yes writes to the /var/log/secure, which renders stunnel.log to be redundant.

Second, the debug of 7 is more than we need for production. Under normal load, does not log anything, including a restart of the service. Debug 5 gets the starts and stops, but logs three lines per transaction. It's a thin line to tread, but looks like I'm going to use 5.

Next problem is the fact that, as presented, the server will accept connections from anyone. The client's PEM was unused. We need to tell the server force clients authenticate. We'll do this with key exchange. Add two lines to the server's config file:

CAfile=/etc/stunnel/server.ca
verify=2

We won't really have a CA, but it will be close enough. Copy the server.pem to server.ca, which we modify in a few minutes.

On the client:

echo "-----`hostname`-----" > `hostname -s`.crt
sed -ne '/BEGIN CERT/,/END CERT/p' \
    <server.pem   >>`hostname -s`.crt

Ship this off to the server.

On the server, we will add the CRT we just transferred from the client to the CA we created a few moments ago:

cat clientfile.crt   >> server.ca

Restart stunnel on both ends. Unless a client's CRT has been added to the server's CA, they can't get in.

Chinatown Arch: Almost Done

They unveiled the DC Chinatown Arch. Still have some of the scaffold left. Newly refurbished, nice and pretty. I'm covering for a coworker on the morning shift, so the sun is at the wrong angle. I'll have to get a better picture next week.

Console is not yet active for guest

I was barking up the wrong tree on this one. I had a VM that would not start, displaying the message:

Console is not yet active for guest

My confusion was caused by the fact that if I shutdown a different VM, this one worked fine. Furthermore, this was effecting multiple VMs, besides just the two. I even went so far as to call tech support on this one. Gasp!

Needless to say, I solved this before they did. The problem is a quirk in the RHEL5 virt-clone command. This command is suppose to copy a VM image, while making the needed changes in the descriptor file in the process. It does not, however, change the VNC port number. This has the effect of forcing two machines to use the same video card. In the directory with the VM descriptor files, try:

cd /etc/xen
grep "^vncdisplay" * | sed 's/"//g' | sort -n -k3

Any duplicate values define a conflict-- whoever boots "powers on" first, wins. The looser gets power, but can't initialize their BIOS.

To resolve, edit the descriptor file, and assign a unique value: But watchout! Make sure it is unique across the cluster, otherwise a migration could cause a collision.

Now I guess I got to tell Red Hat.

Limited Edition Star Trek Waffles

I bought all they had. I've cornered the market! Just imagine: in a couple years, there no telling how much these will be worth... After all-- they're limited edition. Watch out E-bay!

HowTo stunnel

I ran into a problem that I thought stunnel might be the solution, but found Fedora's stunnel RPM to be in complete. First it has no init script and second it's documentation is lacking. Further the same problems exist with the stunnel.org code (which I couldn't get to compile!) I'm sure there is more than two problems, but...

Over at Gaztronics.net, there is a working init script. Grab it and save it to /etc/init.d. Next we need an stunnel.conf (also not included, though there is an example... which won't work.) I found a good set of example files over at Edna Narrabilis' Blog. These may need a few tweaks, but since I'm working on an remote mySql issue, these are good.

Notice in Edna's configs, there are really only two differences between the client and the server. First, not the client=yes directive: just to be safe, add client=no to the server side. Second, notice the client's connect line includes the server's name. Drop her configs on each sides /etc/stunnel directory.

Now that we've got the Fedora RPM, Gaztronics init.d, and Edna's configs, we need an SSL certificate. When you generate the certs on both sides, it is important that for Common Name, you use the fully qualified, DNS resolvable, name for each machine.

As I mentioned, I made a few tweaks. For instance, I moved the PID out of /tmp. I also forced a few other options. Just for good measure, do this first:

mkdir /opt/stunnel
chown nobody /opt/stunnel
chmod 700 /opt/stunnel

Server syslog=yes debug=7 output=/opt/stunnel/stunnel.log   cert=/etc/stunnel/stunnel.pem pid=/opt/stunnel/stunnel.pid foreground=no setuid=nobody setgid=nobody client=no [mysql-svr] accept=6639 connect=3306

Client syslog=yes debug=7 output=/opt/stunnel/stunnel.log cert=/etc/stunnel/stunnel.pem pid=/opt/stunnel/stunnel.pid foreground=no setuid=nobody setgid=nobody client=yes [mysql-client] accept=localhost:6639 connect=fqdn.target.tld:6639

One thing to notice: we are flipping the port as they go across the network from 3306 to 6639. This way we can use iptables to block 3306 as adefense against automated SQL injection attacks. I found that If I tried to do the client accept at 3306, it didn't work, but I think that's a mySql thing.

Virtual PC 2007 VM Component Files

I'm finally satisfied with my Virtual PC 2007 install on my Vista laptop. Just for reference, the VM is made up of several files in a dedicated directory... just like VMware. In the table below, the first item is the file extension, followed by the extension's name, followed by a description of what the file does.

vmc = Virtual Machine Code, VM descriptor file
vhd = Virtual Hard Disk, the actual image (may be more than one)
vud = Virtual Undo Disk, changes to disks are discarded when
            the VM is powered off (great for testing things)
vsv = Virtual SaVe state, allows the VM to be paused

Citrix XenServer P2V

An interesting feature of the Citrix XenServer install CD is the option to execute a physical to virtual (P2V) migration. Having significant experience with VMware Converter, I thought I'd use this to create a VM from a Windows XP instance sitting on a drive that I never use. From my point of view, it would be a legal version of XP, though I'm sure Microsoft's lawyers would argue otherwise.

The first shot at P2V failed with a message specifying a partition error with /dev/dm-5. My immediate concern was that the USB multi-card reader was causing trouble. I disconnected it, and tried again with the same results.

A little research found a completely unexpected result. Turn out Citrix P2V only works on Linux, and not Windows. Looks like the old NTFS patent issue. Further, it only works on "older" Linux, somewhere in the Fedora 8 and before range. The problem doesn't seem to have anything to do with USB, but instead the way the migration tool interprets the device mapper.

I decided to hold off on further use of Citrix P2V, for the time being, but am continuing research on doing a manual P2V of that Windows instance.

Trouble Makers

Metro has an advertising campaign meant to convince people to be nice to handicapped people. Then I noticed something about the posters: Its the same two people causing all the trouble. So I figure these are wanted posters, not advertisements. If you see them, grab'em!

Vista VM Disk Size < 12Gb

I recently installed Vista in a VM on my Linux virtualization platform, running Qemu/KVM. Today, I installed a Vinat on a VM on my Vista laptopn, running Microsoft Virtual PC 2007. As part of the experiment, I wanted to how small I could make the virtual disk. On my Linux platform, I gave Vista 20 gigabytes, and was disappointed to find that it took about 12GB of space.

This time, I gave it 10GB, expecting to get an error, and found that it installed on about 7GB, This seems to imply that Vista, which no long prompts for "what do you want to install" makes these decisions for you based upon the space it sees. By giving it less space, it put less crap on the drive. I was able to clean about another 500MB by deleting Games and few other silly things.

Unfortunately, the VM would not install SP1 from disk, as it needed 4GB to extract. By adding a second virtual drive of 5GB as a "D Drive", SP1 was able to install. I actually removed the second drive once loaded. Thus, it looks like it could be possible to run Vista on 8GB, but 12GB seems to be the base.

*** Update 8/12/2009 ***
Using a second disk was too unstable. After installing SP1 and a dozen security patches, the 8GB disk reached capacity. and the VM would lock up. There was no easy way to shift components from the C: drive to D:. I ended up rebuilding at 12GB, and after updates have 3GB left-- acceptable for small testing tasks.

Dual Boot Citrix Xenserver 5

I'm very proud of this hack. So proud that it's actually a quad boot, but two of the four are Windows, which we don't care about, so we'll stick with the title of dual boot. Here's the scenario:

A couple weeks ago, I did the VCP-310 cert, with the anticipation of rolling that into the VCP-410 next month. Until then, I'm going to do the Citrix CCA cert. This means we need a Citrix Xenserver... Unfortunately, the only machine I have that is not doing something is my oldest kid's desktop. Now, this is not as bad as it sounds, as he's off doing a summer session at some university somewhere, or something.

Since he's gone, we'll wipe the drive and load Citrix. Not so fast! He's got Windows on that thing and uses it as a game station when he's in town. I'd already partitioned it and loaded it with boot to Fedora as well as Windows, so Citrix should be a non-issue. Except, Citrix Xenserver thinks it should be the only thing on the system... Just like ESX.

So, here we go:
1. Disconnect the hard drive and load a second hard drive, pinned out as master.
2. Load Citrix Xenserver in the default configuration. Reboot and test.
3. Disconnect second hard drive and pin out as slave. Cable it and the original drive to the primary controller.
4. Boot to BIOS and ensure both are visible. This ensures that both are pinned correctly.
5. Boot off the previously existing Linux partition to Single. Use fdisk -l to validate that the second drive is visible.

(This is where it starts to get hairy.)

6. The master appears as /dev/sda and the secondary as /dev/sdb. Mount the second drive:

mount /dev/sdb1 /mnt
ls -l /mnt

Notice anything unusual? No boot partition. Citrix is loading the MBR with a LILO like loader.
7. Inspect /boot:

chain.c32
config-2.6.18-128.1.6.el5.xs5.5.0.496.1012kdump
config-2.6.18-128.1.6.el5.xs5.5.0.496.1012xen
extlinux.conf
extlinux.sys
initrd-2.6.18-128.1.6.el5.xs5.5.0.496.1012kdump.img
initrd-2.6.18-128.1.6.el5.xs5.5.0.496.1012xen.img
initrd-2.6-xen.img
mboot.c32
menu.c32
System.map-2.6.18-128.1.6.el5.xs5.5.0.496.1012kdump
System.map-2.6.18-128.1.6.el5.xs5.5.0.496.1012xen
vmlinuz-2.6.18-128.1.6.el5.xs5.5.0.496.1012kdump
vmlinuz-2.6.18-128.1.6.el5.xs5.5.0.496.1012xen
vmlinuz-2.6-xen
vmlinuz-kdump
xen-3.3.1.gz
xen-3.3.1.map
xen-3.3.gz
xen-3.gz
xen.gz

Some of this looks familiar. We just need to arrange it into our existing grub.conf.
8. I looked at an old FC6 Xen platform's grub.conf:

title Fedora Core (2.6.20-1.3002.fc6xen)
    root (hd0,0)
    kernel /xen.gz-2.6.20-1.3002.fc6
    module /vmlinuz-2.6.20-1.3002.fc6xen ro root=/dev/hda3
    module /initrd-2.6.20-1.3002.fc6xen.img

(Why in the world am I still running Fedora Core 6? Because, it is most similar to RHEL5.)
9. A closer look at /boot will confirm that many of those files are actually symlinks. So lets add the following to grub.conf:

title Citrix Xenserver 5.5.0
    root (hd1,0)
    kernel /xen-3.3.1.gz
    module /vmlinuz-2.6-xen ro root=/dev/sdb1
    module /initrd-2.6-xen.img

Notice the root line-- hd1,0 not hd0,0-- we're on the second drive. Notice the vmlinuz line-- we are using /dev/sdb1-- that's what we mounted.
10. Reboot... and watch it fail!

I'll save you the agony of troubleshooting. First, in the Fedora example, boot is partition 1, so we prefix the files with a slash (/). Citrix did not use a boot partition: root is partition 1. Therefore, we need to prefix the files with the directory: /boot/

Second, we mounted /dev/sdb1, which was the Xenserver's root partition. Yet, when Xenserver boots, it sees the drive as /dev/hdb1. This is because I happen to have Fedora 8 loaded, and effective F8, all drives are sd, even though the drives are actually IDE rather than SCSI. Since Xenserver is based on el5 (look at the vmlinuz links) it sees the drive as hd.

Here's what actually worked:

title Citrix Xenserver 5.5.0
    root (hd1,0)
    kernel /boot/xen-3.3.1.gz
    module /boot/vmlinuz-2.6-xen ro root=/dev/hdb1
    module /boot/initrd-2.6-xen.img

Xen error "could not find any free loop device"

On one of my Xen systems, I added a disk to a VM, but when I attempted to start the machine, it threw up an error:

could not find any free loop device

With a little help fron Google, I found a solution.

Normally, I run m VMs in LVM because they are faster to create, easier to resize, and perform better. In this case, the VM had to be an image file for easy conversion from Xen to ESX. There were four VMs with two disks each. That's 8 disk total. (Yes, I realize you can count, but that number is important.

Turns out that each image file uses a loop device, as if mounting an ISO file. By default there are only 8 /dev/loop's. (See, I told you it was important.) Here's how we add more:

ls -l /dev/loop* | wc -l
          8
echo "options loop max_loop=64" >> /etc/modprobe.conf
rmmod loop
modprobe loop
ls -l /dev/loop* | wc -l
          64

F11: service libvirtd restart

Fedora Core 4 introduced the ability to run Xen virtualization, but it required a kernel recompile. When 5 came out, it was a question of significant config, but no recompile. Fedora Core 6 allowed for single click Xen virtualization during install. It was easy, stable, and effectively the same model deployed in Red Hat Enterprise Linux 5.

From a Xen standpoint Fedora 7 and 8 were incremental releases, with 9 being a breakthrough release. I say that, because Xen was pulled as a feature of 9-- if you wanted virtualization, you were back to compile and manual module load. The reason was a decision by Red Hat to move away from Xen (as Xensource had been purchased by Citrix) and to adapt Qemu-KVM.

With Fedora 10, Qemu-KVM was the native virtualization engine, but there was a major problem. The virtualization layer was running as service. This meant that if the service was restarted, all the VM's reboot. Mucho bado.

The good news is that in F11, it looks like the kernel virtualization modules (KVM) have been integrated with the kernel such that a restart of the service only effects the Qemu management layer. So, go ahead:

service libvirtd restart

Fedora on MS Virtual PC 2007

Finally got a good load of Fedora running in MS Virtual PC 2007, on my Vista laptop. ended up with Fedora 9, simply because it was the most recent 32 bit version on had on my server. I load 32 rather than 64 bit, because HP was so rude as put 32 bit Vista on a 64 bit laptop. Theoretically, the AMD Turion64 X2 should be able to run a 64 bit guest in a 32 bit host OS, but we'll try that later.

A good reference for loading Fedora in VPC2007 can be found on Sean Earp's Blog. He suggested using a Grub option of noreplace-paravirt, which I didn't need. The options that I needed for Fedora 9 were:

clocksource=pit noapic vga=0x700

The vga= directive was a fun one. A value of 0x700 doesn't actually work. Instead, it kicks you to a text based menu that lets you select the mode at boot time. This will let me experiment, until I find the one I like. So far, I'm favoring 0x303.

Next step is to move the image to a USB key.

rPath Cloud Computing Video

I've been spending allot of time lately telling people at work not to talk about cloud computing while all the time focusing on it myself. (Don't tell them.) I found this video by rPath. Very informative and well done.

Newservers.com, Pt 3

The Windows guy is going to be jealous. Yesterday, I saved an image on the "small" server I set up at Newservers.com, then canceled the server. Today, I allocated an different model server, and applied the image. It worked. He suspected it would fail due to drivers. Ha!

An important detail I had misunderstood about their billing: You are billed while the server is allocated, not powered on. To check this, last evening, I issued a poweroff command just before starting the hour long walk, subway, bus, and car ride home. When I got home, I checked the invoice, and, just as expected, it had incremented another $.11. Once I did a Canceled Server, the billing stopped.

And that makes sense: If you have data on the drive, but power off the server, you are still costing them money. Once you de-allocate, they can hand it off to someone else.

As a side note, I found a couple other services to disable: lvm-monitor (they build everything in root) and hidd.

Newservers.com, Pt 2

In response to a comment, I wanted to take a moment to wax philosophic on the concept of the Infrastructure as a Service (IaaS) business model.

I have a silly little website, dougbunger.com, that is hosted on a virtual machine... Xen, of course. The VM does a few other small functions, but represents Platform as a Service (PaaS). To me, the physical machine is irrelevant, as long as I get a Linux platform.

From a business standpoint, the provider (VPSlink.com) is paying for the hardware, rack space, power, and telecom. These are fixed costs for a system with fixed resources. As long as his pricing is such that at 30% resource utilization, he is covering his fixed costs, then any VMs over 30% are profit. The main thing to remember is that after a stable customer base of >30% is achieved, when an "extra" VM disappears, it costs nothing. Effectively, the top 50% of the server is free.

(Yes, I'm 10% short, but when that box is at 90%, its time to worry... You don't want to degrade the "good" customer's VMs.)

Now lets consider the Newservers.com model. In this case imagine ten machines all with the same fixed cost. When a physical server is allocated to a customer, you are offsetting the fixed cost of the server. When it is not allocated, you are "loosing money". Yet, we can never allow the server farm to reach even 80% capacity, since the core of our business model is spare capacity. No capacity, no elasticity, as Amazon might say. This means that we always need two machines empty, and as such "loosing money".

To further complicate things, with VMs, we can always cross that 90% mark. We can let some customers be degraded to get us over a surge. With physicals, however, there is no way to sell eleven boxes, if all we have is ten.

In the end, the whole thing boils down to good capacity planning and accounting. My hope is that as the pricing models mature, we'll see more of the IaaS clouds. As for Newservers.com, they've already saved me the cost of an HP DL380 G5 that I need for the next 60 days.

BTW: Thanks for the comment!

My $0.11 Computer: Newservers.com

Just for fun (and professional development) I leased a server from a company called Newservers.com that market themselves as a "bare metal cloud". Within 15 minutes, I had an account on a physical Dell 1955 machine running Linux. The funky part, is that they claim they only charge $0.11 per hour while the box is powered on. Oddly, this does not seem to be a VM. Here's what we got:

[root@server1868 ~]# cat /etc/redhat-release
CentOS release 5.3 (Final)
[root@server1868 ~]# head -1 /proc/meminfo
MemTotal: 1026612 kB
[root@server1868 ~]# grep "model name" /proc/cpuinfo
model name : Intel(R) Xeon(TM) CPU 2.80GHz
[root@server ~]# fdisk -l

Disk /dev/sda: 36.4 GB, 36420075008 bytes
255 heads, 63 sectors/track, 4427 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 3824 30716248+ 83 Linux
/dev/sda2 3825 4085 2096482+ 82 Linux swap / Solaris
[root@server ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 29G 1.7G 26G 7% /
tmpfs 502M 0 502M 0% /dev/shm

It's a little bloated. Let's lighten the load:

[root@server rc3.d]# for J in kudzu portmap nfslock mdmonitor rpcidmapd rpcgssd bluetooth netfs pcscd autofs yum-cron cups gpm anacron atd yum-updatesd ; do chkconfig $J off; done

And start to secure this puppy:

[root@server ~]# sed -i "s/=disabled/=permissive/" /etc/sysconfig/selinux
[root@server ~]# touch /.autorelabel
[root@server ~]# vi /etc/hosts.deny     # ALL : ALL
[root@server ~]# vi /etc/hosts.allow     # sshd : safe.location
[root@server ~]# vi /etc/ssh/sshd_config     # PermitRootLogin no
[root@server ~]# useradd blah blah blah -G wheel     # unpriv user in wheel group
[root@server ~]# passwd yada yada yada     # strong password
[root@server ~]# visudo     # enable wheel

Cool. Now lets see where this takes us.

Converting a Xen Paravirt VM to Fullvirt

We had a VM that would occasionally try to load the wrong kernel at boot time. The machine has to load the para-virt ELxen kernel to successfully start, but it might try to load a PAE instead. As a result the VM would fail to boot.

It might seem that the solution would be to fix why it was loading the wrong kernel, but, for reasons we will not discuss at the moment, we needed to accept that it would load the wrong kernel occasionally. So, our mission became to move the VM from para-virt to full-virt. As it turns out, it was just a case of modifying the VM config file.

Comparing the two configs, we see some differences:

diff vm-para vm-full
< bootloader='/usr/bin/pygrub'
< vfb=['type=vnc,vncunused=1,keymap=en-us']
---
> builder = "hvm"
> kernel = "/usr/lib/xen/boot/hvmloader"
> boot = "c"
> pae = 1
> acpi = 1
> apic = 1
> localtime = 0
> device_model = "/usr/lib64/xen/bin/qemu-dm"
> sdl = 0
> vnc = 1
> vncunused = 1
> keymap = "en-us"

This tells us that the para-virt VM uses a different Grub and that it needs a set of variables to describe its connections to the outside world. The full-virt VM, has a much wider set of variables, and uses a different boot mechanism.

So, how do we convert?
1. With VM running, install a standard kernel, using rpm -ivh
    or yum install kernel.
2. Power off VM.
3. Edit config file, removing the two line specified by "<" above.
4. Add the lines specified by ">" above. Make sure to remove the leading greater-than (>) symbols!
5. Start VM.

RDP on Vista Premium

I used Fransblog's termserv patch to enable RDP on my Vista Home Premium VM. Worked perfectly the first time. Thanks Frans!

Fedora On Virtual PC 2007

While trying to install Fedora on Virtual PC 2007 running on my Vista Home Premium laptop, I was plagued by boot hangs. Usually, the CD would fail to boot on:

running /sbin/loader

It seems the way around this was to add a directive to the installer:

clocksource=pit

I'm going to blame this on my AMD Turion64 CPU, as few others had come across the same problem.

The problem that had gotten everyone else was the graphics levels between Linux and VPC are not compatible. The solution to that was to add:

vesa

This forced Linux into a 16 bit color mode. Once the install was complete, both options needed to be added to GRUB.

The next problem was the network. I had to attach the VM's network to the physical interface on the laptop. I had expected to be able to use a NAT interface, but I suspect this was my own fault for using my network install server rather than an DVD image.

Alas, my first successful install was Fedora Core 4, but when it booted up, I could not login. The clocksource was running so fast, PAM timed out the username and password within a matter of seconds. In all fairness, FC4 is eons old. I found a 32 bit DVD ISO for FC6 and F9 on my install server, so I'll give them a shot and see if they run any better.

Windows Virtual PC 2007

After building a Vista VM today, I needed a downloadable copy of SP1. That's when I stumbled upon Windows Virtual PC 2007. I knew of its existence, but must admit, I didn't realize it was free. Another thing about it astonished me:

Virtual PC 2007 runs on: Windows Vista™ Business; Windows Vista™ Enterprise; Windows Vista™ Ultimate...

Oh there were others, but no Vista Home or Home Premium. I guess I need to fork out a couple hundred dollars to upgrade.

Or, I could fire it up and see what happens. I grabbed the installer, launched it on my Home Premium laptop, and sure enough, it complained that it was not supported. It seems however, that "unsupported" does not mean "not allowed" I chose to continue, and the install finished.

I launched the program, and it immediately complained again. Yet... There is a checkbox: "Don't show this message again" Next thing you know, I'm in an installation wizard.

Now we get to find out what this thing will do. I launched the wizard to install a Fedora instance. When presented with the Operating System choices, I found the following list:

Windows 98
Windows NT
Windows 2000
Windows XP
OS/2
Windows Vista
Windows NT Server
Windows 2000 Server
Windows Server 2004

What! OS/2? You're kidding, right? I think I threw those disks away about 10 years ago. And what about Linux, damn you.

Other

Okay.

In the short wizard, I allocated 312 meg of RAM and 8 gig of disk. The VM appeared in the Virtual PC Console window, and I selected Start. Once the VM launched, I was able to click the CD menu item and select "Use Physical Drive", but it was already too late in the POST. I clicked the Action menu item, and selected "Reset". This time it launched the installer from CD.

Now, we get to see if it will install.

Vista /etc/hosts File

As we all know, each iteration of Windows gets more like Linux, and Vista is the most Linux-like so far... Except for the fact that it sucks, but then Microsoft does have to differentiate their product somehow.

Take for instance the /etc/hosts file. In Vista, it is at:

Local Disk (C:)
  Windows
    System32 (Syswow64)
      Drivers
        etc
          hosts

Unfortunately, you can't edit the file. You're not allowed... its not your OS, so you can't edit it. Its Microsoft's OS.

But we got a hack for that.

Click Start (even though it doesn't say "Start" any more.) In the Start Search box, type "command". Right click the displayed icon and select Run as administrator. This will give you a command prompt. Yeah-- just like Linux. Execute:

cd c:\windows\system32\drivers\etc
notepad hosts

Add the needed entries, save, close, then exit to discard the command prompt.

More Yahoo Silliness

I've seen this picture on Yahoo mail a few times promoting a video on the history of their mail service. If you a actually look at the picture, you may notice something funny... The machine in the picture is an IBM 3178C terminal designed to be connected, via coax, to an IBM mainframe. This device was capable of 80x25 green text. I could not be made to display color, nor could it be connected to a phone line.

The good news is that it could be connected to the internet using a text based browser, called Charlotte (as is Charlotte's Web). This was way before Yahoo... The only way we had to search the internet was with Gopher.

Yahoo Has Hacked Browser History

This is scary, but it is not the first time this has happened in the last few weeks. I'll search for an airfare, then have Yahoo Mail present me with a banner add advertising the exact destinations for which I searched.

As an experiment, I opened Yahoo Mail in a tab, and opened the inbox. Next, I opened another tab, and visited Expedia. There, I searched for an airfare. I returned to the Yahoo Mail tab, and clicked "Check Mail". I was presented with a Netflix ad.

Lets try again. This time Travelocity: search for airfare in a different tab, return to the Yahoo Mail tab, click "Check Mail", and here's what I got...



Theoretically, this should not be possible for security reason. One site should not be able to read another's cookies, and no one should be able to access a history. I tried the same process in IE7, same results. Even works on Linux. The only factor seems to be the site searched: Expedia does not work, Travelocity does. I've seen others (obviously Orbitz), but more travel sites than any. I guess that's what I use the web for most, except technical stuff... certainly not porn.

BTW, the trip to Reykjavik was too expensive. So much for given the bankrupt country money. Maybe I'll go to California.

FC-6 VM On F11 Platform

I recently upgraded one of my virtualization platforms from F10 to F11. It is significantly more stable and refined. An odd thing happened, however. The target system is an AMD-64x2 with 5G. Its was hosting:

FC6 application server
F8 VNC server
WXP Pro with RDC (need it for my current contract)
F10 on 4 node cluster for clustering R&D
F8 on 2 node cluster for web development
F8 application server

Under F10, the box gave great VM performance while at an idle.

Under F11, the FC6 VM pegged the virt-manager's performance graph at 45% of the physical CPU cycles. Yet, within the VM, top saw nothing. Effectively an entire core spinning away on nothing! I re-imaged (via kickstart) the VM as FC8, and now that same machine sits at half a percentage point. I'm not sure I have an explanation other than a different kernel or virt drivers.

Big Prop

This was an odd thing to see on the ride home. Looks like a C-130 cargo plane propeller. It was being towed by a pickup truck.

Scaffold in Chinatown, Pt 4

Maybe David Copperfield is going to make it disappear.

Sculpture Garden Fountain

In the background is the National Gallery rotundra and Washington Monument.

US Capitol

From the window of the commuter bus... Which I got to ride for free. Yesterday's driver forgot his hole punch, and used a pen to mark out the ticket. Today's driver punched over yesterday's mark.

Scaffold in Chinatown, Pt 3

This is getting exciting. Turns out the crew is working at night. About 8pm they close down H St. How truly civilized.

Xen error: xc_dom_find_loader

One of my Xen virtualization platforms crashed yesterday, throwing the error:

Error: (2, 'Invalid kernel', 'xc_dom_find_loader: no loader found\n')

Had the hardest time figuring out the cause.

Then, for whatever reason, I did a df -h and found my the /var/lib/xen mount was at 100%. I wiped out the contents of the save subdir, and moved all the boot_ files to /tmp. Suddenly, all my VMs could start.

Scaffold in Chinatown, Pt 2

They've added another layer of scaffold, but haven't actually done anything. It must be the same crew that has about half of all the subway escalators offline... you never actually see them working.

Pennsylvania Avenue

At the Nation Archives, looking south east toward the Capitol.

Scaffold in Chinatown

Looks like somebody is going to work on the Chinatown arch. I'll have to keep an eye on this.

Running Firefox Across an SSH Connection

I've always been annoyed by the Firefox "feature" of executing locally, even if initiated remotely. Consider this situation: You are running Firefox on your Linux workstation, and SSH into a server. On the server's command line, you launch Firefox. Rather than opening a new window which is X-tunneled back through SSH, it opens a new tab on the local Firefox. Argh!

Here's the solution:

[bungerd@lnxqa20 ~]$ MOZ_NO_REMOTE=1 firefox
[bungerd@lnxqa20 ~]$ firefox -no-remote

Either works, but the second seems more intuitive.

The Library of Congress

It was a nice day. I looked like a nice picture.

Power Sources For DC-Baltimore Area

I got this in my power bill this week: a break down on the origins of the electrical power supplied by BGE (Baltimore Gas and Electric, which is owned by Constellation Energy.) The good news is very little oil. I already knew a significant portion was nuclear.

What is interesting is the Renewable energy column:

No solar. We've got a hundred miles of shoreline, and no wind power. Oh, and I love that wood fired generators are considered under the renewable energy category. The best part of wood power is that we consume forests, thus reducing our nature carbon filter, and discharge carbon at the same time. How very efficient.

This chart is a perfect example of one of the single biggest reasons expounded for NOT using solar of wind power: Politicians have claimed for decades that we should not use solar or wind, because we can't run cities like DC on solar and wind. Therefore, it is worthless technology.

I'm going to hand key the stats below to seed the search engines:

Coal 51.2%
Oil .3%
Natural Gas 6.4%
Nuclear 33.2%
System Mix 4.3%
Renewable Energy
* Captured Methane Gas .3%
* Geothermal 0%
* Hydroelectric 2.8%
* Solar 0%
* Solid Waste .1%
* Wind 0%
* Wood or Biomass 1.5%

Unraiding a Mirrored LVM Physical Volume

Here's the problem I've got: I have a mirrored RAID-1 that acts as the physical layer for an LVM volume group. One of the disk elements in the RAID mirror needs to be permanently removed. We could simply fail that element and be happy, but the question is whether I can convert the LVM to a partition without effecting the logical volume.

Be fore warned: we'll need to boot to get this to work.

Initial config:

[root@gfs1 ~]# ls -l /mnt
total 16
-rw-r--r-- 1 root root 0 2009-06-01 11:32 hello-there
[root@gfs1 ~]# pvscan
PV /dev/sda2   VG VolGroup00   lvm2 [4.88 GB / 2.59 GB free]
PV /dev/md0    VG changeme     lvm2 [4.99 GB / 3.99 GB free]
Total: 2 [9.87 GB] / in use: 2 [9.87 GB] / in no VG: 0 [0 ]

We want to change /dev/md0 to /dev/sdb1 without killing the hello-there file.

First, we remove the disk we want to keep from the array. We're doing this because the array is going to be destroyed.

[root@gfs1 ~]# umount /mnt
[root@gfs1 ~]# mdadm /dev/md0 -f /dev/sdb1
mdadm: set /dev/sdb1 faulty in /dev/md0
[root@gfs1 ~]# mdadm /dev/md0 -r /dev/sdb1
mdadm: hot removed /dev/sdb1

To prevent the RAID from restarting at boot time, we need to change the partition type from FD to 8E. Use fdisk, per SOP, or...

[root@gfs1 ~]# echo -e "t\n 8e\n p\n w\n" | fdisk /dev/sdb

Let's we nuke the raid. To accomplish this we are going to change the partition type from FD to 83.

[root@gfs1 ~]# echo -e "t\n 83\n p\n w\n" | fdisk /dev/sdb
[root@gfs1 ~]# echo "y" | pvremove /dev/md0 -ff

Notice we are setting this to a standard partition rather (83) rather than LVM (8e). This is because we do not want the kernel initializing the partition in the next step. Now, we do our first reboot.

During the boot sequence, neither disk will be recognized as a RAID element. The disk we want to keep will be a physical volume, because we tagged it as 8E. The disk we want to loose will appear to be an unformatted partition.

When the boot is complete, log back in, and check the LVM status:

[root@gfs1 ~]# pvscan

One of two things happened: You got lucky and your volume is displayed-- jump to the last step. Nine out of ten times, you have to reinitialize the volume with the correct UUID. You did record the UUID before you started... didn't you?

Oops.

No worries. Try this: (remember your UUID will be different.)

[root@gfs1 ~]# vgcfgrestore changeme
Couldn't find device with uuid '7ZXhzB-Bsm0-w9be-cu57-EPDx-xk4Q-f9vHdv'.
Couldn't find all physical volumes for volume group changeme.
Restore failed.
[root@gfs1 ~]# pvcreate --uuid '7ZXhzB-Bsm0-w9be-cu57-EPDx-xk4Q-f9vHdv' /dev/sdb1
Software RAID md superblock detected on /dev/sdb1. Wipe it? [y/n] y
Wiping software RAID md superblock on /dev/sdb1
Physical volume "/dev/sdb1" successfully created
[root@gfs1 ~]# vgcfgrestore changeme
Restored volume group changeme
[root@gfs1 ~]# pvscan
PV /dev/sdb1 VG changeme lvm2 [4.99 GB / 4.89 GB free]
PV /dev/sda2 VG VolGroup00 lvm2 [4.88 GB / 2.59 GB free]
Total: 2 [9.87 GB] / in use: 2 [9.87 GB] / in no VG: 0 [0 ]
[root@gfs1 ~]# vgscan
Reading all physical volumes. This may take a while...
Found volume group "changeme" using metadata type lvm2
Found volume group "VolGroup00" using metadata type lvm2
[root@gfs1 ~]# lvscan
inactive '/dev/changeme/keepme' [100.00 MB] inherit
ACTIVE    '/dev/VolGroup00/LogVol00' [2.00 GB] inherit
ACTIVE    '/dev/VolGroup00/LogVol01' [288.00 MB] inherit
[root@gfs1 ~]# lvchange -ay /dev/changeme/keepme

Mount the logical volume, and your done.

The On-Call Blackberry

I got wrangled into having to use a Blackberry as part of an on call support gig. I'd never used a Blackberry before, always having carried Palm devices. This particular one is a RIM BlackBerry 8830.

...and it sucks!

The interface is absolutely horrible. It's like using an original Mac with a trackball-- today, not twenty years ago when the original Mac and trackball were breakthrough technology. Why doesn't it have a touch screen? How 1990's.

Even the apps are cluncky. How is it that people think these things are useful, helpful, cool, or intuitive? Palm is five years ahead of this piece of crap, and they are two years behind the iPhone. Even the MS Windows Mobile (WCE) systems are better than this. And to think that RIM has been the business package of choice.

I just don't get it.

Atomic Veterans

A friend at work mentioned that his father had "claimed" to have take part in atomic bomb tests in the 1950's , where unprotected soldiers were exposed to radiation. He wasn't sure he believed the story. I found this video on YouTube of the mission: Operation Buster-Jangle.

Yep, it was true. Not real smart, but true.

(Unfortunately, They won't let this one be embedded.)

Cloud Storage Connect

This script will allow cluster nodes to connect to each other's ISCSI shares to create a cloud storage platform. I assume a high level of experience going into this. Maybe a howto should follow.

Carve out free space as a partition on each system. Define the partition as a Physical Volume in the Volume Group cloud. Share the partition via ISCSI (tgtd service), and start the service on all nodes. Execute the following on each node:

#!/bin/sh
#
# join nodes of a GFS cluster via iSCSI
#

if [ "$1" == "join" ]; then
  ACT="login"
elif [ "$1" == "leave" ]; then
  ACT="logout"
else
  echo "Usage: iscsi-cloud join|leave"
  echo "GPL 2009, dougbunger"
  exit 99
fi

grep "clusternode.*name" /etc/cluster/cluster.conf > /tmp/iscsi-nodes
sed -i "s/nodeid=.[0-9]*//" /tmp/iscsi-nodes
sed -i "s/votes=.[0-9]*//" /tmp/iscsi-nodes
sed -i "s/sed -i "s/[ \"=<>\t]//g" /tmp/iscsi-nodes

  ME=`hostname | cut -d\. -f1`
  for J in `grep -v $ME /tmp/iscsi-nodes`; do echo "$ACT node $J:"
    # discover
    L=`iscsiadm -m discovery -t sendtargets -p $J 2> /dev/null`
    T=`echo $L | cut -d\ -f2`
    echo "...$T"
    if [ "$T" != "" ]; then
      P=`echo $L | cut -d, -f1`
      echo "...$P"
      # bind
      iscsiadm -m node -T $T -p $P --$ACT
    fi
  done
vgscan
lvscan

exit

Restart the clvmd and lvm2-monitor service on all nodes. From any node, execute:

vgdisplay cloud

This will show the total space available across the cloud. Turns our system-config-lvm is cloud aware, and will keep all nodes in sync when creating or adjusting LVs.

Physical Layer Monitor

Here's a little script I whipped up to solve an argument as to whether the server was going to sleep or no packets were being routed into the box. The echo line could be tee'd for historical reference:

#/bin/sh
TRUE=1

while [ $TRUE = 1 ]; do
  killall tcpdump 2> /dev/null
  mv /dev/shm/netwatch.txt /dev/shm/netwatch-b.txt
    tcpdump > /dev/shm/netwatch.txt 2> /dev/null &
    IP=`grep -c " IP " /dev/shm/netwatch-b.txt`
    ARP=`grep -c " arp " /dev/shm/netwatch-b.txt`
    echo "`date +%T` packets=$IP arps=$ARP"
  sleep 10
done

Domaine Trois Freres Chardonnay

My son, the chef in training, asked for a white wine to cook chicken. I opened a bottle of a cheap Italian white, but it had "corked". Living by the philosphy that if a wine is not good enough to drink, its not good enough for cook with, he poured the bottle down the drain.

Next in line was this nondiscript French chard. It had a light flavor, perhaps just a little more acidic than I'd prefer, but far less than a Chilean. On par with New Zealand. The wine is a respectable 5 of 10, but the chicken was fablous.

Montresor Soave

My standard soave. Not as smooth as the Bolla, but simple, unlike some of the others that are more golden. An enjoyable 5 of 10.

Taurino Salice Salentino

This southeast Italian rosso was too heavy for my tastes. It reminded me of an earthy chianti. This was a riserva 2004, very similar to the Ruffini Il Leo, maybe a dollar cheaper. 4 of 10.

Squid Reverse Proxy Server

I have a small problem for which a Squid reverse proxy server was the obvious problem, but I had the hardest time getting this to work as advertised. Why was it difficult? Undocumented options.

Here's the problem: I've got three webservers with identical content. I want eveybody from comcast.net to hit one, verizon.net to hit the second, and everybody else to hit the third. Squid can be configured to grab the requests, do the lookup on the domain, and forward them to the correct server. Here's a snippet of the configuration that worked:

http_port 80 accel defaultsite=gfs2.terran.lan
cache_dir null /null

acl first  src comcast.net
acl second src verizon.net
acl others src 0.0.0.0/0


cache_peer 192.168.69.59 parent 80 0 proxy-only \
  no-query no-digest originserver name=gfs3
cache_peer_access gfs3 allow first
http_access allow first

cache_peer 192.168.69.60 parent 80 0 proxy-only \
  no-query no-digest originserver name=gfs4
cache_peer_access gfs4 allow second
http_access allow second

cache_peer 192.168.69.61 parent 80 0 proxy-only \
  no-query no-digest originserver name=gfs2
cache_peer_access gfs3 allow others
http_access allow others

Yummy, Delicious, tar

Create an archive, based on date. The man page says use:

-N, --after-date DATE, --newer DATE

But what's the format for date? Further, if you try:

tar -czvf --newer "11 Mar 2009" www-incr-0311.tar.gz \
    www

...it creates a bogus file, --newer, which is a bear to delete.

(Try it. Frustration is good for the soul. The trick is to use the absolute path to file.)

Turns out, with tar, the dash-dash options must go at the end of the command line:

tar czvf www-incr-0311.tar.gz www \
    --newer "11 Mar 2009"

As for the date format, I decided to use mtime and an all numeric format:

tar -czvf www-incr-0311.tar.gz www \
    --newer-mtime "20090301"

Updating XenServer

I'm a little disappointed that the XenCenter management platform for XenServer only runs under Windows, but then VMware VIC only runs under Windows, so I shouldn't really expect much from them. What's more odd is the way the XenServer is updated. Since the box is a bastardized version of RHEL 5, I expect Yum to be the update mechanism. Here's how it works, instead:

1. XenCenter polls Citrix for updates and provides a list.
2. Items are downloaded via an IE download manager plug-in to the Windows station.
    (Select "Save", not "Install"!!!)
3. Navigate to the download folder, double click the xsupdate file.
4. XenCenter will launch with a system update wizard.
5. Select the servers to be updated.
6. Sit back and watch all the pretty colors flash by.

Just don't like having to use Windows to update Linux.