Tracking SSH Tunnels

Native to Secure Shell (SSH) is the ability to create point-to-point, encrypted, tunnels.  The function was designed to provide legacy protocols, such as mail (SMTP/POP) with encryption.  A user could login to an SSH server in their company's DMZ, open a tunnel from their laptop to the server, and redirect their mail client through the tunnel.  On the surface, this sounds like a good idea: it protects the exchange of company data from the "protected" corporate intranet to users "in the field".

But, as with all good things, there is room for abuse.  Consider the opposite scenario:  What if a user inside the corporate intranet SSH'ed to the DMZ server and built a tunnel to allow them to surf the web, thus bypassing the content filters?

Granted, content filters are just a way for the man to oppress middle class workers.  By censoring free thought, the 1% is able to keep the 47% running on the hamster wheel of consumerism.  Hear me, my brothers!  There will come a day when the proletariat will raise up and declare their freedom from the jack-booted thugs of Wall Street and their Illuminati masters.

But I digress...  Where was I?  Oh yes, SSH tunnels. So the question is this:

How can we monitor the SSH tunnels defined on the server to ensure they are not being abused?

Much to my surprise, the answer is:  You can't.

There does not seem to be any mechanism for determining what tunnels exist, and here's why.  The tunnel is defined on the client end, where the SSH application is always listening.  When the client receives a packet that matches a tunnel, the packet is shipped to the server with handling instructions.  When the server gets the packet, it opens the needed socket, fires it off, then closes the socket.  In other words, the connection from the server to the destination is not persistent... it behaves more like UDP than TCP.

Since a socket is opened, it is possible to capture it with lsof -i, but since the socket is transient, trying to catch it in a while/do loop is a matter of pure luck.

This means we have two choices, one of which shouldn't count.

In order to catch someone using a tunnel to surf out of the DMZ, we need an IPtables rule to catch the outbound packets.  As it turns out, any packet originating from a tunnel will use the server's IP address as the source address.  We only need to log the initial connect, so we only need to log the SYN flag.  To further complicate things, our abusive user has to be using a proxy, so we can't restrict our checks on port 80 and 443.

iptables -A OUTPUT -s 192.168.0.1
  -o eth0 -p tcp --syn
  -j LOG --log-prefix "out-syn "

Here, we are looking for OUTPUT, since we are assuming that this DMZ machine is supposed to be building tunnels.  The (-s) address is the address of the DMZ machine.  In this case (-o) eth0 is the internet side of the machine and eth1 would be the intranet side of the machine.  Notice that no port number is assigned to the (-p) TCP statement.  Lastly, we are going to log this message.  (The trailing space in the quotes is significant.)

This rule will catch bad tunnels, but ignore good tunnels, on the grounds that good tunnels will use (-o) eth1 to get to the intranet resources.

If you'll recall, I said there were two choices.  The second is this:

iptables -A OUTPUT -s 192.168.0.1
-o eth0 -p tcp --syn
-j DROP

In this case, we are refusing all outbound TCP traffic from the DMZ machine.  (Since DNS is UDP, we can still resolve the addresses of the inbound SSH connections.)  As stated above, we are allowing the good tunnels, since they use (-o) eth1.

So which of the two rules shouldn't matter?  The first:  We shouldn't have to "catch" abusive users, we should just stop them.  Of course, we could use both lines to first log them and, second, prevent the connection.  This allows us to know who the abusers are, and bitch slap them for their feeble attempt-- for they are probably using Windows workstations, and deserve to be degraded.

What's that you say Mr. Boss?  You want me to prove abuse exists before locking down the DMZ.  Okay, we implement rule number 1, log the abuse, and then later lock down with rule number 2.

What's that you say Mr. Boss?  Prove the abuse exists without implementing rule number 1.  Ah...  No can do.

Oh well, if you want me, I'll be in my cube.  Listening to Slacker internet radio, via an SSH tunnel, through the DMZ.

Good news, bad news

First, the good news: the ceiling is fixed...

..and the floor is done.

The bad news? Still no kitchen.

And on a side note, ML asked if any of the walls I knocked out were "load bearing". So far, it seems not. I guess that's good news too!

The Peter Principle

Everybody knows The Peter Principle as "people get promoted to their level of incompetence."

Unfortunately, that's wrong.

The Peter Principle is the title of a book published in 1969, by Dr Laurence J Peter and Raymond Hull.  Dr Peter did the research, Hull ghost wrote book.  I recently picked up a copy from The Book Thing free store in Baltimore, that was printed in 1970.  The front cover states:

In a hierarchy, every employee tends to rise to his level of incompetence.

Unfortunately, that's not really the principle, either.

The problem is this:  These concepts are introduced and explained on page seven of the book, but the book is so poorly written, that even though it was a run-away best seller, no one actually read it.  They bought it, talked about it at cocktail parties (remember... this was 1970) but they didn't actually read it.  They read the cover and mutually agreed with each other's lies of understanding. 

Its kind of like going to Las Vegas: no one want to admit they are the only person in the world who didn't get their room "comp'd", so they lie.  Thus the of "the Las Vegas comp" is a self perpetuating fable.

In reality, The Peter Principle is about the behavior of human systems, which Dr Peter calls "a new science, hierarchiology, the study of hierarchies," and the level of incompetence thing is one small part of the pie.

Have I finished the book?  No, but I am struggling through the text.  At this point its a challenge.  I shall not be defeated by a small tome of yellowed pages!

And what wonders been gleaned from my pain and suffering?  Thus far, only that Dr Peter is an elitist pig.  A slightly observant and insightful elitist pig, but a pig none the less.  My evidence?  From page 44:

[He] managed, by hard study, to master a foreign language.  It is quite possible that he would have to fill one or more posts in the company's overseas sales organization before being brought home and promoted to his final position of incompetence as sales manager.  Study created a detour in [his] hierarchal flight plan.

In other words, be a good drone and do what I tell you.  You will ultimately fail in life and efforts at self improvement will only delay the inevitable.

The Silicon Valley people have a new saying: "Fail fast.  Fail cheap."  (I hate Silicon Valley people.)  The logic is that if you are going to fail, it is better to do it sooner than later.  This use to be called "cutting your losses."  For Silicon Valley people, its about recognizing failure on the horizon, accepting that you have over-reached, and moving on to your next meal ticket.  But this is not what Dr Peter is proposing. 

For Dr Peter, it is about recognizing that you will fail, giving up on any chance of enjoying what you're doing, and moving as quickly as possible to the job that you will inevitably hate, where you will spend the rest of your life being ineffective, and suffer the scorn of your co-workers.

Wow.  I just realized that not only is this book poorly written, but its depressing, too.  No wonder nobody read it.

Stupid Yahoo Password Criteria

For about a week, I've been wrestling with my Yahoo! password.  My old, but still functional, Palm Centro mobile phone has an app to connect to Yahoo mail, but it recently stopped working.  Given that it failed the day after I changed my password, one might claim that it was a self inflicted injury, but no...  it was Yahoo's fault for storing 450,000 passwords in clear text which, of course, got hacked and published.

The smart thing was to change the password.  What Yahoo failed to explain was that in order to be able to login to your account on their mobile site, you have reset your password from a desktop computer, using the password requirements for the mobile site.  Unfortunately, the password criteria checker they use is Javascript, and it is not configured with the password criteria used on the mobile site.

Bottom line:

You can use special characters !@#$, but not %^&*.

My password contained the percent sign.  I could login from my Windows and Linux machines using IE or Firefox, even using the m.yahoo.com URL to force the browser to the mobile site.  I could not login from my Palm Centro across the SprintPCS network using either the mobile browser or mail app.  Just to prove that this was not a Palm problem, I also could not login from my Android E-reader tablet.

As soon as I changed my password to use use a "good" special character, rather than a "bad" special character, all previously denied devices worked.

Use awk To grep "this but not that"

I've run into this situation several times over the decades, bit for some reason I never researched an elegant solution. Consider the case of grep'ing to see if a process is running. The simple solution is:

ps -ef | grep "ntpd" 

The problem is that if there is one process matching the regex, this will report two processes, because it will also report the grep process that is grep'ing the process stack. Its kind of like taking a picture of yourself in the mirror. The generic solution to this is:

ps -ef | grep "ntpd" | grep -v "grep" 

In other words, lets launch another grep to grep out the grep. This is only slightly less efficient than taking a picture of yourself in the mirror, then Photoshopping the camera out of the picture.

Today, I found the elegant solution, and its... awk to the resuce!

ps -ef | awk '/ntpd/ && !/awk/' 

Here, awk is taking the stream and searching for a line that has ntp and (&&) does not (!) have awk.

Ceiling Fan Effectiveness

While investigating why my home is not cooling evenly, I ran a quick test.  Using my infrared thermometer, I ran some scans on two rooms.  The test was done at noon, with the air conditioner enabled, set on 85.  The outside temperature as measured by the air conditioner (in the shade) was 89.

In room one (R1) the temperature at the junction of an outside wall, an inside wall, and the ceiling, was 85.9 degrees Fahrenheit.  In the room two (R2) the temperature at the junction of the same outside wall, same inside wall, and same ceiling, was 86.4 degrees.  Needless to say, a temperature variation of half a degree might seem negligible, certainly within an acceptable margin of error, except for one thing:

In R1, the ceiling fan had been running four hours.

This raises an interesting question, if running the ceiling fan for four hours only cools the room by half a degree, is it really working?  Yes, because its not a ceiling fan's job to cool a room: its job is to circulate air.

But wait!  It get's more interesting. 

The temperature of the fan's motor housing was 97 degrees.  This means that the fan was acting as a heat source.  Of course the ceiling fan is equipped with a light kit.  With the light illuminated for one hour, the temperature of the globe around the single incandescent bulb increased from ambient to 95 degrees.  This means the light was also acting as a heat source.  (The four bulb "tulip" light kit in R3 was 103 degrees, using CFL bulbs.  The three bulb halogen fixture in R2 was 118 degrees.)

This experiment causes me to question the value of ceiling fans.  In both R1 and R2 the temperature at the baseboard was 81 degrees.  Conventional wisdom dictates that the fan in R1 would pull cooler air up, or push hotter air down.  By circulating the air, the room would be more evenly cooled.  Yet, the measurements indicate the effect of circulation is not significant.  On top of costing electricity to operate, its possible that the fan is adding heat, not subtracting.

So why do people use ceiling fans if they don't work?  One word: breeze.  People think ceiling fans work, because they can feel a breeze, which seems to have a cooling effect.  (In reality, a breeze is only effective on bare skin because it assists evaporation of perspiration.)

Thus, I will conclude with this philosophical question:  

If a ceiling fan is running in a room,
and there is no one around to feel it,
does it do any good? 

No.

Tear Down The House

To those that suggested that the solution to uneven cooling of my home was to tear the house down... I'm way ahead of you. A couple weekend ago, I took out a ceiling and two walls.

The ceiling had to come out:  There was a leak in the upstairs plumbing that had caused the ceiling to sag in two places.  Once the leaks were fixed, a the ceiling was "repaired" by a couple of "professionals".  And then it sagged again.  So, out it went.

And while you're taking out the ceiling, might as tear down some walls.  The red line represents a wall that separated the everyday dining room from the formal dining room, which has spent the majority of the last five years as a storage room.  The purple line represents the wall that separated the kitchen from the formal dining room.  The yellow circle is around the light switches, that were in the walls, and now just hang down for easy reach.

The big black thing in the background is the Big Fucking Refrigerator (BFR).  The difference between a BFR and a refrigerator is that a BFR damages doors and walls when it goes in the house.  The BFR did sit in the corner where the red and purple walls had been.  Now its against the outside wall.

Home Not Cooling Evenly

I have a running joke about my house:  I think they installed the insulation backwards, because in the winter, the house is colder then it is outside, and in the summer it is hotter.  Though not completely accurate, I have always been perplexed by the thermal dynamics of this property.

Lets start off with some background.  Like most of the Washington, DC, area (and east coast cities) my home is a "town" house.  Generally speaking, a town house is wider than a "row" house.  A town house is typically two rooms wide, whereas a row house is one room wide.  In both cases, the home would be several stories tall and several rooms deep-- potentially half a city block deep.  (In Tennessee, a house that is one room wide and several rooms deep is called a "shotgun" house, because you could shoot your shotgun in the front door, and blow out the back door.  Sigh...  Tennessee was such a fun place.)

In my case, when you walk in the front door, you can turn right to the living room, walk down a flight of stairs to the master suite, or walk up half a flight to the kitchen, dining room, family room.  So, that's two and a half stories.  From the downstairs master suite, you can go down another level to the computer bunker (ie: basement.)  From the kitchen, you can go up a story to more bedrooms. Thus, we're at four and a half stories.  Turns out there is enough room in the roof, for another 15x17 room, but its never been built out.

The dominate feature of the town house, is the pseudo-spiral staircase.  I call it pseudo-spiral, because its a rectangular box, the size of an elevator shaft, running from bottom to top.  There are three stairs, a set of four stairs at 45 degree angles, then three stairs, and two stairs at 45 degrees.  (If your not good at geometry, you just climbed one story and turned 270 degrees, so you are facing to the left of where you started.)  There is a small landing, and then another sequence of steps.  Repeat this once down and twice up.

I call it the M.C. Escher house:
 

But here's what you really need to know about the house:  Using my new Harbor Freight Non-contact Laser Thermometer, I have verified that at the lowest climate controlled point of the house, to the highest climate controlled point in the house, there is a thirty degree temperature variation.

Yes, that right: 30 degrees Fahrenheit!  With the thermostat set on 85 degrees, and an outside temperature of 75 degrees, the baseboard of the master suite is 65 degrees.  The largest bedroom upstairs has a 12 foot vaulted ceiling.  At the peak of the ceiling, the temperature is 95 degrees.  And these are ambient temperatures-- the AC is not running!

First questions: Why such a huge temperature variation?  Simple answer:  Heat rises, so the stairwell acts as a ventilation silo that allows all the hot air to rise, and cool air to sink.

Second question:  How do I fix it?

And so we begin...

Harbor Frieght Infrared Thermometer

This weekend, I bought a Harbor Freight Non-contact Laser Thermometer.  I'll explain why in a later post, but I gotta say, I'm impressed with how well this thing performs... especially since it costs $39.99... and I had a 20% off coupon!  :)

First, the laser doesn't "do" anything--  its actually just a pointer to help aim the thing.  Its really a focused infrared measurement device.  The instructions state that it has an accurate range of about 8 feet.  After that, the accuracy begins to drop, but from my minimal experimentation, the loss of accuracy is negligible.  Truthfully, if its accurate within +/- five degrees, it will be worth the price.

The device uses one nine volt battery, which gets depleted fairly quickly. 

Note:  The instructions do not explain how to install the battery.  The front part of the hand grip (black part) slides down.  At the top of the grip, near the trigger, there are four ridges on each side.  Hold the top of the "gun" in your left hand, squeeze the ridges with the thumb and forefinger of your right hand, and slide the grip away from the top of the gun.

 Here's some backyard fun.  With fire!

Flames: 598 degrees Fahrenheit

  Almost ready...

Coals: 946 degrees Fahrenheit

 Sure beats licking the coals to see if they are hot enough.

Welcome Back

David and Paul both complained, separately, that I had not updated my blog in a while. I didn't realize that it had been six months. Time flies when you're having fun... Not!

You see, most of my posts are based on R&D for work related projects, or humorous (read: absurd) events that happen during the day. Recently, much of my day is tied up with logistical silliness. Nobody wants to read a blog post about delivery guys getting mad at me because I wont accept a server rack, because the inventory team won't give me permission to bring it in the building, because my boss didn't get their approval on the rack before he bought it. Boring.

If I wanted to write boring posts that nobody cared about and would not enrich the lives of humanity, I'd post to Facebook.

Having said that, I've started a few personal projects that might be interesting.