Fall 2003
|
I'll bet you did not know that servers were built for more than just serving! They can serve other purposes, like say, being a sponge.
(Thanks to Alan for taking pictures of this humerous event.)
In order to keep the heat and noise in my house down to a minimum, I moved my file/web/email server, nicknamed Ogre, under the house. It's a 4 section server, one main CPU unit (with 6 drives) and 3 additional drive arrays, totalling over 40 SCSI drives. Everything is built for fault tolerance and there are also multiple battery backups to keep it running. The crawlspace seemed like a good fit since I could get network and power to the server and still secure it from outside influence.
It just so happened that there was a massive rainstorm one morning. We had several inches of rain in the span of a
few hours and I noticed something was up when my Winamp stopped playing music. Checking the circuit breakers showed that the crawlspace was having issues.
I decided to go underneath and have a look...
What I found was pretty awful/amazing. The water had flooded my crawlspace... about 18 inches deep at several places. The lowest thing on the makeshift rack was a battery backup. That unit was affected first by the water and caused the power short. The drive array directly above the battery, now powered off, was then submerged... Soaking fans, power supplies and 10 drives. I managed to get all of the wet components out from the crawl space and out where I could dry them off. I took the drives into work where I could use compressed air and some fine tools to get all of the dirt and film off the electronics. On my way out the door, I dropped one of the drives on the concrete... I saw some resistors come off of the drive, so that one was now dead. The remainder I was able to clean at work. Whew.
Getting it all put back together, with a single dead drive, nine cleaned drives, a new drive chassis (borrowed from work), and
a new battery backup (Best Buy rules!) was loads of fun. I was able to get the system up and running again without too much trouble. The drive array came
back online and scanned clean! No loss of data! Wooohooo! I purchased a few tables to give Ogre a few more feet of elevation. Now I've got a fairly
stable server and a much quieter house. No more rain problems.
Photo-quality links: | 
|
