We make a huge thing about making sure that we build in backups, disaster recovery, redundancy, but there’s one thing that unless you have your own true backup and redundancy, you only have about 30 minutes that separates you between all the technology at your fingertips and whipping out a Dixon Ticonderoga.
Today at about 3:45 p.m. all of a sudden all of our battery backups on all of our workstations and servers starting madly beeping even though the power was still on. It appeared that we were having a bit of a brown out and our battery backups were kicking in to make up the difference. [And yes, we’d found that buying EVERY workstation a battery backup is cheaper than possibly losing a spreadsheet or project and the power goes out. We make sure there’s even a backup battery on the phone system and the network switch so we can quickly save and shut down our workstations. About 4 p.m. the power straightened up and we went on with our day.
Well tonight about 7:30, we’re working along and “BEEEEEEEPPPPPP” there goes the power again. While we did have the functional battery backups, I found we needed one more thing. Emergency power lights. We have one in the office that turns into a flashlight, but obviously not enough. I found myself walking down the hallway with one hand on the wall visualizing the doorways and openings to get to where the emergency flashlight was. We shut down the server fully this time since we felt that the power truly was going to stay out for a bit this time.
Check those battery backups… yank the cord of the UPS from the wall and make sure that your workstations and servers stay up ….long enough for you to turn them off.
The moral of this story is that for all our worrying about “up time” and “true redundant servers” and “ redundant DSL connections”, sometimes what you really need most in the world is just a flashlight so you can go turn off the server and go home early for the night.