I want to be clear, the problems with the website yesterday were my fault. I broke all sorts of cardinal rules that I would never think of doing on a client’s server, and completely wrecked mine (temporarily!).
You know what they say about the shoemaker’s children going barefoot; when Service Pack 2 for Exchange was release I was all over my clients with reasons to implement fast, and they all did. So when Rick and I were trying to configure my Mobile 5 device to sync to my Exchange Server using Push technology, I was embarrassed before my friend by the realization that a year after release my own server was running on SP1. I bowed my head in shame and promised him that it would be fixed that very night.
Of course I am in Ottawa and I am always a little weary about doing real work to production machines when physical access is unavailable; I knew that if I did damage I would not be able to get my hands on the physical box for another two days. However what I had to do was really basic stuff, and did not anticipate blowing things up too badly.
Of course the SP install went well but then without thinking, as I am apt to do with SBS when things change, I reran the CEICW. In this morning’s National Post crossword puzzle there was a clue whose answer was WHOOPS. That is exactly what I felt the second I pressed GO.
For those of you who are (and for those who are not) SBSers you know that the CEICW (Configure E-Mail and Internet Connection Wizard) is a great way to make things right with connectivity. What you might or might not realize is that as soon as you have multiple external IP addresses – a non-standard configuation for SBS – you should disable the CEICW forever, as it will always break things beyond simple repair.
A second tip that I always tell my fellow SBS Admins is that if you have created or altered ANY of your ISA (Internet Security and Acceleration Server) firewall policies, to export these before you run the CEICW, as it alters them often in ways that are unpleasant.
To make things worse I did not have access to my network documentation, so when I started playing with ISA and IIS configurations (on a server that I am not currently the primary administrator) a lot of it was educated guessing. Unfortunately I knew Daniel was on an airplane so I started playing around. Soon my host Bradley Bird (VP, Ottawa Windows Server User Group) came in and we set up two Terminal Server connections and started searching for answers. To our credit we were almost there when Daniel came back on-line, and took us across the goal line to victory.
All in all the website (and more) were down for about two hours Thursday evening, and I take responsibility for it. I also credit Daniel and Bradley for getting it back up as quicly as they did. I also look forward to retiring the current server early in the new year when our new corporate sponsor gives us a faster and more reliable server on which to host MITPro, because then I will be able to break anything I want without affecting the community!
To all of our members who are celebrating this week I want to wish you a Merry Christmas, Happy Channukah, Kwanza, and any other festival I may forget. To all of you I wish you all the best for a happy and healthy New Year and look forward to seeing you all in 2007.