The break fix of patching

The other day in New Orleans I talked about how I hadn’t changed my patch management strategy in 6 years.  Never patch on Patch Tuesday, wait for Dead Body Wednesday  and don’t install service packs until later.  This “patch management” strategy is the “managed services model”.  When I was speaking with an attendee (and forgive me I can’t remember  who I was talking to on this) and they said they were installing patches the other day and it was a bit difficult to break them up and know what issues were caused with what patch as they were installing like 64 of them.

“64?” I said, “why so many?” And it was because they were a break fix client they said. 

Bear with me here… and I don’t mean to sound like you shouldn’t be aware of the consequences of what I’m about to say, but I would like to throw this out on the table.

If a client is a break fix client, I think you do a disservice to that client to not enable automatic updates.  The patching process works the best when you take patches in monthly chunks.   Installing 64 patches in one afternoon, if you did have some interaction, you’d never figure out which one of the 64 patches was the one that caused the interaction.

While you could easily argue with me that turning on automatic updates on a break fix client was a disservice to that client, I think NOT enabling automatic updates on a client that you only see once a year is also a disservice.

At a minimum I think you should ensure they sign a document that indicates they know you are leaving with updates turned off and that they understand that they are accepting the risks of that setting.

So what do you think?  I think if you don’t see that client, that they should take the risk of automatic updates versus the risk of 64 patches in one afternoon.  Patches are best consumed in little bits.

4 Thoughts on “The break fix of patching

  1. Dean on May 30, 2007 at 7:23 am said:

    I agree that you should not leave automatic updates off for a year but I disagree that yo should not install all the patches at once. For one thing after a year either the patches are going to work or they would have been pulled. Second who’s going to divide 64 patches into say 6 sets and waste all that time over how many weeks. Third Microsoft has so broken automatic updates trying to fix the Svchost issue that they need to scrap it and start over. It used to work just fine not it’s a piece of shi_ .

  2. Craig on May 30, 2007 at 10:06 am said:

    In the same situation, I recommend Automatic Updates is turned on. Set it to install on Tuesday at 3am and you will have a full week after a patch tuesday to alleviate the risk of installing on the first few days.

  3. I am in an ethics discussion about the best approach at the moment.

    A stable network only becomes unstable when a change occurs be it hardware or software. This we all know.

    If the customer has a stable but unpatched network, and you patch it (because that is best practice) causing the network to become unstable, would the customer be entitled to refuse to pay for your time as the changes you introduced caused them a loss in productivity?

    Would you pay for your car’s annual service if afterwards it kept misfiring? Would you want loss of earnings compensation if you had to take time off work to get it back to the garage?

    On the other hand –

    If the customers stable but unpatched network became compromised due to an employee going to a non line of business website, the customer could blame you for not installing the patches (despite the employee not adhereing to company internet usage policy).

    The only way out is to get the get the customer to choose the option that they feel happiest with after pointing out the risks of all the options, get them to sign on the dotted line, confirming that understand. I would recommend that patching fullstop be treated as a seperate hourly billable service.

    It would be nice if MS could write a patching system that does not launch a denial of service attack on each workstation (100% CPU usage) – most hackers would be proud to have such a system wide result!!!

  4. I agree with you in principle, but there are always exceptions:

    1. It’s the first time you visit a client and they’ve never installed a patch, ever. What do you do? Turn on AU and run? Or leave them turned off?

    2. Your client has ageing hardware and Microsoft breaks the AutoUpdate client so that turning on AutoUpdates makes you public enemy number 1 every day for a couple of hours when everyone’s computer is frozen.

    In an ideal world it is a no-brainer that auto updates should be enabled, but a baptism of fire is not a good experience for anyone. I woudl say, if the customer can be persuaded to let you install and manage WSUS for them, then you can trickle the patches into service in a manageable way. Otherwise, I think the greater service is to leave AutoUpdates alone and let them learn their lesson the hard way.

Post Navigation