Guarded Mutexes

I’ve been in an ongoing conversation with some folks about the new Guarded Mutexes that are being used in Windows Server 2003 and higher. They’re a little bit different than previous synchronization mechanisms, so they deserve their own consideration.

Guarded mutexes were introduced into the memory manger of Windows Server 2003 as a performance optimization over FAST_MUTEXes and other synchronization primitives. As you know, in order to acquire a fast mutex, typically the IRQL must be raised to APC_LEVEL, or else the driver must be in a critical region. Otherwise, the potential for deadlock exists due to APCs. The two mechanisms of disabling APCs are different in one respect: raising to APC_LEVEL disables special kernel APCs.

However, as OSR’s Tony Mason, points out, raising the IRQL to APC_LEVEL requires an expensive access to the APIC on a multi-processor computer, leading to a performance hit. In order to address this, Microsoft has introduced guarded mutexes, which merely disable all forms of APCs without adjusting the IRQL.

The following are some guarded mutex functions exported by my 3790 XP-64 kernel:


lkd> x nt!*guarded*
fffff800`00843bb0 nt!KiWaitForGuardedMutexGate = 
fffff800`00855970 nt!KeAcquireGuardedMutexUnsafe = 
fffff800`008266f0 nt!KeEnterGuardedRegion = 
fffff800`0081ddc0 nt!KeInitializeGuardedMutex = 
fffff800`00855940 nt!KeReleaseGuardedMutexUnsafe = 
fffff800`00935e80 nt!WmipTraceGuardedMutex = 
fffff800`0081dd60 nt!KeAcquireGuardedMutex = 
fffff800`008267a0 nt!KeTryToAcquireGuardedMutex = 
fffff800`0081dcb0 nt!KeReleaseGuardedMutex = 
fffff800`00826710 nt!KeLeaveGuardedRegion = 

…and the structure they operate on:


lkd> dt nt!_kguarded_mutex
nt!_KGUARDED_MUTEX
   +0x000 Count            : Int4B
   +0x008 Owner            : Ptr64 _KTHREAD
   +0x010 Contention       : Uint4B
   +0x018 Gate             : _KGATE
   +0x030 KernelApcDisable : Int2B
   +0x032 SpecialApcDisable : Int2B
   +0x030 CombinedApcDisable : Uint4B

A little imagination will let you guess approximately how to use these new routines, although they don’t show up in my DDK or IFSKit.

There are a couple of interesting programming considerations that guarded mutexes raise. The first is that it is now no longer sufficient to test the IRQL for APC_LEVEL if you want to see if special kernel mutexes are enabled. The main impact I’ve seen so far is that it’s not safe to call Zw* file management routines or send IRPs to filesystems just because the IRQL is PASSIVE_LEVEL. You must also test to make sure that special kernel APCs are not disabled.

The even better part is that, as of today, there is absolutely no way to make that test. There is a field in the KTHREAD structure that controls whether or not special kernel apcs are enabled:


lkd> dt nt!_kthread
   ....
  +0x0bc KernelApcDisable : Int2B
   +0x0be SpecialApcDisable : Int2B
   +0x0bc CombinedApcDisable : Uint4B
    ....

…but there is no exported API to test that field. Testing it directly is a bad idea, of course, so what’s a programmer to do?

The simple answer, as far as I’m concerned, is to just post all file I/O off to a worker thread if there’s any chance at all that you could be getting called from the memory manager. Someday there will be a new API (According to Neal Christiansen from Microsoft, someday == Srv03 SP1), but until then, better safe then sorry.

2 Replies to “Guarded Mutexes”

  1. Are you sure about that?

    lkd> u nt!KeAreApcsDisabled

    nt!KeAreApcsDisabled:

    8050f74d 64a124010000 mov eax,fs:[00000124]

    8050f753 83787000 cmp dword ptr [eax+0x70],0x0 ; <— tests a ULONG and not a USHORT!

    8050f757 0f95c0 setne al

    8050f75a c3 ret

    lkd> dt nt!_KTHREAD

    .

    .

    .

    +0x070 KernelApcDisable : Int2B

    +0x072 SpecialApcDisable : Int2B

    +0x070 CombinedApcDisable : Uint4B

    It seems as if KeAreApcsDisabled won’t return 1 unless both kernel and special kernel APCs are disabled, (despite what the documentation says; I guess it’s now wrong, since the Win2003 DDK claims special APCs will still be delivered…). This is on Windows Server 2003 SP0 plain-x86.

Leave a Reply

Your email address will not be published. Required fields are marked *