Adjusting Exchange 2010 DAG Failover in High Latency Networks

Some Exchange 2010 DAG implementations have DAG members that are separated by high latency WAN networks.  In these networks you may find that the increased latency causes unexpected DAG failovers or failures. 



This is especially likely to happen with a two node DAG with a File Share Witness (FSW).  When network latency increases to the point that the cluster heartbeat threshold is reached, the node furthest from the FSW will go offline.  The node that’s in the same LAN as the FSW will stay online because it maintains quorum.



There are two properties that specify cluster health, as measured in heartbeats.

  • CrossSubnetDelay specifies the heartbeat interval (in milliseconds) between subnets. The default is 1000 milliseconds (1 second).
  • CrossSubnetThreshold specifies how many heartbeats can be missed between subnets before cluster failover (or failure) occurs. The default is 5 heartbeats.

With the default settings, any WAN latency that causes 5 missed heartbeats over 5 seconds causes the cluster to fail. This matches the settings used by the SameSubnetDelay and SameSubnetThreshold properties.



If WAN latency causes unexpected cluster failover or failures, adjust the CrossSubnetDelay value to its maximum value of 4000 milliseconds (4 seconds) and the CrossSubnetThreshold property to its maximum value of 10,  With these settings the cluster will not failover or fail until there are 10 missed heartbeats, 4 seconds apart (40 seconds).



This is accomplished from Powershell by doing the following:



  • From one of the DAG members open the Windows Powershell Modules in Administrative Tools.  This will launch Powershell and import all the Windows Powershell modules for installed features, including the new Windows Failover Cluster module.
  • Run the following one-liner to configure the maximum values:



$cluster = Get-Cluster; $cluster.CrossSubnetThreshold = 10; $cluster.CrossSubnetDelay = 4000



  • Check your settings using the following cmdlet:

Get-Cluster | fl *




Since cluster properties are instantly replicated between all nodes in the cluster, this only needs to be configured from one node in the DAG.  The changes go into effect immediately and there is no need to restart services or the server.



Note that you can configure the same properties using cluster.exe, but I’m using Powershell here because cluster.exe is deprecated in Windows Server 2008 R2.

How to Prevent WSUS from Offering Exchange SP1 Update Rollups

Microsoft’s servicing strategy for Exchange Server 2010 is to release updates every six to eight weeks.  They do this using Exchange service packs for major updates and update rollups for minor updates.  Update rollup packages are cumulative.  For example, if you apply Update Rollup 5 on a computer that is running Exchange 2010 SP1, you receive all the fixes in that specific update package together with all the fixes that were released in all earlier update rollup packages.



Microsoft recommends using Windows Update to update Exchange Server 2010.  However, the Exchange team always announces the update on the EHLO Blog, along with the download link, before it’s available in Windows Update.  It is usually published to Windows Update about four weeks later.



This causes an interesting issue for “early adopters” of update rollups.  If you install the latest update rollup on a new Exchange server before it becomes available on Windows Update, WU will offer the latest update that’s published.  For example, say you build a new Exchange 2010 SP1 server today and install update the latest update (Update Rollup 6 for Exchange Server 2010 SP1 at the time of this writing) before it’s available on Windows Update.  When you run Windows Update, it will offer Update Rollup 5 for SP1 as an important update.






You will need to ignore the “important” update until UR6 for SP1 is released to Windows Update.  An important thing to know is that if you choose to hide the update in Windows Update (right-click the update and choose “Hide update”) Windows Update will offer the previous update (Update Rollup 4v2 for Exchange Server 2010 SP1).



While this is not the end of the world, I find it annoying to have Windows Update notify me about missing important updates that I don’t need.  It’s also unclear what happens when UR5 installs on top of UR6.  The version number stays the same at 14.1.355.2.



If you find you have new Exchange 2010 SP1 servers that have only UR6 installed, you can prevent Windows Update from offering a lower level update rollup by creating a key in the registry for the earlier update rollup.  Add the following key to the registry on the Exchange 2010 server:





[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Products\AE1D439464EB1B8488741FFA028E291C\Patches\ED0E3CF125E478A45898D3FE635D1A48]



This is the key for Exchange Server 2010 Update Rollup 5 for Service Pack 1.  Only the key needs to exist, no other registry settings under that key are necessary.






Once you enter the key, run Windows Update and check for new updates.  It will no longer offer Update Rollup 5 or any of the previous Exchange 2010 SP1 update rollups on that Exchange 2010 server.

Migrate Your Exchange 2007 Custom Prompts the Easy Way!

There’s a hidden gem in the Exchange scripts folder you can use to easily export custom prompts (AKA Auto Attendants) from Exchange 2007 to 2010.  The script is called MigrateCustomUMPrompts.ps1 and it’s located in the \Program Files\Microsoft\Exchange Server\V14\scripts folder on the Exchange 2010 server.



Custom prompts are audio files made available to all Outlook Voice Access users and callers who dial in to UM auto attendants.  They are stored in different places, depending in the Exchange version.



In Exchange 2007 custom prompts are stored in the custom prompt publishing point, which is really just a UNC file share.  Exchange 2007 then uses the Microsoft Exchange File Distribution service to replicate the audio files in the share to the \\<servername>\ExchangeUM folder to all the UM servers in the same dial.



Exchange 2010 simplifies this model and stores the custom prompts in the system mailbox {e0dc1c29-89c3-4034-b678-e6c29d823ed9}.  This mailbox is created when you run Setup /PrepareAD.  If the system mailbox {e0dc1c29-89c3-4034-b678-e6c29d823ed9} mailbox is missing in your environment, you can run Setup /PrepareAD again to recreate it.  By using an Exchange 2010 system mailbox, custom prompts can be backed up and restored along with other mailboxes in a database.



The MigrateCustomUMPrompts.ps1 script runs the Import-UMPrompt cmdlet which requires the Windows Desktop Experience feature.  For this reason you will probably need to run the script from your Unified Messaging server.



To use it, simply open the Exchange Management Shell and change to the Exchange scripts folder.  You can do this quickly by running the following command:



cd $exscripts


It looks dirty, but it isn’t.  :)



Now run MigrateCustomUMPrompts.ps1.  The script takes no parameters.  It will connect to the custom UM prompt location on one of the Exchange 2007 servers and import the prompts into the Exchange 2010 system mailbox.






Easy, peasy!  Sure beats the old way.

Messageware OWA Desktop Review

I don’t normally do product reviews, but I had the opportunity to work with Messageware’s new product OWA Desktop and thought I’d share my thoughts.  This software extends Outlook Web App’s functionality so that it behaves even more like the real Outlook client and integrates OWA into the Windows desktop environment.



OWA Desktop was awarded the prestigious Best of TechEd 2011 award by Microsoft TechEd attendees this year in Atlanta, GA.  Messageware, a Toronto based company, has been extending OWA functionality since 2003.  Their goal is to make OWA as close to Outlook as possible.  This makes it extremely useful for companies that can’t or don’t want to deploy a full blown Outlook client on desktop workstations.



OWA Dekstop runs in the Windows notification area whenever the user logs on and gives one click access to most OWA functions.

Right-click OWA functionality from the Windows notification area



With a single click users can open the full OWA application, compose a new email, create an appointment or meeting, change settings, etc.  Notice in the screenshot below that OWA Desktop opens like an HTA application.  There are no browser controls, such as the address bar, forward or back buttons, or browser menus.  This gives OWA Desktop a clean look and better use of screen real estate.



The OWA Desktop application



New message notifications come in two flavors.  One option uses balloon pop-ups in the notification area. Balloon pop-ups can optionally provide a text preview of the new message.  A single click opens the full message using OWA Desktop. 



Balloon pop-up with message preview

The other option displays a “View Unread” scrollable window that displays all the unread messages in the Inbox.  Here you can open an item, reply/reply all to an item, mark an item as read/unread, or open the full OWA Desktop application.



“View Unread” window pop-up



One of the other nice features is the ability for users to import and export data using OWA.  By right-clicking the OWA Desktop icon, you can import holidays for dozens of countries or import/export contacts.  This is very useful for Office 365 users.



OWA mport/export functions

The OWA Desktop MSI package is only about 6.5MB in size (much smaller than 258MB used by Outlook 2010) and is easily deployed using System Center Configuration Manager (SCCM) or Group Policy software distribution.  There are no server-side components required.



User account setup uses Exchange Autodiscover for easy setup and configuration.  You can also configure multiple OWA Desktop accounts to be open simultaneously on the same computer.  The screenshots in this article show two different colored icons in the notification area, one for each Exchange email account.  Traveling mobile users will appreciate the ease of changing timezones in Options.



Account Setup uses Exchange Autodiscover



Because OWA Desktop runs in the notification area when users login to Windows, they are unlikely to close OWA accidentally.  OWA Desktop runs in a discreet browser window so users don’t need to continually login to OWA when they close their browser.



Messageware also created an OWA Desktop Deployment Pack which allows deployment customization and branding to personalize the look of OWA Desktop.



I’ve worked with Messageware for a few years, primarily with their OWA PlusPack product for OWA 2003 and OWA 2007.  This new product provides easy access to OWA right from within Windows.  You really should check it out.