The Default SMTP Virtual Server instance was failing to start after recovering from a System State Backup. Because this was a Domain Controller the server had to be started in Directory Services Restore Mode (DSRM) in order to do the ntbackup restore. When booted into DSRM, Chkdsk kicked in and did a scan before launching into Safe Mode. What I didn’t know initially was that Chkdsk had gotten hold of the Exchange ‘Queue’ directory. Here is what we did to bring it back and recover the previously queued mail.
When the Default SMTP Virtual Server was failing to start, the application log reported event id 429 which mentioned that the specified mail queue directory is not valid. We checked the ’vsi 1’ folder and found that the ‘Queue’ folder was missing and in its place was a file with the name ‘Queue’. We deleted the file and created a folder named ‘Queue’. Once this was done, we were able to start the Default SMTP Virtual Server instance.
Now that Exchange was running again with its new ‘Queue’ folder that was great, but what happened to the messages that were previously in the Exchange Queue before the System State Restore had been done? We found that checkdisk had placed them in the ‘Found.000’ folder. We replayed the messages by putting them back into the ‘Pickup’ folder. Mail normally flows into the Pickup folder first and then will automatically move to the Queue folder before moving into individual mailboxes. There ended up being a couple corrupt ones that would just sit in the Queue folder so we deleted those in Windows Explorer.
Prashant Joseph who works with Microsoft’s Enterprise Messaging Support sent this to me at the resolution of a call Monday morning. Great stuff so passing it along. What I thought was cool is there is also a URL redirector for the ExBPA at www.exbpa.com if you want a quick link to get there.
Notes from Prashant:
1) ExBPA (Exchange Server Best Practices Analyzer Tool)
- This is a tool which can help you to generate reports regarding your Exchange server and Organization.
- This can be installed on either the Exchange server or a client machine.
- The reports generated by this tool provide information regarding any critical Exchange issue. It also provides a link to some online documentation which provide either information or a solution to the issue.
• The Exchange Server Best Practices Analyzer Tool can be downloaded from http://www.microsoft.com/downloads/details.aspx?familyid=dbab201f-4bee-4943-ac22-e2ddbd258df3&displaylang=en
2) ExTRA (Exchange Server Troubleshooting Assistant)
- The Microsoft Exchange Troubleshooting Assistant can help to determine the cause of performance, mail flow and database mounting issues on computers running Microsoft Exchange Server. The tool automates specialized troubleshooting steps for identified symptoms.
- This can be installed on either the Exchange server or a client machine.
* The Exchange Server Troubleshooting Assistant can be downloaded from http://www.microsoft.com/downloads/details.aspx?familyid=4BDC1D6B-DE34-4F1C-AEBA-FED1256CAF9A&displaylang=en
3) Antivirus and Exchange.
• Antivirus software’s can cause numerous issues on an Exchange server. In order to prevent this there are a few things that need to be taken care of:-
Ø File level Antivirus software’s.
# In case if we are using a file level Antivirus software then we need to make sure that the right Exclusions have been setup.
# The following folders must be excluded from Virus scanning:-
a) INETPUB (The default location is C drive)
b) INETSRV (The default location is C:\WINDOWS\SYSTEM32)
c) The Exchange database folder.
d) The Exchange log files folder.
• The below mentioned KB article talks about Exchange and Antivirus.
I’m beginning to feel like Handy Andy and his ATM experience I mentioned earlier today. After giving up on my own resources I called Microsoft Partner Business Critical Phone Support to get a resolution on a Kerberos error that has brought down AD and Exchange on a Win 2k box. Was confident I would make it home for a couple hours sleep before catching a ferry back over to Seattle and starting my week, but now I’m getting concerned. I was on hold from 12:00 AM – 2:00 AM PDT before the first call terminated that had “no one in queue” before me. Called back in at 2:20 AM PDT (25 minutes ago) and back on hold again with “no others in queue.” Must be lunch time on the other side of the world. Good thing I’m at a coffee company tonight. I’ve had four double shot espressos but now I really just want to get this server fixed and head out of here before I end up in Monday morning commuter traffic, or worse yet, afternoon commuter traffic! Kevin B, I know it’s closed, but please add this to the 2007 Community Survey for the Windows Server guys to consider. Hours on hold in the middle of the night are not winning any points in the PR department with me.
After experiencing Terminal Services intermittently hanging after applying Microsoft Updates I’ve come across one way that gets the server back up and responsive most of the time and a backup plan for when it still doesn’t.
First, to cover yourself, it is a good idea to run a system state backup at minimum of the server before applying updates in case you need to get yourself out of a challenge resulting from the update. Then what I’ve started doing (wasn’t an original idea, got it from others in the community) is creating a restart.bat file that consists of shutdown -r -f and a hard return to bounce the server. I then create a one time scheduled task and set it for about 30 minutes out from when I figure the updates should have been applied and the server restarted and back online. Most of the time a restart will “jump start” the Terminal Services and get them running if they hang. If I don’t have to use it, I go into Scheduled Tasks and set the time back a day to disable it until next time. The other trick I ran across was the result of a desperate attempt to get a box back online one night as I was applying updates from my Vista Tablet PC. I’ve since tried this from XP Pro and it does not work, so seems to be a Vista only trick.
If TS/RDP hangs on the server, as Microsoft is beginning to admit is a problem finally, attempt to VPN from your Vista computer to the remote network using the remote domain’s administrator account (I’ve only tried this with Windows PPTP so no idea about IPSec or edge VPN devices such as SonicWALLs but I’m guessing it would fail since the reason this is working has to be because Vista is authenticating as the remote administrative account). Once you have established the Windows PPTP VPN from your Vista box, open a command prompt by selecting off of your start menu, “right clicking” and running as administrator. Once inside the command window, type shutdown -i and drop in the name or IP address of the remote server you are wanting to get back your Terminal Services from and do a restart in the graphical interface. I have had this work all but once or twice (hence the new addition of the scheduled task restart batch file). When the server comes back online TS is normally responsive and you can head off to bed versus getting in the car and taking off to a client site in the middle of the night!
Virtual PC 2007 is not agreeing with the network adapters on my Vista Ultimate HP tc4400 so I’m on my wife’s HP nc6320 running XP Pro while I’m doing a Swing Migration. While on here I went to connect to my desktop back at the office and was tossed into a loop in IE7 prompting me about Active X controls when attempting to connect to my computer in Remote Web Workplace. Had this happen the other day in RWW so guessing others are having similar hassles with it. Here’s how to fix it.
In IE7 go to Tools, Internet Options, Security (2nd tab), Trusted sites, and add your SBS external domain to your Trusted sites. Then click on the Custom level… button about 1/3rd up from the bottom of the Security Window and scroll down to “Download unsigned ActiveX controls” and change the radio button from Disable to Prompt. Click OK and you should be set. If you would like or need to install the Self-Signed SSL Certificate your SBS box created while you are at it so you don’t have to look at the colorful red across your address bar in IE, check out the SBS Team Blog’s post by Wayne McIntyre to take care of that why you’re at it.
On my way Sunday evening to a client site thinking I had it rough. Then l read Handy Andy’s post about his week.
This isn’t in the normal KB articles for antivirus scanning exclusions, so if you come across the following event ID, check out MS KB 887311.
Event Type: ErrorEvent Source: Microsoft ISA Server ControlEvent Category: None
Event ID: 14079
This is from an email I received this week from Mark Clagett over at Microsoft. Running into an issue today where I sent it to an end user so figured I would post it here as well. Thanks Mark for sending out the notice! – Steve ________________________________________________________________________________________________________ I thought you’d be interested in this new Solution Accelerator from Microsoft – it’s called the Malware Removal Starter Kit. It’s a free download from TechNet, and provides you with excellent guidance and tools to help you restore PCs infected with malware. Here’s a quick overview of what the kit can do for your organization. Suggest you take a look!
PCs Infected with Malware? Every day, adversaries attempt to invade your networks and infect your systems with viruses, spyware, and other malware. In other cases, employees can open the door to malware by visiting infected Web sites, opening the wrong e-mail attachments, or running macros that contain viruses.As an IT professional focused on security, you know the risks first hand. You’ve installed antivirus software and you keep your protection updated. Sometimes, though, attacks are successful, and computers get infected. And once they are inside the organization, malware outbreaks can spread with alarming speed, compromising or destroying mission-critical data or personal information. Restore Infected PCs with the Malware Removal Starter Kit!When you discover PCs that have been infected with malware and your current antivirus tools can’t solve the problem, where do you turn next? Is there a way to restore infected PCs without completely rebuilding them from scratch? The Malware Removal Starter Kit, the newest Solution Accelerator from Microsoft, provides free, tested guidance to help you combat malware attacks and restore infected systems—so users can safely get back to work. The kit shows you how to use the Windows Preinstallation Environment (Windows PE) to discover malware by performing a thorough offline scan of your computers, uncovering malware that may be hiding in the operating system. And once malware is located and identified, it can be quickly removed from infected PCs with a number of free anti-malware tools, like the Malicious Software Removal Tool from Microsoft.The Malware Removal Starter Kit answers questions like:
· What are the keys to a reliable, effective response plan to remedy malware outbreaks?
· How do I build a bootable CD that lets me perform offline virus scans?
· How can I discover and remove viruses and other malware hiding in the operating system?
· How does the Malware Removal Starter Kit augment Microsoft’s anti-malware strategy? Key Benefits
The Malware Removal Starter Kit is:
· Effective: Helps you to uncover malware that’s difficult to expose.
· Flexible: Lets you use best approach for the specific problem you’re facing.
· Reliable: Provides guidance thoroughly tested by Microsoft security experts.
· Simple: Offers a solution that is easy to configure and use.· Free: The Malware Removal Starter Kit is a free download from TechNet. Download the free Malware Removal Starter Kit
Accessing the kit is easy, and it’s free! Click here to learn more or to download the kit.
Microsoft will officially end support for SBS 2003 RTM (also know as SBS 2003 SP0) on July 10, 2007. See the SBS Blog for the full story at http://blogs.technet.com/sbs/archive/2007/07/03/sbs-2003-sp-0-support-retirement.aspx.