Category Archives: 13175

Poll: Which new Hyper-V lab server build would you be more likely to buy?

I am preparing to create my 5th generation Super-Fast Hyper-V Lab Server build. As usual, I will create a parts list, photos, videos, and tips about the build on this blog, but I need your help.




I normally stick to a small Micro ATX form factor which currently supports a maximum of 32GB RAM. I currently run this build at home and I’m happy that it doesn’t take much room and uses less power. 32GB RAM is enough to run 6-7 medium/large servers at once 24×7.



Some IT Pros have asked for a build that supports 64GB RAM so they can run more or larger VMs. A 64GB build requires me to use a traditional ATX form factor motherboard with more DIMM slots. This will use more power and will cost about $900 more.



I realize cost is more of factor than size to most folks, but this website shows a comparison of ATX vs. Micro-ATX case sizes if you’re not aware. The microATX case I usually go with is the same form factor as the “barebones” case shown on the website.



I created the poll below so I can determine which build you would like me to go with for my 5th generation server. I really appreciate your input.





Which new Hyper-V server build would you be more likely to buy?













I will be speaking at the IT/Dev Connections conference September 15-19 in Las Vegas. There, I will be hosting two sessions, “Build Your Own Super-Fast Exchange Lab for Under $2,000!” and an open mic forum entitled “Ask the Exchange Experts,” a Q&A session about Exchange and Office 365 migration tips and tricks with fellow MVP Tony Redmond.



I will be bringing my latest Hyper-V lab server build to the lab session and will provide tips on how to build, manage, and use the server to advance your IT career. I hope to see you there!


An Open Letter to Microsoft Learning

Yesterday I was notified of a new video touting how Microsoft Learning is revamping its Exchange 2013 exams and certification requirements for Exchange 2013 SP1. As someone who has worked with Microsoft to help rewrite exams for Exchange 2010 SP1, I was interested to see what @MSLearning had to say. I was met with great disappointment when I was greeted with the following video (since removed, but still found on YouTube).






In response I tweeted on Twitter, “One more reason that customers need the MCM program. I weep for our future.




As a Microsoft Certified Master and someone who takes great pride in the 77 Microsoft certifications I hold, I take Microsoft certifications seriously. As a Microsoft Gold Partner ExtraTeam does, as well, and makes its mark in the professional services industry by hiring the the most highly certified consultants and engineers in the industry.



Judging by the feedback I received to my tweet, I know that other IT Pros share my sense of frustration and dismay about the direction of Microsoft Learning.



Veronica Wei Sopher of Microsoft Learning responded to my tweet, genuinely asking for my feedback – so here it is:



  • Take yourself seriously. “Sesame Street” style videos have no place in a professional certification program. As one person wrote, “The costumes? No names? This needs to feel more work related if the sound is muted.” How do you think this looks to hiring managers? I can’t imagine anything like this coming from the Cisco or CCISP certification programs.

  • Be respectful and show ownership. Many IT Pros, such as myself, have invested significant amounts of time preparing for, taking exams, and maintaining their Microsoft certifications. Many do it on their own time and with their own money. It’s embarrassing and insulting to all IT Pros to be associated with a program that makes fun of certifications and the process.

  • Have integrity. Like other MCM candidates, I spent 21 days in Redmond learning 24×7 about Exchange in the MCM program and I’d do it again in a heartbeat. It was one of the best learning experiences of my life. That’s why it was so disappointing when Microsoft Learning canceled the MCM program without any notice, even to the Exchange product group. When Tim Sneath canceled the program in September 2013, he told us that Microsoft Learning was looking into ways to revamp the program. It’s been nearly a year later and we have heard absolutely nothing. At this time, the highest level of certification that IT Pros can achieve is an MCSE, which is pretty much worthless due to cheating and brain dumps. There has to be a better top-tier certification for Microsoft products than what is available now.



How to delete duplicate Lync Contacts or Lync Contacts folders

A bug in previous versions of the Lync 2013 client caused Lync contacts to be duplicated in the Exchange Lync Clients folder. This makes it very annoying to work with contacts in Outlook, OWA, and mobile devices.








I wrote in an earlier article, Fix for Excessive Duplicate Contacts, that describes how to delete these contacts or folders using OWA or earlier versions of Outlook. This was possible because these older versions did not respect the flag that defines the Lync Contacts folder as a protected folder, like Inbox or Drafts.



You could use MFCMAPI to delete protected folders, as shown below, but this can be cumbersome if you have to do it to many mailboxes — not to mention the fact that you need to grant yourself full access to the target mailbox(es).






A better solution is to have the end-users do it themselves using OWA in Light mode. OWA Light bypasses the protected folder check and allows end users to delete some or all of the Lync contacts, or the entire Lync Contacts folder itself. The best part is that this works from all versions of OWA, even Office 365!



All you need to do is send a URL to the end-users to login to OWA Light with the steps to delete the folder or contacts:

https://outlook.office365.com/owa/?exsvurl=1&layout=light&wa=wsignin1.0

Replace outlook.office365.com with your organization’s OWA FQDN, if necessary. By following this specially crafted URL, users can enter OWA Light to clean it up without affecting their current OWA Premium experience.



From OWA Light select the Contacts folder on the left pane. To delete the duplicate Lync Contacts folder click the link for Manage Contacts Folders, then click the Choose folder to delete drop down list and delete the duplicate folder.







If the user wants to delete Lync contacts from an existing folder, select the Lync Contacts folder and use the checkboxes to select them or use the checkbox at the top to select all the contacts displayed. Then click Delete.






Hopefully this will help those of you who were unable to delete these duplicates because you were running a newer version of Outlook 2013 or OWA. Special thanks to Greyson Mitchem for the tip.


Does your environment need an Exchange 2013 Edge Transport server?




I was asked to write an article for my friends at ENow Consulting, “Does your environment need an Exchange 2013 Edge Transport server?”  The standard consulting answer applies: “It depends.”



If you’re wondering if an Exchange Edge Transport server makes sense in your Exchange environment, I encourage you to head over to the ENow Consulting Blog to read the article.




SMTP Firewall Requirements for Exchange Online

Most of my Office 365 engagements are hybrid projects connecting Office 365 with Exchange on-premises, and most are with larger companies concerned with securing the hybrid deployment.



Exchange Online Protection servers send SMTP emails using a TLS connection usually to the hybrid or Edge Transport server to enable mail flow between cloud and on-prem users. Microsoft does not support any sort of SMTP gateway or appliance between EOP and the Edge or hybrid server. For this reason, customers normally have to open TCP port 25 on the firewall to the hybrid server from the Exchange Online Protection servers.








Companies can secure this SMTP traffic by configuring the perimeter firewall to allow inbound TCP 25 traffic only from Exchange Online Protection servers to the hybrid or Edge servers.



I’ve seen a number of articles that list the public IP addresses used by EOP to send SMTP emails to on-prem customers, but the one true list is maintained in the article, Exchange Online Protection IP Addresses. Currently, this article lists seven IPv4 blocks and one IPv6 block for SMTP delivery to on-prem:

  • 65.55.88.0/24
  • 207.46.51.64/26
  • 207.46.163.0/24
  • 213.199.154.0/24
  • 213.199.180.128/26
  • 216.32.180.0/24
  • 216.32.181.0/24
  • 2a01:111:f400:7c00::/54

Microsoft tries hard to not make changes to this list, but if they do they will update the article. It’s important for firewall admins to know that EOP does not use URLs for root domain routing (also known as Top-Level Domains, or TLDs). You must use the IP addresses listed in the article above.

Up until April 2014, Microsoft used many other IP addresses to send emails from Office 365 tenants to on-prem customers. This is because they maintain another set of IP addresses for something called the High Risk Delivery Pool, which is used to protect the production Exchange Online namespace from “spammy” senders. EOP no longer uses the High-Risk Delivery Pool when sending emails between the customer’s tenant and their on-prem servers.


It’s nice to know that we now have a single source to point to when configuring firewalls for Office 365.


Reporting Outlook Client Versions Using Log Parser

I’m doing a little cross-pollination today. Chris Lehr, one of my colleagues at ExtraTeam, worked up a Log Parser script that produces a report showing all the clients versions connecting to Exchange.  This is very helpful to show which clients are running Office versions in your organization that should be updated prior to migration.  Check out his blog post here.



Outlook Client Version Report

Clients will always get the best experience using the latest version of Office, currently Office 2013 SP1. The best practice is to always update your clients with the latest cumulative update prior to migration. this is especially true when you’re migrating to Office 365, since most updates pertain to Office 365, Exchange Online, and Exchange 2013 compatibility.



If you find that you need to upgrade clients to a new version of Office, I recommend that you install the x86 version of Office to provide the best compatibility with add-ons and third-party products. Some customers think they need to install Office x64 on Windows x64 operating systems, but that’s not the case. See 64-bit editions of Office 2013 for details on when it makes sense to install Office x64.



If you’re an Office 365 customer, I strongly recommend checking out using the Office 2013 ProPlus software deployment that’s most likely part of your Enterprise license. This version of Office 2013 can be installed on up to 5 PCs, iPads, tablets, etc. and is always up-to-date since it’s a cloud-managed service.




Tips for Connecting to Office 365 using PowerShell in Hybrid Environments

The Office 365 portal and Exchange Admin Console are fairly powerful to allow you to manage your tenant and on-premises environments. But as you are no doubt aware, there are many administrative tasks that require you to use PowerShell.



The sequence you usually find on the web to connect to Office 365 via PowerShell is:

[PS] C:\>$LiveCred = Get-Credential -credential admin@contoso.onmicrosoft.com
[PS] C:\>$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.outlook.com/powershell/ -Credential $LiveCred -Authentication Basic -AllowRedirection
[PS] C:\>Import-PSSession $Session

If you are in a hybrid environment with Exchange and Office 365 you will discover that both environments have a lot of the same cmdlets, such as Get-Mailbox, Set-DistributionGroup, etc. This causes a conflict when the Office 365 PowerShell cmdlets are loaded within the Exchange Management Shell. You either need to connect to Office 365 PowerShell from a regular PowerShell console (separate window) or you need to use the -AllowClobber parameter, which overwrites the existing EMS cmdlets with the Office 365 versions. This is not ideal if you are working with both on-prem and cloud objects at the same time.

Proxy creation has been skipped for the following command … Use the AllowClobber parameter if you want to shadow existing local commands
I wrote a PowerShell script called Connect-Office365.ps1 that overcomes these conflicts by using the -Prefix parameter with the Import-PSSession cmdlet. The Prefix parameter tells PowerShell to add the specified prefix to all cmdlets it loads from Office 365. For example, if you set the prefix to “cloud” the Get-Mailbox cmdlet for Office 365 becomes Get-cloudMailbox and the Get-Mailbox cmdlet still applies to on-prem. This way you can use both sets of cmdlets in the same EMS console. The script also connects to the MSOLService so you can use the MSOL cmdlets to manage Azure AD.

Connect-Office365.ps1 with “cloud” prefix
Here’s the four line Connect-Office365.ps1 script:
$LiveCred = Get-Credential -credential “admin@contoso.onmicrosoft.com”
 
$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://ps.outlook.com/powershell/ -Credential $LiveCred -Authentication Basic -AllowRedirection

Import-PSSession $Session -WarningAction SilentlyContinue -Prefix cloud

Connect-MsolService -Credential $LiveCred

I usually copy the script to the C:\Windows folder on my Exchange servers and my management computer so it can be run from any directory whenever I need it.




Announcing the 7th Annual UC Roundtable at TechEd 2014, Houston!





I’m pleased to announce the 7th Annual UC Roundtable at Microsoft TechEd North America 2014 in Houston, TX!






The purpose of the UC Roundtable is to gather Exchange and Lync admins, MCMs, MVPs, Exchange product group members, architects, and experts for a free-flowing discussion about issues, questions, and experiences related to Exchange, Office 365, and Lync Server.  If you work with Exchange, Office 365, or Lync you need to be here!



The UC Roundtable will be held Wednesday, May 14, 2014 from 6:00-8:00PM CDT and will be within walking distance or a short cab ride from the TechEd hotels. A special thanks to my friends at F5 who will be hosting the event for the third year in a row!



Please RSVP to jeff@expta.com for event details and location. I will email you with the location details once they’re set.



Help spread the word and I hope you can join me!






Pending Messages in Exchange Online Will Not Be Delivered

There‚Äôs a pretty big unpublicized bug in Office 365 transport that you need to be aware of.



Messages from Exchange Online Protection (EOP) that cannot be delivered to an on-prem Exchange hybrid server due to a transport problem will be marked as Deferred or Pending.  These messages will NEVER be delivered, even after the transport issue is resolved.



The following example shows a message that was sent at 2:46PM UTC and the status shows as Pending. The last event for this message shows DEFER at 3:29PM UTC with the detail, “The last attempt to deliver the message encountered an error”.



Sample Message Trace from Office 365

For example, say the TLS certificate expires on your hybrid server. Inbound messages from EOP to the hybrid server will queue because the Outbound Connector is using Forced TLS, but the certificate is invalid. If you resolve the problem by renewing the cert or reconfiguring the Office 365 Outbound Connector to use opportunistic TLS the problem is solved for new emails – they get delivered right away, but the Pending messages will never get resubmitted and eventually expire (after 48 hours).



Incredibly, there currently is no way for Microsoft to resubmit these messages like there is for on-prem Exchange. After opening a high-priority case with Microsoft Online, the only “solution” they could give is to contact all the senders and ask them to resend their emails.








Troubleshooting TLS SMTP Connections to Exchange Online Protection

By default Office 365 uses Transport Layer Security (TLS) to send encrypted SMTP emails between Exchange Online and Exchange on-prem. This provides end-to-end encryption of emails between your on-prem Exchange Hybrid Server and Exchange Online Protection (EOP), just like they were the same organization.



TLS utilizes x.509 (SSL) certificates to encrypt the email in transport. Exchange Online and Exchange 2013’s implementation of TLS differs from previous implementations of TLS in several important ways:

  • The certificates used for TLS must be issued by a trusted third-party CA (Digicert, Godaddy, Verisign, etc.)
  • The CRL for the certification authority must be available. If Exchange or O365 can’t read the CRL it will not trust the certificate.
  • The FQDN that the Receive Connector provides in response to EHLO must match the subject name or a subject alternative name on the certificate. SAN certificates and wildcard certificates are both valid for TLS use. If you use a wildcard cert the FQDN used on the connector can be any name that is valid for the wildcard cert’s domain.
  • In previous versions of TLS any certificate could be used to encrypt SMTP traffic, even expired or self-signed certificates. This is not the case with Office 365 and Exchange 2013.
As previously mentioned, TLS is required by default for SMTP communication between Hybrid servers and Exchange Online Protection (EOP). When you run the Hybrid Configuration Wizard it will configure forced TLS for all Send and Receive Connectors. Note that this configuration is updated whenever you run the HCW, so if you reconfigure the connectors for opportunistic TLS or turn it off completely, the HCW will reconfigure them to use forced TLS again.

The following options are available for TLS encryption in Office 365:

  • Force TLS – Email sent across this connector must use TLS. If TLS is unavailable messages will queue until they are delivered or expired.
  • Opportunistic TLS - The connector tries to setup a TLS connection using the STARTTLS verb. If a TLS connection cannot be made the connector falls back to regular ESMTP or SMTP. This is functionally equivalent to turning TLS off.
Normally the use of TLS is configured on Receive or Inbound Connector. You do this to control how email is accepted by your domain and it can be easily configured using the Exchange Admin Console. 
Office 365’s Inbound Connector from Hybrid Server
Send Connectors can also be configured to require TLS. This is used to enforce that email is sent by this connector must only use TLS, and is configured in the shell using the RequireTLS property:

Outbound Connector TLS Settings to Office 365

Using this configuration if the corresponding Receive Connector does not offer or require TLS, messages will queue on the sending server until they are finally delivered or expire.

You can easily tell if a receiving SMTP server is configured to use TLS using Telnet. Install the Telnet Client feature, Telnet to the server using TCP port 25, and look for the STARTTLS verb after issuing the EHLO domain.com command. For example:

Telnet servername 25

If you see that the STARTTLS verb is missing, the server is not offering TLS. If your Send Connector is configured to require TLS, the messages will queue on the sending server with the error,
451 4.4.0 Primary target IP address responded with: “451 5.7.3 STARTTLS is required to send mail.”

There are several reasons that the STARTTLS verb might be missing:

  • The receiving server is not configured to Force TLS or use Opportunistic TLS.
  • The sending server’s IP is on an SMTP block list (aka SMTP blacklist or SMTP blocklist). Office 365 will not attempt to send TLS traffic with a server it can’t trust.
  • The receiving server is configured to only respond to SMTP (not ESMTP) commands. TLS is part of the ESMTP protocol.
  • Your firewall is doing some form of email inspection and is filtering “unsafe” verbs from the SMTP conversation. Some examples of firewalls that do this are:
Office 365 uses internal and external SMTP blocklists to protect the network. If the sending server is on one of these black lists the EOP servers will not offer TLS. You can test this by sending a non-TLS email to EOP using Telnet. See Brian Reid’s article, “Cannot Send Emails to Office 365 or Exchange Online Protection Using TLS“. The response from the EOP server will tell you which block list you’re on and how to request removal.

If the receiving server requires TLS, but the sending server is not configured to use a TLS certificate messages will queue on the sending server with the error,
451 4.4.0 Primary target IP address responded with: “451 5.7.3 Must issue a STARTTLS command first.”
The fix here is to configure the Send Connector to use TLS and a valid certificate using the Enable-ExchangeCertificate cmdlet, such as:
Enable-ExchangeCertificate -Thumbprint 5DC5902752816FD2FC51D5564C363F68D8F7FFC4 -Services SMTP
Make sure to use Get-ExchangeCertificate to get the correct certificate’s thumbprint for the command above.

Hopefully this information will help you understand TLS and will assist you with troubleshooting.