In Exchange Server 2010, there is no more single instance storage (SIS). This was a feature which allowed multiple attachments and messages to be stored once with a reference being held in each mailbox. The concept which had been with Exchange for years, had been intended to save space.
So what happened? To hear it from the exchange team, disks are cheap and the ability to have users on separate servers and in different databases decreased the value of SIS to the point where it is no longer relevant. I would say more complex to implement effectively too. So the decision was to invest the resources in other techniques which would have a similar impact on database size while applying to the current reality that users are not really in the same database anymore.
Underlying differences in architecture:
Prior to Exchange 2010 the database was simpler in that all messages existed in database wide tables as shown in this diagram:
In Exchange 2010, the database evolved so that the tables which existed as database wide were now contained within individual mailboxes. So, what you find now is an architecture where each mailbox is its own complete database structure as described here:
Where did they invest the resources:
- Compression of Message Headers and Bodies using LZ77 algorithm
Here is what the exchange teams says about compression:
Is compression the answer to replacing single instancing all together? The answer to that question is that it really does depend. There are certain scenarios where SIS may be viable:
- Environments that only send Rich-Text Format messages. The compression algorithms in Exchange 2010 do not compress RTF message blobs because they already exist in their most compressible form.
- Sending large attachments to many users. For example, sending a large (30 MB+) attachment to 20 users. Even if there were only 5 recipients out of the 20 on the same database, in Exchange 2003 that meant the 30MB attachment was stored once instead of 5 times on that database. In Exchange 2010, that attachment is stored 5 times (150 MB for that database) and isn’t compressed. But depending on your storage architecture, the capacity to handle this should be there. Also, your email retention requirements will help here, by forcing the removal of the data after a certain period of time.
- Business or organizational archives that are used to maintain immutable copies of messaging data benefit from single instancing because the system only has to keep one copy of the data, which is useful when you need to maintain that data indefinitely for compliance purposes.
What does the community think?
Concerns about backup and restore times increasing:
- Microsoft Answer: With a well designed VSS disk to disk backup effects should be minimal
Concerns about attachments and multiple copies:
- My suggestion is to implement SharePoint which does a much better job of doing document management and will have the same effect. Sending links to documents is a much better method of maintaining the current state. Although this isn’t an answer to every scenario, there are ways of managing attachments for large environments where this can be an issue.
Concerns about bucking the trend of de-duplication and going the other direction
- MS has moved data reliability, resiliency, and integrity up from the HW (RAID) level to the application level. The app layer gives us a lot more overall protection than the HW layer where the controllers just look at the bits in on the disk and make sure they are "correct" without knowing if the data inside the bits are ok.
In summary the loss of SIS will likely lead to greater adoption of attachment handling software. I certainly would not be surprised if we see specific functionality added around this for new versions of Exchange. For those planning for today I seriously would look at leveraging document management feature of SharePoint or other third party products. In the mean time training your workforce to send links to central files share is a much better way of handling attachments within your organization.
Jeff Loucks
Chief Strategy Officer | Winrox | 425-577-7377