MADSolutions - Blog

Goodbye to Single Instance Storage!

Posted under Email Management, Exchange Server

In the 1990’s when email use started to become more popular than faxes, systems like MSMail, All-in-one, Banyan Blue, shark mail, lotus notes, groupwise and many others sprouted up throughout the email management landscape. Systems like MSMail with their post office style file folder system utilized message transfer agents (MTA)’s that were little more than file transfer processes.

These agents put email into the file system where they would await an MSMail client to come along to retrieve the data in the directories (such as MAI, MBG and ATT) and then store it in local MMF files. These systems served their purpose in their day but as the number of users of email grew and the distances email was sent increased, newer and more robust systems took their place. These older type systems often used file transfers, X400 and other protocols for message transfer unlike the modern day generally accepted standard of SMTP.

Data was often stored multiple times across various devices. One of the key issues with MSMail was the storage of a user’s email in MMF files which seemed to continue to grow and eventually become corrupted leaving administrators with challenges in retrieval. With the introduction of systems like Microsoft Exchange 4.0 the email was no longer stored per user but instead in a shared database on a server with a single instance of messages and attachments to minimize space. With Microsoft Exchange 5.5 and the introduction of a feature rich client (Microsoft Outlook 97 released in January 16, 1997), the number of users finding it so easy to drag and drop files into messages or with just a simple click, attach files the size of email messages and the number of emails and their size seemed to grow exponentially. Mailbox quotas that once were less than 10 megabytes soon increased to 50, 100, 500 or more megabytes in size. Despite storage requirements, email usage across cities, provinces, countries and even across the world continuing to expand to the masses and well beyond the business world. Thank goodness with the introduction of Microsoft Exchange 4.0 Microsoft included a “single instance storage” feature that only stored one copy of data or files when they were duplicated across multiple mailboxes.

With the ever-increasing amounts of data that users send and attempt to store in email, one can only be thankful for single instance storage…however…as the number of servers increased and as the number of databases increased, single instance storage became less and less effective. After all single instance storage only worked at the database level, not across databases or across servers. This single instance storage also came with a performance price tag but it was a trade off to keep storage costs down.

So now here we are in 2010 and Microsoft has pretty much taken the market with its highly scalable, highly robust messaging platform that more than 70% of the world’s corporations rely on 24-7, 365 days a year. With Microsoft Exchange versions 4.0, 5.0, 5.5, 2000, 2003 and 2007 single instance storage helped companies keep storage costs down, however with the cost of storage dropping dramatically and to provide greater performance and bang for your buck, Microsoft Exchange 2010 no longer provides single instance storage in favour for a more improved compression algorithm and the ability to use JBODs even if they are only SATA drives on an inexpensive server. Database availability groups (DAGs) help prevent data loss by having multiple copies of the database enabling companies to inexpensively implement local redundancy and site resiliency. The trade off? Storage space increasing dramatically. What you need is a exchange mailbox management system like MADSolutions Email Management Server which continues to include a Single Share feature that allows administrators to keep storing a single instance copy of attachment files that are stored across multiple users’ mailboxes.

Posted by SGeisel on May 16, 2011 —