Do I really need to archive all this email?

Storage managers are waking up to the realisation that they need to dedicate a large amount of storage to email.

Has your company entered the world of email archiving, or is it on the agenda for 2006? Have you asked yourself if you really want to archive all the email? Have you thought about how much email traffic your company sends and receives? Do you know how much this is increasing year upon year and the impact this will cause on your storage? The chances are that you don't.

Everyone can relate how many emails they receive a day and compare that to the volume of email from a couple of years ago, but what will this be in two years time? Storage managers are waking up to the realisation that email storage demands are huge, and despite attempts to curb users demands, they are fighting a losing battle and need to dedicate a large amount of storage to email.

The question that comes to the fore is whether the email truly needs to be archived or whether it is simply easier to archive everything because no-one can be bothered to answer the question.

Or is it that they can't see any easy way to clarify and then resolve the issues? So what are the problems? There are three main ones, being the volume of mail, understanding the reasons for wanting to archive and the decision on what to archive? We all know email volumes are high, but quite how high, and what will it be next year? Respected industry analysts, such as Sara Radicati and Michael Osterman, report that email volumes are increasing by around 20-25% per annum.

Coupled to this is the increasing size of email.

This is slightly lower at around 15-20% per annum.

But compounding these figures we get an annual increase in email data of around 40% per annum.

That gives us around double the amount of data every two years.

This is unremitting, with none of the analysts predicting any fall from this rate of increase.

This is way beyond the figures most storage managers and email system managers have been catering for.

There are two concepts for archiving, firstly to take a secure copy of every mail as it is received/leaves the user (usually known as compliance archiving) and secondly to remove large or old emails from the system, usually by a process known as message-stubbing to reduce the size of the message store and its impact on system performance.

It may surprise those of you who are about to enter archiving for the first time, but a large proportion of those who have already entered this world, did not understand what the needs were in the first place.

Differing objectives came into play.

Legal and Human Resources departments wanting every email retained for legal reasons, while system managers wanting to archive for system capacity and performance reasons.

The two do not necessarily coincide in terms of solutions.

There are many laws and industry regulations that either dictate to us or advise us to archive data.

Some of them dictate for how long the data must be retained and even what to do on the expiry of the retention period.

On reading the laws and regulations more carefully, they usually indicate which job functions or departments these relate to, such as mergers and acquisitions or Board level management of a quoted company.

However, other laws apply more generally.

Freedom of Information laws require public bodies to retrieve information on almost any subject, while Data Protection laws require companies to be able to locate information related to an employee or ex-employee.

There may well be the need to archive existing data, to relieve the burden on the messaging system.

Some of this data could be more than 5 years old.

Do we really need to archive this data for corporate reasons, or for the users own productivity needs, or does it remain in the system because the user simply did not delete it at the time meaning it has long been forgotten and is no longer required.

If you sift through all the laws and regulations that apply to your company, you can produce a matrix of what is essential to retain, what is important to retain for user productivity and what is of no value at all.

When you have understood both of your reasons to archive and the data/employees/departments that need to archive, then a good approach is to look at the problem in reverse.

As in "what do you not want to archive".

When you start making this list, then you will you will start to see patterns of 'essential' and 'non-essential' email.

To achieve this, you will need to exploit features of your archiving solution.

Multiple repositories will enable you to archive different sets of data to different locations, for example for employees email can be archived to optical media as it is unlikely to accessed with any frequency .

This will aid in the overall management of the data and in the physical location of the data (i.e fast access near-line devices or off-line media).

Just as important (in terms of managing the storage) is the ability to select which data is and is not archived.

If you get it right, you can make a decision on which data should be deleted, and gaining agreement on from Legal need not be that onerous! For example does anyone apart from the marketing department really need to keep files such as .mp3, .avi, .mpg, .wav? You would probably be surprised how much of this is stored in your email system.

And once the data has been there for more than 60 days, the employee who retained it has probably forgotten about it.

Of course before you delete such data you need to ensure your written corporate e-policy states that 'the company has the right to _' From understanding your business needs, the requirements to retain and the ability to remove or not archive particular email; and exploiting the advanced features of your chosen solution, will enable you to identify the archiving criteria.

From here it should be a relatively simple step of creating email policy management which can first identify and delete the unwanted data before archiving the remaining or chosen emails to single or multiple repositories; each of which could have its own retention periods.

This should be easy to set-up and administer and will result in a reduction of the volume of email to be stored and retained, reduce the processing and more importantly dramatically reducing the demands on the storage.

Thinking about your ongoing archiving needs can reap benefits in the long term and improve the return on investment in your archiving decision.

C2C is exhibiting at Storage Expo 2006 the UK's largest and most important event dedicated to data storage.

Now in its 6th year, the show features a comprehensive FREE education programme and over 90 exhibitors in the National Hall, Olympia, London from 18 - 19 October 2006.

Back to top