Transforming data management into a competitive advantage

Today's world of compliance, data breaches and disasters has put data retention on the business agenda, as organisations far and wide create copies upon copies of their data to keep themselves "safe" and "compliant".

However, compliance is complicated. Many businesses have started to keep everything, forever—it's easier and less risky than drawing a line in the sand and deciding which data to keep and which data to discard. As a result, data retention has become a very costly exercise, consuming large amounts of storage with little to no value apart from "being compliant".

Data has historically been retained for two primary reasons: to recover from a data loss situation or disaster, or to satisfy legal and regulatory requirements. It's the latter that typically demands a much larger spend and creates a greater risk factor for organisations due to the complexities around retention.

As an example, according to the Australian Securities and Investment Commission (ASIC), all companies are required to keep a wealth of information about each of their employees, such as their ABNs and wage records, for seven years. At the same time, under tax law, small businesses must keep evidence of all transactions and be able to explain them for up to five years. There are other types of records that need to be kept for much longer, for example, patient health records, architectural and engineering designs, financial transactions, an interaction between teachers and students, the list goes on…

With so much data around, it's vital that organisations can truly extract value from their data to achieve competitive advantage. Here are a few ways your organisation can do just that.

1. Understand your data retention requirements. With privacy legislation such as the GDPR and Notifiable Data Breaches scheme now in effect, it's imperative that organisations understand what data they collect, why it's collected, where it lives and how it is managed. Check with your legal team and decide which of your datasets need to be retained and for how long. There is no need to keep backups or archives of all data for seven years, so come to terms with the requirements and act accordingly. Remember, if you can't justify why you collect a particular data set, you must reconsider.

2. Choose an appropriate long-term storage medium. Many organisations are turning away from tape, and traditional storage arrays, to deliver the data storage required for their retention needs. Cloud storage has increasingly become the answer to challenges in scalability and security, which in turn deliver compliance.

3. Choose the right data management solution. Some data management solutions struggle to apply long-term retention policies to the appropriate data sets, and typically require organisations to run separate labour and system resource intensive jobs (like monthly and quarterly backups). For the best results, organisations should run an intelligent scheduling and storage policy engine that automates this activity, leveraging efficient synthetic full backup runs combined with highly efficient deduplication and automated, intelligent storage tiering for the lifecycle of your data.

4. Ensure your data is recoverable and accessible. What is the point of keeping all this data for long periods of time if you can't get it when you need it most? Occasionally it is important to run a verification job of any important data to be kept for long-term retention. This used to be more associated with the risks of tape mediums failing, however is still used for disk-based targets to ensure data integrity. It is also important to factor redundancy and resiliency, from disasters to fault tolerance. Test your recovery and access data at least once a quarter.

5. Transform your data into an advantage. Data can serve more purpose than just legal and regulatory e-discovery requests. Rather, data can be turned into information that can be used to drive the outcomes your business needs. By using content indexing, data can be made human-friendly, searchable and readable to enable collaboration and self-service access. Data can also be layered with analytics and workflows that allow you to know what data you have, contextualise it, apply rules to it, protect it and use it. 

Data retention and management do not need to be a burden on your organisation. No doubt we have entered a new dawn of data. And while this brings with it new compliance challenges, it also presents limitless opportunities for data to be activated to better serve your organisation.