Story image

The trouble with data hoarding — and how to fix It

09 May 18

Article written by StorageCraft APAC head of sales Marina Brook

There’s a US reality TV show called Hoarders that features people who compulsively acquire stuff and are unwilling or unable to discard it. Eventually, they get buried.

A lot of companies are behaving the same way. They’ve become data hoarders. And, like those unfortunate folks on the TV show, they’ll soon find themselves flooded with data and struggling with the cost of managing it—if they aren’t already.

Companies are brought to this point by the strong belief, now widespread in business, that the worst thing they can do with a piece of data is to throw it away. It’s an understandable impulse. They want to keep all their data because they never know which innocent piece of data might suddenly become hugely important, today or tomorrow.

Data accumulation has become a problem for companies both large and small. Not long ago, 500 terabytes of data used to be solely the concern for a Fortune 500 company. Today, it is a problem for lots of small and midsize companies as well.

For example, a healthcare customer stores medical images such as CT scans and mammograms for multiple radiology offices. Recently they upgraded to high-resolution 3D mammography, which meant their storage requirements for each image went from a few gigabytes to tens of gigabytes. In a short space of time, they went from an organisation dealing with data on a few hundred TB-scale to one needing to manage storage on a petabyte scale.

Solutions that worked perfectly fine just five years ago simply don’t work any longer. In these ‘old days’, companies would just drop data into their existing storage infrastructure. But when their data is growing at 120% per year, that’s not an option because they will end up either swapping out their entire storage infrastructure every year or two or adding disparate storage silos to address data growth.  Either option is bad for business in terms of cost, complexity, cohesion and continuity.

Today, companies need to figure out quickly which information is critical to their business, and which information can be neglected. They need to understand which data should be pushed to the cloud, so it’s always available, and which data can be stored locally.

The answer to this challenge lies in self-organising storage that applies intelligence and, specifically, machine learning to the management of information. In this scenario, real-time analytics done by the storage system itself decide what the optimal placement for data is and what the optimal protection for any element of information within a dataset may be. This is the only way people are going to keep up with the explosive growth of data that has reached a scale and size that humans can no longer handle effectively.

The future where machine-learning algorithms go through the content of data and establish relationships has already begun. As a result, organisations are able to organise data in its proper context and ensure that datasets with similar context are matched together, making it easier to manage and make sense of mountains of information.

To tackle an organisation’s growing data issue, my advice is to start with digestible chunks. Don’t try to pretend you know what the world of technology and data storage is going to bring next year. Don’t go out and buy a storage system that hopefully will be sufficient five years from now. The digital world moves far too fast and is far too fluid for that kind of guesswork.

Instead, start small with a strategy that scales over time. Take sensible steps that you can test and prove as you go. You’ll be light on your feet and ready to respond as business changes, as the type of data being stored changes, as the technologies in the market change, as regulations change, as your company changes.

So, go ahead and hoard that data. But be smart about it. Be nimble and flexible. Go with a system that’s scalable and requires a relatively small investment now, so you can develop with agility and accuracy in the years ahead.

Dimension Data nabs three Cisco partner awards
Cisco announced the awards, including APJ Partner of the Year, at a global awards reception during its annual partner conference.
WatchGuard’s eight (terrifying) 2019 security predictions
The next evolution of ransomware, escalating nation-state attacks, biometric hacking, Wi-Fi protocol security, and Die Hard fiction becomes reality.
Why the adoption of SAP is growing among SMEs
Small and medium scale enterprises are emerging as lucrative end users for SAP.
Exclusive: How the separation of Amazon and AWS could affect the cloud market
"Amazon Web Services is one of the rare companies that can be a market leader but remain ruthlessly innovative and agile."
HPE extends cloud-based AI tool InfoSight to servers
HPE asserts it is a big deal as the system can drive down operating costs, plug disruptive performance gaps, and free up time to allow IT staff to innovate.
Digital Realty opens new AU data centre – and announces another one
On the day that Digital Realty cut the ribbon for its new Sydney data centre, it revealed that it will soon begin developing another one.
A roadmap to AI project success
Five keys preparation tasks, and eight implementation elements to keep in mind when developing and implementing an AI service.
The future of privacy: What comes after VPNs?
"75% of VPN users said they are seeking a better solution for cloud networks."