IT Brief Australia logo
Technology news for Australia's largest enterprises
Story image

The trouble with data hoarding — and how to fix It

By Patrick Martlew
Wed 9 May 2018
FYI, this story is more than a year old

Article written by StorageCraft APAC head of sales Marina Brook

There’s a US reality TV show called Hoarders that features people who compulsively acquire stuff and are unwilling or unable to discard it. Eventually, they get buried.

A lot of companies are behaving the same way. They’ve become data hoarders. And, like those unfortunate folks on the TV show, they’ll soon find themselves flooded with data and struggling with the cost of managing it—if they aren’t already.

Companies are brought to this point by the strong belief, now widespread in business, that the worst thing they can do with a piece of data is to throw it away. It’s an understandable impulse. They want to keep all their data because they never know which innocent piece of data might suddenly become hugely important, today or tomorrow.

Data accumulation has become a problem for companies both large and small. Not long ago, 500 terabytes of data used to be solely the concern for a Fortune 500 company. Today, it is a problem for lots of small and midsize companies as well.

For example, a healthcare customer stores medical images such as CT scans and mammograms for multiple radiology offices. Recently they upgraded to high-resolution 3D mammography, which meant their storage requirements for each image went from a few gigabytes to tens of gigabytes. In a short space of time, they went from an organisation dealing with data on a few hundred TB-scale to one needing to manage storage on a petabyte scale.

Solutions that worked perfectly fine just five years ago simply don’t work any longer. In these ‘old days’, companies would just drop data into their existing storage infrastructure. But when their data is growing at 120% per year, that’s not an option because they will end up either swapping out their entire storage infrastructure every year or two or adding disparate storage silos to address data growth.  Either option is bad for business in terms of cost, complexity, cohesion and continuity.

Today, companies need to figure out quickly which information is critical to their business, and which information can be neglected. They need to understand which data should be pushed to the cloud, so it’s always available, and which data can be stored locally.

The answer to this challenge lies in self-organising storage that applies intelligence and, specifically, machine learning to the management of information. In this scenario, real-time analytics done by the storage system itself decide what the optimal placement for data is and what the optimal protection for any element of information within a dataset may be. This is the only way people are going to keep up with the explosive growth of data that has reached a scale and size that humans can no longer handle effectively.

The future where machine-learning algorithms go through the content of data and establish relationships has already begun. As a result, organisations are able to organise data in its proper context and ensure that datasets with similar context are matched together, making it easier to manage and make sense of mountains of information.

To tackle an organisation’s growing data issue, my advice is to start with digestible chunks. Don’t try to pretend you know what the world of technology and data storage is going to bring next year. Don’t go out and buy a storage system that hopefully will be sufficient five years from now. The digital world moves far too fast and is far too fluid for that kind of guesswork.

Instead, start small with a strategy that scales over time. Take sensible steps that you can test and prove as you go. You’ll be light on your feet and ready to respond as business changes, as the type of data being stored changes, as the technologies in the market change, as regulations change, as your company changes.

So, go ahead and hoard that data. But be smart about it. Be nimble and flexible. Go with a system that’s scalable and requires a relatively small investment now, so you can develop with agility and accuracy in the years ahead.

Related stories
Top stories
Story image
trust
9/10 Aussies to stop spending if personal data compromised
"Based on the patterns we are seeing among Australian consumers, it is evident that trust in a brand is exceptionally important."
Story image
Artificial Intelligence
AI-based email security platform Abnormal Security valued at $4B
"A new breed of cybersecurity solutions that leverage AI is required to change the game and stop the rising threat of sophisticated and targeted email attacks."
Story image
CRM
Zendesk announces new conversational CRM solutions
“The last few years have made it obvious that digital is the front door, convenience is paramount and relationships are anchored in conversations."
Story image
Workato
Workato unveils enhancements to enterprise automation platform
"The extra layer of protection with EKM, zero-logging, and hourly key rotation gives customers a lot more visibility and control over more sensitive data."
Story image
Microsoft
SAS Viya on Microsoft Azure to deliver 204% return - study
The Forrester Total Economic Impact study finds SAS Viya on Microsoft Azure brings a 204% return on investment over three years.
Story image
Cybersecurity
The 'A-B-C' of effective application security
Software applications have been a key tool for businesses for decades, but the way they are designed and operated has changed during the past few years.
Story image
SAS
New SAS service overcomes subscription fatigue for media companies
SAS has launched SAS 360 Match which helps media companies move towards a AVOD model to generate revenue as subscribers cancel.
Story image
Digital Transformation
Unlocking the next digital frontier for educational institutions
Understanding where to invest in technology can be challenging for education institutions, especially after the COVID-19 disruptions.
Story image
Excel
Could your Excel practices be harming your business?
While Excel has been the de-facto standard for budgeting, planning, and forecasting, is it alone, enough to support organisations in the global marketplace that’s facing rapid changes due to digital transformation?
Story image
Artificial Intelligence
SAS unveils AI experience to improve kids' batting abilities
SAS has created The Batting Lab, an interactive experience using AI, computer vision and IoT analytics to help kids improve their baseball and softball swings.
Story image
Cybersecurity
Video: 10 Minute IT Jams - An update from IronNet
Michael Ehrlich joins us today to discuss the history of IronNet and the crucial role the company plays in the cyber defence space.
Story image
Wireless
SmartCIC, BICS partner to expand wireless service options
SmarCIC has partnered with BICS to increase choice for organisations using fixed wireless services, expanding existing carrier relationships for its CELLSMART division.
Story image
BeyondTrust
BeyondTrust integrates Password Safe solution with SailPoint
BeyondTrust has announced the integration of BeyondTrust Password Safe with SailPoint identity security offerings.
Story image
Artificial Intelligence
Updates from Google Workspace set to ease hybrid working troubles
Google Workspace has announced a variety of new features which will utilise Google AI capabilities to help make hybrid working situations more efficient and effective.
Darktrace
Threat actors are exploiting weaknesses in interconnected IT/OT ecosystems. Darktrace illuminates your entire business and takes targeted action to stop emerging attacks.
Link image
Story image
Sift
Sift shares crucial advice for preventing serious ATO breaches
Are you or your business struggling with Account Takeover Fraud (ATO)? One of the latest ebooks from Sift can provide readers with the tools and expertise to help launch them into the new era of account security.
Story image
SaaS
Atturra partners with Focus HQ to support Aus organisations
Atturra has executed a partnership agreement with Focus HQ, to resell and support the company's Australian developed SaaS-based portfolio management platform.
Story image
Safety
Voxel hits total funding of $18M following ongoing wins
Since raising its seed round in September, Voxel has grown at pace, by decreasing on-site injuries by upwards of 80% and increasing operational productivity.
Story image
Telstra
Telstra, Google and Accenture launch 5G AR experience for AFL
Telstra, Google and Accenture are developing a new 5G powered augmented reality (AR) experience at Melbourne's Marvel Stadium for the footy season.
Story image
Fortinet
Fortinet's Security Fabric hits new record for integrations
The Fortinet Security Fabric has surpassed 500 technology integrations with more than 300 Fabric-Ready Technology Alliance Partners.
Story image
SaaS
Absolute Software expands Secure Access product offering
Absolute Software is enhancing its Secure Access product portfolio, enabling minimised risk exposure and optimised user experiences in the hybrid working environment.
Softiron
For every 10PB of storage run on HyperDrive vs. comparable alternatives, an estimated 6,656 tonnes of CO₂ are saved by reduced energy consumption alone over its lifespan. That’s the equivalent of taking nearly 1,500 cars off the road for a year.
Link image
Story image
Remote Working
IT teams deploy powerful technologies to enable remote work
"We found that IT teams mastered the challenges of remote work last year in large part by employing powerful yet easy-to-use technologies."
Story image
Sustainability
Grasping the opportunity to rethink the metrics of a sustainable data centre
A data centre traditionally has two distinct operations teams: the Facility Operations team, and the IT Operations team. Collaboration between them is the key to defining, measuring, and delivering long-term efficiency and sustainability improvements.
Story image
Testing
Google and CSIRO use AI to help protect the Great Barrier Reef
Google has partnered with CSIRO in Australia to implement AI solutions that help protect the Great Barrier Reef.
Story image
Phishing
Google reveals new safety and security measures for users
Google's new measures include automatic two step verification, virtual cards and making it easier to remove contact information on Google Search results.
Story image
Ivanti
Ivanti and Lookout bring zero trust security to hybrid work
Ivanti and Lookout have joined forces to help organisations accelerate cloud adoption and mature their zero trust security posture in the everywhere workplace.
Story image
Adyen
Adyen expands partnership with Afterpay as BNPL payments increase
Adyen has expanded its partnership with AfterPay allowing more of Adyen’s merchants in more countries worldwide to use the BNPL provider.
Story image
Cybersecurity
A10 Networks finds over 15 million DDoS weapons in 2021
A10 Networks notes that in the 2H 2021 reporting period, its security research team tracked more than 15.4 million Distributed Denial-of-Service (DDoS) weapons.
Story image
Manufacturing
HINDSITE wins Aerospace Xelerated Pitch Challenge with solution to support Boeing
Brisbane-based startup HINDSITE was the winner of the first ever Pitch Challenge organised by Aerospace Xelerated in partnership with Queensland XR Hub. 
Story image
Alteryx
Decision Inc. Australia enters partnership with Alteryx
Independent data and analytics consultancy Decision Inc. Australia has partnered with automated analytics company Alteryx, expanding its offering to clients.
Exabeam
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
Remote Working
How organisations can meet employees' changing expectations
The global employment market has shifted dramatically in favour of employees, sparking the so-called great resignation, in which people are leaving unsatisfying roles in search of greener pastures.
Story image
Artificial Intelligence
SAS launches human-focused responsible innovation initiative
SAS has launched a responsible innovation initiative, furthering its commitment to equity and putting people first.
Story image
Artificial Intelligence
Clear Dynamics closes $35M funding round, invests in global growth
The funding is a major milestone and speaks to Clear Dynamics’ vision for AI-enabled ‘composable’ enterprise software, the company states.
Story image
Cybersecurity
Managed service providers: effective scoping to avoid costly vendor pitfalls
Managed security services are outsourced services focusing on the security and resilience of business networks.
Story image
Application Security
What are the DDoS attack trend predictions for 2022?
Mitigation and recovery are vital to ensuring brand reputation remains solid in the face of a Distributed Denial of Service (DDoS) attack and that business growth and innovation can continue.
Story image
Remote Working
How zero trust and SD-WANs can support productive remote working
The way people connect with applications and data has changed, users are remotely accessing resources that could be stored anywhere from a corporate data center to the cloud.
Story image
Data Center
Preventing downtime costs and damage with Distributed Infrastructure Management
Distributed Infrastructure Management (DIM) can often be a lifeline for many enterprises that work with highly critical ICT infrastructure and power sources.
Story image
Sustainability
Power at the edge: the role of data centers in sustainability
The Singaporean moratorium on new data center projects was recently lifted, with one of the conditions being an increased focus on power efficiency and sustainability.
Story image
Talend
Talend introduces new data health solutions for businesses
Talend has announced its latest version of Talend Data Fabric, with the release of Talend Trust Score enabling data teams to establish a foundation for data health.
Story image
Ransomware
Cybersecurity starts with education
In 2021, 80% of Australian organisations responding to the Sophos State of Ransomware study reported being hit by ransomware. 
Story image
SaaS
Cloudflare launches instant serverless database for dev teams
"Today we’re announcing our first serverless database which we expect will quickly become one of the largest databases in the world."
Story image
Telstra
Telstra and Silver Trak Digital delivers 5G to the cinemas
Telstra and Silver Trak Digital say they've launched Australia's fastest and most secure delivery of content over 5G for cinemas.