Why the data warehouse is back and better than ever
The data warehouse has acquired something of a reputation for being yesterday's technology in recent times. Businesses and organisations have switched their focus from developing and running their own centralised repositories for data to exploring digital era alternatives.
But that's not to say the data warehouse itself is lacking in merit. In fact, the reverse. As data continues to play an increasingly important role in the prosperity – and survival – of companies and organisations of all stripes, a thoughtful data storage strategy has become more important than ever.
Infrastructure for a different era
It's easy to home in on the shortcomings of the traditional data warehouse model. Around in some guise since the 1970s, the data warehouse was traditionally regarded as a major capital purchase. Organisations would estimate their data requirements two to six years ahead, based on a maximum use scenario, and purchase software and hardware accordingly.
It was a model which could act as a disincentive for storing and analysing large amounts of data for commercial gain, courtesy of the fact that doing so was expensive and could result in the capacity of the data warehouse being exceeded sooner than expected.
Once that occurred, organisations faced a choice between reducing the amount of data they retained and shelling out big bucks for additional capacity when, oftentimes, only an incremental increase was required.
Data storage for data-driven days
Embracing data analytics has become a key goal for Australian organisations, as they attempt to transform digitally and target new markets and customers.
The perils of not doing so have been well documented. According to a report from Gartner analyst Alan Duncan, developing a data-driven corporate culture is vital to the success of medium-sized enterprises in particular, as it's the means by which they'll uncover new digital business opportunities.*
A deeper understanding of customers and better-informed planning and decision making are two of the benefits a data-driven culture can deliver.
The transformation begins by empowering employees to access a deep pool of data, from inside and outside the organisation. This is unlikely to happen if data storage is on rations or the infrastructure in use doesn't allow for the collection, retention and analysis of an array of structured and unstructured data.
The opportunity cost of an abstemious storage model is impossible to quantify but potentially immense.
Enterprises are increasingly finding that the most valuable insights generated by data analysis don't necessarily come from the mission-critical data, such as transaction records, which they've always stored as a matter of course.
Analysing weblogs and social media interactions – data which may have been viewed as ephemeral or low in value, historically – can yield previously undiscovered gems that result in competitive advantage or profitable new business opportunities.
Data warehousing – but not as we know it
Implementing a cloud-based data warehouse can eliminate the need to decide whether data is mission critical or ripe for discarding before there's been an opportunity to analyse it.
The pay-as-you-go model scotches the storage scarcity mindset and opens the possibility of exploiting a plenitude of data to provide answers to business questions and challenges, both longstanding and emerging.
Unlike its traditional precursor, a cloud-based data warehouse can store both structured and semi-structured data. That makes it an ideal repository for the sort of stream data which the digital economy and the Internet of Things are generating in abundance. It can hold the key to competitiveness via continuous improvement. That's the North Star for Australian enterprises which aspire to survive and thrive, post the Fourth Industrial Revolution.
