IT Brief Australia logo
Technology news for Australia's largest enterprises
Story image

Does your data stack up? What it takes to build a great modern data stack

By Contributor
Mon 4 Apr 2022

Article by Technology Partners from Fivetran alliances manager, Ignatius Bourke and Sisense APAC director of channels and alliances, Rohan Persaud.

While the business world remains deep in the throes of sweeping uncertainty amidst the pandemic and volatile economic events, one thing remains certain: organisations need to be data-driven to survive.

It’s time to take a new approach. Businesses can’t rely on what used to work. Instead, they must focus on what will work. And to that, data is the answer. But not just any data; organisations need to take advantage of the right data at the right time to make swift decisions for their own good. 

“When you look at the new business norm today, it’s all about the shift from descriptive analytics or the, ‘What happened in the past?’ to predictive analytics, ‘What’s going to happen?’ to, even more, prescriptive analytics, which says, ‘What do I need to do?’” says Sisense, APAC director of channels and alliances, Rohan Pursaud.

“At the end of the day, a data strategy looks at new business models in the right context thanks to insights.”

According to Gartner, however, many organisations are struggling to get business value from their analytics. Through 2022, only 20% of analytic insights will deliver business outcomes. And when it comes to AI, Gartner predicts that 80% of projects will remain “alchemy, run by wizards whose talents will not scale in the organisation” this year.

Implementing analytics projects or an analytics organisation is one thing, but extracting value from it is quite another. That’s where a modern data stack (MDS) comes in. Simply put, an MDS is a suite of tools used for data integration. And it’s a boon for businesses of all sizes. 

Breaking down the modern data stack

The function of an MDS is to facilitate the ingestion of data from multiple sources into a centralised location where it can then be analysed. 

  • It’s made of four basic components:
  • data ingestion solution
  • data warehouse or data lake
  • transformation tool; and
  • business intelligence (BI) tool

But assembling this MDS doesn’t have to be complex. Instead, it’s all about choosing combinations that best fit your business. After all, the ultimate goal of an MDS hinges on using data to create the best customer experience possible.

An MDS is more than the sum of its parts or its specific tools. What makes a stack modern is its ability to meet the different demands caused by modern data problems at each phase of the data lifecycle. What happens to data from when it is created to when it is ready to be actioned as an insight.

“That’s how data strategy is tied to growth – it’s about tackling new complex questions, questions we never anticipated,” adds Rohan.

The starting point: ingestion tools

The first step in building a data stack is identifying exactly what you’re trying to calculate and measure. This will help guide you toward the type of data you need to source. 

There are many ingestion tools to choose from, ranging from simple open-source software to more automated and lower-maintenance solutions.

The power of business intelligence (BI)

BI tools are the final component of a great MDS. Increasingly powerful and easier to use, BI tools allow users to access data insights from anywhere. They can be leveraged to facilitate more efficient teamwork, increase business agility, make sense of the outpouring of data, and better personalise marketing messages. 

Transformation and data warehouses

A data warehouse’s primary function is to provide a central repository for sourced data and prepare it for reporting and analytics. A data lake is also helpful and recommended for all historical data, whether it’s used for immediate analysis or analysis later on. Choosing the right data ingestion tool makes it near effortless to move data from sources to repositories. 

Open-source tools, like data build tool (DBT), can further simplify the process when coupled with connectors like Fivetran to transform data that’s been loaded into the warehouse.

“We enable organisations to monetise their data by reducing the heavy lifting. You save the effort of three people, two years of work, and about $400,000 of a project scope by switching to Fivetran,” says Technology Partners at Fivetran alliances manager, Ignatius Bourke.

7 benefits of a modern data stack:

  • Reduced costs: An MDS can reduce data engineering costs by more than 90% by eliminating the need to build and maintain data pipelines or normalise data from denormalised APIs. 
  • Increased productivity: An MDS allows your data team to be more productive because available data increases without consuming in-house resources.
  • More data projects: With more time and more data, your team can focus on new analytics projects.
  • Extra time: An MDS will drastically decrease report generation time and ensure reports are up-to-date, saving your company time. 
  • Better data reliability: The burden of extract, transform, load (ETL) maintenance will be eliminated, and data reliability will improve with an MDS.
  • Improved data literacy: Intuitive and easy-to-use, modern BI tools are designed to make data accessible to business users and technical employees alike.
  • Creation of new metrics: Additional data sources and a simple-to-use BI tool allows business to create a host of new metrics. This is made possible in two ways: by richer data that enables new types of cross-analysis and by greater data access across departments, allowing more employees to propose new metrics based on their specific competencies.

Access more insights from Technology Partners from Fivetran alliances manager, Ignatius Bourke, and Sisense APAC director of channels and alliances, Rohan Persaud here.

Related stories
Top stories
Story image
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Let’s clear the cloud visibility haze with app awareness
Increasingly, organisations are heading for the cloud, initiating new born-in-the-cloud architectures and migrating existing applications via ‘lift and shift’ or refactoring.
Story image
Remote Working
Successful digital transformation in the hybrid work era is about embracing shifting goalposts
As organisations embraced remote working, many discovered they lacked the infrastructure needed to support history’s first global load test of remote work capabilities.
Story image
Vectra AI
Understanding the weight on security leader’s shoulders, and how to shift it
Millions of dollars of government funding and internal budgets are being funnelled into cybersecurity to build resilience against sophisticated threats, indicating how serious this issue has become.
Story image
Alarming surge in Conti Ransomware Group activity - report
A new report has identified a 7.6 per cent increase in the number of vulnerabilities tied to ransomware in Q1 2022.
Story image
ChildFund launches new campaign to protect children online
ChildFund says WEB Safe & Wise aims to protect children from sexual exploitation and abuse online while also empowering them to become digitally savvy. 
Story image
Customer experience
Research unveils precarious customer loyalty for retailers
New research has found customers are reassessing established brand loyalties as their priorities and behaviours shift.
Story image
New Relic enters multi-year partnership with Microsoft Azure
New Relic has announced a strategic partnership with Microsoft to help enterprises accelerate cloud migration and multi-cloud initiatives. 
Story image
9/10 Aussies to stop spending if personal data compromised
"Based on the patterns we are seeing among Australian consumers, it is evident that trust in a brand is exceptionally important."
Story image
Data Center
Preventing downtime costs and damage with Distributed Infrastructure Management
Distributed Infrastructure Management (DIM) can often be a lifeline for many enterprises that work with highly critical ICT infrastructure and power sources.
Story image
Sift shares crucial advice for preventing serious ATO breaches
Are you or your business struggling with Account Takeover Fraud (ATO)? One of the latest ebooks from Sift can provide readers with the tools and expertise to help launch them into the new era of account security.
Story image
Digital Transformation
How to modernise legacy apps without compromising security
At a time when digital transformation has become central to business, even the most important applications come with a ‘use-by’ date.
Story image
Artificial Intelligence
AI-based email security platform Abnormal Security valued at $4B
"A new breed of cybersecurity solutions that leverage AI is required to change the game and stop the rising threat of sophisticated and targeted email attacks."
Story image
Could your Excel practices be harming your business?
While Excel has been the de-facto standard for budgeting, planning, and forecasting, is it alone, enough to support organisations in the global marketplace that’s facing rapid changes due to digital transformation?
Story image
Nutanix study reveals financial services sector lagging with multicloud adoption
Nutanix has released new research that reveals the financial services sector is lagging behind when it comes to multicloud adoption.
Story image
Artificial Intelligence
How to ensure ethical deployment of AI implementations
The increase in automation and machine technology such as AI and machine learning has unlocked a whole new level of scale and service to organisations. 
Story image
Power at the edge: the role of data centers in sustainability
The Singaporean moratorium on new data center projects was recently lifted, with one of the conditions being an increased focus on power efficiency and sustainability.
Story image
HINDSITE wins Aerospace Xelerated Pitch Challenge with solution to support Boeing
Brisbane-based startup HINDSITE was the winner of the first ever Pitch Challenge organised by Aerospace Xelerated in partnership with Queensland XR Hub. 
Threat actors are exploiting weaknesses in interconnected IT/OT ecosystems. Darktrace illuminates your entire business and takes targeted action to stop emerging attacks.
Link image
Story image
Airwallex launches new bank feed integration with NetSuite
Airwallex has launched a new bank feed integration with NetSuite, developed in partnership with NetSuite solution partner, Onlineone.
Story image
Digital Transformation
The impact of COVID-19 on healthcare environments and care delivery
The COVID-19 pandemic has revolutionised the healthcare industry while overcoming staff shortages, social distancing requirements, and lockdowns.
Story image
More than 40% of banks worried about cloud security - report
Publicis Sapient's new report finds security and the lack of cloud skills and internal understanding of business benefits are big obstacles for banks moving to the cloud.
Story image
Digital Transformation
Pluralsight and Ingram Micro Cloud team up on cloud initiative
Pluralsight has teamed with Ingram Micro Cloud to build upon cloud competence and maturity internally, and externally support partners’ capabilities.
Story image
Data solutions
South Australia state satellite makes significant progress
South Australia’s first state satellite has successfully completed the Critical Design Review (CDR), moving it closer to providing tangible data solutions.
Story image
Data Protection
Barracuda launches new capabilities for API Protection
"Every business needs this type of critical protection against API vulnerabilities and automated bot attacks," Barracuda says.
Story image
Telstra enters into new RSP agreement with Opticomm
Telstra has entered into an RSP agreement with Opticomm (A Uniti Group Limited subsidiary) to provide network fibre services to customers.
Story image
Supply chain
Jetstack promotes better security with supply chain toolkit
The web-based resource is designed to help organisations evaluate and plan the crucial steps they need to establish effective software supply chain security.
Story image
Cybersecurity starts with education
In 2021, 80% of Australian organisations responding to the Sophos State of Ransomware study reported being hit by ransomware. 
Story image
New vulnerabilities found in Nuspire’s Q1 2022 Threat Report
“Threat actors are quickly adjusting their tactics and these exploits tend to get industry attention, but the threat posed by older and attacks still persists."
For every 10PB of storage run on HyperDrive vs. comparable alternatives, an estimated 6,656 tonnes of CO₂ are saved by reduced energy consumption alone over its lifespan. That’s the equivalent of taking nearly 1,500 cars off the road for a year.
Link image
Story image
Legrand unveils Nexpand, a data center cabinet platform
Legrand has unveiled a new data center cabinet platform, Nexpand, to offer the necessary scalability and future-proof architecture for digital transformation.
Story image
Cradlepoint expands its Cellular Intelligence capabilities
Cradlepoint has announced additional Cellular Intelligence capabilities with its NetCloud service.
Story image
Cloud Security
Aqua Security createa unified scanner for cloud native security
“By integrating more cloud native scanning targets into Trivy, such as Kubernetes, we are simplifying cloud native security."
Story image
Workato unveils enhancements to enterprise automation platform
"The extra layer of protection with EKM, zero-logging, and hourly key rotation gives customers a lot more visibility and control over more sensitive data."
Story image
A10 Networks finds over 15 million DDoS weapons in 2021
A10 Networks notes that in the 2H 2021 reporting period, its security research team tracked more than 15.4 million Distributed Denial-of-Service (DDoS) weapons.
Story image
Rubrik Security Cloud marks 'next frontier' in cybersecurity
"The next frontier in cybersecurity pairs the investments in infrastructure security with data security giving companies security from the point of data."
Story image
Application Security
What are the DDoS attack trend predictions for 2022?
Mitigation and recovery are vital to ensuring brand reputation remains solid in the face of a Distributed Denial of Service (DDoS) attack and that business growth and innovation can continue.
Story image
A third of companies paying ransom don’t recover data - report
Veeam's report finds 76% of businesses who are victims of cyberattacks paid the ransom to recover data, but a third were still unable to get their information back.
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
Infoblox's State of Security Report spotlights Australian remote work hazards
Attackers exploit weak WiFi, remote endpoints, and the cloud, costing 50% of organisations over $1.3 million in breach damages.
Story image
Qualys updates Cloud Platform solution with rapid remediation
The new update is designed to enable organisations to fix asset misconfigurations, patch OS and third-party applications, and deploy custom software.
Story image
Artificial Intelligence
SAS announces new products amid cloud portfolio success
Analytics and AI company SAS is deepening its broad industry portfolio with offerings that support life sciences, energy, and martech.
Story image
Remote Working
Australia’s remote workers face connectivity and security issues
SOTI's new report finds better video conferencing technology and improved security measures are top concerns for remote workers in Australia.
Story image
Grasping the opportunity to rethink the metrics of a sustainable data centre
A data centre traditionally has two distinct operations teams: the Facility Operations team, and the IT Operations team. Collaboration between them is the key to defining, measuring, and delivering long-term efficiency and sustainability improvements.