Does your data stack up? What it takes to build a great modern data stack

Article by Technology Partners from Fivetran alliances manager, Ignatius Bourke and Sisense APAC director of channels and alliances, Rohan Persaud.

While the business world remains deep in the throes of sweeping uncertainty amidst the pandemic and volatile economic events, one thing remains certain: organisations need to be data-driven to survive.

It's time to take a new approach. Businesses can't rely on what used to work. Instead, they must focus on what will work. And to that, data is the answer. But not just any data; organisations need to take advantage of the right data at the right time to make swift decisions for their own good.

"When you look at the new business norm today, it's all about the shift from descriptive analytics or the, 'What happened in the past?' to predictive analytics, 'What's going to happen?' to, even more, prescriptive analytics, which says, 'What do I need to do?'" says Sisense, APAC director of channels and alliances, Rohan Pursaud.

"At the end of the day, a data strategy looks at new business models in the right context thanks to insights.

According to Gartner, however, many organisations are struggling to get business value from their analytics. Through 2022, only 20% of analytic insights will deliver business outcomes. And when it comes to AI, Gartner predicts that 80% of projects will remain "alchemy, run by wizards whose talents will not scale in the organisation" this year.

Implementing analytics projects or an analytics organisation is one thing, but extracting value from it is quite another. That's where a modern data stack (MDS) comes in. Simply put, an MDS is a suite of tools used for data integration. And it's a boon for businesses of all sizes.

Breaking down the modern data stack

The function of an MDS is to facilitate the ingestion of data from multiple sources into a centralised location where it can then be analysed.

• It's made of four basic components:
• data ingestion solution
• data warehouse or data lake
• transformation tool; and
• business intelligence (BI) tool

But assembling this MDS doesn't have to be complex. Instead, it's all about choosing combinations that best fit your business. After all, the ultimate goal of an MDS hinges on using data to create the best customer experience possible.

An MDS is more than the sum of its parts or its specific tools. What makes a stack modern is its ability to meet the different demands caused by modern data problems at each phase of the data lifecycle. What happens to data from when it is created to when it is ready to be actioned as an insight.

"That's how data strategy is tied to growth – it's about tackling new complex questions, questions we never anticipated," adds Rohan.

The starting point: ingestion tools

The first step in building a data stack is identifying exactly what you're trying to calculate and measure. This will help guide you toward the type of data you need to source.

There are many ingestion tools to choose from, ranging from simple open source software to more automated and lower-maintenance solutions.

The power of business intelligence (BI)

BI tools are the final component of a great MDS. Increasingly powerful and easier to use, BI tools allow users to access data insights from anywhere. They can be leveraged to facilitate more efficient teamwork, increase business agility, make sense of the outpouring of data, and better personalise marketing messages.

Transformation and data warehouses

A data warehouse's primary function is to provide a central repository for sourced data and prepare it for reporting and analytics. A data lake is also helpful and recommended for all historical data, whether it's used for immediate analysis or analysis later on. Choosing the right data ingestion tool makes it near effortless to move data from sources to repositories.

Open source tools, like data build tool (DBT), can further simplify the process when coupled with connectors like Fivetran to transform data that's been loaded into the warehouse.

"We enable organisations to monetise their data by reducing the heavy lifting. You save the effort of three people, two years of work, and about $400,000 of a project scope by switching to Fivetran," says Technology Partners at Fivetran alliances manager, Ignatius Bourke.

7 benefits of a modern data stack:

• Reduced costs: An MDS can reduce data engineering costs by more than 90% by eliminating the need to build and maintain data pipelines or normalise data from denormalised APIs. 
• Increased productivity: An MDS allows your data team to be more productive because available data increases without consuming in-house resources.
• More data projects: With more time and more data, your team can focus on new analytics projects.
• Extra time: An MDS will drastically decrease report generation time and ensure reports are up-to-date, saving your company time. 
• Better data reliability: The burden of extract, transform, load (ETL) maintenance will be eliminated, and data reliability will improve with an MDS.
• Improved data literacy: Intuitive and easy-to-use, modern BI tools are designed to make data accessible to business users and technical employees alike.
• Creation of new metrics: Additional data sources and a simple-to-use BI tool allows business to create a host of new metrics. This is made possible in two ways: by richer data that enables new types of cross-analysis and by greater data access across departments, allowing more employees to propose new metrics based on their specific competencies.

Access more insights from Technology Partners from Fivetran alliances manager, Ignatius Bourke, and Sisense APAC director of channels and alliances, Rohan Persaud here.