ANZ 2023 Predictions: What Lies Ahead for Data Management

There is no doubt that COVID-19 accelerated the push towards digital business, with companies forced to respond quickly to market forces and pivot their offerings to a purely online model. As the dust settles on this period in time, the general consensus is that many organisations have embraced this new digital business model for good. 

Even if brick-and-mortar businesses do come back full swing, a considerable amount of their business transactions will still take place digitally. Further to this, consumers expect more from their service providers now – they expect a fast, seamless digital transaction every time and will not hesitate to find an alternative if their needs are not met. In fact, digital and data-driven business innovation may differentiate the leaders from the rest of the pack.

Data is at the heart of digital business, and managing data and related infrastructure with proper strategy and planning will be key to business success. That's why we foresee a lot of innovation taking place in data management infrastructure and architecture-related fields. Here are the top five trends related to data and analytics that will have the most impact in 2023. 

Trend #1: As recession looms, companies will look to optimise infrastructure cost 

Whether Australia is in recession or not, companies are actively cutting costs and reducing IT infrastructure, which has always been an easy choice for CEOs. While computer and storage costs continue to be reduced through the usage of cloud, it can still lead to huge bills for organisations, given their heavy investments in data and analytics infrastructure. Thanks in part to the breadth of choices of storage, computing, and applications, companies often take a rip-and-replace strategy to modernise their data and analytics efforts. That approach is not only costly, but it can often lead to disruption in IT operations. In 2023, more companies will see IT focusing on modern, non-disruptive ways to update their IT infrastructure, whether their data resides entirely in one cloud, multiple clouds, or in a hybrid environment, including on-premises. 

Trend #2: While multi-cloud gets real, FinOps in cloud becomes necessary 

For many companies, strategic data assets are spread across multiple clouds and geographical locations, whether that is because various business units or locations have their preferred cloud service provider (CSP) or because mergers and acquisitions have led these assets to reside in different cloud providers' boundaries. As more data continues to move to the cloud and different geographies see the prominence of certain cloud providers vs. others, there is accelerated adoption of multi-cloud architecture for multinational corporations. Currently, there is no easy way to manage and integrate data and services across these different CSPs. Failure to address this problem always results in data silos and a fragmented approach to data management, leading to data access and data governance complications.  

Also, contrary to popular belief, cloud costs are increasingly becoming a material expense due to the sheer volume of data and related egress charges, to name a few. For many organisations, cloud investments do not deliver the economic and business benefits intended. As a result, they are leveraging FinOps to provide a framework for controlling cloud costs and usage, identify cost vs. value, and understand ways to optimally manage it across modern hybrid and multi-cloud environments. In the coming year, expect FinOps to gain momentum as a critical initiative to help companies better manage their hybrid-cloud and multi-cloud spending.  

Trend #3: Accelerated adoption of data fabric and data mesh 

Over the past two decades, data management has gone through cycles of centralisation vs. decentralisation, including databases, data warehouses, cloud data stores, data lakes, etc. While the debate over which approach continues, the last few years have proven that data is more distributed than centralised for most organisations. While there are numerous options for deploying enterprise data architecture, 2022 saw accelerated adoption of two data architectural approaches to better manage and access the distributed data – data fabric and data mesh. There is an inherent difference between the two. Data fabric is a composable stack of data management technologies, and data mesh is a process orientation for distributed groups of teams to manage enterprise data as they see fit. Both are critical to enterprises that want to manage their data better. Easy access to data and ensuring it's governed and secure is important to every data stakeholder – from data scientists all the way to executives. Afterall, it is critical for dashboarding and reporting, advanced analytics, machine learning, and AI projects.  

Both data fabric and data mesh can play critical roles in enterprise-wide data access, integration, management and delivery when constructed properly and with the right data infrastructure in place. So in 2023, expect a rapid increase in the adoption of both architectural approaches within mid-to-large size enterprises. 

Trend #4: Ethical AI becomes paramount as commercial adoption of AI-based decision-making increases 

Companies across industries are accelerating the usage of AI for their data-based decision-making. Whether it's about social media platforms suppressing posts, connecting healthcare professionals with patients, or large wealth management banks granting credits to their end consumers.

However, when artificial intelligence decides the end result, there is currently no way to suppress the inherent bias in the algorithm. That is why the Australian Federal Government has launched reviews of the regulatory and legal framework for AI to ensure that our legal system remains fit for purpose. A specific example of this is the Digital Technology Taskforce's enquiry into automated decision-making and AI regulation, which was launched in March 2022. Other regions have also undertaken such reviews with regulations, such as the proposed EU Artificial Intelligence Act and Canada's Bill C-27 (which may become the Artificial Intelligence and Data Act if enacted), starting to put a regulatory framework around the use of AI in commercial organisations. These new regulations classify the risk of AI applications as unacceptable, high, medium, or low risk and prohibit or manage the use of these applications accordingly.  

In 2023, organisations will need to be able to comply with these proposed regulations, including ensuring privacy and data governance, algorithmic transparency, fairness and non-discrimination, accountability, and auditability. With this in mind, organisations have to implement their own frameworks to support ethical AI, e.g. guidelines for trustworthy AI, peer review frameworks, and AI Ethics committees. As more and more companies put AI to work, ethical AI is bound to become more important than ever in the coming year. 

Trend #5:  Augmentation of data quality, data preparation, metadata management and analytics 

While the end result of many data management efforts is to feed advanced analytics and support AI and ML efforts, proper data management itself is pivotal to an organisation's success. Data is often called the new oil because data and analytics-based insights are constantly propelling business innovation. As organisations accelerate their usage of data, it's critical for companies to keep a close eye on data governance, data quality and metadata management. Yet, with the growing amount of volume, variety and velocity of data, these various aspects of data management have become too complex to manage at scale. Consider the amount of time data scientists and data engineers spend finding and preparing the data before they can start utilising it. That is why augmented data management has recently been embraced by various data management vendors, where, with the application of AI, organisations are able to automate many data management tasks.  

According to some of the top analyst firms, each layer of a data fabric (namely data ingestion, data processing, data orchestration, data governance, etc.) should have AI/ML baked into it to automate each stage of the data management process. In 2023, augmented data management will find strong market traction, helping data management professionals focus on delivering data-driven insights rather than being held back with routine administrative tasks. 

While these are the five most important trends in our mind, there are other areas of data and analytics practice which will shape how digital business will not only survive but thrive in 2023 and beyond. The last three years have definitely taught us that digital business is not really a fall-back option when the world cannot meet in person; rather, it is where the future lies. Hopefully, your organisation can gain some insights from this article as you lay out your digital business plan.