Data security and governance to take centre stage in 2023

Data security and governance will take centre stage in 2023, according to new trends predictions from Databricks. 

Bede Hackney, Regional Vice President at Databricks ANZ, says the spate of high-profile cyber attacks against Australian organisations in the latter half of the year has brought data security to the top of the corporate agenda. 

"With new laws just passed that will see organisations face up to $50 million for serious data breaches, we are likely to see even greater focus placed on ensuring data security," he says. 

"However, Australia must be careful this doesn't come at the expense of innovation. Leveraging open source technology will be critical to getting this balance right. 

"Advances in secure data-sharing standards based on open source change the equation because enterprises now have a way of securely sharing their data while avoiding the pitfalls of opacity and getting locked into a single vendor. Entering 2023, cybersecurity will be top of mind and we will see an uptick in 'secure sharing', the idea and ability to share data internally and externally while centrally governing its access and without duplicating the data."

IT Will Expand its Hiring Criteria  

Hackney says finding and retaining high-calibre IT talent has been a critical challenge for Australia this past year. 

"According to the National Skills Commission, nearly one third of the most in-demand occupations are in the ICT sector. This challenge is only going to continue if Australia is to meet the goal set by the Australian government and Tech Council of Australia of 1.2 million tech jobs by 2030," he says. 

"We foresee a softening of the IT market as companies look to grow more prudently next year. As part of that, companies will need to look at hiring differently. We think we'll see companies hire more employees early in their careers and also look to expand things like internships and year-end programs to support that."

Hackney says new graduates are also increasingly wanting to work with modern, open-source technology stacks to develop skills they can take with them throughout their careers. 

"Selecting a leading data platform will help organisations attract the best talent while also reaping the benefits from the technology itself. With remote work here to stay, organisations will continue to look for talent across geographies and from less traditional technology backgrounds," he says.

"As a result, we'll see companies invest in robust training and development programs to better train employees who may not have the deepest technical expertise but can work across a diverse range of roles."

Organisations Will Look to Consolidate Data and Simplify Architecture

With data security in the spotlight, Hackney says more Australian organisations are rethinking how they currently manage and store their data. 

"A problem we are still seeing is that organisations today have to manage too many systems. Too many disjointed data pools lead to duplicated, siloed, and locked-up data, which is not only timely and costly to manage and analyse but more critically, leads to security issues. If organisations are to grow profitably, they must invest in the right technology to support that growth from the outset," he says.

"We anticipate more organisations will cut back on vendor spending to simplify their data architecture where they can combine data science and data analytics and draw from a single source of truth. 

"Adopting a data architecture like the Lakehouse which combines hindsight and predictive analytics can help organisations achieve more with less. The medium to long-term cost saving opportunities significantly outweigh traditional approaches, incentivising organisations to accelerate their modernisation projects to realise these savings more quickly."

AI will help customers manage rising costs in an uncertain economic environment

Many enterprises have rushed into the cloud, and while it has led to productivity gains, it has also created ballooning data management costs, Hackney says. 

"In an increasingly uncertain macro environment, we see AI as playing more of a role in reducing costs by understanding customers' workloads over time and automatically right-sizing their deployments for optimal cost and productivity trade-offs," he says.

"Cost will become a critical focus over the next year as organisations turn to AI and machine learning to streamline their operations and manage the areas they can control in an increasingly volatile economic climate."