Exclusive: How Teradata is supporting the Australian Government

In an enlightening conversation with Simon Williams, the Architecture Leader for the Australian Federal Government at Teradata, the future of data and digital transformation within government sectors was discussed in depth. Williams, with decades of experience at the intersection of technology and public policy, delved into the evolving landscape of government data management and the pivotal role of emerging technologies such as artificial intelligence and cloud computing.

Williams began by tracing the history and evolution of Teradata, a company at the forefront of big data since the late 1970s. He remarked, "For over 40 years Teradata has been the platform of choice for large organisations that need to manage their data at-scale." He noted that Teradata's early customers included large retail companies in the US that needed the power of a massively parallel processing architecture to support their data warehousing requirements. Since then Teradata has become a global leader in enterprise data solutions, operating in regions including the US, Europe, the Middle East, and Asia-Pacific.

Expanding on Teradata's global operations, Williams shared that the company, headquartered in San Diego, has a significant presence worldwide, including offices in Sydney, Singapore, London and Tokyo. He elaborated on the employee structure, stating, "Globally, we have around 7,000 employees. Everything from back-end development labs and technical support to front-line sales, consulting, architecture and professional services.

Discussing Teradata's Innovations, Williams introduced VantageCloud Lake, the latest in a line of products that mark the transition from on-premise to cloud-based capability. "VantageCloud Lake is our implementation of the Data Lake and Data Lake House paradigms, deploying technologies such as OpenTable format, with Apache Iceberg and Delta Lake, and the separation of Compute and Storage".  

"We have completed IRAP on AWS for our VantageCloud Enterprise platform, with more work being done with other Hyperscalars. These are just some of many examples where Teradata is investing in development to drive innovation for its customers," Williams explained.

Reflecting on the accelerated shift to hybrid and multi-cloud systems during the COVID-19 pandemic, Williams commented, "We had already been developing our cloud portfolio prior to the pandemic, but there has been an acceleration of Cloud adoption for many of our customers post-COVID." This proactive strategy has positioned Teradata well to navigate the increasing demand for flexible, scalable cloud solutions.

Williams expressed particular enthusiasm about the role of artificial intelligence in enhancing government data management. He detailed how Teradata's platform has evolved, stating, "Our platform has evolved from being an Enterprise Data Warehouse to now including an integrated analytics and AI/ML capability." He added, "We have a technology called ClearScape Analytics, which provides well over 150 AI/ML functions embedded within the environment."

Discussing AI's role further, Williams highlighted a feature that simplifies the interaction with data: "Another new Teradata technology that is being rolled out right now is called ask.ai, which in essence is a natural language processing engine that allows a user to ask a question about their data in plain language which then gets translated into an SQL query. This will allow non-IT users to better access data and will be a game changer for democratising data within government.

Turning to the role of data in the Australian public sector, Williams emphasized its critical nature, "Data is the lifeblood of government. It's critical... the volume, variety and velocity of data is increasing all of the time." He discussed government's need to leverage data more effectively to improve public services, "Over the years Government has increasingly used data to provide better outcomes; however data is not being used as effectively as it could be. Government needs to re-assess the value of its data holdings to materially improve outcomes based on the enhanced use of data".

On the topic of Data and Digital Government, Williams reflected on Australia's advancements but acknowledged the need for ongoing improvement. "A recent OECD study – the 'Digital Government Index' ranked Australia as number five globally in terms of digital capability across government. We've come a long way in terms of our maturation, but there's still a long way to go – in fact, it's a journey that, by definition doesn't end," he explained.

Looking towards the future, Williams expressed excitement about the potential for generative AI in government settings. He speculated on the integration of foundational, large language models with existing capability to create new outcomes, saying, "At Teradata we're looking at ways of leveraging large language models with our analytics and SQL engine capabilities to produce new outcomes that weren't easy to achieve before."

Williams concluded the interview with optimism about the continuous evolution of technology in government. "It's always on the move, and Teradata is at the forefront of that. These are exciting times," he stated, underscoring the dynamic nature of the industry and the continuous opportunities for innovation and improvement in government data management. This comprehensive perspective from a leader in technology architecture highlights the pivotal role of data and AI in shaping the future of public sector policy and service delivery.