Story image

Top Coles exec talks about the strength of big data

13 Sep 2016

The head of big data at Coles is an Irishman named Niall Keating. He's worked in the area of big data for four years, having previously been at Telstra.

If you aren't an Australian, Coles is a dominant supermarket chain in Australia and Telstra the largest telco in the land. Clearly both large enterprises work with a lot of data.

He's also a major advocate of the open source Hadoop eco-system for handling big data. This eco-system is developed by the open source focused Apache Foundation. He was also the first person in Australia to reach out to HortonWorks, a key tool in this open source big data world.

His job is to modernise and simplify the retailer's information systems, in particular, it's big data technologies.

"Coles has spent a lot of time with proof of concepts and trying out different technologies. And now in the last couple of years, we've really seen what we can do with that [big data] eco-system. It's now being deployed in earnest to solve business problems," Keating says.

Scott Gnau, Hortonworks Chief Technology Officer at HortonWorks, believes that big data technologies have become a more established part of the IT landscape. No longer are they just pipe dreams.

"Coles' experience really echoes or mirrors what we've seen in other regions. Starting with tests and pilots but now delivering more business value and it being worked into production," Gnau says.

Big data adoption in Australia

Keating also sees that Hadoop and big data technology adoption has been a little slow in Australia.

The challenges that he sees are culture and skills. Culturally, that IT departments in Australia are risk averse. He believes that they need to get that balance right of adopting new technologies versus not making mistakes and managing costs. "Every enterprise should have an innovation agenda," says Keating.

While he sees universities producing data scientists, there is a skills shortage in software engineers holding back big data adoption in Australia, the kind of people that can build scalable and distributed back end systems.

The future

Keating sees a number of start-ups being created around big data, either as technology providers or just at their core using big data to develop new products and services. Already corporates like Telstra and Westpac have invested in these type of ventures.

Read about an innovative example; Data Republic here.

"Data will be at the core of products going forward and drive customer experience. You can see this with Uber and AirBnb already," Keating says.

He also sees that technologies like IOT and Cloud are linked to the useful rise of big data in the enterprise.

"The next challenge is real time. Real time is quite easy, but real time at scale can be quite hard," he continues.

Gnau agrees that real-time is going to be the way of the future.

"It's important for the business to adjust and be able to react very quickly. Customers and even businesses will expect this real-time reaction in the future," Gnau says.

On Hadoop and Hortonworks​

Keating likes the Hadoop eco-system because it's open source foundation creates competition between engineers globally. Which in turn pushes the technology ecosystem forward.

"This creates opportunities but also that flexibility, innovation and constant change creates challenges for enterprises," he continues.

"That's where we need organisations like HortonWorks to put that wrapper around the open source innovation, to give us a solid platform to build our enterprise applications on top of," he explains.

HortonWorks sees their role as adding dependability, scalability, reliability to the Hadoop open source eco-system. Which in turn enables ease and acceleration of adoption of the tools.

Read more about HortonWork's latest release here.

GitHub launches fund to sponsor open source developers
In addition to GitHub Sponsors, GitHub is launching the GitHub Sponsors, GitHub will match all contributions up to $5,000 during a developer’s first year in GitHub Sponsors.
Check Point announces integration with Microsoft Azure
The integration of Check Point’s advanced policy enforcement capabilities with Microsoft AIP’s file classification and protection features enables enterprises to keep their business data and IP secure, irrespective of how it is shared. 
Why AI will be procurement’s greatest ally
"AI can help identify emerging suppliers, technologies and products in specific categories."
Are AI assistants teaching girls to be servants?
Have you ever interacted with a virtual assistant that has a female-based voice or look, and wondered whether there are implicitly harmful gender biases built into its code?
Google 'will do better' after G Suite passwords exposed since 2005
Fourteen years is a long time for sensitive information like usernames and passwords to be sitting ducks, unencrypted and at risk of theft and corruption.
Hackbusters! Reviewing 90 days of cybersecurity incident response cases
While there are occasionally very advanced new threats, these are massively outnumbered by common-or-garden email fraud, ransomware attacks and well-worn old exploits.
Data#3 to exclusively provide MS licences to WA Government
The technology services provider has won two contracts with the Western Australia Government, becoming its sole Microsoft licence provider.
Why cash is no longer king in Australia
Australia is leading the way in APAC for granting credit on B2B transactions.