Story image

A look at how DataOps can drive significant business value

05 Apr 2018

Article written by MapR VP of technology strategy Dr Crystal Valentine 

In 2018, big data analytics will become a core competency within Australian enterprises rather than merely a nice-to-have. In today’s data-driven world, organisations are already being tasked with collecting and curating data on a massive scale to increase revenue and gain competitive advantage, as well as improve operational efficiency. Without this focus on data, businesses will fall behind.

According to Telyste, around a third of Australian CIOs are already are or looking to use big data analytics for sales and marketing applications. Moreover, IDC estimates that by 2020, data-as-a-service will generate 80% of revenue for large businesses in Australia.

As the importance of big data within the organisation increases, the role of data science and machine learning is also rapidly growing. This is because data science and machine learning techniques are playing an increasingly vital role in driving value for big data projects. For example, machine learning can be used to help identify credit card and insurance fraud, to determine the health of factory machinery and the likelihood of machine failure, as well as helping native advertising become more intelligent.

This intelligence will infiltrate all areas of business. According to IDC, by 2019, 85% of enterprise apps will use artificial intelligence (AI), 65% of consumers will interact with AI-driven chatbots, and half of all digital transformation initiatives will leverage AI.

What is DataOps and how does it drive value?

DataOps is an agile methodology for developing and deploying data-intensive applications, including data science and machine learning. A DataOps workflow supports cross-functional collaboration, with an emphasis on both people and processes, as well as the empowering platform technologies that underpin it. While DataOps focuses largely on people and processes, it also requires an enterprise-grade platform to enable collaboration and the sharing of data.

Rather than having teams work from siloed technology and data, a collaborative effort requires the use of a single, unified data platform. Using a single platform is key to agility, as it reduces the need to copy or move large data sets. With a unified data platform, organisational data access and privacy policies can be enforced holistically across organisations.

A DataOps methodology offers many benefits to data-driven organisations and specifically has a positive effect on business agility. With DataOps, data scientists can iterate rapidly to improve deployment models. With DataOps, data scientists also won’t get bogged down in doing the “plumbing” work of finding, copying, curating, and transforming data.

The agile approach to big data and data strategy

Despite businesses making increasing investments in data science and machine learning, many of them still grapple with how to deliver value due to the logistical challenges of managing big data. A key challenge is that extensive time and resources are required to work on data engineering and to create the “glue” code to put together data pipelines. As a result, many businesses are still struggling to adopt an agile approach to leveraging big data. While there are many factors contributing to this, one of the most critical is a lack of a unified, secure platform for data science and analytics.

An agile approach to big data is collaborative and creative, with changes and feedback an integral part of a project’s evolution. A successful agile data strategy combines all the elements of big data analysis and makes the data visible, shareable, reproducible and standardised.

Traditionally, raw data has been viewed as something to be extracted, transformed and loaded into warehouses before it can be analysed. Today leading organisations are instead looking to leverage enterprise-grade big data platforms for a cost-effective, flexible and fast way to access and secure multiple data types. When combined onto a platform, all of these elements can help create a more agile, creative, and collaborative environment.

Big data will not bring value to an organisation until it is leveraged to impact either top-line growth or profitability through increased operational efficiency. While DataOps is still emerging as an enterprise practice to organise the work of small teams, it represents a significant new trend that will serve to increase collaboration, agility and operational efficiency. The recognition that data-intensive applications have their own set of considerations and challenges when it comes to managing and securing large, complex data sets, - will mean a major a paradigm shift for Australian enterprises.

Microsoft urges organisations to tackle data blindspots
Despite significant focus placed on CX transformation, over a third of Australian organisations claimed that more than one in five of their projects failed.
Raising the stakes: McAfee’s predictions for cybersecurity
Security teams and solutions will have to contend with synergistic threats, increasingly backed by artificial intelligence to avoid detection.
Renesas develops 28nm MCU with virtualisation-assisted functions
The MCU features four 600 megahertz CPUs with a lock-step mechanism and a large 16 MB flash memory capacity.
DOCOMO ranked world's top mobile operator in 5G SEP applications
NTT DOCOMO has been ranked the world's leading mobile operator in terms of applications for candidate standard-essential patents.
Exclusive: Ping Identity on security risk mitigation
“Effective security controls are measured and defined by the direct mitigation of inherent and residual risk.”
CylancePROTECT now available on AWS Marketplace
Customers now have access to CylancePROTECT for AI-driven protection across all Windows, Mac, and Linux (including Amazon Linux) instances.
Gartner’s top 10 data and analytics trends for 2019
Data is the fuel for the modern world, and analytics the engine. Gartner has compiled the top 10 trends to watch this year.
How CIOs can work with colleagues to drive new competitive advantages
"If recent history has taught us anything, it’s that the role of the CIO is always changing, and that it won’t stop changing anytime soon."