IT Brief Australia logo
Technology news for Australia's largest enterprises
Story image

Three steps to creating a data-driven enterprise

By Contributor
Tue 1 Feb 2022

By New Relic APJ chief technology officer, Michael Fleshman.

One of the biggest challenges facing organisations today is how to truly harness the power of data - to determine what and how to measure, understand what the data actually "means," and prioritise what data to look at or act on to achieve maximum impact for the business.

A large part of the effort associated with digital transformation revolves around bringing data together to create a clear, top-level view of all data being ingested and ensure that these resources sit at the core of how a business operates.

At the highest level, there are three principles that underpin successful digital transformations through the creation of a data-driven enterprise.

Adopt a cross-functional, data-centric approach

By definition, the entire organisation needs to participate in a coordinated and coherent process to become a data-driven organisation. That means linking data from customer experience, business teams, finance, product, engineering and operations together in a way that allows alignment and optimisation to take place.  

Creating that framework is a material competitive advantage for companies undergoing digital transformation. According to the McKinsey & Company study on "Ingredients for a successful Digital Transformation", the top success factor (that separated winners from losers by almost a 3:1 ratio) was "Implementation of digital tools to facilitate analysis of complex information". The next two factors were: 

  • Adapted business processes to enable rapid prototyping and testing with customers
  • Created networks of cross-functional teams with end-to-end accountability

Businesses face several key issues when it comes to the implementation of their vision. The first is the sheer volume of possible data to be monitored.

The second is the selection of tools needed to manipulate the data across different time-horizons (real-time, in the next few days, this month, this quarter, this year, year-on-year) and for different stakeholders (business, product, marketing, engineering, eCommerce, etc.) and the economics associated with acquiring and running the data tools themselves. Making this even more challenging is the shortage of available data science/engineering professionals.

As a result, fragmentation of information across disparate, incompatible tools is an issue for most companies out there. Ironically, quantifying the impact of this fragmentation is difficult to impossible, given there are very few organisations (if any) that have extensively rolled out observability across the entire organisation to allow them to quantify the impact.

Enable collaboration across the enterprise

Collaboration is not just about SREs, operations personnel or developers working together. It's about how these functions collaborate with and support product management, growth teams and finance teams to work as a cohesive unit. This cohesivity enables scalability and the ability to manage more complex environments.

Many organisations don't have a clear view of their data. A common reason is that they don't have a central/integrated data management platform. It is also common to find several similar data stores existing in different parts of the company and no systems that provide a holistic view across these sources.

Another problem arises when organisations make investments in data tools but cannot integrate or normalise data across these tools sufficiently, which eventuates in creating multiple versions of "the truth" and the requirement to constantly try to determine which data to follow. Timeliness is also a major issue: if it takes two weeks to get the data needed today, then, of course, it won't be very helpful. 

In all these cases, it's difficult to make data-substantiated decisions if an organisation doesn't have the data or isn't confident in it. In addition to being suboptimal from a business efficiency standpoint, the less visibility one has, the more likely this will translate into performance and customer experience problems.

While achieving a "single pane of glass" view is more of an optimistic goal, what we're looking for is the equivalent of a diamond: where all of the information is holistically related, and different stakeholders can view it through their "facet" (e.g. their function or related KPI, OKR or dashboard).

The key to achieving all of the above is to have a strong culture that first and foremost understands the value of a data-centric organisation - which includes observability - but is extended to the organisation as a whole. These cultures will invest in data and observability and will constantly be looking to evolve to that "multi-faceted diamond". Their proportional investments in data (and observability) reflect this culture, and not surprisingly, they are better able than their peers to manage their business.

Implement AI, ML to handle scaled complexity

From an observability standpoint, the main job of Artificial Intelligence (AI) and Machine Learning (ML) is to help us humans deal with the exponential explosion of data. The sheer volume of data available for analysis, along with the number of possible relationships between the different data sources and points, makes it impossible for humans to sift through and react to the data in real-time.

AI and ML are good at doing several things with this sea of data: pattern recognition, correlation and predictability (in a probabilistic sense) - and can achieve this analysis at speeds and volumes several orders of magnitude greater than humans can.

Through correlation, AI and ML can quickly detect relationships that would not be easily detectable otherwise and bring these relationships to the attention of either humans or other systems. This includes detecting patterns that commonly occur before an issue takes place, allowing the team to investigate and determine if the anomaly is benign or an early sign of an impending issue that should be addressed before it impacts the customer or the business.

One well-known example of these ML capabilities is Google's AI arm DeepMind. Despite the tech giant's data centers already being highly optimised by some of the best engineers on the planet, DeepMind algorithms identified additional optimisations that reduced cooling costs by a further 40 per cent.

Even more savings (or reduction in additional costs) will be achieved from a resource perspective due to these algorithms minimising the need to materially scale teams as data volume scales. However, the ultimate value is customer experience and satisfaction as AI and ML reduce downtime and increase performance.

Although tech teams can now automate much more than they once could with these new AI and ML capabilities, there probably won't be a complete handover to automation any time soon. Instead, organisations should look at the current state of AI and ML in observability to keep up with the volume and complexity while minimising the need to scale the teams responsible for observability.

To get near that state of "total automation", a much broader percentage of an organisation's data will need to be incorporated in the observability framework. In addition, more sophisticated ways of helping the algorithms learn will need to be implemented.

By New Relic APJ chief technology officer, Michael Fleshman.

Related stories
Top stories
Story image
Employment
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Migration
Let’s clear the cloud visibility haze with app awareness
Increasingly, organisations are heading for the cloud, initiating new born-in-the-cloud architectures and migrating existing applications via ‘lift and shift’ or refactoring.
Story image
Remote Working
Successful digital transformation in the hybrid work era is about embracing shifting goalposts
As organisations embraced remote working, many discovered they lacked the infrastructure needed to support history’s first global load test of remote work capabilities.
Story image
Vectra AI
Understanding the weight on security leader’s shoulders, and how to shift it
Millions of dollars of government funding and internal budgets are being funnelled into cybersecurity to build resilience against sophisticated threats, indicating how serious this issue has become.
Story image
Apricorn
Data backup plans inadequate, data still at risk - study
The Apricorn 2022 Global IT Security Survey revealed that while the majority organisations have data backup plans in place, data for many are at risk.
Story image
Broadband
Telstra enters into new RSP agreement with Opticomm
Telstra has entered into an RSP agreement with Opticomm (A Uniti Group Limited subsidiary) to provide network fibre services to customers.
Story image
Digital Transformation
The Huawei APAC conference kicks off with digital transformation
More than 1500 people from across APAC have gathered for the Huawei APAC Digital Innovation Congress to explore the future of digital innovation.
Story image
Artificial Intelligence
AI-based email security platform Abnormal Security valued at $4B
"A new breed of cybersecurity solutions that leverage AI is required to change the game and stop the rising threat of sophisticated and targeted email attacks."
Story image
Microsoft
New Relic enters multi-year partnership with Microsoft Azure
New Relic has announced a strategic partnership with Microsoft to help enterprises accelerate cloud migration and multi-cloud initiatives. 
Story image
Ransomware
Cybersecurity starts with education
In 2021, 80% of Australian organisations responding to the Sophos State of Ransomware study reported being hit by ransomware. 
Story image
Excel
Could your Excel practices be harming your business?
While Excel has been the de-facto standard for budgeting, planning, and forecasting, is it alone, enough to support organisations in the global marketplace that’s facing rapid changes due to digital transformation?
Story image
Remote Working
Australia’s remote workers face connectivity and security issues
SOTI's new report finds better video conferencing technology and improved security measures are top concerns for remote workers in Australia.
Story image
Digital Transformation
Trading up: It's time to swap core systems for flexible digital applications
This year will see more oranisations planning and commencing high tech renovations that will shake up the way they operate.
Story image
Workato
Workato unveils enhancements to enterprise automation platform
"The extra layer of protection with EKM, zero-logging, and hourly key rotation gives customers a lot more visibility and control over more sensitive data."
Story image
Nozomi Networks
Nozomi Networks, Siemens reveal software integration
Nozomi Networks and Siemens have extended their partnership by embedding Nozomi Networks’ software into the Siemens Scalance LPE local processing engine.
Story image
Lightspeed
Lightspeed launches all-in-one marketing platform in A/NZ
ECommerce provider, Lightspeed has launched a new all-in-one marketing solution, Lightspeed Marketing & Loyalty in Australia and New Zealand.
Story image
Qualys
Qualys updates Cloud Platform solution with rapid remediation
The new update is designed to enable organisations to fix asset misconfigurations, patch OS and third-party applications, and deploy custom software.
Story image
Sustainability
Power at the edge: the role of data centers in sustainability
The Singaporean moratorium on new data center projects was recently lifted, with one of the conditions being an increased focus on power efficiency and sustainability.
Story image
Manufacturing
HINDSITE wins Aerospace Xelerated Pitch Challenge with solution to support Boeing
Brisbane-based startup HINDSITE was the winner of the first ever Pitch Challenge organised by Aerospace Xelerated in partnership with Queensland XR Hub. 
Story image
Data solutions
South Australia state satellite makes significant progress
South Australia’s first state satellite has successfully completed the Critical Design Review (CDR), moving it closer to providing tangible data solutions.
Story image
trust
9/10 Aussies to stop spending if personal data compromised
"Based on the patterns we are seeing among Australian consumers, it is evident that trust in a brand is exceptionally important."
Exabeam
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
SaaS
Rubrik Security Cloud marks 'next frontier' in cybersecurity
"The next frontier in cybersecurity pairs the investments in infrastructure security with data security giving companies security from the point of data."
Story image
Fintech
Airwallex launches new bank feed integration with NetSuite
Airwallex has launched a new bank feed integration with NetSuite, developed in partnership with NetSuite solution partner, Onlineone.
Story image
Amazon Web Services / AWS
RedShield leverages AWS to scale cybersecurity services
"Working with AWS gives RedShield the ability to mitigate significant application layer DDoS attacks, helping leaders adopt best practices and security architectures."
Story image
Digital Transformation
How to modernise legacy apps without compromising security
At a time when digital transformation has become central to business, even the most important applications come with a ‘use-by’ date.
Darktrace
Threat actors are exploiting weaknesses in interconnected IT/OT ecosystems. Darktrace illuminates your entire business and takes targeted action to stop emerging attacks.
Link image
Story image
Data Center
Preventing downtime costs and damage with Distributed Infrastructure Management
Distributed Infrastructure Management (DIM) can often be a lifeline for many enterprises that work with highly critical ICT infrastructure and power sources.
Story image
Customer experience
Research unveils precarious customer loyalty for retailers
New research has found customers are reassessing established brand loyalties as their priorities and behaviours shift.
Story image
Infrastructure
Nutanix study reveals financial services sector lagging with multicloud adoption
Nutanix has released new research that reveals the financial services sector is lagging behind when it comes to multicloud adoption.
Story image
Cybersecurity
More than 40% of banks worried about cloud security - report
Publicis Sapient's new report finds security and the lack of cloud skills and internal understanding of business benefits are big obstacles for banks moving to the cloud.
Story image
Attain
Revenue operations is taking centre stage
As the business world continues to evolve, new demands need to be met to keep up with the ever-changing landscape. 
Story image
Sift
Sift shares crucial advice for preventing serious ATO breaches
Are you or your business struggling with Account Takeover Fraud (ATO)? One of the latest ebooks from Sift can provide readers with the tools and expertise to help launch them into the new era of account security.
Story image
Artificial Intelligence
CFOs using digital workers and AI to prevent unnecessary loss
New technology is now allowing CFOs to use digital workers to automate their accounting processes, making it easier for them to avoid unnecessary losses.
Story image
Application Security
What are the DDoS attack trend predictions for 2022?
Mitigation and recovery are vital to ensuring brand reputation remains solid in the face of a Distributed Denial of Service (DDoS) attack and that business growth and innovation can continue.
Story image
Digital Transformation
Pluralsight and Ingram Micro Cloud team up on cloud initiative
Pluralsight has teamed with Ingram Micro Cloud to build upon cloud competence and maturity internally, and externally support partners’ capabilities.
Story image
Cybersecurity
A10 Networks finds over 15 million DDoS weapons in 2021
A10 Networks notes that in the 2H 2021 reporting period, its security research team tracked more than 15.4 million Distributed Denial-of-Service (DDoS) weapons.
Story image
Artificial Intelligence
SAS announces new products amid cloud portfolio success
Analytics and AI company SAS is deepening its broad industry portfolio with offerings that support life sciences, energy, and martech.
Story image
Digital Transformation
The impact of COVID-19 on healthcare environments and care delivery
The COVID-19 pandemic has revolutionised the healthcare industry while overcoming staff shortages, social distancing requirements, and lockdowns.
Story image
CRM
Zendesk announces new conversational CRM solutions
“The last few years have made it obvious that digital is the front door, convenience is paramount and relationships are anchored in conversations."
Story image
Cloud Security
Aqua Security createa unified scanner for cloud native security
“By integrating more cloud native scanning targets into Trivy, such as Kubernetes, we are simplifying cloud native security."
Softiron
For every 10PB of storage run on HyperDrive vs. comparable alternatives, an estimated 6,656 tonnes of CO₂ are saved by reduced energy consumption alone over its lifespan. That’s the equivalent of taking nearly 1,500 cars off the road for a year.
Link image
Story image
Sustainability
Grasping the opportunity to rethink the metrics of a sustainable data centre
A data centre traditionally has two distinct operations teams: the Facility Operations team, and the IT Operations team. Collaboration between them is the key to defining, measuring, and delivering long-term efficiency and sustainability improvements.
Story image
Customer experience
Gartner recognises Okta for abilities in Access Management
Okta has announced it has been recognised as a Customers' Choice for the fourth time in a row in the Gartner Peer Insights "Voice of the Customer" report.