IT Brief Australia logo
Technology news for Australia's largest enterprises
Story image

Q&A: Cloudera on building the perfect hybrid cloud architecture

By Sara Barker
Mon 13 Sep 2021

Cloudera's APJ cloud director Stevie Walsh chats to us about hybrid cloud and the democratisation of data.

One of the most common pieces of advice to organisations is that an entirely on-prem IT infrastructure is seldom workable for most organisations’ IT needs, so the hybrid infrastructure is the best of both worlds. Is there a trick (or perhaps several tricks) to making the most of all these environments?

Many organisations take a well-intentioned leap into a hybrid cloud environment without taking the entire digital transformation journey into account - essentially putting the cart before the horse. 

They’re on the cloud, but their data could be spread across multiple clouds. There are also situations where they must split their data between the cloud and on-premise, which makes it even more challenging to access, manage and secure.

All clouds have their benefits, however, there are common challenges that need to be addressed to make the most of today’s modern cloud environments. These include:

  • Greater amounts of data coming from multiple sources 
  • Data capture at the Edge, where it must be ingested and analysed in real-time
  • Multiple systems that can be problematic to integrate
  • Concerns related to data security and accessibility of data within an organisation
  • Multiple public and provide clouds and data warehouses within a single enterprise customer

It’s also important to understand which workloads are best suited for each deployment – the public cloud is ideal for intermittent, seasonal, ‘batchy’ or unknown (newly developed workloads) whereas a private cloud is best for highly utilised workloads. Hybrid cloud environments can help organisations support increasingly remote workforces, improving scalability and control while maintaining business continuity.

Having a single platform for both public and private provides organisations with the possibility to choose the right destination for workloads or launch, develop or sandbox new workloads in the cloud and then move them on-prem.

In many cases, data needs to be used by both analysts and less technical users, as well as machine learning or automated systems. Does data need to be ‘re-engineered to cater for all these different users and applications?

 A decentralised structure allows for greater productivity benefits from data and analytics efforts. Customers use data to accelerate innovation efforts by combining existing technologies in new ways, or “upcycling” existing technologies by applying them to a different domain to solve a different problem.

Data democratisation is the idea of empowering every employee (not just the data scientists) with the training and tools to be data-savvy. Compared to the IT team, business user groups like HR, marketing or product development have a more in-depth understanding of the data collection context and will be able to come up with domain-specific actionable insights. 

For example, the marketing team in a financial services organisation may need data modelling to develop a recommendation engine for lead capture. Within the same company, the customer support team might be on using data modelling to understand and reduce customer churn. 

The roadblock, however, is often legacy data systems that don’t have the high-speed processing power to support the instantaneous delivery of ensuring the right data reaches the right people at the right time. 

Applying Machine learning (ML) is another area of significant untapped potential – businesses that have adopted a successful ML strategy are outperforming their competitors by over 9%.

To help make it faster and simpler for businesses to unlock greater ML capabilities, we recently released Applied ML Prototypes (AMPs). Everyone is trying to do more with their data and AMPs enable data teams to go from idea to a fully working ML use case in a fraction of the time.

Enterprise data clouds help organisations use analytics at every stage of the data lifecycle, working at speed to extract the true value from their data. They also enable users to control who has access to data in a secure manner that maintains data integrity. 

Speaking broadly about the democratisation of data, what does this mean in a general enterprise/business context? 

With data playing an increasingly strategic role across all industries, data democratisation is something that should no longer be considered optional. Providing employees with access to data ensures they can gain better insights to make informed decisions quickly. Benefits can come in the form of financial or operational improvements such as increased operational efficiencies, new revenue streams, increasing average spend per customer, resilience and cost savings. To get started on data democracy, businesses need to look at structure, tools and training. 

One of the barriers to data democracy is, ironically, organisational structure. It is common for businesses to have centralised data analyst teams. It is also common for data to be kept in disparate silos, and data teams need time to extract and consolidate the relevant data. Such structures can often delay the decision-making process. Removing these data silos ensure more people can access the data, empowering them to make smarter, data-driven decisions.

As businesses push data and analytics out to more people within their organisation, it’s important they manage risk around sensitive information and also consider how to remain compliant with governmental regulations related to data privacy and locality. There are various strategies to achieve these goals, but the key is simplifying the security and governance end to end.

Being able to deliver consistent security and governance across all clouds - public or private rather than multiple security solutions stitched together reduces operational complexity, risk, and ongoing costs. 

How can data democratisation be applied to different industry sectors?

Locally, within A/NZ, telco providers are leveraging customer experience analytics to build a 360-degree view of their customer journey across all channels and lines of business. From this, they’re building prediction models to identify a customer base for personalised offers, even during this uncertain time.

When it comes to the in-person retail experience, technology and widespread access to data is proving a game-changer. Businesses can now combine the power of connected technology with a physical presence, placing beacons and sensors in stores to track purchase paths of customers by identifying ‘hot’ and ‘cold’ spots throughout the store. In addition, they can utilise facial recognition technologies to measure customer moods and sentiments.

On the supply chain front, retailers can use IoT devices to track goods, for instance, embedding devices within refrigerated trucks carrying perishable foods to measure temperature and humidity.

There are times when organisations need timely and accurate data to make quick decisions, such as in emergency response situations. How does data democratisation fit into these situations and these decision-making processes?

In an emergency response situation, the need for speed is mission-critical, to make sense of data accurately and quickly. Organisations need to be able to ingest and process data in real-time. Decisions can then be made faster and with confidence, often as a situation is unfolding.

For example, to mitigate the impacts of power outages, utility companies are now using advanced data analytics and machine learning to monitor and manage capacity, build predictive capacity models, identify bottlenecks, and prioritise and plan network expansion decisions. They’re also using advanced analytics to predict network capacity, mitigate business disruption and effectively ‘keep the lights on’. 

Another of our Australian based customers is in the financial services industry. For them, having a single pane of glass view across all their data is critical to support their fraud detection programme, mitigating risk for themselves and their customers. Existing in-house tools were inadequate in managing their data workloads considering the increasing scale of data clusters and demand. The company harnessed our platform to process large volumes of data in real-time and deliver analytical insights on transactions for reporting. This in turn enabled the customer to detect issues and irregularities, proactively avoiding fraudulent transactions.

Is there anything else you would like to add?

Data is a strategic asset that warrants its own strategy in the form of an enterprise data strategy. It’s very easy for organisations to lose sight of their initial motivation of migrating to the cloud. What I hear all too often is, “we’re going cloud – why are we doing this again?”. Sticking to strategy, or being able to steer the ship as needed, is critical especially when it comes to scaling data and ensuring strong governance and security. 

Equally important is separating data strategy from cloud and infrastructure strategy to give customers choice and flexibility. Delivering all these elements relies on the right set of tools to start with, consideration of end-to-end data lifecycle and building complex data pipelines with a single platform.

Related stories
Top stories
Story image
CISOs need to consider a risk-based cybersecurity strategy
Rather than talking in terms of attack vectors and vulnerabilities, CISOs and security decision-makers must look at actual business risk.
Story image
CM Group recognised as Best Overall MarTech Company
MarTech Breakthrough has ranked CM Group as the Best Overall MarTech Company for the third year running and Cheetah Digital as Best Overall Marketing Campaign Management Solution.
Story image
Unified Communications
Gold Coast private hospital improves comms with Alcatel-Lucent Enterprise solutions
With further demands placed on health workers as a result of the pandemic, they need efficient and adaptable solutions that help them effectively serve their communities.
Story image
Workday research finds A/NZ organisations becoming leaders in digital agility
New research from Workday and IDC has revealed that Australia and New Zealand are becoming leaders in digital agility.
Story image
University of Melbourne
The University of Melbourne implements data ecosystem with Talend
Talend has announced that The University of Melbourne (UoM) has selected Talend Data Fabric for its cloud data platform.
Story image
Contact Centre
Treasure Data launches new customer experience functionality
Treasure Data has introduced a new strategic vision and product positioning for Customer Data Cloud, reflecting the company's focus.
Story image
Video: 10 Minute IT Jams - An update from Mendix
Mendix is a low-code platform used by businesses to develop mobile and web apps at scale, and Jornt joins us today to discuss how these offerings work, and what benefit they have in the development process.
Story image
Lucid Software
Lucid Software expands enterprise offerings with enhanced slack apps
Lucid Software has expanded its enterprise offerings with enhanced slack apps for its Lucidspark and Lucidchart technology.
Story image
Australian IT security concerns higher than before pandemic
Australian organisations are more concerned about cyberattacks than they were prior to the COVID-19 pandemic, according to a new survey.
Story image
Digital Transformation
Federated change is the best path to digital evolution
Businesses that can successfully manage the exponentially expanding masses of data produced by modern consumers will be the businesses that survive and prosper.
Story image
Artificial Intelligence
Gartner unveils key emerging tech to watch in 2022
"Such technologies present greater risks for deployment, but potentially greater benefits for early adopters," says Gartner.
Discover why a localised cloud ERP is the way to go for Australian businesses.
Link image
Story image
Cyber attacks
Dramatic uptick in threat activity with exploits growing nearly 150%
"While it’s not a surprise given increased attack opportunities like remote work, it’s still a worrying development and one we cannot ignore."
Story image
Investment in APAC cold storage to reach $5 in next decade
Investment in Asia Pacific’s cold storage market is expected to grow fivefold in the next decade, according to JLL.
Story image
Lacework launches new capabilities for better threat detection
Lacework has announced new capabilities that enable organisations to uncover more critical threats to their infrastructure and empower teams.
Story image
Data analytics
Pressure on orgs to up their data analytics game - study
A recent report from Sisense highlights data transmission, analysis, and risk management remain top concerns for data professionals in APAC.
Story image
Digital Transformation
Why it is imperative to keep up with technology advancements
Technology is constantly evolving, which is why digital transformations can often be complicated. What legacy items stay? Will the new and old work together? Or is there a whole new system that we need to introduce?
Story image
Artificial Intelligence
Is your chatbot bringing down the customer satisfaction score?
The top 10 reasons why chatbots are failing to meet customer expectations and what you must do to avoid that.
Story image
Data protection: How SMEs can secure online consumers’ trust
Consumers believe how a company treats their data reflects how they treat their customers, according to a new survey.
Story image
Hybrid Cloud
The essential guide to digital transformation by SolarWinds
Digital transformation is a buzzword thrown around all the time by companies, but what does it actually mean and why is it important? SolarWinds breaks it down.
Story image
Dynatrace extends application security capabilities for runtime environments
Dynatrace has announced that it has extended its Application Security Module to detect and protect against vulnerabilities in runtime environments.
Story image
Sophos reveals latest ransomware trend impacting orgs
Sophos has announced in a new whitepaper that Hive, LockBit and BlackCat, three prominent ransomware gangs, consecutively attacked the same network.
Story image
Australian Defence Force
Cloudera and ADF partner for e-health management system
The Australian Defence Force (ADF) has selected Cloudera as a strategic data partner to help transform its e-health management system.
Story image
Kaspersky uncovers new attacks by advanced persistent threat group
The attacks involved modifications of the well-known malware, DTrack, as well as the use of a brand-new Maui ransomware.
Story image
Artificial Intelligence
Concentric AI protects sensitive data with new capabilities
The new capability reveals sensitive data shared across email and business messaging platforms and highlights who has inappropriate access to content.
AWS Marketplace
Learn how security orchestration, automation, and response (SOAR) enhances your security strategy.
Link image
Story image
Avast reveals zero-day exploits targeting Chrome and Microsoft
Avast, released its Q2/2022 Threat Report today, revealing a significant increase in global ransomware attacks, up 24% from Q1/2022.
Story image
Artificial Intelligence
Appier achieves historically high growth rate of 56% YoY
"Our strong momentum over the past two quarters underscores Appier's significant growth alongside our customers."
Story image
Data breach
Weak breach data disclosure laws for IP theft leaves vital Australian industries vulnerable
Infoblox has seen unprecedented levels of demand for cyber protection from companies outside of the Critical Infrastructure Act who have discovered security events and those who fear falling victim to IP theft.
Story image
Can biometrics help? 123% increase in Gen Zs scammed online
In the three years leading up to 2022, the number of Gen Zs who fell victim to online scams rose by 123%, according to Ping Identity.
Story image
Gartner Magic Quadrant
Gartner names Lookout a Visionary in 2022 Magic Quadrant
Gartner has recognised Lookout as a Visionary in the 2022 Magic Quadrant for Security Service Edge (SSE) and one of the top three offerings in the 2022 Gartner Critical Capabilities for SSE report.
Story image
Digital Transformation
Top tips for making your finance transformation program a resounding success
Planning to make 2023 the year you embark on a wholesale finance transformation program? It’s a move that will stand your enterprise in excellent stead as you navigate the complexities of the post-Covid business landscape.
Story image
Cloud Security
Tenable makes additions to Cloud Security portfolio
Tenable has announced additions to Tenable Cloud Security that represent the next step in assessing threats related to cloud vulnerabilities.
Story image
Tech job moves
Tech job moves - Fastly, INX, Kinly, SmartBear & Vectra AI
We round up all job appointments from July 29 - August 12, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Why security needs to shape your journey to the cloud
It's estimated that 80% of workloads could be in the cloud in the next few years. How can you make all that data secure?
Story image
Why enhancing bot protection for web and API endpoints matters
The trouble with bots is that they aren’t all bad. Unfortunately, this can make it challenging to detect malicious bots that find their way into your system and threaten your business.
Story image
Data Protection
Advancing genomic sequencing and public health with digital infrastructures
Right before our eyes, we've witnessed the development of the COVID-19 vaccine in record time. An enormous achievement in an otherwise lengthy task that previously took, on average, 10-15 years.
Story image
Education sector seeing highest volumes of cyber attacks
When breaking down the numbers to education attacks by region in July 2022, A/NZ was the most heavily attacked.
Story image
Enterprise Resource Planning / ERP
Why the right ERP (and partner) is crucial to an innovative and successful business
Enterprise Resource Planning (ERP) is a foundational step to ensuring a robust business model; here's why choosing the right one could be vital to ensuring long-term success and innovative results.
Story image
Customer experience
Scope Australia selects NICE CXone platform to streamline operations
One of Australia’s largest providers of disability support services, has chosen the NICE CXone platform to help streamline its contact centre operations.
Story image
Augmented Reality
TeamViewer remote access software integrated into RealWear Cloud
TeamViewer has announced a major expansion of its partnership with RealWear, a leading provider of assisted reality wearable solutions for frontline industrial workers.