IT Brief Australia logo
Technology news for Australia's largest enterprises
Story image

Q&A: SnapLogic CTO makes the case for investment in machine learning

By Contributor
Thu 8 Oct 2020
FYI, this story is more than a year old

SnapLogic field chief technology officer Brad Drysdale discusses the roadblocks to a successful machine learning implementation and the ways they can overcome them.

What are the barriers for firms when it comes to machine learning (ML) adoption?

The ML process can seem daunting for many organisations due to its unpredictable and experimental nature. 

When IT teams initially start going through the data to deploy ML algorithms, most probably won’t know the type of data that is required. A lot of exploration must be done before IT decision-makers have an idea of what data will be useful, and which ML algorithms will work best to solve a particular problem. 

Other technical challenges include being able to automate data access. When organisations have formulated clear policies that allow easy access to real-time data, they need to consider how to set up a channel or a pipeline to access the data. 

Organisations also need to ensure that there is availability of constant real-time data. ML models should not be trained on a single fixed set of data, so organisations need to set them up so that they can retrain their models to adapt to the changing behaviour of the data and the systems that they’re working with. 

Additionally, there is a significant talent shortage. While the number of qualified data scientists is growing, there are only a small number of data scientists that we can produce each year. 

How can these barriers be overcome? 

There are concerns about access to certain types of data, particularly when you have different groups of employees or other stakeholders coming in at different times to work on projects. So organisations should consider filtering out any potentially sensitive information from the data first so that the rest can be used to deploy ML algorithms. 

Another issue to overcome is how to fulfil the demand for data scientists. While it’s great to see that more data scientists are emerging in the workplace, a lot of time still goes into training them, so supply is still not keeping up with the rising demand. 

However, more people who have been trained in other areas, such as senior business analysts and software engineers, are increasingly expanding their knowledge of data science and ML, which can help bridge that gap.

Additionally, organisations will have IT business analysts who have experience with handling databases, even if they’re not programmers, they’re still analytically minded, so they can take advantage of ML through self-training too. 

All of these developments are following a positive trend, as tools and platforms are beginning to allow a broader range of users to engage with ML and make it useful for them. 

What are the biggest misconceptions businesses have concerning ML?

I see two main misconceptions about ML that relate to its complexity and capabilities. First, businesses often think that ML is very complex and requires PhD’s to get value out of an implementation. 

Many relatively simple ML algorithms can be applied to business data to provide predictions or classifications.  On the other hand, there is an unrealistic conception that ML is a panacea to all business problems. The sweet spot is understanding the realistic capabilities of different, well-understood ML algorithms and match them with the right business data to derive real value. 

How can businesses address the data science skills gap?

Businesses of all sizes should be working with universities or creating apprenticeship programmes to bring in fresh talent. 

For example, the Computer Science department at the University of San Francisco offers a project course for both undergraduates and graduates for one term, where they typically do small project work with an industry sponsor. This not only allows students to work with a variety of different companies, but it also makes the recruitment process for businesses far easier. 

Another way to help bridge the skills gap is through investment in technology that will lift the burden off IT professionals. 

Low-code/no-code platforms are a prime example of this, as they can enable data tasks to be undertaken by people outside of the IT department working in the lines of business. 

Currently, having the right data to deploy ML algorithms is an incredibly time-consuming process. A lot of time is spent trying to get access to and sift through vast volumes of disorganised data with manual coding, leaving IT professionals little time to focus on higher-value tasks. 

By investing in the right low-code/no-code technology, businesses can easily automate data pipelines, giving all departments regular access to real-time data, and make ML processes as seamless as possible with little to no coding required. 

How will businesses benefit if they invest more in emerging technologies and skills training? 

Businesses can look at how investment in emerging technologies will benefit them in two ways – either to get ahead of their competition or to prevent their organisation from becoming obsolete.

Businesses need to follow and even move ahead of technology trends to not only offer a better experience and more effective utilisation of their resources but continue to provide the services that their users and customers expect. 

Eventually, all organisations will need to adopt ML simply because that will become an expectation, so that applications and services can better anticipate what their users are attempting to do and to provide recommendations or predictions that enable them to achieve their goals more rapidly. 

This doesn’t just apply to just investment in the technologies, but the skills training as well. Businesses need to ensure that people are trained well to utilise these technologies, but also continue to help expand their skill set to harness the full potential of these emerging technologies. 

Related stories
Top stories
Story image
Artificial Intelligence
Frost & Sullivan recognises Genesys as leader in new reports
Frost & Sullivan has recognised Genesys as a leader in the cloud contact centre market for its robust cloud and digital capabilities.
Story image
Manhattan Associates
Shortening the click-to-customer cycle through smart technologies
Speed of delivery without accuracy is a dealbreaker for consumers. How can retailers operating in an omnichannel environment overcome the challenge of click-to-customer cycle times.
Story image
Vectra AI
Vectra’s inaugural Partner of the Year Awards revealed
APAC companies Baidam, Firmus, ShellSoft and Macnica have been recognised in Vectra AI's inaugural Partner of the Year Awards.
Story image
Cybersecurity prompts upgrade for 1.3 billion electricity meters
ABI Research finds Advanced Metering Infrastructure (AMI) and cybersecurity concerns are prompting the upgrade of 1.3 billion electricity meters by 2027.
Story image
Tech job moves - Forcepoint, Malwarebytes, SolarWinds & VMware
We round up all job appointments from May 13-20, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Customer experience
Gartner recognises Okta for abilities in Access Management
Okta has announced it has been recognised as a Customers' Choice for the fourth time in a row in the Gartner Peer Insights "Voice of the Customer" report.
Story image
Nutanix study reveals financial services sector lagging with multicloud adoption
Nutanix has released new research that reveals the financial services sector is lagging behind when it comes to multicloud adoption.
Story image
Lightspeed launches all-in-one marketing platform in A/NZ
ECommerce provider, Lightspeed has launched a new all-in-one marketing solution, Lightspeed Marketing & Loyalty in Australia and New Zealand.
Story image
Let’s clear the cloud visibility haze with app awareness
Increasingly, organisations are heading for the cloud, initiating new born-in-the-cloud architectures and migrating existing applications via ‘lift and shift’ or refactoring.
Story image
Revenue operations is taking centre stage
As the business world continues to evolve, new demands need to be met to keep up with the ever-changing landscape. 
Story image
A10 Networks finds over 15 million DDoS weapons in 2021
A10 Networks notes that in the 2H 2021 reporting period, its security research team tracked more than 15.4 million Distributed Denial-of-Service (DDoS) weapons.
Story image
Supply chain
Jetstack promotes better security with supply chain toolkit
The web-based resource is designed to help organisations evaluate and plan the crucial steps they need to establish effective software supply chain security.
Story image
Sift shares crucial advice for preventing serious ATO breaches
Are you or your business struggling with Account Takeover Fraud (ATO)? One of the latest ebooks from Sift can provide readers with the tools and expertise to help launch them into the new era of account security.
Story image
HINDSITE wins Aerospace Xelerated Pitch Challenge with solution to support Boeing
Brisbane-based startup HINDSITE was the winner of the first ever Pitch Challenge organised by Aerospace Xelerated in partnership with Queensland XR Hub. 
Story image
Nozomi Networks
Nozomi Networks, Siemens reveal software integration
Nozomi Networks and Siemens have extended their partnership by embedding Nozomi Networks’ software into the Siemens Scalance LPE local processing engine.
Story image
Vectra AI
Understanding the weight on security leader’s shoulders, and how to shift it
Millions of dollars of government funding and internal budgets are being funnelled into cybersecurity to build resilience against sophisticated threats, indicating how serious this issue has become.
Story image
Qualys updates Cloud Platform solution with rapid remediation
The new update is designed to enable organisations to fix asset misconfigurations, patch OS and third-party applications, and deploy custom software.
Story image
More than 40% of banks worried about cloud security - report
Publicis Sapient's new report finds security and the lack of cloud skills and internal understanding of business benefits are big obstacles for banks moving to the cloud.
Story image
Digital Transformation
The Huawei APAC conference kicks off with digital transformation
More than 1500 people from across APAC have gathered for the Huawei APAC Digital Innovation Congress to explore the future of digital innovation.
Story image
SPS network now available to CrescoData eCommerce customers
CrescoData, a Pitney Bowes Company and PaaS business in the commerce space, says its customers can now connect to the SPS Commerce Retail Network.
Story image
Rubrik Security Cloud marks 'next frontier' in cybersecurity
"The next frontier in cybersecurity pairs the investments in infrastructure security with data security giving companies security from the point of data."
Story image
Alarming surge in Conti Ransomware Group activity - report
A new report has identified a 7.6 per cent increase in the number of vulnerabilities tied to ransomware in Q1 2022.
Story image
Data Center
Preventing downtime costs and damage with Distributed Infrastructure Management
Distributed Infrastructure Management (DIM) can often be a lifeline for many enterprises that work with highly critical ICT infrastructure and power sources.
Story image
The most common online scams in Australia
No one is safe from online scammers, and many of these scammers have capitalised on the pandemic, using this confusing time to attack more people than ever.
Story image
New Relic enters multi-year partnership with Microsoft Azure
New Relic has announced a strategic partnership with Microsoft to help enterprises accelerate cloud migration and multi-cloud initiatives. 
Story image
Trojan cyber attacks hitting SMBs harder than ever - Kaspersky
In 2022 the number of Trojan-PSW detections increased by almost a quarter compared to the same period in 2021 to reach 4,003,323.
Story image
Digital Transformation
Pluralsight and Ingram Micro Cloud team up on cloud initiative
Pluralsight has teamed with Ingram Micro Cloud to build upon cloud competence and maturity internally, and externally support partners’ capabilities.
Story image
Cloud Security
Aqua Security createa unified scanner for cloud native security
“By integrating more cloud native scanning targets into Trivy, such as Kubernetes, we are simplifying cloud native security."
Story image
Telstra enters into new RSP agreement with Opticomm
Telstra has entered into an RSP agreement with Opticomm (A Uniti Group Limited subsidiary) to provide network fibre services to customers.
Story image
Fastly acquires Glitch, enables faster developer innovation
"This acquisition brings together two of the worlds best ecosystems for application development into a single, seamless developer experience."
Threat actors are exploiting weaknesses in interconnected IT/OT ecosystems. Darktrace illuminates your entire business and takes targeted action to stop emerging attacks.
Link image
Story image
New Relic
New Relic launches vulnerability management platform
New Relic has introduced New Relic Vulnerability Management to help organisations find and address security risks faster and with greater precision.
Story image
Digital Transformation
The impact of COVID-19 on healthcare environments and care delivery
The COVID-19 pandemic has revolutionised the healthcare industry while overcoming staff shortages, social distancing requirements, and lockdowns.
Story image
Infoblox's State of Security Report spotlights Australian remote work hazards
Attackers exploit weak WiFi, remote endpoints, and the cloud, costing 50% of organisations over $1.3 million in breach damages.
Story image
Data backup plans inadequate, data still at risk - study
The Apricorn 2022 Global IT Security Survey revealed that while the majority organisations have data backup plans in place, data for many are at risk.
Story image
Could your Excel practices be harming your business?
While Excel has been the de-facto standard for budgeting, planning, and forecasting, is it alone, enough to support organisations in the global marketplace that’s facing rapid changes due to digital transformation?
Story image
Equinix announces milestones on sustainability commitments
Equinix has released its 2021 Sustainability Report which outlines progress, innovation and accomplishments on key ESG commitments.
Story image
Silver Peak
The path to an adaptive, modern network
Managing and securing the network looks different than it did just two years ago—especially given that most of these networks are made up of multi-generations of infrastructure stitched together over time.
Story image
Data Protection
Barracuda launches new capabilities for API Protection
"Every business needs this type of critical protection against API vulnerabilities and automated bot attacks," Barracuda says.
Story image
Remote Working
Australia’s remote workers face connectivity and security issues
SOTI's new report finds better video conferencing technology and improved security measures are top concerns for remote workers in Australia.
Find out how a behavioural analytics-driven approach can transform security operations with the new Exabeam commissioned Forrester study.
Link image
Story image
Cybersecurity starts with education
In 2021, 80% of Australian organisations responding to the Sophos State of Ransomware study reported being hit by ransomware. 
Story image
Application Security
What are the DDoS attack trend predictions for 2022?
Mitigation and recovery are vital to ensuring brand reputation remains solid in the face of a Distributed Denial of Service (DDoS) attack and that business growth and innovation can continue.
Story image
Digital Transformation
Trading up: It's time to swap core systems for flexible digital applications
This year will see more oranisations planning and commencing high tech renovations that will shake up the way they operate.