IT Brief Australia logo
Technology news for Australia's largest enterprises
Story image

Four factors to consider before using AI to augment data analytics

By Contributor
Fri 10 Sep 2021

Article by Snowflake data platform architect, Field CTO Office, Rishu Saxena.
 

As a rapidly developing technology, artificial intelligence appears to have much to offer organisations battling with a rising tide of data.

While traditional approaches to analytics limit the size of the data sets that can be analysed, adding AI means those limits can be significantly increased. This shift opens up new opportunities to gain better insights and deliver significant business benefits.

With this in mind, four key factors need to be considered when determining when and how AI and machine learning tools should be used to augment traditional statistical analysis.
 

The size of the data set

When it comes to analytics, the most important variable is the size of the data set. When it gets beyond a certain number of rows and columns, traditional statistical tools quickly run out of capacity to identify all the correlations and understand what’s interesting or important.

AI tools can augment traditional approaches by allowing staff to analyse more data and identify insights or relationships that would otherwise remain hidden. These tools can also increase awareness that the available data sets do not provide enough information for accurate predictions.

Working with larger data sets ensures higher accuracies on a wide range of algorithms, which these days are only a click away.
 

Explainability versus performance

Another consideration is the level of ‘explainability’ required for problems. For example, if you’re learning to control a self-driving car, you don’t need to understand the impact of every variable on the output. You only need a high degree of confidence that it is safe.

However, if you are predicting whether a piece of machinery will fail, you need to understand the impact of each variable, including temperature, humidity, and the rotation rate of the machine. If not, the output won’t be useful because it doesn’t allow you to take corrective action.

Explainability is important in other contexts, too. If a model produces results that align with intuition, then the model you spent time building may not be used. For instance, if a model predicts that a certain change will increase sales by 40%, but you can’t explain why, people may reject it because they don’t trust the result.
 

Selecting the right algorithm

Different algorithms have different advantages and disadvantages. For example, in forecasting, you might choose between ARIMA, ARMA, Prophet, or LSTM. It’s recommended to consider which are best supported by the broader community and the feature differences between them. Other questions to consider include their limitations and what biases they might inherently introduce.

These are nuanced considerations that are not equally weighted and will depend on the business or analytical problem at hand. Working with domain experts and carefully considering the use case will be critical to choosing the most appropriate algorithm.

Mapping domain knowledge to the problem space is also an important factor when selecting the right algorithms. For example, you may find that there are multiple algorithms to handle classification. However, classification is a diverse area, and what constitutes the right algorithm may vary depending on the nuance of an industry or use case.
 

Achieving execution velocity

For most organisations, true value is found in the data and not in tuning the algorithm so that practitioners can accelerate their use of machine learning with AutoML tools. These tools allow teams to test 20 or 30 algorithms against the same data and compare the performance of those models. That allows for a level of iteration and visibility not previously possible, and it’s a powerful way to test classic algorithms in a much more expansive way.

However, there are trade-offs here too. If five models produce similarly high-performance scores, how do you decide which one is preferable? One option is to conduct A/B tests using a subset of the data to gather empirical evidence about which model truly has the best performance. This is a good general MLOps practice, both when a model is first created and whenever a model is updated.

Adding business value

By adding AI and ML capabilities to a data analyst’s toolbox, a business can extract significantly more value from its data than is currently the case. However, several key factors need to be considered.

Taking the time before rolling out these tools to understand their potential (and limitations) will ensure expected benefits are achieved. 

Related stories
Top stories
Story image
LG Electronics
LG Electronics brings on Aussie AV distributor Alloys
LG Electronics Australia Business Solutions has appointed Australian AV distributor Alloys as a new distribution partner.
Story image
Energy
Sustainability huge factor for APAC data centre managers
A new report reveals that 85% of data centre managers in APAC believe that sustainability will significantly impact operations and decision making.
Story image
Pegasystems
Marketing technologies set to increase revenues - study
Marketing leaders say that investment in new marketing technologies (MarTech) will drive significantly increased revenues.
Story image
Equinix
Aussie businesses buckle down on digital tech investments
Australian businesses are planning major investments in digital technologies to support ambitious expansion plans.
Story image
Document Management
NZ's FileInvite raises $10M in latest investment round
FileInvite has raised $10 million in Series A investment to fast-forward the extinction of email for requesting and collecting documents online.
Story image
Malware
Colt launches new SASE Gateway solution with Versa
Colt Technology Services’ customers now have access to an integrated full SASE solution that brings together SD WAN and SSE features.
Story image
Collaboration
Enterprise service management: the importance of a one-stop shop
In an online world, employees and end-users want one place to go for all their questions and requests. Intranet technology and self-service portals are useful tools that help serve this purpose.
Story image
Artificial Intelligence
Eight top DevSecOps trends to support IT innovation in 2022
The use of DevSecOps practices is growing, as it is increasingly seen as the best way to produce high-quality and secure code. So what are the current trends?
Story image
Wiise
Four things wholesale distributors need to consider for FY2023
In a post-pandemic world, there are many things for a distribution business to juggle. ERP solutions company Wiise narrows down what companies should focus on.
Story image
Media
Registrations for the W.Media Sydney Cloud and Datacenter Convention 2022 now open
Are you a C-Level executive looking to enhance your knowledge in the cloud and data center space in order to get the best results for your company?
Story image
Documentation
Adobe study finds lack of digital trust and utilisation in Australian Government agencies
New research commissioned by Adobe has revealed a significant lack of digital trust within Australian Government departments, along with the continued underutilisation of key digital processes.
Story image
Low-code
Appian unveils low-code certification program in Australia
Appian has announced a program to provide the next generation of low-code developers with access to education on the subject and certification to foster career opportunities.
Story image
Remote Working
RDP attacks on the rise, Kaspersky experts offer advice
"Given that remote work is here to stay, we urge companies to seriously look into securing their remote and hybrid workforce to protect their data."
PwC
PwC's Consulting Business and PwC's Indigenous Consulting are proud to play an important role in helping Australian Indigenous Mentoring Experience build IMAGI-NATION, a free online university for marginalised communities around the world.
Link image
Productivity
Discover the 5 ways your ERP may be letting you down. Is your current system outdated, difficult to manage, and costing you a fortune?
Link image
Story image
Payroll
How New South Wales state departments achieved cloud migration success
State departments in New South Wales are heading to the cloud to achieve better workflow solutions, and one company is paving the way for their success.
Story image
Amazon
What brands can expect from Amazon Prime Day in Australia
Amazon Prime Day is the annual two-day shopping event, kicking off this year from July 12-13 and is the global online shopping platform's biggest sales event. 
Story image
Accounting
Four factors to consider when choosing the right job accounting solution
Progressive job-based businesses can achieve success by strengthening their ability to quantify every cost attributable to the delivery of an outcome for a customer.
Story image
Training
Barracuda customises training to fit ACSC Essential Eight
Barracuda has announced that its Security Awareness Training now provides a customised training curriculum in line with the Australian Cyber Security Centre’s (ACSC) Essential Eight.
PwC
WSLHD and PwC’s Consulting Business came together to solve through the challenges of COVID-19. A model of care was developed to the NSW Health Agency for Clinical Innovation guidelines with new technology platforms and an entirely new workforce.
Link image
Story image
ABI Research
75% of 5G networks to advance to 5G-Advanced by 2030
5G-advanced is set to launch in 2025, and 75% of 5G networks will upgrade by 2030, according to ABI Research. 
Story image
Artificial Intelligence
Siemens expands NVIDIA partnership for industrial metaverse
Siemens is expanding its partnership with NVIDIA to enable the industrial metaverse and increase the use of AI-driven digital twin technology.
Story image
Cybersecurity
Without trust, your security team is dead in the water
The rise of cyberattacks has increased the need for sound security that works across any type of business, but with any change, buy-in is essential. Airwallex explains why.
Story image
Southern Cross Cable
Southern Cross Cable launches the SX NEXT cable to connect NZ to the world
The new Southern Cross NEXT fibre cable (SX NEXT) is set to connect Australasia to the US and further enhance connectivity between New Zealand, Australia, and the US.
Story image
Artificial Intelligence
Accenture shares the benefits of supply chain visibility
It's clear that gaining better visibility into the supply chain will help organisations avoid excess costs, inefficiencies, and complexity to ultimately improve their bottom line.
Story image
Enterprise Resource Planning / ERP
Five ways your ERP is letting you down and why it's time for a change
Wiise explains while moving to a new system may seem daunting, the truth is that legacy systems could be holding your business back.
Digital Transformation
Discover the 5 signs your business is ready for a cloud-based ERP. Is your business being left behind as more of your competitors switch to the cloud?
Link image
Story image
Artificial Intelligence
Dynatrace extends automatic release validation capabilities
Dynatrace has extended its platform release validation capabilities to improve user experience at every stage of the software development lifecycle.
Story image
Ransomware
Examining the future of ransomware threats with Vectra’s CTO
As customers' valuable data move to the cloud, so will ransomware. What is the current landscape and what do we need to know?
Story image
Telecommunications
WiFi as a Service market to reach $26 billion through 2032
As a result of the easy management of wireless infrastructure over cloud services, WaaS is experiencing rapid growth.
Story image
Samsung
Monitors are an excellent incentive for getting employees back
The pandemic has taught us that hybrid working is a lot easier than we would’ve thought, so how can the office be made to feel as comfortable as home? The answer could be staring you in the face right now.
Supply chain
Discover the 4 critical priorities for wholesale distribution businesses in FY23. Are you worried about how supply chain issues may affect your business in 2023?
Link image
Story image
SaaS
Ping Identity appoints Deloitte Australia as a partner
Ping Identity has appointed Deloitte Australia as a Consulting Technology Partner, uniting its offerings with the company's consulting services.
Story image
Metaverse
How the metaverse will change the future of the supply chain
The metaverse is set to significantly change the way we live and work, so what problems can it solve in supply chain management?
Story image
Cybersecurity
Palo Alto Networks' cloud security platform receives IRAP assessment
"We provide help protect all forms of compute, cloud native services and access to data within public and private sectors."
Story image
Airwallex
How Airwallex helps businesses achieve globalisation success
As markets continue to shift, businesses need to be able to provide the same quality of service for customers regardless of where they are located around the world.
Project management
Discover the 4 crucial factors for choosing the right job-costing solution. Is your team struggling to cost jobs and keep projects running on budget?
Link image
Story image
Data Protection
Five signs your business is ready to move to the cloud
Many organisations are thinking about moving to the cloud. But what are the signs you are ready, and what are the reasons to move?
Story image
Document Management
Regaining digital trust and enhancing digitisation in Australian Government agencies
Having a digitised ecosystem of documents, tools and data can help bolster security, improve workflow and ultimately create better services.
Story image
Cybersecurity
Tech and data’s role in the changing face of compliance
Accenture's study found that 93% of respondents agree or strongly agree new technologies such as AI and cloud make compliance easier.
Story image
Microsoft
Elders signs five-year agreement with Microsoft to boost innovation
Australian agribusiness Elders has signed a five-year agreement with Microsoft that looks to transform its customer experience, efficiency and sustainability outcomes.
Story image
CSG
To win at 5G, telcos must tame their quoting chaos
The catalogs of CSP (communication service providers) market offerings are set to explode as new digital services emerge, powered by B2B2X business models.
Story image
Digital Transformation
Munro Footwear Group chooses Boomi to assist with digitisation strategy
Munro Footwear Group (MFG) has selected Boomi to assist with its eCommerce and digital transformation roadmap goals.
Story image
Security Information and Event Management (SIEM)
LogRhythm updates SIEM Platform with latest innovations
LogRhythm has announced the launch of version 7.9 of the LogRhythm SIEM Platform and updates to LogRhythm NDR and LogRhythm UEBA.