IT Brief Australia logo
Technology news for Australia's largest enterprises
Partner content
Story image

Scaling AI: Making machine-learning models more effective and efficient

By Contributor
Mon 29 Nov 2021

Article by Infosys SVP & industry head, global markets, Raja Shah.

High-quality, clean and appropriately labelled data is undeniably crucial in today's world. 

Companies are increasingly dependent on the ability of AI and machine learning (ML) models to provide real-time insights that drive business and customer engagement outcomes. 

With an exponential increase in data, AI and ML algorithms are integral to leveraging this data effectively. This is key to enabling everything from self-driving cars, cashier-less shopping services and even cancer detection.

In the realm of the telecom world specifically, we see AI and ML being used for various use cases that enhance customers' experiences of solutions and services. This includes speech recognition and activated commands which have become almost must-have smart features in today's fast-paced world.

And with increasing reliance, the quality of data and data models that minimise unconscious bias from human data labellers is even more important. With customer behaviour and genome analysis more prevalent for customer mapping, telecoms can confidently hyper-personalise offerings when data is effectively cleansed.

As the data is crucial, the design and testing process, which includes data cleansing and labelling, must be extensive to minimise bias in data. The industry is awash with new and dedicated data labellers, such as San Francisco-based start-ups Scale AI and Sama.

Google and Amazon also complete gargantuan manual labelling tasks, especially in the legal and healthcare industries, but often charge businesses a particularly high fee.

Across all these data labelling services, there is no guarantee that the output will be comprehensive, unbiased, or free from noise, which adds a risk of flawed outcomes and inefficiencies. The length of time required to successfully clean and label data is often too long for agile companies.

At Infosys, we understand that 25-60% of ML projects costs come from manual labelling and validation of data. Expenditure on these tasks seems to be increasing, with little guarantee of quality. AI consultancy Cognilytica estimates enterprises will collectively spend US$4.1 billion on data labelling by 2024. 

So, what's a faster and more effective way to reduce bias and deliver clean data for hungry ML algorithms? 

An approach that combines intelligent learners and programmatic data creation is required. By allowing AI to do the heavy lifting for deskilled data labelling, overall bias can be reduced and efficiency and effectiveness can ultimately be boosted. Here are some of the ways this transformation can take place:

Active Learning 

During the active learning process, an intelligent learner examines unlabelled data and picks parts of it for further human labelling. Using a classifier can help control what data is selected and helps address areas that haven't been optimised for machine learning. This makes the labelling process active rather than passive and, in turn, increases data quality.

Active learning was recently used in the legal industry to label contractual clauses. Through the process, data accuracy increased from 66% to 80%, even when using fewer data points, while the cost and time involved were also significantly lower. 

In a situation where an AI-based decision appears biased, it is easier to interrogate and find the reason why. The result of a Netflix recommendation, for example, is based on a set of rules driven by user data. If the rules appear to be displaying biased results, while complicated, the machine learning model can be investigated to find out why and corrected to remove perceived bias.

Distant supervision 

Using distant or weak supervision to programmatically create data sets is the best way to use AI at scale. In both approaches, a labelling function is programmed to create labels from input datasets. That means distant or weak supervision can combine noisy signals and resolve conflicting labels without any sort of reference to a "ground truth".

Distant supervision produces noise-free training data using distance knowledge bases. By looking across multiple data sources and databases, distant supervision can map the metrics for machine-based learning models.

The process has 98% accuracy, but there may still be noise in the label depending on the type and number of knowledge bases in the training data available. One challenge with this model is that finding distant knowledge bases can be difficult, and ML engineers need expert domains to help them uncover the appropriate information. 

When data needs to be sourced from unreliable sources, it is best to use weak supervision.  

Synthetic data generation 

When data and labelling functions don't yet exist, there's an option to make up the data. 

Amazon took this approach at its new Go Stores, which are small convenience stores where no check-out is required. Amazon created virtual shoppers using graphics software, which in turn trained computer vision algorithms about how to learn what real-world shoppers select off the shelf. 

NASA's Perseverance mission to Mars also saw the entire Martian landscape synthetically captured using synthetic data generation. 

Like the virtual shoppers, synthetic data has the same representative characteristics as the real-world data from which it is derived. The data must have exposure to converse use cases and outliers to reduce uncertainty and ensure it is fair, safe, reliable and inclusive.

This can be seen in the case of churn prediction. Churn prediction is about analysing relevant data to identify factors indicating that a given customer is a flight risk. If you know which customers are about to cancel their subscription or terminate their contract, you can take proactive measures and prevent them from leaving. This can be created without data being generated by calls which may be annoying to the customer and who may have already been contacted for other services by the same provider.

AI projects require quality labelling of data in a timely manner. At the moment, about one-quarter of the time devoted to a machine learning task is spent labelling – well above the 3% of time devoted to developing algorithms.

As large corporations seek to scale AI into every part of their business, they will likely struggle with the trade-off about how to make the process work effectively and efficiently. But active learning, distant supervision and synthetic data generation can do the heavy lifting and significantly reduce costs and increase the efficiency of deskilled data labelling while also improving the quality required to achieve powerful AI models into the future.

For more information on Infosys, visit:

Related stories
Top stories
Story image
Thales on recruitment hunt for next disruptive innovations
"Recruiting new talent is part of Thales's belief in the power of innovation and technological progress to build a safer, greener and more inclusive world."
Story image
Palo Alto Networks' cloud security platform receives IRAP assessment
"We provide help protect all forms of compute, cloud native services and access to data within public and private sectors."
Story image
Jamf introduces new content filtering solution for education providers
Jamf has announced the launch of Jamf Safe Internet, a new offering that looks to deliver a safe online experience to students while offering better management options for admins.
Story image
Video: 10 Minute IT Jams - An update from Paessler
Sebastian Krüger joins us today to discuss how unified infrastructure monitoring enables MSPs to seamlessly deliver services to their clients.
Story image
Artificial Intelligence
Salesforce announces new innovations for financial services
Salesforce has launched expanded financial services that offer more targeted and trusted automation to help teams unlock insights, deliver better customer service, and drive operational efficiencies.
Story image
State Library of Victoria
State Library of Victoria entrusts Oracle support and security to Rimini Street
“Our finance team are very happy with the support and security that Rimini Street provides, which keeps our assets and our customers secure."
Story image
How organisations can mitigate IoT and IIoT security risks
IoT and IIoT come with inherent risks because they are often deployed faster than they can be secured, putting organisations in danger of cyber threats. Here are tips on how to mitigate those risks.
Story image
SentinelOne integrates with Torq to empower security teams
"With Torq, security teams can extend the power of SentinelOne to systems across the organisation to benefit from a proactive security posture.”
Story image
Identity and Access Management
Ping Identity named a Leader in Access Management
Ping Identity has been named a leader in the 2022 KuppingerCole Leadership Compass report for Access Management. 
Story image
Artificial Intelligence
Vectra AI named as AWS security competency partner
Threat detection and response company Vectra AI has announced that it has become an Amazon Web Services Security Competency Partner.
Story image
Enterprise Resource Planning / ERP
Five ways your ERP is letting you down and why its time for a change
Wiise explains while moving to a new system may seem daunting, the truth is that legacy systems could be holding your business back.
WSLHD and PwC’s Consulting Business came together to solve through the challenges of COVID-19. A model of care was developed to the NSW Health Agency for Clinical Innovation guidelines with new technology platforms and an entirely new workforce.
Link image
Story image
Voice recognition
Renesas and Cyberon expand services with voice recognition
“We are honoured to collaborate with Renesas to simplify the development of embedded voice recognition functions."
Story image
Intel Labs unveils integrated photonics research advancement
"This new research demonstrates that its possible to achieve well-matched output power with uniform and densely spaced wavelengths."
Story image
Monitors are an excellent incentive for getting employees back
The pandemic has taught us that hybrid working is a lot easier than we would’ve thought, so how can the office be made to feel as comfortable as home? The answer could be staring you in the face right now.
Story image
SAS wins Microsoft ISV 2022 Partner of the Year award
"We formed the SAS and Microsoft strategic partnership with a shared goal of making it easier for customers to drive better decisions in the cloud."
Story image
Digital Fingerprint
Decline in counterfeit cherries after digital fingerprinting
Reid Fruits says there’s been a dramatic decline in counterfeit products for its cherries over the past three export seasons to Asia because of digital fingerprinting.
Story image
Adobe study finds lack of digital trust and utilisation in Australian Government agencies
New research commissioned by Adobe has revealed a significant lack of digital trust within Australian Government departments, along with the continued underutilisation of key digital processes.
Story image
Tech and data’s role in the changing face of compliance
Accenture's study found that 93% of respondents agree or strongly agree new technologies such as AI and cloud make compliance easier.
Story image
Artificial Intelligence
Dynatrace extends automatic release validation capabilities
Dynatrace has extended its platform release validation capabilities to improve user experience at every stage of the software development lifecycle.
Digital Transformation
Discover the 5 signs your business is ready for a cloud-based ERP. Is your business being left behind as more of your competitors switch to the cloud?
Link image
Story image
Four factors to consider when choosing the right job accounting solution
Progressive job-based businesses can achieve success by strengthening their ability to quantify every cost attributable to the delivery of an outcome for a customer.
Story image
How Airwallex helps businesses achieve globalisation success
As markets continue to shift, businesses need to be able to provide the same quality of service for customers regardless of where they are located around the world.
PwC's Consulting Business and PwC's Indigenous Consulting are proud to play an important role in helping Australian Indigenous Mentoring Experience build IMAGI-NATION, a free online university for marginalised communities around the world.
Link image
Story image
Australian consumers loyal to retailers who deliver speed and visibility
SOTI finds extensive order visibility and speed are the most important factors for turning one-off customers into loyal, long-term buyers.
Story image
Progress launches latest version of network visibility solution
In Flowmon 12 network solution, Progress has expanded its support for public cloud provider flow log monitoring and launched new features.
Story image
How New South Wales state departments achieved cloud migration success
State departments in New South Wales are heading to the cloud to achieve better workflow solutions, and one company is paving the way for their success.
Story image
New VMware offerings improve cloud infrastructure management
VMware has unveiled VMware vSphere+ and VMware vSAN+ to help organisations bring benefits of the cloud to existing on-prem infrastructure.
Story image
Artificial Intelligence
Accenture shares the benefits of supply chain visibility
It's clear that gaining better visibility into the supply chain will help organisations avoid excess costs, inefficiencies, and complexity to ultimately improve their bottom line.
Story image
MYOB snaps up Sydney-based management software specialists
MYOB has announced the acquisition of Sydney-based business management software and support specialists, GT Business Solutions.
Story image
Four things wholesale distributors need to consider for FY2023
In a post-pandemic world, there are many things for a distribution business to juggle. ERP solutions company Wiise narrows down what companies should focus on.
Story image
BT builds on Equinix partnership with new cloud offering
BT has launched a next-generation cloud connectivity offering extending its global network into strategic carrier-neutral facilities (CNFs) and building on its existing partnership with Equinix.
Story image
Online shopping
Consumers want speed, visibility in return for brand loyalty
72% of Australian shoppers want complete online order visibility and 63% are loyal to retailers who deliver goods the fastest.
Story image
Without trust, your security team is dead in the water
The rise of cyberattacks has increased the need for sound security that works across any type of business, but with any change, buy-in is essential. Airwallex explains why.
Project management
Discover the 4 crucial factors for choosing the right job-costing solution. Is your team struggling to cost jobs and keep projects running on budget?
Link image
Story image
Honeywell named Frankston facility services provider
Honeywell has been named the joint facility services provider for Frankston Hospital’s AU$1.1 billion redevelopment.
Story image
The next stage for 5G in thermal materials - IDTechEx
IDTechEx says higher frequency deployments, such as mmWave devices and very different station types such as small cells, present their own technological evolution and, with it, thermal challenges. 
Story image
Industry-first comprehensive risk-based API security enhances protection
Application Programming Interfaces (APIs) have become a crucial part of operating web and mobile application businesses and are causing significant economic growth in the digital sector.
Story image
How the metaverse will change the future of the supply chain
The metaverse is set to significantly change the way we live and work, so what problems can it solve in supply chain management?
Story image
Data Protection
Five signs your business is ready to move to the cloud
Many organisations are thinking about moving to the cloud. But what are the signs you are ready, and what are the reasons to move?
Discover the 5 ways your ERP may be letting you down. Is your current system outdated, difficult to manage, and costing you a fortune?
Link image
Story image
Colt launches new SASE Gateway solution with Versa
Colt Technology Services’ customers now have access to an integrated full SASE solution that brings together SD WAN and SSE features.
Supply chain
Discover the 4 critical priorities for wholesale distribution businesses in FY23. Are you worried about how supply chain issues may affect your business in 2023?
Link image
Story image
Evonik relies on Getac F110 tablet to control autonomous robot
The aim of the project is to evaluate the practicality of an automated robotic maintenance and inspection solution in the chemical industry.