IT Brief Australia logo
Technology news for Australia's largest enterprises
Story image

SnapLogic: How AI bias is holding back adoption

By Contributor
Mon 13 Jan 2020
FYI, this story is more than a year old

Article by SnapLogic CTO Brad Drysdale

Artificial intelligence (AI) is already having a considerable impact on businesses across industries, and its influence is showing no signs of slowing down.

There are few sectors that have not yet been touched by AI.

However whilst the development and continued innovation around AI is going to bring benefits to business and our wider society, it is important that any potential risks are managed appropriately to prevent AI bias and allow AI to advance responsibly. 

Research from SnapLogic found that ethical and responsible AI development is a top concern amongst IT leaders, with 94% believing that more attention needs to be paid to corporate responsibility and ethics in the application of AI.

But it is not just IT and business professionals who are concerned about AI.

As governments also look to implement AI strategies, they too are making it clear that transparency and responsibility will be a top priority.

The New South Wales government recently announced that it will be launching its AI strategy in March 2020 and pledged that transparency would be top of the agenda.  

There’s no question that the insights AI offers can be highly beneficial, however, we must also recognise its limits in providing perfect answers.

Data quality, security, and privacy concerns are real and until these are addressed, the AI regulation debate will continue. 

What is holding AI back?

AI bias is a key hurdle that must be overcome, occurring when an algorithm delivers prejudiced results due to incorrect assumptions in the development process.

AI bias is often ‘built-in’ through the unconscious preferences of the humans who create the programme or select the training data.

Issues can also be found throughout the entire process, for example in the data-gathering stage, where weighting procedures can result in incorrect conclusions to be made about certain data sets.

Bias is a real concern as we become increasingly reliant on AI and we’ve already seen legal cases emerge where groups have forced the disclosure of how algorithmic processes make decisions.

An example of this involved teachers who were not paid performance bonuses.

They won damages when it was realised that the algorithm assessing eligibility for the bonus did not take into account class sizes – a factor found to be highly significant in pupil attainment. 

Unless AI bias is addressed and eradicated, we can expect public trust of AI to remain an issue as well as potentially more legal cases as organisations and individuals seek complete transparency over how AI makes decisions. 

Where does bias creep into AI processes?

The difficulty with AI bias is that it can be challenging to pinpoint exactly where it enters the system. Bias can form at any stage of the learning process and it does not always relate to training data alone – it can also emerge whilst collecting data, setting objectives, or preparing the data for training or operation.

The initial process of collecting, selecting, and cleaning data is commonly associated with AI bias.

In this early stage, bias can arise if data outliers are perceived as irrelevant and aren’t thoroughly tested, allowing for prejudices to be accidentally introduced.

This can result in certain factors being mistakenly favoured by the AI, such as gender.

For example, if a successful male-dominated business uses AI to screen candidates and the AI is trained on the CVs and employment data of current employees, it is likely to develop a bias towards males.

It may disregard any female applicants for interviews as they don’t fit the success pattern of the company as it currently exists.

Care must be taken when addressing this, as a simple fix like removing the sex of employees from the training data may not work.

Instead, the AI algorithm may identify patterns of male-dominated hobbies as indicators of desirable employees, for example. 

Setting the objectives for a deep learning model can also be where bias forms, but those handling the process can prevent this by setting contextual objectives in order for recommendations to be accurately generated.

Finally, bias can also be introduced during the stage where data is prepared for processing.

This often results in certain attributes for the algorithms being prioritised over others.

It is imperative that this stage is completed accurately, as the choice of what attributes should be considered or ignored will have a significant impact on the accuracy of the results.

Designing a data pipeline that can handle exceptions is crucial to ensure there is sufficient data for good training outcomes. 

If we don’t know exactly where AI bias stems from, how can we prevent it?

IT and business decision-makers need to be aware of possible bias and how AI can be used in a way that doesn’t encourage it or allow it to be accidentally introduced.

Testing is paramount – bias is often only discovered when a system goes live, by which time the issue can become far more challenging to address.

Testing the system against expectations as it develops and involving a diverse group of stakeholders in the evaluation is critical to its accuracy and success. 

Progress in combating likelihood of bias

When it comes to investigating the source of bias, it is often found that human involvement, feeding into the underlying systems, is responsible for introducing prejudices.

However, once this has been identified and resolved, developers should also check the underlying data to confirm that it is fully representative of all factors that could inform the business decision. Many developments have been made in regard to this – algorithms have now been created that can effectively detect and reduce bias, which is a significant step in the right direction. 

The European Union’s GDPR regulation is an example of government attempts to avert the negative effects of AI bias.

GDPR gives consumers the right to an explanation of how automatic decisions have been made based on their data.

It also protects consumers as AI, and the various profiling methods it is used for, cannot be used as the sole decision-maker in choices that can have a significant impact on the rights or freedoms of individuals.

For example, AI alone cannot decide if someone is eligible for a bank loan. 

Driving forward data-focused approaches 

As industries look to reduce the potential risks in order for AI to advance responsibly, it is paramount that AI-enabled results reflect global, diverse data.

There are valid concerns that AI decisions often reflect biases of first-world cultures, which work against the less well off in society.

To combat this, developers need to ensure the inclusion of inputs from wider, more globalised and diverse data. 

In addition, building AI models using original data can help to eliminate bias as there is greater scope for ideas and actual evidence to feed into AI systems that can, therefore, evolve to offer insights beyond the typical first-world perspective.

This data-driven approach would also build far more flexibility and responsiveness into AI systems and expose them to a more comprehensive and diversified set of global considerations.

A data-driven approach is definitely the way forward, but to allow for this, it is important that there is a focus on developing systems that break down data silos, enable seamless data integration, and ensure consistent data access and information flow.

This is possible without the need for expert software developers, as there are intuitive, self-service tools on the market that can integrate large volumes of data between different systems. 

Data is key for any AI system and ultimately, there must be regulations in place that protect against bias, but also allow for continual data access and information flow.

The more decisions that are made as a result of incorrect data from prejudiced assumptions, the more challenging it will be for AI to continue to innovate and progress.

Those who invest in ensuring they can access a wide and diverse pool of data will benefit the most by identifying the true value of AI and preventing bias from influencing decisions. 

Related stories
Top stories
Story image
Thales on recruitment hunt for next disruptive innovations
"Recruiting new talent is part of Thales's belief in the power of innovation and technological progress to build a safer, greener and more inclusive world."
Story image
Palo Alto Networks' cloud security platform receives IRAP assessment
"We provide help protect all forms of compute, cloud native services and access to data within public and private sectors."
Story image
Jamf introduces new content filtering solution for education providers
Jamf has announced the launch of Jamf Safe Internet, a new offering that looks to deliver a safe online experience to students while offering better management options for admins.
Story image
Video: 10 Minute IT Jams - An update from Paessler
Sebastian Krüger joins us today to discuss how unified infrastructure monitoring enables MSPs to seamlessly deliver services to their clients.
Story image
Online shopping
Consumers want speed, visibility in return for brand loyalty
72% of Australian shoppers want complete online order visibility and 63% are loyal to retailers who deliver goods the fastest.
Story image
Ericsson and Ciena, Telstra enhance service capacity for Telstra's optical network
Ericsson, Telstra, and Ciena have announced new enhancements to Telstra's Next Generation Optical Network, which will increase the service capacity of Telstra's optical network to 400 GE (Gigabit per Second Ethernet).
Story image
Remote Working
RDP attacks on the rise, Kaspersky experts offer advice
"Given that remote work is here to stay, we urge companies to seriously look into securing their remote and hybrid workforce to protect their data."
Story image
Colt launches new SASE Gateway solution with Versa
Colt Technology Services’ customers now have access to an integrated full SASE solution that brings together SD WAN and SSE features.
Story image
Four things wholesale distributors need to consider for FY2023
In a post-pandemic world, there are many things for a distribution business to juggle. ERP solutions company Wiise narrows down what companies should focus on.
Supply chain
Discover the 4 critical priorities for wholesale distribution businesses in FY23. Are you worried about how supply chain issues may affect your business in 2023?
Link image
Story image
Ivanti puts spotlight on power of employee digital experiences
The report revealed that 49% of employees are frustrated by the tech and tools their organisation provides and 64% believe this impacts morale.
Story image
Tech job moves
Tech job moves - Bitdefender, Cohesity, Fortinet & MODIFI
We round up all job appointments from June 27-30, 2022, in one place to keep you updated with the latest from across the tech industries.
Story image
Enterprise Resource Planning / ERP
Five ways your ERP is letting you down and why its time for a change
Wiise explains while moving to a new system may seem daunting, the truth is that legacy systems could be holding your business back.
Story image
Macquarie Data Centres
Macquarie deal to pioneer CO2-cutting data centre tech in Australia
Macquarie Data Centres has signed a multi-year deal with ResetData, an Australian first provider using Submer data centre technology. 
Story image
New VMware offerings improve cloud infrastructure management
VMware has unveiled VMware vSphere+ and VMware vSAN+ to help organisations bring benefits of the cloud to existing on-prem infrastructure.
Story image
How the metaverse will change the future of the supply chain
The metaverse is set to significantly change the way we live and work, so what problems can it solve in supply chain management?
Project management
Discover the 4 crucial factors for choosing the right job-costing solution. Is your team struggling to cost jobs and keep projects running on budget?
Link image
Story image
BT builds on Equinix partnership with new cloud offering
BT has launched a next-generation cloud connectivity offering extending its global network into strategic carrier-neutral facilities (CNFs) and building on its existing partnership with Equinix.
Story image
Intel Labs unveils integrated photonics research advancement
"This new research demonstrates that its possible to achieve well-matched output power with uniform and densely spaced wavelengths."
Story image
Appian unveils low-code certification program in Australia
Appian has announced a program to provide the next generation of low-code developers with access to education on the subject and certification to foster career opportunities.
Story image
Without trust, your security team is dead in the water
The rise of cyberattacks has increased the need for sound security that works across any type of business, but with any change, buy-in is essential. Airwallex explains why.
Digital Transformation
Discover the 5 signs your business is ready for a cloud-based ERP. Is your business being left behind as more of your competitors switch to the cloud?
Link image
Story image
Industry-first comprehensive risk-based API security enhances protection
Application Programming Interfaces (APIs) have become a crucial part of operating web and mobile application businesses and are causing significant economic growth in the digital sector.
Story image
Artificial Intelligence
Vectra AI named as AWS security competency partner
Threat detection and response company Vectra AI has announced that it has become an Amazon Web Services Security Competency Partner.
Story image
How organisations can mitigate IoT and IIoT security risks
IoT and IIoT come with inherent risks because they are often deployed faster than they can be secured, putting organisations in danger of cyber threats. Here are tips on how to mitigate those risks.
Story image
Multi Cloud
Cloud is a tool, not a destination
For many years, “cloud” has been thought of as a destination which has led to a misguided strategy that sees an enterprise trying to shift all its applications to a single cloud provider – regardless of the specific needs and nuances of each individual workload.
Story image
How New South Wales state departments achieved cloud migration success
State departments in New South Wales are heading to the cloud to achieve better workflow solutions, and one company is paving the way for their success.
Story image
Tech and data’s role in the changing face of compliance
Accenture's study found that 93% of respondents agree or strongly agree new technologies such as AI and cloud make compliance easier.
PwC's Consulting Business and PwC's Indigenous Consulting are proud to play an important role in helping Australian Indigenous Mentoring Experience build IMAGI-NATION, a free online university for marginalised communities around the world.
Link image
Story image
Australian consumers loyal to retailers who deliver speed and visibility
SOTI finds extensive order visibility and speed are the most important factors for turning one-off customers into loyal, long-term buyers.
Story image
Artificial Intelligence
Accenture shares the benefits of supply chain visibility
It's clear that gaining better visibility into the supply chain will help organisations avoid excess costs, inefficiencies, and complexity to ultimately improve their bottom line.
Story image
Artificial Intelligence
Dynatrace extends automatic release validation capabilities
Dynatrace has extended its platform release validation capabilities to improve user experience at every stage of the software development lifecycle.
Story image
Four factors to consider when choosing the right job accounting solution
Progressive job-based businesses can achieve success by strengthening their ability to quantify every cost attributable to the delivery of an outcome for a customer.
Story image
Adobe study finds lack of digital trust and utilisation in Australian Government agencies
New research commissioned by Adobe has revealed a significant lack of digital trust within Australian Government departments, along with the continued underutilisation of key digital processes.
Story image
Data Protection
Five signs your business is ready to move to the cloud
Many organisations are thinking about moving to the cloud. But what are the signs you are ready, and what are the reasons to move?
Story image
Dicker Data
EXCLUSIVE: Why women in IT makes good business sense - Dicker Data
The Federal government wants to bolster female participation in the tech industry to at least 40% by 2030. Here's how one homegrown Australian company has already reached that goal.
Story image
Monitors are an excellent incentive for getting employees back
The pandemic has taught us that hybrid working is a lot easier than we would’ve thought, so how can the office be made to feel as comfortable as home? The answer could be staring you in the face right now.
Story image
What brands can expect from Amazon Prime Day in Australia
Amazon Prime Day is the annual two-day shopping event, kicking off this year from July 12-13 and is the global online shopping platform's biggest sales event. 
WSLHD and PwC’s Consulting Business came together to solve through the challenges of COVID-19. A model of care was developed to the NSW Health Agency for Clinical Innovation guidelines with new technology platforms and an entirely new workforce.
Link image
Story image
How Airwallex helps businesses achieve globalisation success
As markets continue to shift, businesses need to be able to provide the same quality of service for customers regardless of where they are located around the world.
Discover the 5 ways your ERP may be letting you down. Is your current system outdated, difficult to manage, and costing you a fortune?
Link image
Story image
State Library of Victoria
State Library of Victoria entrusts Oracle support and security to Rimini Street
“Our finance team are very happy with the support and security that Rimini Street provides, which keeps our assets and our customers secure."
Story image
Sutton Tools deploys Infor M3 CloudSuite for manufacturing
Sutton Tools has also implemented the Infor OS cloud operating platform, including Infor Intelligent Open Network and Mongoose.
Story image
Identity and Access Management
Ping Identity named a Leader in Access Management
Ping Identity has been named a leader in the 2022 KuppingerCole Leadership Compass report for Access Management.