IT Brief Australia logo
Technology news for Australia's largest enterprises
Story image

What happens when AI falls into the wrong hands

By Contributor
Mon 15 Apr 2019
FYI, this story is more than a year old

Article by Deep Instinct SVP Stuart Fisher.

Think about what would happen when attackers start using the power of deep learning and machine learning for their advantage?

That being said, currently, the use of AI for attackers is mainly being used in academia and not in practical attacks.

But there’s a lot of talk in the industry about attackers using AI in their malicious efforts, and defenders using AI as a defence technology.

Here’s how to make sense of it all.

There are three types of attacks in which an attacker can use AI:

1. AI-boosted/based cyber-attacks – In this case, the malware operates AI algorithms as an integral part of its business logic. For example, using AI-based anomaly detection algorithms to indicate irregular user and system activity patterns.

Unusual patterns can lead to different malware behaviour, increased or decreased evasion and stealth configurations, and communication times. Situational awareness is implemented in malware for a long time, but AI can offer much more accurate and adaptive approaches.

An interesting use case can be found in DeepLocker, presented by IBM Security researches in Black Hat USA 2018. DeepLocker is encrypted ransomware which autonomously decides which computer to attack based on a face recognition algorithm - meaning only when the target is recognized by the camera (after using face recognition techniques to identify) the attack takes place.

There are other hypothetical use cases, which might be a part of malware business logic. Consider “Anti-VM”, for instance. Sophisticated malware tends to check if it runs on a virtual machine (VM), to avoid operating its malicious activities on sandboxes, which will reveal the file is malicious, or to avoid being analysed by a security researcher, which might reveal how it works.

In order to assist their Anti-VM efforts, malware writers can train a VM environment classifier, that would get environment details (e.g., registry keys, loaded drivers, etc.) as features and understand whether the host the malware is running on is a VM or not. Moreover, such a model can resolve some of the difficulties malware have when they run on cloud hosts, which are also VMs, but not security research-oriented VMs, increasing the malware spread.

2. AI-based attack infrastructure and frameworks – in this case, the malicious code and malware running on the victim’s machine do not include AI algorithms, however, AI is used elsewhere in the attacker's environment and infrastructure – on the server side, in the malware creation process etc.

For instance, info-stealer malware uploads a lot of personal information to the C&C server, which then runs an NLP algorithm to cluster and classify parts of the information as “interesting” (credit card numbers, passwords, confidential documents, etc.).

Another example for this would be #TheFappening attack, where celebrity photos stored on iCloud were leaked. An attack like this one could have taken place on a much larger scale if it was an AI facilitated attack. For instance, computer vision machine-based algorithms could be used to review millions of pictures and identify which of them contains celebrities and then expose only the matching ones, similar to the ones leaked in #TheFappening.

Another example of an AI-facilitated cyber-attack can be a spear-phishing attack, as described in the report: The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation.

In phishing techniques, the target is “fooled” by a superficial trustworthy façade tempting them to expose sensitive information/money, whereas a spear-phishing attack involves collecting and using the information specifically relevant to the target, causing the façade to look even more trustworthy and more relevant.

The most advanced spear-phishing attacks require a significant amount of skilled labour, as the attacker must identify suitably high-value targets, research these targets’ social and professional networks, and then generate messages that are plausible within this context. Using AI – and specifically generative NLP models, this can be done at a much larger scale, and in an autonomous way.

3. Adversarial attacks – In this case, we use “malicious” AI algorithms to subvert the functionality of “benign” AI algorithms. This is done by using the same algorithms and techniques used in traditional machine learning, but this time it’s used to “break” or “reverse-engineer” the algorithm(s) of security products.

For instance, Stochastic Gradient Descent which is a technique used to train deep learning models which can be used by adversaries to generate samples that are misclassified by machine learning or deep learning algorithms.

One example of adversarial learning is placing a sticker in a strategic position on a stop sign, causing it to be misclassified by an image recognition street sign classifier as a speed limit sign.

Another example for this attack is injecting malicious data streams into benign traffic in order to cause an anomaly detection-based network intrusion detection system (NIDS) to block legitimate traffic effectively causing Distributed Denial of Service attacks (DDoS).

Such attacking techniques have been developed by researchers against computer vision algorithms, NLP and malware classifiers.

We believe the AI vs. AI trend will continue to increase and cross the boarders from academic POCs to actual full-scale attacks as computing powers (GPUs) and deep learning algorithms become more and more available to the public.

In order to have the best defence, you need to know how attackers operate. Machine learning and deep learning experts need to be familiar with these techniques in order to build a robust system against them.

Related stories
Top stories
Story image
Bridgestone Australia uses Dematic's AGVs to optimise warehouse operations
Bridgestone Australia has deployed Dematic's Automated Guided Vehicle solution across its new Melbourne warehouse in Truganina.
Story image
Marketplacer and Adobe accelerate partnership for enhanced commerce solutions
Marketplacer has accelerated its partnership with Adobe in order to further enhance the global commerce marketplace.
Story image
Rubber Monkey gears up for Aussie market with latest capital raise
Rubber Monkey is seeking to raise up to NZ$2.5 million of new capital through online investment platform, Snowball Effect.
Story image
ABI Research
NaaS market expected to reach $150B by 2030 - research
"The market is immature and fragmented, but telco market revenue will exceed US$75 billion by 2030 if they act now and transform to align with requirements."
Story image
Enterprise service management: the importance of a one-stop shop
In an online world, employees and end-users want one place to go for all their questions and requests. Intranet technology and self-service portals are useful tools that help serve this purpose.
Story image
Palo Alto Networks' cloud security platform receives IRAP assessment
"We provide help protect all forms of compute, cloud native services and access to data within public and private sectors."
Story image
Businesses unprepared to defend against ransomware attacks
Ransomware attacks continue to impact organisations worldwide with high costs, but businesses are still largely unprepared.
Story image
How organisations can mitigate IoT and IIoT security risks
IoT and IIoT come with inherent risks because they are often deployed faster than they can be secured, putting organisations in danger of cyber threats. Here are tips on how to mitigate those risks.
Story image
What brands can expect from Amazon Prime Day in Australia
Amazon Prime Day is the annual two-day shopping event, kicking off this year from July 12-13 and is the global online shopping platform's biggest sales event. 
Story image
Monitors are an excellent incentive for getting employees back
The pandemic has taught us that hybrid working is a lot easier than we would’ve thought, so how can the office be made to feel as comfortable as home? The answer could be staring you in the face right now.
Project management
Discover the 4 crucial factors for choosing the right job-costing solution. Is your team struggling to cost jobs and keep projects running on budget?
Link image
Story image
Australian consumers loyal to retailers who deliver speed and visibility
SOTI finds extensive order visibility and speed are the most important factors for turning one-off customers into loyal, long-term buyers.
Story image
Appian unveils low-code certification program in Australia
Appian has announced a program to provide the next generation of low-code developers with access to education on the subject and certification to foster career opportunities.
Story image
How New South Wales state departments achieved cloud migration success
State departments in New South Wales are heading to the cloud to achieve better workflow solutions, and one company is paving the way for their success.
PwC's Consulting Business and PwC's Indigenous Consulting are proud to play an important role in helping Australian Indigenous Mentoring Experience build IMAGI-NATION, a free online university for marginalised communities around the world.
Link image
Story image
To win at 5G, telcos must tame their quoting chaos
The catalogs of CSP (communication service providers) market offerings are set to explode as new digital services emerge, powered by B2B2X business models.
Story image
Your tools, your choice: why allow employees to choose their own devices?
Jamf Australia says giving your team the freedom to work with their digital device of choice could help to attract and retain top talent in a tight labour market.
Story image
Enterprise Resource Planning / ERP
Five ways your ERP is letting you down and why it's time for a change
Wiise explains while moving to a new system may seem daunting, the truth is that legacy systems could be holding your business back.
Story image
Supply chain
Supply chains continue to be disrupted, enterprises embrace circular economy
“Businesses urgently need to find a solution that can help them to manage this disruption, and transition to a circular economy."
Story image
Four things wholesale distributors need to consider for FY2023
In a post-pandemic world, there are many things for a distribution business to juggle. ERP solutions company Wiise narrows down what companies should focus on.
Story image
Security Information and Event Management (SIEM)
LogRhythm updates SIEM Platform with latest innovations
LogRhythm has announced the launch of version 7.9 of the LogRhythm SIEM Platform and updates to LogRhythm NDR and LogRhythm UEBA.
Story image
Artificial Intelligence
Decision Inc. partners with to expand offering
Decision Inc. Australia has partnered with to expand its offering to clients in the retail, FMCG, manufacturing, supply chain and logistics sectors.
Story image
Artificial Intelligence
Eight top DevSecOps trends to support IT innovation in 2022
The use of DevSecOps practices is growing, as it is increasingly seen as the best way to produce high-quality and secure code. So what are the current trends?
Story image
Industry-first comprehensive risk-based API security enhances protection
Application Programming Interfaces (APIs) have become a crucial part of operating web and mobile application businesses and are causing significant economic growth in the digital sector.
WSLHD and PwC’s Consulting Business came together to solve through the challenges of COVID-19. A model of care was developed to the NSW Health Agency for Clinical Innovation guidelines with new technology platforms and an entirely new workforce.
Link image
Story image
Thales on recruitment hunt for next disruptive innovations
"Recruiting new talent is part of Thales's belief in the power of innovation and technological progress to build a safer, greener and more inclusive world."
Story image
Four factors to consider when choosing the right job accounting solution
Progressive job-based businesses can achieve success by strengthening their ability to quantify every cost attributable to the delivery of an outcome for a customer.
Story image
Tech and data’s role in the changing face of compliance
Accenture's study found that 93% of respondents agree or strongly agree new technologies such as AI and cloud make compliance easier.
Story image
How the metaverse will change the future of the supply chain
The metaverse is set to significantly change the way we live and work, so what problems can it solve in supply chain management?
Story image
Data Protection
Five signs your business is ready to move to the cloud
Many organisations are thinking about moving to the cloud. But what are the signs you are ready, and what are the reasons to move?
Story image
Digital Fingerprint
Decline in counterfeit cherries after digital fingerprinting
Reid Fruits says there’s been a dramatic decline in counterfeit products for its cherries over the past three export seasons to Asia because of digital fingerprinting.
Story image
Artificial Intelligence
Accenture shares the benefits of supply chain visibility
It's clear that gaining better visibility into the supply chain will help organisations avoid excess costs, inefficiencies, and complexity to ultimately improve their bottom line.
Story image
Video: 10 Minute IT Jams - An update from Paessler
Sebastian Krüger joins us today to discuss how unified infrastructure monitoring enables MSPs to seamlessly deliver services to their clients.
Story image
How Airwallex helps businesses achieve globalisation success
As markets continue to shift, businesses need to be able to provide the same quality of service for customers regardless of where they are located around the world.
Story image
MYOB snaps up Sydney-based management software specialists
MYOB has announced the acquisition of Sydney-based business management software and support specialists, GT Business Solutions.
Story image
Artificial Intelligence
Salesforce announces new innovations for financial services
Salesforce has launched expanded financial services that offer more targeted and trusted automation to help teams unlock insights, deliver better customer service, and drive operational efficiencies.
Supply chain
Discover the 4 critical priorities for wholesale distribution businesses in FY23. Are you worried about how supply chain issues may affect your business in 2023?
Link image
Story image
Progress launches latest version of network visibility solution
In Flowmon 12 network solution, Progress has expanded its support for public cloud provider flow log monitoring and launched new features.
Discover the 5 ways your ERP may be letting you down. Is your current system outdated, difficult to manage, and costing you a fortune?
Link image
Story image
Artificial Intelligence
Dynatrace extends automatic release validation capabilities
Dynatrace has extended its platform release validation capabilities to improve user experience at every stage of the software development lifecycle.
Story image
EXCLUSIVE: Finding the best data center for your business needs with datacenterHawk
Companies using cloud are consistently looking for the best storage solutions to suit their enterprise needs and often have to go through rather complex processes in order to find the right fit.
Story image
Without trust, your security team is dead in the water
The rise of cyberattacks has increased the need for sound security that works across any type of business, but with any change, buy-in is essential. Airwallex explains why.
Story image
Artificial Intelligence
Vectra AI named as AWS security competency partner
Threat detection and response company Vectra AI has announced that it has become an Amazon Web Services Security Competency Partner.
Digital Transformation
Discover the 5 signs your business is ready for a cloud-based ERP. Is your business being left behind as more of your competitors switch to the cloud?
Link image