Over recent weeks there have been many reports that some AI tool went rogue and hacked into a system. It started with a funny story about an AI agent hacking into a gym's systems to advance someone in a queue for a morning class. Then it was Open AI and the Hugging Face incident and, this week, it was revealed an AI agent 'hacked' into non-public information at Medicare.
In each case, the AI found a previously undiscovered vulnerability, developed an exploit or attack chain, and infiltrated the system to do something the creator of the agent didn't anticipate.
Guess what? This is what threat actors have been doing for decades. If you're old enough to watch the movie War Games, you'll recall that the hack into the nuclear missile systems happened because of a vulnerability in systems. Perhaps the best-ever hacker movie, Sneakers was based on a similar premise. Find a weakness (in that case social engineering was a big part of the attack chain) and exploit it.
AI accelerates the process. It doesn't break through security - it finds a way through.
The distinction is important. While releasing poorly controlled AI is akin to releasing a virus, the vulnerabilities they exploit are the responsibility of the system owners.
Guardrails for AI are important - just as guardrails for all software are. No one wants software developers to release self-replicating applications that access information, extract it and share it without authority. And no one wants AI agents that do the same.
But what about making system owners accountable for the weaknesses and vulnerabilities that were exploited? This side of the "AI hacking" debate has been ignored in public discourse. In every case, the AI may have executed actions that were unanticipated but they discovered vulnerabilities that were already there.
AI didn't break smash the locks and break through the front door. It found an unlocked side window and climbed in.
The consequences of this new world are significant. Attackers can now operate at machine-speed velocity and volume. That's more attacks coming faster than humanly possible. Defensive and risk management strategies must adapt.
For sufficiently resourced organisations traditional red teaming must employ frontier AI models and tools. When the first Mythos models were revealed and discovered vulnerabilities in open source software that had been undetected for two decades, that should have been a wake-up call about failures in red teaming. Instead, it became a panic about AI.
Smaller organisations without the resources to engage experts are in a more precarious position. SMEs contribute over half of Australia's GDP. While they might have been able to claim being too small for threat actors to care about, they are now prime targets. AI has changed the risk equation.
AI enables threat actors to engage in a high-volume attack method. They can now afford to attack thousands of smaller targets through AI and automation. The attack chain is now so low-cost that even targets that only yield a few hundred dollars yield attackers a significant return on investment.
Basic counter-measures like multi-factor authentication, encrypting data wherever possible and maintaining up-to-date software are critical. The excuse of saying those actions were too annoying is no longer valid.
AI and automation have fundamentally changed the economics of information security. Threat actors can detect vulnerabilities and weaponise them faster than ever before. Organisations that don't adapt to this new world will find themselves the victims of an increasingly volatile and fast moving threat landscape.