Most Australian businesses hit by AI failures, study finds

More than three-quarters of Australian businesses have experienced at least one artificial intelligence failure in the past year, according to a BSI study. It found that 76% of respondents in Australia had encountered an AI-related incident.

Produced with Alma Economics, the study combined economic modelling with a survey of 1,287 business leaders across nine countries. It estimated that unmanaged AI risks could put between 0.1% and 3.6% of annual market revenue at risk across exposed sectors, depending on the type of failure.

In Australia, the findings point to repeated breakdowns rather than isolated incidents. Among businesses that reported an AI failure, 88% said they had suffered multiple incidents in the previous 12 months, while 9% had recorded between 51 and 100 separate failures.

Those failures were not confined to trial projects. The research found that 34% affected internal workflow tools, 21% hit customer-facing systems and 28% disrupted both internal and external applications at the same time.

The operational effects were broad. Half of Australian organisations that had encountered failures reported downtime or efficiency losses, above the global average of 44%. Another 35% faced regulatory or compliance consequences, compared with 29% globally, while 31% reported security breaches, 25% safety failures, 24% direct financial losses and 22% reputational damage.

Companies are still increasing spending on the technology. The study found that 72% of Australian businesses had seen year-on-year AI cost increases, and 77% expected budgets to rise further. Half of respondents said AI spending had diverted funding from other strategic initiatives.

Business leaders also remained confident about adoption despite the problems. The survey found that 81% had seen tangible operational benefits from AI, 72% reported a positive return on investment and 77% planned to expand AI investment.

Governance gap

The report highlighted a gap between adoption and oversight. While 53% of Australian organisations said they had policies covering the safe and ethical use of AI, and 59% had standardised risk assessment frameworks, only 37% had established a formal AI governance program.

Alignment with international standards was lower still. Just 29% of Australian organisations said they were taking steps to align with ISO/IEC 42001, an international standard for AI management systems. That was below China at 37% and slightly below the global average of 31%.

The findings also suggested weak visibility over data sources and tools. Only 38% of Australian respondents said they knew which data sources their business relied on to train or deploy AI tools. Although 37% said their workplaces restricted staff use of unauthorised AI tools, 63% still operated without such restrictions.

Interest in more autonomous systems is also growing. About 88% of Australian businesses said they were exploring, piloting or deploying agentic AI, even as respondents cited security concerns, lack of governance and unclear regulation as major obstacles.

Sector exposure

Alma Economics modelled revenue at risk across several use cases. In management consulting, it estimated that fabricated citations, flawed datasets and hallucinated analysis put 3.46% of sector revenue at risk globally, equivalent to about USD $11.42 billion. Across the Asia-Pacific region, the figure was estimated at USD $3.14 billion.

For Australia's management consulting market, the model estimated that unmanaged generative AI errors threatened AUD $169.4 million in annual revenue in a sector worth AUD $4.9 billion. The analysis drew on an Australian case in which a professional services firm produced a government-commissioned report containing fabricated citations and quotes, leading to revisions and a refund of 22.3% of the contract fee.

In autonomous and agentic AI, the study estimated that runaway computing costs, recursive logic loops and unsupervised workflows left 3.58% of the market exposed annually, or USD $409.9 million globally. Customer-facing aviation chatbots were estimated to leave USD $458.08 million of passenger air transport revenue exposed worldwide through booking and ticketing errors.

Legal services and automated screening were also identified as vulnerable areas. The study estimated that non-existent case law and faulty filings put 1.28% of legal market value at risk, while algorithmic bias in tenant screening and student loan underwriting created smaller but still material exposure.

Susan Taylor Martin, Chief Executive, BSI, said: "As AI reshapes our businesses, economies and society, business leaders need to consider what happens when AI doesn't work as intended. What's clear is that AI failure is becoming a regular occurrence, bringing with it significant costs. We need to encourage leaders to do everything they can to prevent failures while being comfortable sharing what they have learnt when an issue arises. Realising the benefits of AI requires taking responsibility for how it is used. Business has been here before, responding to growing cybersecurity risks over the past two decades by embedding safeguards proactively and rigorously investigating and implementing learnings from AI incidents. Now, we must prioritize putting in place robust governance and guardrails to enable safe innovation that protects us all. Acting early is likely to help reduce exposure to financial and legal liabilities, while also helping to build trust."

Charlene Loo, Managing Director, BSI Australia, said: "AI is moving quickly, and Australian businesses are moving with it. But as adoption accelerates, so does the responsibility to get it right. When AI fails, the consequences can be significant, financially, legally and reputationally. Strong governance, independent oversight and rigorous testing need to be built in from the start. Businesses that want to benefit from AI also need to earn and maintain the trust of the people who use it."