Australian workers mistrust AI as errors hit workplaces

Appian has published research showing that 60% of Australian workers have identified an error made by an AI tool at work. The study also found that only 14% fully trust AI-generated outputs used within their organisation.

According to the workplace survey, more than one in five respondents said employees in their organisation are discouraged from challenging AI recommendations or decisions. The findings point to a gap between the growing use of AI at work and employee confidence in the systems being deployed.

The research suggests AI mistakes are already affecting operations and customer outcomes. One in five workers said the most significant AI error they encountered caused a customer service issue, while 13% said it led to operational disruption.

Almost half of respondents said the most significant AI error they experienced had consequences beyond minor inconvenience, including financial, compliance or reputational impacts.

Trust and oversight appear to be central concerns. Nearly all respondents (94%) said AI should not be allowed to make decisions affecting employees or customers without limits or human oversight.

Workers were also more likely to trust senior management than AI. The survey found that 52% placed greater trust in senior leaders, compared with 29% who said they trusted AI more.

Key concerns

The biggest concern, cited by 25% of respondents, was AI making incorrect decisions. Data privacy and security risks followed at 22%, while 20% pointed to a lack of transparency around how AI reaches decisions.

The results suggest concerns extend beyond isolated mistakes to how organisations govern AI. They also indicate workers want clearer limits on where automated systems can be used and how their outputs can be checked.

Two-thirds of Australian workers said decisions affecting employees or customers should either require human review and approval or always be made by people. A further 28% said AI should be restricted to low-risk decisions.

Charlie Hutchinson, Senior Vice President, Asia-Pacific and Japan, Appian, said workers were reacting to what they were seeing in practice.

"It is hardly surprising that complete trust in AI is so low when workers are catching it making mistakes and are discouraged from questioning its outputs," Hutchinson said.

He said employee hesitation could undercut the expected gains from using AI and warned that organisations risk allowing flawed outputs to stand if staff do not feel able to challenge them.

"For businesses, mistrust creates a lose-lose situation. Employees may avoid AI or spend time checking everything it produces, wiping out the productivity gains it was meant to deliver. Alternatively, they may continue using an output they suspect is wrong because they do not feel able to challenge it. AI needs to operate within the same governance processes and controls as the rest of the business, with people able to step in before a significant error occurs," Hutchinson said.

Governance gap

Appian also cited separate global research by Harvard Business Review Analytic Services on how businesses are handling AI controls. That study found 92% agreed AI agents need rules-based guardrails to operate safely and effectively, yet only 48% said their organisation had defined them.

The contrast suggests many organisations recognise the risks tied to AI use but have yet to put formal controls in place. The issue may become more pressing as AI moves from standalone productivity tasks into business processes that directly affect customers and staff.

Hutchinson said employees often act as an informal check before errors reach customers or influence decisions.

"Every visible error gives workers another reason to question whether AI should be trusted with more consequential work," Hutchinson said.

"Employees are often the last line of defence before an incorrect output reaches a customer or becomes part of a business decision. If they are discouraged from speaking up, organisations remove one of their most important safeguards," he said.

He also argued that governance needs to be designed into AI use from the outset rather than added later.

"Businesses know AI needs rules, but many are deploying it before those rules are in place. Doing so without the right controls can quickly lead to unexpected token costs, operational disruption and accountability gaps," Hutchinson said.

"Governance cannot be bolted on after AI is already influencing decisions. AI needs to be embedded within governed business processes that control what it can access, define the rules it must follow, and determine when a decision must be handed to a person. That gives employees the visibility and confidence to use AI while allowing businesses to capture its productivity benefits without giving up accountability," he said.

The findings add to a broader debate in corporate Australia over how AI systems should be supervised as they are introduced into daily work. They also suggest worker trust may depend less on the presence of AI itself than on whether staff can question, review and override its outputs when needed.

"Australian workers are rejecting unchecked AI. Businesses will only realise the full value of AI if employees build trust, are willing to use it within critical processes, and are confident enough to act on its outputs. That requires governed processes where AI recommendations and autonomous agent outputs can be systematically reviewed, validated or overridden, and where human judgement remains part of important decisions," Hutchinson said.