Australia to legislate safety rules for frontier AI

Australia will legislate mandatory safety and security obligations for frontier AI companies after a rapid review into an AI agent's infiltration of the Medicare Statistics Reporting Service Portal.

The government also plans to establish a regulator with the power to enforce the rules and impose penalties for non-compliance. The package includes mandatory reporting of misaligned AI behaviour and cyber incidents.

The measures mark a shift in Australia's approach to AI oversight. They place formal obligations on frontier model developers and sharpen scrutiny of how AI systems are monitored after deployment.

Industry reaction focused on governance, cyber risk and the practical effect on businesses already using AI in day-to-day operations. Several executives said the framework should prompt organisations to tighten controls around data, visibility and accountability rather than wait for the finer details of the legislation.

Angela Colantuono, Vice President and Managing Director, SAP Australia and New Zealand, said the announcement was a welcome step, but argued Australia must be clear about which AI it is regulating.

"The frontier model debate has consumed most of the oxygen, yet enterprise AI, the AI already running inside Australian businesses and government agencies, is how most organisations will actually move from consuming AI to participating in it. That is where this framework will have the most practical impact.

The organisations I work with that are moving fastest have one thing in common: they have invested in the right foundations: clean data, integrated systems, clear governance and human oversight built in from the start.

The human element sits at the centre of all of it. AI is at its most powerful when it works alongside people, not instead of them, handling the repetitive and time-consuming so people can focus on work that genuinely needs judgement, experience and creativity.

At SAP, we work with some of Australia's largest businesses and government agencies. What we see is clear: the organisations best positioned to unlock that value are those that treat security, transparency and accountability as the starting point, not an afterthought."

Security focus

Fortinet said the package sets a baseline for trust in AI systems, but argued organisations need stronger visibility into the software agents operating inside their networks and business applications. The issue has become more prominent as AI tools take on a wider range of tasks with access to internal systems and data.

Cornelius Mare, Chief Information Security Officer, Australia, Fortinet, said the proposed obligations would give organisations a clearer baseline for trusting the AI systems they adopt.

"Requirements such as security controls across development and deployment, independent model evaluation, and rapid incident notification give organisations a clearer baseline for trusting the AI systems they adopt.

As AI agents take on more tasks across business and government systems, we'll see more focus on agent identities and visibility of rogue agents. Organisations need to know which agents are operating in their environment, what those agents can access, and what actions they're taking. Visibility will be key. The Australian Signals Directorate's recent guidance is a good model here, with governance and visibility built in across each of the domains it identifies.

Although more detail on the Australian AI Standards legislation is likely to come in 2027, organisations shouldn't wait. Treating AI agents like any other identity, with clear ownership, least-privilege access, and continuous monitoring, will put them in a stronger position as the rules take shape."

Nicole Henry, Head of Government Affairs, Australia and New Zealand, Fortinet, focused on how the new structure will work across government. She said the test would be whether agencies can share information and act quickly when incidents emerge.

"The government's decision to legislate mandatory safety and security obligations for frontier artificial intelligence companies, establish a targeted regulator, and accept the rapid review recommendations is a significant step. Success will be measured by improved resilience: whether risks are identified early, information reaches the right people, and harm can be contained.

The government is asking organisations to adapt around AI. Minister Ayres' appointment to administer the Department of the Prime Minister and Cabinet and contribute to overseeing the Office of AI recognises that government must do the same. AI does not fit neatly within one portfolio. The test is whether that alignment at the top produces clear accountability and faster, coordinated decisions across government.

Mandatory incident notification can provide earlier warning, but no single institution will hold the whole picture. The regulator will need to work closely with the Australian AI Safety Institute, ASD, existing regulators, and industry to turn that warning into action.

Good governance depends on visibility. Decision-makers need a shared understanding of emerging risks, clarity about who is responsible, and the ability to act before harm spreads. Australia can build on the leadership and partnerships developed through cybersecurity to create AI governance that keeps pace with the technology."

Business response

Logicalis said trust will be central as companies expand AI use in operations and decision-making. It linked that trust to governance and cyber controls rather than regulation alone.

Lisa Fortey, General Manager, Logicalis Australia and New Zealand, said organisations should not wait for regulation to define best practice.

"AI represents an enormous opportunity for Australian businesses to improve productivity, drive innovation and fundamentally change the way we work. But to realise that opportunity, organisations need to have confidence in the technology they are adopting. Trust is absolutely critical.

As AI becomes more embedded in the way businesses operate and make decisions, cybersecurity and governance can no longer be treated as separate conversations. Organisations need to understand where AI is being used, what information it has access to and, importantly, how they protect their customers, their people and their business.

The government's increased focus on safeguards and accountability is an important step, but I don't believe organisations should be waiting for regulation to tell them what good looks like. We need to be proactive in building security, resilience and responsible governance into our AI strategies from the beginning.

At Logicalis, our focus is on helping customers embrace new technologies with confidence. We don't see cybersecurity as something that should slow innovation down. Quite the opposite. When organisations get the foundations right, they can move faster, adopt AI at scale and unlock real business value.

Ultimately, the organisations that will benefit most from AI won't necessarily be those that adopt it first, but those that build the trust, security and resilience needed to make it sustainable."