AI Firms Must Prove Safety Systems Work Under New Rules

Artificial intelligence companies operating in Australia would have to demonstrate that their safety systems reliably meet the required standard under new regulation proposed by the federal government.

The plan marks a shift in emphasis. Rather than leaving developers to police themselves, Canberra wants firms to show evidence that their safeguards work, and keep working, as the technology evolves and is rolled out more widely.

The core of the proposal is consistency. A company that can point to a single successful safety test would not be enough. Under the approach outlined, providers would need to prove their systems hold up over time and across the situations in which people actually use them.

The move reflects growing pressure on governments around the world to respond to the speed at which AI tools have entered workplaces, schools and homes. Concerns raised by experts and the public include misleading or harmful outputs, the misuse of personal information, and the risk that powerful systems are released before their weaknesses are understood.

Putting the burden of proof on the companies themselves is intended to give regulators and the public something concrete to rely on. Instead of taking a developer's word that a product is safe, authorities would be able to look at the evidence behind that claim.

The proposal is still at the draft stage. Details such as which systems would be covered, how compliance would be assessed, and what penalties might apply to firms that fall short are expected to be a focus of consultation with industry, researchers and the wider community before any rules are finalised.

Industry groups are likely to weigh the benefits of clearer expectations against the cost and complexity of meeting them, particularly for smaller local developers that may have fewer resources than the global technology giants. Consumer and civil society advocates, meanwhile, are expected to push for strong enforcement so that the requirements carry real weight.

For everyday Australians, the practical effect would be felt through the AI products they already use, from chatbots and search tools to software embedded in banking, health and government services. The government's argument is that stronger, verifiable safety obligations should make those tools more trustworthy.

The proposed regime places Australia among a growing number of countries moving from voluntary guidelines towards enforceable expectations on AI developers. How closely it aligns with approaches overseas will also matter, given that many of the most widely used AI systems are built and operated offshore.

As the consultation process unfolds, the central question will be how to set rules strict enough to protect the public without stifling innovation. For now, the government's message to the industry is clear: it is not enough to say a system is safe. Companies will have to show it.