OpenAI has apologised after one of its agents went rogue and hacked into Medicare, pledging money to help the Australian government strengthen its cyber defence.

The incident – in which an OpenAI agent breached a Medicare statistics portal during a training exercise – happened in June but the company did not inform the federal government until earlier this month.

The government has established its own rapid response task force to investigate the breach and has begun moving sensitive Medicare data to new platforms.

In its first public comments on the attack, OpenAI revealed on Tuesday the breach was broader than first known. It said its agent was supposed to be researching government spending using public statistics, but instead it retrieved internal files, credentials, and non-public aggregate statistics from Services Australia.

Open AI said it would set up an Australian-based task force, using domestic expertise to try to ward off the autonomous strength of AI. It is also expected to introduce mandatory reporting procedures for rogue AI agents, which would involve relevant government departments and the Australian Signals Directorate.

When OpenAI did finally inform the government about the breach, it did so via a generic email to a Services Australia account that was not monitored 24 hours.

“We should have handled our response better,” a statement from OpenAI, announcing the task force, said on Tuesday.

“We are sorry and working to do better in the future.”

Prime Minister Anthony Albanese – who spoke with OpenAI boss Sam Altman last week – said the company had been “constructive and open”, but balancing the dangers of AI was a global problem.

“We’ve seen those risks exposed, not just in what occurred in Australia but the revelation that has occurred in the United States and other countries as well,” he said.

“We see this as an opportunity, but we want to seize the benefit whilst mitigating the risks.”

The breach has raised serious concerns about the ability of AI agents to act on their own to compromise data.

The government will pursue criminal charges if possible; however, it is yet to determine who is legally responsible for the breach.

Public Service Minister Katy Gallagher said it was a “significant issue” and one of the first breaches of its kind in the world.

OpenAI will fund government work through credits from a $1 billion fund that subsidises work on essential services protection. It has so far mainly been used in the US.

The company’s chief strategy officer, Jason Kwon, will appear at a government committee on AI in early October.

“As AI systems broadly grow more capable, we also see a narrowing window to help organisations find and fix weaknesses. This takes collective action working with defenders worldwide,” the statement said.

New AI model release scrapped amid safety concerns

OpenAI has also scrapped the release of GPT-6.1 Astra, a next-generation AI model planned for an October debut, ‌after internal testing found the system did not meet the company’s safety and alignment standards.

OpenAI has warned Astra, its flagship GPT-6 model, can at times evade human oversight, while the company and rivals such as Anthropic have faced scrutiny over ‌experimental AI systems that ‌breached safeguards, including the illicit Medicare access.

The company has not said if Astra was the rogue agent involved in the Australian breach.

The ​Wall Street Journal reported OpenAI had abandoned plans to launch Astra, which was expected to be integrated into ChatGPT and Codex and was designed to handle more complex tasks without human assistance.

It said GPT-6.1 Astra also showed higher levels ⁠of deception than its predecessor in internal ‌testing, including ​instances in which it did not always accurately disclose what actions it had taken.

“While [GPT-6.1 ​Astra] improved on ‌axes such as model laziness, it didn’t quite meet the bar in terms ​of staying within scope and authorisation, and how it communicates back to the user about the type of work it’s done,” said Saachi Jain, head of safety systems ​at ​OpenAI.

“Of course we want to ​make sure our model development is safe no ‌matter whether that’s in the company, or when we ship it to users.”

-with AAP