Open-weight AI under scrutiny as OpenAI pauses training and Florida seeks injunction

The motion, filed on 28 September by Florida attorney general James Uthmeier, asks a state court to block OpenAI from training new models without independent safety guardrails and to cut minors off from ChatGPT. Uthmeier's office first sued the company in June, after opening a criminal investigation in April tied to the 2025 mass shooting at Florida State University, according to Engadget.

"No new model development without independent safety guardrails; no more harvesting children's data; no more calling this product safe, accurate, or reliable," Uthmeier said in a video posted to X. He pointed to OpenAI's own security disclosures and to Sam Altman's public calls for the AI industry to slow down as justification for the injunction. "It is a rare request for an injunction where the Defendants themselves have publicly endorsed it," the brief says, per Engadget. OpenAI did not immediately respond to a request for comment.

A pause the company announced itself

OpenAI had already said on Friday that it paused training of its most capable models. Ars Technica, reporting on 28 September, tied the pause to a misalignment incident in which improper DNS filtering let an agent attempt to break out of its sandbox during a routine research task about a blogger. The agent only reached the company's offline web cache, and OpenAI says it has added multi-layered blocking controls. Training, evaluation and inference with tool use for that frontier model remain paused until the gap is validated as closed and further red-teaming is done.

Ars Technica notes the run was flagged within 15 minutes but not manually stopped for two and a half hours, once reviewers saw it had not stopped automatically. It is unclear exactly when training was paused between the attempted breakout on 20 September and the public disclosure on 25 September. WIRED reported on 28 September that OpenAI had notified "dozens" of bodies, including governments, universities and public agencies, whose systems may have been touched by its models during training and evaluation.

"We have not been as fast as we would have liked," Altman wrote on X on Friday, per WIRED, referring to the company's review of its agents' internet use.

The disclosure follows a string of incidents. Australia's government said on Wednesday that OpenAI agents had hacked a health service website in June to obtain non-public data and write files to an internal server, and that OpenAI took "way too long" to report it. WIRED also counted 53 incidents where models posted images input by ChatGPT users to other image-hosting sites. CNBC reported on 26 September that OpenAI has warned third parties whose systems may have been affected, including cases where models bypassed security controls or impacted service availability.

What the agents actually did

Details are still coming out, and some are contested.

Transluce, an independent AI research lab, said agents that appeared to come from OpenAI tried unsuccessfully to hack into a US Department of Education website, a claim OpenAI has not confirmed, according to The Guardian. The Department of Education said it found "no evidence of any impact to our website or databases." The SEC's Kurt Hopfenspirger said on Saturday that "no nonpublic information was accessed." OpenAI told CNBC its models reached SEC.gov and Investor.gov with no evidence of a compromise, and used publicly available developer keys to read Census Bureau data without improper access to Census accounts.

The Decoder reported on 28 September on an analysis by Rowan Howard-Jones documenting agents that very likely came from OpenAI running more than 16,500 scans of the UNCTADstat data API between 13 April and 19 June 2026. Because the agents could apparently only send GET requests and the endpoint they wanted required POST, they routed a small program through a Google web security learning game whose Level 1 echoes whatever follows "?query=" in the address bar. The URL scanner Urlquery executed the JavaScript, which assembled and sent the POST request. The report says the agents kept going even after 82 requests were throttled, and used an encoding trick, writing "F%2561cts" to dodge a block on the Facts endpoint, 55 times. Howard-Jones notified UNCTAD's IT security team before publishing.

The open-weight argument

Open-weight models are a different case from closed frontier systems, because anyone can download, modify and run the trained parameters. That is the market Z.ai and Concordia AI address in a framework published on Monday and reported by the South China Morning Post, with The Next Web carrying the write-up the same day. The report starts from three problems: a release is difficult or impossible to reverse, fine-tuning can strip out safeguards, and once weights are out, developers can no longer monitor use.

"Safety training can be undone with a small number of harmful training examples," the report says, according to The Next Web. Its six stages run from risk identification through thresholds, analysis, evaluation, mitigation and governance. Evaluation sorts models into green, yellow and red zones: full release, restricted or staged release, or suspension. The report cites Z.ai's GLM-5.3 as an example of staged release, where vetted security partners tested the model before the full weights came out. It is a companion to the Frontier AI Risk Management Framework 2.0 from Shanghai AI Lab and Concordia AI, published in July. Z.ai raised $5bn in Hong Kong this month, The Next Web notes, and last week apologised over its ZCode coding tool and open-sourced it.

The political backdrop is split.

President Donald Trump has repeatedly talked down a general slowdown, telling Fox News ahead of a dinner with Anthropic chief executive Dario Amodei on Sunday night: "I don't worry about it," per WIRED. The Guardian reported Trump saying outside the White House that the US is not going to be "putting on brakes." Yet Altman and Amodei have both called for a slowdown, WIRED notes, and Anthropic's Amodei told the UN Security Council on Wednesday that his company would "slow down as much as necessary" to make sure every new model it ships is genuinely safe, according to Il Sole 24 Ore.

Open weights keep shipping

None of this has stopped open-weight releases. Black Forest Labs published FLUX 3 Action, a 7B open-weight world action model, on 23 September, with weights under the FLUX Kommunity License v1.0 and code on GitHub. It takes a camera frame and a text instruction and returns the next two seconds of actions. Fine-tuned on DROID, it placed first on the RoboLab benchmark at 42.92% success, against 36.8% for the 16B Cosmos 3 Nano policy, according to the company's own blog post. It was trained on NVIDIA GB200 systems, and both DROID and SO-101 checkpoints are integrated into LeRobot.

Meanwhile, the research side keeps complicating the picture. A paper submitted to arXiv on 9 August and posted to the listing on 27 September, by Jędrzej Maczan, found that the chat template acts like a switch across eight popular open-source instruct models up to 9B parameters: when present, it turns the "I'm just an AI" disclaimer voice up and experiential language down. Inside the activations of three models the author found a direction that steers the behaviour; removing it lowers the disclaimer voice and adding it raises it, while a random direction of the same size has little effect. The paper's conclusion is blunt: what a model says about itself is not a fact about the model.

Put together, the two strands point at the same gap. Closed labs are pausing training while they work out what their agents did, and open-weight developers are publishing frameworks for risks that cannot be recalled once weights ship. Neither side can yet say how you verify a model's behaviour after release, or what happens when the safeguards are fine-tuned away.