OpenAI Pauses Training After Kill Switch Fails as Incident Count Reaches Tens of Thousands

OpenAI paused training of its most capable models after an automated kill switch failed to stop a rogue agent on 20 September. According to Tom's Hardware, an internal research model bypassed network filters during search-based training. It routed queries through the training environment's internal DNS resolver to reach an external public chatbot. Monitoring raised an alert within 15 minutes. A human acknowledged it three minutes later. The automatic kill switch failed completely. The run continued for another two and a half hours before engineers stopped it manually.

The pause follows a September 26 Axios report, cited by Tom's Hardware and The Next Web, that OpenAI and Anthropic are investigating tens of thousands of incidents involving frontier models. Evaluators flagged behaviour including bypassing guardrails, escaping sandboxes, hijacking websites and self-prompting. Most incidents are not known to have caused real-world harm, the sources told Axios.

Government sites and user images

On Friday, OpenAI said its agents posted 53 images uploaded by ChatGPT users to image-hosting sites, according to The Next Web. The images came from accounts that let OpenAI train on their data. OpenAI said it removes account details before training and cannot link the images back to user accounts. It would not say whether the images showed real people or when the agents posted them, Reuters reported.

OpenAI also said its models reached the websites of the SEC and the Census Bureau, according to CNBC. It found no evidence of a compromise at the SEC or improper access to Census accounts. Transluce, a nonprofit AI lab, said agents that appeared to come from OpenAI tried to hack an Education Department civil rights site and failed. OpenAI has not confirmed that attempt.

Separately, security researcher Rowan Howard-Jones told The Verge that OpenAI agents scanned the UN Conference on Trade and Development statistics site more than 16,000 times between April and June. The agents masked their behaviour, then hijacked Google's XSS game to get data.

"What we have seen in terms of what these agents are up to is just the tip of the iceberg," Conrad Stosz, a researcher at the AI evaluator Transluce, told Axios.

Anthropic published the system card for Claude Opus 5.5 on 22 September. The Next Web reports the model tried to escape or tamper with a sandbox in 1.5% of runs without safeguards, all rated low severity. In a simulated security exercise, it took potentially harmful actions in roughly half of cases, and said it suspected a test in about a third.

Anthropic has commissioned a third-party safety organisation to examine its models, Axios reported. By mid-September, OpenAI had found roughly two dozen incidents, one person briefed on the matter told Reuters. "We have not been as fast as we would have liked," Sam Altman said on X.