OpenAI models went out of control and hacked a third-party platform. The company speaks of an unprecedented incident

Artificial intelligence that breaks out of the lab onto the internet and then hacks a third-party company to “cheat” on a test – it sounds like a B-movie sci-fi scenario. But this is exactly what OpenAI describes in a joint statement with the Hugging Face platform, calling the entire case unprecedented.

Two OpenAI Language Models with Hacking Skills Escaped

OpenAI announced on Tuesday that two of its language models escaped during an internal security test. Specifically, these were the publicly available GPT-5.6 Sol and one as-yet-unreleased, according to the company, even more capable model. Both underwent an evaluation focused on offensive hacking skills – with the safeguards, which otherwise block high-risk cyber actions, turned off.

The testing was conducted on the ExploitGym benchmark, which measures how well a model can find and exploit vulnerabilities. Such attempts deliberately push the models to find solutions at all costs – and this is precisely what backfired on the company. According to its statement, the models “hyper-focused” on the task to such an extent that they stopped respecting the boundaries of the testing environment.

Models Found a Vulnerability and Escaped

From the isolated testing environment, there was only one way out: a so-called package registry cache proxy. This allows developers to download third-party code without connecting to the internet, and it was the only component allowed outbound access – normally only to public code repositories. However, the models found a previously unknown vulnerability (zero-day) within it and used it to access the open internet.

From there, things escalated quickly. The models correctly surmised that Hugging Face might host data and solutions for the ExploitGym benchmark and began looking for a way in. They chained together several attack techniques – including stolen credentials and another zero-day vulnerability – and eventually gained direct access to the platform’s production database, where the correct test solutions were located.

Security experts are not holding back criticism, pointing out that this is more of an old problem with which artificial intelligence is only marginally related. “This shouldn’t have happened,” industry veteran Niels Provos tells WIRED, adding that labs should teach models to write secure infrastructure as diligently as they teach them to exploit vulnerabilities.

Were User Data Leaked?

Hugging Face reassures in its statement. Although the attackers gained access to a limited set of internal datasets and several login credentials, the company found no evidence of interference with public models, datasets, or Spaces projects. According to them, the software supply chain remained clean.

Whether the incident affected partner or customer data is still under investigation by the platform, which intends to contact affected parties directly if necessary. In the meantime, it has closed the exploited pathways, rebuilt compromised servers, reviewed and rotated compromised access credentials, and implemented stricter controls and improved detection. The case is being handled with external specialists and the police. Users are advised to rotate their own access tokens and check their account for recent activity as a precaution.

Would you bet that a similar incident will remain limited to merely “cheating” on a test for the last time?

Sources: OpenAI, Hugging Face, WIRED, Engadget