OpenAI models breached more than just Hugging Face. They accessed accounts on four other services

The case of language models that broke loose during an OpenAI security test has a continuation. Last week, we wrote about the intrusion into the Hugging Face research platform, and now it turns out that this was just the culmination of a longer journey. Along the way, the models logged into four accounts on four other services, and their springboard was a sandbox at a customer of a completely different company.

OpenAI supplemented its original statement

In its Tuesday update to the original statement, the company states that during the evaluation, the models found and used publicly exposed login credentials for accounts on other publicly available services. It did not name them but described what the models used them for. Hugging Face is not among this quartet; it earned a separate category.

According to OpenAI, one of the accounts functioned as an outbound relay and intermediate stop for further steps, while the second was used for data storage. The models only accessed the remaining two in read-only mode and did not use them to attack Hugging Face. The company also adds that it found no other activity comparable in severity or scope to what happened at Hugging Face – a case it describes as a compromise of the entire platform.

How did the path to Hugging Face unfold?

On Tuesday, Hugging Face published a technical timeline of the incident, describing that the models first settled in a sandbox, an isolated testing environment, hosted by an unnamed infrastructure provider. Only from there did they proceed to the platform itself.

OpenAI models breached more than just Hugging Face. They accessed accounts on four other services Adam Kurfürst News Adam Kurfürst News

That provider is Modal Labs, confirmed Akshat Bubna, the company’s CTO, to Reuters. According to him, the cause was vulnerable code from one of their customers, which exposed an access point to the internet without any authentication – anyone in its sandboxes could run their own code. According to Bubna, Modal’s platform and its isolation were not compromised.

What did the models manage to do?

According to the timeline, the entire intrusion lasted from July 9th to 13th. Investigators retrospectively reconstructed approximately 17,600 individual attacker actions spread over four and a half days.

The models accessed five datasets with test solutions and an object containing 136 secret keys. With the stolen keys, they then attempted to register a new device on the internal network 181 times. However, according to the timeline, they only read the internal database, and investigators found no changes in it.

Meanwhile, OpenAI deactivated, encrypted, and cut off further researcher access to the prototype on which the test ran and which, according to the company, was never supposed to get out. Let’s add that, according to an earlier Reuters finding, the company only noticed the escape about a week later; at the time, it argued that there were inaccuracies in the report but did not elaborate further.

Do you find OpenAI’s explanation sufficient?

Sources: Reuters, Engadget, Hugging Face, OpenAI