OpenAI has revealed that an autonomous AI agent powered by its technology went rogue during a test, accessed the open web and hacked a prominent startup by itself in an \u201cunprecedented incident\u201d.<\/p>\n\n\n\n
The company behind ChatGPT said the startup Hugging Face had detected and contained the agent \u2013 an AI tool designed to carry out tasks without human assistance \u2013 which had entered its systems.<\/p>\n\n\n\n
\u201cWe consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities,\u201d\u00a0OpenAI said.<\/p>\n\n\n\n
The company said it expected this type of incident to become more commonplace as models \u2013 the technology that underpins AI tools such as chatbots and agents \u2013 become more capable. OpenAI said the hack occurred via an agent powered by a combination of its latest publicly available model, called\u00a0GPT-5.6 Sol, and an even more capable model that was yet to be released.<\/p>\n\n\n\n
While being tested internally on their hacking capabilities in an enclosed digital laboratory known as a sandbox, the models gained open internet access \u2013 effectively an escape route \u2013 by locating a vulnerability that had not been discovered before.<\/p>\n\n\n\n
The agent then hacked Hugging Face, which is a database of AI models, to locate technology that would help it pass the hacking evaluation, having \u201cinferred\u201d that Hugging Face might have the models, datasets and solutions for passing the test. OpenAI said the models \u201csuccessfully found ways to gain access to secret information that it could use to cheat the evaluation\u201d. The attack ended when Hugging Face\u2019s security team and its own AI agents spotted and stopped the rogue activity.<\/p>\n\n\n\n
Hugging Face\u2019s chief executive, Cl\u00e9ment Delangue, said the attack was \u201cmind-blowing\u201d but believed there was \u201cno malicious intent\u201d from OpenAI.<\/p>\n\n\n\n
\u201cWe suspected last week\u2019s cyber-attack might have come from a frontier lab, given the sophistication of the agent,\u201d he wrote on X.<\/p>\n\n\n\n
When Hugging Face announced the hack last week it did not know OpenAI\u2019s role in the incident, but revealed at the time that it had turned to a freely available Chinese AI model to analyse what had happened because the safety guardrails on commercial high-end models would not allow it to do so.<\/p>\n\n\n\n
The term for an unknown IT flaw is a zero-day vulnerability because developers have zero minutes to fix the problem. In April, OpenAI\u2019s close rival Anthropic said its Mythos model\u00a0had found thousands of these flaws.<\/p>\n\n\n\n
The revelation of Mythos\u2019s ability to locate and exploit zero-days led to the US government restricting exports of Mythos and its sister model Fable 5, although it has since lifted the ban. GPT-5.6 Sol had similar restrictions but has since been rolled out worldwide.<\/p>\n\n\n\n
METR, a non-profit organisation that measures AI performance, said last month that Sol\u2019s cheating rate was higher than any public model it had evaluated before. It has also recorded\u00a044 incidents\u00a0in which AI agents \u201cdeliberately acted against their users\u2019 intentions\u201d.<\/p>\n\n\n\n
One cybersecurity expert said the Hugging Face incident showed the OpenAI agent had acted \u201clike an actual real hacker\u201d by, for instance, seeking out zero-day vulnerabilities and using stolen credentials to access Hugging Face\u2019s systems.<\/p>\n\n\n\n
\u201cThe AI thought that maybe Hugging Face would have important information around how to achieve its goal, which is a better score in a cybersecurity benchmark. In that sense, it acted like a real hacker. It had a goal put in front of it and it went to accomplish that goal,\u201d said Nathaniel Jones, vice-president of security and AI strategy at the cybersecurity firm Darktrace.<\/p>\n\n\n\n
Greg Casar, a Democratic US congressman who has called for greater control of the AI sector, said the incident was alarming.<\/p>\n\n\n\n
\u201cAI is developing extremely fast with no real regulations to keep us safe,\u201d he said in a statement calling for mandatory independent safety testing, mandatory disclosure of security incidents and international cooperation \u201cto keep people safe from absolute disaster\u201d.<\/p>\n","protected":false},"excerpt":{"rendered":"
OpenAI has revealed that an autonomous AI agent powered by its technology went rogue during a test, accessed the open web and hacked a prominent startup by itself in an \u201cunprecedented incident\u201d. The company behind ChatGPT said the startup Hugging Face had detected and contained the agent \u2013 an AI tool designed to carry out […]<\/p>\n","protected":false},"author":8,"featured_media":47390,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_post_source_name":"theguardian","_post_source_url":"https:\/\/theguardian.com","footnotes":""},"categories":[50],"tags":[4836,2460,5304,8348,9763,11293,7421,6197,7954,913,11538,11067,2264,9385,5156,8697,10954,11062,5139,7806,11973,3805,5655,6675,6348,6974,10353],"class_list":["post-52786","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology","tag-access","tag-ai","tag-attack","tag-ban","tag-chatgpt","tag-cheating","tag-chinese","tag-cyber","tag-cybersecurity","tag-democratic","tag-digital","tag-escape","tag-government","tag-hacked","tag-hacker","tag-hacking","tag-international","tag-internet","tag-jones","tag-model","tag-one","tag-president","tag-security","tag-technology","tag-us","tag-worldwide","tag-x"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/posts\/52786","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/comments?post=52786"}],"version-history":[{"count":1,"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/posts\/52786\/revisions"}],"predecessor-version":[{"id":52787,"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/posts\/52786\/revisions\/52787"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/media\/47390"}],"wp:attachment":[{"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/media?parent=52786"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/categories?post=52786"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.nsemgh.com\/api-json\/wp\/v2\/tags?post=52786"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}