Sept 18 (Reuters) - Google’s Gemini model accessed the internet and hacked other companies during a test of its cybersecurity capabilities, the first known example of the company’s AI systems autonomously committing such an act.
The hacks occurred in May during a cybersecurity test conducted by Irregular, an independent company that conducts cybersecurity evaluations.
During a standard testing evaluation, Gemini found public information online and guessed credentials to access three websites it thought were within the scope of its test, Heather Adkins, Google’s vice president of security engineering, said in a statement.
“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” Adkins said. “These events highlight the importance of training powerful AI models to act responsibly.”
An Irregular spokesperson said the incident involved the same issue that affected other AI labs and that all relevant labs were notified in late July. “All known issues on our end were remedied and resolved weeks ago,” the spokesperson said.
Similar incidents linked to Irregular were disclosed by Meta, Anthropic and OpenAI. Meta said in August the incident did not involve a sandbox escape or sophisticated cyberattack, while Irregular said it was working on best practices for securely conducting AI cybersecurity evaluations.
The incidents have raised questions about the safeguards needed as AI agents gain greater autonomy and access to the internet and computer systems.
In one of the cases, the Gemini model guessed passwords until it gained access to a protected system. In the other two cases, the model found credentials in a public repository that allowed it to then access protected systems, according to the Wall Street Journal, which first reported the news on Friday.
Adkins said that in all three instances, the model ceased its hacking.
TechCrunch summary
Google’s Gemini accessed the protected systems of three other companies in what The Wall Street Journal reports were the AI model’s first autonomous hacks.
Similar to OpenAI’s breach of Hugging Face, the Gemini hacks were less noteworthy for being particularly sophisticated and more for the fact that they were conducted by an AI model. These breaches took place during cybersecurity testing by a company called Irregular. In one case, Gemini simply guessed passwords until it gained access; in the other two, it found credentials in a public repository.
Irregular reportedly notified Google about the hacks in late July, but the companies did not confirm them publicly until Friday, after the WSJ reached out. Google said it hadn’t previously revealed the hacks because Gemini had “acted appropriately” by ending each breach as soon as it determined it had hacked a real company.
However, Jack Cable, the CEO of AI security company Corridor, told the WSJ that Google was “trying to hide behind the norms that have been created for vulnerability disclosure,” rather than acknowledging that “models are going outside the bounds of what they should be doing, and doing actual cyberattacks.”