AI Used to Hack AI; How 3 Indians Found Security Flaws in OpenAI

Artificial intelligence is becoming a powerful tool in almost every field. But a recent cybersecurity experiment has shown that even advanced AI systems can be connected to serious security risks.

Three Indian-origin cybersecurity researchers reportedly used Anthropic’s Claude AI to help test security weaknesses in OpenAI’s systems. The researchers were not carrying out an illegal attack. They were testing OpenAI’s security as part of its bug-bounty programme.

The three researchers are Harsh Jaiswal, Mohan Pedhapati and Rahul Maini. They are associated with cybersecurity company Hacktron AI and have experience in finding software vulnerabilities.

According to the report, the team discovered a weakness in OpenAI’s public community forum. The flaw was related to the way certain image files were handled. This weakness could potentially allow code to be run on the forum’s server.

The researchers then used Claude to help them understand the vulnerability and develop an exploit. The AI assisted with tasks such as writing, debugging and changing code during the security test.

The researchers later discovered another weakness involving login tokens. This gave them a possible route to access employee accounts. They eventually found a way into OpenAI’s private GitHub environment through an employee’s Codex account.

However, the researchers themselves connected the different security flaws and decided how to use the access they obtained. Claude was used as a tool during the process, rather than independently carrying out the entire operation.

The complete security experiment reportedly took less than 72 hours. The researchers spent less than $3,000 on AI tokens during the testing process. OpenAI later fixed the vulnerabilities and reportedly paid the team a $6,500 bug bounty.

Mohan Pedhapati is the CTO and co-founder of Hacktron AI. Harsh Jaiswal has more than a decade of experience in vulnerability research and has previously worked with organisations including Project Discovery, Zomato and Cure53. Rahul Maini has also worked in the bug-bounty and penetration-testing field.