AI start-up Anthropic launches bug reporting scheme
Artificial intelligence startup Anthropic launched a vulnerability disclosure program (VDP), managed by HackerOne, in August with bounty rewards up to $15,000 for novel, universal jailbreak attacks that could expose vulnerabilities in critical, high-risk domains such as CBRN (chemical, biological, radiological, and nuclear) and cybersecurity.
A jailbreak attack in AI involves a method for circumventing an AI system’s built-in safety measures and ethical guidelines, allowing a user to elicit responses or behaviours from the AI system that would normally get blocked.
“As we work on developing the next generation of our AI safeguarding systems, we’re expanding our bug bounty program to introduce a new initiative focused on finding flaws in the mitigations we use to prevent misuse of our models,” Anthropic said in a blog post on the revamped program.