What changed
Hugging Face has launched a four-season AI model hacking hackathon aimed at improving the safety of AI agents by identifying and addressing vulnerabilities. The initiative, titled 'FINAL-Bench,' invites participants to test AI models and find ways to bypass safety mechanisms, with the goal of making AI safer for everyone.
What this means in practice
The hackathon is designed to simulate real-world scenarios where AI agents might be exploited. Participants will engage in four distinct challenges, each focusing on a different type of vulnerability: agent hijack, data exfiltration, hidden instruction injection, and kernel breach. Each season has its own set of rules, difficulty levels, and prizes. The top score prize and a special award of USD 500 are available in each season, with the total prize pool amounting to USD 4,500.
Participants can join using their Hugging Face accounts and will be provided with mock tools for testing, ensuring that no real data is exposed. The hackathon is open to anyone with a browser, requiring no coding or GPU access.
The initiative is part of Hugging Face's effort to foster a collaborative approach to AI safety, leveraging the collective intelligence of the community to strengthen security measures.
Limitations and availability
The hackathon is available to anyone with a Hugging Face account and a browser, with no coding or GPU requirements. Seasons 1, 2, and 3 are currently open and run concurrently for 60 days, while Season 4 will start after the closing of the earlier seasons. The hackathon is hosted on Hugging Face, making it accessible to the open AI community. The initiative does not specify any hardware requirements beyond a browser and a Hugging Face account.
