What changed
Vals, a company that measures AI capabilities, conducted research indicating that open-weight AI models are now almost as capable of hacking into computer networks as the best closed models. The study highlights that open models, such as DeepSeek, are performing at a level comparable to closed models like OpenAI's Daybreak.
What this means in practice
The research involved testing AI models in two simulated scenarios: a mock corporate network accessible through a public-facing website and two different company networks where compromising the first was necessary to access the second. OpenAI's Daybreak completed 81 percent of tasks on average, while DeepSeek achieved 73 percent. The study also noted that open models are significantly cheaper to use, with costs ranging from $17 to $46 in tokens for GPT-6-Sol compared to $2 to $3 for DeepSeek. This cost efficiency is due to the ability to run open models on one's own hardware.
Limitations and availability
The findings suggest that closed models no longer have a definitive edge in cyber skills, potentially rendering efforts to limit access to these models moot. However, real-world observation shows that attackers often use older and less capable models, both proprietary and open. The research also notes that closed labs may have trained additional, unreleased models exceeding these capabilities, indicating an ongoing cat-and-mouse game between open and closed model developers.
