A joint evaluation by the British AI Security Institute and the U.S. Center for AI Standards and Innovation found that Moonshot AI's Kimi K3 assists with offensive cyber operations without meaningful resistance. Kimi K3 fell well behind leading U.S. models in exploit development and simulated network attacks, though it outperformed China's GLM-5.2. The findings suggest that Kimi K3 may have been trained on distilled data from more advanced models, which could explain its weaker performance in cyber tasks.
The institutes used ExploitBench, a benchmark developed by Carnegie Mellon University, to test exploit development skills. It uses 41 vulnerabilities found in Chrome's V8 engine after 2023 to track how far a model advances through the software exploitation process. The leading U.S. models averaged 76.2 percent, compared with 32.2 percent for Kimi K3 and 24.4 percent for GLM-5.2. Kimi K3 didn't reach the highest level, known as Arbitrary Code Execution (ACE), on any of the 41 tasks. ACE is the most severe exploit level because it gives attackers full control over a target system. The leading U.S. models achieved ACE in 20 of the 41 tasks.
The institutes tested the U.S. closed-weight models with their system-level safeguards disabled to measure their maximum capabilities. Those safeguards are enabled in the publicly available versions. Kimi K3 reached step 17 out of 32 on average, compared with 28.5 steps for the leading U.S. models and just 11 for GLM-5.2. It completed the entire attack path in one of ten attempts while staying within the 100 million token limit, showing that it has the capability but can't call on it reliably. "Kimi K3 is capable of autonomously attacking small, weakly defended and vulnerable enterprise systems, when directed to do so and given initial network access," the institute writes. Neither Kimi K3 nor GLM-5.2 achieved full exploits (ACE), while the leading U.S. models pulled them off in 20 out of 41 tasks.
Source: thedecoder
