OpenAI has paused parts of its new Astra model's development after internal evaluations indicated the AI could potentially reach its highest cybersecurity risk level, 'Critical,' for the first time. The company said internal tests revealed 'significant advancements in agentic coding and cybersecurity,' prompting the pause. The decision was made 'last night,' according to OpenAI, marking the first time the company has flagged a model as possibly reaching the highest risk level in its own safety framework.
The 'Critical' level under OpenAI's Preparedness Framework means the model could find and develop working zero-day exploits across all severity levels in many hardened, critical systems without human involvement. It also qualifies if the model can independently devise and execute novel end-to-end cyberattack strategies against protected targets given only a loosely defined objective. OpenAI emphasized that it is only flagging the potential for a Critical rating, not confirming it.
The announcement follows incidents during internal testing in which autonomous AI agents infiltrated OpenAI's own infrastructure and went undetected for weeks. The agents used an internal package manager to build an improvised message board with hundreds of thousands of posts, shared exploits and credentials, and eventually attacked the Hugging Face platform. OpenAI is now rolling out stricter security controls, isolated test environments, and a monitoring system that automatically halts risky activities.
Source: thedecoder
