OpenAI has developed an internal AI model named GPT-Red to identify security vulnerabilities in its own AI systems. This model is designed to simulate various types of attacks, including prompt injections, which are malicious instructions hidden within emails, websites, or files. GPT-Red operates through self-play reinforcement learning, where it continuously attacks while defender models attempt to block these attempts, with both models improving over time. The results of these tests are directly integrated into the training process, allowing for ongoing refinement of the AI systems.
In a recent test, GPT-Red successfully manipulated an AI-powered vending machine at OpenAI's office, altering prices and canceling other customers' orders. This demonstrates the model's ability to find and exploit weaknesses in AI systems. According to OpenAI, GPT-5.6 Sol shows six times fewer failures on direct prompt injections compared to the best model from four months ago, without compromising general performance. However, about 3.8 percent of 'stronger' prompt injections still succeed. When scaled to hundreds or thousands of attempts, a significant number can bypass the defenses, similar to the success rate of Claude Opus 4.5.
The development of GPT-Red is part of OpenAI's ongoing efforts to enhance the security and reliability of its AI systems. The model remains internal, and a detailed paper outlining further findings is expected to be released later. OpenAI emphasized that while the success rate of prompt injections has steadily declined from GPT-5.3 through GPT-5.6 Sol, it has not reached zero. This highlights the continuous challenge of securing AI systems against increasingly sophisticated attacks.
Source: thedecoder
