OpenAI has developed GPT-Red, an automated red-teaming model designed to enhance the robustness of its large language models. The model is trained to identify vulnerabilities and improve model safety before wider deployment. GPT-Red is used to adversarially train GPT-5.6, making it significantly more resistant to prompt injection attacks. This approach allows OpenAI to scale its safety efforts alongside model capabilities. The model is trained at the compute scale of some of OpenAI's largest post-training runs, reflecting a substantial investment in improving model safety. Source: openai
GPT-Red is trained using self-play reinforcement learning, where the model and a collection of diverse defender LLMs are trained simultaneously on a broad set of red-teaming scenarios. GPT-Red is rewarded for eliciting a valid failure, such as a successful prompt injection, while the defender models are rewarded for resisting the attack and completing their original tasks. As the defenders become more robust, GPT-Red is forced to discover stronger and more diverse attacks. This training method enables the model to develop advanced capabilities in identifying and exploiting vulnerabilities. Source: openai
AI systems commonly encounter third-party data through browsers, connected apps, local files, and other tools. These affordances are necessary for performing real-world tasks, but they also create more opportunities for malicious actors to influence model behavior. Human red-teaming is a critical part of OpenAI's safety work, helping to uncover vulnerabilities before deployment and put the right safeguards in place. However, human red-teaming alone is difficult to scale, limiting how quickly new failure modes can be identified and incorporated into stronger safeguards. Source: openai
