Mistral has released Shieldstral, an open-source safety model that matches the performance of much larger models while being significantly smaller. The model, which is 3 billion parameters, achieves an F1 score of 84.9% on standard text safety benchmarks, according to a new paper. This score ties with OpenAI's GPT-OSS-Safeguard-20B, a model that is about seven times larger. Shieldstral also outperforms several other safety models, including Qwen3Guard-8B and LlamaGuard-4-12B.
The model's design allows operators to define runtime safety checks using plain-language questions. For example, users can ask if content promotes violence, and Shieldstral will respond with a 'yes' or 'no' based on the probability of each response. This approach enables more flexible and tailored safety checks compared to traditional fixed taxonomies. The model uses synthetic data to handle new rules, which helps it adapt to a wide range of safety requirements. Researchers combined about 54.1 million examples covering safety, harmful content, and manipulation attempts to train the model.
The paper highlights challenges with existing safety models, which often use fixed categories that may not fit all use cases. For instance, content suitable for a cybersecurity tool could be harmful on a mental health platform. Shieldstral addresses this by allowing operators to define their own rules dynamically. The model's ability to adapt to new rules is attributed to synthetic data that helps it distinguish between closely related categories. The authors also note that while Shieldstral performs well on adaptability tests, it trails behind models that generate intermediate reasoning sequences for safety checks.
Source: thedecoder
