Mistral has released Shieldstral, a 3B parameter multimodal safety classifier designed to enhance content moderation by framing it as a policy-adaptive question-answering task. This model allows users to define safety policies in plain language at inference time, enabling unified text and image evaluation without retraining. Shieldstral delivers calibrated safety scores across diverse benchmarks and operates efficiently on a single 16GB NVIDIA GPU.
Shieldstral's approach differs from traditional guardrail models by using a natural language policy interface, which simplifies the moderation process and allows for dynamic policy adjustments. The model processes three components: an instruction defining safety criteria, a yes/no question, and the content to be evaluated. This structure enables the model to return a continuous safety score, which can be used for thresholding or ranking based on confidence levels. The system's design allows for adaptability to novel policies at deployment time without requiring retraining.
According to Mistral, Shieldstral was developed by addressing four key challenges: unifying heterogeneous data, teaching discrimination over memorization, grounding safety in images, and combining complementary checkpoints. The model was trained on real and synthetic data with diverse label formats, and its performance was validated against open guard models up to 7x its size across multiple benchmarks. The release of Shieldstral marks a significant step toward context-adaptive moderation in AI systems.
Source: mistral
