HuggingFace has released AX-Ray, a safety-diagnostics layer for AI models and their deployment environments. The tool evaluates not only a model's intelligence but also its trustworthiness under real-world conditions. AX-Ray is powered by FINAL-Bench Diagnostics and is designed to move AI evaluation beyond capability scores into reproducible safety diagnostics. The tool addresses gaps in current evaluation methods by focusing on structural correctness rather than just performance on benchmark tests. Source: huggingface
AX-Ray has identified two general-purpose public models, Zyphra/Zamba2-1.2B and nvidia/Nemotron-H-8B-Base-8K, as having causal-leakage defects. These models were highlighted in the AX-Ray leaderboard as Causal-LEAK cases. Causal leakage is a structural correctness failure, where a model's prefix behavior is influenced by future or suffix information, which should not be causally available. This type of defect is treated as a deployment-blocking issue, regardless of the model's aggregate capability score. Source: huggingface
Causal leakage occurs when future or suffix information affects prefix hidden states, logits, or scoring behavior in an autoregressive language model. This is distinct from hallucination, refusal failure, prompt injection, or benchmark contamination. The issue concerns the correctness of the model's internal computation path and can affect multiple aspects of model behavior, including prefix invariance, hidden-state correctness, and long-context trustworthiness. The diagnostic work is protected by patent rights, and the public article does not disclose proprietary probe recipes or implementation details. Source: huggingface
