Amazon Bedrock has introduced automatic policy refinement to streamline the process of improving Automated Reasoning policies. Previously, refining these policies required manual diagnosis, editing, and retesting. The new feature automates the diagnosis and correction of failing tests, allowing users to approve changes before they take effect. This enhancement is designed to reduce the friction in policy development, which customers previously identified as the biggest challenge. Automated Reasoning checks in Amazon Bedrock Guardrails use formal verification to ensure answer correctness by translating natural language into formal logic. The system delivers up to 99% verification accuracy, as reported in the GA announcement. Testing a policy involves attaching input/output text with expected results, enabling users to identify where the policy diverges from their intent. Automated Reasoning checks surface two distinct failure signals that map to the two-step validation pipeline: translation and validation. The refinement engine diagnoses failing tests and proposes formal-logic fixes. Customers told us that this iterative tuning creates the biggest friction point in policy development. In this post, we walk through two new refinement modes: Iterative Refinement for rule issues, and Ambiguous Variable Refinement for language issues. For each mode, we show a complete API workflow (start, poll, retrieve) and a repeatable console workflow for turning failing policies into passing ones. The refinement engine analyzes how the current rules diverge from the source document and your feedback. It proposes a set of candidate changes (new rules, edited rules, added variables) that bring the policy in line. The convergence loop iterates internally, generating a candidate change, simulating its effect on your saved tests, and checking whether the previously failing tests now pass. The iteration happens internally, though: you don’t observe each intermediate attempt, and you don’t need to shepherd it. What you receive is the converged result: a proposed diff that shows exactly which rules changed, which variables changed, and how the change affects every test in your suite. The review gate appears after convergence, allowing you to select Accept changes or Discard changes. Accepting writes the changes to your DRAFT policy. Discarding leaves everything exactly as it was. Iterative Refinement requires at least one test attached to your policy. Without a failing test signal, there’s nothing to drive the refinement. Use this mode when the translation is correct (right variables, right values) but the validation result is unexpected. Do not use it when the finding is TRANSLATION_AMBIGUOUS. That’s a language problem better addressed by Ambiguous Variable Refinement. Through the API, refinement runs as an asynchronous build workflow. Using the AWS SDK for Python (Boto3), the flow has four steps: export the current policy definition, start the workflow, poll for completion, and retrieve
Source: awsml
