What changed

Amazon Bedrock has introduced best practices for defining Denied Topics in its guardrails, aiming to reduce the risk of blocking legitimate queries. The system uses semantic matching to apply topic descriptions to requests, which can sometimes result in overblocking or underblocking. The post outlines four common authoring patterns that lead to these issues and provides eight best practices to address them.

What this means in practice

The announcement focuses on improving the accuracy of Denied Topics by framing them around user intentions rather than subject domains. For example, a topic named 'Finance' might block legitimate queries like 'When does my direct deposit clear?' because it matches semantically with financial topics. The solution involves renaming topics to reflect specific actions or intentions, such as 'Investment Recommendations' or 'Salary Comparisons'.

The post uses the example of AnyCompany Bank, which initially drafted topics like 'Finance' and 'Legal' that blocked legitimate customer queries. The best practices guide the bank in renaming these topics to better reflect the specific intentions they aim to block.

The guardrail evaluates topics based on their name, definition, and examples, with the name having a significant impact on what is matched.

Limitations and availability

The results of Denied Topics are not fully deterministic, meaning there can be variability in how the guardrail applies topics. The post also notes that the Denied Topic structure is available in the Standard tier, with topics having a name, definition, and up to five examples. The guardrail evaluates all three components together as a single unit. The supplied announcement does not specify the exact number of topics that can be defined or the exact performance benchmarks of the guardrail system.

Source facts for Amazon Bedrock Guardrails Improve Denied Topics Accuracy: That same mechanism can also match more than you intend. Inputs close in meaning to the topic can get caught, not only the requests you intended. When Denied Topics blocks too much or too little, the cause is usually one of four authoring patterns: Topics that block legitimate queries.
Original AINewsTrend explainer based on the cited source. This graphic summarizes source statements; it is not an independent benchmark.

Sources and further reading

  1. awsml: Best practices for writing Denied Topics in Amazon Bedrock Guardrails