What changed
Amazon Bedrock has introduced best practices for defining Denied Topics in its guardrails, aiming to reduce the risk of blocking legitimate queries. The system uses semantic matching to apply topic descriptions to requests, which can sometimes result in overblocking or underblocking. The post outlines four common authoring patterns that lead to these issues and provides eight best practices to address them.
What this means in practice
The announcement focuses on improving the accuracy of Denied Topics by framing them around user intentions rather than subject domains. For example, a topic named 'Finance' might block legitimate queries like 'When does my direct deposit clear?' because it matches semantically with financial topics. The solution involves renaming topics to reflect specific actions or intentions, such as 'Investment Recommendations' or 'Salary Comparisons'.
The post uses the example of AnyCompany Bank, which initially drafted topics like 'Finance' and 'Legal' that blocked legitimate customer queries. The best practices guide the bank in renaming these topics to better reflect the specific intentions they aim to block.
The guardrail evaluates topics based on their name, definition, and examples, with the name having a significant impact on what is matched.
Limitations and availability
The results of Denied Topics are not fully deterministic, meaning there can be variability in how the guardrail applies topics. The post also notes that the Denied Topic structure is available in the Standard tier, with topics having a name, definition, and up to five examples. The guardrail evaluates all three components together as a single unit. The supplied announcement does not specify the exact number of topics that can be defined or the exact performance benchmarks of the guardrail system.
