What changed

Amazon Bedrock now supports explicit prompt caching for open-weight models. This feature allows users to mark reusable prefixes in their prompts, enabling the system to reuse previously computed states and reduce input costs and response latency. The change is part of an update that includes both implicit and explicit caching modes.

What this means in practice

Open-weight models on Amazon Bedrock can now use explicit prompt caching, which lets users mark the end of each reusable prefix with a cache breakpoint. This allows for more control over which parts of the prompt are cached and reused. The system automatically manages the cache, and users can read cache usage from each response. The feature is available through the Chat Completions API, and users can apply it to scenarios like coding agent sessions.

Limitations and availability

The availability of explicit prompt caching depends on the model and the API. Users should check the model card for details on supported caching modes, minimum prefix length, and how long entries stay cached. The feature is available in regions where the model is supported, and users need an AWS account with access to the selected model. The feature is not available for all models and requires specific setup steps, including installing Python packages and configuring the environment.

Source facts for Amazon Bedrock now supports explicit prompt caching for open-weight models: That context can be a system prompt, reference documents, tool definitions, or the conversation so far. Open-weight models on Amazon Bedrock can reuse it across requests, which helps reduce input costs and response latency. Prompt caching reuses the computed state of a matching prompt prefix, so repeated context is read from cache instead of being processed again.
Original AINewsTrend explainer based on the cited source. This graphic summarizes source statements; it is not an independent benchmark.

Sources and further reading

  1. awsml: Use prompt caching with open-weight models on Amazon Bedrock