What changed
Amazon Bedrock has introduced prompt caching for OpenAI models, allowing users to store and reuse input prompts across multiple requests. This feature helps reduce costs by avoiding the need to re-send the same context repeatedly.
What this means in practice
Prompt caching works by storing the input prompt, including instructions, tools, and content, and reusing it on subsequent requests. This means that the model does not have to re-read the entire history of the conversation, which can significantly lower costs. The service can either implicitly select the reusable prefix or explicitly mark it using a prompt_cache_breakpoint. Users can combine both approaches for optimal results.
Limitations and availability
Prompt caching is available for OpenAI models on Amazon Bedrock. The feature requires that the exact same text is passed to the model for caching to work. Users must ensure that the agent’s instructions and tool lists are serialized consistently. The prompt_cache_key is used to identify which requests should share a cache. The service also has a time-to-live (TTL) setting, which defaults to 30 minutes. The supplied announcement does not specify the exact cost savings or performance improvements, but it highlights the reduction in token processing and the potential for more efficient agent workflows.
