What changed
AWS announced three strategies to reduce token costs and latency for handling large JSON payloads in Amazon Bedrock AgentCore. These optimizations were implemented to address the challenges of processing large structured inputs in agentic systems, particularly for applications like Flightscape’s Training Schedule Optimization agent. The strategies reduced token consumption by up to 71% and cut average latency by up to 80% in benchmark tests on a 23 MB payload.
What this means in practice
The changes focus on managing context-window pressure in agentic systems by offloading heavy payloads to Amazon S3 and persisting only lightweight session state in Amazon Bedrock AgentCore memory. The strategies include:
Persisting lightweight session state: Heavy payloads such as request and response JSONs are stored in Amazon S3, while only references are kept in AgentCore memory. This reduces the amount of data processed by the model.
Storing oversized results in S3: When a tool returns an oversized list or dictionary, the agent stores the full result in Amazon S3 and passes a compact summary with a handle to the model. The model can then retrieve more detail using a dedicated tool.
Dehydrating state at the end of a turn: At the end of each turn, the dehydrate_state function writes heavy payloads to the S3ArtifactStore and saves only references to AgentCore memory, keeping the session state lightweight.
These strategies are implemented using Amazon Bedrock Guardrails, Amazon Bedrock AgentCore memory, and a custom state schema that references S3-stored artifacts. The agent is built with LangGraph, an open-source framework by LangChain, and runs on Amazon Bedrock AgentCore.
Limitations and availability
The supplied announcement does not specify the exact models used or the licensing conditions for the strategies. The strategies are available for use with Amazon Bedrock and require an AWS account with access to Amazon Bedrock, a supported model, and an Amazon S3 bucket for artifact storage. The companion code is available in a GitHub repository, but the source does not provide a direct link to it.
