AWS has introduced a new observability solution to monitor AI agents running across on-premises and multi-cloud environments. The solution, called AgentCore Observability, supports frameworks such as Strands Agents, LangGraph, and CrewAI, allowing users to track agent performance and behavior regardless of where they are deployed. This capability is part of the broader Amazon Bedrock AgentCore platform, which enables the building, connecting, and optimizing of AI agents at scale.
To enable observability for agents running outside AWS, users must configure the AWS Distro for OpenTelemetry (ADOT) for auto-instrumentation in non-AWS environments. This setup allows telemetry data such as traces, metrics, and logs to be sent to the AgentCore Observability dashboard. ADOT handles the complexities of exporting telemetry from non-AWS environments and uses SigV4 authentication with AWS Identity and Access Management (IAM) credentials to securely route the data. The solution also relies on environment variables to specify OpenTelemetry settings for routing and authentication.
The observability solution integrates several AWS services, including Amazon CloudWatch for telemetry ingestion and storage, Amazon Bedrock AgentCore Observability for specialized monitoring dashboards, and IAM for securing authentication between external environments and AWS. This observability is essential for responsible AI practices, as it provides visibility into agent reasoning chains, tool invocations, and model outputs, enabling users to detect hallucinations, monitor for harmful responses, and track token usage for cost governance.
Source: awsml
