Amazon has introduced Nova Act, a new tool designed to enhance user experience (UX) testing by analyzing web interfaces through vision and action. Traditional UX testing methods often struggle with scalability and adaptability, especially when dealing with dynamic content and evolving user journeys. Nova Act addresses these challenges by mimicking human reasoning when navigating interfaces, allowing for more comprehensive and efficient testing processes. This model processes visual information to understand page layouts and identify interactive elements, making it adaptable to interface changes that would typically break traditional automation tools like Selenium or Playwright. The solution also incorporates generative AI to enable parallel execution of user flow testing at scale, offering a new approach to UX analysis and improvement. Source: awsml
Nova Act operates through a multi-layered architecture that supports documentation processing, orchestration, execution, and analysis. The documentation processing layer uses Amazon Simple Storage Service (Amazon S3) to store user guides and flow specifications, which are then ingested into an Amazon Bedrock Knowledge Base for semantic similarity searches. AWS Lambda, integrated with Claude 4.5 Sonnet, transforms these inputs into detailed testing scenarios. This system generates step-by-step interaction paths for Nova Act to execute, allowing for comprehensive testing of user flows across various device types and interaction patterns. The orchestration layer manages test execution at scale using Amazon DynamoDB and AWS Lambda, while the execution layer leverages Amazon ECS with AWS Fargate for parallel test runs. The analysis layer processes results using Amazon Bedrock to identify usability scores and friction points, providing actionable insights for UX improvements. Source: awsml
The solution also provides a setup guide for deployment, requiring Node.js v20 or newer, npm v10.8 or newer, an AWS account, and the AWS Cloud Development Kit (AWS CDK). The deployment process involves cloning a GitHub repository, configuring the environment, and running a deployment script. After deployment, users can create test flows using automatic generation from documentation, manual definition, or a hybrid approach. The hybrid method is recommended for establishing baseline coverage from existing documentation and supplementing with manually defined flows for specific test cases or edge scenarios. Source: awsml
