What changed

AssemblyAI implemented a cost-optimized speech-to-text inference solution on AWS, using Amazon EC2 Spot Instances to achieve 70% cost savings compared to On-Demand pricing. The solution maintains service level agreements (SLAs) and supports data residency requirements for regulated industries.

What this means in practice

AssemblyAI's architecture uses Amazon ECS, EC2 Spot Instances, and Amazon SQS to manage GPU-intensive inference workloads. The system handles interruptions by returning in-flight jobs to an SQS queue, allowing other healthy instances to complete them. This approach ensures minimal impact on end users, with only a small increase in processing time. The solution is deployed across European AWS Regions to meet data residency requirements for healthcare, financial services, and legal industries.

Limitations and availability

The solution is available on AWS Marketplace and requires Amazon EC2 Spot Instances across multiple GPU families, including g4dn, g5, and g6. It supports inference for speech-to-text, speaker detection, sentiment analysis, and PII redaction. The architecture is designed for high-volume workloads and is currently deployed in US West (Oregon) and Europe (London) regions. The supplied announcement does not specify exact pricing or availability beyond AWS Marketplace.

Source facts for AssemblyAI optimizes speech-to-text inference costs on AWS with EC2 Spot: Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot Speech-to-text inference is computationally intensive. At scale, this means thousands of GPU instances running continuously. Running entirely on On-Demand Instances would be cost-prohibitive.
Original AINewsTrend explainer based on the cited source. This graphic summarizes source statements; it is not an independent benchmark.

Sources and further reading

  1. awsml: Cost-optimizing AssemblyAI speech-to-text inference on AWS with Amazon EC2 Spot