Amazon SageMaker AI released coding agents for deploying Hugging Face models, streamlining the process from days to hours. The tool supports real-time endpoints and multiple deployment modes, including scale-to-zero and serverless inference.
The deployment process includes setting up autoscaling, CloudWatch alarms, and the correct serving container from the AWS Deep Learning Containers catalog. These skills are open source, using Python and the AWS Command Line Interface (AWS CLI), and work across macOS, Linux, and Windows.
"Both coding agents initially chose Text Generation Inference (TGI) as the serving container to deploy, an understandable choice given that T. TGI build available in the Region predated Qwen3’s architecture and couldn’t load the model. The endpoint failed its health check. The agent bumped the TGI version, redeployed, failed again, and pivoted to vLLM."
The problem with unguided agents is that they may make wrong decisions, leading to fragile or costly endpoints. The skills prevent this by incorporating current deployment knowledge, such as the need for vLLM with recent Qwen models and Python 3.13 compatibility.
The skills cover the end-to-end deployment workflow, orchestrating five AWS services: Amazon SageMaker AI hosts the endpoint, IAM provides the execution role, Amazon ECR and AWS Deep Learning Containers supply the serving image, and Amazon CloudWatch powers the alarms. All helper scripts call these services through Boto3 and the AWS CLI.
The announcement follows the release of Hugging Face Skills, which are now integrated with SageMaker AI to enhance deployment efficiency. Amazon SageMaker AI did not say how the skills will evolve, and the source raises the question of how frequently deployment knowledge changes.
Source: awsml
