What changed
AWS has released a new Ray Train Deep Learning Container (DLC) that simplifies distributed training on Amazon EKS by integrating the Elastic Fabric Adapter (EFA) and Ray Train. This container includes all necessary components such as CUDA, NCCL, and the AWS OFI NCCL plugin, eliminating the need for custom image builds. The DLC is available in Amazon Elastic Container Registry (ECR) Public and is tested as a single unit, ensuring compatibility and performance across multiple nodes.
What this means in practice
The Ray Train DLC allows users to set up a multi-node training cluster with minimal configuration. By using a single tagged image, users can deploy both Ray head and worker nodes without separate images. The KubeRay operator manages the cluster creation, ensuring that each pod connects to the head node and utilizes EFA for inter-node communication. This setup enables efficient training across multiple GPUs, with the sample provided demonstrating fine-tuning of Qwen2.5-1.5B using PyTorch Fully Sharded Data Parallel (FSDP) on eight NVIDIA L4 GPUs.
Limitations and availability
The DLC is available as a public image in ECR Public, and users can also access it via private ECR URIs. The image includes pre-installed libraries such as DeepSpeed, flash-attn, and Transformer Engine, which are compiled for CUDA 13.0.2. Users can customize the training environment by specifying GPU requirements, network transport settings, and additional Python packages through environment variables and runtime_env. The sample uses the tag public.ecr.aws/deep-learning-containers/ray:train-ml-cuda-v1.1, which is pinned to a specific version for reproducibility. The solution is available in a sample repository and uses the same KubeRay-on-EKS foundation as other Ray-based workflows.
